Big data and numbers may seem to drive the world these days, but human factors can play a dizzying role when it comes to statistics and medical treatments.
For those fascinated by numeracy in health care, writer Hannah Fry, in a readable New Yorker essay, details how medicine and patients alike have been bedeviled by attempts to quantify life-and-death decision making.
She tracks centuries of investigators experiments in applying rationality, logic, and mathematics to human lives and their care by doctors and others, reporting about Adolphe Quetelet, an 1830s Belgian astronomer and mathematician:
“He developed the idea of a ‘Social Physics,’ and began to explore the possibility that human lives, like planets, had an underlying mechanistic trajectory. There’s something unsettling in the idea that, amid the vagaries of choice, chance, and circumstance, mathematics can tell us something about what it is to be human. Yet Quetelet’s overarching findings still stand: At some level, human life can be quantified and predicted. We can now forecast, with remarkable accuracy, the number of women in Germany who will choose to have a baby each year, the number of car accidents in Canada, the number of plane crashes across the Southern Hemisphere, even the number of people who will visit a New York City emergency room on a Friday evening … But it’s one thing when your aim is to speak in general terms about who we are together, as a collective entity. The trouble comes when you try to go the other way—to learn something about us as individuals from how we behave as a collective.”
Fry shows how errors in evaluating statistics could confuse authorities investigating if Harold Shipman, a British doctor who cared for the elderly, was merely unlucky or a stone-cold killer. Spoiler alert: Though numbers might suggest his patients’ death rate was not aberrant among so many who were so old, more than 200 fatalities were blamed on the infamous late MD’s lethal prescribing.
Fry also recounts the value of a key metric — the number needed to treat or NNT. (It’s an invaluable clarifier for patients that has gotten even more useful as doctors and researchers develop ways to present it, I’ve found). She turns to David Spiegelhalter, author of, “The Art of Statistics” and a consulting investigator in the Shipman case, to describe the NNT:
“Every day, millions of people, David Spiegelhalter included, swallow a small white statin pill to reduce the risk of heart attack and stroke. If you are one of those people, and go on to live a long and happy life without ever suffering a heart attack, you have no way of knowing whether your daily statin was responsible or whether you were never going to have a heart attack in the first place. Of a 1,000 people who take statins for five years, the drugs will help only 18 to avoid a major heart attack or stroke. And if you do find yourself having a heart attack, you’ll never know whether it was delayed by taking the statin. ‘All I can ever know,” Spiegelhalter writes, “is that on average it benefits a large group of people like me.’ That’s the rule with preventive drugs: for most individuals, most of those drugs won’t do anything. The fact that they produce a collective benefit makes them worth taking. But it’s a pharmaceutical form of Pascal’s wager: you may as well act as though God were real (and believe that the drugs will work for you), because the consequences otherwise outweigh the inconvenience.”
In my practice, I see not only the harms that patients suffer while seeking medical services, but also their struggles to access and afford safe, efficient, and excellent medical care. This has become an ordeal due to the skyrocketing cost, complexity, and uncertainty of treatments and prescription medications, too many of which turn out to be dangerous drugs.
Research shows that doctors are more harried than ever, urged by rigorous studies to deal with patients and their needs in many different ways — but not necessarily with accompanying guidance about how to create more time or to deal with financial consequences of doing so. So, instead, medical care givers may breeze through vital information and explanations for patients about their care. This can abridge their fundamental right to informed consent. That means they are told clearly and fully all the important facts they need to make an intelligent decision about what treatments to have, where to get them, and from whom.
But if patients are easily baffled or confused by numbers — and they are — on the effectiveness of medications or therapies or how those affect their longevity or quality of life, medical scientists themselves also may struggle with math and its application. She explains well a raging controversy over validation of studies using the “p-value”:
“A stranger hands you a coin. You have your suspicions that it’s been weighted somehow, perhaps to make heads come up more often. But for now, you’ll happily go along with the assumption that the coin is fair. You toss the coin twice and get two heads in a row. Nothing to get excited about just yet. A perfectly fair coin will throw two heads in a row 25% per cent of the time—a probability known as the p-value. You keep tossing and get another head. Then another. Things are starting to look fishy, but even if you threw the coin a thousand times, or a million, you could never be absolutely sure it was rigged. The chances might be minuscule, but in theory a fair coin could still produce any combination of heads. Scientists have picked a path through all this uncertainty by setting an arbitrary threshold and agreeing that anything beyond that point gives you grounds for suspicion. Since 1925, when the British statistician Ronald Fisher first suggested the convention, that threshold has typically been set at 5%.”
OK, it may seem esoteric. What’s the problem with p-values in studies? Fry reported on how, for example, this standard can make a hash out of data from landmark studies that suggest that patients ought to take aspirin to prevent heart attacks, once seemingly convincing information that since seems much less so:
“In science, the situation is starker, and the stakes are higher. With a threshold [or p-value] of only 5%, 1 in 20 studies will inadvertently find evidence for nonexistent phenomena in its data … this is far from being only a theoretical concern. In medicine, a study of 49 of the most cited medical publications from 1990 to 2003 found that the conclusions of 16% were contradicted by subsequent studies. Psychology fares worse still in these surveys (possibly because its studies are cheaper to reproduce). A 2015 study found that attempts to reproduce a hundred psychological experiments yielded significant results in only 36% of them, even though 97% of the initial studies reported a p-value under the 5% threshold. And scientists fear that … the fluke results tend to get an outsized share of attention.”
In other words, Americans — patients and medical professionals alike — get bombarded with “findings” from studies that purport, based on research and data, to recommend we do everything for our health from eating dirt to standing on our heads five times a day. The silly stuff we, perhaps, discard with common sense. But as costs, complexities, and uncertainties soar in patients’ options with life changing and lifesaving drugs and treatments, solid studies and data, interpreted correctly become paramount. We can educate ourselves about research, math and statistics, and what’s good and what’s hokum. We’ve got a lot of work to do — and it may be easier and sounder to become savvy about what can be daunting stuff while we’re well and uninjured, so we don’t need to get harsh education from grim reality.
