When regular folks don’t understand statistics, they can be misled. When pollsters don’t understand statistics, they can lose races that should have been won. When doctors don’t understand statistics, people can suffer.
Professor Gerd Gigerenzer of the Harding Center for Risk Literacy gave more than 1,000 gynecologists a set of facts about mammography and asked, “How many women who test positive actually have breast cancer?” Only 21 percent of these specialists could come up with right answer.
It’s a problem played out repeatedly in journal articles and media accounts of medical research.
With the focus on suicide prevention after the Germanwings crash, I’m sure we’d be delighted to read, as a university press release reported, “With a [simple blood] test like ours, we may be able to stem suicide rates by identifying those people and intervening early enough to head off a catastrophe.”
The journal article reported that the presence of the “suicide” gene increases the risk of suicide 1.15-fold.
What does this mean? In 2010, 13 people out of every 100,000 died as a result of suicide. Among carriers of the “suicide gene,” fewer than 15 out of 100,000 took their own lives. This de minimus improvement in predictive ability is worth next to nothing.
Of course, this is far from the only example of statistically misleading claims.
We’d all love an Alzheimer’s test too. A group of British researchers obliged, with a study claiming it had identified proteins that could distinguish patients with mild cognitive impairment (MCI) who later developed the disease from MCI patients whose faculties did not decline. Indeed, the researchers claimed it predicted whether a patient developed Alzheimer’s 87 percent of the time.
That sounds pretty darned accurate. Consider this, however: Only 10 percent of those with MCI develop Alzheimer’s. If you simply predicted that none of these patients would get the dreaded disease, you would be accurate an even better 90 percent of the time.
So what’s going on here?
With 100 MCI patients, 10 will develop Alzheimer’s — the true positives — and 90 will not — the true negatives. This test will falsely identify 29 percent of the 90 true negatives, or 26 cases, as people who will develop the disease. Meanwhile, nearly nine of the 10 true positives will be correctly identified (the 87 percent accuracy).
That means only a quarter of those predicted to get Alzheimer’s will actually develop the disease.
Even some tests that have made it into common usage are of questionable value.
That case has been made for mammograms, but prostate cancer screenings are equally iffy.
Gigerenzer compared two groups of 1,000 men each. One set had regular prostate-specific antigen (PSA) testing; the other didn’t. Seven of the men who had been tested regularly died of prostate cancer, as did an equal seven who had not been screened. Of the men who had been screened, 160 found out only after a biopsy that the PSA test was a false positive — that is, they were told they might have the disease but did not. Of course, none of those who were not tested went through the difficulty and expense of a biopsy.
In addition, 20 of the 1,000 men who had been screened were diagnosed with, and treated for, prostate cancer unnecessarily — in fact they were never suffering from the disease. So the test produces lots more tests but isn’t saving many lives.
Medications, too, are used or recalled under false statistical pretenses.
In Britain, health officials warned that an oral contraceptive doubled the risk of thrombosis. While true, they neglected to mention the doubling was from a 1 in 7,000 chance to a 2 in 7,000 chance. Women dropped the pill in huge numbers. One result: an additional 13,000 abortions.
Living in the modern world requires statistical literacy for practitioners and consumers alike. Failing the test carries real costs.
Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the minority leader of the Senate and the Democratic whip in the House.