If you knew that a woman who had an abortion had double the risk of long-term mental health problems, would you be more likely to think it should be restricted?
What about if it one in ten of all mental health problems was linked to abortion? Perhaps you might start thinking there's a bit more evil in allowing women to terminate pregnancies than you had previously thought.
In September last year a massive study was released that claimed just this. The study was the type medical experts tend to think of as the very best: a meta-analysis, which means it gathered all the results of many different studies, to provide a much bigger sample of women, and, supposedly a much more reliable result.
It made a splash. The UK’s Daily Telegraph uncritically reported the study, which included more than 877,000 women, noting the author, Priscilla Coleman, said her motivation was to produce an “unbiased analysis of the best available evidence”.
Information websites, such as Real Choices Australia, which describes itself as dedicated to “education around the issues of unintended pregnancy and abortion”, also repeated the findings.
There was just one problem: the study was crap. An analysis published in the academic journal Contraception this month has detailed the many and varied ways this so-called meta-analysis was unreliable.
Just to name a few: 11 of the 22 studies included in the analysis were by the author herself, and no reason was given as to why others were excluded. Some of the studies that were picked were analysed incorrectly or were dodgy in the first place. Oh, and Coleman seemed to think that it was acceptable to compare the mental health of women who had abortions with that of women who were never pregnant in the first place, rather than women who had proceeded with the unintended pregnancy.
The Contraception paper concluded: “Unfortunately, much of the published research… claiming to find an association between abortion and mental health is shot through with methodological and analytical errors and shortcomings, and it is used to misinform policy and clinical practice, both of which have consequences for women's reproductive decision-making, health and well-being”.
And, let’s face it; once the cat is out of the bag with “research” like this, it’s very hard to put it back in.
But the scariest thing is that this type of bad research is not limited to controversial issues like abortion.
If you want to know more, I can’t speak highly enough of the new book Bad Pharma, by UK doctor Ben Goldacre.
See, the truth is, you are being lied to every day. Even - unwittingly - by your doctor. In his book Goldacre shows how the whole system the world uses to test, approve, and monitor medicine is broken.
For a taste of his argument, in this TED talk Goldacre tells a story of a group of researchers that decided to examine all antidepressants that the US drug regulator approved over a 15 year period.
They found there was pretty much a 50/50 split between studies given to the regulator that showed the particular drug they were looking at worked, or showed it didn’t.
Yet when they looked for those same studies in academic journals, only three of the negative ones were published, while all but one of the positive trials was published.
“You can see what a staggering difference there was between reality and what doctors, patients commissioners of health services and academics were able to see,” Goldacre says. “This is a cancer at the core of evidence-based medicine. If I flipped a coin 100 times but then withheld the results from you for half of those tosses, I could make it look like a had a coin that always comes up heads… [and] this is exactly what we blindly tolerate.”
Perhaps that’s why Priscilla Coleman was so able to get her flawed, misleading study into a prestigious journal like the British Journal of Psychiatry.
But for those of us who care about women’s access to abortion – and how we treat everything from mental illness, cancer to heart disease – this is very scary indeed.
* You can get Bad Pharma here.