DAR
Interesting book review published in Skeptic Magazine. Excerpts:
***
Snake Oil Science
by Harriet Hall, M.D.
I could condense this review into three words: “read this book!”
The term “complementary and alternative medicine” (CAM) is relatively new, but the treatments it encompasses are not. Before we had science, all we had to rely on was testimonials and beliefs. And even today, for most people who believe CAM works, belief is enough. But at some level, the public has now recognized that science matters and people are looking for evidence to support those beliefs. Advocates claim that recent research validates CAM therapies. Does it really? Does the evidence show that any CAM therapy actually works better than placebos? R. Barker Bausell asks that question, does a compellingly thorough investigation, and comes up with a resounding “NO” for an answer.
Bausell is the ideal person to ask such a question. He is a research methodologist: he designs and analyzes research studies for a living. Not only that: he was intimately involved with acupuncture research for the National Center for Complementary and Alternative Medicine (NCCAM). So when he talks about what can go wrong in research and why much of the research on CAM is suspect, he is well worth listening to.
...SNIP...
The fact that pain relief follows treatment doesn’t necessarily mean that the treatment caused the pain relief. This is only one of the many impediments to correct thinking that plague our fallible human brains. Bausell describes some of those other impediments. He shows how patients, doctors, and researchers are all equally likely to fool themselves, and why the most rigorous science is needed to keep us from reaching false conclusions.
book cover
Bausell’s thorough discussion of the placebo phenomenon is illuminating and invaluable. He covers the history of research on placebos and tells some fascinating anecdotes. He argues that placebo response is not just imagination. It is a learned phenomenon, a conditioned response. You respond to a placebo pill because you have previous experience of being helped by pills. Morphine injections in dogs cause a side effect of salivation: after a while, you can inject water and they will respond with salivation. Physiologic effects from placebo are always smaller than with the real thing, but apparently they do occur. The evidence for objective physiologic effects may not be entirely convincing, but it is certain that pain and other subjective symptoms respond to placebos. And there is even research suggesting a mechanism: the release of endogenous opioids, pain-relieving chemicals produced by our own brains. If you counteract those chemicals with a narcotic antagonist like Narcan, you can block the placebo response.
...SNIP...
Research is full of pitfalls. Negative studies tend not to get published (the file drawer effect). Research done by believers and pharmaceutical companies tends to be more positive than research done by others. Studies from non-English speaking countries are notoriously unreliable for various reasons — 98% of the acupuncture studies from Asia are positive, compared to 30% from Canada, Australia, and New Zealand. The researcher may delegate the actual research to others, who may make undetected mistakes or deliberately skew results to please their boss. Double blind studies may not be truly blind: subjects may have been able to guess which group they were in. Subjects who are not responding may drop out. People who believe in homeopathy are more likely to volunteer for homeopathy studies. Researchers may put a positive spin on their findings or reach conclusions that are not justified by the data. Even if the research is impeccable, we arbitrarily use p=.05 as the measure of statistical significance, and this means there is a 5% probability that the results will appear falsely positive just by chance.
There are more pitfalls, and Bausell covers them all. When you come right down to it, no experiment is beyond criticism, and most published research is wrong. So how can we decide which studies are credible? We now have published guidelines such as the 22 item Consolidated Standards of Reporting Trials (CONSORT) checklist to assess the quality of randomized controlled trials, but Bausell offers some simpler criteria that can rule out the worst offenders:
* Subjects are randomly assigned to a CAM therapy or a credible placebo
* At least 50 subjects per group
* Less than 25% dropout rate
* Publication in a high-quality, prestigious, peer-reviewed journal
Using this simple 4-item checklist, he reviewed all the CAM studies published in The New England Journal of Medicine and The Journal of the American Medical Association from 2000–2007. 14 met the criteria, and all were negative. When he expanded his search to include the Annals of Internal Medicine and Archives of Internal Medicine, he ended up with 22 studies, only one of which was positive (exactly what you would expect from the 5% rule if none of them worked).
...snip...
Full article here