Why Bogus Therapies Seem to Work - CSI

December 5, 2014
Public discussion of scientific topics such as global warming is confused by misuse of the term “skeptic.”
Why Bogus Therapies Seem to Work
At least ten kinds of errors and biases can convince intelligent, honest people that cures have been achieved when they have not.
“Nothing is more dangerous than active ignorance.”
—Goethe
Those who sell therapies of any kind have an obligation to prove, first, that their treatments are safe and, second, that they are effective. The latter is often the more difficult task because there are many subtle ways that honest and intelligent people (both patients and therapists) can be led to think that a treatment has cured someone when it has not. This is true whether we are assessing new treatments in scientific medicine, old nostrums in folk medicine, fringe treatments in “alternative medicine,” or the frankly magical panaceas of faith healers.
To distinguish causal from fortuitous improvements that might follow any intervention, a set of objective procedures has evolved for testing putative remedies. Unless a technique, ritual, drug, or surgical procedure can meet these requirements, it is ethically questionable to offer it to the public, especially if money is to change hands. Since most “alternative” therapies (i.e., ones not accepted by scientific biomedicine) fall into this category, one must ask why so many customers who would not purchase a toaster without consulting Consumer Reports shell out, with trusting naivetë, large sums for unproven, possibly dangerous, health remedies.
For many years, critics have been raising telling doubts about fringe medical practices, but the popularity of such nostrums seems undiminished. We must wonder why entrepreneurs’ claims in this area should remain so refractory to contrary data. If an “alternative” or “complementary” therapy:
is implausible on a priori grounds (because its implied mechanisms or putative effects contradict well-established laws, principles, or empirical findings in physics, chemistry, or biology),
lacks a scientifically acceptable rationale of its own,
has insufficient supporting evidence derived from adequately controlled outcome research (i.e., double-blind, randomized, placebo-controlled clinical trials),
has failed in well-controlled clinical studies done by impartial evaluators and has been unable to rule out competing explanations for why it might seem to work in uncontrolled settings, and,
should seem improbable, even to the lay person, on “commonsense” grounds, why would so many well-educated people continue to sell and purchase such a treatment?
The answer, I believe, lies in a combination of vigorous marketing of unsubstantiated claims by “alternative” healers (Beyerstein and Sampson 1996), the poor level of scientific knowledge in the public at large (Kiernan 1995), and the “will to believe” so prevalent among seekers attracted to the New Age movement (Basil 1988; Gross and Levitt 1994).
The appeal of nonscientific medicine is largely a holdover from popular “counterculture” sentiments of the 1960s and 1970s. Remnants of the rebellious, “back-to-nature” leanings of that era survive as nostalgic yearnings for a return to nineteenth-century-style democratized health care (now wrapped in the banner of patients’ rights) and a dislike of bureaucratic, technologic, and specialized treatment of disease (Cassileth and Brown 1988). Likewise, the allure of the “holistic” dogmas of alternative medicine is a descendant of the fascination with Eastern mysticism that emerged in the sixties and seventies. Although the philosophy and the science that underlie these holistic teachings have been severely criticized (Brandon 1985), they retain a strong appeal for those committed to belief in “mind-over-matter” cures, a systemic rather than localized view of pathology, and the all-powerful ability of nutrition to restore health (conceived of as whole-body “balance”).
Many dubious health products remain on the market primarily because satisfied customers offer testimonials to their worth. Essentially, they are saying, “I tried it and I got better, so it must be effective.” But even when symptoms do improve following a treatment, this, by itself, cannot prove that the therapy was responsible.
The Illness-Disease Distinction
Although the terms disease and illness are often used interchangeably, for present purposes it is worth distinguishing between the two. I shall use disease to refer to a pathological state of the organism due to infection, tissue degeneration, trauma, toxic exposure, carcinogenesis, etc. By illness I mean the feelings of malaise, pain, disorientation, dysfunctionality, or other complaints that might accompany a disease. Our subjective reaction to the raw sensations we call symptoms is molded by cultural and psychological factors such as beliefs, suggestions, expectations, demand characteristics, self-serving biases, and self-deception. The experience of illness is also affected (often unconsciously) by a host of social and psychological payoffs that accrue to those admitted to the “sick role” by society’s gatekeepers (i.e., health professionals). For certain individuals, the privileged status and benefits of the sick role are sufficient to perpetuate the experience of illness after a disease has healed, or even to create feelings of illness in the absence of disease (Alcock 1986).
Unless we can tease apart the many factors that contribute to the perception of being ill, personal testimonials offer no basis on which to judge whether a putative therapy has, in fact, cured a disease. That is why controlled clinical trials with objective physical measures are essential in evaluating therapies of any kind.
Correlation Does Not Imply Causation
Mistaking correlation for causation is the basis of most superstitious beliefs, including many in the area of alternative medicine. We have a tendency to assume that when things occur together, they must be causally connected, although obviously they need not be. For example, there is a high correlation between the consumption of diet soft drinks and obesity. Does this mean that artificial sweeteners cause people to become overweight? When we count on personal experience to test the worth of medical treatments, many factors are varying simultaneously, making it extremely difficult to determine what is cause and effect. Personal endorsements supply the bulk of the support for unorthodox health products, but they are a weak currency because of what Gilovich (1997) has called the “compared to what?” problem. Without comparison to a similar group of sufferers, treated identically except that the allegedly curative element is withheld, individual recipients can never know whether they would have recovered just as well without it.
Ten Errors and Biases
The question is, then: Why might therapists and their clients who rely on anecdotal evidence and uncontrolled observations erroneously conclude that inert therapies work? There are at least ten good reasons.
The disease may have run its natural course.
Many diseases are self-limiting — providing the condition is not chronic or fatal, the body’s own recuperative processes usually restore the sufferer to health. Thus, before a therapy can be acknowledged as curative, its proponents must show that the number of patients listed as improved exceeds the proportion expected to recover without any treatment at all (or that they recover reliably faster than if left untreated). Unless an unconventional therapist releases detailed records of successes and failures over a sufficiently large number of patients with the same complaint, he or she cannot claim to have exceeded the published norms for unaided recovery.
Many diseases are cyclical.
Arthritis, multiple sclerosis, allergies, and gastrointestinal complaints are examples of diseases that normally “have their ups and downs.” Naturally, sufferers tend to seek therapy during the downturn of any given cycle. In this way, a bogus treatment will have repeated opportunities to coincide with upturns that would have happened anyway. Again, in the absence of appropriate control groups, consumers and vendors alike are prone to misinterpret improvement due to normal cyclical variation as a valid therapeutic effect.
Spontaneous remission.
Anecdotally reported cures can be due to rare but possible “spontaneous remissions.” Even with cancers that are nearly always lethal, tumors occasionally disappear without further treatment. One experienced oncologist reports that he has seen twelve such events in about six thousand cases he has treated (Silverman 1987). Alternative therapies can receive unearned acclaim for remissions of this sort because many desperate patients turn to them when they feel that they have nothing left to lose. When the “alternatives” assert that they have snatched many hopeless individuals from death’s door, they rarely reveal what percentage of their apparently terminal clientele such happy exceptions represent. What is needed is statistical evidence that their “cure rates” exceed the known spontaneous remission rate and the placebo response rate (see below) for the conditions they treat.
The exact mechanisms responsible for spontaneous remissions are not well understood, but much research is being devoted to revealing and possibly harnessing processes in the immune system or elsewhere that are responsible for these unexpected turnarounds. The relatively new field of psychoneuroimmunology studies how psychological variables affect the nervous, glandular, and immune systems in ways that might affect susceptibility to and recovery from disease (Ader and Cohen 1993; Mestel 1994). If thoughts, emotions, desires, beliefs, etc., are physical states of the brain, there is nothing inherently mystical in the notion that these neural processes could affect glandular, immune, and other cellular processes throughout the body. Via the limbic system of the brain, the hypothalamic pituitary axis, and the autonomic nervous system, psychological variables can have widespread physiological effects that can have positive or negative impacts upon health. While research has confirmed that such effects exist, it must be remembered that they are fairly small, accounting for perhaps a few percent of the variance in disease statistics.
The placebo effect.
A major reason why bogus remedies are credited with subjective, and occasionally objective, improvements is the ubiquitous placebo effect (Roberts, Kewman, and Hovell 1993; Ulett 1996). The history of medicine is strewn with examples of what, with hindsight, seem like crackpot procedures that were once enthusiastically endorsed by physicians and patients alike (Skrabanek and McCormick 1990; Barrett and Jarvis 1993). Misattributions of this sort arise from the false assumption that a change in symptoms following a treatment must have been a specific consequence of that procedure. Through a combination of suggestion, belief, expectancy, cognitive reinterpretation, and diversion of attention, patients given biologically useless treatments can often experience measurable relief. Some placebo responses produce actual changes in the physical condition; others are subjective changes that make patients feel better although there has been no objective change in the underlying pathology.
Through repeated contact with valid therapeutic procedures, we all develop, much like Pavlov’s dogs, conditioned responses in various physiological systems. Later, these responses can be triggered by the setting, rituals, paraphernalia, and verbal cues that signal the act of “being treated.” Among other things, placebos can cause release of the body’s own morphinelike pain killers, the endorphins (Ulett 1996, ch. 3). Because these learned responses can be palliative, even when a treatment itself is physiologically unrelated to the source of the complaint, putative therapies must be tested against a placebo control group — similar patients who receive a sham treatment that resembles the “real” one except that the suspected active ingredient is withheld.
It is essential that the patients in such tests be randomly assigned to their respective groups and that they be “blind” with respect to their active versus placebo status. Because the power of what psychologists call expectancy and compliance effects (see below) is so strong, the therapists must also be blind as to individual patients’ group membership. Hence the term double blind — the gold standard of outcome research. Such precautions are required because barely perceptible cues, unintentionally conveyed by treatment providers who are not blinded, can bias test results. Likewise, those who assess the treatment’s effects must also be blind, for there is a large literature on “experimenter bias” showing that honest and well-trained professionals can unconsciously “read in” the outcomes they expect when they attempt to assess complex phenomena (Rosenthal 1966; Chapman and Chapman 1967).
When the clinical trial is completed, the blinds can be broken to allow statistical comparison of active, placebo, and no-treatment groups. Only if the improvements observed in the active treatment group exceed those in the two control groups by a statistically significant amount can the therapy claim legitimacy.
Some allegedly cured symptoms are psychosomatic to begin with.
A constant difficulty in trying to measure therapeutic effectiveness is that many physical complaints can both arise from psychosocial distress and be alleviated by support and reassurance. At first glance, these symptoms (at various times called “psychosomatic,” “hysterical,” or “neurasthenic”) resemble those of recognized medical syndromes (Shorter 1992; Merskey 1995). Although there are many “secondary gains” (psychological, social, and economic) that accrue to those who slip into “the sick role” in this way, we need not accuse them of conscious malingering to point out that their symptoms are nonetheless maintained by subtle psychosocial processes.
“Alternative” healers cater to these members of the “worried well” who are mistakenly convinced that they are ill. Their complaints are instances of somatization, the tendency to express psychological concerns in a language of symptoms like those of organic diseases (Alcock 1986; Shorter 1992). The “alternatives” offer comfort to these individuals who for psychological reasons need others to believe there are organic etiologies for their symptoms. Often with the aid of pseudoscientific diagnostic devices, fringe practitioners reinforce the somatizer’s conviction that the cold-hearted, narrow-minded medical establishment, which can find nothing physically amiss, is both incompetent and unfair in refusing to acknowledge a very real organic condition. A large portion of those diagnosed with “chronic fatigue,” “environmental sensitivity syndrome,” and various stress disorders (not to mention many suing because of the allegedly harmful effects of silicone breast implants) look very much like classic somatizers (Stewart 1990; Huber 1991; Rosenbaum 1997). When, through the role-governed rituals of “delivering treatment,” fringe therapists supply the reassurance, sense of belonging, and existential support their clients seek, this is obviously worthwhile, but all this need not be foreign to scientific practitioners who have much more to offer besides. The downside is that catering to the desire for medical diagnoses for psychological complaints promotes pseudoscientific and magical thinking while unduly inflating the success rates of medical quacks. Saddest of all, it perpetuates the anachronistic feeling that there is something shameful or illegitimate about psychological problems.
Symptomatic relief versus cure.
Short of an outright cure, alleviating pain and discomfort is what sick people value most. Many allegedly curative treatments offered by alternative practitioners, while unable to affect the disease process itself, do make the illness more bearable, but for psychological reasons. Pain is one example. Much research shows that pain is partly a sensation like seeing or hearing and partly an emotion (Melzack 1973). It has been found repeatedly that successfully reducing the emotional component of pain leaves the sensory portion surprisingly tolerable. Thus, suffering can often be reduced by psychological means, even if the underlying pathology is untouched. Anything that can allay anxiety, redirect attention, reduce arousal, foster a sense of control, or lead to cognitive reinterpretation of symptoms can alleviate the agony component of pain. Modern pain clinics put these strategies to good use every day (Smith, Merskey, and Gross 1980). Whenever patients suffer less, this is all to the good, but we must be careful that purely symptomatic relief does not divert people from proven remedies until it is too late for them to be effective.
Many consumers of alternative therapies hedge their bets.
In an attempt to appeal to a wider clientele, many unorthodox healers have begun to refer to themselves as “complementary” rather than “alternative.” Instead of ministering primarily to the ideologically committed or those who have been told there is nothing more that conventional medicine can do for them, the “alternatives” have begun to advertise that they can enhance conventional biomedical treatments. They accept that orthodox practitioners can alleviate specific symptoms but contend that alternative medicine treats the real causes of disease — dubious dietary imbalances or environmental sensitivities, disrupted energy fields, or even unresolved conflicts from previous incarnations. If improvement follows the combined delivery of “complementary” and scientifically based treatments, the fringe practice often gets a disproportionate share of the credit.
Misdiagnosis (by self or by a physician).
In this era of media obsession with health, many people can be induced to think they have diseases they do not have. When these healthy folk receive the oddly unwelcome news from orthodox physicians that they have no organic signs of disease, they often gravitate to alternative practitioners who can almost always find some kind of “imbalance” to treat. If “recovery” follows, another convert is born.
Of course, scientifically trained physicians are not infallible, and a mistaken diagnosis, followed by a trip to a shrine or an alternative healer, can lead to a glowing testimonial for curing a grave condition that never existed. Other times, the diagnosis may be correct but the time course, which is inherently hard to predict, might prove inaccurate. If a patient with a terminal condition undergoes alternative treatments and succumbs later than the conventional doctor predicted, the alternative procedure may receive credit for prolonging life when, in fact, there was merely an unduly pessimistic prognosis — survival was longer than the expected norm, but within the range of normal statistical variation for the disease.
Derivative benefits.
Alternative healers often have forceful, charismatic personalities (O'Connor 1987). To the extent that patients are swept up by the messianic aspects of alternative medicine, psychological uplift may ensue. If an enthusiastic, upbeat healer manages to elevate the patient’s mood and expectations, this optimism can lead to greater compliance with, and hence effectiveness of, any orthodox treatments he or she may also be receiving. This expectant attitude can also motivate people to eat and sleep better and to exercise and socialize more. These, by themselves, could help speed natural recovery.
Psychological spinoffs of this sort can also reduce stress, which has been shown to have deleterious effects on the immune system (Mestel 1994). Removing this added burden may speed healing, even if it is not a specific effect of the therapy. As with purely symptomatic relief, this is far from a bad thing, unless it diverts the patient from more effective treatments, or the charges are exorbitant.
Psychological distortion of reality.
Distortion of reality in the service of strong belief is a common occurrence (Alcock 1995). Even when they derive no objective improvements, devotees who have a strong psychological investment in alternative medicine can convince themselves they have been helped. According to cognitive dissonance theory (Festinger 1957), when experiences contradict existing attitudes, feelings, or knowledge, mental distress is produced. We tend to alleviate this discord by reinterpreting (distorting) the offending information. To have received no relief after committing time, money, and “face” to an alternate course of treatment (and perhaps to the worldview of which it is a part) would create such a state of internal disharmony. Because it would be too psychologically disconcerting to admit to oneself or to others that it has all been a waste, there would be strong psychological pressure to find some redeeming value in the treatment.
Many other self-serving biases help maintain self-esteem and smooth social functioning (Beyerstein and Hadaway 1991). Because core beliefs tend to be vigorously defended by warping perception and memory, fringe practitioners and their clients are prone to misinterpret cues and remember things as they wish they had happened. Similarly, they may be selective in what they recall, overestimating their apparent successes while ignoring, downplaying, or explaining away their failures. The scientific method evolved in large part to reduce the impact of this human penchant for jumping to congenial conclusions.
An illusory feeling that one’s symptoms have improved could also be due to a number of so called demand characteristics found in any therapeutic setting. In all societies, there exists the “norm of reciprocity,” an implicit rule that obliges people to respond in kind when someone does them a good turn. Therapists, for the most part, sincerely believe they are helping their patients and it is only natural that patients would want to please them in return. Without patients necessarily realizing it, such obligations are sufficient to inflate their perception of how much benefit they have received. Thus, controls for compliance effects must also be built into proper clinical trials (Adair 1973).
Finally, the job of distinguishing real from spurious causal relationships requires not only controlled observations, but also systematized abstractions from large bodies of data. Psychologists interested in judgmental biases have identified many sources of error that plague people who rely on informal reasoning processes to analyze complex events (Gilovich 1991, 1997; Schick and Vaughn 1995). Dean and colleagues (1992) showed, using examples from another popular pseudoscience, handwriting analysis, that without sophisticated statistical aids, human cognitive abilities are simply not up to the task of sifting valid relationships out of masses of interacting data. Similar difficulties would have confronted the pioneers of pre-scientific medicine and their followers, and for that reason, we cannot accept their anecdotal reports as support for their assertions.
Summary
For the reasons I have presented, individual testimonials count for very little in evaluating therapies. Because so many false leads can convince intelligent, honest people that cures have been achieved when they have not, it is essential that any putative treatment be tested under conditions that control for placebo responses, compliance effects, and judgmental errors.
Before anyone agrees to undergo any kind of treatment, he or she should be confident that it has been validated in properly controlled clinical trials. To reduce the probability that supporting evidence has been contaminated by the foregoing biases and errors, consumers should insist that supporting evidence be published in peer-reviewed scientific journals. Any practitioner who cannot supply this kind of backing for his or her procedures is immediately suspect. Potential clients should be wary if, instead, the “evidence” consists merely of testimonials, self-published pamphlets or books, or items from the popular media. Even if supporting articles appear to have come from legitimate scientific periodicals, consumers should check to see that the journals in question are published by reputable scientific organizations. Papers extolling pseudoscience often appear in official-looking periodicals that turn out to be owned by groups with inadequate scientific credentials but with a financial stake in the questionable products. Similarly, one should discount articles from the “vanity press” — journals that accept virtually all submissions and charge the authors for publication. And finally, because any single positive outcome — even from a carefully done experiment published in a reputable journal — could always be a fluke, replication by independent research groups is the ultimate standard of proof.
If the practitioner claims persecution, is ignorant of or openly hostile to mainstream science, cannot supply a reasonable scientific rationale for his or her methods, and promises results that go well beyond those claimed by orthodox biomedicine, there is strong reason to suspect that one is dealing with a quack. Appeals to other ways of knowing or mysterious sounding “planes,” “energies,” “forces,” or “vibrations” are other telltale signs, as is any claim to treat the whole person rather than localized pathology.
To people who are unwell, any promise of a cure is especially beguiling. As a result, false hope easily supplants common sense. In this vulnerable state, the need for hard-nosed appraisal is all the more necessary, but so often we see instead an eagerness to abandon any remaining vestiges of skepticism. Erstwhile savvy consumers, felled by disease, often insist upon less evidence to support the claims of alternative healers than they would previously have demanded from someone hawking a used car. Caveat emptor!
References
Adair, J. 1973. The Human Subject. Boston: Little, Brown and Co.
Ader, R., and N. Cohen. 1993. Psychoneuroimmunology: Conditioning and stress. Annual Review of Psychology 44: 53-85.
Alcock, J. 1986. Chronic pain and the injured worker. Canadian Psychology 27(2): 196-203.
—. 1995. The belief engine. Skeptical Inquirer 19(3): 14-8.
Barrett, S., and W. Jarvis. 1993. The Health Robbers: A Close Look at Quackery in America. Amherst, N.Y.: Prometheus Books.
Basil, R., ed. 1988. Not Necessarily the New Age. Amherst, N.Y.: Prometheus Books.
Beyerstein, B., and P. Hadaway. 1991. On avoiding folly. Journal of Drug Issues 20(4): 689-700.
Beyerstein, B., and W. Sampson. 1996. Traditional medicine and pseudoscience in China. Skeptical Inquirer 20(4): 18-26.
Brandon, R. 1985. Holism in philosophy of biology. In Examining Holistic Medicine, edited by D. Stalker and C. Glymour. Amherst, N.Y.: Prometheus Books, 127-36.
Cassileth, B., and H. Brown. 1988. Unorthodox cancer medicine. CA-A Cancer Journal for Clinicians 38(3): 176-86.
Chapman, L., and J. Chapman. 1967. Genesis of popular but erroneous diagnostic observations. Journal of Abnormal Psychology 72: 193-204.
Dean, G., I. Kelly, D. Saklofske, and A. Furnham. 1992. Graphology and human judgement. In The Write Stuff, edited by B. and D. Beyerstein. Amherst, N.Y.: Prometheus Books, 342-96.
Festinger, L. 1957. A Theory of Cognitive Dissonance. Stanford: Stanford University Press.
Gilovich, T. 1991. How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life. New York: Free Press/Macmillan.
—. 1997. Some systematic biases of everyday judgment. Skeptical Inquirer 21(2): 31-5.
Gross, P., and N. Levitt. 1994. Higher Superstition. Baltimore: Johns Hopkins University Press.
Huber, P. 1991. Galileo’s Revenge: Junk Science in the Courtroom. New York: Basic Books.
Kiernan, V. 1995. Survey plumbs the depths of international ignorance. The New Scientist (April 29): 7.
Merskey, H. 1995. The Analysis of Hysteria: Understanding Conversion and Dissociation. 2d ed. London: Royal College of Psychiatrists.
Melzack, R. 1973. The Puzzle of Pain. New York: Basic Books.
Mestel, R. 1994. Let mind talk unto body. The New Scientist (July 23): 26-31.
O'Connor, G. 1987. Confidence trick. The Medical Journal of Australia 147: 456-9.
Roberts, A., D. Kewman, and L. Hovell. 1993. The power of nonspecific effects in healing: Implications for psychosocial and biological treatments. Clinical Psychology Review 13: 375-91.
Rosenbaum, J. T. 1997. Lessons from litigation over silicone breast implants: A call for activism by scientists. Science 276 (June 6, 1997): 1524-5.
Rosenthal, R. 1966. Experimenter Effects in Behavioral Research. New York: Appleton-Century-Crofts.
Schick, T., and L. Vaughn. 1995. How to Think About Weird Things: Critical Thinking for a New Age. Mountain View, Calif.: Mayfield Publishing.
Shorter, E. 1992. From Paralysis to Fatigue: A History of Psychosomatic Illness in the Modern Era. New York: The Free Press.
Silverman, S. 1987. Medical “miracles": Still mysterious despite claims of believers. Psientific American (July): 5-7. Newsletter of the Sacramento Skeptics Society, Sacramento, Calif.
Skrabanek, P., and J. McCormick. 1990. Follies and Fallacies in Medicine. Amherst, N.Y.: Prometheus Books.
Smith, W., H. Merskey, and S. Gross, eds. 1980. Pain: Meaning and Management. New York: SP Medical and Scientific Books.
Stalker, D., and C. Glymour, eds. 1985. Examining Holistic Medicine. Amherst, N.Y.: Prometheus Books.
Stewart, D. 1990. Emotional disorders misdiagnosed as physical illness: Environmental hypersensitivity, candidiasis hypersensitivity, and chronic fatigue syndrome. Int. J. Mental Health 19(3): 56-68.
Ulett, G. A. 1996. Alternative Medicine or Magical Healing. St. Louis: Warren H. Green.
Barry L. Beyerstein