Back in February, Mark Crislip and I both deconstructed an article written by Dr. Reynold Spector that appeared in the March/April issue of Skeptical Inquirer (SI), the flagship publication for the Committee for Skeptical Inquiry (CSI). The article was entitled Seven Deadly Medical Hypotheses, and, contrary to the usual standard of articles published in SI, it used a panoply of spin, bad arguments, and, yes, misinformation to paint a picture of seven horrifically deadly “medical hypotheses,” most of which, even if the reader accepted Dr. Spector’s arguments at face value in a worst case scenario, weren’t actually all that deadly at all, with the alleged deadliness of the others being in dispute. In addition, Dr. Spector painted a picture of medical science that is not nearly rigorous enough. While we at SBM would probably agree that much of medical science is insufficiently rigorous, given how so-called “complementary and alternative medicine” (CAM) or “integrative medicine” (IM) has found a prominent place in medical practice in all too many academic and private medical centers, Dr. Spector got it so wrong that he wasn’t even wrong when he conflated preliminary, hypothesis-generating studies with the big, randomized, phase III clinical trials necessary to achieve FDA approval for a new drug or device. This latest article by Dr. Spector seemed to be of a piece with his previous article in the January/February 2010 issue of SI entitled The War on Cancer A Progress Report for Skeptics, which was so negative in its assessment of scientific progress against cancer that for a moment I was wondering if I were reading NaturalNews.com or Mercola.com.

Unfortunately, Seven Deadly Medical Hypotheses itself is not yet online on the CSI website; so readers without a subscription to SI cannot at the present time judge for themselves whether Mark and I were too harsh on Dr. Spector, but our criticisms, along with that of SBM partner-in-crime Harriet Hall, did have an impact. Seemingly genuinely stunned at the level of criticism leveled at an article published in SI, SI’s editor Kendrick Frazier, to his credit, invited several responses to Dr. Spector’s article, which Harriet Hall, Mark Crislip, Carol Tavris, Avrum Bluming, and I eagerly provided. These letters were originally scheduled to be published a couple of issues ago, along with Dr. Spector’s response. Unfortunately, publishing in dead tree media being what it is, Harriet Hall and I were disappointed to find that the latest issue of SI still didn’t contain our rebuttals. Fortunately, Mr. Frazier has posted this material online for your edification, although, again, I wish he had also published the original article as well.

What other skeptics said about Seven Deadly Medical Hypotheses

Before I deconstruct Dr. Spector’s response to criticism, I think it’s worth briefly commenting on some of the other criticism that Dr. Spector garnered, the first of which appeared in the letters section of the July/August 2011 issue of SI in the form of two letters, one by Dr. James Kolter and one by a professor of social science named Robert J. Miller. Unfortunately, both appear to be primarily single issue critiques, rather than taking on the entire wrong gestalt of Dr. Spector’s article. For example, Dr. Kolter, as both Dr. Crislip and I did in our SBM blog posts, pointed out that the evidence indicting hormone replacement therapy (HRT) after menopause as a cause of breast cancer is not as definitive and clear-cut as Dr. Spector represented it, while Dr. Miller took Dr. Spector to task for a “seeming too-easy dismissal of a positive role of fresh fruit and vegetables in disease prevention.” Both of these are, of course, valid criticisms, but the miss the meta-issues underlying Dr. Spector’s medical pseudoskepticism, one of which Mark Crislip, in his usually insightful manner, picks up on right away:

The article “Seven Deadly Medical Hypotheses” has fundamental flaws. The argument, as I understand it, is that medical researchers fail to use the hypothetical/deductive approach to decide upon research agendas and therefore waste a “vast quantity of resource to disprove them.”

The argument as presented is circular. The author uses information from completed studies to prove the studies should not have been done in the first place. He fails to consider the state of knowledge at the time the studies were done and the cumulative nature of medical knowledge.

Meanwhile, Carol Tavris and Avrum Bluming pointed out a more irritating aspect of Dr. Spector’s entire attitude:

Of course, scientists and other skeptics can and do disagree with one another when it comes to interpreting data. Our primary objection to Spector’s discussion of HRT is not his conclusion but his tone of certainty, his oversimplification of an enormous body of research, and his unwillingness to question received wisdom from the WHI. Even some investigators who share his belief that the relative risks of HRT warrant concern acknowledge that the absolute risks from this treatment are small. In one worst-case analysis, researchers calculated that a fifty-year-old woman taking estrogen and progestin for ten years has a 96 percent chance of remaining free of breast cancer versus a 98 percent risk if she does not take HRT.

I will note that, as a breast cancer surgeon, I do not quite agree with Tavris and Bluming with their apparent glee in trashing the Women’s Health Initiative study that led to the publication of the famous paper in the Journal of the American Medical Association back in 2002 concluding that estrogen/progestin combined HRT increased the risk of breast cancer, a finding that Tavris and Blum apparently do not accept. Of course, this is an example of how scientists and other skeptics can disagree when it comes to interpreting data.

Finally, Harriet Hall, a.k.a. The Skepdoc, weighs in with a devastating characterization of Seven Deadly Medical Hypotheses:

Spector’s article was doubly disappointing because he had so many excellent points that were sabotaged by the way he presented them. Medical science is far from perfect, but it is a cooperative, self-correcting endeavor that constantly criticizes itself and is constantly improving. We need continued rational skepticism and constructive dialog, not oversimplified contrarian polemics.

Dr. Hall nailed it (as usual). Dr. Spector could have made some very valuable points about the complexity and difficulty of interpreting scientific evidence in medicine or in designing clinical trials. Instead, he decided to go all Mike Adams on us.

Since I’m the one writing this particular post, I won’t bother to quote from my own letter in response to Dr. Spector’s article. If you’re enough of a glutton for punishment or for some odd reason need more David Gorski in much the same way that the world needs more Mark Crislip, feel free to read it yourself. Regardless of your opinion of my characteristic verbosity and false sense of modesty, at the risk of using a reverse argumentum ad populum, I will point out that, in the several months since Dr. Spector’s article appeared, I had a hard time finding any medical skeptic with anything positive to say about it. Still, just because Dr. Spector’s article garnered a lot of negative attention in the skeptical blogosphere doesn’t necessarily mean that he didn’t make some valid points. No doubt, if he is as right as his dogmatic certainty would appear to indicate, Dr. Spector will have prepared excellent rebuttals to our criticisms that address each salient point that each of us raised, at least within the space constraints allowed (which, in all fairness, were significant), right?

Wrong.

Dr. Spector doubles down

The first thing I noticed reading Dr. Spector’s response to criticism is that Dr. Spector does not respond particularly well to criticism. Perhaps he should become a blogger. Of course, not responding well to criticism is not necessarily that horrible a thing in and of itself, as long as it doesn’t devolve into never listening to anyone and some well-intentioned criticism can still get through. On the other hand, these latter two conditions can be a death knell for a scientist, because one of the chief skills needed to succeed in science other than the ability to do good science is the ability to handle criticism with grace and to respond to it with evidence. Outside of science, though, the ability to double down when criticized can be a required positive job skill, particularly in politics, public relations, and, of course, punditry. How else can one explain Rush Limbaugh, Sean Hannity, and Ann Coulter? Like Limbaugh, Hannity, or Coulter, Dr. Spector reacts to criticism primarily by doubling down on his previous statements.

The second thing that I noticed in Dr. Spector’s response is that he is the poster child for a common saying we have in medicine that goes something like, “‘Perfect’ is the enemy of the good.” In other words, yes, in an ideal world it would be lovely if every single medical study had a perfect hypothesis and followed the rigorous methodology of the randomized clinical trial analyzed using exactly the correct statistical analysis. However, many questions don’t easily lend themselves to that approach, and, more importantly, I point out again that there is a difference between hypothesis-generating studies, which are generally smaller, less controlled, and sometimes don’t even have a hypothesis at all (after all, they’re called “hypothesis-generating” for a reason) and final, hypothesis-testing experiments and clinical trials that lead up to FDA approval. Both have their role in science, the former early on in the discovery process, the latter after the discovery process is well under way and approaching an actual, usable treatment that needs to be validated in a phase III randomized clinical trial. Also, such a rigid insistence on methodological perfection above all else smacks of what a favorite blogger of mine (alas, now retired) Revere called “methodolatry.” No doubt Dr. Spector would retort that I say that as though it were a bad thing, to which I would respond that it is a bad thing when taken to an extreme and Dr. Spector does take it to an extreme. Each type of study has its proper place in medical research, from basic science, to translational, to animal studies, to preliminary clinical trials, to the final, big phase III trials whose results are used to apply for FDA approval.

Be that as it may, the last thing that comes through in Dr. Spector’s response is that he appears to think that we do not understand his arguments or the basics of medical science. In fact, Dr. Spector arrogantly doubles down on his previously bad arguments while at the same time accusing skeptics of not understanding him:

It is clear from reading these letters that the correspondents do not understand how to evaluate the medical literature—how to discriminate the wheat from the chaff. They do not understand the hierarchical nature of proffered evidence. For example, I doubt any of the correspondents understand Hill’s criteria for assessing causal relationships in epidemiology/observation studies (see below). I recommend they read Spector and Vesell (2006) or the online methodological supplement to my paper on nutrition in SKEPTICAL INQUIRER (Spector 2009). They should also consult the FDA regulations for what constitutes proof in drug development.

I realize that we’re not supposed to use NSFW language here on SBM. Heck, I’m even the managing editor who says that we bloggers are not supposed to use NSFW language here on SBM, with Dr. Novella’s agreement. So instead of saying what I was thinking as I read this paragraph, I will instead say, “What a pair of fetid dingo’s kidneys!” In any case, I’ll simply point out that reading the medical literature and “separating the wheat from the chaff” is part of my job. It’s what I do, both in my “real world” job and as editor and blogger at SBM. Readers can peruse my work here at SBM and judge for themselves whether I (and all the other bloggers) do a good or poor job at it. As for not understanding “Hill’s criteria,” well, I must admit to a tiny chuckle when I read that sentence, given how often we’ve written about it here on SBM in the context of countering the use and abuse of Hill’s criteria by promoters of unscientific medicine. In fact, Mark Crislip, who wrote one of the letters, also wrote a rather nice discussion of Hill’s criteria. While no one would expect that Dr. Spector would have searched SBM for posts on Hill’s criteria, it is rather presumptuous of him to leap to such conclusions based solely on our letters.

It’s also rather presumptuous of Dr. Spector to assume that I (or others) didn’t read any of his other papers. For instance, I read Spector and Vesell (2006). I also read his SI article Science and Pseudoscience in Adult Nutrition Research and Practice. The latter article spent a great deal of verbiage to conclude that “In general, the most powerful method to establish the truth of many nutritional hypotheses, when contributory causality is postulated, is the prospective, randomized, controlled trial.” No kidding. Nowhere in the article, however, does Dr. Spector acknowledge just how difficult, sometimes impossible even, it often is to do a prospective, randomized, controlled trial studying hypotheses involving the role of nutrition in disease. Sure, it’s possible to do for a subset of nutritional trials (supplementation with individual vitamins or nutrients, for instance), but even then it’s not easy. When it comes to more general questions, such as whether a diet rich in certain fruits and vegetables prevents cancer, it might be either too expensive or too difficult to ensure adequate adherence to control and experimental diets, making such a trial impractical or even downright impossible. What would Dr. Spector have us as scientists do when confronting such questions? Throw up our hands and say, “Since we can’t do a rigorous, randomized, controlled clinical trial, I guess we’ll never know the answer”? Apparently so. I would counter: For such questions, science is either forced to use less rigorous methodology and try to determine if there will be a confluence of evidence from a variety of techniques that leads to an answer or give up, and I’m not one for giving up on important questions. Once again, Dr. Spector doubles down:

Three correspondents claim I don’t understand that the evaluation of clinical science is not easy. Gorski talks about the “messiness of science-based medicine.” Crislip states “medicine advances slowly and erratically.” Hall states that “scientists are doing the best they can.” What they do not understand is that one of the reasons for this sorry state of much of the clinical literature is that many published studies do not have a clear hypothesis, are underpowered, or use inadequate methods (e.g., epidemiology/observation studies attempting to “prove” causal connections).

One wonders what Dr. Spector thinks of the epidemiological studies that first demonstrated a causal link between smoking and lung cancer. It’s notoriously difficult (albeit not impossible), for example, to induce lung cancer in experimental rodents using tobacco smoke or extracts from tobacco smoke. It would also be highly unethical to do a randomized clinical trial to test whether smoking causes lung cancer in humans. Seriously. I ask Dr. Spector in all earnestness how he would have demonstrated that smoking causes lung cancer in humans, if not accepting strong results from epidemiological studies as sufficient.

Ironically, Austin Bradford-Hill (yes, that Hill, of Hill’s criteria) was a pioneer in demonstrating the link between smoking and lung cancer, and his criteria came partially out of that work. Teaming up with Sir Richard Doll, Austin Bradford-Hill published a case-control study in 1950 that showed that smoking greatly increases the risk of lung cancer. The two followed up that study with a long-term prospective study known as the British Doctors Study, which followed the health of 30,000 British doctors for several years, which was a prospective cohort study. Neither of these were controlled, randomized clinical trials. In any case, Hill’s criteria were designed specifically to determine when a disease or condition associated with an environmental exposure could be considered to be caused by that exposure, which, of course, can’t ethically be done in general through a randomized clinical trial. Thus, Dr. Spector implicitly recognizes that for environmental exposures causality can be scientifically inferred from a confluence of epidemiology and other evidence. For everything else, apparently, if it isn’t a randomized clinical trial, to Dr. Spector it’s crap. In any case, Dr. Spector seems to leap back and forth between advocating randomized clinical trials for everything and invoking Hill’s criteria, the latter of which do not necessarily require randomized clinical trials specifically, although #8 (experiment) can, depending on the specific exposure, be interpreted as requiring an RCT of some kind. The problem is, again, that because it’s not acceptable ethically to do an RCT for an environmental exposure that results in disease in a lot of questions, Hill’s criteria #8 will automatically go unfulfilled.

Dr. Spector seems particularly stung over the criticisms of his take on hormone replacement therapy, so much so that he spends a considerable fraction of his limited space in answering them, pointing out that estrogen can cause mammary cancer in female rodents (true) and that it can cause uterine cancer in humans (also true). All of this is rather beside main criticisms I had, given that in general I accept that mixed progestin/estrogen HRT increases the risk of estrogen receptor-positive breast cancer, although more recent evidence suggests that estrogen-only HRT probably does not, or if it does it does so so modestly that it is difficult to detect the increased risk in epidemiological studies. I also can’t help but point out that more recent work suggests that in some cases estrogen HRT can be protective against breast cancer. Once again, the evidence is more complex than how Dr. Spector represents it, as this analysis of the Women’s Health Initiative Estrogen-Alone Trial demonstrates.

In any case, Spector’s arguments miss the point entirely, as he piously proclaims:

In fact, I believe a good physician should use only proven therapies for which the risk/benefit ratio favors the patients. That was never the case with HRT, a known carcinogen. Fortunately, when my sister asked me if she should take HRT for her menopausal symptoms in the mid-nineties, I said no; it’s too risky and its benefits are uncertain.

How condescending of Dr. Spector! As if we believe that a physician should use unproven therapies for which the risk/benefit ratio doesn’t favor the patient! (Look for that sentence to be quote-mined some day.) The point was that it wasn’t so clear at the time and medicine has to use the best available evidence that it has at the time. Even now, the situation is not as clear as Dr. Spector makes it out to be.

Finally, I must admit that Dr. Spector truly disappointed me with his last paragraph, where he doubles down yet again, in essence referring readers to an article many of whose points were easily refuted:

To answer the questions on cancer chemotherapy, I recommend my SKEPTICAL INQUIRER article “The War on Cancer” (Spector 2010). Moreover, the problems with chemotherapy keep surfacing. For example, it was shown that for lung cancer patients, hospice care with minimalist chemotherapy was better than standard chemotherapy in terms of both quality of care and longevity (Temel et al. 2010). The scandals associated with the so-called targeted chemotherapy—such as the ineffective, incredibly expensive, and risky Avastin in breast cancer treatment—escalate (Tucker 2011). Finally, the authors of a recent long, thoughtful article (Smith and Hillner 2011) argue persuasively, as I did, that there is tremendous overuse of chemotherapy. The authors point out that “some oncologists choose chemotherapy in order to maximize their practice income. A system in which one half the profits in oncology [practice] are from drug sales is unsustainable.” Where is the well-being of the patient in this? A thoughtful, empathetic reader will weep.

Actually, I suspect that if Austin Bradford-Hill were alive today, he’d weep at Dr. Spector’s obsessive worship of his nine criteria to the exclusion of all else, but I digress.

When called out on his claims, note how Dr. Spector retreats and revises his argument to something that is more easily defensible. Remember, Dr. Spector didn’t argue just that chemotherapy is overused and that it doesn’t do much good in certain circumstances, such as advanced solid malignancies like non-small-cell lung cancer. He went far, far beyond simply suggesting that we need to be more science- and evidence-based in our use of chemotherapy. In fact, let’s revisit what he did say, which was to label chemotherapy as a “deadly medical hypothesis”:

From a cancer patient population and public health perspective, cancer chemotherapy (chemo) has been a major medical advance.

I also note that I spent quite a bit of verbiage in my last post on this issue refuting much of what Dr. Spector wrote in his “war on cancer” article; so I will not repeat it here. I also pointed out how including chemotherapy as a treatment for cancer as one of his Seven Deadly Medical Hypotheses was off-base and exaggerated the problems with chemotherapy as a cancer treatment far beyond the oncological evidence. On the one hand, while it disappoints me that Dr. Spector apparently never bothered to click on the link I provided to my original criticism of his article, it does not surprise me.

As for the rest, Temel et al is a relatively small but nonetheless provocative study telling us something we already know, namely that chemotherapy doesn’t do very much good for metastatic non–small-cell lung cancer, and I’ve already written quite a bit about the problems with Avastin in breast cancer. This is a controversy that has gone both ways in breast cancer even as Avastin has proven value in other cancers, such as colorectal cancer. Again, if all Dr. Spector had argued was that chemotherapy is overused in some cancers, he’d have gotten little or no argument from me. As for what Smith and Hillner actually argue in their NEJM article, it is not primarily that there is huge overuse of chemotherapy, but rather that oncologist behavior should be changed in the following ways:

1. Target surveillance testing or imaging to situations in which a benefit has been shown.
2. Limit second-line and third-line treatment for metastatic cancer to sequential monotherapies for most solid tumors.
3. Limit chemotherapy to patients with good performance status, with an exception for highly responsive disease.
4. Replace the routine use of white-cell–stimulating factors with a reduction in the chemotherapy dose in metastatic solid cancers.
5. For patients who are not responding to three consecutive regimens, limit further chemotherapy to clinical trials.

Among other things, they also suggest switching to symptom-directed care when chemotherapy success is highly unlikely. While there is an implication that chemotherapy is overused, the article is more about sticking as closely as possible to science- and evidence-based treatment and recalibrating expectations in order to activate hospice earlier in cases when treatment success can’t be expected. None of these are radical; none of these question the paradigm of chemotherapy use against cancers for which its use is evidence-based. All the authors do is to advocate for integrating hospice care and sticking to evidence-based chemotherapy regimens whose toxicity is lower, all in order to increase the quality of life of cancer patients.

Science: A spectrum of evidence

As I’ve pointed out before, Dr. Spector appears to be someone whose career has been primarily involved in drug development in the later stages. At least, if that’s not the case, I have a hard time figuring out how he can (1) conflate the necessary evidentiary burden required for hypothesis-generating experiments with late-stage randomized clinical trials designed to win FDA approval; (2) lecture critics with a straight face by telling us to read FDA regulations regarding what constitutes proof in drug development and referring us to a paper of his in which he concludes the breathtakingly obvious, namely that the controlled randomized clinical trial is the gold standard for clinical evidence; and (3) dogmatically present a simplified view of scientific and clinical evidence that brooks no ambiguity. All of this bespeaks a narrowness of vision that probably serves him well for defined, limited projects that ask questions like, “How can we design a drug to target this receptor?” and “How do we confirm or refute that the drug works?” However, when it comes to vision that will lead to medical breakthroughs that aren’t simply using existing knowledge to design better new drugs? Not so much, given his obvious contempt for the necessarily less rigorous preliminary studies, the majority of which will fail but a few of which will lead to more rigorous studies demonstrating the efficacy of a new therapy. In other words, sticking to Dr. Spector’s methodology will produce science that is rigorous, safe, and completely uninnovative. I also can’t help but wonder when was the last time Dr. Spector actually took care of a patient, given his obvious intolerance for ambiguity.

No one is arguing against more scientific rigor, least of all me, and where Dr. Spector notes deficiencies in the science being presented that seem reasonable, I’ll be with him. However, when he labels medical hypotheses that are not really hypotheses as deadly when they are not really deadly and then doubles down on the same pseudoskeptical arguments when criticized for them, I’m most definitely not with him there. It needs to be remembered that each of the types of experiments and studies used in science, from small basic science investigations to hypothesis generating genome-wide association studies and next generation sequencing studies of tumors, to the most rigorous phase III clinical trials designed to win FDA approval, are all tools designed to answer specific questions. Each has its place, and a “one size fits all” approach to scientific evidence doesn’t work. Demanding FDA-level evidence for every question makes little sense. This is not a straw man, either; it is what Dr. Spector genuinely appears to be advocating, at least to me. The problem is that Dr. Spector dogmatically clings to the methodolatry of his randomized clinical trials and his FDA regulations in much the same way that a minister, rabbi, or imam holds on to his holy book. That might be a form of science, but it is not good science, and it’s definitely not skepticism, which is why I hope not to see any more of Dr. Spector’s work in SI.

<!--<!--