Why is my mind so clean and pure?  Because I am always changing it.

In medical school the old saying is that half of everything you learn will not be true in 10 years, the problem being they do not tell which half.

In medicine, the approach is, one hopes, that data leads to an opinion.  You have to be careful not to let opinion guide how you evaluate the data.  It is difficult to do, and I tell myself that my ego is not invested my interpretation of the data. I am not wrong, I am giving the best interpretation I can at the time. For years  I yammered on about how it made no sense to give a beta-lactam and a quinolone for sepsis until a retrospective study suggested benefit of the combination.  Bummer. Now when I talk to the housestaff about sepsis, I have to add a caveat about combination therapy.  It is why my motto is, only half jokingly,  ”Frequently in error, never in doubt”.

At what point do you start to change you mind?  Alter your message as a teacher?  Have new behavior?  Medicine is not all or nothing, black and white.  Changes are incremental, and opinions change slowly, especially if results of a new study contradict commonly held conclusions from prior investigations.

Nevertheless, I am in the process of changing my mind, and it hurts.  I feel like Mr. Gumby.

It is rare that there is one study that changes everything; medicine is not an Apple product.  Occasionally that there is a landmark  study that alters practice in such a dramatic way that there is a before and after.  As I write this I cannot think of a recent example in infectious diseases, but I am sure there is one.  The problem is that once practice changes, it seems as we have always done it that way.

For me, three is the magic number.  One study that goes against received wisdom warrants an ‘interesting, but give me more.’

Two studies, especially if using different methodologies with the same results gives and well, two is interesting, but I can argue against it.’ However, with two studies the seed of doubt is planted, waiting to be watered with the water of further confirmation.  Yeah. Bad metaphor.

Three studies with different methodologies independently confirming new concepts?  Then I say, ‘I change my mind. My brain hurts.’

There are now three studies concerning the issue of efficacy of the flu vaccine in the elderly.  You might remember my discussion of the Atlantic article several months ago. In that entry I discussed two articles  that suggested the flu vaccine may be less effective in the elderly than the studies demonstrated.

The argument was that the elderly who received the influenza vaccine were healthier at baseline than those that didn’t receive the vaccine and the deaths during flu season was not due to the protection from the vaccine, but due to the fact that healthier people are less likely to die when they get ill. In part this was demonstrated by showing decreased deaths in vaccinated populations when influenza was not circulating.  If insomnia is a problem, you can go back and read my post.   To quote my favorite author, me, I said

“One, it is an outlier, and outliers need confirmation. The preponderance of all the literature suggests that influenza vaccine prevents disease and death. If you do not get flu, you cannot die from flu or flu related illnesses. When outliers are published, people read them, think, “huh, that’s interesting”, but there is going to have to be more than one contradictory study to change my practice. But if “study after study” shows mortality benefit, and one study does not, it is food for thought, but not necessarily the basis of changing practice. The results, above all, needs to be repeated by others… In medicine we tend to be conservative about changing practice unless there is a preponderance of data to suggest a change is reasonable. Except, of course, if our big pharma overlords take us to a good streak house.”

Now we have a third article, “Evidence of Bias in Studies of Influenza Vaccine Effectiveness in Elderly Patients” from the Journal of Infectious Diseases.

In the study they examined the records of the elderly in the Kaiser Health System, their vaccination records, and their risk of death.  And the results were interesting.

“The percentage of the population that was vaccinated varied with age. After age 65, influenza vaccination increased until age 78 in women and age 81 in men, then decreased with increasing age. Vaccination coverage also varied in a curvilinear fashion with risk score, increasing with risk score to a risk score percentile of ?80%, then decreasing. In addition, as the predicted probability of death increased, vaccination coverage increased. Vaccination coverage was highest among members with a probability of death of 3%–7.5%. Those with a predicted probability of death in the coming year of 17.5% had a decreasing likelihood of influenza vaccination”

They then looked at mortality when flu was not circulating.

“A change in the pattern of vaccination had a striking effect on mortality. For members > 75 years old who had been receiving influenza vaccinations in previous years, not receiving a seasonal influenza vaccination was strongly associated with mortality in the months ahead (Table 1). A person who had received an influenza vaccination every year in the previous 5 years had a more than double probability of death outside the influenza season if he or she missed a vaccination in the current year, compared with a person who was vaccinated as usual (odds ratio, 2.17; P < .001). On the other hand, if a person did not receive any seasonal influenza vaccination in the previous 5 years, then receipt of a vaccination in the current year was associated with a greater probability of death. “

If they had a history of flu vaccine for five years and missed it, the probability of death went up.

If they did not have a flu vaccine for five years and got one, the probability of death went up.

They suggest in the first case, the patients may have had an increase in their co-morbidities and as a result did not get the vaccine and died of underlying diseases. Their increased risk of death was from accumulating prior illnesses.

In the second case, people who were healthy and did not seek care subsequently developed diseases that lead them to a doctor who advised the vaccine.  Their increase risk of death was due to new illnesses.

Either way, the uptake of the flu vaccine is more complicated than I had suspected and makes interpretation of efficacy of the vaccine in prior studies harder to evaluate.  The table shows an unexpected relationship between age, risk of death and use of the flu vaccine.

They say in the discussion

“We showed that, despite strong efforts to increase vaccination among the elderly population, vaccination is relatively low in the oldest and sickest portions of the population. Persons 65 years old with a 17.5% chance of death in the upcoming year are less likely to receive the influenza vaccine. Because persons who are most likely to die are less likely to receive the vaccine, vaccination appears to be associated with a much lower chance of dying; thus, the “effectiveness” of the vaccine is in great part due to the selection of healthier individuals for vaccination, rather than due to true effectiveness of the vaccine. Previous studies have argued that worsening health is associated with increasing vaccination. We found this to be a curvilinear relationship, in which increasing illness means increasing vaccination, up to a point, and then, as people come closer to the end of life, there is a decrease in vaccination coverage.”

They do not say the vaccine is not effective, but they suggest that there is a bias that may make the vaccine appear more effective in the elderly than it really is.  Reality is often more complex than one would think at the beginning.

After three studies I am reasonably convinced that efficacy of the flu vaccine in the elderly is potentially not as well understood as I had thought.

So do I think the flu vaccine is no longer useful in the elderly?  No.  I still think it is a reasonable intervention but it may not have the efficacy I would like.  But I have always known that, for a variety of reasons, the flu vaccine is not a great vaccine. But it is better than no vaccine. There are, as discussed in the earlier post on the vaccine, many lines of evidence to show that the flu vaccine has benefit; at issue is the degree of the benefit.  Perhaps what is needed is a better vaccine with adjuvants or multiple injections to get a better result in the elderly, who respond poorly to the vaccine.  Or perhaps it will be better to focus on increasing vaccination in those who care for or have contact with the elderly.  But when I talk to my patients and residents, when I get to part about flu vaccine efficacy, I will be a little more nuanced, use more qualifiers. I will tell them that the vaccine is like seat belts.  It does not prevent all death and injury, but if you had a choice, would you not choose to use seat belts?

In the end the data has to change the way I think about medicine, not matter how much it hurts.

Compare and contrast that with the anti-vaxers who have the belief that vaccines cause autism.  They look for data to support the pre-existing belief and ignore contrary data.  Opinion does not follow from data.

The most representative statement of their approach is on the 14 studies website where they say  ”“We gave this study our highest score because it appears to actually show that MMR contributes to higher autism rates.”

The key phrase in the whole site. Data that supports their position is good, data that does not is bad. What makes a study good is not its methodology or its rigor, or its reproducibility, or its biologic plausibility,  but if it supports vaccines casing autism.

Dr. Wakefield, as has been noted over the last week, had his MMR/autism paper withdrawn from Lancet not for bad science, but for dishonest science.  In medicine you can be wrong, but you cannot lie.  If the results of medical papers were shown to be fabrications, such as the papers of Scott S. Reuben, no one the medical field would defend the results.  Dr. Reuben, as you may remember, was found to have fabricated multiple studies on the treatment of pain.  Nowhere can I find web sites defending his faked research.  No suggestions it was due to a conspiracy of big pharma to hide the truth. No assertions that he is still a physician of great renown.   He lied and is consigned to ignominy.   Physicians who used his papers as a basis of practice no longer do so, or so I would hope.

The response to Dr. Reuban is in striking contrast to the defense of Dr Wakefield, where bad research combined with unethical behavior, results in

“It is our most sincere belief that Dr. Wakefield and parents of children with autism around the world are being subjected to a remarkable media campaign engineered by vaccine manufacturers reporting on the retraction of a paper published in The Lancet in 1998 by Dr. Wakefield and his colleagues.

The retraction from The Lancet was a response to a ruling from England’s General Medical Council, a kangaroo court where public health officials in the pocket of vaccine makers served as judge and jury. Dr. Wakefield strenuously denies all the findings of the GMC and plans a vigorous appeal.”

Opinions did not change when the Wakefield paper was demonstrated to be not just wrong but false, the researcher’s behavior unethical, and the study could not be reproduced using similar methodologies.  Instead, the defense of Dr. Wakefield became, well, like a Jim Carrey shtick. The Mask defends retracted autism research. Fire Marshall Bill on the medical literature.  Jenny and Jim’s defense does make more sense read as comic performance art.  Andy Kaufmann would have been proud.

I wonder if the more grounded in fiction an opinion is, the harder it is to change, the more difficult it is to admit error.  I have to admit I cannot wrap my head around the ability of people to deny reality.  It is the old Groucho line come to life, “Who are you going to believe, science or your lying eyes?”

So I will, I hope, keep changing my mind as new information come in.  It is what separates real health care providers from acupuncturists and homeopaths and naturopaths and anti-vaxers.  It is what some truly great minds admit to doing.  As one deeper thinker and better writer than I said, kind of,

“The other terror that scares us from self-trust is our consistency; a reverence for our past act or word, because the eyes of others have no other data for computing our orbit than our past acts, and we are loath to disappoint them.

But why should you keep your head over your shoulder? Why drag about this corpse of your memory, lest you contradict somewhat you have stated in this or that public place? Suppose you should contradict yourself; what then? It seems to be a rule of wisdom never to rely on your memory alone, scarcely even in acts of pure memory, but to bring the past for judgment into the thousand-eyed present, and live ever in a new day. In your metaphysics you have denied personality to the Deity: yet when the devout motions of the soul come, yield to them heart and life, though they should clothe God with shape and color. Leave your theory, as Joseph his coat in the hand of the harlot, and flee.

A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines and anti-vaxers. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day. — ‘Ah, so you shall be sure to be misunderstood.’ — Is it so bad, then, to be misunderstood?”

I get a lot of email asking me about various alternative therapies and supplements. A recurring theme on this blog has been the hyperbolic claims of alternative practitioners and supplement makers, and while I can’t answer every email, I can at least address some of them in the blog. Supplements are often marketed using unsupported health claims to which is appended the Quack Miranda Warning, essentially allowing the makers to say that the pill will have such and such a benefit, while simultaneously denying any responsibility for the claim.  Since the FDA isn’t examining these claims, it’s worth while to ask our own questions.

The latest email concerned a product called CardioFuel. Let’s take a closer look at this stuff.

According to the distributor:

CardioFuel is the most profound energy producing supplement on the market today! It does something like no other can: Increase energy at the most basic metabolic level, by increasing ATP (the biochemical energy unit of transfer) production. More ATP means more energy reserves to overcome chronic disease, beat the competition, and handle the everyday stressors of today’s fast paced world!

So to be taken seriously, there should be evidence that this product: 1) increases ATP, 2) increases “energy reserves”, and 3) helps overcome chronic disease and “the competition”. First, it is not possible to directly measure ATP in a human being under normal clinical conditions, so any claims about this must be an inference from markers of ATP metabolism, or a guess. We’ll see what the literature says about this below. Second, we need an operational definition of “energy reserves”. Does this mean fat stores? Glycogen stores? These things are measurable to an extent.  Finally, we can do a literature search to see if CardioFuel or an acceptable analog has been tested for its effect on relevant outcomes.

First, what is ATP?

ATP is adenosine triphosphate, a biological molecule with many functions, among them the transfer of energy. ATP is produced in several ways, most famously in the Krebs cycle, a complicated biochemical process which premeds are mercilessly forced to memorize. ATP contains three phosphate bonds, and the third bond contains a great deal of energy, energy that the body uses to fuel many biochemical processes. Each molecule of ATP contains a d-ribose moiety, a simple sugar upon which the molecule is built. One of the claims being made by the CardioFuel folks is that if we ingest more d-ribose, we can make more ATP and be more “energetic”.

First, ATP synthesis, like most biochemical processes, is subject to feedback regulation; ATP production and its byproducts feed back to reduce further ATP production. Second, it is not clear to me that simply providing more of this particular substrate would significantly boost ATP production. But with my limited knowledge of biochemistry, it seems like an interesting question to investigate.

CardioFuel claims that its “ATP boosting” properties are  not just from ribose, but also from other molecules such as carnitine and coenzyme Q10.  There is nothing in the published literature that I can find to support these claims.

From my perspective as a physician, I want to see results.  Regardless of what is posited to happen at the cellular level, I want to see outcomes studies supporting the claims that d-ribose, or preferably CardioFuel itself actually does what is claimed, which the marketer further specifies:

I created CardioFuel to help my patients who suffer serious energy-depleting chronic diseases, such as: heart disease (PVD), diabetes, neuromuscular disease, fibromyalgia, lung disease, Chronic Fatigue Syndrome, kidney disease, HIV/AIDS, etc. These diseases deplete ATP, which CardioFuel rapidly replaces, dramatically improving health, vitality, and Quality of Life (QOL).

A PubMed search for “CardioFuel” turned up nothing.  Of the ingredients claimed to boost energy, d-ribose is the one with the most literature—literature which is not favorable.  There are a number of pilot studies looking at d-ribose in exercise.  For example, one study of twelve cyclists found that, “D-ribose supplementation has no impact on anaerobic exercise capacity and metabolic markers after high-intensity cycling exercise.”  A slightly larger study of rowers compared dextrose (d-glucose) with ribose and found that, “…the dextrose group showed significantly more improvement at 8 weeks than the ribose group.”  Another small study found that,” ribose had no effect on performance when taken orally, at the dose suggested by the distributor.”Since I couldn’t find literature specific to CardioFuel, and my literature search failed to find significant support for claims of the ingredients of CardioFuel, I went to the CardioFuel website for further guidance.  Tellingly, there are no references to studies of CardioFuel itself but only for its purported ingredients. There are some chaotic lists of incomplete references.  For example, for coenzyme Q10, some of the references are listed, some just quoted without citation, and there is little data related to the claims of CardioFuel.  The section on “elite athletes” addresses ribose directly, but not with outcomes studies.

I can find no support for the claims made by the sellers of CardioFuel. But under the Dietary Supplement Health and Education Act of 1994 they are allowed to make these claims as long as they use the Quack Miranda Warning. I couldn’t find such a warning on any page of the website so I guess I must assume that all of the claims made by the company are verifiable. I can’t seem to verify them myself, but the data must be somewhere. Right?

Last week I wrote about a study that purported to show that antidepressants have no effect in mild to moderate depression. A careful reading of the paper shows that the authors dramatically overstated their findings, particularly in their public statements to the media. The study has another implication beyond the misleading claims about antidepressants. It is an object lesson in an ongoing and disturbing phenomenon in mainstream journalism, the wholesale reprinting of press releases of scientific papers instead of reading and analyzing the papers themselves.

Pick up any newspaper or magazine and you can read about the latest scientific breakthroughs in cancer, Alzheimer’s or heart disease. Just keep in mind that what you are reading is probably a commercial message direct from the authors, not an accurate representation of the paper itself. Medical journalists are supposed to interpret the findings of recent medical publications and present them to the general public in ways that they can understand. They are supposed to provide context for the discovery, explaining what it might mean for disease treatment or cure. Yet, they rarely do. Instead, they simply copy the press release.

Most people are unaware that scientists issue press releases about their work and they are certainly unaware that medical journalists often copy them word for word. Instead of presenting an accurate representation of medical research, medical journalists have become complicit in transmitting inaccurate or deceptive “puff pieces” designed to hype the supposed discovery and hide any deficiencies in the research.

Imagine if a journalist reviewing the newest Ford cross-over vehicle didn’t bother to drive the car, but simply copied the Ford brochure word for word. Could you rely on the journalist’s evaluation? Of course not. Yet that is precisely what medical journalists are doing each and every day.

Not surprisingly, there is a scientific paper describing recent trends in medical press releases. The paper in the of Annals of Internal Medicine, Press Releases by Academic Medical Centers: Not So Academic?, by Woloshin, and colleagues finds:

Of all 113 releases about human studies … [f]orty percent reported on inherently limited studies (for example, sample size <30, uncontrolled interventions, … or unpublished meeting reports). Fewer than half (42%) provided any relevant caveats…

Among the 87 releases about animal or laboratory studies, most (64 of 87) explicitly claimed relevance to human health, yet 90% lacked caveats about extrapolating results to people…

Twenty-nine percent of releases (58 of 200) were rated as exaggerating the finding’s importance…

Almost all releases (195 of 200) included investigator quotes, 26% of which were judged to overstate research importance…

Although 24% (47 of 200) of releases used the word “significant,” only 1 clearly distinguished statistical from clinical significance. All other cases were ambiguous …

Why is this a problem? The harm extends beyond the obvious point that it is deceptive, and a failure of medical journalists to do their job, which is to interpret the accuracy and relevance of scientific publications when writing about them. Because medical journalists credulously publish press release as if they were true, they are constantly publishing conflicting reports, contributing to the public’s distrust of medical research. Each day seems to bring a new report of a food, or a drug that will prevent or cure cancer. Within a week or a month or a year, the journalists are reporting that that food or drug does not prevent or cure cancer.

To the public, it looks like medical researchers are constantly making mistakes. Today they claim that a food will prevent cancer. Next month, the same food will be found to cause cancer. In reality, medical research never demonstrated either claim, but medical journalists reported preliminary findings or flawed research as if they were definitive even though that was untrue.

The willingness of journalists to pass on the information in press releases without checking is not just a function of laziness. Journalists often lack the knowledge of science and statistics that is needed to analyze the paper. Moreover, journalists appear to suffer from a misunderstanding of the scientific literature. Publication of a scientific paper is not the end of a process confirming the truth of a paper; it is only the beginning. Publication does not mean that the findings should be accepted uncritically; it merely means that the findings are worthy of being included in the ongoing public discussion that characterizes science.  The findings of the paper may ultimately be deemed worthless or wrong.

The Annals of Internal Medicine has done an important service in bringing this disturbing practice to light. You can’t believe what you read about medical research in newspapers and magazines because medical journalists are simply copying press releases, not analyzing the research for accuracy or relevance. Therefore, in the interest of accuracy and relevance, I must disclose a caveat to this important scientific paper. In what surely is an unintentional irony, The Annals of Internal Medicine publicly unveiled the paper and its findings by issuing a press release.