Errors trigger retraction of study on Mediterranean diet's heart benefits

Ask just about anybody, and you'll probably hear that a healthy diet is one full of fruits and vegetables, olive oil, nuts and fish — what's called Mediterranean diet. A lot of research has suggested people who eat this way tend to be healthier, but it's been harder to prove whether that is because of the diet or some other factor. So in 2013, many took notice of a study in the New England Journal of Medicine that seemed to provide some proof. The study found that people eating the Mediterranean diet supplemented with olive oil were 30 percent less likely to experience a heart attack, stroke, or death from cardiovascular causes than people assigned to a low-fat diet. People who stuck with a Mediterranean diet supplemented with mixed nuts had a 28 percent lower risk than those asked to follow a low-fat diet. The results got wide media attention, including from NPR. But the New England Journal of Medicine retracted the paper Wednesday because of problems in the way the study was carried out. A retraction is a last resort for medical and scientific journals, a sign to readers that the results are no longer trustworthy and are beyond correcting. Although retractions are relatively rare — taking down fewer than 1 in 1,000 published articles — the rate is increasing. The authors of the NEJM paper are replacing the 2013 paper with a corrected version that shows people following the diet had a similarly reduced level of heart attacks and strokes. The major change is softer language about the conclusions.

The revised paper says only that people eating the Mediterranean diet had fewer strokes and heart attacks, not, as the original paper claimed, that the diet was the direct cause of those health benefits.

Of course, a change in one paper — even a high-profile one — doesn't mean that researchers have lost faith in the benefits of the Mediterranean diet. "I don't know anybody who would turn around from this and say, 'Now that this has been revealed, we should all eat cotton candy and turn away from the Mediterranean diet,' " says David Allison, dean of the School of Public Health at Indiana University in Bloomington.

Still, without the strong language touting the benefits of the diet in the original paper, "the legs have been kicked out from under it, in some sense," says Allison, a longtime nutrition researcher whose research has at times been partially funded by the food industry.

What prompted the Mediterranean diet researchers to soften their language?

A persistent British anesthesiologist named John Carlisle.

He knew very little about analyzing the details behind clinical trials until a few years ago, when he wrote a letter to an anesthesiology journal bemoaning the fact that his field was polluted by one researcher's data that many suspected were problematic. The journal editor told Carlisle, a practicing anesthesiologist at Torbay Hospital in Torquay, England, to prove it.

So Carlisle did. He read up on statistical methods and looked over more than 160 trials by the researcher, Dr. Yoshitaka Fujii, and analyzed how likely it was that the people had been randomized to different treatments. Randomization is part of the gold standard for clinical studies because it reduces the risk of bias and allows researchers to determine cause-effect relationships.

Carlisle found the odds were infinitesimally small that Fujii had randomized people properly. Since Carlisle's findings were published in 2012, medical journals have retracted more than 160 papers by Fujii — the most retractions for any one researcher, by a large margin, according to Retraction Watch. Carlisle's analysis drew accolades — and added him to the ranks of a small but growing number of such sleuths. The success of the approach prompted Carlisle to expand his scope to include randomized trials from other researchers and in other fields. After the Fujii paper, he spent two years applying the "Carlisle method" to thousands of clinical trials published in eight journals. In 2017, he reported that roughly 2 percent showed signs of problems. Among the more than 5,000 trials Carlisle analyzed, 934 were published by the New England Journal of Medicine. He flagged 11. One was the 2013 study showing the Mediterranean diet was responsible for the reduced rates of heart attack and stroke. Today, NEJM is replacing the 2013 study and correcting five others on different topics. The Mediterranean diet paper is being retracted because it turns out Carlisle was correct. When the lead author of the paper, Dr. Miguel Ángel Martínez González of the University of Navarra Medical School, saw Carlisle's analysis, he conducted a detailed audit of the study and quickly noticed some problems.

It turns out approximately 14 percent of the more than 7,400 study participants hadn't been assigned randomly to either the Mediterranean diet or a low-fat one. When couples joined the study together, both had been picked to follow the same diet. At one of the 11 participating study sites, the lead investigator had assigned the same diet to an entire village and didn't tell the rest of the investigators.

"This affected only a small part of the trial," says Martínez González. When the researchers reanalyzed the data excluding the nonrandomized people, the results were the same, he adds.

Still, because everybody wasn't randomly assigned to different groups, the study can no longer claim the diet directly caused those health benefits. "We need to tone down the results, but it is just a little bit," he says.

Many other studies have shown that people eating the Mediterranean diet have lower risks of various ailments, without claiming cause and effect.

But would the New England Journal of Medicine, one of the world's most selective and prestigious journals, have published the Mediterranean diet paper from Martínez González if the randomization issues and softened conclusion had been included in the first place? "It is difficult to answer hypothetical questions," New England Journal of Medicine spokeswoman Jennifer Zeis says. "We believe the evidence [in favor of the diet] is still strong, but not as strong as a randomized study in which the randomization was executed flawlessly." Carlisle praised the journal's response. "I think that the NEJM editorial team responded very maturely to my paper," he says. "They took the possibility of a problem seriously and acted quickly and thoroughly."

Alison McCook is the editor of Retraction Watch, a website that reports on scientific retractions and related issues. It is a project of The Center for Scientific Integrity, a nonprofit organization that has been funded by the John D. and Catherine T. MacArthur Foundation, the Laura and John Arnold Foundation and the Helmsley Charitable Trust. This story is a partnership with Retraction Watch, which you can find on Twitter: @retractionwatch Copyright 2019 NPR. To see more, visit https://www.npr.org.