I still expect our stroke medical professionals to create the following diet protocols. Not useless guidelines; PROTOCOLS!
For stroke prevention; for dementia prevention; for cognitive improvement; for cholesterol reduction; for plaque removal; for Parkinsons prevention; for inflammation reduction; for blood pressure reduction.
https://www.vox.com/science-and-health/2018/6/20/17464906/mediterranean-diet-science-health-predimed
Nutrition science is supposed to tell us how to eat. It’s in the midst of a crisis.
Share
The million-dollar question in nutrition science is this: What should we eat to live a long and healthy life?
Researchers’ answers to this question have often been
contradictory and confusing. But in recent decades, one diet has
attracted the lion’s share of research dollars and public attention: the
Mediterranean way of eating. And in 2013, its scientific cred was
secured with PREDIMED, one of the most important recent diet studies published.
The study’s delicious conclusion was that eating as the
Spanish, Italian, and Greeks do — dousing food in olive oil and loading
up on fish, nuts, and fresh produce — cuts cardiovascular disease risk
by a third. As Stanford University health researcher (and nutrition science critic) John Ioannidis put it: “It was the best. The best of the best.”
Not anymore. Last week, the prestigious New England Journal of Medicine pulled the original paper from the record, issuing a rare retraction. It also republished a new version of PREDIMED, based on a reanalysis of the data that accounted for the missteps.
PREDIMED was supposed to be an example of scientific excellence in a field filled with conflicted and flawed studies. Yet it now appears to be horribly flawed.
At first, I thought this could be the beginning of the
end of nutrition science. There have been too many poorly executed and
disappointing studies over the years, too many research dollars wasted.
(We’ve also just learned about industry influence in the National Institutes of Health’s alcohol studies, and infighting that brought down what was supposed to be the “Manhattan project for nutrition.”)
But after spending the past few days talking with some of
the brightest minds in nutrition research and epidemiology, I now feel
the PREDIMED retraction is actually cause for hope. Maybe even a new
beginning for the field.
Yes,
studies with big flaws pass peer review and make into high-impact
journals, but the record can eventually be corrected because of
skeptical researchers questioning things. It’s science working as it
should, and the PREDIMED takedown is a wonderful example of that. This
process should bring us a step closer to what really matters: informing
people who want to know how to eat for a healthy life.
Why the PREDIMED trial was such a big deal
As I’ve reported,
nutrition science has done a great job of finding ways to address
diseases of nutrient deficiencies, like scurvy. But today, our greatest
health problems relate to overeating. People are consuming too many
calories and too much low-quality food, bringing on chronic diseases
like cancer, obesity, diabetes, and cardiovascular disease.
These illnesses are much harder to get a handle on. They
don’t appear overnight but develop over years. They’re not usually
related to one cause; they’re caused by many lifestyle and genetic
factors in concert. And fixing them isn’t just a question of adding an
occasional orange to someone’s diet. It involves looking holistically at
lifestyle behaviors, like diet and genetics, trying to tease out the
risk factors that lead to illness.
The trouble is, most of what we know about nutrition’s effects on chronic disease comes from observational data.
Researchers track what large numbers of people eat over time and then
look at their rates of disease, trying to tease out relationships in the
data. Do people who drink more red wine have lower rates of heart
disease? Is meat associated with an early death?
Because these studies aren’t controlled like experiments,
they can’t tell us whether one thing caused another thing to happen.
Researchers try to use statistics to control for some of these
“confounding factors,” but it’s impossible to catch all of them.
That’s why the randomized controlled trial (RCT) is
considered the gold standard for evidence in health research. With these
trials, scientists randomly assign their study participants to one of
two groups (or sometimes more). One group gets a treatment or
intervention; a control group gets a placebo. Again, they give
participants these assignments in a random fashion, meaning the members
of the study have an equal chance of being selected for each group. And
if there’s a difference in health outcomes at the end of the study, it’s
fair to say the intervention was the cause.
But there’s a catch with RCTs: It’s extremely difficult
and expensive to run this kind of study for long enough and in numbers
that are large enough to yield meaningful answers.
This is exactly why PREDIMED stood out. “It was a
randomized control trial, it was long-term, and it had clinical outcomes
on things that mattered,” said Ioannidis. “It was the prototype of the
best that had been done.”
What PREDIMED found
The study was conducted in Spain and tracked more than 7,400 people at high risk for cardiovascular disease, recruited through 11 study sites.
The study participants were randomly assigned to one of
three groups: getting advice about following a Mediterranean diet as
well as free extra-virgin olive oil delivered to their home; getting
advice about following a Mediterranean diet as well as free nuts to
their doorstep; and, for the control group, getting advice about
following a low-fat diet.
The main endpoints the researchers tracked were major
cardiovascular events — strokes, heart attacks, death from
cardiovascular causes. They stopped the trial early, after a median
follow-up time of nearly five years, because the effects of the diet
seemed to be so dramatic. The Mediterranean diet, when supplemented with
lots of olive oil or nuts, could cut a person’s risk of cardiovascular
disease by a third.
It’s rare to see an effect size that big, even in studies
on medications. What’s more, the study’s early halting seemed to
exaggerate the results in the minds of the public.
“The magnitude of the diet’s benefits startled experts,” the Times reported
when the trial came out. “The study ended early, after almost five
years, because the results were so clear it was considered unethical to
continue.”
What went wrong with the study
The findings helped heave olive oil and nuts into the realm of the sacred. They were catnip for journalists (“Mediterranean Diet Shown to Ward Off Heart Attack and Stroke,” the New York Times story read. “Spanish Test: Mediterranean Diet Shines In Clinical Study,”
proclaimed NPR’s headline.) And they spawned a cottage industry of
studies by other scientists, who used the PREDIMED data to run hundreds
of different analyses.
But it turns out the trial wasn’t properly run.
John Carlisle, a British anesthesiologist (and legend in medical statistics), first called attention to the fact that something seemed fishy about PREDIMED in this June 2017 paper.
In the study, he applied a statistical test to 5,087 randomized trials,
including PREDIMED, to see if the groups randomized in the studies were
too similar to happen by chance. He found they were, suggesting
problems with the trial’s randomization process.
One of PREDIMED’s authors, nutrition researcher Miguel Ángel Martínez González,
told Vox that upon seeing the Carlisle paper, he reached out to his
colleagues on PREDIMED’s steering committee about reexamining the data.
He told me they also wrote to the NEJM about the potential errors, which led to a plan to recover old data and fish for errors — though the editor of the journal told Science it was they who reached out to the researchers.
Martínez González, his colleagues, and editors at NEJM spent months poring over the data. They discovered several major problems:
- The researchers at one of the study sites, overseeing 11 clinics, failed to randomly assign individual people to the different diets, instead assigning everybody in the same clinic to the same diet.
- In cases where more than one person in a family was participating in the study, they were all assigned to the same diet, again instead of being randomized. (And while the authors knew about this deviation, it wasn’t reported in their original study as it should have been.)
- Researchers at another study site failed to properly use the randomization table, which is supposed to guide researchers in how the randomization is done.
Altogether, these errors meant that some 1,588 of the
study’s participants — or a little more than 20 percent — weren’t
properly randomized in one way or another.
Needless to say, this isn’t how a randomized trial should
work. The researchers should have made sure each study site and clinic
involved in the trial was following strict procedures on how to
randomize, continually following up with and checking in on them,
Indiana University School of Public Health dean David Allison told me.
Martínez González insists that the problems with the
randomization were not clinically meaningful, that they don’t
fundamentally change the conclusions of the study. “The aim of
randomization is to get balanced groups. There were no imbalances in the
three groups regarding the clinical characteristics,” he said.
Mainly for that reason, the researchers only slightly watered down their conclusions in the revised version of the study
published on June 13. “We are now more sure than ever of the strength
of PREDIMED and the robustness of the results,” Martínez González told
me. “We honestly reported these small departures from the strict
randomization procedure — but this has no importance regarding the
methods and results of the trial.”
“We
have seen that this is the deepest scrutiny ever done to a trial,” he
added. “I think that if you put the magnifying glass to any published
trial, things will appear.”
Other scientists remain skeptical of the retooled PREDIMED
Researchers who weren’t involved in the study aren’t so
sure about PREDIMED’s retooled conclusions. They feel the randomization
problems suggest sloppiness or deliberate data manipulation and that
upon further scrutiny, more errors will indeed materialize that will
shake PREDIMED’s conclusions even more.
“They only came to know what was wrong with the
randomization after it was published? That’s really weird, and then I
get nervous about [other] details,” said Cecile Janssens,
a research professor of epidemiology at Emory University. “It was weird
they retracted it and weird they republished it immediately, and I feel
there may be other things wrong with this study.”
On her Twitter feed, she’s now pointing out flaws in the retooled report and asking others to help her do the same:
Hilda Bastian, who also specializes in picking apart trials, has pointed out other red flags in the rebooted trial on her blog.
“The reboot, and the choice of data and calculations reported, leave a
lot of questions,” Bastian told me over email. “It begs the question of
how impartial a journal is able to be, when one of its star papers
explodes.”
For Richard Bazinet, a University of Toronto nutrition
researcher, the biggest worry isn’t just that the study itself is wrong;
it’s all the other studies out there that are based on PREDIMED’s data.
“People have taken that [PREDIMED] data set and
reanalyzed it for other things over and over and over again. There are
probably a hundred papers in the literature. I can’t even figure out
what the process is going to be to [fix this],” he said. “That’s the
thing that’s got me up at night.”
PREDIMED should be a beginning, not an ending
We can probably expect to hear more about PREDIMED in the
coming months. And the only way to move toward better and more robust
nutrition science is to learn from the debacle.
“A big [lesson] is how important it is to take post-publication critiques seriously and act on them,” said Bastian.
While many people had pointed to flaws in the trial over
the years, it was Carlisle’s statistical critique, and the fact that it
wasn’t ignored, that led to the retraction. “The better, critical
systematic reviews had already pegged this study as being particularly
weak evidence, and they needed to be taken more seriously too,” she
added.
Another big issue is the need for improved transparency,
Bastian said. “For such a big trial, the protocol, methodological
detail, and even data are extraordinarily skimpy in key areas — and that
goes for the reanalysis too.”
“I wouldn’t give up [on nutrition science],” said
Ioannidis. “But we need to take at a minimum some steps forward: Perform
more, long-term, large randomized control trials — like PREDIMED but do
it right. We need to share the data, and make them publicly available,
have people be able to look at them and see that they get the same
results.” People in many other fields of science, notably psychology, are calling for similar steps to strengthen science.
For now, he added, “[Nutrition epidemiology] is a field
that’s grown old and died. At some point, we need to bury the corpse and
move on to a more open, transparent sharing and controlled experimental
way.” PREDIMED is probably that point.
So what should I eat?
The PREDIMED retraction isn’t a total dismissal of
Mediterranean eating patterns, but it’s a reminder of just how difficult
it is to come by clear advice on precisely what people should eat for
their best health.
Layered on top of the difficulties with studying
nutrition is the fact that people have very strong feelings about food —
from scientists to study authors to the media — and these feelings bias
the research and how it’s interpreted. Food is cultural, it’s social,
it’s about our family histories and where we grew up, and it’s something
we all have experience with, Allison noted. It’s no wonder “people
become zealots” when they talk about diet.
Despite what the zealots say, there’s lots of data
suggesting humans can survive — and even thrive — on myriad eating
patterns. The optimal diet for any individual probably varies, but in
general, you can’t go wrong with lots of fruits and vegetables, little
junk food, and not too many calories. If you don’t smoke too much, don’t
drink too much, and exercise regularly, you’ll probably be okay. Of
courses, that’s easier said than done for the vast majority of people.
No comments:
Post a Comment