https://www.medpagetoday.com/blogs/themethodsman/69974?
Beware of "Lord's paradox"
A randomized trial found that ginkgo biloba may improve cognitive function after an ischemic stroke. But the study may not be fully-flowered, as F. Perry Wilson, MD, explains in this 150-Second analysis. In fact, it may be a prime example of a statistical artifact known as "Lord's paradox."
A randomized trial, appearing in the journal Stroke and Vascular Neurology, found that ginkgo biloba – the ubiquitous supplement – improved cognitive function among patients with acute ischemic stroke.
Or... did it? The trial brings up an interesting statistical quirk, but one that can have big implications for studies like this.
Researchers in China enrolled 348 patients who had an acute ischemic stroke within the past 7 days. They were randomized 1:1 to the ginkgo group or usual care.
This is problem number one. No placebo control. Patients knew they were getting ginkgo – an ancient herb that might help improve their brain function. That knowledge alone might be enough to move the needle a bit on cognitive tests.
How much was the needle moved?
The primary outcome here was based on the Montreal Cognitive Assessment, or "MOCA."
This is a validated test for cognitive impairment, with higher scores being better – much like the Mini-Mental Status Exam we know and love.
Take a look at the MOCA scores over time in the two arms of the trial.
At the beginning of the trial, the MOCA scores in both arms of the study were about the same – 19 or so. By 180 days later, scores in both groups had gone up a bit – a bit more in the ginkgo group, but not enough to achieve statistical significance.
But I said earlier that ginkgo "improved outcomes." What gives?
Well, the primary outcome was not just MOCA score, it was change in MOCA score from baseline to 180 days.
What you see here is that change, and indeed, the ginkgo group improves more over time than the control group.
But how is that possible? How can the average MOCA score be the same in both groups, but the change in MOCA score be significantly different? Well this is a statistical phenomenon known as Lord's paradox and it can happen when one simply examines change scores without taking into account baseline scores.
This picture illustrates Lord's paradox.
You can see how, overall, the scores end up in pretty much the same place, but on the individual level, every person in the treatment group does better than every person in the control group.
So what we're seeing is a rather weak effect. A strong ginkgo effect would be visible regardless of how you analyze the data. And the lack of an adequate placebo control is probably enough to account for the small observed benefit. So for now the data on this particular herb is not fully-flowered.
F. Perry Wilson, MD, MSCE, is an assistant professor of medicine at the Yale School of Medicine. He is a MedPage Today reviewer, and in addition to his video analyses, he authors a blog, The Methods Man. You can follow @methodsmanmd on Twitter.
A randomized trial, appearing in the journal Stroke and Vascular Neurology, found that ginkgo biloba – the ubiquitous supplement – improved cognitive function among patients with acute ischemic stroke.
Or... did it? The trial brings up an interesting statistical quirk, but one that can have big implications for studies like this.
Researchers in China enrolled 348 patients who had an acute ischemic stroke within the past 7 days. They were randomized 1:1 to the ginkgo group or usual care.
This is problem number one. No placebo control. Patients knew they were getting ginkgo – an ancient herb that might help improve their brain function. That knowledge alone might be enough to move the needle a bit on cognitive tests.
How much was the needle moved?
The primary outcome here was based on the Montreal Cognitive Assessment, or "MOCA."
This is a validated test for cognitive impairment, with higher scores being better – much like the Mini-Mental Status Exam we know and love.
Take a look at the MOCA scores over time in the two arms of the trial.
At the beginning of the trial, the MOCA scores in both arms of the study were about the same – 19 or so. By 180 days later, scores in both groups had gone up a bit – a bit more in the ginkgo group, but not enough to achieve statistical significance.
But I said earlier that ginkgo "improved outcomes." What gives?
Well, the primary outcome was not just MOCA score, it was change in MOCA score from baseline to 180 days.
What you see here is that change, and indeed, the ginkgo group improves more over time than the control group.
But how is that possible? How can the average MOCA score be the same in both groups, but the change in MOCA score be significantly different? Well this is a statistical phenomenon known as Lord's paradox and it can happen when one simply examines change scores without taking into account baseline scores.
This picture illustrates Lord's paradox.
You can see how, overall, the scores end up in pretty much the same place, but on the individual level, every person in the treatment group does better than every person in the control group.
So what we're seeing is a rather weak effect. A strong ginkgo effect would be visible regardless of how you analyze the data. And the lack of an adequate placebo control is probably enough to account for the small observed benefit. So for now the data on this particular herb is not fully-flowered.
F. Perry Wilson, MD, MSCE, is an assistant professor of medicine at the Yale School of Medicine. He is a MedPage Today reviewer, and in addition to his video analyses, he authors a blog, The Methods Man. You can follow @methodsmanmd on Twitter.
No comments:
Post a Comment