Monday, March 19, 2012

What Reading that White Rice and Diabetes Study Actually Told Me


The short version of what it told me was that the BMJ is actively and willfully abusing the public's trust, and it also told me that many low-carb and paleo diet guru bloggers and tweeps don't care about bad nutritional epidemiology so long as it vilifies carbs.

Here's the longer version starting with the BMJ.

Last week the BMJ published another observational dietary study out of Harvard. This time the study's focus was on white rice and it involved a meta-analysis of the medical literature on white rice consumption and its impact upon the risk of developing type 2 diabetes.

According to the study's authors, prior studies on white rice and type 2 diabetes had mixed results, and so in order to try to illuminate them, they undertook a literature review. In the end the authors' exclusion criteria left them with only 4 studies detailing 7 separate cohort analyses. Follow ups in these cohorts ranged from 4 to 22 years, and of them, 2 had exceedingly low incidences of type 2 diabetes development and so the authors had to perform some statistical sleight of hand to include them.

Of the 7 cohort analyses, 2 did not control for differences in dietary consumption patterns (probably important if you're looking at the impact of dietary factors on diabetes development), 1 did not control for family history of diabetes, 6 did not control for income/socioeconomic status, and none controlled for consumption of other refined grains or sugars. Let me go over that last one again not one of their 7 cohort analyses controlled for consumption patterns of refined grains or added sugars (though 2 controlled for consumption of "bread" and "noodles" but these weren't broken down any further).

Pooling the data together led researchers to a relative risk of type 2 diabetes development of 1.27 for the highest white rice consumption with a 95% confidence interval possessing significant heterogeneity and starting at 1.04, which for the non-statisticians reading this means the finding is just barely interesting.

When the researchers then stratified the data by ethnicity they found that for Asian populations the pooled relative risk was 1.55 with a confidence interval starting at 1.2, and for Western populations a pooled relative risk of 1.12 with confidence intervals ranging from 0.94-1.33, meaning that according to their findings, the impact of white rice on diabetes risk only seems to exist in Asians.

Translation?

The most generous way I can spin it would be that this study, using pooled cohorts that left out tremendously important controls and considerations, when analyzed, suggested that white rice consumption increases the risk of diabetes development in Asian, but not Western, populations.

The worst way to spin it? The fact the cohorts used to determine this study's conclusions failed to consider incredibly relevant diabetes confounders like family history of diabetes, socioeconomic status, and dietary consumption patterns, including the dietary consumption of other categories of refined grains, makes quantifying the effect on diabetes development due to white rice consumption from this data set impossible.

And yet it was published in the BMJ?

But that's not the truly shocking part. This is. The BMJ published an accompanying editorial that rightly called the paper out on its methodological and statistical inadequacies and in conclusion stated,
"Although the findings of the current study are interesting they have few immediate implications for doctors, patients, or public health services and cannot support large scale action. Further research is needed to develop and substantiate the research hypothesis"
Yet what's the title and first line of the very same BMJ's press release regarding this statistically and methodologically weak paper,
"White Rice Increases Risk of Type 2 Diabetes

The risk of type 2 diabetes is significantly increased if white rice is eaten regularly, claims a study published today on bmj.com.
"
Where the next 6 paragraphs of the press release continue in that same conclusive and important sounding vein only to end with this last line,
"In an accompanying editorial, Dr Bruce Neal from the University of Sydney suggests that more, bigger studies are needed to substantiate the research hypothesis that white rice increases the chances of getting type 2 diabetes."
Guess which 7 of 8 press release paragraphs the media paid attention to?

Here are a sampling of actual media headlines as a consequence of the BMJ's misleading press release,
"Eating White Rice Daily Up Diabetes Risk" - CBS News

"White Rice 'Could Cause Diabetes'" - The Independent

"Think Twice About Rice? New Study's Advice" - ABC News

"White Rice Raises T2 Diabetes Risk, Claim Academics" - The Daily Telegraph

"White Rice Linked to Diabetes Risk" - WebMD

"White Rice May Increase Your Risk of Diabetes" - MSNBC
And of course the Twitterverse went crazy too and rice bashing tweets abounded, and from some rather influential tweeps including Dr. Sanjiv Chopra, Dean of Continuing Education at Harvard Medical School, the Harvard School of Public Health itself, journalist Greta Van Susteren, the American Society of Nephrology, the Drudge Report, and many, many, many, many more.

Now I'm not suggesting white rice is a wise food to consume. On the contrary, I generally recommend people try to minimize its consumption, but to be very clear, this study does not in any formative way, shape or form have the strength to draw any conclusions whatsoever on the specific impact white rice has on the risk of developing type 2 diabetes.

And what the hell is up with the BMJ? Publish a paper so weak that you feel the need to co-publish an editorial questioning the paper's design and conclusions and then simultaneously put out a press release that in turn purposely glosses over and misinforms the media about the paper's weak conclusions?

And lastly, where is the low-carb and paleo bloggers' and tweeps' outrage? We saw it surge like never before when a far more rigorously controlled and designed observational study on red meat came out just a week before, and yet here there's pretty much critical silence (Zoe Harcombe being a notable exception).  There were even examples where prominent low-carb and paleo folks retweeted the press releases to their followers as if they were facts, when just days before those very same tweeps were tweeting their outrage at what they saw as the meat study's undeserved promotion.

The real lessons here?  Confirmation biases and impact factors sure do cloud our judgements and analyses.