Monday, February 06, 2012

Why that Diet Soda/Stroke Paper is Worthless and a Failure of Peer Review


Ugh.

So without spending too much time on this, here's the thing, that paper that purported daily diet soft drink consumption was associated with several vascular risk factors including strokes? It's worthless, and it's a glaring failure of peer review.

Why?

Because the authors didn't even attempt to control for dietary quality, and moreover, the dietary recall data itself was obviously inherently flawed.

First the control issue. As I'm sure you're aware, what we eat has a tremendous impact upon our risk of developing various chronic diseases. Consequently not accounting for the folks who for instance ate 10 meals a week from fast food restaurants, take aways or diners, versus those who actually ate at home and transformed raw ingredients would likely skew the data. But even if you want to try to suggest that such differences would be accounted for by the dietitians conducting the dietary recall effort, if all you actually analyzed at the end of the day was amounts of consumed protein, carbohydrates and fats, what you'd fail miserably in doing would be to actually usefully compare the quality and caliber of the diets you were studying. By not looking at the quality of the macronutrients you'd be comparing quinoa to white rice, salmon to bacon, and olive oil to Crisco. Yet that's exactly what the authors did. And it's certainly not at all implausible that folks who regularly indulge in lower caliber dietary choices assuage some of their dietary guilt and build their own health halos by choosing a diet beverage over a fully loaded one.

But even if the authors accounted for (as they should have at the very least tried) meals out, vs transformed raw ingredients, as well as the actual quality of dietary macronutrients this study would still be useless.

Why?

Because according to the dietary recall data presented in their study, and despite an average body mass index of 28 (overweight), the self-reported overall caloric intake of the study population was a measly 1,575 calories, whereas 2000 NHANES data pegs the average for men between the ages of 59-79 to be between 2,123-2,590 and women between 1,596 and 1,828. And that's 12 year old NHANES data.  Given what we've seen with obesity rates, it's certainly quite plausible if not exceedingly likely, that average caloric consumption has risen over the past decade and change.

So basically here we have a study where we know the self-reported dietary recall is inaccurate and where the authors didn't even attempt to account for the actual quality of the participants' diets, that's making conclusions about the impact of diet soft drink consumption on strokes and suggesting diet was a controlled for variable? 

How this got through peer review is completely beyond me, but the worst part is the coverage. This paper got plenty of press making its publication not only an embarrassment to the Journal of General Internal Medicine and its peer review process, but also a powerful source of misinformation in a world that certainly doesn't lack for nutritional confusion.

Gardener, H., Rundek, T., Markert, M., Wright, C., Elkind, M., & Sacco, R. (2012). Diet Soft Drink Consumption is Associated with an Increased Risk of Vascular Events in the Northern Manhattan Study Journal of General Internal Medicine DOI: 10.1007/s11606-011-1968-2