Tuesday, March 27, 2012

What Reading That Eat Chocolate be Thinner Article Actually Told Me


It told me that the University of California in San Diego's PR department is beyond shameless, and that the Archives of Internal Medicine will publish pretty much anything.

Here's the press release in its entirety (highlights are mine),
"Regular chocolate eaters are thinner

Katherine Hepburn famously said of her slim physique: "What you see before you is the result of a lifetime of chocolate." New evidence suggests she may have been right.

Beatrice Golomb, MD, PhD, associate professor in the Department of Medicine at the University of California, San Diego, and colleagues present new findings that may overturn the major objection to regular chocolate consumption: that it makes people fat. The study, showing that adults who eat chocolate on a regular basis are actually thinner that those who don't, will be published online in the Archives of Internal Medicine on March 26.

The authors dared to hypothesize that modest, regular chocolate consumption might be calorie-neutral –in other words, that the metabolic benefits of eating modest amounts of chocolate might lead to reduced fat deposition per calorie and approximately offset the added calories (thus rendering frequent, though modest, chocolate consumption neutral with regard to weight). To assess this hypothesis, the researchers examined dietary and other information provided by approximately 1000 adult men and women from San Diego, for whom weight and height had been measured.

The UC San Diego findings were even more favorable than the researchers conjectured. They found that adults who ate chocolate on more days a week were actually thinner – i.e. had a lower body mass index – than those who ate chocolate less often. The size of the effect was modest but the effect was "significant" –larger than could be explained by chance. This was despite the fact that those who ate chocolate more often did not eat fewer calories (they ate more), nor did they exercise more. Indeed, no differences in behaviors were identified that might explain the finding as a difference in calories taken in versus calories expended.

"Our findings appear to add to a body of information suggesting that the composition of calories, not just the number of them, matters for determining their ultimate impact on weight," said Golomb. "In the case of chocolate, this is good news –both for those who have a regular chocolate habit, and those who may wish to start one.
"
Holy awesomesauce batman! Eat chocolate, get thinner! If you're not eating it already, maybe you should! Chocolate is a magic fairy food that has unknown substances that not only cause chocolate's calories not to count, but actually makes chocolate a net negative calorically!

Only they're not, and they don't, and there is no growing "body of evidence" suggesting there are magically calorie neutral or negative foods (though there are certainly differences indirectly consequent to different foods' impacts upon satiety).

So what did the study actually show?

The study looked at 975 men and women aged 20-85 who filled out a single food frequency questionnaire as part of their enrollment in a study that was meant to look at the non-cardiac impact of statin drugs. Included in the questionnaire was the question, "How many times a week do you consume chocolate?". The authors then looked at the relationship between chocolate frequency and BMI controlling for:

  1. Fruit and vegetable intake
  2. Saturated fat intake
  3. Mood
  4. Number of days of week active for at least 20 minutes.

That's it, that's all? Nothing else?

Ummmmm, last time I checked people ate more than just chocolate, fruits, vegetables and butter.

Honestly I seem to recall reading or learning somewhere, maybe it was in med school, that people sometimes eat things like beef, fish, chicken, nuts, lentils, pulses, dairy products, and candy. Oh, and don't they also sometimes have breads, pastas, and cereals? And does anyone other than me drink alcohol?

Would how much they ate or drank of those things matter in a study looking at their weights?

And do you think a question like, "How much chocolate do you consume?", would have been helpful in assessing its effect? And are the only non-food related variables that affect weight mood and exercise? How about maybe controlling for medications and medical conditions that affect weight, socio-economic status, education, sleep duration, marital status, smoking, etc., those other pesky things that have been shown to actually impact upon BMI?

Here's part of the authors' conclusions from the paper,
"The connection of higher chocolate consumption frequency to lower BMI is opposite to associations presumed based on calories alone, but concordant with a growing body of literature suggesting that the character - as well as the quantity - of calories has an impact on metabolic syndrome factors"
Really? A growing body of literature suggests that there are foods whose calories don't count? I notice that there are no actual references provided for that particular statement.

So to recount - basically here we have a study with no controls whatsoever rendering conclusions impossible, authors who rather than mention their study's pretty much insurmountable methodological limitations instead made up a "growing body of literature" on magic calorie neutral or negative foods, a press release that spins it all as fact and as a result, as of early this morning, less than 24 hours after publication, there were already 443 chocolate makes you thin stories on the newswire to further misinform an already nutritionally confused world.

Once again I'm left scratching my head trying to understand how this could possibly have made it to - let alone passed - peer review, and why it is that ethics and accuracy don't seem to matter to the folks who write press releases, or to the respected researchers who are drawing these unbelievably irresponsible and over-reaching conclusions despite undoubtedly knowing better. It also makes me wonder just how exactly they all manage to sleep at night.

[Even more amazing? This study was NOT funded by the chocolate industry]

28 comments:

  1. Anonymous6:02 am

    What was the funding source for this study. Hopefully tax dollars weren't wasted on it. I'd feel better if some cocoa or chocolate trade group was behind it.

    ReplyDelete
    Replies
    1. Thanks - forgot to add that the study was not in fact funded by the chocolate industry (just did so).

      Delete
    2. Anonymous7:21 am

      In that case, I really question the oversight over the funding mechanisms. The more I read studies, the more I realize how utterly craptastic most of them are from a design and analysis standpoint.

      Delete
  2. The study was so bad that even the chocolate people didn't want to be associated with it!

    ReplyDelete
  3. What an epic fail.
    All I seem to be reading these days, both on the blogs and for my own work, are poorly-designed, poorly executed, poorly analyzed studies that somehow are published in peer-reviewed journals and are thus labelled as "credible" science. What a joke.

    ReplyDelete
  4. Reminds me of the funny chocolate quote "Diet tip: Eat a chocolate bar before each meal. It’ll take the edge off your appetite, and you’ll eat less."

    ReplyDelete
  5. I agree, that study is worthless. I am also not a fan of Beatrice Golomb in general. She exaggerates the risks of statins and minimizes the benefits.

    ReplyDelete
  6. A quote comes to mind about bogus research "garbage in...garbage out". Thank you Yoni for calling attention to the quality of how the "research" was conducted in this case. It sounds more like a poll with no scientific basis.
    The media is hard up for news to promote this information as fact.

    ReplyDelete
  7. The kind of attention these poorly done studies are getting in the news makes me furious. Thanks for getting the other side out there!

    ReplyDelete
  8. Anonymous9:53 am

    They attempt to correct for total calorie intake using a Food Frequency Questionnaire. Is this an accepted/commonly used method in the field?

    ReplyDelete
    Replies
    1. Came back to this post to ask this exact question =)

      Delete
    2. Sometimes it's the best they can do and there are literally dozens of studies out there discussing different coefficients to apply to them to try to make up for known inaccuracies.

      Delete
    3. So they did account for over-all calories? And the fruit and vegetables is just an add on question?

      Should that not mean calorie intake was controlled for?

      "However, calories were otherwise not in adjustment models because chocolate inherently contains calories and adjustment could justly be deemed inappropriate—overstating the benefits of chocolate to BMI"

      ^ Under the table.

      Delete
  9. "And does anyone other than me drink alcohol?"

    Crap studies like this certainly may drive me to drink...

    ReplyDelete
  10. Anonymous10:51 am

    to tell you the truth i think that this should mean that there should be a greater look into chocolate and the diet. i know that they didn't look into EVERYthing that the people ate, but if they did? and monitored what they ate and how much they exercise?
    is it wrong that people who love chocolate want to see something like this looked into at a deeper? no. the study could have been more inclusive and looked deeper, but it could spawn a study that would do so.

    ReplyDelete
  11. Anonymous11:21 am

    If they're going to take crappy data and try to form some sort of logical relationship between two vaguely statistically linked variables, I still don't understand why it wouldn't occur to them to consider the opposite causal relationship: thin people eat more chocolate because it tastes good and they're not worried about their weight; fat people eat less chocolate (or underestimate their intake on surveys) because they feel like though should be trying to lose weight by eating less junk food.

    ReplyDelete
  12. Anonymous11:25 am

    Also, if they're going to make the argument that chocolate is somehow a calorie-neutral food due to some ill-defined "metabolic effects", then shouldn't they at least have controlled for caffeine intake as well?

    ReplyDelete
  13. Anonymous11:35 am

    Peer reviewed? Seriously?
    Disgraceful.

    ReplyDelete
  14. Anonymous4:40 pm

    "The connection of higher chocolate consumption frequency to lower BMI is opposite to associations presumed based on calories alone, but concordant with a growing body of literature suggesting that the character - as well as the quantity - of calories has an impact on metabolic syndrome factors"

    Wait. Where in that statement does it say chocolate calories don't count? I don't see that. I see a statement that possibly what we eat matters, just like how much we eat.

    Given that the findings are counter-intuitive, that seems like a reasonable speculation. We can't blame the study authors because the media blew it all out of proportion. Looks to me like they've raised a question, not found an answer.

    ReplyDelete
  15. Sorry, but it appears you are propagating bad science by posting this (with links). What I would have asked is whether the chocolate was in addition to their daily calories or were they part of their daily intake. No food is bad. We all choose our poison, be it chocolate, alcohol, chips or fruit. Because eating too much of even "healthy" food can make you fat just as fat as chocolate calories can.

    ReplyDelete
  16. Anonymous7:16 pm

    Umm... anyone who says that no food is bad has never eaten in a public school cafeteria. Ack. He's drawing attention to bad science to try to stem its propagation in the media. He's providing a voice of sanity in a world where too many people fall for studies like this and gleefully say science supports their poison of choice as a "healthy" food. It's an opportunity to examine why the study is bad, from an expert, and thus learn how to discern good science from bad. These bad studies help keep people nutritionally confused and struggling. Thanks for the commentary, Dr. Freedhoff.

    ReplyDelete
    Replies
    1. Sorry, last link didn't work. Link was to my post where I wholly agree that of course there's such a thing as a bad food.

      Delete
  17. Yoni,
    I was really hoping there was a possibility that chocolat was a "free" food. Ridiculous.

    ReplyDelete
  18. Interesting article, everyone talk about this topic at the moment in Australia. The obesity rate is already very high here and message such "eat chocolate, it will make you fit" are a complete non sense to me. I have shared your article on Twitter and FB.

    ReplyDelete
  19. You might enjoy this cartoon. Chocolate cartoon

    ReplyDelete
  20. That's a poor critique. The study shows a negative and significant correlation between frequency of chocolate eating and BMI; adding controls increases the strength of that correlation. You criticize the study for not having enough controls, but doesn't suggest what they should be; controlling for fruit and vegetable intake (known to be associated with lower BMI) and saturated fats intake (not just butter, as you falsely accuse, and known to be associated with higher BMI), effectively controls for caloric sources other than chocolate. Adding a control for grains and nuts or similar would likely overspecify the model.

    The implication of the study is NOT "Eat chocolate, get thinner!" which is a classical misinterpretation of correlational results. The correct interpretation is that eating chocolate is something that thinner people do more frequently than more overweight people, contrary to common expectations. The discussion highlights the possibility that given this evidence chocolate may have dietary effects that should be further tested, as clearly explicated in the final section of the article.

    If you were a critically thinking scientist instead of an inflamed reactionist, you would proffer an alternate hypothesis that would effectively explain the observed correlation between higher frequency of eating chocolate and lower BMI, with and without controls. Yet you only bring ad hominem attacks against the authors and the journal.

    What's your explanation? You sit on your high horse about the quality of the study when it's your clarity of thought (including statistical reasoning) that's clearly left wanting.

    ReplyDelete
    Replies
    1. Anonymous7:15 pm

      It makes me sad that you're probably a voter.

      Delete
  21. Anonymous10:00 am

    A doctor who drinks alcohol. I am amazed.

    And yes, I am indeed serious. Enjoy ruining your health.

    ReplyDelete