Wednesday, October 17, 2018

Wow! 29 Teams of Analysts, One Identical Data Set, One Identical Research Question, 29 Different Outcomes.

This is an incredible paper, Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results, saw 61 analysts (in 29 teams), be given the same data set meant to address the same research question (are soccer referees more likely to give red cards to dark skinned players than light skinned players).

The outcomes?

20 teams found a statistically significant positive effect, while 9 teams did not, and where effect sizes ranged (in odds-ratio units), despite all teams working from the same data set, from 0.89 to 2.93 (where 1.0 would be no effect).

Why so many differences?

Because results depend a great deal on any team's chosen analytic strategy which in turn is influenced by their statistical comfort and choices and their interplay with their pre-existing working theories.

Now these results weren't incentivized examples of p-hacking. The authors of this study point out that the variability seen was based on "justifiable, but subjective, analytic decisions", and while there's no obvious means with which to ensure a researcher has chosen the right methodology for their study, the authors suggest that,
"transparency in data, methods, and process gives the rest of the community opportunity to see the decisions, question them, offer alternatives, and test these alternatives in further research".
Something all the more important in cases where authors might in fact have biases the would incentivize them to favour a particular outcome, and why I wish I was offered more in the way of stats and critical appraisal in medical school (and maybe less in the way of embryology for instance).

[Photo by Timur Saglambilek from Pexels]

Bookmark and Share

Monday, October 15, 2018

From The Education Alone Isn't Enough To Change Behaviour File: Fast Food Edition

Two weeks ago I gave a talk at Ottawa's 6th Biennial Championing Public Health Nutrition Conference. I was part of a group of speakers talking about the how can it possibly not be published yet new Canada Food guide.

I was struck, both during the other presenters talks, and during the question and answer period, how focused people were on how the Food Guide will be utilized by individuals.

In my opinion, as a direct tool, it pretty much won't be. That's not to say it can't or won't have an impact on Canadian dietary patterns (it will by way of its impact on policy), nor that a person who picked it up couldn't choose to follow it, but rather speaks to the simple fact that education alone doesn't seem to be enough to change behaviour. Because time and again we learn that education, even when tied to terrifying events like heart attacks, doesn't seem to be able to consistently lead people to sustain consequent lifestyle changes, nor does genetic knowledge of specific disease risks.

The reasons why are likely myriad, but probably boil down to a combination of normal human nature and change being difficult, along with the impact of a person's food environment and social determinants of health.

For a food related example of this, take this recent paper regarding perceptions about the consumption of fast food. In it, among many other statistics, the authors note that 73% of weekly fast food consumers reported that they believed fast food wasn't good for them.

When it comes to behaviour change, knowledge alone does not seem to correlate particularly strongly with power.

Bookmark and Share

Saturday, October 13, 2018

Saturday Stories: Labour Antisemitism, The Name Of The Dog, And a Risk Conundrum

Man Booker prize winner Howard Jacobson, in The Atlantic, with perhaps the definitive piece on Jeremy Corbyn and Labour's antisemitism.

Taimer Safder, in The New England Journal of Medicine, with a lovely read about the name of the dog (do read this one before it disappears behind a paywall).

Lisa Suennen, in Venture Valkyrie, on the conundrum of divergent ways to evaluate cardiac risk that span from biology to social determinants of health.

[photo by Alexandru Rotariu via Pexels]

Bookmark and Share

Thursday, October 11, 2018

On Physicians Who Support, Promote, And Recommend, Only One Type Of Diet

Oh they're out there.

Tunnel vision physicians who believe that everyone should be vegan, or be intermittently fasting, or in ketosis, or on an incredibly low-fat diet, or vegetarian, or low carb high fat, - and I'm sure the list goes on.

It's a head scratcher for me because a physician's training ought to have them know better.


Because for virtually every medical problem, multiple therapies and therapeutic modalities exist. And because physicians know that some drugs work better than others with different patients - sometimes predictably, and sometimes unpredictably, and that sometimes people have adverse reactions to certain drugs that require them to try alternatives.

Diets are the same.

Whether for weight management, general health, or the treatment of particular medical conditions, certain patients, sometimes explicably and sometimes not, will do better with different diets, both in terms of the impact that diet has on whatever they're trying to treat, but also on their ability to enjoy that diet enough to sustain it long term.

And so even if there were a scientifically proven best diet for a particular issue (and for weight, plainly at this point, there isn't), there'll still be some people for whom it fails, and some people for whom its adverse effects on their lives leads to its discontinuation, and if they happen to be on that diet because they're following or seeing one of those MDs who is so stuck on there being only one diet to rule them all, I guess they're just out of luck.

So what drives those MDs? I think the answer varies. For some it's likely the extension of their own personal experience and success with a particular dietary approach. For others, it may be the consequence of literal or intellectually sunk costs. And finally some may not have sufficient background to evaluate much on their own and instead simply parrot an eloquently delivered diet zealot's talking points (perhaps especially in the cases of MDs converted by other MDs). But regardless of why one thing's for sure, the promotion of one right or best diet isn't good medicine, it compromises patient care, provides oxygen to the fire of fads, serves as catnip for publishers, the media, and the public, and solidifies the notion that there are dietary demons and deities, all of which in turn torches the hope of improved nutrition related scientific literacy in society.

Nutritional populism is a bad look irrespective of which diet it happens to be promoting.

[Photo by Anthony DeRosa from Pexels]

Bookmark and Share

Tuesday, October 09, 2018

About That New Lancet Study: Maybe Don't Expect 9 Year Olds To Change Their Own Food Environments

It's difficult for me to imagine what exactly researchers expected would happen as a consequent of this study's intervention which hoped to help children with obesity.

The Effectiveness of the Healthy Lifestyles Programme (HeLP) to prevent obesity in UK primary-school children: a cluster randomised controlled trial, enrolled 9 and 10 year olds from 32 different UK schools and randomly assigned some schools to deliver a year long curriculum to the children which included,
"dynamic and interactive activities such as physical activity workshops, education sessions delivered by teachers with short homework tasks, drama sessions, and setting goals to modify behaviour"
And while parents were involved, their involvement was dictated by their children primarily who in turn were instructed to "reflect on their own behaviours and goals" with their parents.

Various weight related outcomes, activity related outcomes, and dietary choices outcomes over 2 years were collected, and the results weren't in any way exciting, with pretty much no differences found between study and control groups on any weight related or physical outcome.

But should anyone have expected anything different?

Are there really those out there who believe that if you teach 9 and 10 year olds in school that they should eat less and better and exercise more, that they'll do so? Fully grown adults with newly diagnosed weight or diet related diseases rarely do, so why should children? Or that 9 and 10 year olds who themselves have zero responsibility for their food environments, even if they actually "reflected" on their behaviours and goals with their parents, could see their food environments appreciably and sustainably change?

And what of these kids, and especially of the kids who already have obesity? It does not appear as if this study even attempted to explore whether or not the 2 year long intervention had any negative psychological impact. But certainly, if the crux of the program is to teach 9 year olds that they are personally responsible for their lifestyle choices, I think it would be fair to consider the possibility that the program will lead some to question their self-worth, self-efficacy, body-image, and potentially affect their relationships with food and even risk disordered eating. It may have also been important to study whether or not there was any increase in weight related bullying in the intervention schools.

All this to say, relying on 9 and 10 year olds to modify what for them, given they're in charge of next to nothing related to when, where, and what they eat, are almost certainly unmodifiable food environments, wholly unsurprisingly, isn't an effective plan. While I am supportive of robust programs that work with parents to change their families' lifestyles (disclosure, I'm the medical director of just such a program), focusing just on the kids is akin to focusing all of your efforts on lecturing life's passengers and ignoring the drivers, and where the drivers aren't just the kids' parents, but their food environments as a whole.

Bookmark and Share

Saturday, October 06, 2018

Wednesday, October 03, 2018

Guest Post: Does the BMJ publishing group turn a blind eye to anti-statin, anti-dietary guideline & low-carb promoting editorial bias?

Truth be told, I'm fond of the British Journal of Sports Medicine, but as anyone who reads it knows, it has this strange habit of publishing articles about non-sports related dietary guidelines, the dangers of cholesterol lowering medications, and on the alleged superiority of low-carb diets. That's an odd thing, or maybe it's not, for as today's 4 guest posters (Drs Nicola Guess, Ian Lahart, Duane Mellor, and David Nunan) lay out, it may simply reflect the editor in chief's personal bias. So have a peek at their story, and if while you're reading you're on the fence, ask yourself if it would be odd for the American Journal of Clinical Nutrition to publish ankle sprain treatment guidelines, or if the Journal of the American College of Cardiology published a review of the efficacy of orthotics for plantar fasciitis? And note, at the end of their guest post is a link to an open letter to the BMJ that they've penned, which by following the link, you can also sign.

Scientific journals have the potential to allow researchers to keep up to date with developments in their field, to publish their own research, and to comment on the research and ideas of their peers. Journal editors play a vital role as impartial gatekeepers of this process, and importantly they should ensure their own personal beliefs and prejudices do not impact decisions related to content that is published in their journal.

Here we provide a synopsis of an example of poor gatekeeping we are currently experiencing and how this can skew the scientific discourse in favour of a personal agenda. A full account of this story is available here.

In April 2017, an opinion piece editorial entitled ‘Saturated fat does not clog the arteries: coronary heart disease is a chronic inflammatory condition, the risk of which can be effectively reduced from healthy lifestyle interventions’ was published in the British Journal of Sports Medicine (BJSM), and then repeatedly promoted by the journals Twitter account, which is jointly run by the journal’s Editor in Chief (EIC).* Promotion of an editorial is completely normal—often an EIC will highlight an article or opinion piece that they believe to be of interest. However, it is also normal and expected that the editor acknowledges and welcomes debate and rebuttals from others who disagree with points made in the published article. Two of us (David Nunan and Ian Lahart) emailed the EIC immediately following publication of the original article offering a rebuttal, but received no response. After three months without reply, we published an open rebuttal in PubCommons (latterly Pubpeer) highlighting what we thought were important deficiencies in the original article.

We were surprised to see the EIC tweeting the saturated fat article a year later and stating that ‘importantly’ the editorial had not had any rebuttals. We immediately contacted the EIC again, and after a series of emails received acceptance that our rebuttal would be published in the BJSM.

An important point--one that we believe highlights the bias in gatekeeping at the BJSM—is that the original article was published “open access”, meaning it is was made freely available to academics, public and the media. This is important for openness and access to science, and also allows interested members of public to read articles which frequently are hidden behind a paywall. This of course is good practice. Key here though is that decisions to make articles such as these open access is made entirely by the EIC of the BJSM.

Naturally in the spirit of open debate, we would only consider it reasonable that our rebuttal would be published open access, along with some social media promotion. This way, readers could read both the original article and the rebuttal and consider all the arguments presented. However, we were told by the EIC that our article would not be made available as open access, but that we could instead pay for it (£1,950) to be made free of charge to read.

We were further concerned and surprised when we examined other articles on similar subjects also unrelated to the remit/readership (e.g., dietary guidelines, statins) published in the BJSM. Of 10 such articles, all were open access, they all had narratives that denigrated current dietary guidelines and/or statins and promoted an exclusively low-carb dietary approach. All of the articles were authored by supporters of these narratives, with some writing two or more articles. The EIC, via the BJSM twitter account, has regularly promoted these narratives on social media. Four rebuttals/counter arguments to these articles have been published (including our most recent) – none of these were made available as open access by the EIC. There has also been next to no promotion of these rebuttals via social media from the BJSM twitter account.

Furthermore, during the two-month period we were communicating with the editorial team to have our article published open access, or at the very least a footnote added to state we had been denied free open access (both requests were turned down), the BJSM produced two podcasts from authors of 2 of the 10 free articles, including the one in question here.

To be clear, our rebuttal was not in complete disagreement with all the points made in the original editorial. Our rebuttal was more about using robust methods to emphasize the strength of the evidence and highlighting knowns and unknowns that were overlooked in original editorial. Furthermore, we have healthy disagreements amongst ourselves about the evidence in this field (e.g., dietary guidelines). These disagreements, however, should be debated openly in the scientific literature. The EIC’s role is to facilitate this in an unbiased manner and ensure systems are in place to prevent biases skewing the scientific discourse to the journal’s audience. Imagine if a journal only published and promoted open access articles on the effectiveness of aspirin to prevent heart attacks, yet hid every rebuttal (highlighting potential harms) quietly behind a paywall?

We are concerned about the editorial conduct and procedures of the BJSM. Given the journal is part of the BMJ publishing group (governed by the British Medical Association (BMA)), this also raises questions over governance across over 50 of its journals The BJSM is also co-owned by the British Association of Sports and Exercise Medicine (BASEM). We think this is worth pursuing further and have written an open letter to each of these organizations requesting the issues raised here are looked into.

Our open letter is available for signing (and reading) by clicking here.

[*it would be reasonable to question the fit of such an editorial to the journal’s scope and readership: “…provides original research, reviews and debate relating to clinically-relevant aspects of sport and exercise medicine, including physiotherapy, physical therapy and rehabilitation.”]

Dr Nicola Guess is a lecturer in the Department of Nutritional Sciences at King’s College London in the UK and a Registered Dietitian. Her research interests are on the effect of diet on the pathophysiology of type 2 diabetes. 

Dr. Ian Lahart is a senior lecturer in exercise physiology and researcher at Institute of Human Sciences, University of Wolverhampton. He completed his PhD in the role of exercise in breast cancer. Through his PhD work, he conducted an exercise randomised controlled trial in women with breast cancer. Ian is also the lead author of a recent Cochrane collaboration review on the effects of exercise in women with breast cancer post-adjuvant therapy. Through his role as a research fellow at Russells Hall hospital, Dudley, UK, he helped set up and manage a MacMillan funded exercise-based cancer rehabilitation service. Although his research focus is on the role of exercise in breast cancer rehabilitation and survivorship, he has additionally worked with patients with other cancers, arthritis, cardiovascular disease, and diabetes and related metabolic conditions. He is also interested in the communication of science and meta-research—a field of research that investigates research practices and quality.

Dr. Duane Mellor has worked clinically as a dietitian, mainly in diabetes management and education and then as a researcher in clinical trials. However, reflecting back on the first 2 decades of his career he has begun to question a number of aspects of nutrition and dietetic practice. He is now interested in looking at evidence in nutrition, both in terms of causality and quality along with how this is communicated to the public by the media. Looking to challenge thinking in this area, to consider aspects of benefit and the risks of harm, ultimately looking at how the public can be best supported to eat food they enjoy that also supports good health.

Dr. David Nunan's career in academic research started over 15 years, with a focus on clinical care and evidence-based medicine over the past 8 years. Upon completion of his PhD, he joined the Centre for Evidence-Based Medicine and his role is now divided between research, teaching and outreach activities.

Bookmark and Share

Monday, October 01, 2018

Meta-Analysis Of Low-Carb Meta-Analyses Finds The Ones Most Excited About Low-Carb Diets Are Of "Critically Low Quality"

Meta-analyses, studies that combine a slew of relevant studies to come to one larger conclusion, are undoubtedly valuable, but that doesn't mean there isn't plenty of room for debate about their findings.


Because their findings depend on the criteria they used in order to determine which studies should be included. So when considering a meta-analysis on the impact of low-carb diets (LCD), variables that would affect outcome might include the definition of LCD (ie how many grams per day of carbohydrates constitutes a low carb diet), the duration of the diet, the number of databases searched, how risk of bias was assessed and applied, and investigation of the causes of heterogeneity to name just a few of those found in the more complete (AMSTAR) list seen here:

And in fact, a study analyzing the quality of meta-analyses of low-carb diets was recently published in Obesity Reviews, and its findings fall in line with my very admitted confirmation bias which sees low-carb diets being as good or as bad as any other diet, and that at the end of the day, what matters more than the diet prescribed is diet adherence.

The authors found that,
"critically low quality (low-carb diet/LCD) meta-analyses showed superiority of LCD for weight loss while moderate quality showed inconsistent results, and high quality showed little or no difference"
Of course all of the studies included looked at overall losses between different prescribed diets, but in my opinion, that may not be the best way to evaluate them.

Because as the DIETFITs study so elegantly illustrated, there are people who do incredibly well with low-carb or low-fat diets, while other people do incredibly poorly, and all within the same study population.

I would argue further that this is true for any diet.

All this to say, be wary both of any study or meta-analysis that crowns one diet better than another, and of anyone suggesting that a particular diet isn't worth trying. One person's best diet is another person's worst.

(Photo by Jenna Hamra from Pexels)

Bookmark and Share

Saturday, September 29, 2018

Saturday Stories: Remembering, Revenge, And Not Debating

Anita Hill, February 2018
Jessica Shortall, in Medium, with a rage and sorrow inducing piece about everything she can remember.

Jennifer Weiner, in the New York Times, on wanting to burn the frat house of America to the ground.

Laurie Penny, in Longreads, on not debating.

And here's a recent live podcast I did with Darya Rose at this year's Fireside conference where we cover the basics of successful weight management and why I think chocolate's more important to success than hunger.

[Photo By Gage Skidmore, CC BY-SA 3.0, Link]

Bookmark and Share