First, some background.
In early July, the well respected Cochrane Review folks put out a piece on dietary salt reduction and cardiovascular disease. Their review basically had three non-overlapping and somewhat contradictory conclusions. The first (gleaned from their plain language summary) was,
"Cutting down on the amount of salt has no clear benefits in terms of likelihood of dying or experiencing cardiovascular disease"The second was,
"Our findings are consistent with the belief that salt reduction is beneficial in normotensive and hypertensive people"While the third was,
"The challenge for clinical and public health practice is to find more effective interventions for reducing salt intake that are both practicable and inexpensive"The media pretty much only reported on that first bit, with headlines screaming, "It's Time to End the War on Salt", "Review says salt not responsible for heart attacks", "Study Denies Any Link between Sodium Intake and Heart Risk", and "Now salt is safe to eat — Health fascists proved wrong after lecturing us all for years".
I blogged about how it seemed to me, many of the journalists who wrote about the Cochrane Review, must not have bothered reading it, given the message it contained, truly wasn't the message they conveyed. Of course I'm just some doc with a blog, and admittedly, I'm not a statistician, nor am I a hypertension researcher, maybe I misread something?
Well, Drs. Feng J. He and Graham A MacGregor are hypertension researchers (and certainly their likely confirmation bias' are that salt's bad), and they decided to pen a commentary on the Cochrane Review that was published this past week in the Lancet.
Summarizing their commentary - if you exclude the one paper in the Cochrane Review's analysis that was poorly designed (the one on patients with heart failure whose diuretic medications weren't adjusted when patients were placed on low-salt diets (a bad plan) which had negative outcomes for salt reduction), the remaining 6 papers, when combined to increase power, demonstrated a statistically significant reduction in cardiovascular events with salt reduction, and a non-statistically significant reduction in all cause mortality.
It's a very compelling little commentary, as confirmation bias or not, their arguments seem quite rational, and while it received some play in the press, it didn't receive near the same play as what seems to be the preferred new controversial narrative, that salt's no longer a worrisome thing.
So that leads me to my question, one I've posed before, is the ultimate responsibility of the journalist to the public, or to the press?
My idealism has me on the side of the public, but my cynicism (realism?) has me on the side of the press, knowing full well that telling the public what they want to hear will trump evidence more often than not.
Too bad too, think of the immense public health benefits of a unified press corps that preferentially reported science over hype, truth over contrarianism, and thoughtful discussion over grabby headlines.
Posts
Dr Freedhoff, I love that caricature and, of course, your expanded take on the Cochrane Review's reports on salt intake studies. I think that in your previous blog on this topic (to which you provide the link here), the poster "Anonymous" hits the nail on the head with this comment: ... it seems clear to me that these studies simply further an important point; cooking wholesome meals with real unprocessed ingredients will fix just about any problem, including salt issues.
ReplyDeleteCouldn't have said it better myself :-) and in that connection, three cheers for farmers' markets!
A lot of news outlets pride themselves on offering helpful news to support a healthy and happy life, so there should be some responsibility. People's health shouldn't be trivialized to get a sensational headline.
ReplyDeleteCochrane Review needs new writers, ones who can say what they mean without contradiction and waffling.
ReplyDeleteOkay, Yoni, I'll bite on this one. :)
ReplyDeleteYou cite three "somewhat contradictory" conclusions from the Cochrane Review.
The first was clear and straightforward: reducing salt has no clear benefits on CVD or mortality. You took that from the plain-language summary, but it says exactly the same thing in the results and in the non-plain-language abstract. This is simply what the data showed! If you read the discussion, it's not ambiguous at all:
"We found no strong evidence that salt reduction reduced all-cause mortality in normotensives or hypotensives."
You then cite two further conclusions that aren't "conclusions" at all. They're the opinions of the people who wrote the review. And when they say "our findings are consistent with the belief that salt reduction is beneficial," what they're saying is that their failure to prove their hypothesis doesn't PROVE the null hypothesis. Which is true, of course -- but in no way is this a "conclusion" that follows from their data!
And your third conclusion, about the "challenge for clinical and public health practice," also isn't a conclusion from the data. It's an opinion.
Now, I have nothing against opinions. I write my own all the time. But it seems strange to me to criticize the media for writing about the findings of a study without rather than focusing on the opinions of the researchers.
As for the Lancet commentary... wow. Do a systematic review, get results you don't like. Manually remove the study whose results disagree most strongly with your preconceived notion of the answer, and redo the analysis -- presto, we were right all along! I don't know what that is, but it's not science.
Here's a thought experiment: try replacing "reduce salt" with "take Alex's Magic Happy Pills." Now imagine I run an experiment that fails to prove that taking my Happy Pills extends life by a decade. However, because of the nature of the scientific method, it also fails to DISPROVE that my happy pills extend life by a decade. As a result, in my conclusions I write that the study "is consistent with the belief that Alex's Magic Happy Pills extend life by a decade." How should the media report this study? Should they focus on the data, or the opinion of the study authors?
Last point: I'm not a salt nihilist. I certainly watch my salt intake, and I remain unsure of the long-term effects of high salt intake. But I don't agree that the risk of potential harm justifies some sort of journalistic code of omerta to remain silent about findings that run counter to prevailing orthodoxy -- especially when the studies in favour of that orthodoxy have been getting a free pass for decades.
Thanks for the great comment Alex.
ReplyDeleteYour point regarding conclusions vs. opinions is a fair one, though I think it would have also been fair, given the context in which the conclusion was wielded by much of the media, to include mention of the opinions as well.
Regarding the reanalysis, there we'll need to disagree some. The concerns of He and MacGregor, regarding the potential development and risks of hyponatremia subsequent to not modifying patients' diuretics on a low sodium diet are valid, and may well have contributed to the negative outcome. Further, as mentioned on Twitter, using a single day's sodium excretion as a surrogate for a long term consumption pattern isn't particularly wise, though no doubt sodium excretion is the gold standard means of studying intake. I know if you checked mine today, it'd be fairly low, whereas yesterday, given what I ate, it'd have been much higher.
Willett's date on unchanging sodium excretion over time is fascinating, and surprising, but I don't think discredits the potential role of dietary sodium reduction on blood pressure, it just calls to question the notion that we're eating much saltier diets than before.
All told, I'm not averse to questioning dogma, I just don't feel the media provided a balanced take on this piece and instead, purposely tried to conjure up more controversy than was warranted by the studies in question.
So for me my consternation wasn't about some need for the media to stay silent on the study, it was about spin and balance. Of course you might fairly argue that balance has been provided by years of anti-salt biased reporting, but course that would just be the same problem spun in a different direction.
(BTW - I'd love some of Alex's magic happy pills.)
Thanks for the response, Yoni -- and of course, ultimately I agree with you that balance in articles is preferable and sadly elusive, including in the SciAm article. (On the other hand, I thought your original take on the JAMA paper back in May was excellent: open-minded, considering three different possible explanations, and accepting that for now we don't actually know what the answer is.)
ReplyDeleteI still feel, though, that the current debate about the Cochrane Review is dripping with double standards. I mean, sure, single-day sodium excretion has limitations. But how many times is that raised in discussions of studies that SUPPORT sodium reduction? How many articles about last year's NEJM prediction that salt reduction would save 44,000 lives a year noted that these findings were partly based on studies that use single-day sodium excretion, which "isn't particularly wise"? Raising questions of "journalism and ethics" shouldn't be a cudgel that's only used to chastise one side of a debate.
Maybe I could propose a two-tiered distinction that would keep everyone happy: science journalists should be allowed to report on the findings of studies, while public-health journalists will be obliged to report what the study authors believe the findings should have been. :)
I'm guessing bloggers have far more luxury in the way of being able to say, "I don't know" than science journalists (or public health ones).
ReplyDeleteCan't imagine many editors wanting my very much tripoded onto a fence post take on the JAMA article over a more forceful pro-sodium piece.
There are certainly some wonderfully thoughtful and balanced science journalists out there...but I'm guessing as with most things in life, it's the flashy, controversy that gets the most attention (and has the most sticking power).
Perhaps I should be getting frustrated with editors rather than reporters...or with the public for not wanting real science/evidence based reporting.