Right off the top let me say I'm not well versed enough in statistics to know who's right.
On one side of the fence are the findings of Christakis and Fowler, famously published in the New England Journal of Medicine that posited obesity is socially contagious. Non-statistically, their paper didn't sit right with me, but as far as stats go, I'm no maven.
On the other side of the fence is a new paper published by Russel Lyons who posits that Christakis' and Fowler's work is a great example of statistical illiteracy, and that the conclusion drawn from their data, that obesity is socially contagious, is severely flawed and can't be made.
Lyons' paper, in a nutshell, gives statistical meat to my gut's firm belief - that shared environments and self-selection may well be explanatory for the clustering of obesity in social networks. That folks whose lifestyles may be more conducive to obesity, may well gravitate towards one another, and/or that people living in geographically/socially similar environments, environments that may contribute to the risk of obesity, together share increased risks and outcomes.
Statistical arguments aside (frankly they're way over my head and I couldn't begin to venture a guess who's right and who's wrong), what was most fascinating to me was Lyons' discussion of his paper's publication.
If Lyons' hypothesis is correct, his paper's a big deal. It refutes one of the most widely publicized studies of the decade, one that's translated itself into millions of dollars of grants, countless news stories, and even a book that's been published in 20 different languages.
So what happened when he tried to publish it?
The New England Journal of Medicine and the BMJ rejected it outright, without peer review. JAMA, the Lancet and the Proc. Nat. Acad. Sci. all rejected it next, this time because they have policies of not publishing critiques of articles they themselves didn't publish.
Next Lyons submitted his paper to a statistics journal. The journal, Stat. Sci., did send it out for peer review. 2 of the 3 referees recommended publication without revision, and the 3rd, clearly not an obesity researcher, stated that while they agreed with Lyons' conclusions, that the subject was not important enough to warrant publication. Stat. Sci.'s editors agreed with reviewer number 3, and rejected the paper.
Eventually Lyons' work was published in the journal Statistics, Politics, and Policy, whose impact factor rates at 0.857. Contrast that with the impact factor of 50 that the New England Journal of Medicine enjoys.
The entire experience has led Lyons to use his paper as a call to action to establish a journal whose subject matter is made up solely of study critiques. Were such a journal available, it would create a venue for publication of important criticisms, further protect the public from bad statistical analyses, and potentially serve as an incentive for researchers to double check their work.
All in all, even if you're not a statistician, Lyons' paper is worth a sober read and reflection, and here's something else to chew on - the journalists who were originally all over Christakis' and Fowler's work? I'd bet every last penny I've got that not a single one of them were skilled enough in statistical analysis to analyze it. Really, why should they have been? They're journalists, not statisticians. No, instead they smelled a good story, and ran with it. Those same journalists who shouted from the rooftops that obesity's contagious? I'm betting the vast majority of them are going to be silent on this one, yet wouldn't re-reporting be the socially responsible, ethical, and journalistic right thing to do?
Now I know that plenty of reporters read this blog. Would love to hear from you. Am I off base?
Lyons, R. (2011). The Spread of Evidence-Poor Medicine via Flawed Social-Network Analysis Statistics, Politics, and Policy, 2 (1) DOI: 10.2202/2151-7509.1024