I haven’t paid much attention to astronomy since the onset of puberty — but of course this piece caught my eye on Friday: BBC science reporter Pallab Ghosh writing for Undark on “Exoplanets, Life, and the Danger of a Single Study,” prompted by the uncritical media coverage (then walked back) of two papers published last month announcing the discovery of water vapor on a exoplanet called K2-18b.
The discovery got lots of people excited that K2-18b could be a “potentially habitable super-Earth,” as EarthSky phrased it. Except that lots of astronomers jumped in (on Twitter) to say that the SIZE of K2-18b plus the discovery of water vapor meant that it was almost certainly more like Neptune than Earth, a “mini-gas giant” dominated by hydrogen and helium — i.e., NOT habitable.
So why weren’t those astronomers called in the first place? And what does “the danger of a single study” mean for research-driven organizations?
Ghosh, who’s been a science journalist for 35 years, breaks it down this way:
- Uncritical single-study stories are increasing in frequency — because of a) decreased resources and training for skeptical science reporting, b) more sophisticated and coordinated science communication and c) increasing pressure from and on scientists to overhype studies.
- Too many journalists think “peer-reviewed” means “proven.” Most of the initial media coverage of the K2-18b didn’t reflect dissenting views. That’s because, as Ghosh puts it, “many journalists believe that if research has been published in a peer-reviewed journal, it must be credible, and they make the mistake of reporting the research uncritically.”
- Journalists ask hard questions, even of basic research papers. Papers aren’t “tablets of stone” handed down from “omniscient researchers,” and it’s the job of science journalists “to challenge what we are told” — not just for controversial topics like GMOs, cloning and climate change, but for basic science discoveries, where there “is often plenty of debate.”
- The discovery narrative sucks — “‘people used to think x and now, because of this discovery they think y’ is not the way science works, and it quite frankly makes for boring copy.” And yet it continues to dominate science reporting.
- Hype has always been part of science and starts with scientists, Ghosh says — both the hype borne of “researchers’ natural enthusiasm” and that borne of “their sometimes-deliberate efforts to drum up publicity in order to secure research funding.” (Ghosh is really good here on how the Holy Grail for cancer cures etc. are “seemingly always between five and 10 years away.”
- Sophisticated science comms shops have made the hype train even bigger today, building “an information pipeline that runs uninterrupted from scientists to press officers to the news media.”
- The result: too often, stories that “are their own brand of fake news,” Ghosh argues. “A great science story counts for nothing if it gives readers a misleading impression or paints a cartoonish, one-dimensional picture of how science works.”
- In essence, we’re back to the 1980s, says Ghosh, when single-study stories were the norm and journalists were science communicators, translating and explaining.
How does/can this dynamic change? Ghosh doesn’t offer ways forward, other than journalists getting better at what they do. Here are my thoughts:
- The single-study economy matches the demands and distribution dynamics of the attention economy really, really well — it’s a rational response to pressures from both funders and the news cycle, and the results (when you can get them) potentiate each other. It will be very hard to break up this synergy.
- Journalists hate getting burned. Only a series of explosive scandals can introduce the skepticism necessary among journalists to rebalance their need to file with the need to be journalists and “challenge, weigh, and assess,” as Ghosh puts it.
- Your research-driven organization could achieve significant differentiation (and competitive advantage) by positioning itself as the group that put single studies in the context of your field’s literature — in essence, in standing up for science as opposed to hype. It’s significant that Ghosh played the “fake news” card in regard to our single-study addiction.
- Better, more compelling “science communication” is not enough if it’s mindlessly about adding to the noise and confusion. Are we promoting just another fake discovery narrative, or adding to real knowledge?
Or perhaps, as Aaron Carroll wrote this morning about yet another round of diet-soda-is-bad-for-you studies:
As long as the culture of science demands output as the measure of success, these studies will appear. And given that the news media also needs to publish to survive — if you didn’t know, people love to read about food and health — we’ll continue to read stories about how diet soda will kill us.
And single studies that don’t give us the full picture of what the science says.
Congratulations to Pallab Ghosh for opening another front in this critical conversation.