Lot of that combo going around, not least in science. Michael Eisen and Rob Tibshirani argued this week that the deluge of coronavirus-related preprints requires “American scientists and journalists to join forces to create a rapid-review service for preprints of broad public interest. It would corral a diverse contingent of scientists ready to comment on new preprints and to be responsive to reporters on deadline.” So: basically, fast peer review. (In the now-mandatory group stamp of approval for any slightly disruptive concept in research, more than 100 scientists have signed on to Eisen and Tibshirani’s proposal.)
Andrew Gelman, while saying he’s supportive of the idea, gives it a damning critique. His objections:
- Peer review has never been a guarantee of quality — so why would it be in Eisen and Tibshirani’s model? (And, in fact, it hasn’t during the coronavirus, either.)
- Academic power structures would have to be skirted — good luck.
- The journalist/sci comm industrial complex has selection bias toward stories of discovery, scandal and academic pissing matches. So papers that don’t get the stamp of approval from the Eisen/Tibshirani review machine but that match any of these paradigms (usually discovery) will probably still get covered, defeating the point of rapid review.
But Gelman isn’t just describing research during the pandemic. Points 1, 2 & 3 have been the status quo for decades — how research is framed and marketed by its authors and communicators and networks, as well as how it’s adjudicated. (Read some of Gelman’s interactions with journalists if you doubt the sweep of his claims.)
If we’re at a point where review is so structurally flawed, susceptible to power and dismissable in the face of false but irresistible narratives, then faster review in the service of that dysfunctional system isn’t the answer — in a pandemic or out of it.
The answer is to stop feeding this system and start communicating expertise and the state of science, not individual papers.