Chipotle just announced its “Real Foodprint” initiative — an app that purports to give customers “five key metrics” on the “sustainability impact” of 53 “real” ingredients used in Chipotle meals compared with “conventional” ingredients.
And now my scare quote key is broken …
Jayson Lusk is skeptical of Chipotle’s claims, even though the campaign is fronted by Bill “The Science Guy” Nye (wink wink). As Lusk notes, while we might want simple metrics of our food’s environmental impact right on the package, such metrics are very hard right now to produce credibly. “An environmental label,” he wrote in 2015, “would require more extensive (and more costly) monitoring and tracing, and if it is at all accurate, one could have two Wheaties boxes that are nutritionally equivalent but with very different environmental impacts.”
Chipotle is marketing, and Bill Nye isn’t a real scientist. So while Lusk is right, many of us are also predisposed to see through the scare-quote science that is Real Foodprint — and sneer at those who might fall for it.
It’s harder to see (much less see through) scare-quote science when it’s about something that’s become for us an article of faith as well as a political badge — say, masks and COVID-19.
For instance, the new modeling from the University of Washington’s Institute for Health Metrics and Evaluation (IHME), which says that universal mask use through next spring could prevent nearly 130,000 deaths from COVID-19 in the United States.
That number (along with the prediction of perhaps 500,000 deaths by March, even with social distancing mandates) got headlines. And headlines were, one strongly suspects, the point, amidst a culture war about masks and the virus led by a U.S. president who selectively mocks mask wearing and says the country is on the verge of putting the virus behind it.
But according to reporting in The New York Times, the model “does not take into account the treatments available now for people who are hospitalized” or the drop in death rates among hospitalized patients with COVID-19 since last spring.
Omitting those variables would seem ill-advised when trying to produce accurate modeling for a crisis not know for standing still.
In a STAT piece covering the IHME model, University of Washington biostatistician Ruth Etzioni makes a good counterargument: the model’s absolute numbers (e.g., the 500,000) are less important than the comparisons it makes between different scenarios — such as “with” and “without masks” — which she thinks should drive policy and behavior change:
”We don’t need a model to tell us that we should all be wearing masks, we don’t need a model to tell us that if we continue the way we’re going, we’re going to see tens of thousands more deaths within the next couple of months,” said Etzioni. “But sometimes when a person provides a model and you see these curves and you see these numbers, it helps appropriately freak you out.”
Which — again, one suspects — is the point. The freak out is what’s important. The desire to freak people out comes first.
Models are not predictions, as Zeynep Tufekci reminds us. At best, they help you chop off the long-tail catastrophes through actions that might at the time look like overreactions (like a wearing a mask outdoors all the time, or mandating outdoors mask-wearing). In this sense, the IMHE model could give cover to policymakers who want to institute universal mask mandates in order to avoid lockdowns in the dead of winter.
Then again, while some models are useful, all models are wrong — some more than others. The IHME’s COVID model in spring, described by STAT as Donald Trump’s “oracle of choice,” got lots of publicity for being first out of the gate, and then drew heavy criticism from other epidemiologists. IHME eventually adjusted its methods to fall in line with other models.
But just because one model has a long-tail catastrophe doesn’t mean we should act on it. What is our standard for reconciling these claims across a number of models? What constitutes replicability?
Let’s not underestimate the perils of imposing masking mandates based on modeling whose freak-out numbers rest on questionable assumptions.
And let’s not underestimate the corrosiveness — to public trust in science — of using scare-quote data to freak people out of their pandemic fatigue.
When you see freak-out science, make sure to check for the scare quotes.
When Bill Nye finishes his sofritas bowl, maybe he can help this campaign out.