How researchers get heard

Instagram, Teens & What Research Says

Everyone is piling on Facebook over the recent revelations that it has research showing Instagram is widely harmful to teenage girls.

Except the research doesn’t show that.

Psychology professor Laurence Steinberg wrote about the holes in the research at length in a recent opinion essay published in The New York Times, “Does Instagram Harm Girls? No One Actually Knows.” His lede:

Amid the pillorying of Facebook that has dominated the latest news cycle there is an inconvenient fact that critics have overlooked: No research — by Facebook or anyone else — has demonstrated that exposure to Instagram, a Facebook app, harms teenage girls’ psychological well-being.

Yes, Facebook conducted its own survey research of teenage users of Facebook products (like Instagram). And yes, Frances Haugen, a former Facebook employee who recently testified before a US Senate subcommittee criticizing various Facebook practices, has said that Facebook had kept the research results secret because the results showed that about 30% of adolescent girls reported using Instagram made them feel worse about themselves.

But none of that proves anything, says Steinberg. While the correlation is concerning, he writes, “such a finding should be used as a starting point for research, not a conclusion.” Why?

  • Because we are the worst explainers of what we do. Psychological research has over and over cast doubt on the explanatory power of interviews and survey self-reporting (as the Facebook study relied on) to provide reliable explanations for why we feel or behave the way we do — so we can’t take a “30% say they feel worse after using Instagram” claim at face value.
  • Because the Facebook study didn’t account for confounders. There wasn’t a control group of people who didn’t use Instagram in the Facebook study. So we can’t be sure that another reasonable variable (such as a poor initial mental state or family conflict) wasn’t also contributing to the teens’ Instagram experience and subsequent mental state.
  • Because preexisting evidence for such a link is weak at best. “Of the better studies that have found a negative correlation between social media use and adolescent mental health, most have found extremely small effects,” writes Steinberg, “so small as to be trivial and dwarfed by other contributors to adolescent mental health.”
  • Because you could read the Facebook study the other way even more easily. The Facebook survey also found that, as Steinberg reports, “three times as many teens said Instagram made them less anxious as said made them feel more so and nearly five times as many reported that Instagram made them less sad as that it made them sadder.” (It’s important, Steinberg argues, to be just as skeptical of these positive correlations as of negative ones.)

Three other points Steinberg doesn’t mention:

  • The 30% of teen girls who said in Facebook’s survey that Instagram made them feel worse about their bodies was based on only 150 respondents.
  • Those 150 respondents only talked about Instagram’s role in how they felt about their bodies if they had already reported having body image issues.
  • Another study that used a nationally representative sample and the PHQ-8 scale to measure depression in respondents found that 43% of respondents said using social media makes them feel better, not worse, when they’re depressed, stressed or anxious. Only 17% said it makes them feel worse.

If you’re not a researcher, you might still be saying: This is just typical research caution. It just makes sense that social media is depressing kids. I know Instagram is bad for my daughter (even though I use it myself). We need that minimum age restriction on social media use!

This argument is the one made by two previous Times opinion essays on Instagram and teenage girls — one by Times columnist Lindsey Crouse, and another by psychologists Jonathan Haidt and Jean M. Twenge. Both make a correlative argument about teenage depression and the rise to media domination of social media in the early 2010s — in Haidt and Twenge’s case, the rollout of the “like” button or its equivalent across many platforms. Crouse cherry-picks the studies she cites. Haidt and Twenge admit that research is inconclusive, but keep asking, in effect: What else could be causing the decline in teen mental health if not social media? (A question, as we’ve seen, answered by Steinberg.)

Steinberg’s approach is a whole-of-evidence argument, as oppose to Haidt and Twenge’s correlative/common-sense argument or Crouse’s mostly common-sense argument. Another way of putting this: Haidt and Twenge cite papers, while Steinberg evaluates what all the papers tell us.

In a perfect world, we would read Steinberg’s essay first, then read Crouse’s and Haidt and Twenge’s. That order would prime readers to be more skeptical of the latter two emotional, of-course-it-is arguments.

Of course, that kind of priming is not what happens IRL. Instead, we’re much more likely to read a headline and our friends’ commentary on the headline, served up by an algorithm that favors content we’re likely to agree with. IRL, Steinberg’s argument gets drowned out by headlines saying “Facebook has known for a year and a half that Instagram is bad for teens despite claiming otherwise.”

I’m a little surprised the Times even published Steinberg’s essay, given the deep skepticism of its recent reporting and opinion writing on Facebook and tech at large, of which the Crouse and the Haidt and Twenge pieces are prime examples.

Still: Reminding audiences about the standards for research validity and that one study does not an evidence base make are essential functions of research-based public experts. Journalists by and large don’t catch this stuff. In fact, they go into the tank for it.

Takeaway: As a researcher, you have a simple but massive advantage over everyone else making research-based claims in public: You know the quality of evidence for your arguments and everyone else’s. It’s your job to know.

The challenge today is getting enough people (or the right people) to recognize that advantage. It’s lonely work, and Laurence Steinberg does not have a ton of Twitter followers all of a sudden.

But to paraphrase Wayne Gretzky: We (the public) miss 100% of the shoddy research and evidence gaps you (the public expert) don’t call out.