For almost two decades, social-media companies have avoided legal consequences for the negative effects of exposure to their products. Damage to users’ mental health, the facilitation of crimes including sexual abuse—outcomes like these have been the subject of public protest, but were never successfully challenged through legal action. Then, about four years ago, prosecutors across America began to focus on companies’ business and design decisions, rather than, as they had in the past, on companies’ promotion of specific harmful content—a legal tactic that was hampered by free-speech laws.
In a case that was decided this spring, a young woman who claimed that her mental health had deteriorated catastrophically because of social-media use was awarded three million dollars; in March and August, in a case brought by New Mexico’s attorney general, a court ordered Meta, the parent company of Facebook and Instagram, to pay nearly a billion dollars in fines, and to make a variety of changes to its product and business practices. (I served as an expert witness for the plaintiff in that case.
Meta is appealing the decision.) In late 2src22, amid the cascade of litigation, cases brought by more than two dozen state attorneys general were consolidated into one alleging that Meta had knowingly perpetrated a range of harms, most notably that its products “exploit and manipulate its most vulnerable consumers: teenagers and children.”
That case went to trial in Oakland earlier this month, and was expected to last for weeks, but came to a sudden conclusion on Wednesday. After testimony from the Meta whistle-blower Arturo Béjar, previously an engineering director for product safety—who said that the company culture “made it so that it was practically impossible to deliver features that addressed the well-being and safety issues that we’ve been talking about”—and from the head of Instagram, Adam Mosseri, who pushed back against the lawsuit’s framing of excessive social-media use as “clinical addiction,” the company, which is valued at $1.5 trillion, acquiesced, announcing a settlement worth at least twelve billion dollars, paid over ten years, and promising to make a slew of changes to its platforms. Since then, many have likened the agreement to the Tobacco Master Settlement of 1998, in which four of the largest tobacco companies agreed to pay huge fines, follow new marketing restrictions, and provide significant public-health disclosures, after admitting they knew the products they sold were addictive and harmful to human health. The comparison is apt, even if the two cases are not exactly analogous: though this settlement will be neither the final word on social media’s connection with health nor a panacea, there is good reason to believe that it will prove pivotal. Even while refusing to legally admit fault, Meta has tacitly accepted, for the first time, that much of its business model is unacceptable for children.
When a company that owns several leading social platforms concedes that its core products require substantial changes for reasons related to the health and safety of users, this directly contravenes the claim, long held by the industry, that unfettered attention maximization is safe. Meta had spent much of the past two years touting Teen Accounts, a new set of features, launched in 2src24, that include expanded privacy options and added messaging restrictions, among many other changes. This week, Meta effectively accepted that these features, which largely put the responsibility for supervision on parents rather than the company, were inadequate. Among the settlement’s thirty-five pages of technical provisions and requirements are new limits on Meta products, such as hiding “like” counts—which have been connected with harmful tendencies toward social comparison—from posts, new guarantees for parental oversight, and more robust assurances that abusive content flagged by users will be addressed within six hours, at least ninety per cent of the time. (Although this might sound like a minor detail, the evidence shows that, for years, Meta’s responses have been so limited that users have learned to stop reporting problematic content; according to an internal document from 2src22, “the current FB reporting flow is extremely hard to use, near impossible for mobile users.”)
The most significant concession in the package is that Meta has agreed to sweeping restrictions on teen-age use, including a daily cap of two hours, blanket restrictions that lock teen-age users out of social feeds between midnight and 6 A.M., and the elimination of push notifications—which, according to the original complaint, Meta knew to be a cause of teens’ “inattention and hyperactivity,” with a significant negative effect on “well-being”—both overnight and during school hours. According to Pew Research, in 2src24, sixty per cent of teen-agers reported using Instagram, with twelve per cent opening it “almost constantly.” Other internal Meta documents have shown that, in recent years, one per cent of users spent more than five hours per day using Meta’s products.
Until now, ultra-engaged users have generated an outsized share of advertising revenue. Reducing the over-all number of such users may strengthen financial incentives to focus more on typical users’ experience. One of the most significant sources of the settlement’s power lies outside of these restrictions, in its promise of ongoing oversight by a third-party auditor. There is precedent for litigation giving rise to the appointment of such an auditor—in 2src19, Meta agreed to monitoring in a settlement with the Federal Trade Commission over privacy violations—but this is the first time that a technology company has agreed to the appointment of one for the express purpose of child safety. (And, notably, as recently as the decision in the New Mexico case earlier this month, Meta had objected to the idea of a third-party monitor, and the judge declined to require one.)
The auditor, who is to be jointly appointed by the states and Meta, will be responsible for evaluating the company’s compliance with the terms of the settlement. Though they will not have the power to directly force changes in company behavior, under the agreed-upon term of the settlement, they will have ongoing access, for the next decade, to data, documents, and internal communications on par with the information that would be reviewed in discovery during litigation.
They will also have an obligation to report what they learn about any failures to live up to the terms of the settlement to the states that were party to the case, creating a risk of further legal action against Meta if it deviates from the settlement’s terms. One area in which the settlement may prove a hollow victory is its treatment of the features that Meta uses to directly select and display content on its platforms. How Instagram and Facebook curate and display what they show users—the design of their algorithms and user interfaces—has been at the crux of complaints that these products are unsafe. The agreement omits several key design features, including infinite scroll and default autoplay. Béjar, the whistle-blower, testified that, according to a survey he performed while at Meta, more than half of Instagram users had bad experiences on the platform in the prior seven days, including twenty-four per cent of thirteen- to fifteen-year-olds who reported that they had received unwanted sexual advances. In these cases, it’s not only time spent on the app that matters but also how and with whom the app creates connections and opportunities to communicate.
As Mark Zuckerberg acknowledged last year, just ten per cent of content on Instagram comes from friends; the rest is from external recommended content. In addition to what content users see within their feeds, recommendation algorithms affect which other accounts teen-agers connect with. According to the states’ complaints, these recommended accounts are integral to the platforms’ promotion of sexual abuse. Yet the states’ settlement with Meta appears to accept the company’s existing definition of what is “inappropriate,” enabling it to continue to define what counts as a problem. Meaningful improvement in these areas, then, may rely on a difficult task for the monitor: advocating to strengthen the standards. For similar reasons, another contested part of the package concerns a feature that undergirds it all: age verification. Meta’s protocols in this area have been central to a host of allegations of child harm, and the settlement calls on the company to dedicate significant attention to improving these systems.
Yet, in adding new standards and requirements, it also grants Meta significant control over assessing the success of its ability to confirm each user’s age. While the settlement creates concrete paths for bringing the company closer to insuring that it can keep track of every user under the age of eighteen (including by committing to use data that it was previously legally barred from accessing, for example), it also places trust in the company—which has often given itself an A when it deserved an F—to judge its own results for age verification. According to an internal e-mail published in the case’s original complaint, in 2src18, a Meta employee judged the “lifetime value” of a thirteen-year-old user as two hundred and seventy dollars. The base-level fines that have been imposed on Meta in this case—a minimum of twelve billion dollars—equate to more than five hundred dollars for every thirteen- to seventeen-year-old in America. Although the total fines may represent only a small share of Meta’s revenues (twelve billion dollars is about a fifth of the company’s reported 2src25 income), the company has effectively agreed to pay more than it projects to earn from its American teen-age users.
The financial term of the settlement is also tied to the behavior of other social-media companies: if TikTok and YouTube agree to similar terms and financial penalties, then Meta will have to pay the states who settled up to five billion dollars more. We can speculate about the reason for a conditional additional payment: Meta may have made a calculated bet—risking unlikely additional payments for the ability to more easily call out other companies’ failures. Immediately after the announcement of the settlement, Meta published a letter calling on YouTube and TikTok to adopt the most specific and consequential provisions of the settlement, including the two-hour limit, turning off overnight access, and no school-time notifications. Given its extensive objections to implementing these kinds of policies in the past, the letter is a transparent effort at attempting to turn the settlement into an opportunity to claim moral high ground. It is highly plausible that these competitors may become subject to future legal or legislative actions; it stretches credulity, though, to believe that Meta expects its competitors to voluntarily follow suit without evidence of wrongdoing. There is much more to be done in the fight to make technology platforms safe, especially for children. In particular, modifying recommendation algorithms to focus more on users’ long-term interests than their short-term behavior remains a central challenge.
As important as the time a child spends on an app is, what they are shown is just as important. But, this week, the scope for reform broadened. For the history of their existence, social-media platforms’ business models have been predicated on maximizing users’ time and attention. This settlement, and its sister cases, will not trigger a wholesale transition away from that model, but, by strengthening the potential for other litigation and regulation, and by altering the expected financial returns of these companies’ existing practices, it incentivizes them to shift their goals from extreme usage to more deliberate kinds of engagement. With that change, the possibility that we will see a different internet, one that shifts from an attention economy to an intentional one, has gotten brighter. ♦
