2014–2026

Evidence timeline

539 passages quoted from Meta's internal records, court filings and congressional testimony, in date order. Each is labelled with its source.

Internal documents only

539 excerpts · Page 4 of 22

2018

May 2018

Knowledge

Internal documentPresentationTime Spent: The Research Journey

"We recruited 8 teens from the survey, and found they thought they were spending too much time on social media, becoming emotionally exhausted, and felt they had to manage their accounts rather than enjoy them"

Addiction & problematic useNotifications

• "These teens reported that while their phone didn't wake them at night, it was a constant distraction during both the night and the day" • "Before bed, these teens used their phone to decompress after a long day, but felt compelled to respond to messages/notifications from friends" • "During the day, these teens said their phones were a constant distraction, both because of notifications and the convenience of immediate use, so they would unanimously use a feature that limited notifications to certain times of the day" • "A survey of US teens (n=770) revealed that 84% had taken a break from digital technology in the last year, spanning a few days to over a month" • "We recruited 8 teens from the survey, and found they thought they were spending too much time on social media, becoming emotionally exhausted, and felt they had to manage their accounts rather than enjoy them" • "They were able to catch themselves when their social media use 'got out of hand,' but wanted tools that helped them track their usage, give gentle reminders when they started to reach 'excessive amounts' of app time, and/or provide subtle suggestions to try offline activities instead"

Youth Targeting

Internal documentMessagesRe: Rob's Youth Update - Sep 7

Robert Chen: "What can we build for 9th graders? 8th graders? 7th graders? 6th graders?"

"Winning Teens = Winning Generations" "Thesis: Facebook, if it is to win with teens, will win with teens (13-18 y/o's) on a generation by generation basis." "Thus, we cannot hope to convert current 13-18 y/o's to Facebook if they already have a different set of primary social apps (in the US these are SC and IG)" "We will, instead, need to capture people before they are set in their social media use" "What can we build for 9th graders? 8th graders? 7th graders? 6th graders?" "What can we build for when people join a new school? How can we build products that make it so that when you start Middle School, Facebook is the best product to use and the way you connect with your peers?"

Knowledge

Internal documentPresentationTime Spent MVP

US Teen: "When I'm sleeping, I get a lot of Instagram notifications"

Addiction & problematic useNotifications

"More people are using Instagram longer, and more frequently, than ever before." "MAP + 37% YoY" "DAP + 36% YoY" "Avg daily time spent + 16% YoY" "Avg daily sessions/DAP + 9% YoY" "Avg time spent/session +8.1% YoY" "WHEN USAGE IS NOT WELL-MANAGED, IT CAN Interrupted sleep Impacted focus Guilty usage" US Teen: "When I'm sleeping, I get a lot of Instagram notifications." US Teen: "Because when you're asleep, you don't want anybody texting you or trying to message you. And, if you are trying to study for your midterms, you would want to focus." US Teen: "If somebody needs to stay focused while they're studying or at work, that's a good thing to have. Because I mean, most people will have their phone off to the side, so if it's constantly lighting up, and you see things, that gets really distracting." US Adult: "I'm in the middle of an important writing assignment. I decided I wasn't going to use Instagram until I was fully finished with it. So I didn't use it all day. It was really difficult. I noticed my thumb automatically clicks on the icon on my phone, and I thought 'this is sick. how addicted am I to this app?"

June 2018

Knowledge

Internal documentReportFacebook "Addiction": A response to the public narrative based on clinical and neuroscience research

"It may be a problem if Facebook seems rewarding based on the principle of unpredictability, while the inherent value of the reward is lacking"

Addiction & problematic useContent AlgorithmAutoplayLikes+2

"It may be a problem if Facebook seems rewarding based on the principle of unpredictability, while the inherent value of the reward is lacking • Notifications with little or no relevance, and that come at unpredictable times • Prominent novel content that creates unwanted distractions • News Feed stories with seemingly 'random' content and unpredictable order • Other use of unpredictable rewards, such as delays to load" "Facebook's rewards are easily accessible, available all the time, limitless, and don't include built-in controls. For people who tend to have difficulty regulating their consumption of rewards, it may be easy to use Facebook 'too much.'" • "Continuous content without breaks" • "Signals of availability that lead to expectations of quick responses (read receipts, green dots signaling online presence)" • "Constant updates including Like counts" • "Lack of feedback about amount of use" • "Log in info (username and password) automatically saved leading to easy, immediate access" "It may be a problem if there are cues for Facebook use that influence people's behavior based on automatic tendencies, when they actually don't want to be using Facebook. • Unwanted push notifications and badges, especially ones that are on by default • Stories or videos that autoplay • Immediacy of seeing News Feed"

Knowledge

Internal documentReportFacebook "Addiction": A response to the public narrative based on clinical and neuroscience research

"Facebook may be rewarding and habit-forming"

Addiction & problematic use

"There is no evidence that Facebook use meets the clinical definition of 'addiction.' The way the media portrays Facebook 'addiction' can be exaggerated and inaccurate. However, Facebook may be rewarding and habit-forming. Therefore some people may experience genuine problems related to their Facebook use." "Neuroscience research has shown that Facebook does activate the brain's reward system" "For people who tend to have difficulty regulating their consumption of rewards, it may be easy to use Facebook 'too much.'" "If someone's primary way of coping with difficult emotional experiences is to go on Facebook, they may become reliant on Facebook and have trouble cutting back on use" "The negative emotion itself could become a trigger for use - so there would be constant real life (non-Facebook) cues that could drive someone to use 'too much'" "In addition to Facebook being rewarding/reinforcing, some people feel like their Facebook use is difficult to control, or has become a habit."

Youth Targeting

Internal documentMessagesVideo M-team review CONFIDENTIAL

Fidji Simo: "we lost this teen generation but have a shot at getting kids" (forwarded by Brian Boland)

"Lots of talk about the fact that we lost this teen generation but have a shot at getting kids, especially with Messenger kids, and should therefore 1) build more kid experiences and 2) find ways to transition from kid experiences to main FB. Video is obviously a huge kid use case no one is building, although we determined that IGTV was well positioned there."

Knowledge

Internal documentMessagesRe: Question about the value of message data

"this is the worst of the worst stuff that gives you nightmares, over which children have taken their own lives :("

Child sexual exploitationDirect Messaging

Adam Mosseri: "Kevin and Mikey are uncomfortable scanning Direct message data, which I didn't realize until last week, though they're supportive of using meta data. I'd like to make the value of scanning messages directly clear — do you guys have any idea how we might quantify the value for problems like grooming more specifically? There is the added complexity of being inconsistent across surfaces, but I thought I'd start by trying to clearly articulate the up-side." Guy Rosen: "How about we organize a shadowing session for you & them to see the kind of stuff CO / investigators find? It helps to paint a picture. For grooming - or rather, the broader bucket we're calling IIC (inappropriate interactions with children) splits roughly into these tiers (going from least worst to worst worst): • Tier 3: non-policy violating creepy behavior e.g. adult contacting 100s of minors on messenger with "hey sexy" • Tier 2: online sexual harassment e.g., sending photos of (adult) genitalia to a minor • Tier 1: soliciting or real-world harm c. soliciting CEI b. contact offense (suggesting an in-person meeting) a. content sextortion (using obtained CEI to force further acts) <-- this is the worst of the worst stuff that gives you nightmares, over which children have taken their own lives :( Of the above, we can't do much of anything on Tier 1 & 2 without proactive scanning. Also, because kids are either embarrassed and/or think they're in a real relationship, this tends to go unreported. There are also other things which we should discuss but I am okay to tackle the child stuff first, e.g. financial scams which aren't necessarily targeted at minors, but can also include financial sextortion (getting intimate images and then asking for money). These are typically run as little business out of (usually) West Africa, and often intersect with impersonation (fake attractive female, military and public figures, celebrities - even posing as Mark or Wehner to scam people on a 'Facebook lottery', etc. -- the latter we can probably tackle from the impersonation work though)." "The meta way I think about it is that both FB & IG are discovery services, not just communication services: you can do discovery e.g. search for girls a certain age in a certain place, etc. and then engage them. (At least on WA you can't do the discovery bit.) I think we have to be extra protective of what's happening on discovery services because otherwise they are basically massive 'victim discovery services'."

July 2018

Knowledge

Internal documentReportProblematic Facebook use: When people feel like Facebook negatively affects their life

"In academic experiments, smartphone notifications caused inattention and hyperactivity among teens, and they reduced productivity and well-being."

Addiction & problematic useNotifications

"In this study, we find that that 3.1% of Facebook users in the U.S. experience serious problems with sleep, work performance, or relationships that they attribute to Facebook and find difficult to change. We refer to this as 'problematic Facebook use,' in line with prior academic research on 'problematic Internet use,' which has identified problem rates of 2-8%." "The most liberal definition would be that 14% of our survey respondents report spending 'a lot more' time on Facebook than they want to." "Alternately, 11% report one of the negative life impacts (serious problems with sleep, work performance, relationships, or that FB has had a 'very negative impact' on their life)." "Problematic use is highest among teens and people in their 20s, consistent with previous findings that younger people generally have more problems with self-regulation." "People who feel like they have a problem receive more push notifications and 'view through' a greater proportion of them. Though they also respond more quickly to those notifications, those differences are accounted for by age, friend count, and tenure. In academic experiments, smartphone notifications caused inattention and hyperactivity among teens, and they reduced productivity and well-being. This suggests the value of 'Do Not Disturb'-related products in reducing or limiting notifications, or in identifying contexts where notifications may be undesirable."

August 2018

Youth Targeting

Internal documentMessages, PresentationMarket Landscape Review

"Acquiring people when they are young (e.g., 13) results in the highest lifetime value because their long-term retention is the highest when they are young"

"'The young ones are the best ones'" "The difference in long term retention can be dramatic, with the people who signed up for Facebook when they were young teens being 100% higher than those who joined FB as young adults" 2. "We care because it:" • "Makes the case for why teens are important for Facebook and how important they are relative to other age groups" "Findings thus far (PRELIMINARY)" 1. "The younger you are when you join Facebook (e.g., for year of birth 1999, if you join when you are 13 compared to 18), the higher your long term retention is" 2. "The more recently you were born (e.g., year 2001 instead of 1991), the higher your long term retention" 3. "The earlier you joined Facebook (e.g., 2009 instead of 2015), the higher your long term retention" "Very Preliminary LTV Findings" • "Acquiring people when they are young (e.g., 13) results in the highest lifetime value because their long-term retention is the highest when they are young" • "Retention overpowers the fact that ARPU is lower for young people" • "Retention overpowers the fact that future revenue is discounted" • "However, we have a bit of a correlation issue" • "People who join Facebook at a young age are different from people who don't join Facebook at a young age" • "How might we adjust for this?"

Profits over Safety

Internal documentMessagesRe: Changing default age and improving age gating during registration

The Growth Team pressed to standardize the default age at 25 despite safety objections

Antigone Davis: "Was there a discussion on using 25 as the default? We get hammered on this issue quite a bit on the safety side. It is seen as a signal to age lie. Not a safety issue, but it also seems likely to create an even greater mismatch census data to ads targeting." Shane ████: "25 has been the default on most surfaces for a while. iOS is the outlier. It was supposed to be fixed a long time ago and just never went out. On most of the surfaces we have a pop-up that asks you to confirm that 25 is really your birthday if you don't bother to change it." Antigone Davis: "And yes, I get we use 25 and this is a consistency issue but rather than push for 25 everywhere, I'd encourage us to think about present date everywhere or other options if that has real cost. The current 25 default creates real costs on the policy side and is cited to us with some frequency." Shane ████: "I've done a fair bit of pushing on the 25 point and have gotten resistance. The age gate change seemed like it would be bigger impact for us, so I've focused on getting that cleared. I'll push a bit more on understanding the impact, but I think it's significant. It's also just a pain to enter birthdays starting from 0, so there is a UI/user reason for defaulting to something closer to the mean age. Is there another age that you think would be better? Our general advice has been that it can't be close to 13." Shane ████: "They're fine to wait. The age gating was entirely for policy—they'd rather not ship it. Having inconsistent default ages creates some issues for them, but probably as much an issue for us as it's hard to explain why we have different behavior on android & iOS. I think we can do some more due diligence internally, but I don't think we should talk to external stakeholders until we get a better sense of the cost of changing these and how willing growth is to make changes." Antigone Davis: "On 25, safety folks find the 25 default objectionable and would prefer no default. The default makes age lying more passive. Are they open to testing how no default impacts sign ups? Do they have data that with no default fewer people sign up or more people lie?" Shane ████: "Getting pressure from the growth team to try to close out the default age at account creation as they'd like to get all our surfaces on the same default. The estimated impact of defaulting everyone to 0 is > 10M MAP. As part of the review of age practices we're doing I think we're likely to suggest doing this, but I don't think we're locked on that position and it will obviously take some discussion. Given that the AoC workstream is still being discussed, I'm recommending that we just hold things the way they are until we make a decision. That said, if Growth insists on doing something, I don't see the risk as that high of moving iOS into alignment with Android and our other surfaces. But if you feel strongly, I can try to hold them. Let me know." Shane ████: "Sorry. iOS is currently 0 and they want to make default 25. Cost of moving all surfaces to 0 is > 10M MAP yearly." Antigone Davis: "To be clear, you are asking if I am okay letting growth move iOS default to 25 like it is everywhere else just until we come back to them with an informed global recommendation (e.g., policy recommends we default everybody to 0)? And the likely recommendation from policy will be to default everybody to 0?" Shane ████: "Basically. I'm not positive policy will win the discussion of setting the default to 0, but, yes, wanted to see if you were ok with us going to a consistent approach of 25 for the moment until we can make a company decision on a long-term approach." Antigone Davis: "I guess so but I am inclined to use the discrepancy to ask them to explain why they have selected 25 – especially if we are going to lose the 0 option down the road and likely end up with 25 due to inertia. This approach has totally screwed up what we know about people's ages both for safety and ads targeting and if the idea is that we lose some people by forcing them to input info and we want a default let's at least universally default to an age under 18 so we accidentally over safeguard rather than under safeguard. That might not fix our ads targeting issues but it will help with safety and privacy."

Profits over Safety

Internal documentMessagesRe: Changing default age and improving age gating during registration

"Did growth agree to 2 hours [the duration of Facebook's age-gate block on underage registrations] because it is meaningless?"

Shane ████: "We're going to update age gating such that on mobile surfaces after you fail to enter a valid birthday a 2nd time during registration (i.e. you say you're under 13 twice), we block the app from registering new people for 2 hours or until you force close it." Antigone Davis: "Got it on session. FWIW, I think it would be good to test what amount of time sounds reasonable to stakeholders. My guess is 2 hours will come across as arbitrary without data to back up that people don't come back after 2 hours to try again. I would expect 24 or 48 hours will" Antigone Davis: "(Sorry accidentally hit send),... 24 or 48 hours will come across as the kind of block U13s will perceive as permanent (for now) but 2 hours will be seen as 'you don't take this issue seriously...' Another option might be random times but no less than two hours — at tleast then we can say we are trying to make it harder for them to figure out how to game the system." Antigone Davis: "If asking for more than 2 hours means this never ships, we should take 2 hours. But I think it is better to have a time validated by external stakeholders or based on data. What is the impact 2 hours has on growth? Did growth agree to 2 hours because it is meaningless? Have we looked at accounts that try again after 2 hours to see what percent might be U13 v people who just hit the wrong age twice? If we want to claim this as a win with our policy stakeholders it has to seem meaningful to them. (PS - we should think of a term other than age gating or verifying for this as it isn't what product means by either, more like a speed bump than a gate)."

Youth Targeting

Internal documentMessagesRe: Rob's Youth Update - Sep 7

Robert Chen: "'the young ones are the best ones.' You want to bring people to your service young and early."

"Lifetime Value (LTV) This number is core to making decisions about your business. Lifetime value is the cumulative total 'value' (usually expressed as 'profit') you expect from a customer/user. With this number, we can make better decisions regarding how much to spend on each user. Generally, you do not want to spend more than the LTV of the user." "Shared initial findings on long-term retention. Short summary is the 'the young ones are the best ones.' You want to bring people to your service young and early. This is the most fascinating work I have done recently. Look for more from me and ████"

September 2018

Profits over Safety

Internal documentPresentationMarket Landscape Review: Teen Opportunity Cost and Lifetime Value

"[Y]oung teens... are the users with the potential for highest loyalty and value over time"

"All figures are presented as ballpark approximations since many simplifying assumptions were made." "The major takeaway from this work is that we are in the right range if we spend $100-$200 per teen." "The lifetime value of a 13 y/o teen is roughly $270 per teen" • "However, do note that older users have much higher average revenue per users (ARPU) than younger users" ◦ "A case could be made to pursue young adults if we are able to increase their long-term retention" "Because of the high value of young teens, more product teams, and not just the Youth Team, should care about young teens as those are the users with the potential for highest loyalty and value over time" "If we are looking to spend near or more than the lifetime value ($270) per young teen, then that spending should result in extra scrutiny" "Given the importance of lifetime value, we should find a long term owner for the model—Youth leadership likely needs to make the case to Finance" "Analytics should track long term retention to understand app health and user behavior"

Profits over Safety

Internal documentMessagesRe: Rob's Youth Update - Sep 7

Robert Chen: "the lifetime value of a 13 y/o teen is roughly $270 per teen"

• "Teen Opportunity Cost and Lifetime Value. Caveat: There are lots of assumptions going into the below numbers. Consider them ballpark estimations." ◦ "Just in the US, we have an opportunity cost of $150 million a year in 2019...this rises to $1.8 billion per year in 2030" ◦ "The total present value loss is ~$12 billion" ◦ "By 2030, Facebook will have 30 million fewer users than we could have otherwise if we do not solve the teen problem" ◦ "The lifetime value of a 13 y/o teen is roughly $270 per teen" ▪ "Spending above that should result in much more scrutiny"

Concealment

Internal documentMessagesRe: Releasing child safety stats pre CSER v2

"given the size of the CEI removal number, I think [reporting] it separately may better contextualize it so people aren't scared of what's happening on FB... we need approval from Mark to release CEI [data] at all"

Child sexual exploitation

Andrea Saul: "My feeling is that we should release everything together post-election, with one possible exception of the CEI data. We want to have a moment where we talk about all we do in the child safety space, and given the size of the CEI removal number, I think doing it separately may better contextualize it so people aren't scared of what's happening on FB. Emily is pulling together what the narrative itself will look like, so we can see that before making any decisions – and we need approval from Mark to release CEI at all, which the data team is taking to him in the next couple of weeks for a decision." Vishwanath ████: "Strongly agree with both the points on not using fake accounts in the context of elections as well as the child safety data." 2. "Child Safety is going to be hard to explain and talk about given the nuance and nature of the numbers. I fear the nuance will get lost in the large set of numbers for CSER and feel talking about Child Safety earlier would really be valuable and reduce the risk of misunderstanding." Guy Rosen: "Starting a thread on potentially releasing CEI numbers before CSER v2 to prevent the nuance being lost when we publish. This seems like a very reasonable thing because the numbers do seem scary w/o context. The one additional thought I wanted to float here is to package it with some 'good news' - either: a. new product development we can talk about in the space b. I've always wanted to do this: success stories of cases we've helped children? We literally save lives (recent example here) and we have historically been worried about talking about it. I wonder if we need to lean into this more similar to how we've talked about suicide success stories. Also given some of the conversation e.g. in the UK seem to be moving from just CEI to Grooming, showing that we literally have cases where AI detected something and a child's life trajectory is changed for the better could be valuable (seriously I get goosebumps writing this)."

Knowledge

Internal documentMessagesRe: Releasing child safety stats pre CSER v2

Antigone Davis: "we do very little to keep U13s off our platform"

Child sexual exploitation

"The biggest challenge here is that for the most part no one thinks FB makes people suicidal but people (especially policymakers and law enforcement) do think we create the opportunity for children to be groomed. Until we have really strong mechanisms for preventing inappropriate adult/minor contact, I think we risk generating more negative press than positive – especially when we do very little to keep U13s off our platform."

Profits over Safety

Internal documentMessagesI would have thought that he would be more an advocate of harm prevention

"I would have thought that [Mark Zuckerberg] would be more an advocate of harm prevention"

Child sexual exploitationDirect Messaging

Guy Rosen: "btw I prewired Mark a little about encryption / scanning his initial reaction was 'in the end it'll all be encrypted' But I explained the net harm concept etc He needs to think about it - mostly he'll have context for when we eventually bring it to his attention, sooner or later." Jonathan ████: "I would have thought that he would be more an advocate of harm prevention -- but it sounds like we're going to have to make the case against encryption by default" Jonathan ████: "Here's how I'm thinking about it: https://fb.quip.com/6YAJAIJpUGMs" Guy Rosen: "I think we need to get really crisp at the 'victim discovery platform' argument BTW I think safety orgs make the point, I'll fwd something from Antigone that says that"

October 2018

Knowledge

Internal documentMessagesRe: Body-enhancing filters & Well-Being Lit Review request

"there is a lot of competitive pressure and a lot of market demand for filters that go much more directly into the beautification space. And if we test any of these things, they will undoubtedly perform well. But just because people like and want something in the short term doesn't mean it's healthy for them."

Body image & eating disordersFilters

Jennifer Guadagno: "I wanted to follow up on the request we chatted about yesterday to understand more about possible effects of using 'body enhancing' filters on people from the well-being angle. I found an external researcher/clinical psychologist who specializes in body image issues who'd be able to summarize the literature and put forth a perspective on this, completed last week of the October. I wanted to check in with you to see if the proposed summary/request matches what you're after and if the timing works. If all looks good, I'll get this kicked off asap. Request: Summarize relevant social science literature to understand possible positive, negative, or neutral effects of using (or having the option to use) photo manipulation techniques that alter the look or shape of the body. Understand who is most likely to engage in body-altering photo techniques. Also understand who is most likely to be negatively impacted by using body-altering photo techniques and viewing body-altered photos, and what the impacts are." Jennifer Guadagno: "Attached is the literature review on 'Consequences and Implications of Selfie Manipulation on Well-Being' as it relates to well-being and mental health. It was researched and written by a researcher and licensed psychologist at Duke who specializes in eating disorders and body image issues among adolescents and adults." Margaret Stewart: "Thanks so much for this report, ████. Given that there is selfie filter work happening in a few areas (FB Camera, Messenger) I think it would be good for us to be consistent in our policies around so-called beautification filters... I also think it's important that we study the report that Jennifer has gathered to understand that even though people may reporting wanting and liking these effects, there still may be negative affects on people in the long run, and we need to weigh that carefully. This is especially true with the youth/teen where we see s growing body of research that social media may be driving significant increases in rates of anxiety and depression, esp among young women. See this recent short summary of the research by Jonathan Haidt: https://twitter.com/JonHaidt/status/1057231466076258304" "I'd love for us to maintain a principled stance on avoiding default filters that change the way people look 'in real life,' avoid skin lightening (though it's OK to lighten the entire image for artistic effect), avoid anything that distorts the face shape akin to plastic surgery unless it is a part of a 'fantasy filter' affect, etc." "It sounds like some of the things that Dantley's team are considering are a fairly nuanced tweak to our current policies (per some details he shared on another thread). I haven't see the latest on the Messenger filters but would be great to see a demo to see how far they diverge from the current guidelines." "This is a hard issue to navigate because I know there is a lot of competitive pressure and a lot of market demand for filters that go much more directly into the beautification space. And if we test any of these things, they will undoubtedly perform well. But just because people like and want something in the short term doesn't mean it's healthy for them." "I hope we can think carefully about this and maintain a more principled stance so we can not only avoid the bad press that could arise from launching more explicit beautification filters, but also feel proud that we are doing the right thing here. And if these new filters are not a significant divergence from our guidelines I apologies for the drama!" "Would be great to gather the right group of people to discuss, review the filters live, and see if we can agree on the right tweaks to the guidelines if needed."

Youth Targeting

Internal documentMessagesMarket Strategy Updates - 10/5/18

"We should acquire teens as young as possible"

[Highlights]: 2. "Ecosystems - US User Lifetime Value (Phase 1): Based on a conservative set of assumptions, we find the lifetime value (LTV) of users in the United States to be between $200 and $350. People who join FB as 13 year-olds have an LTV of approximately $350, higher than all other teen ages (13-18), due to their very high long term retention. We should acquire teens as young as possible. Phase 2 of the work will explore how the assumptions may change.." "Long term retention is highest at 40%+ for age 13 and then drops to about 15% in the 20s and then rises to roughly 30% in the 50s"

Knowledge

Internal documentReport"Addiction"/Problematic Use Design Audit Plan

"These may lead to feelings of being manipulated, feeling lack of control related to certain behaviors, and feelings of dependence on checking or going on Facebook that could be related to lower well-being - and could be fueling the subjective, colloquial experiences of feeling 'addicted' to Facebook."

Addiction & problematic useAutoplayNotifications

"We hypothesize that unpredictable, low value rewards and cues that promote frequent, automatic, undesired behaviors are contributing to reward learning and habit formation in a way that is not aligned with users' intentions or preferences." "These may lead to feelings of being manipulated, feeling lack of control related to certain behaviors, and feelings of dependence on checking or going on Facebook that could be related to lower well-being - and could be fueling the subjective, colloquial experiences of feeling 'addicted' to Facebook." "Partner with design to refine a definition of what aspects of features promote 'addictive'-like/problematic behaviors to target in the audit and how they're different from/related to dark patterns. Specifically, features are problematic if they promote: • Unpredictable, low-value rewards (e.g., certain notifications) • Low-control or automatic behaviors (e.g., autoplay) • Frequent behaviors that are pressured or unwanted (e.g., read receipts) • Define other criteria" "Q: How is an 'addictive'-like design element similar vs. different from a 'dark pattern'? A: 'Dark pattern' is a UX term referring to interactions that are deceptive, or that trick you into doing something you didn't want to do. These include misleading or manipulative features or settings that are designed to encourage a particular behavior that is not necessarily in the user's best interests. These can include single-instance behaviors." "'Addictive'-like design elements are those that promote repeated or poorly regulated behaviors that may have a negative impact on the individual over time as they continue behaviors that aren't in their best interests." "A dark pattern deviates from user expectations; an 'addictive'-like design results in a behavior pattern that is not wanted or valuable for the user over time (the function of the design is to promote repeated maladaptive or low-control behaviors)."

Knowledge

Internal documentMessagesInstagram Insights

████: "Teens weaponize IG features to torment each other"

Bullying & harassment

"# INSTAGRAM CONFLICTS Greetings! Here are the results from our interview study (N=24) of the what, who, how, and impacts of conflicts on Instagram." "**TL;DR** * *Conflicts* are distressing interactions between users about their perceived differences. * Most IG conflicts are between real-world connections. Current classifiers and policies do not address many of these conflicts. * Most IG conflicts use comments. * Teens weaponize IG features to torment each other, often without violating standards. * IG conflicts are common and painful; most participants regret engaging in conflicts." "**Opportunities include:** * Encourage kind interactions and help users accept their differences. * Help (ex-) friends, family, and romantic partners better manage their online conflicts. * Highlight current comment controls; develop new comment controls. * Discourage users from tormenting each other; help teens manage subtle attacks. * Give 'frontal lobe assists' to help users exercise self-control and avoid conflicts."

Profits over Safety

Internal documentPresentation2019 Marketing Budget

"IGTV - Get teens to view more content on IGTV (KPI: IGTV PR)"

Direct Messaging

"3 KEY OBJECTIVES TO SUPPORT INSTAGRAM'S MISSION... Be teens' favorite place to share everyday moments and get closer to the people and things they love. $127.3M Total ($89.8M Opex, $37.5M HA, +89% y/y)" "Objective 1 Establish Instagram as the favorite place for teens to freely express themselves with friends IG Stories - Get teens to share more everyday moments (KPI: Stories producer PR) IG Direct - Get teens to message their friends more everyday (KPI: Direct Sender PR)... $83.0M (65% of total) $60M Opex; $23.0M HA" "Objective 2 Position new bets for success (IGTV, Shopping, Direct app) IGTV - Get teens to view more content on IGTV (KPI: IGTV PR)... $29.3M (23% of total) $20.3M Opex; $9.0M HA" "Objective 3 Reinforce that Instagram is the safest and kindest online platform Well-being - Get teens, parents, educators, and policy makers to believe Instagram is a safe and kind platform through well-being features and programs (KPI: stat sig lift in 2 sentiments - safe place to express myself and encourages a kind community)... $15.0M (12% of total) $9.5M Opex; $5.5M HA"

November 2018

Knowledge

Internal documentReportIG IIC Prevalence Analysis - a/c priv

"We estimate that there are 3.5M profiles conducting Inappropriate Interactions with Children... among all IG DM Sender MAP who sent a message to a minor"

Child sexual exploitationDirect Messaging

"We estimate that there are 3.5M profiles [95% CIs: 2.9M to 4.1M] conducting Inappropriate Interactions with Children (IIC) among all IG DM Sender MAP who sent a message to a minor (a population of 320M). In percent terms, this is 1.1%." [Assumptions and caveats]: "'Potential Groomers' Review. There are a large amount of accounts marked as 'potential groomers' (18M [16.6M to 19.5M]). In these cases, the reviewer was either unsure of the profile's user age, or there was ambiguity in the conversation. We will be re-submitting the 'potential groomers' to a double review to be extra sure on this number, and it is possible that the IIC numbers could increase with additional review. We have already conducted double review on accounts marked as IIC." "Countries we didn't review. We were only able to review profiles in 22 markets due to labelling resources. This group represents 73% of this population (233M/320M). The above 3.5M number is based on assuming that the countries we did not review have the same rate of IIC activity as the 22 countries we did review. If we do not make this assumption and restrict to the 22 countries we reviewed, our estimate is 2.55M [2.1M to 3.0M] IIC profiles (1.1% of this group)." • "3.5M groomers (1.1%) [95% CIs: 2.9M to 4.1M], • 2.7M missing label (0.85%) [2.2M to 3.3M], • 18M potential groomers (5.6%) [16.6M to 19.5M], • 104M minor age liars (32.6%) [101M to 107M], • 192M non-groomers (59.9%) [189M to 195M]"

Knowledge

Internal documentPresentationSocial Comparison on Instagram

"51% of people experience social comparison on IG"

Body image & eating disorders

"Social comparison is common on IG. 51% of people experience social comparison on IG. They either compare their accomplishments to others or observe other people to decide how they should act 'sometimes' or more often." "Women and teens are more prone to social comparison, especially negative social comparison." "Negative social comparison is triggered mostly by photo/video, in particular regarding beauty, fitness, fashion, travel, and romantic relationships, posted by weak ties such as celebrities, strangers, and acquaintances." "Negative social comparison lowers well-being (loneliness, life satisfaction, self worth, and self efficacy), and people with lower well-being may be more prone to negative social comparison." "Most people (38%) report that the upsetting feeling caused by negative social comparison only lasted for 'less than a few minutes' when it happened last time, however the cumulative duration is substantial: 33% of people have been feeling worse about themselves on IG for 'several months to a year'." "TEEN GIRLS AND YOUNG WOMEN ARE PARTICULARLY PRONE TO NEGATIVE SOCIAL COMPARISONS" "Compared to 25+ years old men, the odds for teen girls to do negative SC are 8x; for young women 5x, controlling for tenure, engagement level, no. of follow(er)s, and country."

Knowledge

Internal documentReportSSI Desk Research

"We don't have a clear understanding about who reaches out for help on Instagram, the effects of posting (or not posting) on subsequent real world behavior, or the effects of seeing SSI content on viewers. We also don't understand the barriers that people face when seeking or receiving help from others on IG."

Suicide & self-injuryHashtags

"From 7/6/18 - 8/6/18: ~16K unique IG users admitted suicidal thoughts or intent and ~21K unique users admitted self-injury. Of these, between 45% and 60% are teens, even though only 18% of DAP are teens. On a daily basis, 5.1 million users, or 0.75% of DAP, are exposed to SSI-related content each day." "Teens keep one step ahead of IG's community guidelines against glorifying self-injury by changing hashtags as they are shut down" "Seeing SSI admissions on Instagram is significantly associated with increased time spent as well as posting SSI content. However, it is unclear what the causal mechanism is; those pre-disposed to self-harm could be actively seeking out SSI content" "Extreme communities online (those that promote and support beliefs generally unaccepted by society, such as eating disorders, self-harm, or suicide) provide a false sense of belonging but promote injurious behavior." "We don't have a clear understanding about who reaches out for help on Instagram, the effects of posting (or not posting) on subsequent real world behavior, or the effects of seeing SSI content on viewers. We also don't understand the barriers that people face when seeking or receiving help from others on IG." "Contagion and triggering are serious risks, although academics used to be confused about suicide contagion. That being said, we don't know how to identify users who are most vulnerable to SSI content or key intervention points."

Sources include internal documents, public filings and press reports. “Alleged by plaintiffs” denotes the plaintiffs’ characterization, not Meta’s own words. Source labels identify provenance, not a finding of fact.