Evidence timeline

9 Mar 2021Internal documentReportHow much topics people see are linked to appearance comparison on IG

"People who see more content about western pop stars like Selena Gomez, Miley Cyrus, Ariana Grande, and Cardi B feel worse about their appearances."

Body image & eating disordersContent Algorithm

"Some topics are likely to make viewers, especially teen girls, feel worse about their bodies: Fashion and beauty (especially makeup, skincare, hair removal, nail art); Relationships (which includes dating, romance, sex, and intimacy); Western pop stars who emphasize their bodies (e.g., Selena Gomez, Cardi B, Ariana Grande); Images that emphasize women's bodies generally (e.g., cleavage, swimwear). Collectively, these topics comprise 1/4 of what people see on Instagram (1/3 for teen girls)." "People who see more content about western pop artists like Selena Gomez, Miley Cyrus, Ariana Grande, and Cardi B feel worse about their appearances. These musicians regularly focus on their bodies in their posts... though the list is mostly female celebrities, Justin Bieber and the wrestler John Cena are also associated with greater appearance comparison." "We should reduce exposure to celebrity content about fashion, beauty, and relationships, while increasing exposure to close friend content of all kinds. Potential product directions include: ... Not recommending celebrities to follow that post primarily fashion and beauty content. People can find these accounts on their own, but we shouldn't amplify their influence through recommendations." "Only 13% of the content that people see comes from friends (15% for teen girls), a median of 31 pieces of friend content per day (46 pieces of friend content for teen girls). Thus, for every piece of friend content a teen girl sees, she sees 5x as many pieces of content from top accounts. We should make it easier for teen girls to find and consume content from their friends, especially close friends." "Seeing more friends post about these topics is not associated with worse levels of appearance comparison on Instagram... The problems associated with content about relationships (e.g., dating, coupledom, sex) and fashion (including beauty) are coming from non-friends." "In a recent listening session, one creator described Explore as 'a landmine for everything I want to avoid on IG' (@kendramorous) because it triggers appearance comparison." "[T]een girls report feeling worse when friends mimic celebrity beauty standards."

11 Mar 2021Internal documentMessagessuper high risk, last time it was launched #killchallenge went viral

The keyword challenge hashtag is "super high risk" and caused "#killchallenge" to go viral

Hashtags

"hey do you have context on hashtag challenges?… super high risk, last time it was launched #killchallenge went viral. recipe for e.g. tide pod challenge disaster"

11 Mar 2021Internal documentMessagessuper high risk, last time it was launched #killchallenge went viral

We know we need to constantly re-win this segment as there is a new group of 13-year-olds every year

"Youth and Teens are critically important to Instagram. While Instagram has strong market-fit with Teens, we know we need to constantly "re-win" this segment as there's a new group of 13-year-olds every year and the competition over their Social Media engagement has never been more fierce. We have an opportunity to responsibly introduce young people to Instagram. To that end, we're building a new pillar within Instagram focused on Youth. Our goal is to ensure that Instagram continues to have a strong fit with teens by offering as much of Instagram's value to tweens as possible. ** Tweens are teen-aspirational - they want access to the same Instagram that teens get. Parents want a safe space for Tweens to grow and experiment free from any potential harm, e.g. online bullying. Our product will build at the intersection by providing access to Instagram for tween by requiring parental content, transparency and controls."

23 Mar 2021Internal documentMessagesno way for users to report U-13 users on Facebook in app

"For reporting, currently, there is no way for users to report U–13 users on Facebook in app and takes over 7 screens for users to report U-13 users on IG."

"For reporting, currently, there is no way for users to report U–13 users on Facebook in app and takes over 7 screens for users to report U-13 users on IG."

23 Mar 2021Internal documentMessagesno way for users to report U-13 users on Facebook in app

"On enforcement, we currently have a backlog of over 2.5 million cases age-related reports to review."

"On enforcement, we currently have a backlog of over 2.5 million cases age-related reports to review."

26 Mar 2021Internal documentMessagesI felt bad for everyone involved in this next round of litigation

"Guess what, they're reviving private by default LOL" ████: "I felt bad for everyone involved in this next round of litigation"

Darius Kilstein: "Guess what, they're reviving private by default LOL" Darius Kilstein: "mygod" Darius Kilstein: "Was Private By Default (we give them the option to choose private and public and private is defaulted)" ████: "Yeah, I heard that this morning... I felt bad for everyone involved in this next round of litigation"

2 Apr 2021Internal documentMessagesi know, it's pretty dispiriting to think about... we have no idea what the long-term effects will be

"i know, it's pretty dispiriting to think about. and the fact that we have no idea what the long-term effects will be for this generation that has grown up comparing themself to something that's...totally fake"

Body image & eating disordersFilters

Elizabeth Crenshaw: "some flags for this crew: • *MIT Tech Review piece on beauty filters* -- it's not a great piece and does a lot of speculating about the long-term impacts of filters on identity + distortion of reality, esp for young girls. it's focused on the industry at large with fairly equal balance between us + snap, TT. worth a read for sure. • *Insider on Hailey Bieber + mental health* -- talks about how she takes a break when bullying gets intense, and how it helps her mental health. seems like some of our product work could align nicely with this." ████: "This makes me so sad to read. Especially knowing how hard we fought to prevent these on IG. 'But her younger sister, Sophia, who was a fifth grader at the time, disagrees. 'I definitely was—me and my friends definitely were,' she says. 'Twelve-year-old girls having access to something that makes you not look like you're 12? Like, that's the coolest thing ever. You feel so pretty.''" Elizabeth Crenshaw: "i know, it's pretty dispiriting to think about. and the fact that we have no idea what the long-term effects will be for this generation that has grown up comparing themself to something that's...totally fake" ████: "One question I think we can re-raise is the 'promotion' of these beautification filters. I know most people discover filters from stories, but to the extent we're promoting any (either in the effects gallery under the 'appearance' tab or in tray through ranking), I think it's worth re-looking at our decision and confirming with product that we're not promoting these types of filters." ████: "Agreed. I'm also a little concerned about a category being called 'appearance' in general, especially after our discussions around the Reels tool. But I know these are different use cases" ████: "I have a hard time believing that we'd do this, but it is still worth the effort IMO. Given all the continued coverage on the impact of beauty filters on youth, I'd want us to consider age-gating these effects to U18 accounts that we have (stated, hard-matched, etc.)." ████: "Especially b/c over the last year we've considered more age-gated experiences and defaults." ████: "would there ever be a world where an individual user could opt out of seeing these filters? like a specific evolution of the mute tool - 'mute appearance filters'" ████: "I would love to see a test like this and compare it to the number of uninstalls or drops in MAP or weekly actives because I imagine giving people more choice over some of the features they see might reduce the nuclear option of uninstalls or staying away for periods of time" ████: "i know there is a thin line between choice and simplicity but I feel like it could definitely be a tradeoff worth exploring for some features like that"

9 Apr 2021Internal documentMessagesGirls as young as 8 start to lose self-confidence b/c of negative self image and social comparison

"Girls as young as 8 start to lose self-confidence b/c of negative self image and social comparison."

Body image & eating disordersContent Algorithm

████: "Would we consider not surfacing certain content in our rec surfaces that we know to be triggering?" ████: "The way that we've identified borderline harmful in other categories" Vaishnavi Jayakumar: "Totally - we're working on a borderline ED policy right now to address this and are just kicking that off" ████: "The images that ████ sent yesterday of an explore feed is really problematic, and the ownership of that is also on us since it's an unconnected, recommended surface." Vaishnavi Jayakumar: "Yeah - the challenge is that when you zoom in, any one of those photos probably wouldn't be triggering. But in an explore grid with all these photos stacked against one another, it's pretty overwhelming" ████: "Girls as young as 8 start to lose self-confidence b/c of negative self image and social comparison. It's when they become aware of their bodies vs. others." Vaishnavi Jayakumar: "Yeah and we're playing the same role today in their perception of self-identity that magazines and music videos played when we were growing up" ████: "Exactly" Vaishnavi Jayakumar: "And we all know how much those visuals impacted how we saw ourselves - I wanted to have pin straight long hair and hated my curls" ████: "That's exactly right. And the problem i had with the intervention proposal is that the magazines had to take more ownership of what they showed in their mags knowing that young girls were the biggest consumers of their publication." ████: "Also, one leverages user data in a way that I want us to think through holistically. It's invasive, in many ways, to know that we're analyzing the content you consumer and recommend something else. Outside of narrowly defined harmful content streams." ████: "there are really meaty legal issues separate and apart of the policy and public health concerns. Largely by way of deceptiveness (which I think is something we should start to leverage more)"

9 Apr 2021Internal documentMessagesHonestly the only insight I see in these charts is that teens are really into using IG at 11pm when they probably should be sleeping

████: "Honestly the only insight I see in these charts is that teens are really into using IG at 11pm when they probably should be sleeping :("

Addiction & problematic use

"Honestly the only insight I see in these charts is that teens are really into using IG at 11pm when they probably should be sleeping :("

21 Apr 2021Internal documentReportAddressing viral suicide & self-injury challenges across the FOA

"We have noticed an increasing rate of SSI viral challenges on the platform" but have historically only dealt with those events once media outlets catch wind of them

Suicide & self-injury

"An SSI viral challenge is an event that promotes or encourages others to complete a challenge related to SSI, including self harming or suicide." "We have noticed an increasing rate of SSI viral challenges on the platform." "We've historically taken a reactive approach to dealing with viral events (i.e. we deal with it once it's become so viral that media outlets catch wind of it)."

21 Apr 2021Internal documentReportUnderage & Age Verification Data 1-Pager

"We identified a backlog of >2M reports and appeals to be reviewed in [U13] age-related flows, and managed to automate decisions of 600k of those so far. However, jobs are still growing faster than our review capacity, so we currently have 2.6M jobs in backlog."

"Identity Integrity spun up an Age Verification Task Force in January '21 to quickly understand our existing state of under/over age 13 verification and identify quick wins to put the company in a stronger position. This was prompted by increasing regulatory concerns and a changing competitive landscape regarding youth." "We identified a backlog of >2M reports and appeals to be reviewed in age-related flows, and managed to automate decisions of 600k of those so far. However, jobs are still growing faster than our review capacity, so we currently have 2.6M jobs in backlog." "We got $2.5M operating expense budget approved to increase CO capacity, there has not been any increase in capacity granted from ops, and it will take us some time to staff because of hiring lead time." "33% of IG users don't have stated age" "Reviewing for IG underage appeals has been stopped since COVID, which means that checkpointed IG accounts currently have no route to regain access to their account." "Current underage review protocols might not be effective." "The current U13 protocols rely on reps making a decision based on account photos, which can be difficult especially if account photos are of teens (e.g. it is hard to tell if someone is 13 or 14)" "Out of 3rd party reports that have been checkpointed, 77% of them are predicted to be over 18 by the adult ML classifier for FB and 76% for Instagram, suggesting that the reviewing process might not be an efficient or accurate way to decide if an account is U13"

23 Apr 2021Internal documentMessagesRe: Youth Family Data Circle - State of Youth Integrity

Progress on child safety issues is hampered by lack of capacity. Central Integrity has 1 Data Scientist on Child Safety.

"I'm very concerned about the lack of capacity, especially given our efforts on Youth across the company. I don't believe this is well understood by leadership and would like us to raise awareness of this, so we can advocate to staff the teams better." "Perhaps the biggest issue: progress on [child safety issues] is hampered by lack of capacity. Indeed, we didn't dwell so much on gaps on existing areas of coverage but they are substantial as well. Central Integrity has 1 [Data Scientist] on Child Safety."

23 Apr 2021Internal documentMessagesUp until last half, even Child Safety CI had no workstream set up to do bulk enforcement against FB Groups with clearly violating content in either CEI or MS

"Up until last half, even Child Safety CI had no workstream set up to do bulk enforcement against FB Groups with clearly violating content in either CEI or MS[Minor Sexualization]"

Child sexual exploitationDirect MessagingGroups

████:"[W]e're not very good at proactively detecting and taking down MS[Minor Sexualization] Groups and it's a big problem we need to tackle sooner rather than later b/c they areVERY publicly visible and leads to PR fires and stakeholder complaints and they get deployed against us as proof points that we're not taking child safety seriously" ████: "Up until last half, even Child Safety CI had no workstream set up to do bulk enforcement against FB Groups with _clearly_ violating content in either CEI or MS[Minor Sexualization] ████: "We also didn't have an implicit minor sexualization policy stood up until this half" ████: "Child safety was not one of our org's priorities, as they focus on higher prevalence issues historically like misinformation, civic/health/etc. Issues, nudity/sexual solicitation, but we are looking to get it on the radar a bit more with risk tiering" ████: "[T]he way we keep getting distinguished from the Apple's of the world is that we're connected to the social graph and a public surface, i.e. FB Groups… where bad actors can _signal_ one another… and then enter a private messaging thread… so it's one seamless flow of discovery → connection → harm" ████: "PLUS, we currently have liability risk under existing law for minor sex trafficking, sex trafficking and prostitution in general"

28 Apr 2021Internal documentMessagesif/where tweens message and if IGY should consider beyond Direct

"Hey folks do we have data or research you can point to around early teens (or tweens) trends on messaging and competition? ████: "████ did an analysis with 13-14yrs old as a tween proxy"

████: "Hey folks do we have data or research you can point to around early teens (or tweens) trends on messaging and competition? Specifically * how early teens use ig direct (or other messaging systems) - any analysis on content/JTBD there? * how tweens and early teens use snap * how tweens and early teens use avatars * for tiktok or ig, where do external private shares go? to snap, imessage?" Darius Kilstein: "Data wise we have nothing on tweens as there aren't any that use Instagram" Darius Kilstein: "We have teens as a bucket, we never looked at stated-age for those who tell us they are 13-15. So we dont have anything there and that would be net-new work." ████: "████ did an analysis with 13-14yrs old as a tween proxy" ████: "Here's the very wip deck that I'm putting together on US 13-14 year olds! I haven't dug too deep into direct yet, but slides 24 and 27-29 might be relevant!" ████: "@silent these are so great, thank you all! Very helpful to influence convos around if/where tweens message and if IGY should consider beyond Direct." ████: "@silent let's please get consistent in our data and research on the language and age brackets for youth. Aligned with ████ and ████ on this, comms went out to IG Research this week and encourage us to keep consistent with this: * Young Kids (0-5) * Kids (6-9) * Tweens (10-12) * Early Teens (13-15) * Later Teens (16- legal maturity that can vary by country)"

4 May 2021Internal documentMessageswhat ever happened to the Nielsen deprivation study?

"What ever happened with the Nielsen deprivation study?"

Alex Dow: "what ever happened with the Nielsen deprivation study? I think I lost track when covid hit. I know there was one or more pilot studies completed, but did they ever do the full thing?" Moira Burke: "There were two one-wave pilot studies and the findings have been shared with leads internally, but afaik we're not planning to totally do a longer-term study. The findings were basically that people's expectations totally drove the results, and we weren't able to use the data to disentangle those expectation effects." Alex Dow: "well, shit. that seems like a result worth publicizing on its own. Not that anyone would want to hear it from us. maybe that is a study for FORT, what is the role of expectation effects on social media deprivation studies."

7 May 2021Internal documentMessagesthe daily number of minor victims having sex talk conversations in IGDM with adults is 38x compared to FB (~200k victims/day)

"The daily number of minor victims having sex talk conversations in IGDM with adults is 38x compared to FB (~200k victims/day)."

Child sexual exploitationDirect Messaging

Pavni Diwanji: "[T]here was decision made to encrypt all private messages for u13. This has severe impact on safety as we know. I am trying to argue, perhaps foolishly, that its not the best decision for our users." Sarah ████: "Internal child safety data analysis found that the daily number of minor victims having sex talk conversations in IGDM with adults is 38x compared to FB (~200k victims/day). When normalized for the number of teens, the relative prevalence on Instagram compared to Facebook is about 5x." Vaishnavi Jayakumar: "Big fan of @Hitomi Hayashi-Branson's research on IG teen safety that just wrapped up, where she interviewed several of our expert child safety organisations around the world. One of the key findings around both comments and DMs: Children are regularly exposed to sexualising content and comments - girls in particular have come to just expect that they're see this type of content and have these experiences" "As a supporting anecdote, another IG researcher who was recruiting teens for a messaging study met a potential recruit who described a predatory grooming message she received from an older man. The potential recruit was disturbed by this but also mentioned that she regularly gets these kinds of messages, usually at least once a day."

12 May 2021Internal documentMessages[Flagging] Thorn Report on Child Safety Research Communications Plan

45% of children age 9-12 use Facebook daily and 40% use Instagram daily

Thorn Report: - 45% of children age 9-12 use Facebook daily. - 40% of children age 9-12 use Instagram daily. Pavni Diwanji: "This is a big WHOA, if these numbers are to be believed. I am curious how Thorn is generating these numbers and what is our assessment of these numbers. If we feel that numbers are in the right ballpark, its hard to justify bringing more kids onto our platform before we make it better, afford better protections. And if numbers are not in right ball park, can we make a case or refute these?"

Jun 2021Internal documentMessageswe really do recommend a LOT of high-NAC content to teens!

Shilpa Mody: "We really do recommend a LOT of high-NAC content to teens"

Body image & eating disordersContent Algorithm

"[I]t turns out that at the high end, we really do recommend a LOT of high-NAC content to teens! Like for the top 10% of teens, our recs in Explore are 57% high-NAC :/"

27 Jun 2021 – 8 Jul 2021Internal documentTableBad Experiences and Encounters Framework (BEEF) Survey: Data collection: June 27 - July 8, 2021

"Younger people report higher rates of every issue"

Hate speechDrugsChild sexual exploitationSuicide & self-injury+1

"A little over half of respondents (51.6%) experienced at least one issue in the past 7 days" "Issues are largely caused by strangers, with negative social comparison (15.5%) and bullying target (14.6%) having the highest rates of known instigators." "93.8% of unwanted sexual advances are from people the respondent doesn't know, the highest percentage of the 12 issues" "Respondents felt the least supported by Instagram when they were the target of bullying, had difficulty posting, or felt Instagram took down content when it shouldn't." "All issues cause some reduction in self-reported creation behavior, with impersonation hindering creation the most."