Antigone Davis: "In getting feedback from policy leadership w/r/t policy's integrated youth plan, Erin Egan asked about the development of company positions and 'overarching strategy' regarding the spectrum of youth issues: Age Assurance, Addiction/Problematic Use/Time Spent, 13 v 16, Bullying,... Her exact words:"
"We will build out our positions regarding related issues we are seeing crop up, such as around addiction. It would benefit us to firm up an overarching strategy across the spectrum of youth issues, especially as we consider different equities within our products and as the issues continue to evolve."
Antigone Davis: "I responded that to some degree this sounds like the work that is happening within the Centralized Youth Team, CSN, PA Marketing,... That said, I do have some sense of what she is seeking beyond the current work and why. I'd like to talk it through with this small group and see what, if anything, we should consider doing."
████: "As I understand parental gatekeeping is much larger for 13yo - should we focus more on 14-17? Do we have data that shows that if 13yo don't start using IG - it is much harder to get them excited to join IG at 14+?"
████: "████ has a slide on this I believe! TL;DR is that it's important to acquire them early on"
Darius Kilstein: "What we learnt from FB teen declines is that each incoming year is worse than the last - so to some extent, what you are saying is probably true in reality (i.e. if you dont join when you are 13 you are much less likely to do so when you are 14)"
"One flag for folks is that we have to be extra careful when conducting research to avoid conversations that highlight <13 yo with IG accounts. I'm seeing some of these questions, particularly around the impact of parent sentiment to teen usage and engagement flirting with that line. This isn't to say to not do this research but just know there will likely be a fair amount of PXFN pushback."
"Context: following a recent child safety incident (detail) an XFN team across Policy, Product, Legal and Public Affairs came together to re-evaluate an exception that allows 18-20yo to message minors (13-17yo) on Instagram. This exception stands in contradiction to our public statements and witness testimony that adults are not allowed to message minors on Instagram that don't follow them.
"Decision: The XFN were unable to align on a path forward and escalated three options to product leadership last week, with Adam attending as decision maker.
- Option 1: Status Quo, 18-20yo can continue to message minors that don't follow them
- Option 2: Remove the exception altogether, effectively bringing us into compliance with our public commitments. Minors would still be able to initiate conversations with 18+ adults. The new added requirement is that for 18-20yo to message minors, the minor is required to proactively follow the adult.
- Option 3: Keep the exception with an age gap cap of 2 years, meaning a 19yo could message a 17yo but not a 15yo, for example.
Adam supported option 3 as it allows teens in similar age ranges to continue messaging one another, supported by Product and Policy. Legal ████"
"**Public Affairs POV:** In practice, option 3 may represent significant legal and reputational challenges. It will require us to update public materials on this protection, trigger briefings to regulators like the IDPC, FTC and ICO to flag the change (therefore disclosing the gap), and may necessitate updates and corrections to prior congressional testimony.
While it's reasonable that 18yo and 17yo should be able to message one another, there are ramifications to publicly disclosing the longstanding and continued existence of this gap. Specifically, we believe that we'll likely have to confirm that 20yo have been able to message 13yo for a number of years following the launch of this protection. Furthermore, it could be perceived that a decision to formally adopt the exception (even with the age range cap) contradicts our youth well-being campaign goal to demonstrate aligned interests with parents.
Option 2 would bring us into compliance and not require us to make such disclosures or corrections. We believe we can explain the existence of the initial exception if it became a matter of public knowledge, provided it's remediated."
"IG DIOps proactively reviewed IG surfaces by assessing the search results of 16 Queries, Tags and Emojis (e.g. tweenxstars, #cutegirls and Little # &) 50+ suspicious accounts were found, and we observed patterns indicating the existence of a much larger network of similar accounts."
"Among 50+ *suspicious accounts mentioned *in the video, we observed the following patterns: Accounts shared media of scantily clad children's pictures of which some are ai-manipulated or simply stolen from other accounts Classified as not 'real' pictures but CGI-animated (eg. 17841457894122934) or in case of organically derived : they seems to be just allegedly 'stolen' from other users (eg 17841457443294988, 17841457303684351). Accounts having '3d' in their IG Username (eg. 'claire_3dwaifu', 'lola_unreal3d') indicative of 'Computer Generated Imagery' Telegram accounts were found, possibly facilitating purchases of CGI media of child-like images in sexually suggestive positions. We observed concerning patterns which might indicate the existence of a network of malicious accounts: IG Users 'evebloom3d', 'lolawaifu3d', 'allnatural_3dbabes', 'unrealcandies_cgi' sharing Child Sexual Exploitation Imagery, having 31k+ Common Followers. Total Followers of 'evebloom3d', 'lolawaifu3d', 'allnatural_3dbabes', 'unrealcandies_cgi' were 83k+. We flagged 116 IGIDs; 101 Non-Recommendable posts of Sexually Explicit and Suggestive Imagery with Insufficient Coverings and 15 posts violating Community Guidelines for Child Safety."
"We wanted to share some (rather concerning) early data insights and next steps, as we looked into recommendability and media quality of US Teens' Top recommended Reels."
"Anecdotal evidence indicates that Teens see higher levels of non-rec content. An ex-Meta employee recently shared feedback that his teenage step-daughter's Reels feed is highly offensive with content including: 'people chugging alcohol and vomiting, blatantly racist jokes, pedophilia jokes, videos of people dying in car crashes, etc.'."
"11% of US Teen's Weekly Top 200 Reels on Reels Tab/Chaining was non-recommendable.
- These 200 Reels made up 3% of US Teens' Reels consumption on Reels Tab/Chaining.
- Here's a media fiddler of some non-rec examples within the top 200 Reels for US Teen on Reels tab. To provide some examples: the top 1 Reel by VPV used the N word and depicted gun (regulated goods), the 9th Reel by VPV depicted violence and physical bullying, the 13th involved sexually suggestive positioning, and the 36th was disturbing."
"Why did we not find out how dangerous US Teen Top Recs were earlier?
- We did not pay enough attention to country variance. We have global *teen* impression level prevalence measurement (row 6), which is in fact lower than global *adult* impression level prevalence (row 7). However, from the rows 4-7 in the table above, we see that US prevalence is higher than the rest of the world, and US teen prevalence even higher than US adults."
Leah ████: "Product pivoted to recommending allowing 18-20 year olds to message unconnected minors if the age gap is <2 years after learning that blocking threads from 18-20 year olds to teens would impact 7.7% of threads to teens."
"Data shows blocking threads from 18-20 year olds to teens would capture .68M daily new potentially harmful threads. Harmful threads are based on MaCSA scores, negative feedback from minors, CSE reports including sextortion, etc."
Leah ████: "MWB has scoped 3 options, each of which has voice (i.e., new threads blocked) and safety (i.e., potentially harmful threads prevented) impacts:
• Option 1: Prevent 18+ year olds from messaging people aged 17 and below who don't follow them
◦ Blocks 2.9M new threads daily
◦ Prevents 680,000 potentially harmful threads each day
• Option 2: Allow 18 year olds to message teens who do not follow them only if age gap <= 1 years
◦ Blocks 1.9M threads daily
◦ Prevents 440,000 potentially harmful threads each day
• Option 3: Allow 18-19 year olds to message teens who do not follow them only if age gap <= 2 years
◦ Blocks 1.5M threads daily
◦ Prevents 360,000 potentially harmful threads each day"
"MWB believes that the disparate safety impact between Options 1 and 2—240,000 additional potentially harmful threads each day—can be largely mitigated in H2, but we haven't seen data on those mitigations."
"Our competitors in social connection are leveraging school integrations as a way to keep in touch with their community. Can we take this a step further and use school networks as a lever for acquisition?"
"How can we position Instagram as integral to navigating school relationships, especially during transition periods?"
"In H1'22, we proposed defaulting new teens into Take a Break based on positive feedback from teens, parents, regulators, and press."
"Why we did not pursue it
1. Leadership shared that the bar for defaults is high and should consider policy/legal requirements, experimental data showing strong adoption, and behavioral data that a majority of users would support a given product.
2. Testing without confidence in launching was considered challenging, as it is difficult to walk back what many would see as a positive step for well-being. We still face this challenge, but it's more defensible if we move forward with some level of default (ie., U16 v U18)."
"[A]wareness of opt-in control features is modest (23-34% aware of time management features)"
"97.5% of teens have not adopted Take a Break or Quiet Mode, indicating opportunity for intervention"
"Youth Industry Practices specify the need to default a break nudge (default either break nudge or daily limit plus break reminders in 2 of 3 competitors)"
"Perceived low control over Instagram use is a common problem (-20-40% of youth IG users)"
"35% of late night time spent for p90+ teens comes from sessions over over 15 minutes long."
"One of the few differences in borderline hate speech policy is that IG does not allow 'Questioning innocence of victims, such as claiming the victims are responsible or provoked the incident'"
Hate speechBullying & harassmentSuicide & self-injuryChild sexual exploitation+3
Melissa ████: "I'm not surprised we have bad hashtags. our search safety team is a few employees who look at a list of popular hashtags every week to decide which to filter out :( it's super low tech… We also do very little to make sure handles are appropriate which is so creepy and problematic… The fact that we recommended the pedo account that activist viewed once to all her followers is really disappointing"
Melissa ████: "Content moderators not enforcing the rules is such a big problem. Could be the biggest problem on right now"
Melissa ████: "It's a really big problem that you view one media and then we start recommending many like it. we gotta solve the rabbit hole issue"
Melissa ████: "We think rater recall could be as low as 50%, and the non-rec labelers only know English ... they label 0 media/week as hate speech, bullying, or suicide and self injury even though we know the prevalence is higher. and they frequently get easier topics wrong that don't require language comprehension, like disturbing content"
Tomas Kabbabe: "You'd think we JUST started working on integrity with issues like this one"
Tomas Kabbabe: "[I]t feels like we're so behind where we should be"
Melissa ████: "I mean you see the chaos of the way products are shipped at Meta… We don't do comprehensive, holistic overviews often enough… We don't hold enough people responsible for metric monitoring. we only staff teams when we want to build, not maintain… the central integrity org is pivoting to focus on "voice" aka false positives and overenforcement. Their other big new focus is "integrity value": trying to justify how integrity work adds value to the business. as a result, they've stopped investing in SWOP, which is our Meta metric for content violations!
Melissa ████: "integrity is an adversarial space. there's millions of people who will profit from finding ways to exploit it, so they're always trying. product teams change the way things work all the time and don't move in lockstep with integrity"
Melissa ████: "and there's just SO MANY TYPES OF HARM. it's hard to think of all of them for everything always. just because there's no harm on an issue type or surface today doesn't mean there won't be harm on hashtags tomorrow, so someone makes the call that it's not a priority and then boom, things evolve"
Melissa ████: "and non-rec is owned and operated by the small team on RI, so we don't benefit from Meta's best practices on making sure raters actually label the violating content as violating"
"Problematic Use Definition
1. A perceived lack of control over usage of a technology or social media platform and
2. This lack of control is perceived to lead to negative life impacts (e.g., in domains such as relationships, work/school, sleep, etc)"
"Personal life interference from app use is highest for younger users, and tracks with age"
"We measure perceived low control in two ways:
1. Perceived low attentiveness around platform use (i.e., spending more time than wanted, checking and consuming content without realizing it)
2. Perceived low capability to change platform use (i.e., low confidence in ability to reduce time, checking, or interactions)"
"Youth (<18) and adults (18+) report perceived low control (of both types) at generally similar rates: overall rates are highest for youth and young adults"
"Prioritization. I think this has improved meaningfully in the past 12-24 months. Adam has been extremely clear about our focus on teens. This, combined with a more concise set of topline metrics, and more consistency in our stated product priorities, means that we're a more focused organization and individual teams can make better decisions about what to focus on."
"We want to share a summary of learnings and suggestions for next half in this post... Given sampled teen and overall data, we verified that (1) more teen data leads to consistent teen NE wins and wins for overall NE in most cases; (2) teen data is more efficient in improving teen NE compared with non-teen data; (3) non-teen data is indispensable for model performance for teens especially given its large volume - so it is always suggested to use all non-teen data as it improves teen NE as well."
"Suggested next steps (1) we saw teen core and capped session wins for the secondary model tuning for teens with model trained on upsampled data and we are discussing launch paths to materialize the wins; (2) upsampling teens is beneficial for both teen NE and overall NE so given teen is more top priority and teen data is more efficient for teen NE we should prioritize upsampling teens."
"Given that teen data is more efficient for model performance for teens, we can prioritize upsampling the teen part and checking along the way if it breaks the balance of teen proportion in the training data. And given the usefulness of non-teen data for teen model performance, we should almost always use 'ALL' data that is available even if we want to optimize teens only."
"We have tried two approaches to up-weight teen training losses (1) direct up-weight and (2) add teen dedicated tasks... Add additional tasks for selected production tasks such that they only predict for teen data. This way, losses will be applied more for Teen data, resulting in more emphasis on teen model performance. This is essentially up-weight teen train loss without affecting calibration... This work is more promising compared to direct up-weight as (1) we see mild teen NE wins and neutral overall NE with smaller loss weight for teen dedicated tasks... this has also been launched for FB YA."
"Over the last two years, MWB has been focused on understanding and reducing problematic use for Instagram Users."
"Why does problematic use matter? Research and analysis has revealed three primary reasons for reducing problematic use:
1. Teen WB: Research has shown that problematic social media use is both prevalent among teens and is a driver of churn: 56% of IG teens surveyed say it's difficult to manage time on social media, and teens reporting life interference were 2.1x more likely to delete Instagram (source).
2. Parental Trust: ~ 1/4 parents surveyed consider total screen or late night time spent to be a top social media concern.
3. Regulatory Risks: recent state legislation (UT) prohibits U18 late night social media use without parental consent, and new federal guidance calls for more restrictive social media use policies for teens"
"Overall, Instagram had an amazing half. Sharing, messaging, engagement, growth, and revenue are all at all-time highs… The one important soft spot is teens in developed markets."
"Scientific evidence suggests that harmful content exposure as well as excessive and problematic social media use primary areas for concern."
"Teens who spend more than 3 hours/day on social media face double the risk of poor mental health outcomes"
"Among IG users, teens experience the highest prevalence (2.5x) of bullying and harassment and are more likely to experience severe forms. This is one of the top concerns reported by parents globally."
Body image & eating disordersSuicide & self-injuryHate speechBullying & harassment
"Teens have higher prevalences of violating content than non-teens (1.4x-1.8x SWOP), driven by elevated levels of bullying, SSI, hate speech, nudity and VNI."
"SSI/ED has a disproportionately large teen audience on IG with SSI/ED teen SWOP being 2.5x that of adults."
"Some user reports have not been accurately handled by Meta… The key concern had to do with reports of potential child safety issues not being addressed by Meta… we anticipate that our user report recall will continue to be challenging at the conclusion of the task force."
"Analysis of a golden set of known-violating Child Sexual Exploitation profiles/content suggested that reps have ~22% recall on Instagram profiles and ~42% recall on Instagram content, meaning of all the known violating content reviewed by reps as part of this golden set, they would have taken down only ~1 in 5 profiles and ~2 in 5 pieces of content."
Sources include internal documents, public filings and press reports. “Alleged by plaintiffs” denotes the plaintiffs’ characterization, not Meta’s own words. Source labels identify provenance, not a finding of fact.
Meta & Children's Online Safety