Meta & Children's Online SafetyChild Safety on IG: Actor behaviors and opportunities
Raw text
Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 1 of 47 AMENDED Exhibit 210 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 2 of 47 Well-Being Child Safety on IG Actor behaviors and opportunities HIGHLY CONFIDENTIAL (COMPETITOR) Hitomi Hayashi-Branson Product Research hitomih@instagram.com Daniel Alpert Data Scientist dalp@fb.com Exhibit 33 1.30.25 mlr ΜΕΤΑ3047MDL-031-00192305 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 3 of 47 GOAL To summarize and present early data explorations on bad actors who seek to interact inappropriately with or harm minors on Instagram. TRIGGER WARNING - This deck includes images that illustrate bad actors behaviors. These include (but are not limited to) IIC, CEI, and MS. HIGHLY CONFIDENTIAL (COMPETITOR) 2 ΜΕΤΑ3047MDL-031-00192306 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 4 of 47 CONTEXT A recent exploration into IIC on IGDM, revealed that there are ~38x as many victims on IG as on FB. This is mostly due to there being many more minors on IG messaging with adults. Despite this, CEI deletes on IG are "21x lower and IIC auto-enforcing on IG is 90x lower compared to on FB. A unique challenge that Instagram faces is understanding new and unique surfaces (e.g., hashtag pages) that are likely vectors of abuse which we have yet to explore deeply. IIC Tier 2 Victim Prevalence (US English) - Facebook Victim Count 3.98K -19.37% d/d -23.22% w/w No data m/m 5.5 K 5 K 4.5 K 4K IIC Tier 2 Victim Prevalence (US English) - Instagram As of 2020-08-17 As of 2020-08-17 Victim Count 28. Jul 30. Jul 1. Aug 3. Aug 5. Aug 7. Aug 9. Aug 11. Aug 13. Aug 15. Aug 17. Aug 152.11k -10.48% d/d -5.27% w/w No data m/m 250 K 200 K 150 K 100 K 28. Jul 30. jul 1. Aug 3. Aug 5. Aug 7. Aug 9. Aug 11. Aug 13. Aug 15. Aug 17. Aug Announcing IIC Tier 2 Prevalence Measurement on IG Direct: https://fb.workplace.com/groups/135709743752034/permalink/624917758164561/ HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192307 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 5 of 47 TL;DR 1 Actors' intent range from malicious to non-malicious. Malicious intent includes those motivated by underlying sexual interest in children, those would take advantage of situations the engage in CEI, and those that are commercially driven. 2 Likely targets are minors, women, teen accounts, and vulnerable populations. Women appear to be more highly targeted by actors. Vulnerable populations that may be at risk include: LGBTQ youth (particularly young boys), and users who may struggle with mental health issues. 3 Passive/Low effort behaviors include: Seeking/discovering, viewing, and collecting inappropriate content, stalking targets, commenting inappropriately, and networking with other actors. 4 Active/High effort behaviors include: Contacting and developing relationship with targets, creating/ posting and distributing content, and scamming targets. 5 We are only at the beginning of our journey. We have a long road ahead of understanding, sizing, detecting, filling gaps in policy, developing solutions, and ensuring consistent and appropriate enforcement. HIGHLY CONFIDENTIAL (COMPETITOR) 4 ΜΕΤΑ3047MDL-031-00192308 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 6 of 47 DATA ON THE SIZE OF THE PROBLEM AND GAPS 38X MORE 21X FEWER 90X FEWER victims on IG compared to FB. CEI deletes compared to FB. IIC auto-enforcing compared to FB. HIGHLY CONFIDENTIAL (COMPETITOR) 5 ΜΕΤΑ3047MDL-031-00192309 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 7 of 47 993K DATA ON ACTOR BEHAVIORS actors had >=5 impressions on content flagged for CES over a 14 day period. 133K had >=10 impressions and 2.6K had >=50 impressions. 10.8K users following CEl-related hashtags identified by CS investigators. Examples include: #gaykid, #sexykid, #13yearoldboy, #gaykidz. We've also identified 40 additional hashtags that appeared on at least 10 pieces of CEI content in the last week. 55 actors left >=5 comments across 10.5K CES posts over the past 7 days. The most comments left by an actor was 67 comments. HIGHLY CONFIDENTIAL (COMPETITOR) 6 ΜΕΤΑ3047MDL-031-00192310 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 8 of 47 2.2K DATA ON ACTOR BEHAVIORS actors removed for IIC in last 14d. These accounts follow 375K teen accounts on IG.* Note: This may be double counting teens who are followed by multiple violating accounts. 27.6K threads between actors, ultimately flagged for CEI, MS, IIC, and a teen over a 7 day period. 3.9K threads had over 20 messages over 28 days. ~500 actors (flagged over a 7 period), who had threads with at least 10 teens over the last month. ~200 actors had >20 message threads with at least 5 teens over the last month. 5-7.5K pieces of CEl actioned content produced daily by 3-5K actors. 1-2.5K pieces of MS actioned content produced daily by 800-2K actors. HIGHLY CONFIDENTIAL (COMPETITOR) 7 ΜΕΤΑ3047MDL-031-00192311 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 9 of 47 OPPORTUNITIES Note: Opportunities in green are those I believe would be primarily owned by the IG Child Safety team. 1 2 3 4 Understand and size the problem - Explore actors (e.g., their intent, strategies), targets (e.g., who is at risk, how do they respond), interactions (e.g., how do they start, evolve, end), and difference across groups (e.g., countries). Size the problem by key behaviors and interactions. Understand risk across IG surfaces to enable surface-level interventions - Explore how actors target and interact with minors across different surfaces. Pay particular attention to high risk and ephemeral surfaces Live, Reels, Story. Develop ways to prevent actors across surfaces, implement strategies to enforce blockers and penalties. Also seek to understand virality on IG (e.g., on Reels) and how this may increase risk. Identify likely bad actors - Explore ways to improve our ability to identify bad actors on IG across different surfaces (e.g., improving age prediction, understanding more nuanced behavioral signals). [Primarily CI] Improve detection - Improve detection based on our understanding of the problem and actors on IG. Explore how we can detect less overt or disguised abuses (e.g., text overlaid on Stories image). Continue to improve detection as our understanding grows and as actors evolve to continue evading our detection system. Given E2E encryption, explore non-content based signals that we can use to define and detect violations. [Primarily CI] Gaps in Child Safety Detection and Enforcement: https://docs.google.com/document/d/17EN8Qps-GAogmuUOB3LGr0Kpj49UKFxLfvmxDFnYjuM/edit HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192312 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 10 of 47 5 6 7 8 Assess where we are (compared to Facebook) - Assess where we are for measurement, detection, reporting, safeguards, enforcement, and product solutions across surfaces. For all, explore the applicability of existing solutions and adopt as appropriate or develop new solutions. Develop product solutions based on a deeper understanding of this problem on Instagram. Identify and fill limitations and gaps in current solutions. Draw insights and solutions from FB as appropriate. For current and proposed product solutions, refer to "Child Safety Landscape" (cited below). Ensure correct actions for new/niche surfaces - Challenges that Instagram faces are largely due to new surfaces that we have less experience with. E.g., Despite efforts to calibrate FB classifiers to IG, CEI deletes on IG are ~21x lower and IIC auto-enforcing on IG is 90x lower compared to on FB. Ensure consistency in actions we take across violating content and across platforms. In a summary of Gaps in Child Safety Detection, investigator found inconsistencies in enforcement and escalations. Additionally, enforcement of viral content in Stories was found to be inconsistent across platforms. [Primarily CI/CO] Child safety landscape - Instagram: https://docs.google.com/document/d/1qxutBnVxxt ggf4rwPQ1EQcrlLWPrWuc-YvI5yMt1qg/edit Gaps in Child Safety Detection and Enforcement: https://docs.google.com/document/d/17EN8Qps-GAogmuUOB3LGr0Kpj49UKFxLfvmxDFnYjuM/edit All Roads Lead to Improving Age Prediction: https://fb.workplace.com/groups/1411637962558841/permalink/1429502370772400/ Protecting youth from bad actors on Instagram: https://fb.workplace.com/permalink.php?story fbid=638544290380093&id=100026735829495 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192313 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 11 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192314 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 12 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192315 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 13 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192316 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 14 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192317 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 15 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192318 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 16 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192319 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 17 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192320 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 18 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192321 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 19 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192322 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 20 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192323 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 21 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192324 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 22 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192325 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 23 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192326 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 24 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192327 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 25 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192328 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 26 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192329 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 27 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192330 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 28 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192331 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 29 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192332 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 30 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192333 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 31 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192334 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 32 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192335 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 33 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192336 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 34 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192337 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 35 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192338 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 36 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192339 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 37 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192340 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 38 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192341 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 39 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192342 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 40 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192343 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 41 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192344 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 42 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192346 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 43 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192347 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 44 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192348 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 45 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-031-00192349 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 46 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192350 Case 4:22-md-03047-YGR Document 2757-15 Filed 02/20/26 Page 47 of 47 HIGHLY CONFIDENTIAL (COMPETITOR) ΜΕΤΑ3047MDL-031-00192351