Email: [Need Approval by 10/22] Take a Break Content & Test Details

Raw text

Transcribed text as recorded, including headers and formatting fragments.

Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Page 1 of 8 AMENDED Exhibit 265 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation Message From: Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Pavni Diwanji [/O=THEFACEBOOK/OU=EXCHANGE ADMINISTRATIVE GROUP (FYDIBOHF23SPDLT)/CN=RECIPIENTS/CN 10/22/2021 7:10:53 PM Adam Mosseri Miki Rothschild Re: [Need Approval by 10/22] Take a Break Content & Test Details Sent: To: Subject: Attachments: image001.png Looks good to me. Pavni From: Adam Mosseri Sent: Friday, October 22, 2021 9:48 AM To: Miki Rothschild ; Pavni Diwanji Subject: Re: [Need Approval by 10/22] Take a Break Content & Test Details Totally Pavni's call. From: Miki Rothschild Date: Thursday, October 21, 2021 at 9:37 PM To: Adam Mosseri , Pavni Diwanji Subject: Fwd: [Need Approval by 10/22] Take a Break Content & Test Details This seems fine to me but want to check with you two before approving. Sent from my iPhone Begin forwarded message: From: Date: October 21, 2021 at 10:51:59 AM PDT To: Miki Rothschild Cc: IG_Well-being_Leads IGWB Support Leads 1 Shruti Bhutada 1 , Justin Cheng 1 Liza Crenshaw Page 2 of 8 Shilpa Mody Moira Burke Dayna Geldwert I Subject: [Need Approval by 10/22] Take a Break Content & Test Details Hi Miki and WB Leads, We have incorporated the feedback from IG Leads for Take a Break in preparation of the public experimentation (target: 11/1). We are writing to share and get approval on (1) final content & layout and (2) final test details. Please let us know if you are aligned or if there are any HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00171168 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Page 3 of 8 concerns by 10/22 so the team has time to iterate and complete QA before the test starts. TL;DR Content & Design: Based on the feedback, we have replaced the breathing video with a full screen takeover that encourages the user to close IG and shares 4 lightweight tips for taking breaks off the app. The proposed layout has been reviewed and approved by IGDS. Experts and teen ambassadors also shared feedback that they found the nudge and tips helpful. Test Details: We are testing Take a Break in 6 EN countries (UK/IE after IPDC approval), starting with iOS first and followed by Android in ~3 weeks. We are requesting 3% exposure for 2 trigger variants 10 min and 20 min app session lengths before showing the upsell in Explore chaining surface. Across all teens, max exposure to treatment is 0.34%. Please see details on rationale, experience, and test plan below. Product-Specific Goals The primary objective of Take a Break is to provide a moment of interruption that prompts teens to close Instagram and take a break at intervals of their choosing. Our tests goals are to (1) see how the upsell nudge helps drive awareness of the "break reminder" control (2) see meaningful usage over funnel from upsell to control opt-in to actually taking the break (3) balance ecosystem effects (detailed launch criteria shared and aligned over email yesterday) by evaluating the two trigger variants. Product Design Details We are pursuing the "full screen take over with tips" approach agreed upon late last week (Figma board & mocks below). Note: Content and designs reflect our final recommendation except the first icon in the list (we HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00171169 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Page 4 of 8 are waiting to get a new icon from IGDS for "Take a few deep breaths"). Break Reminder Experience 9:41 Instagram ♡ Sunday sunshine #weekendvibes all & corenents Liked by and others Scrolling 9:41 Time for a break? You've set reminders for every 10 minutes Tale a moment to reset by closing Instagram Take a few deep breaths Write down what you're thinking Listen to your favorite song Do something on your to-do list Doce Edit reminde Break reminder 9:41 Instagram Sunday sunshine. #weekendvibes 5 comments 0:41 Time for a break? Reminder to take breaks We'll remind you to take a breed this amount of time af coceftagram u can control how often you gut des from Your Activity 30 minutes 20 minutes 10 minutes 5 minutes You can adjust your break reminders anytime. Edit Off Post break toast: Only shown the first time you exit a break Edit Reminder Done Experiment Details Experience: When teens have spent a relatively long time on the app and start chaining in Explore, encourage them to set up break reminders for regular intervals. If they set up the control, remind them to take a break after they have spent X amount of time (selected by them) in a session. Variants: 2 total 10- and 20-minute app sessions (nudge will appear on Explore chaining). As a reminder, here is the trigger decision slide aligned on with IG leads. Test dates: 11/1 is the earliest to enable public access. We will be at full 3% on 11/9, and test will run for 3 weeks (over Nov code freeze). Prelim data will be reviewed on 11/11; experiment review and launch decision will be made on 11/30. Launch date: Team is aiming for launch in December during Production Yellow Zone. We will need Org leads and Explore leads approval to launch between 12/1 - 12/17; tentative launch date is 12/7. Test Targeting Targeting Countries Overall test exposure % Max % of teens expected to see experience Take a Break All teens (based on stated age) US, CA, GB, NZ, AU, IE (Waiting for IPDC approval) 3% 0.34% HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00171170 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 For context, 3% exposure is required to: Page 5 of 8 1. Identify most effective variants based on stat-sig differences in adoption rates 2. Have the option of analyzing adoption by gender or other core dimensions 3. Include a buffer for any over estimations given cool downs or forecast assumptions. Test Measurement (Same as what was shared in the dedicated email review thread for success criteria) Launch Metrics: (1) Upsell adoption rate (% of users that set and kept a reminder) >= 0.5% (2) Upsell dismissal rate (% of users that dismiss upsell and never set a reminder) <= 20% Guardrail/Counter Metrics: (1) Time spent: regressions < 0.2% (2) Sessions: regressions < 0.2% (3) Creator watch time: regressions < 0.5% Secondary/Understands Metrics: (1) Break reminder CTR (2) Break reminder dismissal rate (3) Take a break exercise completion rate Please let us know if there are any questions/concerns. Thanks, MWB Team From: Date: Wednesday, October 20, 2021 at 9:28 PM To: Miki Rothschild Cc: IGWB Support Leads IG_Well-being_Leads Shruti Bhutada 1 Moira Burke Dayna Geldwert Liza Crenshaw Subject: Re: [Reminder; Need approval by 10/22] [Email Review] MWB H2 Product Success Criteria Hi Miki, Correct! We will cover our plan and progress on estimating improvements in user reported feelings of NAC in the measurement review on 11/4 (next step #3). Best, From: Miki Rothschild Date: Wednesday, October 20, 2021 at 8:57 PM To: Cc: IGWB Support Leads HIGHLY CONFIDENTIAL (COMPETITOR) , IG_Well-being_Leads Shruti Bhutada Shilpa Mody 1 Justin Cheng Moira Burke META3047MDL_RVOL003 META3047MDL-003-00171171 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Page 6 of 8 Dayna Geldwert Liza Crenshaw Subject: Re: [Reminder; Need approval by 10/22] [Email Review] MWB H2 Product Success Criteria Hey, this seems reasonable to me. I think what we're saying is that we want to guard against any ecosystem/engagement metrics that imply we're hurting the user experience and that we'll rely on research to indicate the upside, but not hold ourselves to measurable upside (improvements in user reported feeling of NAC) since we don't yet have a great way to measure that in a test setting. Is that right? Ideally over time we'd want to have a way to measure the Well-being improvement but I recognize it's complex operationally. From: Date: Wednesday, October 20, 2021 at 4:31 PM To: IG_Well-being_Leads Cc: IGWB Support Leads Justin Cheng Shruti Bhutada Shilpa Mody 1 Moira Burke Dayna Geldwert Liza Crenshaw Subject: [Reminder; Need approval by 10/22] [Email Review] MWB H2 Product Success Criteria Hi Miki, Gentle reminder to review our success criteria proposal for the two PO tests for this half: Alt Topic Nudge and Take a Break Upsell (please see email below). For context, once we have the approval from you, our next steps are: 1. Email review with IG Leads on the success criteria (EΤΑ: 10/26). 2. Align with surface teams on the success criteria defined in terms of their critical guardrails and follow up with test results (ETA: 11/8). 3. Review the measurement plan for NAC overall (ETA: 11/4). Please let us know if there are any questions or concerns. Thanks, MWB Team From: Date: Friday, October 15, 2021 at 5:44 PM To: IG_Well-being_Leads Cc: IGWB Support Leads Shruti Bhutada HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 Shilpa Mody META3047MDL-003-00171172 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Dayna Geldwert Justin Cheng Liza Crenshaw Page 7 of 8 Moira Burke Subject: [Email Review] MWB H2 Product Success Criteria Hi Miki and WB Leads, We are writing to share and align on our success criteria for the two PO tests for this half: Alt Topic Nudge and Take a Break. With both of these, we're primarily focusing on the nudge / upsell experiences that are higher reach (0.51% of teens for Alt Topic and 0.34% for Take a Break), since this is where we get the most test signal and where the bar is highest. Reminder of the team mission, which motivates why we're doing these H2 tests: "Help teens become self-aware to take control over experiences that make them feel bad about themselves, and reduce these experiences on the platform overall." Specifically the product impact we're looking for here is behavioral signal for awareness and control. The launch criteria are: 1. See that we're helping people. At this early stage, we're simply looking for signals that these products are valuable and are built on strategies that are either evidence based or grounded in user research. We're looking to hit a bar, not maximize this. 2. See meaningful usage over funnel. Given that those who need solutions the most are a pretty small group, we're looking for usage comparable to other well-being experiences with upsells and don't want to see upsell dismiss rates that are much higher than benchmarks. 3. Balance ecosystem effects. The balance between the known upside of the solution and changes to ecosystem usage metrics. We'd expect some level of effect if there's material usage. 4. Complements external narrative. This is a judgement call that the product and business workstreams for proactive public comms complement each other in a clearly explainable way. We recommend prioritizing our analysis of experiment data on #3. This is where we have multiple test variants so that we can see changes in effects as the upsell/nudges triggers are broader vs. narrower (reminder: in all cases the triggers are general for all teens). For #1, we've already seen: User research suggests that interruptions could help with pulling users out of downward emotional spirals, especially when they are unable to recognize how the content they see is making them feel or unable to mitigate NAC experiences. For #2, benchmarks are: Nudge / upsell CTR >= 0.5% Nudge / upsell dismiss rate <= 20% (Benchmarks are based on comparing with Update Following, Daisy controls, and voter registration.) For #3, we will: Disregard variants with material ecosystem regressions: Time spent: regressions < 0.2% Ο Sessions: regressions < 0.2% HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00171173 Case 4:22-md-03047-YGR Document 2889-45 Filed 03/27/26 Ο Creator watch time: regressions < 0.5% Balance ecosystem effects: Page 8 of 8 Ο If there are multiple variants at this point, then our choice of the best variant prioritizes trading off big reductions in ecosystem regressions for small reductions in adoption. Since #3 is the main analysis focus, we'll separately review a more detailed analysis plan with the related surface teams (pre-reorg, these were in Discovery and F&R). At this time, we don't expect to use surveys or Negative Appearance Comparison-specific metrics in the launch criteria. We'll follow up with a deeper review of Negative Appearance Comparison-specific metrics so that we can move towards using more specific impact metrics for 2022 launch decisions. The Alt Topic Nudge test is starting this week and we'll have initial analysis in late October. For Take a Break, the timing for analysis depends on the test start date, which per the WB review yesterday depends on deciding on an experience option. Thanks, MWB Team HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00171174