Long Term Retention: The Young Ones are the Best Ones

Raw text

Transcribed text as recorded, including headers and formatting fragments.

Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 1 of 16 AMENDED Exhibit 313 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 2 of 16 All Custodians Attachment Count Author 0 METAMAAG-017-01492550 Metadata Begin Family Date End Family Filename MDL BEGBATES MDL ENDBATES Primary Date (PM) METAMAAG-017-01492550 2018/11/21 METAMAAG-017-01492584 Long Term Retention: The Young Ones are the Best Ones and Other Learnings META3047MDL-047-00995641 META3047MDL-047-00995675 2018/11/21 depobook.com SEMANTIC SEMANTIC SEMANTIC SEMANTIC SEMANTIC SEMANTIC SEMANTIC SEMANTIC SEMANTIC DOC TYP E ALIAS EXHIBIT 7 CONFIDENTIAL CONFIDENTIAL Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 3 of 16 Long Term Retention: The Young Ones are the Best Ones and Other Learnings By and (cover photo below) FB Age In Days 4500 4000 3500 3000 2500 2000 1500 1000 500 0 2004 2005 2006 2007 2008 2009 Confirmed Year 2010 2011 2012 2013 2014 2015 2016 1950 2000 1950 2000 1950 2000 1950 20001950 2000 1950 2000 1950 2000 1950 2000 1950 2000 1950 2000 1950 2000 1950 2000 1950 2000 Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year Dob Year TL;DR We observe very stark differences in long term retention in the United States depending on (1) Age at which you join Facebook and (2) Year in which you join Facebook. Establishing long term retention as a standard metric for Facebook to observe and analyze across our analytics and research teams, in addition to product teams more generally, is a major objective of this work. We see that year-of-birth/age and confirmation year are important dimensions through which to understand how different people are using Facebook over time. META3047MDL-047-00995641 METAMAAG-017-01492550 CONFIDENTIAL CONFIDENTIAL Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Our three major findings and associated implications are below. Page 4 of 16 1. Tweens (approximate age 10-12) are special. People who join Facebook as tweens have the highest long term retention out of all age groups. a. Observations i. As an example, for people who joined Facebook in 2016, 12 year-olds (born in 2004) have long term retention (LTR) MAP_28 of 72% at an FB age of 365. 32 year-olds (born in 1984) have long term retention of 25% at the same FB age. This is a 3x difference for a 12 y/o versus a 32 y/o! The difference is similar in magnitude across multiple confirmation years. ii. You can also see this very clearly in the Chart 2 below. There is a green band (indicating high retention) on the right in every confirmation year. The exceptions are in the very early days of Facebook (2004 and 2005) when high schoolers did not have access to Facebook. iii. Caveat: These findings rely on the Age Affinity model, and while it is the best we have for age, there are biases in the model which may influence the results. b. Implications i. ACTION: Given that tweens have long term retention multiple times (~3X) more than adults, understand why this is the case. 1. Facebook as a whole should consider investing more heavily in bringing in larger volumes of tweens. 2. For the Youth Team, prioritize tweens over all other age groups. ii. ACTION: Track down the former tweens who have incredibly high retention on Facebook and find out why they joined Facebook when they did and what keeps them on Facebook till this day. Can we replicate some of those conditions so that we have a greater number of highly retentive tweens in our pipeline? 2. The earlier the confirmation year (the year in which a user joined Facebook), the higher the long term retention. a. Observations i. For example, for somebody who was born in 1953, this user's long term retention is markedly higher the earlier the user joined Facebook 1. Joined 2009: 76% at Day 1,800 2. Joined 2011: 54% at Day 1,800 3. Joined 2013: Projected ~47% at Day 1,800 (48% at Day 1,400) 4. Joined 2015: Projected ~35% at Day 1,800 (42% at Day 700) ii. You can also see this clearly in the Chart 2 below. The early confirmation years are much greener (LTR closer to 100%) than the latter confirmation years despite the early confirmation years despite having users who are of much greater Facebook age. b. Implications i. Facebook's long term retention deteriorates with each passing year. This is unsustainable, and we need to find out why it deteriorates. 1. Discovering why will lead to different product changes. Some hypotheses: a. Competition: Other social services are better and better META3047MDL-047-00995642 METAMAAG-017-01492551 CONFIDENTIAL CONFIDENTIAL Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 5 of 16 b. Nature of underlying population: The people who join Facebook later on are more marginal. Perhaps they have worse connectivity? Or perhaps they have negative sentiment toward Facebook? c. Changes in Facebook's product/the code and pixels: Our product has changed in a way that does not encourage strong long term retention. d. Changes in the friend graph on Facebook: People who joined earlier on have a different graph than those who joined later on. The graph of those who joined earlier on lends itself to better long term retention. e. Changes in accounts created: There could be more secondary accounts (soon abandoned) or more fake accounts. 2. ACTION: People who joined Facebook earlier on are fundamentally different from the people who join today. What makes them or their situation different? Can we replicate what we had before so that people who join Facebook today have higher retention? 3. In a recent confirmation year, 2016, the higher the population penetration by year of birth, the lower the long term retention. a. Observations i. This is counterintuitive, Conventional wisdom is that as population penetration increases, retention increases. ii. Skip to the section "A Fascinating Relationship Between Long Term Retention and Population Penetration" towards the end to find analysis on this. b. Implications i. If the relationship is a causal one, then it suggests that we do not understand how social media products work. Even if it is only correlational, we should seek to understand the drivers. 1. Is it: a. Users who are signing up for Facebook now are extremely marginal users? i. Why are they marginal? Don't like Facebook? Primarily use Snapchat? Have better uses of their time? b. Users who are signing up for Facebook now are already existing Facebook users but are signing up secondary accounts? i. Why doesn't our SUMA model detect this? c. Users who are signing up are not real people i. Why doesn't our fake accounts team detect this? ii. ACTION: This finding appears to challenge our understanding of how social media is supposed to work. It behooves us as Facebook to understand what is happening. Context and Definitions We initially looked into long term retention in order to calculate the lifetime value for teens. What we found was a dataset that was interesting in of itself. We define long term retention as the continued use of Facebook over very long periods of time, hundreds or even thousands of days. For the purposes of this note, we looked the percent of users who were MAP_28 after n number of days. Quick Primer on Charts META3047MDL-047-00995643 METAMAAG-017-01492552 CONFIDENTIAL CONFIDENTIAL Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 6 of 16 ΜΕΤΑ3047MDL-047-00995644 METAMAAG-017-01492553 CONFIDENTIAL CONFIDENTIAL Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 7 of 16 META3047MDL-047-00995645 METAMAAG-017-01492554 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 8 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995646 METAMAAG-017-01492555 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 9 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995647 METAMAAG-017-01492556 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 10 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995648 METAMAAG-017-01492557 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 11 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995649 METAMAAG-017-01492558 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 12 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995650 METAMAAG-017-01492559 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 13 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995651 METAMAAG-017-01492560 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 14 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995652 METAMAAG-017-01492561 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 15 of 16 CONFIDENTIAL CONFIDENTIAL META3047MDL-047-00995653 METAMAAG-017-01492562 Case 4:22-md-03047-YGR Document 2648-42 Filed 01/20/26 Page 16 of 16 CONFIDENTIAL CONFIDENTIAL META3047 MDL-047-00995654 METAMAAG-017-01492563