Facebook's Measurement Strategy for Problematic Use [Well-Being]: Survey & First Results [October 2019]

Raw text

Transcribed text as recorded, including headers and formatting fragments.

Case 4:22-md-03047-YGR Document 2756-47 Filed 02/20/26 Page 1 of 12 AMENDED Exhibit 99 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 2 of 12 https://fb.workplace.com/notes CONFIDENTIAL Facebook's Measurement Strategy for Problematic Use [Well-Being]: Survey & First Results [October 2019] THURSDAY, NOVEMBER 14, 2019 READING TIME: 11 MINUTES POC: Sean Ross, Definition: We define problematic use as experiencing both of the following issues "very often" or "all the time": 1. Lack of control or feelings of guilt over Facebook use. 2. Negative impact in at least one of the following areas: sleep, parenting, social relationships, or productivity. By: & [Internal definition] Executive Summary: We have made significant headway in our measurement strategy: We have a survey to detect problematic use that is deeply rooted in external academic as well as internal quantitative and qualitative user experience research. • We have broken down problematic use sub-areas into time-related and content-related issues, thus making it easier to move these issues, which likely have different product solutions, separately. • We have an updated PU baseline for Facebook which is now more expansive (e.g., additional subdomains included) • We now have a survey to measure problematic use related people problems in response to product changes and are working on unobtrusive measures (e.g., behavioral metrics) to do this even more effectively • We now have a global understanding of problematic use as opposed to only an understanding of the issue in the US acebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 1/11 META3047MDL-079-00000165 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 1. Academic Research on Problematic Use Page 3 of 12 • There is no established or consistently used definition of either internet addiction or problematic use in academic research or clinical practice. • Existing assessments are also inconsistent and don't have a clear measurement framework. The research community is divided in a set of researchers believing that there is true "internet addiction," i.e., a new or emerging form of addiction and thus an actual psychiatric disorder, versus another set of researchers believing that instead the issue is less severe and better described as "problematic internet use" which manifests for clearly defined online activities such as gambling, pornography, or Facebook use (Yellowlees & Marks, 2007). • Frequency-based style of assessments are most common to assess problematic use (see: here: How often does a participant experience feelings of lack of control or negative life impact). Sometimes agree/disagree style assessments (see here and here) are used which should be avoided (more: here) in general due to responses being biased (people having a tendency to wanting to agree with things) and increased cognitive load on respondents.) • Commonly agreed on facets of problematic use include perceived lack of control over the time spent on Facebook; negative feelings about own Facebook usage (e.g., guilt, regret, shame); and negative life impact for: relationships, sleep, & productivity/work. [Note: "Internet addiction scales more commonly ask about key factors of addiction such as: mood modification, salience (i.e., total preoccupation with social media use), tolerance (i.e., increasing amounts of time using social media), withdrawal symptoms (i.e., negative feelings and psychological symptoms such as irritability, anxiety when social media use is restricted), conflict (i.e., interpersonal problems as a direct result of social media usage), and relapse (i.e., returning to excessive social media use after a period of abstinence)" (Bányai et al., 2017). Main issues with current survey measures for problematic use in academia: • Including desire to use [Facebook/internet use] to manage mood/emotions or escape from negative feelings facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ https://fb.workplace.com/notes/ CONFIDENTIAL 2/11 META3047MDL-079-00000166 10/23/24, 2:13 PM https://fb.workplace.com/notes CONFIDENTIAL Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 4 of 12 • Including desire to use [Facebook/internet use] when feeling lonely or isolated • Mere thinking about [Facebook/internet use] when not online • Going on [Facebook/internet] when feeling down • Constructs that may be more in line with personality, such as preferring [Facebook/internet use] to in-person interactions → As a result, we have begun championing our own rigorous measurement frameworks and begun the process of scientifically validating them through research and peer review. 2. Facebook's measurement approach At Facebook, we have decided to focus on problematic use (rather than clinical levels of addiction) for the following reasons: 1. The DSM-V, the key document that clinicians use to diagnose and define psychiatric disorders, the "Diagnostic and Statistical Manual of Mental Disorders," despite a proposal to do so, did not include internet addiction as a mental illness after evaluating the evidence. 2. Evidence so far is more in support of the existence of problematic use than of internet addiction [with exception of gaming where there is some evidence in support of internet addiction], "despite a total lack of methodologically sound research." (Yellowlees & Marks, 2007) 3. Definitions, measurement, and prevalence estimates vary too greatly (e.g., 3%-80% affected) to present an actionable path forward for measurement. 4. Unlike global internet addiction, problematic use distills down into actionable people problems that, at least in theory, are solvable via Facebook features and interventions. Nomophobia, "feeling anxious if not near one's phone," has been discussed as a hallmark of internet addiction, but is much less actionable than "lack of control over time spent," a criterion widely accepted as part of problematic use. 5. Given the difficulty in treating and impacting true psychiatric disorders, we believe that a focus on problematic use will allow us to help a larger group facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 3/11 META3047MDL-079-00000167 10/23/24, 2:13 PM https://fb.workplace.com/notes CONFIDENTIAL Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 5 of 12 of people. Interventions for truly "addicted" individuals may, by definition, need to occur offline. 6. Diagnosis of full-blown addiction would require diagnosis of a mental health professional rather than a survey instrument alone which is not practical Initial measurement approach and foundational research on problematic use at Facebook [PoCs: Moira Burke, Justin Cheng, Elena Davis] • Prior research at Facebook measured extreme problematic Facebook use as control over time spent or fear of missing out in conjunction with negative life impact in sleep, relationships, productivity, or overall. • FB researchers Moira Burke, Justin Cheng, and Elena Goetz Davis have externally published a first measurement framework for measuring extreme problematic use in the US, finding about 3.1% of Facebook users to be affected by it. • Through our publication efforts we are not just advancing our own understanding but also playing a part in improving definition and measurement approach of problematic Facebook use. (Cheng, Burke, & Goetz Davis, 2019) Based on this foundational work we have identified key behavioral and demographic characteristics of those affected by severe problematic use. 3. Where we are today in our measurement of problematic use at Facebook • We have extended our measurement framework to capture a wider range of issues. • We revised our survey questions to be even more closely aligned with concrete people problems in hopes that these are easier to move than attitudinal assessments. • All questions follow the same format quantifying frequency of negative impact resulting from FB use as is standard practice in mental health assessment scales. facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 4/11 META3047MDL-079-00000168 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 6 of 12 • We have extended measurement to a less strict definition of problematic use to be able to improve the Facebook experience for a larger group of people. • We have begun to measure problematic use globally with initial results for top MAP countries and identified that PU is actually a bigger issue in non-Western countries (e.g., PH, IN). • We now have a large, multi-national dataset of participants who have completed the problematic use questionnaire (n=42,000). • We are identifying a set of demographic and behavioral correlates of PU. We are working to identify a small set of behaviors that correlate highly with PU. Product interventions that significantly increase these behaviors could point to these products as being a risk factor for PU. Measurement of these behaviors in product experimentation could serve as a health or guardrail metric. What people problems are part of our current problematic use assessment? Our current measurement framework includes six key subcategories of people problems around problematic use: • Control over time spent: Aspects of the user experience, social norms of one's network, or compulsive habits that make it difficult to regulate one's duration of use or quality of use OR Guilt over time spent: Feeling guilty or ashamed over time spent on FB as a hallmark emotion associated with addictive behaviors https://fb.workplace.com/notes CONFIDENTIAL AND ONE OF THE FOLLOWING: • Sleep disruption: Affecting people's ability to get proper sleep as a result of a) losing track of time or b) encountering content that is upsetting and leads to rumination • Relationships: Affecting relationships negatively through a) neglecting relationships outside of FB or b) negative impact on relationships through disagreements or conflicts on FB • Parenting: Parents' concerns that FB use may negatively impact their children facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 5/11 META3047MDL-079-00000169 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 7 of 12 • Productivity: Affecting productivity or one's ability to be able to make progress on important tasks 4. First Results on Problematic Use via the current measurement framework: Method: Step 1: Four rounds of cognitive testing of previous (strict) problematic use survey items & new iterations via rolling research with 27 interviews completed. [Round 1 US [n=6], Round 2 US [n=5], IN [n=10], ID [n=6].] Step 2: International cross-sectional survey (TH, RU, IN, ID, GB, US, TR, PH, BR, MX, VN). N = 50,000 [survey weighted on age, gender, country, time spent] Under the extended measurement framework, problematic use appears to affect 12.5% of people. Another 11% of people report negative life impact 'very often' but only 'sometimes' lacking control. Problematic use is highest in PH (~25%), IN (~23%), and MX (~14%) → this suggests we may need a global approach to the problem. The US falls in middle of the pack (~10%) Note that we see quite a bit of between-country variability. We may need to prioritize certain countries and might even have to consider whether different solutions are needed to address problematic use in different countries. https://fb.workplace.com/notes/ facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ CONFIDENTIAL 6/11 META3047MDL-079-00000170 10/23/24, 2:13 PM 0.25 0.2 0.15 0.1 0.05 Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 8 of 12 Problematic Use by Country PH IN MX TR VN GB US Countries by prevalence of problematic use H.L BR RU • Different domains of problematic use are correlated (~0.4-0.5 range of correlation) but sufficiently distinct. • Most negatively impactful concerns: • (1) sleep issues due to losing track of time (15%) • (2) difficulty managing time on FB (14%) • (3) parents concerns that their FB use could neg. affect their kids (8% - lower prevalence because it only affects parents, but a possible opportunity area for us [Around 20% of parents affected]) https://fb.workplace.com/notes/ facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ CONFIDENTIAL 7/11 META3047MDL-079-00000171 10/23/24, 2:13 PM https://fb.workplace.com/notes CONFIDENTIAL 20 ST 10 Case 4:22-md-03047-YGR Document 2756-workplaceiled 02/20/26 Page 9 of 12 Subsetted on parents Neg. ships: Productivity impact on Sleep: forgot time Lack of control Relation Neglect children Relation Sleep: ships: Content Conflict Guilt over FB usage Percentage of users who report experiencing each issue very often or all the time First results on behaviors of those high in PU: Those higher in PU have more sessions than those low in PU, particularly low engagement sessions. Those higher in PU spend more time on Facebook than those low in PU, particularly at night. • Those higher in PU respond to more notifications than those low in PU. More repetitive behaviors such as frequent app checks or checking newsfeed several times a session • More feed inventory coming from groups and pages Problematic use may cause issues for our users and this should be our primary business reason to address it. Additionally, problematic use also negatively affects our business in other ways: 20 percent of participants who suffer from problematic use consider deactivation or deletion due to negative life impact of Facebook very often as facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 8/11 META3047MDL-079-00000172 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-4Workplaciled 02/20/26 Page 10 of 12 opposed to less than 5 percent of those who do not suffer from problematic use: https://fb.workplace.com/notes/ CONFIDENTIAL 0.2 0.15 0.1 0.05 0- PU non_PU Percentage of participants who report considering to delete their Facebook account due to perceived negative life impact very often or all the time 5. Possible next steps • Develop a behavioral model for problematic use based on the ground truth of the survey. This model will not be used to score individuals on problematic use, but to identify behaviors we may want to safeguard against and use as heuristics. Run validation tests to validate our behavioral measure via a survey approach to ensure that the model accurately captures PU • Develop a heuristic for problematic use: a small set of behaviors that correlate highly with PU and can serve as directional indicators of whether a product intervention may be moving PU, one way or another. • Evaluate the effectiveness of our problematic use products by evaluating behavioral changes and demonstrating differences in survey responses between individuals who are affected by problematic use who have used the products and those who have not. facebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ nd non 9/11 META3047MDL-079-00000173 10/23/24, 2:13 PM Case 4:22-md-03047-YGR Document 2756-4Workplaciled 02/20/26 Page 11 of 12 6. Opportunities The tech industry's approach to problematic use has been very focused on regulating time spent. As an application developer and platform creator we might have the opportunity to go much deeper. Problematic use is comprised of a variety of problems, and we now have the opportunity to measure more of these issues via our new measurement approach. In this way, it will become easier to systematically attempt to solve the issues related to problematic use in product. 1. Product strategy: We should identify one area of focus among the negative life impact areas where we can make a significant impact for those affected by problematic use. To do so, we need to understand trust in relation to well-being on platform and in relation to well-being products in particular. Then we need to show our trustworthiness, competence and care, in our product development every step of the way so that we can succeed in the realm of behavioral interventions. 2. Cross-product efforts: For many people, problematic use may not be specific to just one app, but to phone use or social networking in general. Given that our company owns several of such applications, we might be able to more easily scale our impact on problematic use. That is, impact on problematic use does not need to stop with the Blue app. Think about the people who may switch frequently between Instagram, WhatsApp, Facebook, Messenger, and Oculus in search for updates, news, perhaps at times alerted by notifications. We might be able to offer a HUB with controls that cut across the family of apps, offering digests summarizing what an individual has missed across our family of apps while unplugging, or features to help people meaningfully connect with family or friends on multiple channels without getting sidetracked in unwanted ways. Together, at FB Inc., we can dream big! We thank our advisory team for the support and impactful guidance they have provided to us: Moira Burke, Jennifer Guadagno, 29 https://fb.workplace.com/notes/ CONFIDENTIAL acebooks-measurement-strategy-for-problematic-use-well-being-survey-first-resul/164034771370608/ 6 comments 10 shares 10/11 META3047MDL-079-00000174 Case 4:22-md-03047-YGR Document 2756-47 Filed 02/20/26 Page 12 of 12 CONFIDENTIAL META3047MDL-079-00000175