Meta & Children's Online SafetyEmail: Re: Daisy Update 8.17.20
Raw text
Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 1 of 9 AMENDED Exhibit 380 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Message From: Sent: To: Kristin Hendrix [/O=THEFACEBOOK/OU=EXTERNAL (FYDIBOHF25SPDLT)/CN=RECIPIENTS/CN= 8/25/2020 2:06:50 PM CC: A Page 2 of 9 ); Justin Cheng Kyle Andrews J; Betty Li Я Subject: Re: Daisy Update 8.17.20 Thanks for sharing this summary and thanks for undertaking additional analyses, Justin! I submitted a request for the analytics notebook. A couple of things - the analyses that Justin shared don't cite the previous findings from IG Research which are less clear about the impact of Daisy. FYI that in March when the last round of Daisy research was being presented to leads, there wasn't complete alignment with how the results were interpreted between all parties involved - that's important context to have as we make the recommendation because leads will likely call this out and likely be very confused here. @Justin can you please coordinate with Kyle and (cc'd here) to get those findings plugged into your update? As it stands currently, it's missing some key citations that will lead to confusion for leads. One other *very* important finding I'll call out from prior research was stemming from Kyle's analyses on highly active teens. This was one of the major drivers of discussions around additional work on Daisy and will need to be addressed somehow - I'm 99% sure this question will come up from leads about how we can either 1) understand this finding more deeply or 2) monitor it carefully if Daisy launches. >>>Given these additional analyses, can we cut it by highly active teens, @Justin? Lastly, we can't conclude that Daisy has a positive impact on well-being as moderated by impacted negative social comparison per se. We need to be really careful about how we frame our verbiage. Though when we ran surveys on Daisy, we included a general measure of, at the time, what best educated guesses around the company thought was a solid measure of well-being, we've determined via our x-FoA well-being measurement research working group with DSS and the well-being research team from CPS and IG that there is not a reliable overall measure of well-being that's well- established and accepted by experts. So we're working hard to be more specific about which factors we're talking about vs. overall well-being. It may be worth having another sync before messaging IG leads - I want to make sure we're communicating this in as clear a way as possible to minimize confusion for them and maximize likelihood our recommendations are fully understood in light of Daisy's history. Thanks again everyone for continuing with the thoughtful conversation and additional analyses to help with this (super!) complex project! Kristin From: Date: Tuesday, August 25, 2020 at 6:29 AM To: Justin Cheng ! >, Kyle Andrews 11 Kristin Hendrix HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071387 Cc: 1 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Subject: Re: Daisy Update 8.17.20 Hey All, 1 Page 3 of 9 Sorry for the delay here. I worked with and Justin offline to make sure I fully understood what new learnings we can leverage around Daisy & social comparison. Justin pulled together a really helpful 1-pager with insights around social comparison in general, as well as Daisy-specific insights. The two Daisy-specific insights are compelling: 1) Negative social comparison decreased more over time for people who have Pure/Popular Daisy compared to people who do not. And 2) People with Pure or Popular Daisy tended to report less social comparison stemming from seeing content with a lot of Likes. It seems we are all in agreement that launching an additional test won't gain further insights, and would only be a re- verification of original insights. My plan is to circle back with IG Leads to share our shared POV, which I'm summarizing to be: A new global test will not gain us better insights, but should be considered if we are want to launch with a Purple Daisy variant that hasn't been tested globally yet. Given the extensive collaboration with FB for our global tests, we don't feel that additional quant or qual research will meaningfully change our narrative. We have two new insights around how Pure Daisy and Popular Daisy have impacted social comparison on IG, which does strengthen our narrative. If IG Leads think these insights are strong enough to strengthen our burden of proof, we should push to move forward. If we don't, we should plan for a roll-back ASAP. Let me know if there are any thoughts/objections. Best, From: Justin Cheng Date: Thursday, August 20, 2020 at 6:26 PM To: Kyle Andrews Cc: 1 Subject: Re: Daisy Update 8.17.20 Hey Kyle, Thanks for the extra context! LI 1 1 1 >, Kristin Hendrix HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071388 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 4 of 9 Separate from whether social comparison should be considered a driver or component of well-being, we know from previous internal and external research that social comparison is linked to multiple negative well-being outcomes (e.g., increased loneliness, worse body image, and negative mood or affect), as well as some positive outcomes (e.g., greater inspiration), so yes, I think it's useful as a measure on its own. Justin From: Kyle Andrews Sent: Wednesday, August 19, 2020 11:24 AM To: Justin Cheng >; Cc: 1 ; Kristin Hendrix >; 1 Subject: Re: Daisy Update 8.17.20 Hi everyone! A few things worth nothing re: how we measured social comparison. In the latest (v3) survey, the question about negative social comparison was worded specifically about likes. Since it's (almost) impossible to compare on likes with likes removed, we talked about the stat sig difference here as more of an induction check. During the past 7 days, how often did you compare the number of likes your posts received to the number of likes other people received? In the previous (v2) survey, the negative social comparison question was asked in a more general way, and we didn't see differences between test and control. Q7: During the past week, how often did seeing the likes and comments on others' posts make you feel worse about your own life? In terms of the differences between now and during the last set of reviews, I think the difference was that during the review we were conceptualizing social comparison as a driver of well-being. And, since reducing comparisons to others via likes didn't influence the "how positive do you feel?" question, the conclusion was that it didn't have a substantial impact on well-being. But, since then, Moira and Justin have argued that reduced negative social comparison is a measure of well-being in and of itself, not a predictor. From that perspective, there's more room to argue that Daisy impacted well-being. Justin, is that an accurate recap? Kyle From: Justin Cheng Date: Wednesday, August 19, 2020 at 1:20 PM To: Cc: Alex Dow Kristin Hendrix 1 Tomas Kabbabe HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 >, Kyle Andrews META3047MDL-003-00071389 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 5 of 9 Subject: Re: Daisy Update 8.17.20 We were examining both Pure Daisy and Popular Daisy. Looking at the survey data, there's some evidence that: People in either the Pure or Popular conditions report less social comparison compared to people in the control (no Daisy) condition Negative social comparison decreases more over time for people in the Pure/Popular conditions compared to the control condition The Pure variant reduces negative social comparison more than the Popular variant Also agree that there may be value in integrating the more recent findings with the research presented at the Q1 reviews. Justin From: Sent: Tuesday, August 18, 2020 5:58 PM To Justin Cheng Cc: Kristin Hendrix 1 >; Kyle Andrews Subject: Re: Daisy Update 8.17.20 I also wasn't in those reviews, but some of those more recent data points that Justin cited around social comparison were analyses that he, Moira, and Shruti Bhutada as part of IG Well-being Mental Health work, and were conducted after Q1 reviews, and thus would not have been included in those discussions. From: Date: Tuesday, August 18, 2020 at 5:20 PM To: Justin Cheng 1 Kristin Hendrix Γ 1 1 Kyle Andrews Subject: Re: Daisy Update 8.17.20 Justin & thanks for sharing your thoughts. When you refer to the Daisy experience, and its positive impact on social comparison, are you referring to what we call "Pure Daisy" (no like counts appearing on any content)? And does this change if we introduce counts back in some form for popular content? It sounds like everyone is in agreement that an additional test is unlikely to produce the results we want to strengthen our narrative. Given that rolling back from Project Daisy is no small feat, I feel strongly that we should not place more users in this experience unless we are entirely confident that there is a gain. HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071390 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 6 of 9 What I'm hearing though is that there might a stronger a narrative from existing research around social comparison. Since I wasn't around for the final reviews of Daisy in Q1, it would be helpful to know how much of what Justin & Alex state below was discussed in reviews? In particular, this finding, "Social comparison may be mild for most people, but at the same time, we've shown (repeatedly) that it's among the top drivers of well-being on FB and IG. We can also show that social comparison does lead to decreased engagement via deactivation.". Do we think there is a chance to revisit our existing data + research to make a stronger case? I'm also concerned about the impact on users who have benefited from this feature if we do roll back. I wonder if we can build this risk into the narrative as well. Best, From: Justin Cheng Date: Tuesday, August 18, 2020 at 7:46 PM To: Cc: 1 Subject: Re: Daisy Update 8.17.20 Hey all! Some thoughts: 1 1 Kristin Hendrix 1 Kyle Andrews Based on the existing and ongoing research, I'd concur with a recommendation to launch some version of Daisy, both on Facebook and on Instagram. I chatted with and agree that it will be difficult to make a much stronger case just based on additional research/experimentation, and if FB app is not particularly receptive. But I do think that we can do additional work in better framing and communicating the existing research, especially in light of our more recent findings. From what I understand, there are two related concerns (and apologies if I'm repeating a lot of what some of you already know): 1. Whether Daisy improves well-being in the first place, and 2. Whether social comparison is even something to be worried about. Will Project Daisy improve well-being? As Moira noted in her summary, we know that even with relatively significant interventions like "thank-a-friend" or "on this day", it's difficult to observe measurable effects on well-being, so we probably want to temper our expectations on being able to observe "large-enough" effects on well-being. That said, the past Daisy and Blue Daisy surveys did show substantial movement in survey responses, and several findings from our recent IG social comparison survey analyses corroborate the Daisy-specific surveys: Daisy may have reduced social comparison that occurs via seeing Likes counts Social comparison may have decreased more over time for people who had Project Daisy The relationship between like counts and social comparison is stronger among adults than among teens Other types of feedback counts (e.g., comment counts) also contribute to social comparison We've also made similar observations on older FB data. Notably, like counts don't appear to be a teen-specific concern, so the skew of FB users to be older shouldn't be a strong factor in these counts being less of a problem on FB. Each of these analyses are flawed in different ways, but triangulating from both quantitative and qualitative research, I think that we have a reasonable case for Project Daisy helping to improve well-being. Is social comparison important enough to care about? HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071391 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 7 of 9 Social comparison may be mild for most people, but at the same time, we've shown (repeatedly) that it's among the top drivers of well-being on FB and IG. We can also show that social comparison does lead to decreased engagement via deactivation. Daisy is such a rare case where a product intervention can improve well-being for almost everyone that uses our products. So even if the benefit is small, especially in the likely case of not launching the "pure" variant, I do hope that we do keep pushing on it. Happy to chat more and hear more about what others think, Justin From: > Sent: Tuesday, August 18, 2020 2:11 PM To Andrews <k Kristin Hendrix ; Justin Cheng a 1 >; Kyle Subject: Re: Daisy Update 8.17.20 Yes, Moira is on leave for a few weeks. I wouldn't presume to speak for her, but I believe she and I are of a similar mind about the value of Daisy. There are downsides to Daisy, but I think it is one of the clearest things (supported by research) that we can do to positively impact social comparison and well-being on IG and we should ship it. I'm amenable to et al.'s recommendation, because I don't believe we will be able to produce sufficiently compelling evidence to overcome the opposition to Daisy in FB App. I wouldn't go so far as to say that more research won't produce a stronger case for Daisy, but more that I'm skeptical we can make it strong enough. Some of this is due to my belief that any imperfect study will be insufficient, and I'm not expecting any study to be perfect. I'd also like to add that, if we can't ship Daisy given the external comms around it and the supporting research, then I am doubtful that we can ship any broad product changes with the purpose of improving user well-being. I know that @Justin consulted with his perspective as well. From: Sent: Tuesday, August 18, 2020 9:45 AM To: Kristin Hendrix on this recommendation, and I think he should weigh in here with >; 1 1 Cc: >; ; Kyle Andrews 1 Subject: Re: Daisy Update 8.17.20 + as well I think Moira is out on covid leave? and thoughts on who the team can talk with? HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071392 Case 4:22-md-03047-YGR From: Kristin Hendrix Sent: Tuesday, August 18, 2020 9:25 AM To: Cc: Π Document 2887-41 Filed 03/27/26 Page 8 of 9 ; Justin Cheng ; Kyle Andrews Subject: Re: Daisy Update 8.17.20 Thanks for sharing this! It's very telling if the folks who were consulted - both within and outside of IG - unanimously agreed that it's the case that another half's worth of work won't lead to a stronger case for Daisy. One thing I wanted to flag is that this particular verbiage "...because we have already studied Daisy so extensively..." is something not everyone is aligned on (eg, esp with teens), so I'd expect pushback from IG Leads and company execs on that assertion. The other piece I'd call out that might spur some back-and-forth discussion with IG Leads and company execs is that we've shifted from thinking about Daisy impacting overall, general "well-being" to more so thinking about it impacting social comparison, which (arguably) is more reliably measurable (and, notably, as called out below, we didn't break it out into positive/negative social comparison). So, the argument of well-being as a construct that is too nebulous to measure - while true - is no longer at the heart of why we'd potentially ship Daisy, which has shifted to focus more so on social comparison specifically. If we haven't already, I'd recommend we also consult with some folks who have historically been pro-Daisy (eg, Moira) to get their input as well. What are the next steps you all see for socializing this recommendation more widely? Thanks again, everyone! From: Date: Monday, August 17, 2020 at 9:17 PM To: Kristin Hendrix 1 Cc: 1 Justin Cheng , Kyle Andrews 1 Subject: Daisy Update 8.17.20 Hi Everyone, Update on Daisy: At the review last week, we discussed that there is a very high bar to make a ship case for Daisy. This half, we have the option of: A) collecting more research/data to convince FB to test again in H1 2021, or B) allocating resources to other initiatives HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071393 Case 4:22-md-03047-YGR Document 2887-41 Filed 03/27/26 Page 9 of 9 The next step is to make a decision on if it's possible for us to get enough net new info this half to succeed at option A. After following up with a wide range of stakeholders across DS, Research, and CDS, its unanimously agreed that another half of work will not lead to a stronger case for Daisy. Because well being measurement is so nebulous, and because we have already studied Daisy so extensively, there are not large areas to explore further. Updates like the Abbreviated Like Count (at a thousand likes) Threshold variant of Daisy, Feed Recs, Shopping tab, and Reels may make the case for Daisy weaker than it was last winter. Therefore, the recommendation is to develop a comms plan to roll back Daisy. If we do proceed with Daisy this half, we might make progress on understanding the following issues: Highly active teens (HAT) meaningful time spent: HAT may find their Daisy time on IG more meaningful if there is additional popular context for media with over 1k likes. Positive validation vs negative social comparison: Previous Daisy surveys did not differentiate between positive and negative social comparison. Popular content detection: The biggest user pain point in Pure Daisy is the inability to detect popular content. The Abbreviated Like Count Threshold variant should somewhat address this, but may put well being benefits at risk, given recent findings that top accounts elicit more comparison than close friends. US teens: The new and improved US teen model may improve our ability to understand teen's behavior on Daisy among the 22M Teen DAP in the US. In particular, we may be able to assess with more certainty if Daisy impacts teen creation (6.8% PR), but we will not have enough power to measure teen deletion because it is too rare (1.9% PR). Revenue confidence intervals: In the most recent Daisy tests, we chose to test two variants using a cluster network test. Both the decision to run a cluster test and the decision to run two variants came at the tradeoff of wider error bars. Since the cluster results were fairly similar to the user A/B test, this half, we could A/B test. And now that we've narrowed on a single variant, we can concentrate all exposures to one variant instead of two, to reduce the size of the error bars around Pure Daisy's revenue regression (-1%±0.9%). With a test exposed to 2% of users, we could reach a MDE of 1% Feed EBR. With a 10% test, 0.5% MDE. Revenue: The Abbreviated Like Count Threshold variant may recover some of the losses, given that we know it recovers half of Pure Daisy's likes regression. IG brand sentiment: This hasn't been measured in previous tests. Feed Recs: The lack of popular context in Pure Daisy regresses DAP by -0.015% with Feed Recs. Shopping Tab: Identifying popular content will likely be important for brand credibility and Shopping's success. Reels: O Launching Daisy may have long term design implications for where we display metrics on Reels and how users interact with it. [early results] Since Daisy reduces Explore time spent by -0.6%±0.3%, it also reduces Reels thumbnail impressions by -0.8% ±0.3%. [early results] Surprisingly, Daisy increases likes on Reels by 1.4%±0.6%, perhaps because the Reels viewer is one of the few places that Daisy users see like counts HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL_RVOL003 META3047MDL-003-00071394