Chat: Meaningful Engagement is a proxy for future retention

Raw text

Transcribed text as recorded, including headers and formatting fragments.

Case 4:22-md-03047-YGR Document 2889-26 Filed 03/27/26 Page 1 of 3 AMENDED Exhibit 217 PLAINTIFFS' OMNIBUS OPPOSITION TO DEFENDANTS' MOTIONS FOR SUMMARY JUDGMENT Case No.: 4:22-md-03047-YGR MDL No. 3047 In Re: Social Media Adolescent Addiction/Personal Injury Products Liability Litigation Case 4:22-md-03047-YGR Document 2889-26 Filed 03/27/26 Page 2 of 3 Message From: Sent: To: Subject: [/O=THEFACEBOOK/OU=EXCHANGE ADMINISTRATIVE GROUP (FYDIBOHF23SPDLT)/CN=RECIPIENTS/CN= 12/15/2021 12:48:49 AM ; Message summary [{"otherUserFbId":100021941319019,"threadFbId":null}] (12/14/2021 06:00:08 PST): >Another example of great data science work: >The Stories ranking model is a multiplicative model. That means we multiply all our predictors together to generate the final ranking score. > >To break it down futher, we have a handful of models that try to predict different events, like p(tap) or p(BFF). Each predictive model generates its own score. Those score gets multiplied against the scores from the other predictive models, to generate a final score that we use to rank all the Stories. https://www.internalfb.com/intern/wiki/Instagram_Home_Ranking/Beginner's_Introduction/Stories_Ranking/1- _Predictive_Model/ > >This half, we a predictive model to predict Stories replies and a predictive model to predict if a user will send a direct reshare of a story. David observed that p(reply) is highest on stories from someone you actually know in real life. But p(reshare) happens more often on interest content. So, pogs would usually have a high score on one model or the other, but never both. As a result, a low score would be multiplied against a high score, leading to the downranking of stories that have a high probability of starting a conversation. So, we are going to start adding together p(reply) and p(reshare), instead of multiplying them against eah other. > >Early tests show that this could end up being one of the most impactful launches of the half, all based on that data science insight that different types of stories drive different types of conversations. (12/14/2021 06:00:08 PST): >One of the highest impact DS projects I've ever seen was the development of a ranking optimization metric, what we now call Meaningful Engagement. This work is honestly what sold me on joining this team. On ecosystems, one of my responsibilities was upleveling the best DS work at IG, and the Meaningful Engagement project was so superior to anything I had ever seen before on any other DS team. > >Ranking is a series of functions that try to maximize a metric. Before Meaningful Engagement, ranking optimized for impressions. Engineers would build new iterations of the ranking models, and if impressions went up, we would ship. Impressions is a bad metric for many reasons. We would try to also consider other types of engagement, but the tradeoff conversations were messy and inconsistent. So, the DS team built a composite metric, combining many forms of engagement. > >Meaningful Engagement is a proxy for future retention. We run a regression, trained on many types of engagement events to predict future session outcomes. This regression determines the value of engagement we can observe in the short term in our experiments, like story replies or dwell time. We are constantly improving Meaningful Engagement, and the results are astounding. The ranking team has consistently driven +0.1% DAU per half for a year now!! It drives revenue! I've never seen anything like it! This half, most of the feed ranking engineering team spent all their time trying to build new predictors to maximize Meaningful Engagement. As a result, we blew our goals out of the water. > >https://fb.workplace.com/notes/1203881303399259/ (12/14/2021 07:58:21 PST): >yess 1 woulu really like to chat with him actually (12/14/2021 07:58:34 PST): >1 can reach out to him as well! and if he comes to the office maybe we can go grab a coffee (12/14/2021 07:59:41 PST): >these examples are really really exciting thank you for sending!!! both you & have been really helpful in outlining what good DS work looks like on your team (12/14/2021 07:59:55 PST): >thought was really good too from our 30 min convo (12/14/2021 08:00:20 PST): >invited me to an experiment review later this week which i will go check out! and then, not sure if there is a team lunch that i could join? (12/14/2021 09:24:52 PST): >so glad to hear that! yes, this team is super talented. I'm constantly impressed CONFIDENTIAL METADCAG-036-00029529 CONFIDENTIAL ΜΕΤΑ3047MDL-152-00029529 Case 4:22-md-03047-YGR Document 2889-26 Filed 03/27/26 Page 3 of 3 (12/14/2021 09:25:12 PST): >yeah he's great, obviously only hires the best (12/14/2021 10:03:25 PST): >I'm setting up lunch for tomorrow, can you make it work? (12/14/2021 11:00:38 PST): >yes i can!! 1 m free from 122 so anytime is good (12/14/2021 11:48:04 PST): >yay! (12/14/2021 16:42:21 PST): >okay here is the low down on lunch: the attendees will be who works on creator feed ranking, won't be in the office tomorrow, but she comes in Trequently. who you know. is also in the office frequently, but he is working remote in LA for the rest of the week. Most of the eng team is based in NY but they haven't been back in the office yet. The eng tech lead, is AMAZING, I can't wait for you guys to meet, but he is remote from Chicago. I also invited DE. We ve been sharing DE with other ranking teams, but we just hired our first dedicated Sharing Relevance DE this week, and she'll start in late January @delay (12/14/2021 16:48:49 PST): >if you want to join any other meetings this week: there is a cross team ranking understand meeting, a stories xfn team meeting, a meeting with s entire ds team, and a meeting with my ds team @silent CONFIDENTIAL CONFIDENTIAL METADCAG-036-00029530 META3047MDL-152-00029530