Meta & Children's Online SafetyChat: We found groups of new teen users who see Reels just filled with sexualized media after sexualized
Raw text
Message From: Tomas Kabbabe [/O=THEFACEBOOK/OU=EXTERNAL (FYDIBOHF25SPDLT)/CN=RECIPIENTS/CN= Sent: 3/14/2024 10:46:07 PM To: Tomas Kabbabe @meta.com]; Melissa Subject: Message summary [{"otherUserFbId":100021609290728,"threadFbId":null}] Kabbabe Exhibit 13 2/4/26 Jenn Dunn RMR CRR @meta.com] >this is so crazy to me https://fb.workplace.com/groups/485465535579115/posts/1572130136912644 Melissa (3/14/2024 12:20:56 PDT): Melissa (3/14/2024 12:21:23 PDT): >We found groups of new teen users who see Reels just filled with sexualized media after sexualized Tomas Kabbabe (3/14/2024 12:21:25 PDT): >hmmm Melissa (3/14/2024 12:21:32 PDT): >like their whole tab Tomas Kabbabe (3/14/2024 12:21:43 PDT): >I wonder if they seek it out? Tomas Kabbabe (3/14/2024 12:22:52 PDT): >it also makes you think if we need to look at non-rec dau metrics instead Tomas Kabbabe (3/14/2024 12:22:59 PDT): >Or on top of prevalence Melissa (3/14/2024 12:24:30 PDT): >we did an initial look into their searches and didn't find anything. And it's a bigger issue for new users so it seems less likely. Definitely an area we need to go deeper into Melissa (3/14/2024 12:24:40 PDT): >What would non-rec dau metrics be? Tomas Kabbabe (3/14/2024 12:25:09 PDT): >% of DAU who see a non-rec impression Melissa (3/14/2024 12:26:19 PDT): >Adam wants a metric like that so there's on going work into it. We call it reach. But I actually think that metric solves a very different problem. If prevalence is 1%, we expect all power users to see at least one non-rec impression. And seeing just 1 is probably not nearly as bad as seeing mostly non-rec media Melissa (3/14/2024 12:26:32 PDT): >So we want to measure personal prevalence: look at an individual's prevalence Melissa (3/14/2024 12:26:49 PDT): >Then you can have a metric like "x% of users have personal prevalence over 20%" Tomas Kabbabe (3/14/2024 12:26:54 PDT): >I'd look at the numbers to be sure of that assertion though Melissa (3/14/2024 12:26:59 PDT): >But that's very expensive to measure Tomas Kabbabe (3/14/2024 12:27:14 PDT): >I agree that power users are more likely to see a non-rec impression Tomas Kabbabe (3/14/2024 12:27:24 PDT): >But I'm not sure why all of them would Melissa (3/14/2024 12:27:31 PDT): >yeah I agree with you Melissa (3/14/2024 12:27:44 PDT): >but as users engage with Reels more, we would expect Reach to climb Melissa (3/14/2024 12:27:48 PDT): >so it's not an ideal metric Tomas Kabbabe (3/14/2024 12:28:06 PDT): HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-113-00077373 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-021-00549273 PLTF EX-04338 0001 exhibitsticker.com PLTF EX PLTF EX-04338 >yeah it is. Maybe if we choose a 1% of the population to sample? Melissa (3/14/2024 12:28:46 PDT): >would still be super expensive! we're doing this exercise with 500 random users this month Tomas Kabbabe (3/14/2024 12:29:26 PDT): >even if you use CEP? Melissa (3/14/2024 12:29:51 PDT): >That's the proposal but I think the data will be junk. Too much error introduced Melissa (3/14/2024 12:31:20 PDT): >The way we found this was so crazy. We looked at Reels VM scores for some users of interest and found that there was nearly a perfect linear correlation between the score and the sexual classifier, to the point that when folks saw the chart they were like "why is the sexual classifier an input into the Reels VM" 101 Melissa (3/14/2024 12:31:40 PDT): >so it seems like an issue with diversity of sourcing, that we source exclusively similar content for these users Tomas Kabbabe (3/14/2024 12:34:12 PDT): >oh yeah it would be great to do an analysis to trace this back to sources Tomas Kabbabe (3/14/2024 12:34:41 PDT): >This was the case with image originality, where two sources were essentially only recommending unoriginal content Melissa (3/14/2024 12:37:22 PDT): >yeah I think we should remove certain seeds and clusters that are low quality. this work incredibly really well on Threads Melissa (3/14/2024 12:37:49 PDT): >link or author name? Tomas Kabbabe (3/14/2024 12:50:11 PDT): >dmitry Melissa (3/14/2024 13:05:14 PDT): >strep throat??? :( :( :( Tomas Kabbabe (3/14/2024 13:06:09 PDT): >yeah it hasn't been fun Melissa (3/14/2024 13:27:21 PDT): >that sucks, I'm sorry Tomas Kabbabe (3/14/2024 13:28:13 PDT): >[emoji] Tomas Kabbabe (3/14/2024 15:21:41 PDT): >qq for you Melissa (3/14/2024 15:21:49 PDT): >yo Tomas Kabbabe (3/14/2024 15:22:26 PDT): >Do we do any sort of filtering/demotion on stories for people who post too many political news? Tomas Kabbabe (3/14/2024 15:22:43 PDT): >Context: Relevance is now one team again so I'm overseeing stories Melissa (3/14/2024 15:22:52 PDT): >congrats :D so fun Melissa (3/14/2024 15:23:13 PDT): >I don't believe we do anything like that but I can double check. I'm not sure what treatments we have for reshares in stories, that would be more relevant probably Tomas Kabbabe (3/14/2024 15:23:13 PDT): >And Mosseri just pinged us mentioning that several creators are complaining that their reach is down specifically on stories Tomas Kabbabe (3/14/2024 15:23:40 PDT): >when we look at examples, they seem to be people who post repeatedly about Israel/Palestine Tomas Kabbabe (3/14/2024 15:24:13 PDT): >Very excited to work again with warren and Akritee HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-113-00077374 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-021-00549274 PLTF EX-04338 0002 Melissa (3/14/2024 15:24:33 PDT): >It's definitely possible that integrity treatments for violence are affecting them Melissa (3/14/2024 15:25:00 PDT): >Do we know it it's a sharp drop, like a launch, or just a decline? I would also guess their audience is getting exhausted... Tomas Kabbabe (3/14/2024 15:25:01 PDT): >Ming is 000 Tomas Kabbabe (3/14/2024 15:26:15 PDT): >it's been mostly a steady decline, with a couple of sharp drops here and there Melissa (3/14/2024 15:27:34 PDT): >We have been rolling out a new version of AQVM that we expected to have an effect on Israel/Palestine creators because we added more violence signal. That would expect their Stories views via discovery in Search and other places, and the creators would probably know if that was the cause because they can check their account status Melissa (3/14/2024 15:27:46 PDT): >just pinged the CSI team that owns the politics treatments Melissa (3/14/2024 15:28:08 PDT): >that sounds to me like an exhausted audience... Melissa (3/14/2024 15:29:08 PDT): >We also have been testing CS-violation demotions in Stories. They were broken. So if the creators are in that test, it could explain it, too Tomas Kabbabe (3/14/2024 15:29:38 PDT): >I think so too, but want to cover all angles Melissa (3/14/2024 15:29:59 PDT): >https://tb.workplace.com/groups/725911788061986/posts/1368203830499442 Tomas Kabbabe (3/14/2024 15:32:00 PDT): >thanks! Melissa (3/14/2024 15:43:19 PDT): >sounds like there isn't any Melissa (3/14/2024 15:43:42 PDT): >no politics treatments in stories Melissa (3/14/2024 15:43:53 PDT): >and an eng (Lucas) checked one of the accounts that complained and didn't find any integrity enforcement on it Tomas Kabbabe (3/14/2024 15:44:07 PDT): >wonderful, thanks! Tomas Kabbabe (3/14/2024 15:44:18 PDT): >It's probably fatigue Tomas Kabbabe (3/14/2024 15:44:40 PDT): >We're of course digging a bit more deeply Melissa (3/14/2024 15:44:42 PDT): >there was a Stories VM tuning that launched January 23 that might have affected the strength of p(misinfo) but I can't find the metrics on it https://fburl.com/deltoid3/xgjujoyp and given Lucas's finding, wouldn't be relevant to at least that account Tomas Kabbabe (3/14/2024 15:45:28 PDT): >Sometimes the ranking system amplifies these effects bc we increase the avg position of the pog and then distribution goes down even further Melissa (3/14/2024 15:46:07 PDT): >yeah HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-113-00077375 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-021-00549275 PLTF EX-04338 0003