Meta & Children's Online SafetyChat: We found groups of new teen users who see Reels just filled with sexualized media after sexualized
Thread View
quoted in an excerpt in the archive.
Thursday 14 March 2024
12:21MelissaWe found groups of new teen users who see Reels just filled with sexualized media after sexualized
12:21Tomas Kabbabehmmm
12:21Melissalike their whole tab
12:21Tomas KabbabeI wonder if they seek it out?
12:22Tomas Kabbabeit also makes you think if we need to look at non-rec dau metrics instead
12:22Tomas KabbabeOr on top of prevalence
12:24Melissawe did an initial look into their searches and didn't find anything. And it's a bigger issue for new users so it seems less likely. Definitely an area we need to go deeper into
12:24MelissaWhat would non-rec dau metrics be?
12:25Tomas Kabbabe% of DAU who see a non-rec impression
12:26MelissaAdam wants a metric like that so there's on going work into it. We call it reach. But I actually think that metric solves a very different problem. If prevalence is 1%, we expect all power users to see at least one non-rec impression. And seeing just 1 is probably not nearly as bad as seeing mostly non-rec media
12:26MelissaSo we want to measure personal prevalence: look at an individual's prevalence
12:26MelissaThen you can have a metric like "x% of users have personal prevalence over 20%"
12:26Tomas KabbabeI'd look at the numbers to be sure of that assertion though
12:26MelissaBut that's very expensive to measure
12:27Tomas KabbabeI agree that power users are more likely to see a non-rec impression
12:27Tomas KabbabeBut I'm not sure why all of them would
12:27Melissayeah I agree with you
12:27Melissabut as users engage with Reels more, we would expect Reach to climb
12:27Melissaso it's not an ideal metric
12:28Tomas Kabbabeexhibitsticker.com PLTF EX yeah it is. Maybe if we choose a 1% of the population to sample?
12:28Melissawould still be super expensive! we're doing this exercise with 500 random users this month
12:29Tomas Kabbabeeven if you use CEP?
12:29MelissaThat's the proposal but I think the data will be junk. Too much error introduced
12:31MelissaThe way we found this was so crazy. We looked at Reels VM scores for some users of interest and found that there was nearly a perfect linear correlation between the score and the sexual classifier, to the point that when folks saw the chart they were like "why is the sexual classifier an input into the Reels VM" 101
12:31Melissaso it seems like an issue with diversity of sourcing, that we source exclusively similar content for these users
12:34Tomas Kabbabeoh yeah it would be great to do an analysis to trace this back to sources
12:34Tomas KabbabeThis was the case with image originality, where two sources were essentially only recommending unoriginal content
12:37Melissayeah I think we should remove certain seeds and clusters that are low quality. this work incredibly really well on Threads
12:37Melissalink or author name?
12:50Tomas Kabbabedmitry
13:05Melissastrep throat??? :( :( :(
13:06Tomas Kabbabeyeah it hasn't been fun
13:27Melissathat sucks, I'm sorry
13:28Tomas Kabbabe[emoji]
15:21Tomas Kabbabeqq for you
15:21Melissayo
15:22Tomas KabbabeDo we do any sort of filtering/demotion on stories for people who post too many political news?
15:22Tomas KabbabeContext: Relevance is now one team again so I'm overseeing stories
15:22Melissacongrats :D so fun
15:23MelissaI don't believe we do anything like that but I can double check. I'm not sure what treatments we have for reshares in stories, that would be more relevant probably
15:23Tomas KabbabeAnd Mosseri just pinged us mentioning that several creators are complaining that their reach is down specifically on stories
15:23Tomas Kabbabewhen we look at examples, they seem to be people who post repeatedly about Israel/Palestine
15:24Tomas KabbabeVery excited to work again with warren and Akritee
15:24MelissaIt's definitely possible that integrity treatments for violence are affecting them
15:25MelissaDo we know it it's a sharp drop, like a launch, or just a decline? I would also guess their audience is getting exhausted...
15:25Tomas KabbabeMing is 000
15:26Tomas Kabbabeit's been mostly a steady decline, with a couple of sharp drops here and there
15:27MelissaWe have been rolling out a new version of AQVM that we expected to have an effect on Israel/Palestine creators because we added more violence signal. That would expect their Stories views via discovery in Search and other places, and the creators would probably know if that was the cause because they can check their account status
15:27Melissajust pinged the CSI team that owns the politics treatments
15:28Melissathat sounds to me like an exhausted audience...
15:29MelissaWe also have been testing CS-violation demotions in Stories. They were broken. So if the creators are in that test, it could explain it, too
15:29Tomas KabbabeI think so too, but want to cover all angles
15:29Melissahttps://tb.workplace.com/groups/725911788061986/posts/1368203830499442
15:32Tomas Kabbabethanks!
15:43Melissasounds like there isn't any
15:43Melissano politics treatments in stories
15:43Melissaand an eng (Lucas) checked one of the accounts that complained and didn't find any integrity enforcement on it
15:44Tomas Kabbabewonderful, thanks!
15:44Tomas KabbabeIt's probably fatigue
15:44Tomas KabbabeWe're of course digging a bit more deeply
15:44Melissathere was a Stories VM tuning that launched January 23 that might have affected the strength of p(misinfo) but I can't find the metrics on it https://fburl.com/deltoid3/xgjujoyp and given Lucas's finding, wouldn't be relevant to at least that account
15:45Tomas KabbabeSometimes the ranking system amplifies these effects bc we increase the avg position of the pog and then distribution goes down even further
15:46Melissayeah