Meta & Children's Online SafetyChat: Flagging this new video Horwitz just sent of the AYSF recommendations
Thread View
quoted in an excerpt in the archive.
Tuesday 31 October 2023
01:31Malia@Carolina I'll be curious of the outcome for these breastfeeding groups; is here the right way to
06:39Kaitlinhttps://www.internalfb.com/cases/4439816
06:42KaitlinLooks like current consensus is to remove as an SoP call for ANSA
06:44CarolinaYes, you can follow C#4438125. For now, PRESC ran a fanout of the original group's admins and was able to disable a total of 25 groups for ANSA violations, with alignment from CPT.
They are now running inauthenticity checks for the admins.
08:45AndyFlagging this new video Horwitz just sent of the AYSF recommendations for one of the Instagram accounts he's used to do this chid safety research.
Can we please look into this? I'd have assumed at this point the steps we've taken would have accounted for at least most of these recommendations.
08:45AndyShared a file
08:59Alisonoof thanks for flagging Andy.
1. We need to nuke these accounts ASAP. @Carolina what is the best way to have ops not only remove the accounts that are being recommended, but also can we do a fanout of the accounts that are following the accounts being recommended here.
2. MaSCA scores. Can we figure out what Jeff's account's MaSCA score is and why it is being recommended not only seemingly minor content, but what also looks like accounts that should have a MaSCA score themselves and be non-rec. @Rishi is this something your team can prioritize today? It seems like our thresholds for AYSF may not be in the right place?
3. For #2 @Carolina can we ensure ops pulls the list of handles before removing the accounts so we can do that review?
09:01GabrielMeta - Clegg 3/21/2025 exhibitsticker.com PLTF EX If we could get a list of these account IDs I think we can then check what % of them had a MaCSA score and whether there are signals we are missing
09:01CarolinaHey! I'll spin up a case for this right now and will investigate. Gabriel, I'll subscribe you so you can get the IDs.
09:04AlisonThank you @Carolina please subscribe me and Andy as well.
09:05CarolinaWill do!
09:05AlisonCan we please specifically ask for whoever is reviewing to list out the IDs somewhere in the case so they can be pulled easily.
09:05AlisonThanks!
09:08Kristina13 Child Safety identified several accounts belong to Jeff. Last we checked, they didn't have MaCSA scores.
#shareadoc comment
https://docs.google.com/spreadsheets/d/1H4W5mnJGEyHo16HISO6wg78dzWdsRssGRYSDdpL_12s/edit#
09:13Alison@Gabriel can we take a look here while ops is pulling the accounts from the video for why his accounts don't have a MASCA score given how often he is searching for/viewing this type of content.
09:18MaliaThere are no MaCSA signals for views.
09:18MaliaThere are MaCSA signals for likes and saves of content. And there are signals for searches *if* the search terms are in the high-precision list.
09:19MaliaWe have not been able to include the wider set from the taskforce in this list as the precision is varying.
09:19GabrielYes @Kristina and @Malia for these accounts 1/ should the MaCSA signals be triggered? is there some gap in what we are detecting?
2/ are there any additional human-driven signals on looking at these accounts which, if automated, would have led to these accounts being considered risky (1 or 2 points)
09:21Alison+1 my point above was based on what I assume the behavior of his accounts are (viewing/searching) we should confirm first if the MaSCA signals should be triggered here (per @Gabriel I's point).
09:28CarolinaAlso, just flagging that I'm using my test user account to surface the accounts, and also started being recommended some myself.
09:28CarolinaShared an image
09:28CarolinaShared an image
09:30AlisonCan you share the case here when you have it? Separately confirming that @Malia you are looking into Gabe's ask above?
@Kristina
09:32KristinaYes I can look into this
09:33SwithinShouldn't these accounts also be auto enqueued for review based on account names esp pizzalink, kidseller? @Diego
09:34CarolinaC#4447847
09:35AlisonGreat IDs are in the case @Gabriel
09:43MelissaAlso looks like a handful do have reactive user reports on the profile, but are sitting in the queues/pending
09:43AlisonSorry what profile?
09:44Melissaa handful of those in the case
09:47DiegoWe have variations of this. Would be helpful to see what are the new names that are being used and we can add them to the TMS banks for enqueuing or auto actioning.
09:48AlisonComing back to this:
1. @Carolina when we have it, please provide an update on the accounts that have been actioned/result of the fanout.
2. @Malia @Kristina @Gabriel put a doc here so we can centralize across the reporters accounts + IDs from the video
https://docs.google.com/document/d/1eZa0GCrc1m_SyUyrNF1Vtv4r910dNQoKdSoq6brEvNs/edit #shareadoc
09:49AlisonGreat can you please review them from the case shared above and add new content to the TMS banks if permissible.
09:51CarolinaDiego I'll include these insights in the case once I'm done with the investigation, but based on initial fan-outs, I'm seeing a lot of keyword obfuscation (i.e. "megha" or "megga" instead of "mega", and instead of "seller", using "sellar" or "zshaller") But also more obvious versions like, "pizza mega sellers" or "trusted cp seller".
09:53RishiGabriel/Malia: for the accounts that are being surfaced here, should they be caught by MACSA? Or are there other CSE-I profile level classifiers that can catch these?
09:57MaliaWe'll take a look and assess. I'm assuming your question here is on the recommended profiles and not JH's.
09:57Rishiyes
09:58RishiCould we use https://fb.workplace.com/notes/311970791222605 for soft actions?
09:58MaliaIs this not live?
09:59RishiReading the note, it seems like we're using it only to enqueue/UFAC
09:59RishiHypothetically, we could use lower thresholds to filter out of recs
10:00MaliaAh yes, ok.
10:00GabrielYeah i think that's right. We didn't look into this for taskforce but might be an option; think we could probably get a precision number based on what's getting enqueued
10:07Carolina@Malia /@Gabriel on the case the accounts that were recommended to Jeff are labeled as "JH AYSF", you'll notice the case now has about 172 IG accounts, the bulk of which I've found using keyword combinations on front-end and conducting fanouts to find device-linked IG accounts.
ok 1mk.
10:08RishiCould we also use rough heuristics for eg, filter these out of recs.
10:09Gabrieland we worth a sidebar: I think this is how we should be doing nonrec more generally; will need a broader conversation incl policy and upstream teams who work on X-Ray image features
10:10Rishiok let me spin up a side chat
10:13Amiti think this is a promising direction. are we going to expand on this line of thinking and actually eliminate borderline seeds from the rec selection process?
10:13Amitis that something we are also discussing
10:15SwithinSome of these do have a ton of stories. We should definitely remove them from seed sourcing. I'm also seeing some UFAC activity on the accounts and am unclear about what those mean
10:15SayedCan we also find out what the stated/predicted ages are for these recommended accounts? It can help inform some of how we are thinking about AYMF for teens.
10:20Amitso is this change to seed sourcing live now or we need to do it?
10:24RishiIt's not live. Once we align on which signals can/should be used, we should probably do both: (a) stop seed sourcing and (b) do a last pass ranking integrity filter.
(a) will take time as it gets deep into the AYMF stack (b) can be a quicker change
10:49GabrielFolks, it might be helpful to step back a little on IG recs;
1/ we have shipped a bunch of things already we can validate they are working as intended
2/ More broadly, AYMF and Content filtering are still not where we would want them to be. There are multiple levers we could pull but all would take some focus and both eng+policy
3/ business outcomes needed and timelines are currently unclear; it's not obvious to me that pivoting back into war room mode across IGWB and XI teams and shipping some hotfixes now would either address the Jeff story issue (given timlines of this week) or set up teams for longer term success but I would like to get some SR steer on this cc @Alison
14:43CarolinaHey all, following up with the latest enforcement figures from the GRO investigation on the AYSF case (C#4447847).
In addition to the 12 accounts featured in Jeff's video, and the 204 IG accounts we disabled from preliminary fan-outs, GRO reviewed 1.6K users that were device linked to the original disabled accounts.
We conducted subject element reviews (profile photo, username, and bio), and disabled 987 additional IG accounts for CSAM trade.
In total, the investigation resulted in 1203 IG account takedowns. The majority of the users were based in India and engaging in trading/selling of telegram links.
As a next step, we're going to collect the common keywords we spotted and will share with Diego and Jonny to inform further TMS banking efforts.
Let us know if you have any questions or have any additional IDs for review.
cc. @Andy @Alison
14:50DiegoThank you, Carolina!