Meta & Children's Online SafetyChat: Flagging this new video Horwitz just sent of the AYSF recommendations
Raw text
Message From: Sent: Faye (FYDIBOHF23SPDLT)/CN=RECIPIENTS/CN 10/31/2023 9:50:49 PM To: /OU=EXCHANGE ADMINISTRATIVE GROUP @meta.com]; Taylor @meta.com]; Rahul Sophie @meta.com]; Corri @meta.com]; Rishi @meta.com]; Lexi Rory @meta.com]; Gabriel @meta.com]; Mallory @meta.com]; Faye @meta.com]; Arcadiy @meta.com]; Komal @meta.com]; Amit meta.com]; Victoria @meta.com]; Tony @meta.com]; Alison @meta.com]; Kathy @meta.com]; Malia @meta.com]; Kate @meta.com]; Yashaswi @meta.com]; Andy @meta.com]; Swithin @meta.com]; Nir @meta.com]; @meta.com]; Stephanie @meta.com]; Carolina @meta.com]; Hudson @meta.com]; @meta.com]; Jimmy @meta.com]; Kaitlin @meta.com]; Ashly @meta.com]; Melissa @meta.com]; Kristina @meta.com]; Mike @meta.com]; Richard @meta.com]; James @meta.com]; Scott @meta.com]; Mike @meta.com]; Sayed r@meta.com]; Sarah @meta.com]; Leah @meta.com] Subject: Message summary [{"otherUserFbId":null,"threadFbId":6427561907365847}] Attachments: video-e0033bc9-90c3-4c67-ac9f-2fd4b6a4f03d-1698767061.mp4; 377124407_284350133924228_7896967635058393494_n.png; 370241162_303782345840542_536791761734269577_n.png follow? Malia (10/31/2023 01:31:09 PDT): >@Carolina I'll be curious of the outcome for these breastfeeding groups; is here the right way to Kaitlin (10/31/2023 06:39:52 PDT): >https://www.internalfb.com/cases/4439816 Kaitlin (10/31/2023 06:42:18 PDT): >Looks like current consensus is to remove as an SoP call for ANSA Carolina (10/31/2023 06:44:12 PDT): >Yes, you can follow C#4438125. For now, PRESC ran a fanout of the original group's admins and was able to disable a total of 25 groups for ANSA violations, with alignment from CPT. > >They are now running inauthenticity checks for the admins. Andy (10/31/2023 08:45:19 PDT): >Flagging this new video Horwitz just sent of the AYSF recommendations for one of the Instagram accounts he's used to do this chid safety research. > >Can we please look into this? I'd have assumed at this point the steps we've taken would have accounted for at least most of these recommendations. Andy (10/31/2023 08:45:20 PDT): shared: video-e0033bc9-90c3-4c67-ac9f-2fd4b6a4f03d-1698767061.mp4 Alison (10/31/2023 08:59:23 PDT): >oof thanks for flagging Andy. >1. We need to nuke these accounts ASAP. @Carolina what is the best way to have ops not only remove the accounts that are being recommended, but also can we do a fanout of the accounts that are following the accounts being recommended here. >2. MaSCA scores. Can we figure out what Jeff's account's MaSCA score is and why it is being recommended not only seemingly minor content, but what also looks like accounts that should have a MaSCA score themselves and be non-rec. @Rishi is this something your team can prioritize today? It seems like our thresholds for AYSF may not be in the right place? >3. For #2 @Carolina can we ensure ops pulls the list of handles before removing the accounts so we can do that review? Gabriel (10/31/2023 09:01:11 PDT): Meta - Clegg 3/21/2025 Exhibit 63 exhibitsticker.com CONFIDENTIAL METANMAG-039-00000003 PLTF EX PLTF EX-06012_0001 PLTF EX-06012 >If we could get a list of these account IDs I think we can then check what % of them had a MaCSA score and whether there are signals we are missing Carolina (10/31/2023 09:01:42 PDT): >Hey! I'll spin up a case for this right now and will investigate. Gabriel, I'll subscribe you so you can get the IDs. Alison (10/31/2023 09:04:15 PDT): >Thank you @Carolina please subscribe me and Andy as well. Carolina (10/31/2023 09:05:44 PDT): >Will do! Alison (10/31/2023 09:05:54 PDT): >Can we please specifically ask for whoever is reviewing to list out the IDs somewhere in the case so they can be pulled easily. Alison (10/31/2023 09:05:55 PDT): >Thanks! Kristina (10/31/2023 09:08:10 PDT): >13 Child Safety identified several accounts belong to Jeff. Last we checked, they didn't have MaCSA scores. > >#shareadoc comment https://docs.google.com/spreadsheets/d/1H4W5mnJGEyHo16HISO6wg78dzWdsRssGRYSDdpL_12s/edit# Alison (10/31/2023 09:13:37 PDT): >@Gabriel can we take a look here while ops is pulling the accounts from the video for why his accounts don't have a MASCA score given how often he is searching for/viewing this type of content. Malia (10/31/2023 09:18:17 PDT): >There are no MaCSA signals for views. Malia (10/31/2023 09:18:46 PDT): >There are MaCSA signals for likes and saves of content. And there are signals for searches *if* the search terms are in the high-precision list. Malia (10/31/2023 09:19:03 PDT): >We have not been able to include the wider set from the taskforce in this list as the precision is varying. Gabriel (10/31/2023 09:19:50 PDT): >Yes @Kristina and @Malia for these accounts >1/ should the MaCSA signals be triggered? is there some gap in what we are detecting? >2/ are there any additional human-driven signals on looking at these accounts which, if automated, would have led to these accounts being considered risky (1 or 2 points) Alison (10/31/2023 09:21:24 PDT): >+1 my point above was based on what I assume the behavior of his accounts are (viewing/searching) we should confirm first if the MaSCA signals should be triggered here (per @Gabriel I's point). Carolina (10/31/2023 09:28:22 PDT): >Also, just flagging that I'm using my test user account to surface the accounts, and also started being recommended some myself. Carolina (10/31/2023 09:28:32 PDT): shared: 377124407_284350133924228_7896967635058393494_n.png shared: 370241162_303782345840542_536791761734269577_n.png Carolina (10/31/2023 09:28:47 PDT): >I'm adding these to the investigation and will take them down as well Alison (10/31/2023 09:30:16 PDT): >Can you share the case here when you have it? Separately confirming that @Malia you are looking into Gabe's ask above? @Kristina Kristina (10/31/2023 09:32:12 PDT): >Yes I can look into this Swithin (10/31/2023 09:33:04 PDT): >Shouldn't these accounts also be auto enqueued for review based on account names esp pizzalink, kidseller? @Diego Carolina (10/31/2023 09:34:35 PDT): >C#4447847 PLTF EX-06012_0002 CONFIDENTIAL METANMAG-039-00000004 Alison (10/31/2023 09:35:58 PDT): >Great IDs are in the case @Gabriel Melissa (10/31/2023 09:43:30 PDT): >Also looks like a handful do have reactive user reports on the profile, but are sitting in the queues/pending Alison (10/31/2023 09:43:53 PDT): >Sorry what profile? Melissa (10/31/2023 09:44:15 PDT): >a handful of those in the case Diego (10/31/2023 09:47:27 PDT): >We have variations of this. Would be helpful to see what are the new names that are being used and we can add them to the TMS banks for enqueuing or auto actioning. Alison (10/31/2023 09:48:14 PDT): >Coming back to this: >1. @Carolina when we have it, please provide an update on the accounts that have been actioned/result of the fanout. >2. @Malia @Kristina @Gabriel put a doc here so we can centralize across the reporters accounts + IDs from the video https://docs.google.com/document/d/1eZa0GCrc1m_SyUyrNF1Vtv4r910dNQoKdSoq6brEvNs/edit #shareadoc Alison (10/31/2023 09:49:19 PDT): >Great can you please review them from the case shared above and add new content to the TMS banks if permissible. Carolina (10/31/2023 09:51:28 PDT): >Diego I'll include these insights in the case once I'm done with the investigation, but based on initial fan-outs, I'm seeing a lot of keyword obfuscation (i.e. "megha" or "megga" instead of "mega", and instead of "seller", using "sellar" or "zshaller") But also more obvious versions like, "pizza mega sellers" or "trusted cp seller". Rishi (10/31/2023 09:53:37 PDT): >Gabriel/Malia: for the accounts that are being surfaced here, should they be caught by MACSA? Or are there other CSE-I profile level classifiers that can catch these? Malia (10/31/2023 09:57:29 PDT): >We'll take a look and assess. I'm assuming your question here is on the recommended profiles and not JH's. Rishi (10/31/2023 09:57:53 PDT): >yes Rishi (10/31/2023 09:58:04 PDT): >Could we use https://fb.workplace.com/notes/311970791222605 for soft actions? Malia (10/31/2023 09:58:28 PDT): >Is this not live? Rishi (10/31/2023 09:59:07 PDT): >Reading the note, it seems like we're using it only to enqueue/UFAC Rishi (10/31/2023 09:59:57 PDT): >Hypothetically, we could use lower thresholds to filter out of recs Malia (10/31/2023 10:00:09 PDT): >Ah yes, ok. Gabriel (10/31/2023 10:00:56 PDT): >Yeah i think that's right. We didn't look into this for taskforce but might be an option; think we could probably get a precision number based on what's getting enqueued Carolina (10/31/2023 10:07:16 PDT): >@Malia /@Gabriel on the case the accounts that were recommended to Jeff are labeled as "JH AYSF", you'll notice the case now has about 172 IG accounts, the bulk of which I've found using keyword combinations on front-end and conducting fanouts to find device-linked IG accounts. >ok 1mk. > Rishi (10/31/2023 10:08:08 PDT): >Could we also use rough heuristics for eg, filter these out of recs. Gabriel (10/31/2023 10:09:40 PDT): and we PLTF EX-06012_0003 CONFIDENTIAL METANMAG-039-00000005 >worth a sidebar: I think this is how we should be doing nonrec more generally; will need a broader conversation incl policy and upstream teams who work on X-Ray image features Rishi (10/31/2023 10:10:56 PDT): >ok let me spin up a side chat Amit (10/31/2023 10:13:22 PDT): >i think this is a promising direction. are we going to expand on this line of thinking and actually eliminate borderline seeds from the rec selection process? Amit (10/31/2023 10:13:39 PDT): >is that something we are also discussing Swithin (10/31/2023 10:15:10 PDT): >Some of these do have a ton of stories. We should definitely remove them from seed sourcing. I'm also seeing some UFAC activity on the accounts and am unclear about what those mean Sayed (10/31/2023 10:15:32 PDT): >Can we also find out what the stated/predicted ages are for these recommended accounts? It can help inform some of how we are thinking about AYMF for teens. Amit (10/31/2023 10:20:49 PDT): >so is this change to seed sourcing live now or we need to do it? Rishi (10/31/2023 10:24:42 PDT): >It's not live. Once we align on which signals can/should be used, we should probably do both: (a) stop seed sourcing and (b) do a last pass ranking integrity filter. > >(a) will take time as it gets deep into the AYMF stack >(b) can be a quicker change Gabriel (10/31/2023 10:49:13 PDT): >Folks, it might be helpful to step back a little on IG recs; > >1/ we have shipped a bunch of things already we can validate they are working as intended > >2/ More broadly, AYMF and Content filtering are still not where we would want them to be. There are multiple levers we could pull but all would take some focus and both eng+policy > >3/ business outcomes needed and timelines are currently unclear; it's not obvious to me that pivoting back into war room mode across IGWB and XI teams and shipping some hotfixes now would either address the Jeff story issue (given timlines of this week) or set up teams for longer term success but I would like to get some SR steer on this cc @Alison Carolina (10/31/2023 14:43:46 PDT): >Hey all, following up with the latest enforcement figures from the GRO investigation on the AYSF case (C#4447847). > >In addition to the 12 accounts featured in Jeff's video, and the 204 IG accounts we disabled from preliminary fan-outs, GRO reviewed 1.6K users that were device linked to the original disabled accounts. We conducted subject element reviews (profile photo, username, and bio), and disabled 987 additional IG accounts for CSAM trade. > >In total, the investigation resulted in 1203 IG account takedowns. The majority of the users were based in India and engaging in trading/selling of telegram links. > >As a next step, we're going to collect the common keywords we spotted and will share with Diego and Jonny to inform further TMS banking efforts. > >Let us know if you have any questions or have any additional IDs for review. > >cc. @Andy @Alison Diego (10/31/2023 14:50:49 PDT): >Thank you, Carolina! PLTF EX-06012_0004 CONFIDENTIAL METANMAG-039-00000006