"this is the worst of the worst stuff that gives you nightmares, over which children have taken their own lives :("

Adam Mosseri: "Kevin and Mikey are uncomfortable scanning Direct message data, which I didn't realize until last week, though they're supportive of using meta data. I'd like to make the value of scanning messages directly clear — do you guys have any idea how we might quantify the value for problems like grooming more specifically? There is the added complexity of being inconsistent across surfaces, but I thought I'd start by trying to clearly articulate the up-side." Guy Rosen: "How about we organize a shadowing session for you & them to see the kind of stuff CO / investigators find? It helps to paint a picture. For grooming - or rather, the broader bucket we're calling IICIICInappropriate Interactions with ChildrenA classifier threshold and content category for interactions between adults and children indicative of grooming or exploitation, referred to internally as "a.k.a. 'grooming'". The source document puts the scale at "~500k victims per DAY in English markets only", three times Messenger's prevalence, while calling it "a preliminary metric" and adding "we expect the true situation is worse".Full entry ↗ (inappropriate interactions with children) splits roughly into these tiers (going from least worst to worst worst): • Tier 3: non-policy violating creepy behavior e.g. adult contacting 100s of minors on messenger with "hey sexy" • Tier 2: online sexual harassment e.g., sending photos of (adult) genitalia to a minor • Tier 1: soliciting or real-world harm c. soliciting CEICEIChild Exploitative ImageryMeta's policy and enforcement category for imagery that sexually exploits children, tracked separately from Minor Sexualization and from Inappropriate Interactions with Children. The source document has executives weighing whether to limit the CEI figure published in the Community Standards Enforcement Report to Facebook only, "to avoid a step-change some time in the future when we lose access to private messaging", at a point when by their own estimate "almost 90%" of egregious CEI content was on Messenger.Full entry ↗ b. contact offense (suggesting an in-person meeting) a. content sextortion (using obtained CEI to force further acts) <-- this is the worst of the worst stuff that gives you nightmares, over which children have taken their own lives :( Of the above, we can't do much of anything on Tier 1 & 2 without proactive scanning. Also, because kids are either embarrassed and/or think they're in a real relationship, this tends to go unreported. There are also other things which we should discuss but I am okay to tackle the child stuff first, e.g. financial scams which aren't necessarily targeted at minors, but can also include financial sextortion (getting intimate images and then asking for money). These are typically run as little business out of (usually) West Africa, and often intersect with impersonation (fake attractive female, military and public figures, celebrities - even posing as Mark or Wehner to scam people on a 'Facebook lottery', etc. -- the latter we can probably tackle from the impersonation work though)." "The meta way I think about it is that both FB & IG are discovery services, not just communication services: you can do discovery e.g. search for girls a certain age in a certain place, etc. and then engage them. (At least on WA you can't do the discovery bit.) I think we have to be extra protective of what's happening on discovery services because otherwise they are basically massive 'victim discovery services'."