Meta & Children's Online SafetyChat: Up until last half, even Child Safety CI had no workstream set up to do bulk enforcement against FB Groups with clearly violating content in either CEI or MS
Thread View
████ name redacted. Different bars may be the same person or different people.
quoted in an excerpt in the archive.
Friday 23 April 2021
09:54████Anything you're aware of on this one @Sarah?
15:35████YES, we are actively working on a set of FoA wide Youth Principles to guide products but in the meantime, we have various implementation guidance docs on when we should gate users out of products and specific threshold guidance for our IIC classifier and Age models.
15:37████https://docs.google.com/document/d/1BN43J1FscB1rpUg4Ng08NFZVQiMY4ZyMgEeyR5a50cY/edit
15:37████Which prugs into: https://docs.google.com/document/d/1xW6Sg1sKO-0-MC_dh-95Mob23ZLfsOWWZQ3qBACmEAM/edit
15:39████Great, thanks! I'll start by pointing to the team to those to begin with.
15:39████And here's a Child Safety 101 guide for products that includes guidance on how to use the Age models for child safety: https://fb.quip.com/DZLFAibk1xPQ
15:42████Another side topic are there any workstreams in progress on minor sexualization groups we're missing?
I've been looking at groups with % of disabled accounts as part of some groups risk tiering work, and the query turns up a bunch of groups like "beauty boys of...." or "looking for a girlfriend/boyfriend age 9, 10, 11, 12, etc."
15:42████I think the way to think about it is the PIG guidance gives you a high level overview of whether or not your product qualifies as "high risk" "sensitive" or "core"
15:43████Johanna mentioned she was a part of something, so I could just pass the queries to her as a starting point
15:43████and it you're high risk or sensitive, it's very much recommended that we include child safety mitigations
15:45████I'm So grad you're doing this! I haven't had the bandwidth to tackle this in earnest to whip up the XFN support but this has been on my TO-DO list for a while now
15:45████FB Groups just completed an integrity sprint in response to the capitol riots
15:45████and as part of that workstream, I was able to identify projects and PM led initiatives that we should piggy back on to clean up the minor sexualization space
15:46████bottom me, no, we're not very good at proactively detecting and taking down MS Groups and it's a big problem we need to tackle sooner rather than later b/c they areVERY publicly visible and leads to PR fires and stakeholder complaints and they get deployed against us as proof points that we're not taking child safety seriously
15:47████Up until tast half, even Child Safety CI had no workstream set up to do bulk enforcement against FB Groups with clearly_ violating content in either CEI or MS
15:47████OK, great, Groups Integrity is within my team's sub-pillar now though the responsibility is now split across more functional teams like Eligibility / Entity & Actor Understanding / Enforcement / De- amplification. We are working with Entity & Actor Understanding on risk tiering for Groups, as our criteria has been very blunt in the past (i.e. >1000 member groups can't have open rooms).
15:48████We also didn't have an implicit minor sexualization policy stood up until this half
15:48████we now do!
15:48████so we can start enforcing against them more aggressively
15:48████Child safety was not one of our org's priorities, as they focus on higher prevalence issues historically like misinformation, civic/health/etc. issues, nudity/sexual solicitation, but we are looking to get it on the radar a bit more with risk tiering
15:49████oh yes, Antigone's guidance to me was for Groups, right after misinfo and civic, Child safety should be prioritized b/c child safety continues to be used as a wedge issue to fight e2ee
15:50████and the way we keep getting distinguished from the Apple's of the world is that we're connected to the social graph and a public surface, i.e. FB Groups
15:50████where pad actors can signal one another
15:50████and 100k for and connect with kids
15:50████and then enter a private messaging thread
15:50████so it's one seamless flow of discovery-->connection-->harm
15:50████if you watch any of the congressional testimony, we get_creamed on this point while Apple gets off scot free
15:51████Right, makes sense. Good to know you are raising it from your side then at the moment. Some of the child safety related signals like % accounts disabled for child safety might be useful inputs/features into the risk tiering models, so I'll be putting together some options for inclusion from our end
15:53████oh, PLUS, we currently have liability risk under existing law for minor sex trafficking, sex trafficking and prostitution in general so yes, good to prioritize that problem area as well
15:53████awesome to hear you're working on Groups Integrity!! Who are your main PM partners so that I can help advocate prioritizing this with them as well?
15:56████(https://www.internalfb.com/profile/view/594874457) is PM on Entity & Actor Understanding.
Tim's Leam owns the risk tiering modeling workstream that is about to begin
15:58████(https://www.internalfb.com/profile/view/1569364532) is the PM on my team Rooms/Events Integrity who filled role. We are now acting kind of like customers for Entity & Actor Understanding's risk tiering and have got Rooms as an example case for their MVP. E.g. if we wanted to give Rooms back to more Groups, we need a good understanding of all the risk and capabilities to filter out low risk from high risk, particularly for minor safety
15:59████Our team also is an integrity starting point for new products on FB App in general, so we are seeing the same issue come in with say Chats in Groups, where we need better risk tiering to enable these features safely
16:02████there's a mini summit on Monday and Tuesday that you might be interested in let me share here
16:03████https.//TD.workplace.com/groups/233981627049481/permalink/1149093758871592/
16:03████heads up too, sadly I won't be working in this space much longer. I decided to switch teams within FB to a product team in the Assistant org, so I'll be handing over much of this work. I'm formally switching sometime next week, but will make sure these problems are on people's roadmaps before I fully step back
16:07████Shared an image
16:08████We'll be so SAD to lose you here BUT I'm sure exciting things await you in Assistant + I'm sure our paths will cross b/c you can't shake child safety from most products!
16:08████congrats on the move!!!
16:11████yah, I'm feeling the same way. Really enjoyed working with both of you and everybody else on Rooms and child safety!
(4/23/2021 16 14:27 PDT):
I remembered seeing on MVP doc for the risk tiering a question in relation to prioritizing child safety: "open question now do we reconcile this against the fact that child safety is not prioritized overall in our subpillar?"
16:15████just so you're aware that this definitely be worth something you advocating for to make sure it gets attention
16:18████awesome! If you'd feel comfortable and have good relationships with and would you mind introducing us?
16:32████ven! let me start a chat thread.
is on PTO til Tuesday, but there is a monthly business review for team in relation to risk tiering on Monday afternoon with our sub-pillar leadership. Your input ngnt give him another voice to include if useful
16:35████^ will set a reminder to start this Monday morning
16:36████thank you!! In the meantime, have a wonderful weekend!
16:36████you too!