"We sourced suicide and self-injury admission content from an external partner"
"Hypothesis: We would not provide resources to any users or delete violating content as we expected neither user nor proactive reports on them"
[Problem #1 – Detection Test]:
• Problem: "We did not provide resources to users who posted suicide or self-injury admission content or resources were provided late"
• Proximate: "Content was neither user reported nor proactively enqueued"
• Proximate: "We had to rely on user reports as content was not proactively enqueued"
• Intermediate: "Content was neither user reported nor proactively enqueued"
• Root Cause: "Violating content scored significantly lower than the established threshold
Enforcement 4.b.ii (Unintended issue)"
• Root Cause: "Currently, we are keeping thresholds high to keep volumes low due to the resource constraints on the number of reviewers we have to review proactively reported content
Enforcement 4.b.i (tradeoff)"
[Problem Summary]:
• Problem: "Not providing resources to users at all or on time"
• Resulting Risk: "Increased risk of imminent harm"
• Work in Progress: "Classifier work expanding to cover markets in Q2"
• Work in Progress: "Resource expansion for NA being discussed"