Guy Rosen: "because content outside of promotion did not violate our policies, we had no detection or enforcement in place at all for any other content, and most worryingly the content was easy to discover through search, hashtags, IG Explore, etc."

"Instagram suicide and self-injury (SSI) issues reveal we are behind where we need to be" "We've always removed content that promotes SSI, but at the advice of mental health experts, we've allowed suicide admission content as it provides an outlet for people to cope and get support. Following the tragic suicide of a 14-year-old girl in the UK that her family linked to her access to self-harm content, Instagram has been under intense scrutiny over the last few weeks around our policies and enforcement." "Policy: after further consultation, we decided that graphic cutting images should be removed as the potential for modeling, normalizing, or triggering self-harm in others outweighs the value of sharing the content. Non-graphic self-mutilation or healed cuts imagery will be left up but blurred / marked as sensitive and removed from discovery." "Enforcement: because content outside of promotion did not violate our policies, we had no detection or enforcement in place at all for any other content, and most worryingly the content was easy to discover through search, hashtags, IG Explore, etc. We have a swat team across IG Wellbeing, Community Integrity and Community Operations sprinting to clean up hashtags, clean up content, apply warning screens, and reduce visibility of potential SSI content from discovery surfaces." "We also really need to build proactive detection for content. This means ensuring we have the review flows set up right to label the content we're finding, and training our classifiers." "To me, this is a painful lesson on what we mean when we say areas are at 'risk' in our maturity framework. Why was SSI in this state? As some folks may remember, in 2017 we had a lockdown around FB Live in particular, focusing on getting help to people expressing suicidal thoughts. The work we did then has resulted in hundreds of wellness checks from first responders that has saved lives. In retrospect, that good work gave us a false sense of security on our SSI work, and as we ramped up our teams we chose to first put people first on other critical areas that sorely needed attention, such as child safety, terrorism and drugs. This half we rebooted our SSI efforts, and literally days before this became a full-blown fire, we were in the first roadmap review for the new CI SSI team – fittingly, the key takeaway was that the space is broader and more complex than the 'imminent harm' work we'd historically done. I'm glad we have a team ready to jump in, but it's painful to see the gap we left."