"The IG Wellbeing team does not sufficiently enforce bi-directional filtering between minors and potential violators on recommendation surfaces. As a result, the potential violators may bring harm to minors through the Reels surface."

"Overall, while IG Reels has generally implemented the SOX controls required for ad products, several areas in content integrity and product features need significant improvements. IA identified 4 High risk, 20 Medium risk, and 3 Low risk observations during the audit. Based on the overall health of the processes, an 'Orange' rating was assigned to this report." "Insufficient reduction of interactions between minors and potential child safety violators." "Private Reels Surfaced to Unconnected Users The IG Reels Product team has not documented the decision and mitigations related to the logging and monitoring of private Reels viewed by unconnected users. As a result, privacy risks associated with user settings may not be identified timely or enforced appropriately. For example, on a sample date, after considering the 12-hour caching time, 1.4% of VPVs on private Reels are viewed by unconnected users. 3.75% of private accounts with Reels VPVs were impacted." "The IG Reels Product team has not documented the decision and mitigations related to the logging and monitoring of private Reels viewed by unconnected users. As a result, privacy risks associated with user settings may not be identified timely or enforced appropriately." "Potential Violators Able to Interact with Unconnected Minors The IG Wellbeing team does not sufficiently enforce bi-directional filtering between minors and potential violators on recommendation surfaces. As a result, the potential violators may bring harm to minors through the Reels surface. On a sampled day tested, 1) Minors and potential violators can view and reply to comments from each other on a third user's Reels. 2) Minors can see and view Reels created by potential violators on the unconnected surfaces (i.e., Explore and Reels tab) and vice versa. Out of all VPVs made by potential violators, 12.23% were on unconnected minors (i.e., not friend, follower, or followee). The percentage is 0.37% vice versa." "The IG Relevance Integrity team does not adequately identify or monitor trends in Reels. As a result, potentially harmful Reels trends may get wide distribution, and the teams may not be able to intervene timely." "The IG Relevance Integrity team does not monitor the efficacy (e.g., False Positive or Negative Rates) of the demotion model for objectionable content (e.g., regulated good, drugs, and violence). As a result, deteriorating model performance over time may go undetected." "The IG Relevance Integrity team does not regularly monitor the importance of the inputs to the objectionable content demotion model (e.g., regulated good, drugs, and violence) to assess its reasonableness. As a result, significant changes in the importance of inputs may go undetected, leading to degraded model performance." "IG Relevance Integrity team has not planned for human intervention when the scheduled daily model training fails consecutively for a few days. As a result, the model may not be trained with the latest production data leading to unsatisfactory performance. For example, in July 2021, only 7 days out of 31 days had successful training. For the other days, the training data table was blacklisted due to upstream data availability issues that were not resolved." "The IG Reels Engineering team does not monitor the percentage of exceptions in blocking Tags and @Mentions based on user settings. As a result, the team may not be able to act timely when the user settings are not honored." "The IG wellbeing team currently does not have standard governance processes for IG Reels comment bullying and offensive audio name models. Specifically, the following are missing: 1. Monitoring of anomalies for important features 2. Monitoring of model performance 3. Training, refreshing, and deprecating of production model As a result, non-maintenance of IG well-being bullying production ML models may result in models not performing as expected, which may cause bad comments not filtered/covered properly."