Instagram Reels Assurance Audit Report

Raw text

Transcribed text as recorded, including headers and formatting fragments.

Instagram Reels Assurance Audit Report Company confidential - for intended recipients only Martin Schneider Anna Tchernina Christophe Yanan Pratima November, 2021 Meta HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000035 PLTF EX-04330_0001 TABLE OF CONTENTS 1. Executive Summary 2. Detailed Results and Management Action Plan 3. Appendix - Communication Channels - Audit Report Rating Guidelines - Observation Remediation Guidelines Company confidential - for intended recipients only 2 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000036 PLTF EX-04330_0002 EXECUTIVE SUMMARY Objective: Instagram (IG) Reels provides users with the ability to create short-form videos that can be discovered by a wide range of unconnected viewers. As Meta focuses on competing in the short-form video market and incentivizes creators to produce original content, Reels has become one of the top priorities for the company. IG teams implemented a series of controls to detect and enforce on policy-violating and low-quality content, enable music and video sharing in compliance with licensing rights, and provide a safe and healthy platform for users. In 2022, all new videos created by users will be leveraging Reels. Given its significance, Internal Audit (IA) partnered with IG Reels cross-functional teams to perform an assurance audit. Scope: IA assessed the design and operating effectiveness of controls in the following areas: - Ads Revenue: SOX readiness controls, including ads delivery and logging. - Content Integrity: filtering and reduction of the distribution of problematic, low-quality, and un-original content. Note that the Community Standards and Content Monetization policies are out of scope for this audit. - Key Product Features: usage of audio library, audio attribution, wellbeing (i.e., youth safety, comments), and content sharing restrictions. - Metrics and Monitoring: accuracy and monitoring of key product metrics (e.g., new producers participation rate)*. Results Summary: Significant improvements needed Overall, while IG Reels has generally implemented the SOX controls required for ad products, several areas in content integrity and product features need significant improvements. IA identified 4 High risk, 20 Medium risk, and 3 Low risk observations during the audit. Based on the overall health of the processes, an "Orange" rating was assigned to this report. Key observation themes are summarized as below: - Insufficient misinformation measurement and monitoring. - Insufficient monitoring for private content viewed by unconnected users, content trends, and demotion model performance. - Insufficient reduction of interactions between minors and potential child safety violators. - Non-compliant use of music by restricted business accounts or within unlicensed regions. Company confidential - for intended recipients only 3 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000037 PLTF EX-04330_0003 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: IG Recommendation Guidelines Observation/Risk Description: Insufficient Measurement and Monitoring of Misinformation on Reels Surface Rating: High The IG Wellbeing Accurate Information team has not established measurement and monitoring for misinformation on Reels. As a result, Reels content with potential misinformation may not be adequately enforced. For example, potential misinformation contents reported by users have low 3PFC enqueue and review rates for IG Reels. Out of all user-reported contents for misinformation, the 7-day average for IG overall enqueue rate is 2.3%. The 7-day average for IG overall review rate compared to enqueued is 3.2% and IG Reels is 0.4%. Management Action Plan: 1. Perform an assessment to identify proper measurement for misinformation (e.g., prevalence, review rate) and determine a reasonable monitoring threshold for IG Reels. 2. Establish monitoring based on the newly defined metrics. Due Date: 1. 1/31/2022; 2. 3/31/2022* Owner: Glenn (Engineering Manager(EM), IG Well-being (IGWB) Accurate Info); Nischal (Product Manager(PM), Central Integrity (CI)) * Due to complexity of defining an appropriate measurement for misinformation, the IGWB Accurate Info team and the CI team have requested a 60-day extension to IA's standard 60-day remediation guideline for High risk observations. Company confidential - for intended recipients only 4 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000038 PLTF EX-04330_0004 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Recommendability Observation/Risk Description: Private Reels Surfaced to Unconnected Users Rating: High The IG Reels Product team has not documented the decision and mitigations related to the logging and monitoring of private Reels viewed by unconnected users. As a result, privacy risks associated with user settings may not be identified timely or enforced appropriately. For example, on a sample date, after considering the 12-hour caching time, 1.4% of VPVs on private Reels are viewed by unconnected users. 3.75% of private accounts with Reels VPVs were impacted. Management Action Plan: 1. Document a privacy decision related to monitoring criteria and cache TTL for IG Reels tab. 2. Investigate the exceptions identified, make remediation plan and implement the plan. 3. Establish monitoring and alert over the invalid media percentage. Due Date: 1. 12/31/2021; 2 & 3. 2/28/2022* Owner: Jennifer (Privacy Program Manager); Wai (Software Engineer(SE)) * Due to the timeline required for privacy review and complexity of investigation, the Privacy team and the IG Reels Product team has requested a 1-month extension to IA's standard 60-day remediation guideline for High risk observations. Company confidential - for intended recipients only 5 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000039 PLTF EX-04330_0005 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Wellbeing Observation/Risk Description: Potential Violators Able to Interact with Unconnected Minors Rating: High The IG Wellbeing team does not sufficiently enforce bi-directional filtering between minors and potential violators on recommendation surfaces. As a result, the potential violators may bring harm to minors through the Reels surface. On a sampled day tested, 1) Minors and potential violators can view and reply to comments from each other on a third user's Reels. 2) Minors can see and view Reels created by potential violators on the unconnected surfaces (i.e., Explore and Reels tab) and vice versa. Out of all VPVs made by potential violators, 12.23% were on unconnected minors (i.e., not friend, follower, or followee). The percentage is 0.37% vice versa. Management Action Plan: 1. Investigate the exceptions identified and make the plan for remediation. 2. Execute the remediation plan accordingly. Establish monitoring and alert using internal standards. Due Date: 1/31/2022 Owner: Haoyang (SE, IGWB Teen Safety Team); Mingliang (Data Scientist (DS), Analytics Instagram Well-being DS Team) Company confidential - for intended recipients only 6 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000040 PLTF EX-04330_0006 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: IG Recommendation Guidelines Observation/Risk Description: Insufficient Identification and Monitoring of Trends Rating: High The IG Relevance Integrity team does not adequately identify or monitor trends in Reels. As a result, potentially harmful Reels trends may get wide distribution and the teams may not be able to intervene timely. Management Action Plan: 1. Determine how the current trend identification system owned by Relevance team can be used for integrity and wellbeing monitoring purposes and establish the plan to include additional signals in H1 2022. 2. Enhance trend identification process and system. Establish metrics to monitor the healthiness of trend and prevalence. Establish comparison for trend identification between recommendation and integrity technologies. Due Date: 1. 1/31/2022; 2. 6/30/2022* Owner: Ming (EM, IG Relevance Integrity); Martin (EM, IG Reels Inspiration) --- Business Area: IG Recommendation Guidelines Observation/Risk Description: Insufficient Monitoring Of Demotion Model Efficacy Rating: Medium The IG Relevance Integrity team does not monitor the efficacy (e.g., False Positive or Negative Rates) of the demotion model for objectionable content (e.g., regulated good, drugs and violence). As a result, deteriorating model performance over time may go undetected. Management Action Plan: Establish monitoring for key integrity model efficacy. Determine at which point model refresh is necessary. Due Date: 3/31/2022 Owner: Ming (EM, IG Relevance Integrity); Lucas (Engineer, IG Relevance Integrity) * Due to the complexity associated with implementation of trend identification and monitoring based on new inputs, the IG Relevance Integrity team has requested a 5-month extension to IA's standard 60-day remediation guideline for High risk observations. Company confidential - for intended recipients only 7 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000041 PLTF EX-04330_0007 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: IG Recommendation Guidelines Observation/Risk Description: Undefined Goals For Misinformation Model Detection Rates Rating: Medium The IG Wellbeing Accurate Information team and the IG Reels Surface team have not worked with the Central Integrity Team to define a goal for the misinformation model detection rate on IG Reels or documented the roadmap for subsequent detection model enhancement. As a result, Reels content with misinformation may go undetected. Management Action Plan: 1. IGWB Accurate Info team & Surface team performs an analysis to determine the reason of low misinformation detection rate. 2. Central Integrity team works with Surface team to establish scoring standards for the new Misinformation Maturity Framework. Define the Maturity Level for Reels. 3. IGWB Accurate Info team collaborates with Central Integrity team to determine if Reels specific features need to be implemented for feature engineering in the existing model. Due Date: 1 & 2. 3/31/2022; 3. 6/30/2022* Owner: Glenn (EM, IG Well-being Accurate Info); Ece (Project Manager, IG Feed & Stories Relevance); Nischal (PM, CI) * Due to the complexity of model update and feature engineering, the IGWB Accurate Info team and the CI has requested a 3-month extension to IA's standard 120-day remediation guideline for Medium risk observations. Company confidential - for intended recipients only 8 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000042 PLTF EX-04330_0008 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: IG Recommendation Guidelines Observation/Risk Description: Insufficient Monitoring of Demotion Model Input Rating: Medium The IG Relevance Integrity team does not regularly monitor the importance of the inputs to the objectionable content demotion model (e.g., regulated good, drugs and violence) to assess its reasonableness. As a result, significant changes in the importance of inputs may go undetected, leading to degraded model performance. Management Action Plan: Establish monitoring over feature importance change, review model importance on a regular basis to identify significant increase and decrease and determine if the change is expected. Due Date: 3/31/2022 Owner: Ming (EM, IG Relevance Integrity) --- Business Area: IG Recommendation Guidelines Observation/Risk Description: Inadequate Intervention of Model Training Failure Rating: Medium IG Relevance Integrity team has not planned for human intervention when the scheduled daily model training fails consecutively for a few days. As a result, the model may not be trained with the latest production data leading to unsatisfactory performance. For example, in July 2021, only 7 days out of 31 days had successful training. For the other days, the training data table was blacklisted due to upstream data availability issues that were not resolved. Management Action Plan: 1. Define the internal standard of when a human intervention should be initiated when model training fails consecutively. 2. Establish automated alert based on the internal standard. Due Date: 3/31/2022 Owner: Ming (EM, IG Relevance Integrity) Company confidential - for intended recipients only 9 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000043 PLTF EX-04330_0009 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Reels Quality Standards Observation/Risk Description: Insufficient Monitoring Of Quality Model Performance Rating: Medium The IG Recommendation Foundation team does not monitor the performance (e.g., Precision, Recall) of the Reels Quality Models. As a result, deteriorating model performance over time may go undetected. Management Action Plan: Establish monitoring of key quality model performance. Determine at which point model refresh is necessary. Due Date: 3/31/2022 Owner: Raymond (SE, IG Recommendation Foundation) --- Business Area: Reels Quality Standards Observation/Risk Description: Missing Owner for Text-Heavy Model Rating: Medium The IG Recommendation Foundation team has not designated an owner for the Text-Heavy Model, or established a process for model maintenance. As a result, users may see excessive text-heavy content on Reels. Management Action Plan: 1. Determine if the model is continuously used by the Recommendation Team. 2. Establish and implement criteria for model training and performance. Due Date: 1. 1/31/2022; 2. 4/30/2022* Owner: Raymond (SE, IG Recommendation Foundation) * Due to competing priorities, the IG Relevance Integrity team has requested a 1-month extension to IA's standard 120-day remediation guideline for Medium risk observations. Company confidential - for intended recipients only 10 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000044 PLTF EX-04330_0010 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Reels Quality Standards Observation/Risk Description: Unmonitored Demotions Rating: Medium The IG Recommendation Foundation team does not log the demotion multiplier (i.e., intensity) and volume for low quality content. As a result, the demotions applied cannot be measured or compared against the original expectation. Management Action Plan: 1. Assess the cost-benefit of different logging approaches and construct a plan for adding logging. 2. Implement the plan to add logging of demotion multiplier applied by the quality models on Reels video. Due Date: 1. 1/31/2022; 2. 4/30/2022* Owner: Raymond (SE, IG Recommendation Foundation); Matthew (SE, IG Recommendation Foundation) --- Business Area: Reels Quality Standards Observation/Risk Description: Insufficient Prevalence Monitoring Of Non-genuine Reels Rating: Medium The IG Recommendation Foundation team does not monitor the combined prevalence of non-genuine Reels videos across key quality areas (e.g. text-heavy, watermarked, with margins, static videos). As a result, non-genuine content may be recommended, discouraging users from creating original content using the Reels functions. Management Action Plan: Establish composite metrics and set up a continuous monitoring dashboard. Due Date: 2/28/2022 Owner: Raymond (SE, IG Recommendation Foundation) * Due to competing priorities, the IG Relevance Integrity team has requested a 1-month extension to IA's standard 120-day remediation guideline for Medium risk observations. Company confidential - for intended recipients only 11 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000045 PLTF EX-04330_0011 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Reels Quality Standards Observation/Risk Description: No Demotion on Blurry Content Rating: Medium The IG Recommendation Foundation team does not have demotion for blurry content. As a result, this may lead to poor user experiences on Reels. Management Action Plan: 1. Establish a plan to build blurry content demotion. 2. Implement the demotion in production. Due Date: 3/31/2022 Owner: Sulman (Director, Software Engineering) --- Business Area: Content Library and Audio Attribution Observation/Risk Description: Non-Compliant Use of Music By Restricted Business Accounts Rating: Medium The IG Reels Product team does not consistently enforce Meta Inc. licensed music rules on restricted business accounts (e.g., Local Businesses, Brands & Products, Companies & Organizations). As a result, Meta Inc. may not comply with the licensing Terms and Conditions (T&C). For example, on a sample date, 29.8% of Reels created by restricted business accounts used licensed music from the audio library, which violates the T&C with the music partners. Management Action Plan: 1. Identify or establish a validate logging table of business category. 2. Ensure the licensed music restriction is applied to business accounts. Due Date: 3/31/2022 Owner: Kevin, SM (IG Reels Audio Eng); Sara (FB SE Music Partner Experience) Company confidential - for intended recipients only 12 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000046 PLTF EX-04330_0012 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Content Library and Audio Attribution Observation/Risk Description: Unlicensed Use of Music in Various Countries Rating: Medium The IG Reels Product and Music Partner Experience teams do not effectively prevent users from using or hearing a song in countries where it is not licensed to Meta Inc. As a result, Meta Inc. may not comply with the licensing T&Cs. Management Action Plan: 1. Perform investigation over the media able to use or hear music in countries where it is not licensed. 2. Implement monitoring on Reels to ensure country gating is working as intended. Due Date: 3/31/2022 Owner: Kevin, SM (IG Reels Audio Eng); Sara (FB SE Music Partner Experience) --- Business Area: Content Library and Audio Attribution Observation/Risk Description: Insufficient Monitoring on Audio Type Data Field Rating: Medium The IG Reels & Camera Data Engineering team does not monitor the value of the audio type data field (e.g., licensed song, remix, original song) within the IG media events table. As a result, any spike in invalid values would not be detected and investigated. For example, for a sample period from 31 days, null values in the audio type field exceeded the agreed-upon 3% internal threshold on 14 different days. Management Action Plan: 1. Implement monitoring on the audio type data field based on internal criteria. (i.e. "audio_type" in table of fct_ig_media_events table) 2. Establish investigation procedure when an alert is generated. Due Date: 3/31/2022 Owner: Chau (DE, IG Reels & Camera DE Team) Company confidential - for intended recipients only 13 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000047 PLTF EX-04330_0013 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Content Library and Audio Attribution Observation/Risk Description: Missing Logging For Muted Downloaded Reels Rating: Medium The IG Reels Audio Engineering team does not log whether a downloaded version of Reels is muted when the criteria for muting are met (e.g., using licensed music, longer than 15 seconds). As a result, IG cannot detect when users are downloading unmuted Reels against the license T&Cs. Management Action Plan: Add a logging field of whether a Reels was muted when user download it. Due Date: 3/31/2022 Owner: Kevin, SM (IG Reels Audio Eng) --- Business Area: Content Library and Audio Attribution Observation/Risk Description: Insufficient Logging of Copyrighted Video and Audio Rating: Medium The IG Reels Audio Engineering team does not log whether Reels are muted or blocked on the user phone due to copyright restrictions. As a result, the product team cannot validate if copyright restriction is working as expected on the surface. Management Action Plan: 1. IG Reels Audio Engineering team to add a logging field whether a Reels audio is muted at the mobile interface when the user is using the copyright audio. 2. IG Reels Audio Engineering team to add a logging field whether a Reels video is blocked when the user is using the copyright video, whose ownership is already claimed. Due Date: 3/31/2022 Owner: Kevin, SM (IG Reels Audio Eng); Mihir (SE, IG Reels Audio ENG) Company confidential - for intended recipients only 14 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000048 PLTF EX-04330_0014 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Audio Page Observation/Risk Description: Reported Audio Page Not Reviewed Rating: Medium The IG Surface Operations team does not review potential violating audio pages (e.g., IG page displaying all Reels associated with the same audio track) reported by users. As a result, offensive audio pages may not be properly enforced, leading to bad user experience. For example, IA uploaded 5 Reels samples, renamed the audios to offensive names and reported the potential violations. 1 sample was marked as non-violating by the human reviewer and the other 4 samples were not reviewed. Each of the 4 samples accumulated over 1000 VPV within a day. Management Action Plan: 1. Onboard Audio Page to the Central Ranker for human review prioritization 2. Monitor the review rate regularly against internal standards Due Date: 3/31/2022 Owner: Twinkle (IG Surface Operations); Ming (SM Manager, IG Relevance Integrity) --- Business Area: Wellbeing Observation/Risk Description: Lack of Monitoring Over Ineffective Block Rating: Medium The IG Reels Engineering team does not monitor the percentage of exceptions in blocking Tags and @Mentions based on user settings. As a result, the team may not be able to act timely when the user settings are not honored. Management Action Plan: IG Reels Engineering team to establish monitoring on Tag and @Mention settings to identify exceptions related to disabling function of Tag and @Mention. Due Date: 1/31/2022 Owner: Charles Sismondo (EM, IG Well-being Bully Team) Company confidential - for intended recipients only 15 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000049 PLTF EX-04330_0015 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Wellbeing Observation/Risk Description: Lack of Governance on IG Reels Comment Bullying and Offensive Audit Name Models Rating: Medium The IG wellbeing team currently does not have standard governance processes for IG Reels comment bullying and offensive audio name models. Specifically, the following are missing: 1) Monitoring of anomalies for important features 2) Monitoring of model performance 3) Training, refreshing and deprecating of production model As a result, non-maintenance of IG well-being bullying production ML models may result in models not performing as expected, which may cause bad comments not filtered/covered properly. Management Action Plan: 1. Transfer the models from IG Wellbeing team to Central Integrity (CI) team. 2. CI will develop feature store alerts to monitor the failure rates and monitor production model performance through policy agnostic metrics. Due Date: 1. 3/31/2022; 2. 5/31/2022* Owner: Yuanfeng (EM, Borderline and Exploratory Technologies Team); Kevin (EM, Hate and Violent Speech Team); Charles Sismondo (EM, IG Well-being Bully Team) * Due to the complexity of cross-functional collaboration and ownership transition, the CI team has requested a 2-month extension to IA's standard 120-day remediation guideline for Medium risk observations. Company confidential - for intended recipients only 16 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000050 PLTF EX-04330_0016 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: SOX - QA Monitoring Observation/Risk Description: Insufficient QA Testing Scenarios Rating: Medium The Reels Product Quality team does not sufficiently test billable event logging scenarios for Instagram Reels. As a result, Meta Inc. may incorrectly record billable events leading to inaccurate IG revenue. Management Action Plan: The QA team will add test scenarios to the existing Instagram Reels QA test plan. Due Date: Closed Owner: Marco (QA Engineering Lead, Product Quality) --- Business Area: SOX - Volume Monitoring Observation/Risk Description: Insufficient Ads Revenue Monitoring Rating: Medium The Instagram Metrics Platform team does not sufficiently monitor week-over-week ads revenue changes for Instagram Reels. As a result, Meta Inc. may inaccurately record IG revenue. Management Action Plan: The Instagram Metric Platform team will implement monitoring for Instagram Reels ads revenue. Due Date: Closed Owner: Heng (SE, IG Metric Platform Team) --- Business Area: SOX - Unit Test of Impressions Observation/Risk Description: Incomplete Unit Tests Rating: Medium The Instagram Metrics Platform team does not sufficiently perform unit tests to validate the accuracy of billable impression and video event logging. As a result, Meta Inc. may incorrectly record billable events leading to inaccurate IG revenue. Management Action Plan: The Instagram Metric Platform team will implement unit tests for Instagram Reels impression and video event logging. Due Date: Closed Owner: Vincent (SE, IG Metric Platform) Company confidential - for intended recipients only 17 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000051 PLTF EX-04330_0017 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Reels Quality Standards Observation/Risk Description: Redundant Models For Watermark Detections Rating: Low The IG Recommendation Foundation team uses almost identical keywords lists in its online and offline models for watermark detection that both serve the same purpose. As a result, the team may spend unnecessary efforts in model maintenance and management. Management Action Plan: Consolidate offline & online models to one robust model Due Date: 1/31/2022 Owner: Raymond (SE, IG Recommendation Foundation) --- Business Area: Metrics Calculation Observation/Risk Description: Lack of Monitoring Over Lost Data Entries Rating: Low The IG Data Engineering team does not monitor the new producer data entries that are dropped unexpectedly during the data transmission from pre-staging table to the staging table. As a result, data completeness issues may go undetected leading to incorrect New Producer Participation Rate metric decline. Management Action Plan: Implement data completeness check to ensure the amount of dropped entries in calculating the metric is within acceptable range. Due Date: 1/31/2022 Owner: Erin (IG DE) Company confidential - for intended recipients only 18 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000052 PLTF EX-04330_0018 DETAILED RESULTS AND MANAGEMENT ACTION PLANS Business Area: Integrity Signal Observation/Risk Description: Inadequate Transparency of Signal Coverage Rating: Low The IG Recommendation team has not requested automated alerts when traffic coverage for integrity signal drops and is notified manually by the Signals and Platform (SIP) oncall when the incoming signals have issues. As a result, the IG Reels surface team may not be aware of upstream data issue timely. Management Action Plan: 1. Increase transparency around signal traffic and improve alert reliability for surface owners 2. Automatically triage to notify appropriate Point-of-Contacts (PoCs) Due Date: Closed Owner: Yuchang (IGWB Signals & Insights Platform) Company confidential - for intended recipients only 19 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000053 PLTF EX-04330_0019 APPENDIX I - COMMUNICATION CHANNEL [Org chart / reporting-line diagram — Product Engineering, Technical Program Management (TPM), Product Management (PM) functions. Most personnel names redacted; box-diagram layout not reliably machine-transcribable.] Company confidential - for intended recipients only 20 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000054 PLTF EX-04330_0020 APPENDIX I - COMMUNICATION CHANNEL [Org chart / reporting-line diagram — Youth & Communities, Product Analytics, Music & Video, Privacy functions. Visible named individuals include Chris Cox, Adam Mosseri, Pavni Diwanji, Tom Alison, Michel Protti, Tomas Kabbabe; most other personnel names redacted. Box-diagram layout not reliably machine-transcribable.] Company confidential - for intended recipients only 21 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000055 PLTF EX-04330_0021 APPENDIX I - COMMUNICATION CHANNEL [Org chart / reporting-line diagram — Central Integrity, Global Operations, Music Business Development, Legal, Product Policy, Finance & Accounting functions. Visible named individuals include Javier Olivan, Justin Osofsky, Jennifer Newstead, Nick Clegg, David Wehner, Guy Rosen, Joel Kaplan, Susan Li; most other personnel names redacted. Box-diagram layout not reliably machine-transcribable.] Company confidential - for intended recipients only 22 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000056 PLTF EX-04330_0022 APPENDIX II - AUDIT REPORT RATING GUIDELINES Rating: Effective Rating Criteria: Limited "Low risk" observations: Processes and controls provide reasonable assurance that risks are managed and objectives will be met. Rating: Some Improvements Needed Rating Criteria: Limited "Medium risk" observations or a number of "Low risk" observations: Processes and controls require improved effectiveness and efficiency to provide reasonable assurance that risks are managed and objectives will be met. Rating: Significant Improvements Needed Rating Criteria: Limited "High risk" observations or a number of "Medium risk" observations: Processes and controls include gaps that require significant improvements to provide reasonable assurance that risks are managed and objectives will be met. Rating: [Unnamed/lowest tier] Rating Criteria: Several "High risk" observations: Processes and controls include critical gaps that require immediate improvement to provide reasonable assurance that risks are managed and objectives will be met. Company confidential - for intended recipients only 23 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000057 PLTF EX-04330_0023 OBSERVATION REMEDIATION GUIDELINES Rating: High Risk Rating Definition: Process and control gaps that may result in (i) significant deficiency or material weakness in internal controls, (ii) severe decrease in operational effectiveness/efficiency, (iii) violations in policy, laws or regulations, or (iv) damage to Meta's reputation leading to significant business impact. Remediation Expectations: Resolve issue within 60 days AROC Communication: Yes Rating: Medium Risk Rating Definition: Process and control gaps that may result in (i) deficiency in internal controls, (ii) significant decrease in operational effectiveness/efficiency, (iii) non-compliance with policy, laws or regulations, or (iv) damage to Meta's reputation leading to moderate business impact. Remediation Expectations: Target remediation within 120 days AROC Communication: Yes Rating: Low Risk Rating Definition: Process and control gaps that may result in (i) minor non-compliance with policy, laws or regulations, (ii) deficiency in internal controls, or (iii) decrease in operational effectiveness/efficiency. Remediation Expectations: Target remediation within 180 days AROC Communication: No Company confidential - for intended recipients only 24 HIGHLY CONFIDENTIAL (ATTORNEY'S EYES ONLY) METANMAG-168-00000058 PLTF EX-04330_0024