"We want to share a summary of learnings and suggestions for next half in this post... Given sampled teen and overall data, we verified that (1) more teen data leads to consistent teen NE wins and wins for overall NE in most cases; (2) teen data is more efficient in improving teen NE compared with non-teen data; (3) non-teen data is indispensable for model performance for teens especially given its large volume - so it is always suggested to use all non-teen data as it improves teen NE as well."
"Suggested next steps (1) we saw teen core and capped session wins for the secondary model tuning for teens with model trained on upsampled data and we are discussing launch paths to materialize the wins; (2) upsampling teens is beneficial for both teen NE and overall NE so given teen is more top priority and teen data is more efficient for teen NE we should prioritize upsampling teens."
"Given that teen data is more efficient for model performance for teens, we can prioritize upsampling the teen part and checking along the way if it breaks the balance of teen proportion in the training data. And given the usefulness of non-teen data for teen model performance, we should almost always use 'ALL' data that is available even if we want to optimize teens only."
"We have tried two approaches to up-weight teen training losses (1) direct up-weight and (2) add teen dedicated tasks... Add additional tasks for selected production tasks such that they only predict for teen data. This way, losses will be applied more for Teen data, resulting in more emphasis on teen model performance. This is essentially up-weight teen train loss without affecting calibration... This work is more promising compared to direct up-weight as (1) we see mild teen NE wins and neutral overall NE with smaller loss weight for teen dedicated tasks... this has also been launched for FB YA."
Meta & Children's Online Safety