Meta & Children's Online SafetyEmail: Acquisition age management launch proposal
Raw text
Case 4:22-md-03047-YGR Document 3012-18 Filed 05/02/26 Page 1 of 5 From: Case 4:22-md-03047-YGR Document 3012-18 Filed 05/02/26 Page 2 of 5 [/O=THEFACEBOOK/OU=EXCHANGE ADMINISTRATIVE GROUP (FYDIBOHF23SPDLT)/CN= Sent: Wed 12/11/2019 3:05:02 AM (UTC) To: @fb.com] Subject: Re: Acquisition age management launch proposal Thank you! I'm a bit confused as to why they want the over/under 13 buttons when they say it didn't work better than the existing age fallback experience (which is what again?) -- maybe I'm reading this wrong but in the very next sentence they say "both the age buckets and o/u 13 get significantly more people through the registration then age fallback." Is the distinction here that the tests didn't impact RETENTION but it got more NEW users? Which is also confusing b/c why would they be going through an age flow if they're old users we're retaining? Or perhaps retention means whether or not they complete the flow? Full paragraph I'm referencing is here and highlighted in blue below: Per feedback from Brian and Alex, over the last month we've tested a new solution - falling back to age buckets or over/under 13 instead of falling back to an age entry screen. These tests were not better than our existing age fallback experience in terms of retention. Both the age buckets & o/u 13 get significantly more people through the registration than age fallback, however the same or more people select an under 13 age and are therefore not able to successfully register. So we net out neutral to negative on/conf. The main test groups are below and you can find a more detailed description of the tests and results can here. From: @fb.com> Sent: Tuesday, December 10, 2019 4:56 PM To: @fb.com> Subject: FW: Acquisition age management launch proposal Check out the highlighted portion below From: Rob Sherman @fb.com> Date: Tuesday, December 10, 2019 at 4:13 PM To: @fb.com>, Subject: Fwd: Acquisition age management launch proposal From: @fb.com> @fb.com> @fb.com>; Molly Cutler @fb.com> @fb.com>; Denise Moreno Sent: Tuesday, December 10, 2019 7:01:48 PM To: Andrew Bocking Cc: @fb.com>; @fb.com>; Rob Sherman @fb.com>; @fb.com>; Brian Hale @fb.com>; Guillermo Spiller @fb.com> Subject: Re: Acquisition age management launch proposal HIGHLY CONFIDENTIAL (COMPETITOR) PRIV META3047MDL-072-00434143 Cc: Case 4:22-md-03047-YGR Document 3012-18 Filed 05/02/26 @fb.com>, Guillermo Spiller @fb.com>, @fb.com>, @fb.com> Subject: Re: Acquisition age management launch proposal Disappointing this idea did not help, PRIV -Brian Sent from mobile, pls excuse typos. From: Andrew Bocking @fb.com> Page 3 of 5 @fb.com> Sent: Thursday, December 5, 2019 6:34:28 PM To: Cc: @fb.com>; Brian Hale @fb.com>; Denise Moreno @fb.com>; @fb.com> @fb.com> @fb.com>; Guillermo Spiller @fb.com>; Subject: Re: Acquisition age management launch proposal Thanks for the thorough update and generally for all the work Acquisition has done here on this challenging area! Brian/Denise - curious to your thoughts. I'm glad we tested these variants but seems we have run out of options. Perhaps you can discuss with and we should then update Alex ASAP if you agree with the conclusion here. Thanks! Andrew From: @fb.com> Sent: Thursday, December 5, 2019 6:30:20 PM To: Brian Hale Cc: @fb.com>; Denise Moreno @fb.com>; Andrew Bocking @fb.com>; Guillermo Spiller @fb.com> @fb.com>; @fb.com>; @fb.com> Subject: Acquisition age management launch proposal Hello all, Wanted to give an update on our recent age fallback tests and provide Acquisition's recommendation on path forward. TL;DR: Acquisition wants to launch our existing Age Management solution: defaulting to today's date on the registration birthday step & age fallback together for a net impact of -12.5M MAP/year across all interfaces on January 15th. We have now tested falling back to age buckets and a binary over/under 13 HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-072-00434146 Case 4:22-md-03047-YGR Document 3012-18 Filed 05/02/26 Page 4 of 5 option. Neither experience was better than the age fallback experience already built in terms of retention. Context: As previously discussed, Acquisition needs to change the way we currently ask for age in registration. Currently, we have a default date set that is at least 25 years before today's date. There are policy concerns that this pushes people under 13 to enter an incorrect birthday. For the last few months we've been testing solutions, and landed on one which includes two major changes: 1. Defaulting to today's date on the registration birthday step. This will be an estimated -174k WAP@14 hit (18.4M MAP/year). 2. The age fallback product across all interfaces. This will be an estimated +54k WAP@14 win (5.9M MAP/year) on top of the change to default to today's date. We net out at a -12.5M MAP/year hit with these two changes. This is -4.5M India MAP/year, -1.5M Indonesia MAP year and -210k US MAP/year. 65% of the impact comes from FB4A, 12% from Lite, 11% from mTouch and 10% from Kite. New test results: Per feedback from Brian and Alex, over the last month we've tested a new solution - falling back to age buckets or over/under 13 instead of falling back to an age entry screen. These tests were not better than our existing age fallback experience in terms of retention. Both the age buckets & o/u 13 get significantly more people through the registration than age fallback, however the same or more people select an under 13 age and are therefore not able to successfully register. So we net out neutral to negative on/conf. The main test groups are below and you can find a more detailed description of the tests and results can here. Screen shot Over/u Control (Age fallback, launch proposal) nder Over/under 13 Over/under 13 swapped and Age buckets horizontal Age buckets vertical age fallback HIGHLY CONFIDENTIAL (COMPETITOR) Control +o/u13 if error META3047MDL-072-00434147 Reg comple- te win u13 bday gain Case 4:22-md-03047-YGR Document 3012-18 Filed 05/02/26 ADECONFIRM < Join Facebook How old are you? Join Facebook How old are you? Choose your age. Join Facebook How old are you? Choose your age Age 13+ Ur Under 13 Next Use Date of Birth 13- Π Page 5 of 5 GIDA Join Facebook < Join Facebook How old are you' Choose the age range that fits How old are you Choose the age range that t 1-4 Years 5-12 1-4 Years 18-21 13-17 Years 18-29 5-12 Years 30-41 30-49 Years 50+ 13-17 Years 50 Use Date of Birth +22k +22k +33k +22k +22k +31k +34k +28k +22k +25k accoun t -9k -12k +5k Ok -3k create d conf -7k -8k +1k Ok -2k dap@2- -2k -2k Ok Ok Ok Like Date of Batb Recommendation: The Acquisition team proposes to launch the two above changes to the birthday step in registration across all platforms on 1/15. We plan to continue to do some iterations on the birthday step in Q1, like error messaging & UI changes, but do not have any other wins that can be achieved quickly that will recover a significant amount of the impact and thus believe we should ship and iterate. Please let me know if there's further discussion needed or if we can proceed with launching. Thanks, HIGHLY CONFIDENTIAL (COMPETITOR) META3047MDL-072-00434148