Meta & Children's Online SafetyChat: appearance comparison is pretty universal but higher in western countries (especially IG's role)
Thread View
████ name redacted. Different bars may be the same person or different people.
Thursday 20 May 2021
06:52Moira Kathleen Ballantyne Burke@silent Hugs, Shruti! We'll miss you.
11:41Moira Kathleen Ballantyne Burkewhen you say "sexualized imagery" do you mean anything that meets the sensitive content definition (that list of FIT/XRay concepts)? Or do you mean a subset / different set defined by guidelines, classifiers, etc.?
11:50████The definition I found from research:
'a person's value comes only from his or her sexual appeal or behavior, to the exclusion of other characteristics; a person is held to a standard that equates physical attractiveness (narrowly defined) with being sexy; a person is sexually objectified that is, made into a thing for others' sexual use, rather than seen as a person with the capacity for independent action and decision making; and/or sexuality is inappropriately imposed upon a person' APA Task Force on the Sexualization of Girls. (2008, February 1). Report of the APA Task Force on the Sexualization of Girls. American Psychological Association. http://www.apa.org/pi/women/programs/girls/report
the first point of 'to the exclusion of other characteristics' is likely why images are more likely to be damaging than other types of content-- they lack context + can be altered to conform even further to 'ideals'.
So based on this definition I could see an argument for downranking based on a set of XRay concepts, though FIT may be too broad. I believe there are already borderline nudity classifiers that could be used.
Ultimately I think we would need a separate investigation into what signals exist and how well they map to agreed upon definition/guidelines (with input/sign off from policy).
12:04Moira Kathleen Ballantyne BurkeAnd for everyone:
(and others) keep asking about who we're focusing our efforts on (e.g., teen girls in the US, teens globally, everyone). I want to be sure we're on the same page about that. Some factors:
* how good is the teen model worldwide?
* how good is the gender model worldwide?
* who has the highest levels of appearance comparison? the data actually say women until 40ish have the highest levels, so there's not a data-driven reason to limit to teens, but there may be defensibility reasons.
* where is comparison worth addressing? data say appearance comparison is pretty universal but higher in western countries (especially IG's role). there are differences in India (lower body image issues, higher IG-specific issues), and in East Asia (higher body image issues, lower IG-specific issues).
who do you think we should be focusing on? we've frequently said "teens" which I've interpreted as "we'd test with teens in the US where the age model is better and comparison is higher, but in theory would expand these products to other countries, and possibly all ages."
How would you answer s question about who we're focusing on?
12:05Moira Kathleen Ballantyne BurkeShared an image
12:05Moira Kathleen Ballantyne BurkeShared an image
12:19████I've been pushing to focus on female teens in the west order to unblock us from being able to do things. This is in large part because the age model is better in western countries, but we could expand as the model improves globally (or use stated age globally in the interim).
If there are specific features that we would be allowed to ship to broader cohorts, awesome.
If the teen-based problem framing doesn't unblock us, then I would recommend we default to the largest group that would receive a benefit.
12:39Shilpa ModyI agree with the framing of "we'd test with teens in the US where the age model is better and comparison is higher, but in theory would expand these products to other countries, and possibly all ages."
12:41Shilpa ModyFor gender, I can't imagine a world where we'd have different product experiences for girls vs boys (nor do I think that's a good idea) so I think that even if we do more research on girls, we should test & launch across genders. For age, I think limiting to teens gives us more leeway because we have more responsibility to protect teens (the problem might suck for adult women as well, but the protection vs free speech / user agency tradeoff feels really different for adults than for teens). For geography, I actually don't have a strong opinion
13:08Moira Kathleen Ballantyne BurkeJust FYI, I had a 1:1 with and he was asking about sexualized content and how that relates to the other concepts in our doc. He thought that changing the language (not calling it "sexualized content") could be more successful. What do you think about backing out to "comparison-inducing" content? (or "sensitive" content?)
13:18Jus-Tin ChengComparison-inducing seems broad, but sensitive content in the way we've been talking about sounds reasonable.
13:36Shilpa ModyBoth "sensitive content" and "comparison-inducing content" are a lot more broad than sexualized, right?
13:37Shilpa ModyI agree that saying sexualized makes it sound quite judgey, but then that's the actual thing that the research has the most compelling case for
13:38Jus-Tin ChengBasically a euphemism I think, since the "sensitive content" definitions we have are fairly representative of the types of content implicated in prior academic research.
13:39Shilpa ModyAnother factor is that sexualized content to me sounds like we decide what is good vs bad for people and write guidelines for it, whereas comparison-inducing sounds like users tell us what is a good or bad for them and we act accordingly
13:41Jus-Tin Cheng- Our evidence for negative effects is certainly stronger for women overall rather than teen girls specifically - Country effects are complex, but we're not quite powered to do particularly deep dives (i.e., we can't really do comparisons by age, gender and country). Hope to be doing more qual deep dives here though.
- Overall, seems reasonable to focus on teen girls in the US, and conservatively aim to have any changes we make to not make things worse for other demographic groups.
13:46████In the quip, Sam references some existing classifier we have as well.
"suggestive content" or "borderline AN&SA" (ANSA = adult nudity and sexual activity)
Another term is "Sexually Suggestive prevalence".
Another is "Nudity & Pornography" and I imagine there would be borderline classifier for these as well.
Sounds like we should add a table to clarify that we could define 'sexual content' in a variety of ways from user defined to existing classifiers to our own FIT/Xray definition, to a new classifier.
13:55Shilpa ModyHow do people feel about "appearance-focused" or "body-focused" as an alternative term to sexualized?
13:58Shilpa ModyAlso outside of naming, how do people feel conceptually about the we decide what is bad vs users tell us what's bad approaches?
14:00Jus-Tin ChengIsn't this more about what people *perceive* is bad for them. vs. what lab experiments / correlation analyses have said about what is bad?
14:02████For teens, I think a paternalistic approach is okay, as they are children.
For adults, I think a personalized approach is better. And personalized could be extrapolated from people like them or could be 1:1 personalized based on their direct signals or some combination.
14:04Moira Kathleen Ballantyne BurkeI love "body-focused" (and am also okay with "appearance-focused"). I agree that "comparison-inducing" is broad (and historically hasn't worked well rhetorically since hearers don't think it's "bad enough").
14:11Moira Kathleen Ballantyne BurkeI lean more toward "we decide what is bad" (though I'd phrase it as "we make policies/classifiers informed by triangulating what experts say with what our qual/quant research indicates"). My concern about "users tell us what's bad" is that (1) any individual piece of content might not be perceived as bad, it's more the collection, and (2) even then, teens may still not recognize how content may be impacting them.
But I'm also game for "people tell us what they think is bad" if that's a more defensible/pragmatic step forward.
14:16████Are we aligned on 'body-focused' as a straight swap for 'sexualized'?
14:38Shilpa ModyThis makes sense and I somewhat agree that it's OK to be paternalistic (obviously in a data-driven way) with teens. I think the question beneath my question was are we in the business of reducing NSC for teens because of the long-term harm that it causes for some teens (body image problems, etc), or because it's a bad experience to feel like crap about yourself over and over? If the former, paternalistic seems right. If the latter, teens telling us what makes them feel bad seems right.
14:58████I am most interested in preventing long-term harm. That seems like the highest impact and most connected to research.
I'm not sure how different the 'bad experience' framing is in practice, assuming that feeling like crap over and over is probably the process by which people develop body image issues.
The more practical issue with teens telling us what makes them feel bad is 1) whether we can collect enough signal at scale to personalize 1:1, 2) whether a survey would be generalizable enough to be effective on this dimension.
It would be interesting to compare the results of an IG defined model vs a teen defined model for 'negative body image' content-- depending on how we frame the questions the overlap could be quite high and the model features (based on what we have available) potentially very similar.
that they two
15:04Moira Kathleen Ballantyne BurkeYep. From a practical perspective, I'm cool with either way of defining/modeling this kind of content. Agree with models could end up being pretty similar regardless. (I'm actually more on the "prevent people feeling like crap all the time" side of things rather than "prevent long-term body image harm," but would be happy solving for either problem.
15:05Shilpa Modyi think one significant difference is that the long-term harm version is saying we want to do something for all teens because of a problem that happens to only some teens (only some teens actually go from crappy experiences of NSC --> body image problems), whereas the bad experience framing is saying the NSC experience is bad in itself so we're doing something for all teens because of a problem that happens to most teens. I don't think that the bad experience version necessitates 1:1 personalziatiion, though of course personalization would be great.
15:07Shilpa ModyI think the way we've created the sensitive content model so far is user-bad-experience-driven, right? Like indirectly, but it's based on survey responses about feeling bad on IG vs. about experiencing greater harms like body image problems
15:10Shilpa ModyI suppose we have an opportunity to take the more harm-driven approach if we pick features that specifically pick out sexualized/body- focused content rather than just all content that predicts appearance comparison
15:13████It's a funnel so feeling bad about your appearance is a necessary first step towards developing body image issues. I'm just most interested in the people at the end of the funnel. we can talk more about the top of funnel, mid funnel, or bottom funnel depending on how that resonates best internally/externally. And yeah it's possible that the features of a model would be different depending on who you ask/what your train it on.
15:17Shilpa Modytrue makes sense! I am also OK with focusing on the large set of people who feel crappy all the time or the small set of people who have body image problems I think the former will be the easier path but who knows
15:35Moira Kathleen Ballantyne BurkeI'm guessing focusing on the top of the funnel will be the easier path internally, too (though don't really know). Are we all aligned on focusing on the top of the funnel then? (at least for our meeting w/ D
15:54████The way I've thought about it is that the product outcome will be pretty similar regardless of which part of the funnel you focus on (reducing prevalence of body-focused content or similar) but the problem definition may have more or less support by research/policy/leads depending on how it is framed.
This is why we are offering multiple problem framings to try and figure out which of them will unblock the product work we think will have impact.
I tend to lean towards the severity of harm argument because 1/ research is compelling, 2/ harder to argue against, 3/ personal bias based on my lived experiences (for example, one of my close friends manages an eating disorder clinic).
But I think we're pretty much all aligned that the key is to just get unblocked in general and we would all be fine with picking any part of the funnel as the 'focus area' so long as it achieves that goal.