Review 2: How Contribution Affects Engagement on Facebook

Raw text

Transcribed text as recorded, including headers and formatting fragments.

From: Lada ████ [/O=THEFACEBOOK/OU=EXTERNAL (FYDIBOHF25SPDLT)/CN=RECIPIENTS/CN=████] Sent: Tue 5/12/2015 7:56:05 PM (UTC) To: Nir ████ [@fb.com] Subject: Re: [tasks] #6826717: Review 2: How Contribution Affects Engagement on Facebook Attachment: contribution2016grinberg_laa1.pdf #6826717 - Review 2: How Contribution Affects Engagement on Facebook RECENT ACTIVITY Lada ████ changed the title to Review 2: How Contribution Affects Engagement on Facebook Lada ████ changed the description to 1. Author(s): Nir ████, Alex Dow, Lada ████, Mor ████ (non-fb, academic collaborator) 2. Manager approved? Lada ████ on 5/12 (Project started in August 2014 while Nir interned with CDS). 3. Does this require a research review? yes, to be approved by Danny ████ on <DATE>. 4. Where is this being submitted? CSCW 2016 (http://cscw.acm.org/2016/submit/papers.php) 5. Due date(s): Full paper 05/22. 1st Notification: July 3, 2015 Revised Submission: July 29, 2015 2nd Notification: August 24, 2015 If accepted would appear: February 2016 6. A brief description of the subject matter: we want to understand user engagement around contribution. To what extent do people expect feedback on contribution? how does contribution effect the amount of interactions with friends? is feedback simply reciprocated or distributed more generously? is there an effect on interactions with public, non-friend content? 7. A sentence or two explaining why Facebook would want to conduct and publish this research: We should do this investigation because it has the potential to shed light on one of the most fundamental process at Facebook: content production. Better understanding of contribution on Facebook is called for due the overall trend of decreasing content production (not discussed in the paper!), which can help guide the design of services that encourage contribution and/or reduce friction. At the very least, this study will have implications for Newsfeed ranking and interface design, but can also provide the "language" for thinking about contribution at Facebook. Proposed procedure: observational data analysis only, no experimental manipulation, no qualitative analysis of content and no interference with users' experience whatsoever. Data you will be collecting and how it will be de-identified for analysis: data will be derived from existing action logs solely and can be de-identified upon derivation. In any case, data would not be retained any further than its upstream tables. Our analysis will compare users behavior around content production and a reference feedback action (like, comment or reshare) separated one week from each other. Our measures of user behavior include quantitative # of site visits, # of likes and # of comments per content view from different types of originators. The content itself would not be analyzed. 8. Is this a full paper, workshop paper, abstract only, or other? Full paper. 9. List any non-public information that you expect to be included in the publication: non-identifying averages of the following stats for users who posted content: # of site visits, # of stories viewed, rate likes/comments per view. There are several stats included, which add validity to the results, but could be expressed in relative terms if that is advisable (please advise!) * what percentage of people did not receive any notifications in 24hour period after positing (it's very small) * number of site visits/24 hour period * session duration * like & comment rates on posts * what percentage of all users produce content & feedback with this separation (might be able to infer overall content production numbers? maybe?) 10. Please confirm that you will not promise to share any data in the publication: data will not be shared and no promises regarding sharing will be given. 11. Does the paper disclose anything novel that we should think about patenting? No. TL/DR; Abstract: Past research on social participation examined motivations for contribution and long-term effects on relationships. The current work augments previous knowledge by focusing on short-term behavior after someone posts content online and examines the mechanisms underlying contribution. Using observational data analysis of individuals' activity on Facebook, we test hypotheses regarding the existence of feedback expectations, changes in the distribution of attention to content and shifts in the decisions to interact with others. We find that those contributing content are intrinsically feedback-seeking, more attentive to friends (but not others) and evaluate content from friends more favorably (in large part due to reciprocity). Overall, we find contribution associated with more active engagement that begins close to 6 hours before contribution and lasts for less than a day. Better understanding of the processes behind contribution will allow for the design of Social Network Sites that support contributors better around times of social participation. Updated paper draft is attached. 13. Are there any controversial or "FB unfriendly" headlines that someone (e.g. a Facebook critic who may be looking for "clicks") could write about this paper? We are asking this question so that you can help us think through (and potentially help mitigate!) the potential PR/Policy risks with your paper. The paper discusses individuals' expectation of feedback and so one could think up a headline along the lines of 'Facebook users conditioned to seek feedback' or some sort of Pavlovian reference. However, not sure how likely that would be. Lada changed the priority to High Lada opened the task. Lada changed the attachments. Added: contribution2016grinberg_laa1.pdf Removed: contribution2016grinberg_draft20150420.pdf Lada commented: We are very late to this due to my unexpected absence last week. We're still hoping to make the deadline in 11 days. We have a complete draft attached that is ready for review which has all results reported. We will continue to revise framing and wording. We are looking for feedback on whether any of the stats we have included should be made relative. Lada changed the tags. Added: public policy review, PR comms review, CSCW16 Comment · Close · Unsubscribe · Assign · Add Subscribers · Tag · Set Priority · Get Activity STATUS: Open PRIORITY: High OWNER: Lada CREATED: Apr 20, 2015 7:50pm by Nir DESCRIPTION: 1. Author(s): Nir, Alex Dow, Lada, Mor (non-fb, academic collaborator) 2. Manager approved? Lada on 5/12 (Project started in August 2014 while Nir interned with CDS). 3. Does this require a research review? yes, to be approved by Danny on <DATE>. 4. Where is this being submitted? CSCW 2016 (http://cscw.acm.org/2016/submit/papers.php) 5. Due date(s): Full paper 05/22. 1st Notification: July 3, 2015 Revised Submission: July 29, 2015 2nd Notification: August 24, 2015 If accepted would appear: February 2016 6. A brief description of the subject matter: we want to understand user engagement around contribution. To what extent do people expect feedback on contribution? how does contribution effect the amount of interactions with friends? is feedback simply reciprocated or distributed more generously? is there an effect on interactions with public, non-friend content? 7. A sentence or two explaining why Facebook would want to conduct and publish this research: We should do this investigation because it has the potential to shed light on one of the most fundamental process at Facebook: content production. Better understanding of contribution on Facebook is called for due the overall trend of decreasing content production (not discussed in the paper!), which can help guide the design of services that encourage contribution and/or reduce friction. At the very least, this study will have implications for Newsfeed ranking and interface design, but can also provide the "language" for thinking about contribution at Facebook. Proposed procedure: observational data analysis only, no experimental manipulation, no qualitative analysis of content and no interference with users' experience whatsoever. Data you will be collecting and how it will be de-identified for analysis: data will be derived from existing action logs solely and can be de-identified upon derivation. In any case, data would not be retained any further than its upstream tables. Our analysis will compare users behavior around content production and a reference feedback action (like, comment or reshare) separated one week from each other. Our measures of user behavior include quantitative # of site visits, # of likes and # of comments per content view from different types of originators. The content itself would not be analyzed. 8. Is this a full paper, workshop paper, abstract only, or other? Full paper. 9. List any non-public information that you expect to be included in the publication: non-identifying averages of the following stats for users who posted content: # of site visits, # of stories viewed, rate likes/comments per view. There are several stats included, which add validity to the results, but could be expressed in relative terms if that is advisable (please advise!) * what percentage of people did not receive any notifications in 24hour period after positing (it's very small) * number of site visits/24 hour period * session duration * like & comment rates on posts * what percentage of all users produce content & feedback with this separation (might be able to infer overall content production numbers? maybe?) 10. Please confirm that you will not promise to share any data in the publication: data will not be shared and no promises regarding sharing will be given. 11. Does the paper disclose anything novel that we should think about patenting? No. TL/DR; Abstract: Past research on social participation examined motivations for contribution and long-term effects on relationships. The current work augments previous knowledge by focusing on short-term behavior after someone posts content online and examines the mechanisms underlying contribution. Using observational data analysis of individuals' activity on Facebook, we test hypotheses regarding the existence of feedback expectations, changes in the distribution of attention to content and shifts in the decisions to interact with others. We find that those contributing content are intrinsically feedback-seeking, more attentive to friends (but not others) and evaluate content from friends more favorably (in large part due to reciprocity). Overall, we find contribution associated with more active engagement that begins close to 6 hours before contribution and lasts for less than a day. Better understanding of the processes behind contribution will allow for the design of Social Network Sites that support contributors better around times of social participation. Updated paper draft is attached. 13. Are there any controversial or "FB unfriendly" headlines that someone (e.g. a Facebook critic who may be looking for "clicks") could write about this paper? We are asking this question so that you can help us think through (and potentially help mitigate!) the potential PR/Policy risks with your paper. The paper discusses individuals' expectation of feedback and so one could think up a headline along the lines of 'Facebook users conditioned to seek feedback' or some sort of Pavlovian reference. However, not sure how likely that would be. TAGS: academics, PR comms review, product_science, CSCW16, 2015, public policy review SUBSCRIBERS: Lada, Danny, Lauri, Alex Dow, Airalee, Nir SCHEDULE: Due May 22, 2015 3:00am COMMENT HISTORY Lada: We are very late to this due to my unexpected absence last week. We're still hoping to make the deadline in 11 days. We have a complete draft attached that is ready for review which has all results reported. We will continue to revise framing and wording. We are looking for feedback on whether any of the stats we have included should be made relative. May 12, 2015 3:56pm Got feedback about this email? Hit us up! Tasks Feedback Customize your notifications at Email Settings How Contribution Affects Engagement on Facebook Nir ^1,2, Alex Dow^2, Lada ^2 and Mor Naaman^1 nir@cs.cornell.edu, @fb.com, @fb.com, mor@jacobs.cornell.edu 1Jacobs Institute, Cornell Tech 2Facebook Inc. ABSTRACT Past research on social participation examined motivations for contribution and long-term effects on relationships. The current work augments previous knowledge by focusing on short-term behavior after someone posts content online and examines the mechanisms underlying contribution. Using observational data analysis of individuals' activity on Facebook, we test hypotheses regarding the existence of feedback expectations, changes in the distribution of attention to content and shifts in the decisions to interact with others. We find that those contributing content are intrinsically feedback-seeking, more attentive to friends (but not others) and evaluate content from friends more favorably (in large part due to reciprocity). Overall, we find contribution associated with more active engagement that begins close to 6 hours before contribution and lasts for less than a day. Better understanding of the processes behind contribution will allow for the design of Social Network Sites that support contributors better around times of social participation. Author Keywords Social Media; Contribution; Social Participation; User Behavior. ACM Classification Keywords H.5.m. Information Interfaces and Presentation (e.g. HCI): Miscellaneous; See http://acm.org/about/class/1998/ for the full list of ACM classifiers. This section is required. INTRODUCTION Is posting on Facebook associated with different kinds of interactions before and after the post is made? Much research has focused on people's motivations to post on social media, but little research has focused on posters' behavior after posting and whether they behave differently than after doing another comparable activity, like liking or commenting on another's post. In this study we draw on existing theories from social psychology, cognitive psychology and behavioral economics to formulate hypotheses about contributors' behavior on SNS. We address three different questions in this work. First, we test whether contributors (those who post their own content at a given point in time) are intrinsically feedback-seeking and visit the site more often after contribution even when no knowledge of feedback exists. Second, we examine whether content consumption practices of contributors change both in quantity and selectivity. Lastly, we investigate changes in interaction with others as reflected in liking or commenting rates on content and quantify the effect of reciprocity in interactions with friends. Better understanding of the psychological mechanisms behind contribution is important for both theoretical and practical reasons. From a theoretical standpoint, our findings provide evidence for the strength of reciprocity and support for the selectivity of our attention in online settings. On the practical side, better understanding of contributors can help encourage contributions, better support users at times of contribution and may even be used to improve algorithmic ranking of content for others. We devise a within-subject, observational data analysis of anonymized log data of Facebook activity from 2.4 million people over a period of 9 days. In our design, we observe user actions on Facebook around times of contribution (without any intervention) and compare those to a baseline of feedback activity from the same individual. Specifically, we consider when an individual posts a piece of content, e.g. writes a post or posts a photo and compare her behavior when she gives feedback on someone else's content. We use measures of activity such as site visits, number of stories read and interacted with in the 48 hours surrounding contribution to learn about the implications of contribution on contributors' behavior. Our contributions are therefore: - First large-scale evidence for within-subject differences in engagement around times of contribution. - Refined theoretical understanding of the psychological mechanisms underlying contributors behavior on social network sites. - Design implications for better support of contribution on social network sites. The next section describes the theoretical framework used to draw hypotheses about changes in contributors' behavior. BACKGROUND We build on theories from various fields to examine behavioral changes of contributors in SNS. We discuss how contribution of content can affect the ways in which individuals use the service, consume content and interact with others on it. Previous research examined the motivating factors and gratification individuals draw from participation in online communities. For example, Dholakia et al. [7] identified five motivating factors for contribution online: purposive value (exchange of information), self-discovery (acquiring knowledge), maintain relationships, enhance social status and entertainment. Other studies [18, 21] examined the motivations for active participation on Wikipedia, finding similar motivations and gratifications. Several Uses and Gratification (U&G) studies examined contribution to SNS and Facebook in particular. Both Joinson et al. [14] and Papacharissi and Mendelson [19] provide evidence that Facebook contribution helps support expressive information sharing and maintain relationships. Preece and Shneiderman [20] describe contributor's recognition and ability to build reputation as a major motivating factors for social contribution. While previous research extensively studied why people contribute, in this work we focus on how contribution affects user behavior at times of social participation, and build on theories from psychology and behavioral economics to explain observed changes. In general, the theory distinguishes between exogenous and endogenous factors that affect user behavior and attitude. Some of the Exogenous factors we refer to here include feedback and direct rewards that contributors receive for their contribution. Endogenous factors we build on include expectations of feedback, reciprocity, updated valuations of self and others, increased sense of group identity and arousal. All these factors may impact key aspects of user behavior around the time of posting, as we explain next. Feedback Expectations and Site Activity Feedback is a key component of any social exchange: it is important both for motivating contributions in the first place [5, 2, 15] and for evaluating social relationships over time [13, 16]. Over time, feedback expectations are formed about who will respond to what message, when and how. Neuroscientists studying habit-learning were able to identify the brain structures that are involved in learning from rewards and adjusting expectations for future behavior. For example, Meshi et al. [17] in a recent study found activity in the Nucleus accumbens, a brain structure that processes rewards, in response to receiving feedback on Facebook. Outside the domain of Facebook, Dopamine neurons were found to be active even in cases where only an indication of reward was present without an actual reward being given [9, 22], demonstrating that mere expectations are enough to affect behavior. In a similar vein contributors on Facebook are likely to learn to associate the act of posting with feedback. Thus, even when no knowledge of feedback exists, the learned association between posting and feedback will lead contributors to seek "rewards" by visiting the site more often. More formally, our hypothesis is that: H1 Following a post, self-motivated site visits will increase. Attention Attention has been studied by cognitive psychologists for many years, resulting in diverging perspectives on attention at various levels. According to Smith and Kosslyn [24] "attention involves selecting some information for further processing and inhibiting other information from receiving further processing". If changes in contributor's selection processes are underway, content consumption can change both in overall quantity and composition. If a person posting content is more engaged, then they might correspondingly also consume more content. However, posting might also take time and energy away from content consumption. They might also shift their attention toward content from their friends, as opposed to pages or other broadcast sources, looking for content that mirrors what they have produced. Therefore, our hypotheses for content consumption are the following: H2.a Following a post, contributors will consume more content. H2.b Following a post, contributors will consume more content from friends. Judgment, Reciprocity and Interaction Rates Contributing content is likely to have an effect on contributors decisions to engage in feedback-giving activities with others. Reciprocity as the social norm of returning a favor, can lead to high interaction rate with friends. Reciprocity is so deeply ingrained in human culture that sociologists such as Georg Simmel and Alvin Gouldner define it as a necessary component for social equilibrium and cohesion [23]. In fact, Gouldner [12] could not find a single society that does not subscribe to the norm of reciprocity. In the realm of CMC, even simple one-way communications such as a Like or short composed communication (as defined by Burke et al. [1]) bares value. Receiving attention from friends, similarly to receiving a gift, calls for reciprocation. Lastly, going through the process of posting content can assist contributors to take the perspective of other contributors, enhance the sense of shared identity and lead to less critical interactions. The Spyglass Model of Self by Goldstein and Cialdini [11] states that "observing the freely chosen actions of others with whom they feel a sense of merged identity, almost as if they had observed themselves performing the acts". Self-other merging is by no means precarious, it stems from processes of perspective taking [6, 10] and social identification [3], which are particularly strong for close relationships. The theory suggest that after posting content contributors would identify more strongly with other contributors, either because of greater sense of group membership accompanying contribution or because they can better relate to their needs and expectations. Hence, we postulate that contributors would judge content from friends more favorably. In contrast to the above theories, fatigue or a fixed-quota policy may explain a decrease in interaction rates after contribution. If contributors consume more content, as postulated in the previous section regarding attention, they may experience fatigue over time and engage in fewer interactions. Similarly, if people have a fixed amount of interactions they can engage in, and more content is consumed, the rate of interactions would decrease. We believe that the additional amount of content consumed would be relatively small and thus neither fatigue nor interaction limits would be dominant in our case. Therefore, our hypotheses are: H3.a Following a post, contributors are more likely to give feedback to friends. H3.b Contributors are more likely to give feedback to those who responded to their content than to other friends. METHODS To test the hypotheses listed above, we devised a quantitative, within-subject, observational data analysis of Facebook logs. Next, we describe the dataset and key measures and variables used for our analysis. Dataset We performed a within-subject analysis that looks at the differences in individuals' behavior between two types of actions they performed on Facebook: an action of contributing original content (C), and providing feedback to others (F). The data were anonymized and content was not analyzed. Contribution is defined as posting original content to Facebook, for example, an individual posting a status update, or uploading a new photo. Feedback is defined as reacting to someone else's content on Facebook: a like, a comment or re-share of others' content. Identifying such pairs of actions for the same users allows us to compare user behavior around contribution with a baseline of user behavior around feedback. In both cases we know that the individuals were active on Facebook on a specific time, making actions C and F good comparison candidates. For the analysis, we identified individuals who performed these two actions one week apart, in any order. In other words, for each individual, we identified contribution action C that occurred a week before or after a feedback action F. Moreover, we required both actions to have been performed mid-week (some time during the 24 hour span of Wednesday Pacific Standard time) to reduce bias from day-to-day variation (also across cultures). The analysis included one such pair of actions from each individual that matched these criteria. Given the selected actions C and F for each individual, we compared their behavior 24 hours before and after each action. We chose a window of 48 hours around actions in order to respect the natural and regular periodicity of human behavior. The matching of actions did not exclude the other type of action from occurring around that same time. For example, it is possible that a given individual contributed content some time before or after the feedback action F selected for the analysis, and vice versa. Stricter filtering, requiring no contribution by the user around the time of the F action selected for analysis, would have resulted in a much smaller dataset, which is less representative of the contributors' population. Our non-strict selection criteria are noisier, but provide a less biased lower bound on the actual, possibly larger, effect size of contribution versus feedback. Our base dataset includes C and F actions for 2.4 million individuals who posted content to Facebook or gave feedback to others on two specific dates, February 11th and February 18th of 2015. This set of individuals and actions is a sample of all users with actions that aligned with the selection criteria for those dates. In the analyses we perform, detailed below, we sometimes use subsets of the base dataset depending on different requirements of each analysis. Measures We now turn to define the measures used in our analysis to test hypotheses described earlier in the Background section. Organic Site Visits The measure of organic site visits refers to the number of site visits that are not initiated by a notification, before any knowledge of feedback is available to contributors. Site visits are counted in terms of sessions, where each session is a sequence of actions of a logged-in user, where actions are less than 30 minutes apart; if the individual was not active for 30 minutes, a new session is started and a new "site visit" is assumed. Sessionization included both server-side and client-side action taking place on any device (mobile or not). When measuring site visits and sessions we want to ignore those visits that are due to offline notifications — users getting email or other alerts about Facebook activity that invites them to come back to the site. Therefore, we examined a subset of contributors for which Facebook did not generate any e-mail, SMS or mobile push notification in the two days preceding an action and the day following it. This subset of contributors did not receive notifications because they disabled offline notifications explicitly in their profile preferences or there was no activity to generate a notification for them. Stories Read We measure content consumption by examining the number of News Feed stories read by contributors in the 24 hours preceding or following an action. Facebook's News Feed is the landing page for users browsing to facebook.com, where content from friends and followed accounts is algorithmically ranked. A story is considered read if it was visible in the central portion of the user's screen for a least two seconds. Again, we use client-side logging done by Facebook to accurately measure read stories across all interfaces and devices. Note that we explicitly exclude stories that originated from the contributor herself as this may artificially increase the number of stories read in the contribution condition, where contribution appears at the top of the user News Feed. Interaction Rate We define interaction rate as the proportion of likes or comments given per News Feed story read by the contributor in 24 hours before or after activity. Interaction rate is the portion of stories read from others (as defined above) that contributors liked or commented on directly from the News Feed. In order words, our measure of interaction rate excludes likes and comments that occur in other parts of Facebook such as Timeline of other users or groups. Here again, any interactions with the contributor's own content (reads, likes, comments) were excluded. Statistical Analysis For most of the analyses described below, we use Difference in Differences (DID) analysis in order to estimate the effect size of contribution while accounting for exogenous variation external to contribution. DID is a common statistical analysis technique used in economics to mimic a random assignment experimental design in observational data analysis. DID estimates the effect of "receiving treatment" (in our case choosing to post) by controlling for a trend evident in the control group (feedback action in our case). Two elements in the way we apply the DID help reduce selection bias and bias due to ordering effects. First, both control and treatment groups in our analysis include the same people. Having the same people in both conditions eliminates individual differences in receiving "treatment", i.e. choosing whether to post content or to give feedback. On the other hand, the sequence of two actions from a single person introduces a secondary bias of interaction effect between subsequent actions. While we cannot rule out that one action may effect another action over a long period of time, our exploratory analysis suggest a diminishing difference in activity after 24 hours. We use a much longer gap of one week in between the actions we examined to further eliminate such interactions. Second, the random order at which contribution and feedback actions may occur (contribution or feedback first) reduces the bias that observed effects are due to a one-time external event that affects only one of the conditions, or other time-based trends like increase in use over time. All of our statistical tests were done using the standard technique of bootstrapping with 10,000 replicas. We estimated means and 95% confidence intervals around them using the bootstrapped samples. Bootstrapping is more stable and asymptotically more accurate than estimates of confidence intervals based on a single empirical sample and normality assumptions [8]. We also favor bootstrapping over traditional paired t-tests since the later tend to yield highly-significant p-values in all cases due to the sheer size of the sample size (150,000 users in our smallest sample). RESULTS We performed a series of descriptive and comparative analyses to better understand the data, and verify that there are no material differences between the contexts in which users performed the different actions (contribution and feedback). [Figure 1: Percent of active users (top panel) in the 48 hours around contribution and feedback actions, with the differences visible on the bottom panel. The 95% confidence intervals were too small to be visible therefore removed from both panels.] Preliminary Analysis A central question to our analysis is how active users are at times around different actions. Figure 1 addresses exactly this question by presenting on its top panel the percentage of users in our sample that were active on Facebook as a function of time, for 24 hour before and after each of the two actions that they have taken. The figure shows two curves for user activity around the action of contribution (solid red line) or feedback action (dashed black line). Data points in the figure correspond to the percentage of the 2.4M users that had an active session during each 20 minute time bin on the x-axis. For example, at the exact time of an action (time 0) all of the users in our dataset were active on Facebook since they either posted content or gave feedback. As a result, the plot spikes for both conditions at exactly 100%. The bottom panel shows the differences between the percent of active sessions around contribution and feedback (in other words, the difference between the solid red and dashed black lines on top). Figure 1 clearly shows that except for the 20 minutes immediately following an action, contributors are more active for several hours both before and after posting content compared to their activity around feedback at the same time frame. The only exception is the 20 minutes shortly after feedback where people are more likely to continue to engage with News Feed content rather than leave Facebook, as 7% of contributors do immediately after posting content. The discontinuity observed at around zero in Figure 1 informed our decision to exclude the hour immediately following or preceding an action from our analysis. While feedback actions are the closest type of active engagement we have for comparison with contribution, we do see short-term differ- [Figure 2: Average duration of sessions around feedback/contribution actions, with the percent of time spent before the action.] ences stemming from the different sequence of user interaction at which feedback and contribution occur. Therefore, for the rest of our analysis we use a window of 48 hours around an action, but exclude the 2 hours centered around an action. Two interesting findings emerge from Figure 1 regarding the higher activity levels around contribution, and its return to baseline levels at the ±24 period. We see that higher activity levels start as early as six hours before contribution and last more than 12 hours afterwards. The fact that contributors are more active even six hours before contribution is surprising and cannot be simply explained by the additional time necessary to conceive and articulate a post. The higher levels of activity after contribution are very likely to be driven, at least in part, by notifications that contributors get due to feedback on their content. Below, when we address hypothesis H1, we show that notifications are not the only factor that explains higher level of user engagement after contribution. Lastly, Figure 1 shows uptick in activity in the 24 mark before and after each action. The increased activity indicates regular patterns in user activity and justify the choice of 24 hours for analysis. Furthermore, the diminishing differences at the ±24 hours period demonstrate that the effect is largely dissolved in a day. We examined the data to make sure contribution sessions are not fundamentally different than feedback sessions. We looked and the length of sessions and the position within a session where actions were recorded. As before, our definition for a session is a sequence of actions that are less than 30 minutes apart from each other. Figure 2 shows the average duration of contribution and feedback sessions. While contribution session lasts more than 80 minutes on average, feedback sessions are significantly shorter, lasting only close to 68 minutes (95% confidence intervals were 20 seconds long, too short to be visible on the relevant scale). The figure also shows that contributions are positioned similarly within a session, with 43.8% of the session time passing by before feedback occurs and 44.7% for contribution. The 1% increase in the position of contribution within session is equivalent of ~50 seconds, which is relatively small and could potentially be explained by the extra time required to compose a post. We also verify that contribution and feedback actions occur at comparable time of day. For example, we make sure our dataset is not biased such that contribution takes place in the morning and feedback at night. By computing the difference in time of day for each pair of user actions, we find that the 95% confidence interval in differences is (-3.2, 3.0) minutes. [Figure 3: Significant increase in Difference in Differences analysis of organic site visits in 24 hours before/after activity (95% CIs). The dashed line designates the DID prediction for levels after contribution based on the trend evident in the Feedback condition. Square brackets highlight the significant increase in site visits of +0.11 on average.] No difference (difference of zero) is well within the 95% confidence interval. Therefore, we conclude that contribution and feedback actions occur at roughly the same time of day. In summary, the preliminary analysis provided evidence for adequacy of our design decisions regarding the 24 hours period and comparison to a baseline of feedback activity. The initial analysis informed our decision to exclude the hour right before and after an action in the rest of the analysis and found that contribution and feedback actions are positioned comparably within sessions and within the day. Feedback Expectations We test hypothesis H1 about an increase in self-motivated site visits by conducting DID analysis on our measure of organic site visits. Recall that for this analysis, we wish to neutralize the effect of notification. To this end, we focus on subpopulation of about 150,000 contributors for which Facebook did not send any offline notifications in the 48 hours preceding an action and 24 hours after. These contributors either chose not to receive offline notifications or there was no activity that led Facebook to generate a notification for them. Figure 3 show the average number of site visits for the same set of contributors before and after contribution and feedback actions. For instance, we see that in the 24 hours before feedback users had an average of 3.9 organic site visits, while closer to 4 site visits after taking the feedback action (excluding the feedback/contribution session itself). Designated by the dashed line is the projected number of organic site visits contributors, if the general trend apparent in the feedback condition occurred at times of contribution. The actual average number of organic site visits after contribution exceeds the projection by 0.11 site visits on average and is highly significant. In other words, the Difference in Differences in this case is 0.11 (+2.6%), which is in line with hypothesis H1. Attention [Figure 4: Difference in Differences in number of Newsfeed stories read in 24 hours before/after activity (95% CIs). Dashed line designate the DID prediction for levels after contribution based on the trend evident in the Feedback condition. Square brackets highlight the significant increase in the number of stories read.] How is content consumption effected by contribution? hypotheses H2.a and H2.b postulate that contributors will consume more content overall and particularly more content from friends, respectively. We test these hypotheses using a DID analysis on the measure of stories read (whether at least 2 seconds were spent viewing a story). Figure 4 present the DID analysis of stories read separately for friends and pages' content. While fewer stories are read on average after giving feedback (evident in the decreasing trend in black), the trend for contribution is positive for content from friends and neutral for pages. Similar patterns emerge when we do not distinguish between friends and pages content - people read (on average) slightly fewer stories after engaging in feedback actions and about three more stories (+2.1%) after contribution. The number of stories read from pages also increases on average by 1.2 (+1.8%) compared to the DID projection. Our findings support an increase in overall content consumption (H1.a) and consumption of friend content (H1.b). Interestingly, only the number of stories read from friends increase compared to pre-contribution levels, suggesting a shift in attention towards friends but not others. These results are consistent with an explanation where contributors are engaged for longer periods of time after contribution (as observed earlier), but only extend their consumption habits towards friends. Can the above changes in users consumption habits be simply explained by changes in the quality of content available to users for consumption? can News Feed ranking cause the above effects? further analysis suggest otherwise. To address the question of content quality and availability we use tie strength between the author of the content and the contributor as a rough approximation for content quality. We conducted DID analysis on content quality and found no significant differences in the distribution of quality content available for consumption. In conclusion, we find that contributors consume more content even before making their contribution, but further in- [Figure 5: Difference in Differences in Liking and Commenting rates with friends/page content in 24 hours before/after activity (95% CIs). Dashed line designate the DID prediction for levels after contribution based on the trend evident in the Feedback condition. Square brackets highlight the effect size when significant.] crease their consumption after contribution both overall and selectively towards friends. We rule out an explanation that content quality or availability through feed ranking is behind the observed effect. Judgment Previous sections established that contributors are more engaged around contribution and consume more content, even though the content quality itself remains the same. Does contribution affect judgment of content as postulated by hypotheses H3.a and H3.b? Figure 5 shows our DID analysis separately for likes and comments on friends and page content. The bottom left panel, for example, shows that before contribution, users comment on 0.74% of the stories they read from friends and that this rate significantly increases to 0.77% after making a contribution. DID that were statistically significantly are highlighted in the figure in square brackets. Across the board, the interaction rate in the contribution condition is significantly higher than the rate in the feedback condition. There is no significant change or even slight decrease in the interaction rate after feedback. In contrast, the interaction rate with friends after contribution increases for both likes (+0.11%) and comments (+0.03%). The relative change in commenting rate on friends content is 4% and 1% for likes. There are no significant changes in interaction rate for pages. These findings provide supporting evidence for hypothesis H3.a and contradict the idea of a "fixed-quota" or decision fatigue over time. [Figure 6: Commenting and Liking rates (95% CIs) on friends' content who responded to the contributor's post (indebted) or not (control), controlling for tie-strength.] How much does reciprocity play a role in the increased interaction rate with friends? hypothesis H3.b postulates that contributors are more likely to interact with friends who responded to their content than those who did not. We test hypothesis H3.b by comparing the interaction rate of contributors with friends who responded to their contribution with those who did not, controlling for tie strength. For example, Alice posted a photo on which Bob commented, but Eve did not. Hypothesis H3.b suggest that when Alice goes on to read content from both Bob and Eve, she is more likely to comment or like Bob's content since she feels indebted to Bob. In order to rule out the explanation that Bob is simply a closer friend, we control for tie strength in the indebted and control groups by using Propensity Score Matching and verifying that groups averages are not significantly different. The aforementioned selection criteria yield a subset of more than 700,000 contributors who viewed content from equally close friends that differ only on their prior response to contribution. Figure 6 shows liking and commenting rates for indebted and control groups of friends. Commenting on content from friends who responded to contribution (indebted condition) occurs at an average rate of 1.71 comments per 100 stories read. In contrast, contributors only commented on content from friends who did not respond to their contribution (control condition) at a rate of 1.22%. As a sanity-check we verify that rates of interaction in Figure 6 are much higher than in Figure 5 for friends, since those are interactions with closer ties (and their matched counterparts). The relative change in commenting rate in the indebted condition is a large 39.5% increase over the control group and a more modest 4.8% increase for likes. These findings are highly significant and supportive of hypothesis H3.b regarding reciprocity. In summary, we see an overall higher interaction rates around times of contribution, with further increase after contribution to friends, but not others. We find that when equally close friends respond to contribution (or not) the likelihood of future interactions with their content increases both in likes and more considerably in comments. CONCLUSIONS AND FUTURE WORK Our most salient finding is that contribution entails a more active "state" both before and after contribution. Across the board, contributors visited the site more often, read more stories and interacted more with content than at other times. It is surprising that higher activity levels start as early as six hours before contribution, a finding that cannot to be simply explained by the extra time required to plan and compose a post. Contributors are not just visiting the site more often, they are more active on it, remain active for longer periods of time and selectively read and interact with others. Self-motivated site visits increased after contribution by 2.6%, which is consistent with the theory about feedback expectations. More stories are read overall, which suggests higher engagement. The decreasing trend evident in the feedback condition is completely reversed in the contribution condition, for both stories read and interacted with from friends, which suggests longer engagement over time. The lack of similar increases for page content suggest a model of selective attention change, which is consistent with changes in arousal and self-esteem. The large effect size we found occurring after contribution is in commenting rate towards friends who responded to contribution. Commenting rate increased close to 40% more than the control and a modest 4.8% increase for likes. The more modest increase in likes may suggest a different function for likes than comments, which is less adequate for reciprocation. The fact that contributors choose to comment more than like content may indicate greater sense of indebtedness on their part. These findings are in line the changes in tie strength highlighted by Burke and Kraut [1] and the preference for composed communications as they call it. It is also possible that some fraction of the apparent reciprocity stems in part from contribution prompted by shared events, where people who post about the same joint activity might be more likely to engage with each others' post. We were not able to discern this, however, since our analysis did not include content. While we dedicated a great deal of thought into carefully designing our analysis, it still has several limitations. First, as a purely observational study our findings are only suggestive of the causal relations. The question of whether a contribution leads to higher engagement or vice versa is particularly interesting and challenging. In particular, elevated levels of engagement prior to producing content could be indicative of a greater desire for social engagement during a given time that could result in both posting behavior and other forms of engagement. Second, by focusing on aggregate measures of activity over a period of 24 hours we average out some of the behaviors that only occur at shorter time spans and lose the ordinal aspect of activity. For example, our measures are likely to smooth effects that happen on the next session immediately after posting. Lastly, selecting into our analysis contributors who were active on two different times a week apart introduces a selection bias. While we did work with a sample of millions of contributors it still does not include all contributors. A different methodology should be used to draw inferences about less active contributors. Future work could explore the contribution effect for different populations and contribution types, and further disseminate feedback received and the time scale of feedback. Different populations (e.g. women and men, young and old) engage differently with SNS and analyzing the contribution effect there can highlight differences in those populations. Contributions themselves differ in form, style, content and amount effort and intent embedded in them, which all call for further exploration of their effect on contributors behavior. For example, does posting a photo vs. a status update effect contributor's behavior differently? Other extensions can further disseminate behavioral changes as a result of feedback and evaluate the time-range for these effects. While most feedback received on SNS is positive, even in sites with weaker sense of identity and friendship than Facebook (see Cheng et al. [4] for details), the question remains as to how feedback effects behavior. A closer investigation can examine the temporal aspect of the behavioral changes we identified and try to link between short-term and long-terms effects. Our findings have both theoretical and practical implications. On the theoretical side, we found empirical evidence for behavioral changes that are not yet well understood. Our finding about reciprocity provides a partial answer to its importance in social interactions. However, additional insight into what drives higher levels of engagement are needed. A more integrative theory is needed to understand behavior in natural settings. On the practical side, our findings can be used to support those producing content and the consumption and feedback surrounding that content. REFERENCES 1. Moira Burke and Robert E. Kraut. 2014. Growing Closer on Facebook: Changes in Tie Strength Through Social Network Site Use. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 4187-4196. DOI: http://dx.doi.org/10.1145/2556288.2557094 2. Moira Burke, Cameron Marlow, and Thomas Lento. 2009. Feed me: motivating newcomer contribution in social network sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 945-954. 3. Maria Rosaria Cadinu and Myron Rothbart. 1996. Self-anchoring and differentiation processes in the minimal group setting. Journal of personality and social psychology 70, 4 (1996), 661. 4. Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How Community Feedback Shapes User Behavior. In Eighth International AAAI Conference on Weblogs and Social Media. 5. Coye Cheshire and Judd Antin. 2008. The social psychological effects of feedback on the production of Internet information pools. Journal of Computer-Mediated Communication 13, 3 (2008), 705-727. 6. Mark H Davis, Laura Conklin, Amy Smith, and Carol Luce. 1996. Effect of perspective taking on the cognitive representation of persons: a merging of self and other. Journal of personality and social psychology 70, 4 (1996), 713. 7. Utpal M Dholakia, Richard P Bagozzi, and Lisa Klein Pearo. 2004. A social influence model of consumer participation in network-and small-group-based virtual communities. International journal of research in marketing 21, 3 (2004), 241-263. 8. Thomas J DiCiccio and Bradley Efron. 1996. Bootstrap confidence intervals. Statistical science (1996), 189-212. 9. Christopher D Fiorillo, Philippe N Tobler, and Wolfram Schultz. 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 5614 (2003), 1898-1902. 10. Adam D Galinsky and Gordon B Moskowitz. 2000. Perspective-taking: decreasing stereotype expression, stereotype accessibility, and in-group favoritism. Journal of personality and social psychology 78, 4 (2000), 708. 11. Noah J Goldstein and Robert B Cialdini. 2007. The spyglass self: a model of vicarious self-perception. Journal of Personality and Social Psychology 92, 3 (2007), 402. 12. Alvin W Gouldner. 1960. The norm of reciprocity: A preliminary statement. American sociological review (1960), 161-178. 13. Mark S Granovetter. 1973. The strength of weak ties. American journal of sociology (1973), 1360-1380. 14. Adam N Joinson. 2008. Looking at, looking up or keeping up with people?: motives and use of facebook. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 1027-1036. 15. Cliff Lampe and Erik Johnston. 2005. Follow the (slash) dot: effects of feedback on new members in an online community. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work. ACM, 11-20. 16. Nan Lin. 1999. Building a network theory of social capital. Connections 22, 1 (1999), 28-51. 17. Dar Meshi, Carmen Morawetz, and Hauke R Heekeren. 2013. Nucleus accumbens response to gains in reputation for the self relative to gains for others predicts social media use. Frontiers in human neuroscience 7 (2013). 18. Oded Nov. 2007. What motivates wikipedians? Commun. ACM 50, 11 (2007), 60-64. 19. Zizi Papacharissi and Andrew Mendelson. 2010. 12 Toward a new (er) sociability: uses, gratifications and social capital on Facebook. Media perspectives for the 21st century (2010), 212. 20. Jennifer Preece and Ben Shneiderman. 2009. The reader-to-leader framework: Motivating technology-mediated social participation. AIS Transactions on Human-Computer Interaction 1, 1 (2009), 13-32. 21. Sheizaf Rafaeli, Tsahi Hayat, and Yaron Ariel. 2009. Knowledge building and motivations in Wikipedia: Participation as Ba.. Cyberculture and New Media. New York (2009), 51-68. 22. Wolfram Schultz, Peter Dayan, and P Read Montague. 1997. A neural substrate of prediction and reward. Science 275, 5306 (1997), 1593-1599. 23. Georg Simmel. 1950. The sociology of georg simmel. Vol. 92892. Simon and Schuster. 24. E.E. Smith and S.M. Kosslyn. 2007. Cognitive Psychology: Mind and Brain. Pearson/Prentice Hall. http://books.google.com/books?id=-x-TQgAACAAJ CONFIDENTIAL META3047MDL-072-01105922 CONFIDENTIAL METANMAG-011-01107589