Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?​
0 votes
in Metrics by (159 points) | 4.1k views

4 Answers

+2 votes

Let me start with few questions:

1. How're we doing it right now? --> Users have an option to 'Like' the post

2. Why did we launch 'Likes' in the first place? --> What do you feel?

Let me think about it. First off, let me segment the different users:

1. Heavy Posters --> These are the 1% of the users who are always active on Facebook. They're posting all the time, adding new friends, commenting, etc. They're are the content creator!

2. Frequent Posters --> These are the 9% of the users, who sometimes participate on daily FB's activities. 

3. Occasional Posters/Lurkers --> These are the 90% of the user who usually lurks around

As a FB PM, I would have seen this pattern and wanted more participation/engagement from all user segments. --> Seems fair assumptions

Let me list the needs/pain-points of each segment:

1. Heavy/Frequent Posters  --> When others don't engage on their posts, they lose motivation to produce content/posts and thereby they come less on FB, and FB loses its power of having an engaging community. 

2. Lurkers --> They like the posts but don't want to comment, they want an easy way to show their appreciation.

Looking at the above problems, 'Likes' seems to be the solution. 10% of users posting and 90% 'Liking'

Now, extending the above analogy, I see we can have more emotions - “love”, “haha”, “wow”, “sad”, and “angry” catering to the distribution of the 90% of the users as not all 90% will have a need to 'Like', they have other emotions like “love”, “haha”, “wow”, “sad”, and “angry”

So, from the above hypothesis, I see # of posts having only 'Likes' will < #posts having 'reactions'

Other Success metrics will be:

1. # of posts with reactions / # posts without reactions

2. Reactions leading to more posts --> # reactions / # total posts --> Cohorts of posts before and after reactions can be measured

3. Reactions leading to more active users --> # unique users before and after reactions

4. Reactions leading to other activities --> 'Add Friends', 'Comments', and 'Chats'


To summarize, I feel reaction will increase overall engagement as we're targeting a larger distribution of user and their emotions, and an ability to portray these emotions will lead to an increase in engagement


by (61 points)
0 votes

First of all, I'd probably need some sort of clarification about what "at launch" means. Are we looking at the performance the very next day after the feature is launched or are we giving ourselves a few weeks to see how things are going to play out? I assume it's the latter and we are going to be evaluating success two weeks after launch.

The goal of reactions feature is to give users more freedom in expressing how they feel about content that gets posted on Facebook. This feature should be particularly appealing to users whose primary method of engagement on FB is via the like button, which, I assume, is going to be the vast majority of FB's userbase.

First, to set some baseline metrics I can look at the historic data to estimate things like 

  1. number of likes over the span of 2 weeks
  2. number of posts over the span of 2 weeks 
  3. avg. number of likes per post
  4. avg. number of likes per user

Second, I can look at the distribution of the sentiment of facebook comments. I am pretty sure this data will already be available because that's probably how they came up with these 5 reactions to begin with. Let's say the distribution looks like this: neutral (50%), funny (30%), sad (15%), surprised (10%), angry (5%). I'd say it's a pretty safe bet that reactions are going to follow pretty similar distribution. 

Next, I am going to make a few assumptions:

  1. Given how these reactions got implemented (you need to hover before they pop up), not all users are going to immediately become aware that this option even exists
  2. In the first two weeks, I'd expect that only around 25th percentile of facebook active users are going to engage with the feature and it'll take the rest of the users more that two weeks to get fully on board

I think these assumptions matter because it is pretty unrealistting to expect 100% of our active users to immediately jump on board of the reactions train. We'll be setting ourselves up for failure if we don't adjust our launch goals accordingly.

Keeping our assumptions and sentimetn distribution in mind, we can proceed to do some math for reaction vs likes volume two weeks after launch. For ease of math, I'll assume that the expected number of likes over 2 week period based on our historic data is 10million. After we roll out reactions 75% of users are going to continue relying on likes to react to post because they are not yet on board with the new feature, so that's 7.5million likes. 25% of the users are going to use reactions instead of likes whenevere appropriate. Based on our sentiment distribution data, we can expect 1.25million likes, 750k haha, 375k sad, 250k surprised, 125k angry reactions. After summing everything up we end up with the following number of likes and reactions per post:

Likes: 8.75million / # of posts
Haha: 750K / # of posts
Sad: 375k / # of posts
Surprised: 250k / # of posts
Angry: 125k / # of posts

I'd expect the numbers of reactions grow and likes decline as more and more people are becoming aware of the feature, but ultimately we are likely to end up fairly close to the distribution of general user sentiment. Unless we continue adding more types of reactions, I think "like" is going to continue being the most popular way of engaging with the post, just because it is the default and the most relevant reaction for the majority of posts.  One thing to note is that these calculations did not accout for the fact that many users were most likely not liking posts that were making them sad or angry, so I think our average number of reactions + likes per user is going to go up due to this and so will the percentage of reactions vs likes.

by (13 points)
0 votes

C: Clarify the Question

Who are the reactions for?
What are the goals of reactions?
Why was it trimmed to the said 5?

I: Identify the Customers

The two main customers for this feature that I see are businesses who need this information, and people who have been resorting to the like button to respond to posts. Who does it add more value to? I would think this would add more value to businesses as it can directly lead to a more efficient monetization with this additional layer of understanding of a user's sentiment. 

There are two aspects of success

1. (Users) Market Adoption and Engagement due to the reactions. Will people post more, respond more, now that there is an option of an appropriate response?

2. (Businesses) How much more efficient can this information make market targeting?

  • Market Adoption
    • Hypothesis: Does the availability of different response options increase the post engagement of the users?
      • Avg count of reactions per post daily pre-launch of non-like reactions
      • Avg count of reactions per post daily post-launch of non-like reactions (hope to see an increase)
      • Avg count of reactions per 7 days (assuming weekends have a spike of activity) pre-launch of non-like reactions
      • Avg count of reactions per 7 days post-launch of non-like reactions
    • Does the availability of different response options instigate people to share more news?
      • Avg number of posts per day pre-launch
      • Avg number of posts per day post-launch
    • Does biasing to positive emotions garner more engagement with the user?
      • Correlation of distribution of categorized posts according to the non-like reaction vs Avg engagements per post.
  • Business Monetization
    • Does targeting using non-like reactions lead to more engagement with the user?
      • Avg count of single clicks on targeted posts
      • Avg count of two-level clicks on targeted posts
      • Avg count of responses (comment, shares, reactions) on targeted posts
Would this number differ by reaction?
I believe that the reaction type is only important for the businesses to understand how to target their products or subjects, but to Facebook it does not make much of a difference. A user is likely to engage in a post that makes them mad, sad, happy, funny etc. Being mad about a cause is not necessarily a bad thing. Sympathizing with a sad cause isn't either. I believe success would differ due to the mixture of each reaction-garnering post, not the reaction themselves. 
0 votes
Clarification Question: What do you mean non-like features here? Do you mean that we are abandoning the likes and switch it to the expressions mentioned above?

Goal: The goal of Facebook I guess is straight forward: to enhance user engagement ( I don't think monetization here is very important)

User Journey:  Let's break it down to various users segmentations.

a) Users who hugely rely on social networks to express themselves and Facebook is already part of their lives; they care more about whether he/she gets more attention when posting. They are eager to attract more comments & reactions.

b) Users who only post when they have important occasions such as birthday, graduation...etc. They constantly update themselves to their social networks. The pain point here is also to receive as many attention as they can.

c) Users who rarely post but they are active on Facebook to keep track of friends. They will click likes to their friends but that doesn't mean they are best friends.

Metrics ( Ranked by Priority and we should conduct A/B testing to the like feature )

User Engagement:  # of posts of each user && # comments in each post && comments received by each user daily  &&  average number of messages of users daily && # of friend request and new connections

User Retention: Average Time of User to Abandon Facebook && User Churn Rate

User Acquisition:   # number of new users signed up

Drawback: My hypothesis is that this feature will distance people and discourage user engagement in a sense that people who don't have a very close relationship won't express certain deep feelings rather than likes. If the user's engagement is down, Facebook's Ads business will also be damaged.

Post answer and get feedback

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.