0 votes
2.9k views
asked in Metrics by (158 points) | 2.9k views

2 Answers

+1 vote

Let me start with few questions:

1. How're we doing it right now? --> Users have an option to 'Like' the post

2. Why did we launch 'Likes' in the first place? --> What do you feel?

Let me think about it. First off, let me segment the different users:

1. Heavy Posters --> These are the 1% of the users who are always active on Facebook. They're posting all the time, adding new friends, commenting, etc. They're are the content creator!

2. Frequent Posters --> These are the 9% of the users, who sometimes participate on daily FB's activities. 

3. Occasional Posters/Lurkers --> These are the 90% of the user who usually lurks around

As a FB PM, I would have seen this pattern and wanted more participation/engagement from all user segments. --> Seems fair assumptions

Let me list the needs/pain-points of each segment:

1. Heavy/Frequent Posters  --> When others don't engage on their posts, they lose motivation to produce content/posts and thereby they come less on FB, and FB loses its power of having an engaging community. 

2. Lurkers --> They like the posts but don't want to comment, they want an easy way to show their appreciation.

Looking at the above problems, 'Likes' seems to be the solution. 10% of users posting and 90% 'Liking'

Now, extending the above analogy, I see we can have more emotions - “love”, “haha”, “wow”, “sad”, and “angry” catering to the distribution of the 90% of the users as not all 90% will have a need to 'Like', they have other emotions like “love”, “haha”, “wow”, “sad”, and “angry”

So, from the above hypothesis, I see # of posts having only 'Likes' will < #posts having 'reactions'

Other Success metrics will be:

1. # of posts with reactions / # posts without reactions

2. Reactions leading to more posts --> # reactions / # total posts --> Cohorts of posts before and after reactions can be measured

3. Reactions leading to more active users --> # unique users before and after reactions

4. Reactions leading to other activities --> 'Add Friends', 'Comments', and 'Chats'

 

To summarize, I feel reaction will increase overall engagement as we're targeting a larger distribution of user and their emotions, and an ability to portray these emotions will lead to an increase in engagement

 

answered by (57 points)
0 votes

First of all, I'd probably need some sort of clarification about what "at launch" means. Are we looking at the performance the very next day after the feature is launched or are we giving ourselves a few weeks to see how things are going to play out? I assume it's the latter and we are going to be evaluating success two weeks after launch.

The goal of reactions feature is to give users more freedom in expressing how they feel about content that gets posted on Facebook. This feature should be particularly appealing to users whose primary method of engagement on FB is via the like button, which, I assume, is going to be the vast majority of FB's userbase.

First, to set some baseline metrics I can look at the historic data to estimate things like 

  1. number of likes over the span of 2 weeks
  2. number of posts over the span of 2 weeks 
  3. avg. number of likes per post
  4. avg. number of likes per user

Second, I can look at the distribution of the sentiment of facebook comments. I am pretty sure this data will already be available because that's probably how they came up with these 5 reactions to begin with. Let's say the distribution looks like this: neutral (50%), funny (30%), sad (15%), surprised (10%), angry (5%). I'd say it's a pretty safe bet that reactions are going to follow pretty similar distribution. 

Next, I am going to make a few assumptions:

  1. Given how these reactions got implemented (you need to hover before they pop up), not all users are going to immediately become aware that this option even exists
  2. In the first two weeks, I'd expect that only around 25th percentile of facebook active users are going to engage with the feature and it'll take the rest of the users more that two weeks to get fully on board

I think these assumptions matter because it is pretty unrealistting to expect 100% of our active users to immediately jump on board of the reactions train. We'll be setting ourselves up for failure if we don't adjust our launch goals accordingly.

Keeping our assumptions and sentimetn distribution in mind, we can proceed to do some math for reaction vs likes volume two weeks after launch. For ease of math, I'll assume that the expected number of likes over 2 week period based on our historic data is 10million. After we roll out reactions 75% of users are going to continue relying on likes to react to post because they are not yet on board with the new feature, so that's 7.5million likes. 25% of the users are going to use reactions instead of likes whenevere appropriate. Based on our sentiment distribution data, we can expect 1.25million likes, 750k haha, 375k sad, 250k surprised, 125k angry reactions. After summing everything up we end up with the following number of likes and reactions per post:

Likes: 8.75million / # of posts
Haha: 750K / # of posts
Sad: 375k / # of posts
Surprised: 250k / # of posts
Angry: 125k / # of posts

I'd expect the numbers of reactions grow and likes decline as more and more people are becoming aware of the feature, but ultimately we are likely to end up fairly close to the distribution of general user sentiment. Unless we continue adding more types of reactions, I think "like" is going to continue being the most popular way of engaging with the post, just because it is the default and the most relevant reaction for the majority of posts.  One thing to note is that these calculations did not accout for the fact that many users were most likely not liking posts that were making them sad or angry, so I think our average number of reactions + likes per user is going to go up due to this and so will the percentage of reactions vs likes.

answered by (13 points)

Post answer and get feedback

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.