Exercise 83 – What metrics would you use to track the improvement of sense of community within Facebook Groups?

Post and review answers and feedback to answers in the comments section of this post.

See also:

How to answer a product improvement question in a product manager job interview

List of product improvement questions for product manager job interviews

4
Leave a Reply

avatar
2 Comment threads
2 Thread replies
1 Followers
 
Most reacted comment
Hottest comment thread
4 Comment authors
Angelina FominalearnlifelongBijanonequest Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
onequest
Guest
onequest

As I see it, Facebook groups provides a way for Facebook users to connect with other users through a shared interest or real life/offline relationship. This space enables these Facebook users to easily socialize in a group like setting; planning events, discussing topics etc. Facebook also allows groups to be private/closed off or more open and public facing. Is this a good way to think about the Facebook groups product and how it’s used? Sure.

I understand Facebook’s mission is to bring the world closer together. A great way to do that is by creating a space for communities that people from all walks of life find interesting, and if they do not exist, allowing any one to create a community they fit into. I think that’s the power of Facebook groups. It brings the world closer by uniting individuals from all over the world over shared interests but even when this happens users may not feel a sense of community.

When I think of sense of community, it needs to deliver on the user goals below:
-is welcoming
-has friendly associations
-is supportive and respectful
-active, alive, and with a heartbeat
-personable and easy to communicate with others

In this case, the user’s goal is to experience a sense of community by doing so we derive our business goal of increasing user engagement with Facebook groups.

When I think about how a user interacts with groups there are some key interactions that come to mind:
1) User discovers the group from an invite, news feed share, or maybe a search
2) User joins the group and has new user experience
3) User is a member of the group and can reads posts, make a post, discuss with other users, view other group media
4) User can come back to the group page with a notification, or a mention by someone in the group
5) User invites friends to come to the group

Of these interactions, I feel as though the 2nd, 3rd, and 5th steps closely align with our goal of improving the sense of community because they are the interactions where users that become members of groups form their impressions, and develop their feelings and opinions about the community. Of course, the sense of community they feel can be changed in the future, but if they are established up front it makes it easier to manage them moving forward.

Let’s look at metrics. Below are the following metrics I’d look at for each step.
Step 2 and 3:
1. % of users that join a group after viewing the group (public group) [activation]
2. % of users that engage (post, comment, react) with the group within 1D of joining [engagement]
3. avg engagement score (comments + likes) per post by a new or returning user in the group [engagement]
4. % of users that friend or follow another user of the group within 1W of joining [engagement]
5. % of users that are returning members compared to new users [engagement]
6. % of 30 DAU [retention]
Step 5:
7. % of users that invite a friend to the group [referral]

Now that I’ve listed some relevant metrics I’d like to evaluate them on how they directly reflect our goal.
1.indicates how effective a page is at showing value to the user (through a large number of recent posts, or # of new members) would reflect an active community.
2. indicates how safe a user may feel, like they are at home in their own skin
3. indicates how supportive and welcoming existing group members are to new and old users, but this may depend on the type of content that is posted
4. indicates how close users are with each other, and how friendly they are, but this may come across as weird behavior and not performed by many users.
5/6. indicates if the community brings repeat value to users
7. indicates if users will promote a group, but there are other reasons a user may invite friends and this may not be used by a lot of users.

Based on above, my primary metrics are: 2, 3, 5 and my secondary metrics are: 1, 7, 6, 4

To test our improvements, at a high level, I’d A/B test them.
-split new users into two groups, half would see the new features and half would not, then I’d compare our primary metrics.

I can discuss more about my testing approaches if you like. But in summary, to measure our improvements to the sense of community in Facebook groups I’d focus on the new users’ experience initially by measuring how they engage with the community, and how the community engages with them, this should tell us if our improvements were a success.

Bijan
Guest
Bijan

Hi Onequest
I think it’s a great answer with a great structure. You’ve described the product and its goal, the customer journey, and the metrics that matter in each step of the journey. Here are a couple areas of improvement:
– Perhaps you can categorize the metrics based on the steps of the customer journey (e.g. awareness, acquisition, activation, etc). You can see the list in my how to answer a metrics question article.
– I would have described the criteria for evaluating the metrics in more detail. A table approach can help you ensure that you’re evaluating all metrics with the same criteria.

learnlifelong
Guest
learnlifelong

Not sure which features you are referring to in this sentence “split new users into two groups, half would see the new features and half would not”. could you eleborate

Angelina Fomina
Guest

What is the meaning behind groups? What is it trying to accomplish?

As you said in the questions it’s trying to create a sense of community. Groups bring together people so they can feel connected, less lonely and more supported in our largely disengaged and individualistic western society. Groups start movements, motivate people to meet offline and help people progress in their professional and personal lives. Groups are places for like-minded communication.

Facebook Groups can be used by:
1. Individual users
2. Businesses [nonprofits, corporate, independent, small to large, etc] instead or in addition to Facebook pages.

For many different purposes:
1. To bring a community together around a cause, topic, interest to – chat with members over wall posts, messages, share resources
2. To help people make connections through the group (add friends)
3. To maintain a personal presence for a business brand and have a high touch of personal interactions with potential customers
4. To bring together a community of people that serve as leads to potential purchase a products/service the organizer is offering
5. A more private group between friends to share unfiltered thoughts with each other

Groups can be 1) public and visible to everyone 2) require user to request access. Posts can also be 1) automatically posted or 2) require approval from a moderator.

Given the above feature goals and philosophy, how would I determine in plain language if a Facebook Group has a sense of community?

1. Tracking growth in members over time. How quickly are communities in different categories growing?
2. Tracking Daily Active Users defined as users that spend 10+ seconds on the group a day, post, like or comment on the main page news feed.
3. Tracking # of private messages that were initiated as a result of members meeting through the group.

The most important metric I would look at is Daily Active Users per group. To expand on this metric I would 1) look at how this metric is trending over time (especially if we made improvements to make groups more engaging) and 2) % of DAU vs. not DAU per group type. Daily activity on a group signals that it’s a sticky community that is providing a lot of value for the members.