+2 votes
2.7k views

You are a Growth Marketing Analyst at FB and  you have been tasked to identify reasons as to why users signing up from a country are not returning to FB after 30 days.

What will be your hypothesis? 

What data/metrics will you look at prove or disapprove each of the hypothesis

How will you prioritize your problem to solve?

How will you measure success of your change?

asked in Problem Solving by (14 points) | 2.7k views

6 Answers

+2 votes

Clarifying questions

  1. is this a new country for FB?
  2. if not, has this started happening recently? how long has it been happening?

Assumption: FB has been in this country for a long time. This started happening recently 3-4 months

[repeat question now] Ok, so the change is recent and the new users who are signing up are not returning after 30 days

Let me think about what a new user needs on FB
- New user is looking to socialize with their friends and engage with meaningful content on the platform

>> Hypothesis #1: they may not be finding enough friends to connect and thus the platform may not be engaging to them

>> Hypothesis #2: New user is looking for content to engage with and there not finding enough engaging content

Data that I will look for: 

  1. # of connections made per new user per month in that country: 
    1. I will compare it within the country before this drop started occuring and also to other similar countries 
    2. I will also look at % of times People You may know (PYMK) feature was shown to the user in their newsfeed and % of connections user made from the feature. This will help me verify if FB showed enough friends and whther the firends were relevant or not. Based on % times this feature was shown, I can devise an A/B test to show this feature more and validate further if it helps improve 
  2. % of non-friend content shown to new user in the newsfeed:
    1. I will again compare it within the country before this drop started occuring and also to other similar countries 
    2. I will like to conduct an A/B test by showing relevant pages or groups to them in the newsfeed (based on their demographic, FB can gues their interest; since they are new users FB may not have enough info about their interests) and by showing the pages, we can guide them to find new content. We see if the test group stays with FB compared to control to validate the hypothesis. 

I will measure the success of the tests in terms of improvement in 30 day retention. Since this is metric we were concerend about, we will measure the improvement in the metric

answered by (17 points)
+2 votes

Comprehend the problem:

1. Look at the demographic data and understand which users are not returning after 30 days. Broadly, using demography user base can be divided into younger generation (<22) and older generation (>22).

2. What stage the product is in that country and why? Product could be initial, growth, mature, decline phase. And in each phase the problem would mean different things. E.g. new users in initial phase are early adopters and them not returning to FB could mean product-market figment issue. I am going to assume FB is in growth phase.

I am going to concentrate on 1 persona i.e. younger generation.

Why are younger generation user not returning to FB after 30 days? 

Let’s understand why they joined FB, their need is to connect to their social network and wider network so that they can develop good relationships and share their life(photos and videos) and world view.

 The Use cases  they would engage with FB could be:

1. Create, share and discover content with their network. Metric: avg. no. Of weekly content created. And avg. no. Of likes or share per week.

2. Engage with broader content pushed by FB posts especially around their interests. Metric: weekly car and weekly share 

3. Interactions with their network either through content or group activities. Metric: no. Of groups joined and weekly contribution 

4. Adding new connections. Metric: weekly connection added.

I will compare data with users who stayed on FB to prioritise my effort. At this stage, I would assume that users who stay back find more value in FB bcos of their connections vs users who leave there is not enough stickiness for them to stay around. So, I would prioritise UC4. I would then look at UC1 and 2 to improve engagement.

How can we improve the no. Of connections? Feature suggestions:

For this we have to check the ctr for ‘people suggestions’ and ‘invite’ features.

If CTR for ‘people suggestions’ is low it could be 

A. Not enough folks to recommend 

B. Feature not visible

Therefore, improve ‘people suggestions’ to not only display people to add but to follow (e.g. influencers).

Second, Show people suggestions at the top of my wall

Third, display as suggestions new connections added by anyone in my circle.

If invites sent out by the user is less, then improve invite feature.

Integrate with phone contact and allow users to send out invite using sms or whatsapp.

Improve the invite message by showcasing what FB has to offer.

For anyone joining using invite link add them to invitee network.

The above feature improvements should be tested using A/B testing and then rolled out widely to arrest the problem of users leaving FB by increasing their network.

 

 

answered by
0 votes

Understand the bell curve for product growth.

Whether the product is in initial, growth, saturation, stagnation(decline) phase in that country.

Assumption #1: Assuming the product is in growth phase.

Analyse whether the user is signing up for facebook or for using some other applicatons (using sign up with facebook).

Assumption #2: Assuming the user is using facebook app.

Analyse which of the user engagement metrics is declining : fb chat, fb groups, fb pages, user posts or quiz/fb apps based posts.

 Analyse the last activity that the user spends time upon before leaving the page.

Solving the problem:

  1. If the fb posts feature is getting less engaging: Engage the users more by showing the posts by dividing into several groups and applying A/B tests: Posts by direct friends, posts by mutual friends, trending posts etc.
  2. If fb pages or fb groups are getting less engaging: Encourage more group posts in the main feed of the users who're the top commentors/ medium commentors/ less commentors.
  3. If quiz/ fb-app posts are less engaging : Suggest(Incentivise) the app developers to create more new /innovative/ trending apps.

Measuring the Success:

Analyse the results of the hypothesis over a month and apply the appropriate changes over the next months and observe the user engagement/ the rate of users coming back within 30 days. 

answered by (42 points)
0 votes

Assumption: users use FB mainly for socializing with their friends. 

This is an on-going existing problem, which was not caused by specific events or incident.

Hypothesis 1: The new user has not formed a specific habit of using the app or she forgot to use the app. 

Hypothesis 2: After the user registered, she does not feel the need of going back to the app

Hypothesis 3: the user is simply not a target user. The user kept going back to the competitor app such as snapchat for social networking. 

Data to be collected:

(1) demographics data about these group of users to understand if they fit into FB's majority user profile. For example, age is an important factor. Teenagers tend to use snapchat. Younger age implies that the users are simply not the target users for FB. Therefore, the solution should be more at the strategic level - whether or not FB should study the competitor's features and customers and enter new territory by developing new features/products/buying other apps

(2) Study those users' friends' usage of FB and separate them into different buckets. If the user has a group of highly actively friends on FB, the user might just forget to use the app or need a little bit more influence from their friends. A reminder email to the user or even remind the user's friend on FB to say hi or interact with the user on FB might help solve the problem.

If the user's friends are not active on FB, then, in general, the user might find FB is not necessary for her to socialize with her friend. Therefore, FB might want to target other than socializing features such as pushing random news or random interesting content to get the user's attention on FB's other functions. 

Measure: 

Post launch of the solutions, tracking user's login behavior, CTR of reminder emails, content they tended to consume, features they use most, interactions with their FB friends, growth of FB friends etc.

answered by (13 points)
0 votes

Clarifying Questions:

If this is a new country for FB, or an existing country.

If not, how long has the problem been going on? And, what percentage of new sign ups don't return to the app after 30 days.

How this rate is different from previous rate?

Assumption: Facebook has been in this country for a long time, this is a recent problem. A problem in the last 3-4 months. %60 of the new sign ups don't return to FB after 30 days. Let's assume that this rate was on average %30 in the prededing 12-month period.

I would begin looking at this problem from several angles.

Angle 1: Context. Hypothsis#1: A specific change in the context, i.e. sociopolitical and economic situation of the country under discussion is causing a significant reduction. Reasons such as low internet speed; ban either from the government or from local movements discouraging engagment and return; etc. 

Angle 2: Customer Demographics. Hypothsis #2: A recent change in new sign up demographics has caused a low return rate?

             Data Collection: Demographic data; comparative analysis of the last 4 month and the preceding 12 months.

Angle 3: Customer Journey. Hypothsis #3: A recent change in customer journey, has caused a low return rate?

  • Technical difficulty. Hypothesis #3a- there is a specific technical problem that discourages user return. Data collection: needing analytics on metrics such as page load speed/last action etc, to identify potential technical causes for low return.

Content Engagement: Hypothesis#3b. The content is not engaging enough for the users; Because this is is recent problem in an existing market, H3b needs to be investigated alongside our information on previous angles. For example, is there a new mode in the country that FB content is not aligned with, for example multipe ads on luxury clothing during an economic crash/ or for example, are we experiencing a change in demographics whereas the content is geered toward previous demography, like if we have increase in more senior users if the content is priamrily geared toward young users.

  • Social Network: Hypothesis#3c.The new sign ups do not have enough friends, thus little engaging activity and content. This problem would occur for example if we have new demographics, such as teenagers or senior people.

Conclusion:

This is a complicated problem with intertwined aspects; data collection and analysis need to be prioritized and designed step by step. First priorities for data collection are:

1-Major changes in contect. 2-Technical issues in user journey 3-Comparative demographic analysis.

Measurement of Improvments:

Obviously is done via the same metrics that initated this prompt: number of new users returning to FB in the same country over the same length of time, 3-4 months (although the latter needs some consideration for seasonal and time sensitive parameters).

answered by (16 points)
0 votes

Clarifying Questions:

If this is a new country for FB, or an existing country.

If not, how long has the problem been going on? And, what percentage of new sign ups don't return to the app after 30 days.

How this rate is different from previous rate?

Assumption: Facebook has been in this country for a long time, this is a recent problem. A problem in the last 3-4 months. %60 of the new sign ups don't return to FB after 30 days. Let's assume that this rate was on average %30 in the prededing 12-month period.

I would begin looking at this problem from several angles.

Angle 1: Context. Hypothsis#1: A specific change in the context, i.e. sociopolitical and economic situation of the country under discussion is causing a significant reduction. Reasons such as low internet speed; ban either from the government or from local movements discouraging engagment and return; etc. 

Angle 2: Customer Demographics. Hypothsis #2: A recent change in new sign up demographics has caused a low return rate?

             Data Collection: Demographic data; comparative analysis of the last 4 month and the preceding 12 months.

Angle 3: Customer Journey. Hypothsis #3: A recent change in customer journey, has caused a low return rate?

  • Technical difficulty. Hypothesis #3a- there is a specific technical problem that discourages user return. Data collection: needing analytics on metrics such as page load speed/last action etc, to identify potential technical causes for low return.

Content Engagement: Hypothesis#3b. The content is not engaging enough for the users; Because this is is recent problem in an existing market, H3b needs to be investigated alongside our information on previous angles. For example, is there a new mode in the country that FB content is not aligned with, for example multipe ads on luxury clothing during an economic crash/ or for example, are we experiencing a change in demographics whereas the content is geered toward previous demography, like if we have increase in more senior users if the content is priamrily geared toward young users.

  • Social Network: Hypothesis#3c.The new sign ups do not have enough friends, thus little engaging activity and content. This problem would occur for example if we have new demographics, such as teenagers or senior people.

Conclusion:

This is a complicated problem with intertwined aspects; data collection and analysis need to be prioritized and designed step by step. First priorities for data collection are:

1-Major changes in contect. 2-Technical issues in user journey 3-Comparative demographic analysis.

Measurement of Improvments:

Obviously is done via the same metrics that initated this prompt: number of new users returning to FB in the same country over the same length of time, 3-4 months (although the latter needs some consideration for seasonal and time sensitive parameters).

 

 

 

answered by

Post answer and get feedback

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.

Post a question

About this site

Product Management Exercises is the best place to get help preparing for product manager interview questions. Any member of Product Management Exercises can post product manager interview questions, submit answers to the questions, and give feedback to other members' answers.

Follow PM Exercises



Report bug

Privacy - Cookies - Terms - Contact - Twitter - Facebook - Slack - RSS