+8 votes
2.3k views
Post and review answers and feedback to answers in the comments section of this post.
asked in Metrics by (1.2k points) | 2.3k views

4 Answers

+8 votes

The Instagram story feature let’s Instagram users take a picture or video and post it for all of their followers (if account is private) or for anyone on instagram (if account is public) to see. User’s can customize their stories by adding filters, text and interactive elements like tags of other Instagram account, links to their website or polls. Users of Instagram can view each others stories and scroll through all of the stories from the people they follow or use the discovery feature to watch stories from people they don’t yet follow on Instagram.

Goals this feature has:
Awareness – letting users know that stories are versatile – that they can be made interactive and that you can view stories from any public IG profile.
Acquisition + Market-Share – this feature helps FB compete with SnapChat and acquire users from SnapChat + acquire brand-name influencers from SnapChat
Activation – getting users that have never or seldom used IG stories to create or view their first story or five stories.
Engagement – getting users to view, engage with and create more stories daily + getting brand-name influencers to choose IG as the platform of choice
Retention – measuring what effect engaging with stories has on other engagement on IG such as increased following, interaction/messaging between users, etc.
Revenue – measuring how increased engagement with stories = more sponsored stories viewed –> more conversions for advertisers –> increase in revenue for FB from advertising

Different user personas of this feature:
1. Users active on IG but not on viewing or posting stories
2. Passive users that view stories but don’t post
3. Active regular users that view and post (under 5K followers)
5. Facebook advertisers – brands that create IG story ads

I’ll focus on two use cases:
1. Users viewing stories
2. Users creating stories

Why? Because having a large and active AUTHENTIC user base of both users that view stories and create stories is important to Instagram. Authentic users (not brands) are most likely to have habitual interactions on IG and more likely to buy from advertising on IG because they are accustomed to using IG for a personal reason (communication with friends) daily.

Customer Journeys

Viewing Stories:
1. Login to / open IG
2. Look at the top where IG stories are located (possible engage with other aspects of IG)
3. Click on 1 story to view
3 – A: Exit viewing stories
3 – B: Continue viewing other stories
3 – C: Engage with viewed story (answer poll, click on link, swipe up to view more)
4. Exit IG or continue engaging with IG (follow, comment, message)

Metrics for “Viewing Stories” use case:
1. % viewed story within a day of signing up for IG [Activation]
2. % of all IG users that view 1+ stories daily [Engagement] + growth of this #
3. % of all IG users that interact with 1+ stories daily [Revenue] + growth of this #
4. Average # of stories viewed per day per user [Engagement]
5. Average # of stories interacted with per day. Interaction = voted on poll, swiped up to view more, clicked on story link [Engagement]
5. % of all IG users that view 1+ sponsored story daily [Revenue] + growth of this #
6. % of all IG users that interact with 1+ sponsored story daily [Revenue] + growth of this #

Creating Stories:
1. OPTIONAL: take photo or story on phone
2. Login to / open IG
3. Click add story
4. Create story by taking video/photo OR post from phone
5. Customize story
6. Post
7. Look at stories to see # of views

Metrics for “Creating Stories” use case:
1. % created story within a day of signing up for IG [Activation]
2. % of all IG users that create 1+ stories daily [Engagement] + growth of this #
3. Average % growth in followers after creation of story per user (segment users by average with 5K followers) [Retention]
4. % of all IG users that are also advertisers and create 1+ sponsored stories daily [Revenue]

Top Metrics:

The metrics from that list that I’d choose to focus on are:
1. % of all IG users that view 1+ stories daily [Engagement] + growth of this #
2. % of all IG users that create 1+ stories daily [Engagement] + growth of this #

Both numbers will let me know the overall health of the IG stories feature and if more people are getting value out of it by viewing or creating stories. Growth in these numbers also directly impacts the # of sponsored stories IG users will see and increases the % of users that interact with sponsored content. Increasing interaction with sponsored content increases the trust advertisers have in IG ads and increases FB ads revenue.

answered by
0 votes

Instagram Stories is a feature that allows users to share pictures and videos as a stream instead of being added to a profile. This feature is particularly useful for users that want to post regular updates without worrying about adding noise for others by over-posting – friends will need to click the stories ring to get the updates, and the stories disappear after a day, which is an added benefit.

The goal of Instagram is to let users create and share the world’s moments, and this sort of an ephemeral form of messaging was missing from Instagram (and was existent in their competitor, Snapchat), so the goal of launching Instagram Stories would be to add new users (from Snapchat) and increase engagement, so we would look at these two primary metrics to evaluate the success of this feature.

A typical customer journey has the customer log into Instagram, start a story, add photos, and share. Then that user’s story shows up in the list of stories to his/her friends, and they get to view the story.

Metrics that we can track for this:

1) % of Instagram users that click on at least one user story
2) % of Instagram users that create at least one story
3) Average number of media items shared in a story
4) DAU, WAU, MAU
a. Stories
b. Instagram
5) Stickiness (DAU/MAU)
a. Stories
b. Instagram
6) Breadth of engagement (MAU/ total registered accounts)
7) User growth
8) Time spent in instagram, stories
9) Number of friends followed
10) Average number of stories viewed per user
11) Number of sponsored ads shown to users
12) Total revenue made by ads shown to users

answered by (59 points)
0
Hi Anon
Thanks for the answer. You’ve provided a a great list of metrics that matter. It will help if you provide some background as to why some of these metrics matter and which ones would be your area of focus. Also, some metrics listed here are not specific enough to tell the interviewer how they relate to instagram stories. Example is “7. User Growth”. You’ll want to be very specific and clear with each metric, what you’re measuring and how they relate to the goal.
Also, after listing your metrics, I suggest you evaluate them based on some criteria. I’ve provided a more detailed instruction in my how to answer a metrics question article https://productmanagementexercises.com/how-to-answer-a-metrics-question/.
Hope it helps.
0
Thanks bijan!
+1
Could you go in more detail how you evaluate each of those metrics Bijan? Your guide (#5) doesn’t go into detail how you weight each of those with other metrics. For example, how do you size impact, and how do you weight whether the impact to the user experience is positive or negative? Thank you!
0
Good question. This is very contextual and depends on the particular product and metric you’re looking at. You’ll have to use your own judgement to decide how the outcome of a user behaviour could impact a user and how significant that impact is. Sometimes, you can argue both ways but as a PM, you’ll have to wrap your head around these things quickly and come up with some reasonable judgement (or at least argument) as to how it impacts UX.

You don’t need to quantify the size of the impact. It’s very high level. If you see a behavior that can have significant on the goal or UX, make sure to explain why it’s significant.
0
I would appreciate if you could share example of step 5 and 6 from your proceedure in context of an answer to a question
0 votes

Great question. I’d like to make the assumption that our primary metric is engagement with monetization coming through an ads-supported mechanism. Is that a fair assumption to make before I continue?

Clarify – Is web UI still operational? App only. Assume App only.

Acquiring users for Instagram will be based on measuring:
-Clicks through generic story bar icon.
-Clicks to specific user stories – key opinion leaders KOL.
-Clicks to stories as a ratio to number of peers publishing stories.
-Number of active KOL/ user.
-Clicks to stories through Direct Messaging references.
-Clicks from outside references?

Activating users:
-Frequency of viewing stories as a function of app opens.
-Number of stories watched before exiting mode.
-Number of stories watched of those active by publisher.
-Length of time spent watching story.
-Number of stories skipped prematurely.

Retention:
-Time spent in story mode vs feed.
-Time spent on the application from users participating in stories+feed vs those feed only vs stories only.
-Number of users participating in stories only and ignoring feed vs stories+feed vs feed only.

Monetization:
-Number of ad exposures skipped completely vs watched completely vs watch partial.
-Number of ad exposures clicked through to vendor.
-Number of those users that acquire product/ service (assume possible).
-Number of users who subscribe to advertisers with exposure and without exposure in demo.

answered by (110 points)
+1
Hi Steve,

Thanks for the answer. You’ve listed a great list of metrics. I have a couple feedback for you:

– I would first go over the functionality of Instagram Stories, the user journey, how it works, etc. It has two benefits for you: 1- It gives you some extra time to think about relevant metrics as you walk through the user flow 2- It helps you ensure that you’re on the same page with the interviewer on the product / feature goal before moving on to next step

– I would have certainly listed a few metrics that represent level of engagement as engagement is a key metric for this feature. Examples of metrics for Engagement would be # of comments per instagram story, # & % of users commenting on instagram stories vs. instagram posts, # of shares / % of users participating, # of comments per instagram story vs. # of comments, etc.

– I think you’ll need a few metrics that represent the size of adoption. Examples would be # of 30 day active Instagram Stories users vs. # of 30 day active Instagram users. Another one could be % of Instagram sessions that include Instagram Stories.

– After listing the metrics, I suggest you evaluate them based on some criteria you find relevant. In the end, you’d like to only suggest only a few key metrics as the key metrics that you’ll be looking at to evaluate the success of Instagram stories. The thinking process behind the evaluation of various metrics and selecting the important ones represents your ability to execute and focus on what matters.

Hope it helps.
0
Thanks Bijan. Appreciate the feedback. I agree a trade-off for success of stories would help.

Also, just a heads up. I don’t think you can comment on instagram stories. You can DM, but the etiquette there would be such I don’t think it would be effective to track success. Watching complete, partial, skip would be the metric on a per story.
–2 votes

Let's talk about what is IG Stories.

IG Stories give users the possibility to post images and videos for their followers and also edit their posts with filters, tags, links, etc. Followers can see the stories. Users can also search for Stories based on location and hashtag.

Users can reply to stories.

Stories disappear after 24 hours and moved to Stories Archive, which is only visible to user.

Now according to this description, what is the goal of stories? I think the goal is to engage users more with IG. User engagement means for IG more Ad Impressions and more Ad clicks.

So having this goal in mind (user engagement), lets go through the user journey.

  1. Discovery/Awareness: User sees other friends are posting Stories, or sees the button to post a story herself.
  2. Engagement: There are different actions that can be defined as Engagement
    • Posting stories
    • Replying to stories
    • Visiting Stories Archive
    • Searching for stories
    • Viewing other people stories
  3. Retention: User returns to post stories, user begins to follow new people who post stories
  4. Convertion/Monetization: User gets the habit of watching stories and so watches also stories from brands. User does not notice the difference between monetized stories and non-brand stories

Since our Goal was increasing engagement, let's concentrate on the engagement phase. But before that. Why do people spend their time on IG? They are curious to know who is doing what, they want to show to their friends what they are doing ===> they want to be entertained and socially accepted. 

How do they get to find who is doing what? They search for people or content, they follow people.

How do they show to their friends what they are doing? The post images, videos

How are they entertained? The more authentic the post is, the more entertained the user is. The content should be aligned to user's interests and earlier behaviour, or it can be targetting the user herself, e.g. a video of this user posted by a friend

How is the user socially accepted? The user gets feedback on what he posts, through likes, comments, and replies (in case of stories)

I also need to remember I have different types of users:

  1. Brands --->Their goal is to market their goods/services. They want to create a community of people who love their goods and hence spread the word in the online community.
  2. Power users ---> Invest more than 3 hours a day on IG. Their life is defined by what they see and the feedback they get on IG.
  3. Passive users: They interact with IG less than 30 minutes a day. They just watch the posts/stories, but never comment or post anything.
  4. Influencers ---> Users with big audience that target a certain industry, for example beauty, music or food.

I will concentrate of power users and passive users. Power users are already spending a lot of time on IG, but do not have as many followers as Influencers. On the other hand, they follow many people. My hypothesis is that if they produce authentic content, the content begins to be interesting enough for passive users as well so that they interact. 

So once again:

BIZ Goal: Increase user engagement

User Goal: Be entertained and socially accepted

------------------------

So let's go back to the engagement phase. What were the actions that the user does in this phase?

  • Posting stories
  • Replying to stories
  • Visiting Stories Archive
  • Searching for stories
  • Viewing other people stories

----------------------------

How to increase engagement? By giving reward. 

Reward:

What is the reward I get by posting a story?

  • Reply ---> Social acceptance
  • More Followers ---> Having more followers means you are not alone and you are socially accepted. User gets the feeling of importance (even if the importance is defined in her small world of followers, for the user, she is the center of this network)

Other users can be rewarded by being tagged in the story. Being tagged is a reward because it means I am connected to poster and poster has cared about me, plus, it potentially could mean I belong to a social group/activity/event and hence I am socially accepted ====> Things I am tagged in activate more joy in my brain.

-------------------------

Power Users

Some metrics to measure the engagement of power users:

Creation

  1. %Stories vs. %posts ----> (share of stories, are we cannabalizing other prodcuts?)

    Content

    My goal here is to simply understand what content is posted as story more often. This helps me to understand what are the trends in using the story. In addition, I can correlate this information with the #visits and #replies to post, to sort stories and give a higher ranking to stories that are more engaging.

    1. %stories which are live video
    2. %stories that are video vs image
    3. %content (e.g. selfie, nature, animal, family photo)
    4. %stories with friend tags 
    5. %stories with hashtags
    6. %stories with filters
    7. %stories with location

    Retention/Engagement

    Creation

    1. %users who post 1+ story/day + growth---> Users want to continue using stories or they have abandoned it
    2. Growth in the avg #views per story in 1 month ---> Users find the concept of Story interesting and want to be entertained with it
    3. [brands]%stories with paid impression (stories from brands/advertisers)
    4. [brands]%stories with paid links (stories from brands/advertisers)

    Viewing

    1. %user who view 1+ story a day + growth
    2. Avg #stories viewed per user /day + growth
    3. Avg #stories interacted with per user /day + growth
    4. Growth/decline in the %stories that get 0 views in the 24 hours that they are visible
    5. Growth/decline in the %stories that get 0 replies in the 24 hours that they are visible
    6. Monthly growth in %users who have replied to at least one story in that month ---> Reward to poster
    7. Monthly Growth in the avg #replies per story  ----> Reward to the poster. Is the content interesting enough that the user answers to it?
    8. Growth in the #followers for users with more than X stories a day. (X can be the average #stories for power users) ---> Having more followers is a Reward for story poster.  Since the followed person posts a lot of stories, the followers are exposed to more stories and might also find stories interesting and begin posting them (needs to be checked if it is true or not)
    9. %users who watch a story after they were tagged in it --> Reward Do users react to this reward?
    10. %tagged users who replied to story ---> Reaction to Reward ---> Tagged user Rewards the poster by replying
    11. [brands] %stories which had 1+ clicks on paid links (stories from brands/advertisers) ---> Revenue for IG and advertisers
    12. [brands]%stories with paid impression with were viewed 1+ times (stories from brands/advertisers) ---> Revenue for IG and advertisers
    13. [brands]%users who followed a brand/advertisers after watching a story / day + growth  ---> Revenue for IG and advertisers
    14. [brands]Growth in #followers for brands/advertisers with more than X stories a day (X can be the average #stories for brand) ---> Revenue for IG and advertisers

    -----------------------------------------------------

    Passive users:

    I also have a metric to know if things have changed for passive users. It is more challenging to get the passive users for this product, because of the nature of product: The story disappears after 24 hours, and passive user is active less than 30 minutes.

    1. Monthly Growth in #users who have posted their first story
    2. Monthly Growth in #users who replied to a story for the first time
    3. Monthly Growth in #users who for the first time followed a user after watching her story

    -----------------------------------------------------

    Prioritisation

    I will prioritise based on my goal for users: Entertainment and social acceptance. I define entertainment with authentic content, and the social acceptance with the reward system mentioned above.

    To find what content attracts more audience (which might be a sign of authenticity) I use the measures defined in section content and try to find a correlation between features of a story (e.g. content, hashtags, links, video, image) and the amount of engagement (views, replies, followers) it creates. If I combine this information with user behaviours/habits/interests, I can rank stories in a more smart way.

    To measure entertainment, I will concentrate on section Retention, Creation 1-2 and Viewing 1--3, 6-10

    To measure revenue, I choose Creation 3,4 and Viewing 11-14

    ---------------------------------------------------

    Summary

    I described why people might want to post stories and what is their goal: Begin entertained and socially accepted. I described how to extract features out of the content to find entertaining content. I also described the reward system (reply, tag, followers) which would correlate with social acceptance feeling. 

    I skipped the passive users for now. When I have assured high content quality then I can concentrate on acquisition of passive users and I follow my progress by metrics mentioned for passive users.

    answered by (82 points)
    0
    Thanks for the answer Pegah. Here are my feedback:
    I think you’ll want to simplify the structure of your answer a bit. Start with the goal (e.g. increase engagement), then pick one user group (e.g. power users), then list metrics for their journey, evaluate them, and prioritize them.
    I’m not a big fan of “Be entertained and socially accepted” as a goal because it’s not easy to determine if user feels entertained and socialized accepted. Retention can represent whether is happy with the feature or not. I would just focus on a more deterministic goal such as engagement.  
    Your criteria for evaluation of metrics should also include things such as impact and confidence. A metric might improve engagement but the impact of it might be small.
    I think it also helps to list fewer metrics. You’ll want to show that out of all the metrics out there, you’re able to focus on what really matters.

    Post answer and get feedback

    Your name to display (optional):
    Privacy: Your email address will only be used for sending these notifications.
    To avoid this verification in future, please log in or register.