+1 vote
68 views
Post and review answers and feedback to answers in the comments section of this post.
asked in Product Improvement by (548 points)
edited by | 68 views

4 Answers

+2 votes

I will first ask the interviewer to clarify what they mean by “improvement” and what the objective of the improvement is. Let’s say they suggest the objective is to make it easier for the users to discover movies / TV shows they like to see.

There are also various Netflix interfaces. The only interface I currently use is the Apple TV / Google TV interface. I will ask the interviewer if it’s ok to focus on the TV console interface. Let’s assume they say yes.

I can think of a few user groups that currently use the Netflix app on the Apple TV and Google TV and search for movies.

– High rating movie users: They are interested in watching movies that are highly rated by the critics.

– Popular actor / actress movies: they chose movies based on the casting crew

– Explorer : They like to browse through movies and chose them based on name, movie cover image, and other things that might come appealing

– Follower: They chose movies to watch based on what’s recommended to them by Netflix

There are a few other user groups I can think of but for the sake of this interview, I will focus on the user group that likes to watch movies that are highly rated. I can think of a few unique needs among this user group:

– They are willing to spend time to do research about movies before watching them
– They care about the ratings of the movies
– They exchange names of good movies to watch with friends that have similar tastes in movies
– They care about the director name as it’s usually an indication of the quality of the movie
– They want to know list of awards the movie has earned

Above are a few unique attributes of this user demographic. Based on the needs described above, I can think of a few solutions that makes the browsing experience more meaningful for these users:

– Enable writing reviews on movies
– Enable search based on ratings
– Enable users to share / recommend movies to each other

Now, I will evaluate each of the ideas listed above based on impact on the customer experience and cost of implementation:

– Enable writing review – medium impact on CX, Cost of implementation is low to medium (Netflix has to develop process for accepting reviews and presenting them in meaningful ways, filtering out spam, and rewarding user who submit reviews. The cost will be low if Netflix partners with third parties such as IMDB to obtain the information.)
– Enable movie search based on ratings, medium impact on CX (some users will miss out on the large assortment of Netflix movies with lower ratings), Cost of implementation is low
– Share with friends : high impact on CX, low cost of implementation

Based on what I’ve listed above, I suggest implementing features 3 to improve the Netflix experience in the Apple TV / Google TV. I would enable users to share / recommend movies to each other from the Netflix app. Netflix can embed this by adding a new section of movies called “recommended by friends”. It enables users to view list of movies recommended by friends. To share a movie, Netflix can present a “recommend ‘TheMovie’ to a friend” option at the end of the movie. Once user clicks on it, they are asked to provide the email address or username of the recipient or select from list of previously contacted friends.

I can also measure the success of this feature by looking at a few metrics:

– rating given to the movie by the recommendation recipient vs. the rating of the movies they generally watch
– % who start watching a recommended movie and watch the whole movie. If the % is higher than the average of this number among these users with other movies, it’s an indication that the recommendation engine is delivering value to the user

answered by (206 points)
0 votes

I really like the way you have articulated your improvement idea. Thank you!

answered by (135 points)
0 votes

I would first start by defining the goal of Netflix, which is to allow users to watch TV shows, movies and other content on any device on a subscription basis.

Next I would discuss the objective of why you want to improve the service?
1. To expand the user base? New market? New customer segment?
2. Increase user engagement: Get existing users to watch more
3. User retention: prohibit existing user churn or unsubscribe

Since Netflix monthly fee is pretty low compared to other paid media (cable), I would not say 3. is an issue.
For the purpose of this exercise, I would foucs on increasing user engagement.

Before getting into the problems, I would look at who my potential user groups are:
1. Users who are sure of what they want to watch – would go and search for specific content
2. Users who are looking for recommendations and would then follow on
3. Both

I would choose 2. and 3.
Now couple of problems the users might be facing:
1. Recommendations do not match interests
2. Not enough latest movies and TV episodes
3. Search not refined enough: can search by name/title/actor etc. but not by metadata
4. Not enough regional/international language titles
5. No. of devices allowed for simultaneous streaming not enough

Tying back to our goal of increasing user engagement, I would shortlist 1 to 4 and suggest the following improvements:

1. Allow tagging a media content and search based on tags. For example, looking for a documentary on travel + Mexico + food —> search result —-> Episode of a Food/Travel TV show showcasing Mexico… you get the point… (High Impact, Medium Complexity)
2. Allow user to subscribe/follow a particular set of content, for example, subscribe to a TV show “SCANDAL” (High Impact, Low complexity)
3. Notify user whenever a new movie/TV episode is added (based on previous views and subscribed content) (High Impact, Low complexity)
4. Allow user to request addition of a movie of choice and/or a TV show (High Impact, High complexity)
5. Allow users to give reviews (High Impact, Medium Complexity)
6. Recommendations based on what your friends watched – “You may also like because your friend watched” and allow to recommend to friends. (High Impact, Medium Complexity)
7. For more accurate recommendations, take a survey/quiz when a user signs up on what type of content do they like to watch. Something similar to other subscription services. (Medium Impact, Medium complexity)

Last step is to measure metrics and KPIs to check if the above changes did lead to increased user enagagment:
1. No. of users writing reviews
2. No. of users who actually completed watching a movie/TV show they started
3. No. of times a user has recommended content to his/her friend
4. No. of times a friend has viewed a recommended video

Hope this helps.

answered by (13 points)
0 votes

I will answer this question by breaking it into sub-questions like these:
1> Describe
2> Map out user journey
3> Clarify scope
4> Choose metric to impact
5> Choose persona
6> Use cases and pain points
7> Prioritize use cases
8> List Solutions
9> Prioritize solutions
10> Summarize

Let’s jump in.
1> Describe: Ask the interviewer what they aspect of Netflix need to be examined.
Who use it?
How are these users interacting with Netflix?
Do they want to improve experience on Netflix.com
OR
Do they want to improve experience on the thousands of smart devices that run Netflix app.
OR
Do they mean improve Netflix by having more original programming or documentaries?

2> Map out user journey:
Now would be a good time to map out the user journey:
> User logs into Netflix
> User gets recommendations of movies
> User searches for a specific movie
> User starts watching this movie
> User usually returns back to Netflix.com to complete the movie
> User sometimes browses new movies & shows and watches them sometimes.

3> Clarify scope: 
Ask interviewer to clarify scope
Do they want improvement in terms of video views, overall signups, or reduction in people dropping off?

Sometimes multiple goals can be achieved.
Also, ask what resources are available for this project ($$, people, etc.)

4> Choose Metric
Finalize the metric
> Let’s say you both agree on “Timer spent per subscriber watching Videos” on Netflix.com

5> Choose persona
Now let’s try & identify the user personas:
a) Family person – 30 to 45 – that uses 1-2 devices and usually watches children’s movies & family shows
b) Young Male – 18 to 30 — that users 3-4 devices and usually watches sports & action movies & thrillers.
c) Young Female – 18 to 30 — that uses 1-2 devices and usually watches documentaries & thrillers & love stories.

Let’s say the interviewer says we want to improve video watch time for Segment a)

6> Use cases
Use cases are extracted from the user journey. Go back and review the user journey that was outlined in previous steps.

Based on the user journey, we can think of several unique use cases:
i> Search for movies. Often times misspelled words are not auto-corrected properly.
ii> Get recommendation of movies. About 40% of users do not find recommendations valuable.
iii> Complete movie over 2-3 days. This is often the case for family people that can’t watch a full movie in one go.
iv) Login into account
v) User explores new shows/movies. Exploring is paired with discovering new shows and about 80% of users go back to what they were watching earlier.

7> Prioritize use cases
Unless there is a specific problem with viewing the videos or logging in, it might make sense to focus on these 3 use cases as these would give us benefits across the board for multiple personas.
a> Search for movies
b> Get recommendation of movies
c> User explores new shows/movies

8> List Solutions
a> Search for movies : Improve search algorithm & auto-correct capability.
b> Get recommendation of movies: Improve recommendation algorithm
c> User explores new shows/movies: Increase new shows/movies offered on the site.

9> Prioritize solutions
Solutions have to prioritized based on cost & ROI.
Let’s come up with some assumptions about the cost and ROI.
a> Improve search algorithm : $300K over 2 yrs, would increase video viewed time by 10%
b> Improve recommendation algorithm : $500K over 6 months, would increase video viewed time by 10%.
c> Increase new shows/movies offered on the site : $10MM over 3 yr, would increase video viewed time by 20%. It would also impact other metrics such as new signups.

Based on these numbers you can already see that there are different directions one can go based on window of opportunity & willingness to spend.

This would be the time to turn to the interviewer and start a dialogue about what would matter most to Netflix.

Each of these solutions have merits and we could definitely pursue each of them independently. However, the answer in such cases is usually, lowest cost, highest ROI.

Therefore the winning strategy would probably be:
b> Improve recommendation algorithm : $500K over 6 months, would increase video viewed time by 10%.

10> Summarize
The proposed solution for investing is recommendation algorithm was selected as it would give us high return in terms of increased page views within 6 months and if it works well it could be serving multiple personas.

In the end, for a company like Netflix, there are often multiple metrics to keep in mind and of course competitive pressure to deal with as well. They will probably be implementing a) and c) simultaneously as well.

answered by (77 points)

Please log in or register to answer this question.

Privacy - Cookies - Terms - Contact