Case Study 3— Zach Schendel, Director of UX Research at Netflix

Read Case Study 1 & 2 here, in case you missed it https://medium.com/@punam.garu/case-study-1-zach-schendel-director-of-ux-research-at-netflix-15c6f57f181a

https://medium.com/@punam.garu/case-study-2-zach-schendel-director-of-ux-research-at-netflix-e8986f840f75

Hi, everyone! So recently I attended this webinar organized by

InVision

with Zach Schendel of

Netflix Design Team

and I found it very informative and interesting.

In the webinar, he discusses the evolution of Netflix’s design and how they went about in deciding the design changes. I am sharing my notes on the webinar here. I have basically organized the information here so that I do not miss out on anything good.

I will be posting 3 blogs to cover this webinar, segmented across the 3 case studies that he discusses.

2013 UI

Situation then

To get to the Navigation Menu, the user had to scroll up or all the way down. And there were some 30 rows of titles the user had to scroll through, in order to get to the bottom of the page.

Observations

1. People hardly scroll up for the navigation menu. It’s just not convenient for them.
2. What they do is, they go back on the navigation and then again come to the homepage, so that they can get the navigation menu on top.

What the Genre Gallery looked like?

1. Within the navigation menu, there was a link to Genre Gallery.
2. On opening that link, the Genre Gallery would open up and it was called categories
3. One main category head was TV shows — where all the TV shows (Netflix and others) were listed.
4. The remaining categories were all the movies, which were grouped into certain genres and listed alphabetically.

Observations

1. There were challenges with the existing Genre Gallery
2. There was a problem in accessing the Genre Gallery because people either don’t know where it is or they might just stumble upon it. But it was not intuitive

Research Methods

The research was focussed on making the navigation and its accessibility better.

Netflix research team Triangulated the analysis, which means that they used more than one kind of research methodology to draw conclusions and come to a decision later.

This is basically, a mix of both qualitative and quantitative research methods.

Card Sorting method gives both qualitative and quantitative data

In-Home Interviews gives rich qualitative data

AB Testing yields quantitative data

Card Sorting Method

Card Sorting Method is a great way to figure out the information architecture and navigation of a product. It basically tries to uncover the mental models of the users, which means that how users store information in their minds.

1. Netflix did this by giving a pile of cards to its research participants Each of these cards have a title written on them, which could either be a movie or series title of any genre.
2. Next, they asked the participant to pick out one card, and then find the rest of the cards which they think will fall into the same group as the first card.
3. Similarly, they asked the participant to create groups for the remaining cards in a way that makes sense to the participant
4. The final step — they asked them to put Labels to each of these groups

Read more about How To Do Card Sorting here — https://www.nngroup.com/articles/card-sorting-definition/

Observations

1. In almost all the cases, the participants had primarily grouped the cards into Movie and TV Show. Under these two categories, the participants had further grouped the cards into other genres.
2. In the then UI, the Genre Gallery wasn’t really showing the categorization the way people were thinking in their minds.

In-Home Interviews

The research team visited users at their homes and tried to understanding:

1. The kind of contexts the users would watch a Movie?
2. The kind of contexts the users would watch a TV show?

Moment of Truth — Family Movie Night

Nobody does a Family TV Show night. Family Movie Night has been the classic way to spend family time together, where the parents and kids get together and they order pizza and popcorn and enjoy a good movie together. Sometime the users might pick a new movie for the kids to watch or maybe they will watch the same favorite movie (especially with kids this is a usual behavior). And the family would bond over this movie and they will have a common topic to talk about.

Hence mixing Movies and TV shows in the same UI make the choosing process even more difficult for the family.

Takeaways

1. Triangulation of Data — It’s important to triangulate data because then it gives the team actionable insights instead of just plain theory. Here, for example, the research team could have just done In-Home interviews and found that people watch TV Shows and Movies in different contexts. But when they did Card Sorting along with this, they could produce substantial data to the Product Team and Design Team, that would prove the exact same thing that they discovered during the In-Home interviews.

This helped the research team in convincing the other two teams to broadly categorize the titles into TV Shows and Movies and then within those categories split into genres.

1. A/B Testing — Sometimes users can be misleading in these sorts of things. They might tell you that they want certain specific things, and when you give them that specific thing it might hinder their behavior or come in the way of the kinds of behaviors you are trying to create. In this case, Netflix did an A/B test with the old menu and new menu. The new menu had the Movie and TV split and within each split, the shows were categorized into genres. Here the users actually did engage more with the features that they said they wanted during the research.

Hence, the new menu design saw more engagement with both the TV category and Movie category during the A/B test. Plus users always had the option to go back to the Home screen and see everything (Movie + TV mix) if they wanted to.

It is important to think about the real user contexts people are in. To find the kind of pain points and unmet needs they are experiencing in those contexts and to design to help those users improve those pain points or to meet those unmet needs.

But when you are doing that, make sure you are actually going where they are and not just creating an experience or bolt on to your current experience that will not engage the users.

So it’s essential to find those desire paths and find ways of balancing design with utility because the users are trying to accomplish a goal.

And the best way to do so is to have interactions with Designers and Product Managers you are working with and think through the business case you are creating and the best way to create the most impactful business case is by triangulating data and getting those insights and new designs out to test and seeing if they actually change business metrics.

-By Zach Schendel

Watch the full video here — https://www.youtube.com/watch?v=ld7RfeZNtXs&feature=youtu.be