Team /
Katie Borgia
Lu Cai
Danny Vu
Role /
Researcher
Timeline /
8 weeks
Moderating a usability session with a participant over Zoom
HULU WATCH PARTY
Understanding users' experiences with starting and navigating Hulu's group streaming feature.
INTRODUCTION
We conducted a usability study on Watch Party, Hulu’s synchronous group viewing feature. The feature allows Hulu subscribers to stream movies and shows together and connect from different locations. It includes a synchronization feature so that users' videos are synced with one another and a group chat where users can interact. This study was conducted for a Usability Studies course.
The goal of the study was to learn how users experience Watch Party. After doing an initial heuristic evaluation of the feature, we developed the following questions to focus on:
-
How easily can users find the Watch Party button?
-
Are users successful in sending links to other Watch Party participants?
-
What obstacles prevent users from starting a Watch Party session?
-
What obstacles prevent users from navigating a Watch Party session?
-
How well do users understand and use the symbols and icons in the video player?
-
How well do users understand the system feedback?
SUMMARY
-
All participants were able to initiate Watch Party and felt confident using the feature again
-
Participants experienced a lot of frustration looking for the Watch Party button and were surprised by the lack of information or prominence about the feature on the homepage
-
The limited interactivity in Watch Party and confusion about synchronization were some reasons participants were unlikely to host a Watch Party
-
Increase discoverability of Watch Party feature, add interactive functions like voice chat, and automatically sync host and guests’ videos
PARTICIPANT PROFILE / RECRUITMENT
We identified the following characteristics for our target user:
To recruit, a screener survey was created and deployed in our personal and University of Washington networks. We recruited 7 participants, 1 for a pilot and 6 for the test sessions.
METHOD
All sessions were conducted remotely and recorded. Sessions were conducted with 1 moderator and another teammate acting as a dummy participant who would join and interact with participants in their Watch Party. The dummy participant would also purposely fast forward their video, triggering an "out of sync" message to appear on participants' screens. We wanted to observe whether participants understood the message and how they would act in response. Participants were asked to think aloud as they completed tasks. The format of each session consisted of the following:
-
Pre-study questions
-
Tasks
-
Initiate Watch Party
-
Invite a friend to your Watch Party (dummy participant)
-
Start the Watch Party
-
Explore the interface
-
Interact with your friend (dummy participant)
-
Start a new Watch Party
-
Post-task "ease/difficulty of completion" Likert scale questions
-
System Usability Scale (SUS) questionnaire
-
Post-study questions
FINDINGS
Data from each participant was organized and tabulated on a chart. All participants were able to complete all tasks. We found that the Watch Party icon was recognizable and clearly understood as all participants were able to quickly identify it. Participants also felt confident starting another Watch Party after their initial experience. They were able to use a shorter and more direct route compared to their first attempt.
To define areas of improvement, we used participants' quotes, their "ease/difficulty of completion" task scores, and other collected measurements and notes to identify where they experienced issues or decreased satisfaction. We used Jakob Nielsen's severity rating scale and determined the severity of the issues by their frequency, persistence, and impact on completing tasks.
Nielsen's Severity Ratings
4
Usability catastrophe: imperative to fix this before product can be released
3
Major usability problem: important to fix, so should be given high priority
2
Minor usability problem: fixing this should be given low priority
1
Cosmetic problem only: need not be fixed unless extra time is available on project
0
I don't agree that this is a usability problem at all
Areas of Improvement
3
Watch Party feature was not easily discoverable
2
Confusion on how synchronization worked in Watch Party
2
Lack of information on participants in Watch Party group
1
Watch Party icon not prominently displayed on the Details Page
1
Limited interactivity functions in Watch Party
RECOMMENDATIONS
Below is a sample of the evidence and recommendations we prepared. For the full list, please refer to our detailed report here.
Severity 3 - The Watch Party feature was not easily discoverable
6 out of 6 participants expressed frustration while trying to initiate a Watch Party
6 out of 6 participants had issues finding the Watch Party button
“Just [looking] for something that will signal that I can activate the Hulu Watch Party — so I don’t know what I’m looking for right now.” - P4
Recommendations:
-
Highlight the feature on the homepage
-
Add multiple opportunities to start a Watch Party, such as within the video player
Severity 2 - The Watch Party synchronization function was confusing
6 out of 6 participants did not understand how their video was syncing with others
“I am concerned that out of sync is something that I would have to click, like it’s not something that’s automatic.“ - P4
“I wonder if I pause it, does it pause for her too? I can’t tell.” - P1
“Say we have 7 people on the call, how do I know who’s out of sync and how do I know how ahead or behind they are?” - P6
Recommendation:
-
Update feature to sync videos automatically
Severity 2 - Lack of information on other participants within messaging feature in Watch Party group
4 out of 6 participants mentioned being unable to identify guests in their Watch Party and in the chat
“How do I see my party?” - P3
“It’s cutting it short for no reason and it’s pretty easy to have 2 people with the same first letter of the name.” - P6
Recommendations:
-
Allow users the ability to choose a display name
-
Provide information on all guests in the Watch Party
Participants wanted more identifying
information and customization
REFLECTIONS
For future iterations of this usability study, we would ask participants to set their video player in their preferred viewing experience. After the sessions, we realized that the chat window becomes hidden if you make the video full-screen which may impact how participants use Watch Party. We would also perform a competitive analysis of similar streaming services' synchronized viewing functionalities.