BBC Sounds TV
Playback onward journeys
BBC Sounds TV app research and design
Challenge & context
The BBC Sounds TV app has a major problem with retention, with our regular users returning just once every 13 weeks.
Objective
We want to understand the various reasons why users do not regularly use the Sounds TV app. We are especially keen to understand why users are drawn to habitual use of audio on demand (AOD) on our competitors products. This will inform our experimentation and feature roadmap for the next couple of quarters.
Discovery
Planning the diary study
BBC Sounds commissioned an external agency to conduct a diary study, we worked with them to plan recruitment and the tasks given to participants in order to shape an effective diary study in accordance to our budget and timescale.
Study overview:
14 Participants
4 day diary study
Post-diary interviews
Post-study analysis
We used affinity mapping to identify themes, looked at previous research and worked with product to identify the theme we would be focusing our efforts.
Emerging themes;
Discovery and onward journeys
Mobile/seamless integration
Personalisation
Search
After considering each theme with the rest of the design team and product manager, we decided to prioritise the “Discovery and onward journeys” theme.
Why start with Discovery and onward journeys?
Less dependent on other teams
We can leverage existing related discovery work
Quicker and greater impact, evidence of existing user behaviour strongest link to this theme
Ideation
Workshops
Using how might we… statements based on testing notes relevant to the chosen theme, I planned and facilitated workshops to ideate on the HMW statements.
The engineering team had stated a desire to be involved in ideation earlier on in the design and development process so I used the workshop as on opportunity to do so, splitting those eparticipating into three small groups with a spread of disciplines.
Each group explored one of the following how might we… statements;
HMW continue a users journey after playback to aid discovery?
HMW encourage more general discovery within the Sounds TV app (most discovery is on our other platforms)
HMW leverage our most common use case (live listening) to encourage more discovery for users?
The ideation was divided into two parts
An Initial discussion, to share knowledge, make notes and quickly sketch and/or discuss ideas
2nd drafts/iterations of ideas explored in part 1, and a hypothesis for the idea
Prioritisation
A follow-up session was used to scrutinise each idea, discussing the impact we think it could have, level of confidence and technical complexity. The hypotheses were each placed on an impact/effort matrix.
Myself and the two other designers on the TV app team, Simon Roberts and Terri-Samantha Middleton selected one idea each to explore in more detail and take it closer toward a viable solution.
There was some duplication of ideas so I began to flesh out two similar hypotheses:
We believe…that displaying relevant on-demand content to the playback page
Will achieve…users playing more on demand content
We will know this to be true…. when we see selects of the relevant content and an increase in on-demand plays per session
We believe...that recommendations in playback e.g. towards the end of a programme
Will achieve…helping users discover content they might enjoy without having to actively search/browse for it
We will know this to be true…if they continue listening to more content (or see an increase in any other actions implemented e.g. bookmark
Iterations - Building out hypotheses with lo-fi wireframes
I explored how this idea could work using low-fidelity wireframes to save time while providing a realistic idea of size and positioning of elements. I shared some ideas with the team that handles tagging of content content across BBC Sounds to understand possibilities and limitations of content recommendations.
Solutions based on displaying content relating to specific artists was deemed too complex but work was being done that should make this more feasible for any future iterations of this idea and if any simpler iterations yeild a positive impact for users, this could provide rationale for prioritising the backend work that needs to be done.
Content & behaviour
I carried out a competitor analysis of TV apps used to listen to audio on demand to analyse the types of behaviours, interactions and content presented to users who are using an app’s player. Features such as auto-play and similar or related recommendations were expected by users so I focused on documenting these journeys.
For ease and efficiency I looked at the curated content already available in BBC Sounds on web and the mobile app to look for any suitable collections that could be ready to go that would require significantly less engineering and editorial effort, or sign-off processes.
Delivery
Refining ideas and planning experiments
Using research insights about content metadata and working with developers helped me design an experience that works well around device and app screensavers and idle states.
It was decided to use A/B tests to discover the effects of these ideas. This was in part due to leveraging and adding value to pending engineering work to implement the ability for running experiments on Sounds TV. It also allowed testing at larger scales. I worked with product, and data analysts to carefully plan these experiments to define the metrics to track.
Impact
The A/B test promoting on-demand content in the full-screen player when users are listening to live radio programmes.
The outcome of the was a positive experiment resulting in a 10.58% uplift in on-demand play starts in the variant indicating a likelihood the hypothesis was valid.
Next steps
After the success of the test the feature was released to 100% of the audience and ran an experiment investigating the on-demand listening context.
I started discovery work for the next iterations of the feature and have been looking at how we can optimise the feature with navigation style, content offering, device behaviour and user segment targeting.
