Increasing trust with music discovery algorithms for higher user engagement

An exploration of the importance of trust in algorithms and ways to increase user engagement with Pandora’s music streaming app

8 min readDec 14, 2020

The King of Music Discovery

In today’s world, we have a plethora of music streaming services to choose from. Pandora, Spotify, Apple Music, Google Play Music, Tidal, Deezer, Soundcloud… the list goes on. But when it comes to music discovery, Pandora has yet to be dethroned. One of Pandora’s greatest and most iconic features is its music discovery algorithm, which is powered by the Music Genome Project. Pandora describes this as the most comprehensive analysis of music ever undertaken to “capture the essence of music at the most fundamental level”. They do this by describing each song with over 450 attributes or ‘genes’ and then using a complex algorithm to organize them.

When the user starts a session with a song or an artist, the algorithm creates a station that plays music based on a variety of factors. They can then train the algorithm to tailor it to their taste by giving songs a thumbs up or thumbs down, which causes the station to update the playlist in real time. Pandora calls this the lean back approach - an experience that requires minimal input from the user while still providing a personalized session. This requires the user to have faith in the algorithm to play what they want to hear. But is there a way to quantify this level of trust? Could the listener be empowered in a way that allows them to be more confident and trust the service to consistently play music they are likely to enjoy?

Importance of trust in algorithm

From as early as the 1950s, researchers have documented many types of predictions in which algorithms outperform humans. It is no surprise that these algorithms have gotten really good with predictions, especially with the advances made in ML and AI in recent years. A recent study conducted by Harvard researchers suggests that people show “algorithmic appreciation” and are more likely to trust it over human judgement.

What does this mean in the realm of music streaming? Given everyone’s unique and individual tastes in music, it is impossible to objectively state whether the discovery algorithm of service A is better or worse than service B. A person’s level of trust over the algorithm could be a huge deciding factor as to which service they choose to use.

I’ve been using Spotify’s Discover Weekly feature for years and grew to love how well it understood me. I estimate that I’d “favorite” one in every eight songs it played, which was a number I was happy with until I stumbled upon Pandora. After spending some time with the app and adjusting the algorithm to suit my taste via thumbs ups and downs, I found that on average I was saving around one in every three songs that played, which was significantly higher than with Spotify. This is obviously anecdotal but having a higher level of control with individual songs in Pandora not only gave me a better experience, but also led me to trust their algorithm more, bringing me to my next point — the human-in-the-loop model.

The human-in-the-loop model

HITL is a simulation that requires human interaction, where the human can influence the outcome of an event or process. This is commonly attributed to flight simulators, self-driving cars, health research and supply chain management to name a few. The idea behind this is that the human has a certain level of control over an algorithm that can affect the results. In the world of self-driving cars, a lot of research is being conducted to find out the optimal level of human control to provide the best experience. This case study on the differences between the human in-the-loop and out-of-the-loop details the pros and cons of both, specifically in the self-driving domain. A lot of research is being carried out in other areas as well to see how humans can be given a high level of confidence with the decisions made by the AI. This paper explores how trust translates as a human operator’s willingness to rely on the actions performed by an autonomous system.

The findings from the research conducted in such domains suggests that humans are more likely to trust AI if they have some degree of control over the outcome. While this is obviously case dependent, I believe this idea can be applied in part to the music discovery application.

Pandora allows users to give songs a thumbs up or a thumbs down, which adjusts the algorithm to suit their preferences. This was taken a step further when the Pandora Modes feature was introduced in March 2019. With Modes, users were given a choice between different types of algorithms used to populate the station. They call this “leaning in just a bit” from their traditional lean back experience. Modes was really well received by their users, indicating that there might be room to lean in just a bit more and give users a higher level of control over their listening session.

Say we introduced a third adjustment feature called Fine-tune, which could be used alongside Modes and Thumbs to personalize the listener’s experience even further. What would that look like?

Here’s a situation

Imagine the following scenario - it’s a slow Friday night at your apartment, and you’re playing an EDM station on Pandora while drinking a whiskey ginger. You give the occasional thumbs up or down, but for the most part, you let the algorithm play the songs it thinks are best. About 40 minutes in, when you’re on your third drink, an absolute banger plays on the station and you love that it’s an instrumental song that is extremely bass heavy. You now want the rest of the songs in the station to be heavy on the bass and not have accompanying vocals, but there is no way to tell the algorithm that.

This could be the intervention point for Fine-tune.

Simplification of the Music Genome Project

The user obviously can’t be slapped with the 450+ attributes to choose from, but what if they were given a few key identifiers of the song that could help them select the vibe for the upcoming songs? For instance, let’s say the current song is different from the previously played songs with a higher tempo, the addition of vocals and the use of violins. If these could be identified and presented to the user in bite sized pieces (let’s call them tags), they could tune the station in real time to suit the mood. The way I envision it, each song would have tags applied dynamically based on the current listening session, meaning that a song could display different tags based on the songs it’s played after.

Concepts

To illustrate this feature, I’ve mocked up two concepts. A “Fine-tune” button is introduced which sits next to the currently existing “My Station”. While My Station controls the station’s Modes, Fine-tine would allow listeners to tell the algorithm exactly what they liked about the currently playing song.

Concept 1: Station Mutation

When the listener selects Fine-tune, a menu slides up from the bottom, similar to how Modes are selected. This menu would display the key attributes, or tags, for that song. Selecting these would adjust the playlist on the fly and provide an even more personalized experience for the current session.

An alternate concept (third image) displays a persistent set of tags eliminating the need for the Fine-tune button and allowing the listener to make changes to the playlist quickly.

Concept 2: Foresight

The Foresight concept takes a slightly different approach. Instead of having the user pick the individual tags, they are presented with a short list of upcoming songs that are tagged with two or three key attributes. My assumption here is, since the likelihood of the listener being familiar with the song and artist is low while in discovery, the tags would give them an idea of what to expect from the song.

With either concept, the key would be to successfully distill the attributes in a way that differentiates the current song with the previously played songs, as well as be easily understood by the user. A better understanding of the Music Genome Project would be needed to work out how this could be done and what the level of effort would look like.

Wrapping up

This article was inspired by a daydream I had a few weeks ago, and I decided to look further into it and create concepts. I believe there is a need to explore the idea of building trust with users in the context of music discovery, as it could be beneficial for a streaming service like Pandora. With a business model that relies heavily on ads, longer user engagement with the service (and hence more served ads) would be a win-win for all parties involved.

Of course, a lot of this is conjecture based on publicly available information that I pieced together. Thanks for taking the time to read this!