Why Pandora (sortof) Sucks

Okay, first of all, Pandora internet radio is pretty amazing, in that it’ll play music all day for free (sortof) and it’s theoretically music that you like. But there’s a few issues with it that have been bothering me for a while, and i finally took some time to figure out if i was just imagining things or if there really was a systemic problem with Pandora.

So i did a test. First of all, this is the FREE version of Pandora, with ads and everything. I have no idea if the problems that i encountered are solvable with a subscription. I suspect not, but i’m not going to pay for the service just to find out.

Here’s the parameters. I set up a Pandora radio station with 12 seed songs:

The intent was to get a range of years but not get too disparate with styles. Right off the bat i realized that i should have included more female artists in the list, but i went with what i’d selected. I did not thumb-up or thumb-down any songs during the run of this test. I noticed while listening to Pandora before this test that i would hear my thumbs-up songs more often, to the point of them becoming tired, so i didn’t want to influence the raw algorithm. For the first dozen or so sessions, i did not skip any songs, but eventually i started skipping songs that i wasn’t interested in either hearing again or hearing at all. Over the course of playing this radio station, i noticed a few things, one being that the artist seems to matter more than the actual feel/BPM/genre of the song when it comes to matching seed songs. I have no way to prove this, but that’s the impression that i got. Given an upbeat funky seed song, i would have expected to get upbeat funky songs from a full spectrum of artists and decades, but Pandora seems to match based more on pre-defined genres than actual song “feel.”

1. Song-play based on seeds

When i was compiling this test, i initially thought that Pandora plays selections based on a particular seed, from 1 to maybe 6 or 7 songs, averaging about 4. Upon examination, it turns out that it’s almost always exactly 4 songs. Here’s a listening session with the songs broken out by what seed they’re pulled from. It’s a fairly good session, pulling from 7 of the 12 seeds.

Here’s another session, pulling from 6 of the 12 seeds, and going back and forth between some. Of note is the section of 4 songs (labeled ?) that appears to have pulled from 4 different seed songs: The Clash, Kongos, Peter Gabriel, and The Clash again.

The first problem with Pandora is that it can get in little mini-ruts. If i happen to not be in the mood for a particular style, i’ll get three more songs of that style before it switches to something else. This is in contrast with setting up a random Pandora station that pulls from a selection of their preset stations. When listening to a selection of stations (like “80s”, or “Dave Matthews” or “Grateful Dead”) you’ll get 1 song from each station before moving on to the next. That makes for a better random listening experience. That’s how we listen to Pandora at work, delving into many genres to keep everyone satisfied.

I did notice that if you skip 2 or 3 songs in a row, Pandora will almost always play one of your seed songs or something by the same artist as a seed song. Before i set up this test, 2 or 3 skips would almost always play one of my thumbs-up songs.

2. Bands

Over the course of this test, Pandora played songs from 290 bands, 72 of them only once, some of them many many more times. Of note is the playing of Toad The Wet Sprocket and Coldplay much more than their seed band Dada. The Allman Brothers Band got high plays because it seeded from both Stevie Ray Vaughan and Robert Randolph. Same with The Police, seeding from Peter Gabriel and The Clash.

3. Songs

Over the course of this test, Pandora played 691 different songs (1,615 song plays, many repeated). I should note here that i included both the studio and live versions of a song as the same song, even though Pandora considers those two different songs. Here are the songs that got 3 or more plays.

Note that only 4 of the 12 seed songs made it onto this list: Guerilla Radio and I Wish, with 5 plays each, Tightrope, with 4 plays, and Sunday Bloody Sunday, with 3 plays. Here’s how many plays each seed song got. (Speaking In Tongues was not played once.)

This brings me to the biggest problem with Pandora. It will play songs similar to a seed song more often than it plays the seed song itself. This is the issue that made me want to do this test in the first place. I had the band Dada as a seed band in my Pandora station and Pandora would play a select few songs by Fastball, Toad The Wet Sprocket, Gin Blossoms, and Tom Petty seemingly every time i listened, yet played a song by Dada only every couple of times. This did not seem right.

The Takeaway

1. Choose your seed songs and seed bands carefully. For me, it seems to be a better bet to choose more obscure artists—ones that i’m not as familiar with—over bands that i know well or that have a number of hits. I was getting the most interesting songs from this test with songs stemming from Jenny Lewis and Hilltop Hoods.
2. As a corollary to #1, be aware that certain bands beget certain other bands. Having The Clash as a seed gave me more Sex Pistols and Ramones than i could ever want. Rage Against The Machine brought in way too much Red Hot Chili Peppers. Peter Gabriel brought in a host of dismal 80s crap.
3. Don’t be afraid to thumbs-down a song, but be aware that this will constrict the number of songs pulling off of a particular seed, possibly making it so that certain other songs play more often.
4. Don’t ever thumbs-up any song, or you will hear it often enough that you will get sick of it.
5. Pandora would benefit from the user being able to set the number of songs pulled from a particular seed before moving to the next. In my case, i would set it to 1.
6. Pandora would greatly benefit from a setting to not repeat anything for a set amount of time. I would probably set this to 2 or 3 weeks.