I thought I knew music. I thought I knew the 90s. Then the data came in.

Recently, I took part in a listener-powered countdown of the top songs of the 1990s on KEXP (an independent radio station, based in Seattle, WA). The premise was simple: listeners submitted their 10 favorite songs from the decade, the votes were combined, and KEXP counted down the results, from 500 to 1.

Naturally, before the countdown started, I had strong opinions; not just about my own selections, but what the overall top picks would be.

I had a pretty good idea which artists had to be near the top. I knew the songs that defined the decade. And, as someone who listened to a lot of music in the ’90s, I felt reasonably qualified to predict what other listeners would choose.

Then the countdown happened.

There were songs much higher than I expected. Songs I assumed would be near the top that weren’t. Artists whose popularity surprised me. And perhaps most humbling of all: songs that apparently meant a great deal to a lot of people that I barely knew at all.

My immediate reaction was some version of: Really? That song? But then the researcher part of me kicked in. Because, of course, the surprising thing wasn’t really the countdown. The surprising thing was how confident I’d been that my personal view of the ’90s would look anything like everyone else’s.

We All Live Inside a Biased Sample

It is easy to overlook, but my mental picture of the best music of the 1990s wasn’t formed by objectively observing the decade. It was built from the music I listened to, the radio stations I heard, the friends I had, the concerts I went to, where I lived, how old I was and countless other experiences. In short, whilst it was a valid version, it was my version, through my eyes, my filters, my feelings.

Over time, those experiences stopped feeling like one person’s particular version of the ’90s and started simply feeling like the ’90s.

That’s a useful reminder for anyone working in research, healthcare or business: our experience of the world is not the same thing as the world.

We naturally give more weight to things that are familiar to us. We assume other people know what we know. We notice evidence that fits with our expectations. And when something is easy to remember, we can overestimate how common or important it actually is.

Of course, none of that means we’re bad decision makers; it simply means we’re human.

Which leads us to another hurdle. While you would expect expertise to make this problem better, in reality, it can actually make it worse. The more experience we have in an industry, category or subject, the easier it can be to believe we understand how other people think about it.

If you work in healthcare every day, terms such as deductible, prior authorization or formulary may feel completely ordinary. If you work on a product for months or years, its value proposition may seem obvious. If you’ve spent dozens of hours studying a customer journey, the “logical” next step may be crystal clear.

But the customer hasn’t been in those meetings. The patient hasn’t read the strategy deck. The consumer hasn’t spent months thinking about the category. And just as importantly, they don’t share our personal history, context or assumptions.

In other words, sometimes the business equivalent of “Everybody knows that song!” is:

“Everybody understands what this means.”

“Obviously that’s the feature people care about.”

“No one is going to choose that option.”

“We already know what customers think about this.”

Those are exactly the moments when it can be worth stepping back to ask: do we actually know that, or does it just feel true to us?

This is why research matters. Good research doesn’t eliminate human bias. Researchers have biases too. But it does give us a way to test our assumptions against the perspectives and experiences of the people we’re trying to understand.

Sometimes the results validate our initial instincts. Sometimes they challenge them. And sometimes the song we hadn’t even considered ends up at the top end of the countdown.

Those moments shouldn’t be viewed as research failing to confirm what we already knew. In many ways, they’re when research becomes most valuable.

If every study simply confirms everything you or your client already believes, what are you actually learning?

The interesting findings are often the uncomfortable ones:

The feature customers care about isn’t the one we’re emphasizing.

The message we think is clear isn’t clear at all.

The problem we assumed was important isn’t particularly important to customers.

The option nobody internally expected people to choose turns out to be surprisingly appealing.

These aren’t inconvenient findings. They’re opportunities to update our understanding of the world. The lesson isn’t “don’t trust your instincts”, but experience matters.

A healthcare professional who has spent years working with patients knows things that won’t appear neatly in a dataset. A product leader develops instincts through thousands of decisions. And researchers themselves bring valuable, accumulated knowledge to every new study.

The mistake isn’t having a hypothesis, it’s forgetting that it’s exactly that. There’s a big difference between: “Based on what I’ve seen, I think X will happen”, and “Everybody knows X will happen”. One invites evidence. The other quietly closes the door on it.

Perhaps the bigger lesson I took from the countdown is to be curious when you’re surprised or presented with data that doesn’t match your expectations, instead of looking to our instincts to help explain the result away.

“Who voted in this thing?”

“That can’t be representative!”

“People must have misunderstood the question!”

Sometimes of course, these are legitimate methodological questions. We as researchers absolutely should interrogate any surprising results.

But there’s another question worth asking first: What if the result isn’t wrong? What if my assumption was? This is a far more interesting place to start.

The song you can’t believe made the list tells you something about the people who voted for it. The healthcare message that doesn’t resonate tells you something about the people reading it. The product feature people ignore tells you something about what they actually value. The finding that surprises the room may be telling you something the room needs to hear.

As researchers, we’re often expected to know what the data will say, but our job isn’t to be right before the research. While experience certainly helps us form better hypotheses, I’ve increasingly come to think there’s another skill that matters just as much: being genuinely interested when we’re wrong.

Ultimately, isn’t really about proving that our initial predictions and understanding is correct. It’s about improving that understanding and using it to further our research and insights.

Sometimes that requires highly sophisticated methodology, carefully designed experiments and complex analysis. While sometimes, it simply requires thousands of people voting for their favorite songs and reminding you that your version of the 1990s was only ever exactly that: your version of the 1990s.

And yes, I’m still surprised by some of their choices! But perhaps that’s the point.

If you also want to see how accurate your predictions are, you can see the full list of KEXP listeners’ favorite songs of the 90s by clicking here.