ROI and the Heisenberg Principle

There are lots of problems with research. Sometimes it doesn’t give you the answers you need, sometimes it doesn’t give you the answers you want and sometimes it feels like you aren’t getting any answers at all.

Over the last few weeks, I’ve been wrestling with a more disconcerting problem. How to stop the research itself interfering with the answers we’re trying to get.

We’re all familiar with the fact that different methodologies are better suited to gathering different types of data, and can influence results to an extent. Based on what it is you’re trying to get to the bottom of, you simply tailor your methodology to be as sympathetic as possible and work with the data on the assumption that your answers are consistently inconsistent, and therefore consistently useful.

This is a bit different. In most cases, the methodology simply influences the results, so the only problem is with the data and how to manage it. For example whatever type of research you use, it won’t alter the ad that you are testing, merely your ability to evaluate it. Unfortunately, researching experiences doesn’t work like that. The research you conduct to evaluate that experience becomes a connected experience for anyone tasked with filling out the survey. Hence, a poor experience answering questions will tarnish the original experience. Not only would the original experience appear to have worked less effectively, but your poor researching of it would actually have eroded its genuine effectiveness.

A kind of ROI Heisenberg principle, if you like, where the act of measuring something fundamentally alters the thing you are trying to measure.

Like I say, I’ve been wrestling with this for a few weeks (after all, worrying about research methodologies is what Christmas breaks are for, right?), and I think I have the answer.

The answer is that there is no answer. This is an unavoidable part of human nature, that our experiences, and the way they impact us are unavoidably interconnected, and are parts of how we build up our sense of who we are and how we feel about the world around us. But that doesn’t have to be a bad thing. In fact it can be a very good thing.

Most people like to be listened to. They like to be heard and feel that they are making a difference, and they respond better to people (or brands) that listen to them. By asking them the questions in the right way, at the right time, then we can not only find out what we want to know, but we can actually improve our effectiveness and ROI while doing it.

Two of the best metrics in terms of sales correlation make use of this principle- Net Promoter Score and ‘brand I like’. They benefit from the dynamic that the respondent feels emotionally invested in the brand, and many advocacy programmes use similar principles to research to establish this feeling.

So next time I’m trying to justify the importance of research, I won’t only be talking about the value of insight and quantified evaluation, I’ll be talking about enhancing the campaign performance in its own right.

Of course, all you have to do next is to make sure that your methodology actually provides a positive experience for respondents, but that’s probably a different blog-post.