Whose research do you trust?
We are constantly bombarded with the results of surveys and polls – especially in the run-up to any election. (He’s a Medicare fraud! She lost millions in our state pension funds! Like most Floridians – an anecdotal observation I have no valid data to support – I am SO glad the ads are over!)
Some of this research is interesting, even fascinating to follow. Still, like any snapshot, research will always miss some information that’s potentially relevant to the issue or interests at hand. (It’s not that they’re deliberately ignoring the metaphorical man behind the curtain, they’re just looking the other way.) In addition, the best constructed research instrument is only as good as its implementation. I cringe every time I open a newspaper or website or hear a TV newscast that purports to report the results of a reader/viewer survey as if it were fact. Even an undergraduate student in a freshman class on research methods could inform the responsible (Bozo) editors that when you are using a self selected, arbitrary respondent population, the results are not representative of the measured views, attitudes, opinions or actions of any other larger population – so who cares?
Useless and misleading research is worse than no research at all because it encourages people to act on misinformation.
Whose research do you trust? And why?
Valid research can be especially valuable to public relations and marcomm professionals. It can enable us to benchmark and then track ROI, the results of strategic programs that are directed to changing the behaviors we seek – behaviors that may be driven by quantitative factors such as click through rates and frequency of website visits. Qualitative research can help us understand other factors that are hard or impossible to quantify.
That said, a step back: It’s important to define what you need to measure, before you worry about proper research design. As K.D. Paine & Partners points out in its excellent measurement blog [http://kdpaine.blogs.com], “The first law of metrics is to only measure that which you can change.”
“If you measure “likes” on Facebook or “followers” on Twittter you will no doubt spend most of your time, collecting “likes” on Facebook, and get some great numbers. However, if there is no evidence that Twitter or Facebook influences sales, market share or anything else your boss is counting, then you’re wasting a heck of a lot of time.”
Bottom line, measure the right things the right way: Or suffer the consequences.