Plugging The Online Survey Reliability Gap
How to make online surveys more reliable
In our last article, we discussed how online surveys suffer from a large quality deficit. In this follow up article, we talk about the things you can do to make your survey data more usable.
The online survey quality deficit
To recap, there are three main weaknesses with online surveys:
- They are not representative.
- Respondents don't always tell the truth.
- People sign up to multiple panels amplifying the problem with fake and ill considered answers.
So does this mean you have to ditch that expensive survey you've just commissioned? No but you need to understand the limitations of the data before making business decisions.
It starts at the beginning
It may seem obvious but it needs saying - only choose a reputable online survey provider. And by this I mean one that has verified that their panel members are who they say they are, have robust policies for dealing with fake, inconsistent and ill considered answers and who can tell you what proportion of responses are deleted for failing quality checks.
So let's assume that you've got the survey data. What happens now?
Triangulation is the key
Online surveys will give you a sense of where things are at and are indicative of views rather than a precise measurement of them. For many organisations, that is sufficient for their needs but if business critical decisions are to be made off the back of the data then more work on validation is required.
Triangulation is key here. Triangulation is a method used to increase the credibility of research. In this context it means comparing the survey results against other data to see whether the narrative is the same. Comparing against known facts is the most obvious thing to do. This could include comparing sales and frequency of purchase data from the research against known data from your CRM.
Publicly available data can also be used to triangulate. For example, using Google Trends to measure movements in pre and post campaign searches against survey data on awareness and subsequent engagement. Do they go in the same direction or are there differences that need further investigation? At a more basic level it could involve comparing trends in social media reach and engagement against brand metrics from the survey. Topic analysis of user reviews can be used to validate issues emerging from the survey and to double check survey data on flows in and out of a brand.
Our article here gives further information on how existing data sources can be used to obtain data on purchase funnel and wider market metrics.
The world of market research has become ever more agile and the ability to get 1,000 completed interviews by lunchtime is great for clients needing quick data upon which to act. Speed, however, must not come at the expense of quality. Poor data is poor data no matter how speedy its assembly.
Clients need to keep two things in mind when using online survey research. First, be aware of the limitations of the data as it will not be that precise. Second, verify against other sources before making business critical decisions.
