Yes, it happens! This is why – and how to avoid it.
Consumer surveys are a quick and reliable way to tap into the needs and wants of consumers: in a matter of hours or days we can get together a targeted group of people and distill their opinions, preferences and wishes into actionable information.
If the panel is of high quality, that is.
To guarantee that the consumer insight gathered is from the right target group and reflects their true preferences and behaviors, consumer market companies employ a number of quality control tactics.
First of all, the panels must be of a certain size, usually at least one thousand people, and demographically representative. But if those one thousand people aren’t giving truthful answers, quarantine or disqualification can be necessary.
How and when does this happen, and what should you know as a survey participant or market researcher?
Trick questions reveal cheaters
Step one is to pick out people who aren’t replying honestly or putting in enough effort to ponder each question carefully. Cheaters can be recognised through many tactics but one of the most common ones is implementing a series of trick questions into the survey.
Trick questions address relatively rare events. You can be asked for example whether you’ve purchased a boa snake during the last month. There are a few questions like that, and on the off chance you’ve actually bought a snake recently, it’s much less likely that you’ve also purchased an exquisite sports car and a couple of other things at the same time.
A close look at the quickest and slowest
In addition to the answers, the answering time is also clocked and looked at. Every survey has a standard answering time, for example 7 or 15 minutes.
If a panelist completes the survey much more quickly – or much more slowly – the chances are that either they haven’t really dived into the topic or that they have surfed the whole internet and done a bunch of other things at the same time. Either way, the quality of the contribution suffers and the answers are deemed low quality.
Answering from a wrong country can lead to disqualification
All survey respondents are geographically tracked to make sure that they actually are based in the country they say they are. This is important because companies usually look for insight and feedback from people in certain target markets and assuring the panelists really live in these locations improves the quality of their answers.
Of course it can sometimes happen that you fill in a questionnaire during holidays or when visiting a family member in another country. However, if all the answers come from Bangladesh while you’re registered as an American, problems are headed your way.
Disqualifications can be either temporary or permanent
What happens then, if you fail the boa snake test or take too much time to fill in your answers? Usually the first step is to exclude the answers from that survey only. So even a highly unlikely shopping history won’t get a quality panelist completely disqualified.
However, recurring quality problems will affect a participant’s quality score. This can lead either to receiving fewer surveys or a temporary quarantine. In some cases, for example when a participant is caught using several email addresses to answer the same survey multiple times, all accounts are blocked.
Too eager participants are blocked, too
Answering panels can be tempting and addictive: well drafted surveys are a pleasure to go through and telling your opinion is fun, right?
Market research companies don’t want the same people answering all the surveys, however, because the variety of answers is an important quality factor.
On the other hand, marketers shouldn’t overwhelm their panelists with too many, too long or too frequent surveys either. When market research companies only send carefully drafted and relevant surveys to consumers that are motivated to share their insights, both parties benefit.
Blog photo Photo by Priscilla Du Preez on Unsplash