1. RN-GBook-480-60-B2B-Banner-5-16
  2. Greenbook 2
  3. Greenbook-Mobile-6.29.16-
  4. mfour_new_1

Get Into Their Shoes Before Getting Into Their Minds

Before you put any questionnaire in the field, take the survey yourself as a respondent. You’ll be surprised what you find.

 

By Ron Sellers

It was such a good idea…in theory.

How many of those moments have you had in your life?  Something seems so smart, so right, so reasonable – then you put it into practice and it flops.

We do this in research and marketing all the time.  I was reminded of this recently while sitting at a Phoenix stoplight watching a sign dancer.  I don’t know what they’re actually called, but local businesses hire people to stand on the corner of a busy intersection holding a sign that says something like, “Big Sale at Courtesy Buick.”  In order to attract attention, the more enterprising among them dance and do acrobatics with the sign.

The one I saw was quite good – that sign was in constant motion, with a series of creative flips and spins.  Really caught my attention.

Problem is, I had no idea what the sign said.  Even though I was really concentrating and staring at the sign, it was in such constant motion that it was impossible to read.  Such a good idea…in theory.

I’m amazed at how much outdoor advertising is like this.  I see billboards all the time with print so small there is zero chance I can read it while zipping by at 65.  Local churches that meet at school gyms or other rented space put up street-side sandwich boards on Sunday morning announcing their service times – they probably looked great to the sign painter standing a foot away from them, but have print so small that no motorist will ever be able to read it from a car.  On a street.  Doing 45 miles per hour.

All of these things sound great in theory, but when placed in a real-world situation, they fail.

There’s a pretty good chance a lot of your questionnaires are exactly like this.  Pre-testing is supposed to correct some of these problems, but even pre-testing won’t catch everything (and with today’s budgets and timelines, how many questionnaires actually get pre-tested anymore?).

There’s one defense against this:  you.  Never, ever, ever put a questionnaire into the field unless you have answered it yourself.  As a respondent.

You design the questionnaire as a researcher.  You pay careful attention to which scales will allow you to run the statistics you want.  You make sure the skip patterns are all right.  You look for priming issues (questions that could be influenced by previous questions).  You make sure the client’s information objectives are addressed.

Respondents care about none of those things.  When completing a survey, most of them simply want to give you accurate answers to questions they can reasonably answer, and be done with a minimum of effort.  Unfortunately, we often try to ask them things they can’t reasonably answer.  We fail to include every possible response option.  We ask for information they don’t remember.  We write confusing questions.  We ask them questions that aren’t truly applicable to their situation.

And those are the people who are really good at questionnaire design.  We all make these mistakes, because we’re researchers, not respondents.

So before your questionnaire goes into the field, become a respondent.  Answer it honestly the way you would if you were taking the survey.  If it’s not applicable to you, make up a persona and play that role – become Gerald, a 23-year-old, single, Black IT professional, or Ruby, a 64-year-old Latino grandmother who attends Mass every week.  Become whomever it takes to qualify and complete the survey.  Then do just that.  You’ll be amazed how many questions suddenly don’t make as much sense as you thought they did when you were the designer.

If that long grid bores you, it will bore Gerald and Ruby.  If you’re not sure what a question really means, Gerald and Ruby will also be unsure.  If there’s no response option that applies to you, Gerald and Ruby will have the same problem.  If a question is not applicable to you, it will not be applicable to Gerald and Ruby.

Every single questionnaire I put into the field, I complete as a respondent.  Usually multiple times (playing different roles).  Invariably, I find ways to make it more clear, more concise, more relevant, and easier to answer accurately.  I do this again after the questionnaire has been programmed (assuming it’s phone or online), because how the respondent hears it or sees it on screen impacts how it gets answered.  There are times the question itself is fine, but the programming creates problems.  If it’s a phone survey, have someone read it to you as the interviewer, because how you hear things is not the same as how you read them.

We do market research in order to get into the minds of our respondents.  But before we get into their minds, we have to get into their shoes, and see the research from their perspective.  Only then will we have questionnaires that are relevant for our needs as researchers, and relevant for our respondents to answer accurately and consistently.

Share
You can leave a response, or trackback from your own site.

5 Responses to “Get Into Their Shoes Before Getting Into Their Minds”

  1. Angry_MR_Client - the only #mrx person you need to follow on Twitter - Ansr.io says:

    September 21st, 2012 at 11:55 am

    […] a sidenote, Ron Sellers’ “Get Into Their Shoes Before Getting Into Their Minds” is a great post on that […]

  2. Ruben Alcaraz says:

    September 22nd, 2012 at 12:38 pm

    Ron

    Great article!

    I am also a believer of going through what respondents will go through as much as possible. Also, your reco is very valid as sometimes the devil is in the details and simple things as wording, sequence. length or skip patterns can yield really bias results.

  3. Lucy Holmes says:

    September 26th, 2012 at 4:26 am

    I work for a charity, and every year we conduct a telephone survey with our service users. I find a good way of testing it (every year before go-live) is to ask our helpline call-takers to do it as respondents. Not only does it mean several eyes have seen it, but the call-takers know our respondents really well, so they can play the part more realistically than I. It only takes them 10 minutes each, but it’s so reassuring for me that the survey flows well, makes sense, allows appropriate answers, does not offend, and is the right length.

  4. Get Into Their Shoes Before Getting Into Their Minds « National-Express2011 says:

    September 30th, 2012 at 8:21 pm

    […] on http://www.greenbookblog.org Sharen mit:TwitterFacebookTumblrPinterestEmailMoreLinkedInLike this:LikeBe the first to like […]

  5. Ron Sellers says:

    October 5th, 2012 at 12:45 pm

    Lucy, it’s always great to hear from someone in the non-profit sector who is doing research. I’ve specialized in research for non-profits during my career, so it’s always a pleasure to hear from someone else in the sector who values consumer insights. Thanks for your comment!

Leave a Reply

*

%d bloggers like this: