Insights Industry News

May 21, 2012

Timing Is Everything: Specific Time Frames in Question Design

Say you want to gauge how often people wash their cars. Or drink orange juice… There are two primary ways you can ask this.

Ron Sellers

by Ron Sellers

0

 

By Ron Sellers

Say you want to gauge how often people wash their cars.  Or drink orange juice, or go to a movie theater, or whatever.  There are two primary ways you can ask this if you’re trying to measure times per month (for instance):

  • “In a typical month, how many times (if any) do you wash your car or have it washed?”
  • “In the last 30 days, how many times (if any) did you wash your car or have it washed?”

Same difference, right?  Pretty interchangeable?  Not so fast.  Are these two questions really asking the same thing?

For one thing, researchers too often are not realistic when asking people to frame their activities within specific periods of time.  If the survey is being completed on August 18, will the respondent really remember whether she had her car washed on July 17 or July 19?  Will she really remember exactly how many times that car has been washed in exactly the last 30 days?

The problem grows as the time period grows.  I can tell you exactly how many glasses of milk I drank in the last 24 hours.  Maybe in the last 72 hours.  In the last week?  I can give you an estimate.  In the last month?  Now I’m down to pure guesswork.

I frequently answer questionnaires demanding that I provide information I have no hope of remembering.  How many times did I rent a car over the last 12 months?  What did I pay when I stayed at the Hilton three months ago (and I’ve stayed at eight other hotels since)?

But aside from that issue, there are some activities that tend to vary from one time period to the next.  People may watch more TV news as elections near, or during football season.  Church attendance is particularly heavy near Christmas and Easter.  Residents of Phoenix may bicycle frequently during our nine months of nice weather, but few are insane enough to break out the ten-speed when it’s 116° in August.  I’m guessing car washing patterns in Minneapolis are not the same in July and January.

Not only are some activities seasonal, but events may mean a respondent who normally does an activity frequently was not able to do that activity recently.  I was reminded of this last year when I completed a questionnaire about my media use.  I normally read the local newspaper every morning, but I had just spent three weeks in Portugal, and my Portuguese isn’t good enough to read the local paper.  So when asked how many times I read a local newspaper in the past 30 days, I had to answer that it was nearly zero – not at all an accurate reflection of my normal media habits.

These disruptions of normal activities may be more common than we realize:  people have to deal with new births, divorces, family members getting sick or dying, travel, health problems, job losses, moving, family visiting from out of town, severe weather, and any number of other issues that may disrupt their typical activities for a period of time.  I watch a lot less TV, and spend a lot more time reading, when I’m on the road than when I’m at home, and I’m on the road a lot.

Asking about a typical 30 days (or week, or year) means the respondent is telling you what his life is usually like, regardless of whether it was like that in the last 30 days.  But it’s also probably more of an estimate.  If I wash my car once a week in the summer, and once a month in the winter, there is no “typical” for me, so now I’m being forced to do math and average it out (which may or may not be accurate).

Asking about the last 30 days may bring a more realistic number, but that number may vary by when the questionnaire was completed, and you may be getting a very unrealistic number because of this.  On socially desirable activities, someone may even falsify the number, under the reasoning that “I usually give to charity every month, even if I didn’t last month, so I should say I gave last month ‘cause that’s what I usually do.”

Neither of these approaches is necessarily the “right” answer.  But as with everything else in research, it’s really critical to understand what you are getting by asking your question in a specific way.  Every word of every question requires thought, and there are no “standard” questions that can just be used over and over in every situation.  Understanding that, and the impact it may have on the resulting data, is simply part of doing the research properly.

0

quantitative research

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Ron Sellers

Are the Fraudsters More Sophisticated Than the Researchers?

Research Methodologies

Are the Fraudsters More Sophisticated Than the Researchers?

It’s amazing what some people will do in order to make a buck-fifty. Two recent studies have brought to light how sophisticated panel fraud has become...

Ron Sellers

Ron Sellers

Still More Dirty Little Secrets of Online Panels

Research Methodologies

Still More Dirty Little Secrets of Online Panels

Nearly half of your panel data is trash. Here is how to fix it.

Ron Sellers

Ron Sellers

Can Political Polls Really Be Trusted?

Can Political Polls Really Be Trusted?

When political polls fail to predict the exact outcome of an election, maybe they’re not wrong…maybe we are.

Ron Sellers

Ron Sellers

Panel Quality Stinks and Clients Are To Blame

Research Methodologies

Panel Quality Stinks and Clients Are To Blame

Why should panel companies improve their results when clients accept the status quo and won’t pay for better?

Ron Sellers

Ron Sellers

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*