Download the
NEW e-book from GreenBook: Insights That Work - best research from the GRIT Top 50 companies | DOWNLOAD NOW

How The Emergence of Dashboard Sampling is Impacting Quarantine Periods

Q1

By Mike Misel

The growth in push-suppliers in the MR industry has been exponential, especially in the US market, and will continue to grow and become a real force in the sample supply chain.  This has been further consolidated by the emergence of dashboard solutions and advanced API technology to make the most of this movement. Push-supply will continue to provide an important sample source at a time where supply is a major consideration, and distractions for consumers – turning their heads from survey participation – are rife.

The surge in popularity of this ‘offerwall’ type survey delivery, however, presents questions surrounding the frequency of respondent participation and what this means for the industry. Quarantine periods exist to protect the respondent experience – as well as help to wheedle out professional respondents – to ensure they aren’t getting bombarded with emails to take surveys.

Upholding these quarantine periods is becoming more challenging as the chain has become more convoluted, resulting in respondents being able to take part in as many surveys as they like on a frequency of their choosing.   In the modern sampling world, we have a new respondent experience concern that didn’t exist when the idea of quarantine periods was conceptualized: device friendly studies. Nowadays most people are checking emails on their smartphone, and getting the 28 minute grid study creates an awful experience when taking it on mobile. So, dashboard sampling alleviates this by presenting opportunities when people are in a situation where they are looking to be monetized, because they are in a sense opting in, in real time.

There is concern in the industry about what this means for survey quality; does it open the doors for professional survey takers where gain of incentives is the main objective (and incentives are of course important to keep consumers engaged). Generally, a researcher may not be aware of the data gathering process and therefore not know about the potential for their insights to be based on opinions of people who take surveys very regularly.

How can this be addressed in a modern-day marketplace?

Some may argue that quarantine periods should be removed altogether, as there is no real way of policing them with the emergence of dashboard sampling, and with a demand for faster insights gathering, some researchers may not be overly concerned about duplication, if a quick return is needed based on a high volume sample.

Of course, those concerned about quality can take the blending route. This is a great solution to employ in any case where researchers are after a more accurate result with less bias or skew, which can sometimes occur from using one panel even when multiple survey-taking isn’t an issue. Respondents in any one panel will share common ground and so to gain a more widely representative opinion, pulling together and blending different sources, can provide a great result.

It’s difficult to state a right or wrong in the case of push supply and potential over use of respondents, as it will be of concern to some in the industry more than others – dependent on the type of insights needed or the project being worked on. But in a world where supply isn’t infinite, additional sources from push-suppliers certainly cannot be ruled out and will only grow further.

In addition, the growing reliance on what can be termed as ‘non-traditional’ sample sources also comes at a time when there is heightened scrutiny on the quality of online samples, highlighted recently by various papers presented at events hosted by the likes of the MRS and Esomar.

Reg Baker (and others) have called on the industry to revisit and review sampling practices, and raised awareness about the importance of insuring that the fundamentals and ‘science’ behind sampling, and setting sample-frames, remains high priority and central to all suppliers and their clients.   So in order to help, here are five tips to consider around using non-traditional sample sources:

  1. Know the source. Expect transparency over the sources you using in terms of how the respondents are recruited and where from
  2. Understand the incentive model
  3. Understand the respondent flow from the supply source origin to the survey and whether there is routing or other techniques applied prior to a respondent reaching your survey
  4. Enquire as to the profiling and targeting capabilities of the supply sources you are using. For example, can the same respondent be contacted in the future should you need to?
  5. Appreciate the potential biases which exist with any source or methodology of sample, and account for those in your sample-frame designs.

Please share...

5 responses to “How The Emergence of Dashboard Sampling is Impacting Quarantine Periods

  1. In my opinion a simple split of the sample across two suppliers is always a good check on the reliability of the sample. Of course this never tells you who is right or wrong, but it sure rings bells when the differences are statistically hugely significant.

  2. In a way, the use of nonpanel sources (or incentivized ad network or real-time or river, however we call them) exposes researchers to a truism of recruitment from which they were previously insulated, namely that sources differ. Even the idea that “panel” is uniform must be challenged. Panel recruitment sources can vary considerably by demographic and over time. As the saying goes, everyone likes sausage, but they don’t necessarily want to know how it gets made.

    I made the argument in this august blog recently that reach and source diversity will distinguish winners from losers in the panel/sample game. Part and parcel of this is that researchers need to understand that they can’t think of sample sources in any sort of uniform fashion or use dated conventional wisdom that one source is intrinsically superior to another. Concretely, this means:

    1. Having the ability to uniquely identify any respondent that touches the system
    2. Keeping a data trail that includes from whence the person came
    3. Tracking participation and response quality by source, which itself implies…
    4. Creating new benchmarks or calibration models that look not only at demographics, but also at attitudes and behavior to compare them against known (or methodologically superior) datapoints to identify outliers and mitigate bias.

    Researchers, particularly those running trackers and normed studies, are actually contributing to the problem by not accepting this reality.

    The term that comes to mind is caveat emptor. In my view this will be our reality for at least the next 3-5 years.

  3. Long before this technology started to be used, the idea of quarantine periods was effectively a myth in highly developed markets. Panels or panel marketplaces can impose quarantine but when the average panelist is on 5-7 panels (estimate from a few sources) the quarantine is rendered useless. If there was one central unifying ID tag given to every respondent across every panel/aggregator/router than quarantine would be possible but our industry would grind to a halt. Our current panel/routing model is broken because it relies on such a narrow spectrum of people to do on average 30-40 surveys a week (recently measured by GMI Lightspeed).

    In my 11 years in online data collection, quarantine periods have gone from a best practice with quality in mind (Once a week on the first panel I worked for – until we became busy) to “only” 2-3 times a day, which isn’t a quarantine period, but rather the absolute limit of how many survey invites you can send without triggering an opt out.

    Knowing a source/incentive model of your sample supplier is seeing a small part of the respondent’s behavior across multiple platforms.

Join the conversation

Mike Misel

Mike Misel

Senior Vice President - Americas, Cint

Sign up to our newsletter

Don't miss out...
Get great insights content delivered straight to your inbox.
I agree to receive emails with insights-related content from GreenBook.


You can manage your email preferences or unsubscribe at any time. GreenBook protects your privacy under the General Data Protection Regulation.