Machine Learning Bolsters Market Research
By Frank Smadja, PhD.
Technology continues to empower marketers to improve the efficacy and personalization of messages. Data informs targeting capabilities and insight supplements understanding. This insight has become more critical as marketing teams need to understand the ‘why’ behind the ‘what,’ making survey research ever more important. To address this need, market researchers are often looking at ever more targeted individuals to participate in survey research.
Take for example, when an insights professional is planning a survey of 1,000 people who drive a specific make and model of motorbike and live in the London area. A traditional survey approach would be to send the survey to the 200 people that have been pre-identified on a research panel and ask many more about their ownership of a motorbike in order to try to get the 1,000 motorbike owners. While this is a longstanding practice, it runs contrary to achieving speed to insight; not to mention does not improve upon the survey experience.
The good news for market researchers that face this problem is help is on the way.
Machine learning technology is emerging that will help online survey tools to predict the answer to the type of questions like “do you own a motorbike.” The technology teaches computers to learn from experience – learning from data without relying on a predetermined equation as a model. Machine learning algorithms adaptively improve performance as available sample numbers for learning increase.
Machine learning has an expanding presence in marketing, such as in online advertising as a tool for making digital campaigns more targeted and personalized. Another way marketers leverage machine learning is in customer experience applications to identify patterns among customer interactions to increase revenue opportunities. Market research is a natural extension of this technology trend.
With machine learning, survey platforms can learn and predict properties of users based on their answers on other questions and demographic and profile data similarities to additional panelists. The technology enables insights professionals to rely less on asking questions in qualifying respondents, which means less wasted time and resources. It aligns with respondent expectations that researchers already have knowledge about them and don’t need to ask basic information questions which can turn off respondents.
Back to our motorbike example, machine learning will enable surveys to primarily target people who are likely to have a high chance of owning a motorbike even though they never specifically answered the question. Based on this learned intelligence, a survey will simply ask respondents to verify this fact as they enter the survey, reducing the number of people rejected and boosting panel satisfaction because the survey is attuned to their profile.
Machine learning addresses a challenge that is impossible to handle by traditional heuristic-based approaches; these methods would not perform well and would be impossible to scale. Survey panels can run into millions of users, with each panelist having thousands of data points – potentially billions of data points in total. Machine learning techniques address this scale issue to learn about panel users based on their activities and then predict their answers to questions.
In examining ways machine learning can advance market research, Toluna has used an open source library developed from Google to investigate and compare a number of learning algorithms. Our research shows that this technology can help improve targeting significantly. Survey design will continue to first look for people who match the target and in a second step, look for people whose predicted answers match the target. We expect this new process will reduce panel fatigue significantly. It will also help drive more completes per survey panel and require less capital expenditure for each survey.
Machine learning technology shows great promise in market research. It can learn from huge amounts of data to generate insights and predict answers without asking irrelevant questions to panelists – improving respondent experience and survey results.