Putting It All Together: The Secret To Combining Knowledge, Skills, Tools and Data Together
By Kevin Gray
Marketing research has been badly wounded and lies in critical condition. Or maybe it’s alive and kickin’. Depends who you ask. In the past decade marketing research has probably faced more sweeping change than at any period in its history. That much can be said. Change has brought with it new threats and new opportunities, and business models that have served us well over the years don’t fit neatly into the new scheme of things. Marketing research is by no means alone in this and most industries have been shaken. Mergers and acquisitions, economic uncertainly and new technology has affected nearly all of us. Globalization is no longer just talk and has brought many very different national cultures into regular contact with one another. Many of us now really do need to be anthropologists in our daily work. Digital is rattling the marketing landscape and we’ve only felt the foreshocks…
In the past three decades there has been something akin to a Cambrian explosion in analytics and many new “species” are only just now beginning to migrate into marketing research. To cite a few examples, there have been striking developments in time-series analysis, Bayesian statistics, mixture modeling and many other areas, to say nothing of the scores of newer techniques often vaguely called machine learners. The amount and variety of data and data collection methods are changing radically. Hype notwithstanding, Big Data and the Internet of Things (IoT) are here to stay. Qualitative research has not been unaffected and is no longer mainly face-to-face focus groups and in-depth interviews. Online qualitative, MROCs, mobile ethnography and text analytics software are now part of the standard qualitative arsenal.
The wealth of new data and technology enables us to do more things quicker and better than ever but they must be mastered and this takes time and money. A flipside of progress, consequently, is that overspecialization has emerged as a significant threat to our industry. New terminology and jargon seem to appear almost daily and confusion about what many new innovations really are and how they fit into MR is pervasive. To make matters worse, some “new” methods are really old methods that have been repackaged; “agile”, for instance, has been defined in creative ways. Communication can easily break down and serious misunderstandings are not uncommon. How will we keep pace with these new developments?
More than ever we will need to have both breadth and depth in our skills profiles. Each of us must have sound knowledge of marketing research fundamentals yet, at the same time, be a specialist in more than one area. All knowledge and skill do not have equal priority, however, and in my view there is a hierarchy than can be depicted as follows:
Core business skills, such as business acumen and marketing savvy, are more fundamental than others and a marketing research organization or department weak in these areas will not be effective. Secondly, because marketing research is meant to make decision-making more scientific, we need to have a solid grasp of scientific thinking ourselves. An understanding of the scientific method separates the real researchers from the amateurs, no small distinction given the amount of money spent on marketing and advertising. The basic tools of the scientific method include probability, sampling, experimentation, statistical inference and programming skills, though arguably it is more a way of thinking than a collection of skills.
Next up the ladder, there are a very large number of research methods that have been used in MR for many years. None of us will ever become proficient in all of them but a basic understanding of on-and-offline qualitative methods, questionnaire design, regression, factor and cluster analysis, to name a few, has become necessary just to be able to communicate with other researchers and clients. Fourthly, some familiarity with specialist techniques such as conjoint, structural equation modeling, predictive analytics and mobile ethnography is also increasingly required, as are an assortment of computer science skills. Lastly, there is knowledge that pertains to a proprietary technique or software product that is the property of a single company.
A few practical tips
More important than the knowledge and skills themselves is how they are utilized. The often esoteric protocols of the scientific method actually have a pedestrian benefit – they minimize the risk of making costly decisions. While it’s not productive to get bogged down in geeky minutia, some technical details can have big business consequences. Below I’d like to offer a few pointers on how to tie knowledge, skills, tools and data together by using core business skills, scientific thinking and a dash of common sense.
Frame the issue: Make sure you understand the key business issues, what decisions must be made, how they will be made and by whom. How you frame the issue drives research design. When planning research, think about the deliverables and the end users first and work backwards. Be careful about the assumptions you make and always be on the lookout for confirmation bias – searching for, interpreting or recalling information in a way that confirms your beliefs. A great deal of time can be wasted correcting mistakes made in the early stages of a research project. These often occur because we haven’t thought things through or have made off-the-cuff assumptions that turned out to be wrong. If advanced analytics will be used it’s best to design them into the research whenever possible. Establish realistic timelines – haste does make waste.
Construct a path diagram: Path diagrams similar to those used in path analysis and structural equation modeling can help you visualize plausible ways variables may be interrelated. They do not need to be elaborate and I’ve found them helpful even when I’ve sketched them out with pencil and paper.
Exploit the researcher’s toolbox: Often more than one research method or statistical technique will get the job done and frequently several are used in conjunction. Steer clear of a cookie-cutter approach to analytics. Time and budget are obviously things to think about and may rule out the approach you feel is ideally suited for a project, so you need to be flexible (without cutting corners).
That fascinating result may be wrong: Even supposedly clean data can have serious errors. Your data may turn out to be quite unrepresentative of your target population. If some pattern seems weird or, on the other hand, looks like a finding that will dazzle your client…it could mean there is a problem with your data. Many bad decisions have also resulted from data dredging. When done by professionals, multivariate analysis is less risky and more informative than looking through reams of cross tabulations.
Keep to the facts: Distinguish between facts, hypotheses, and speculation. Avoid being seduced by sexy interpretations and recommendations that are actually based speculation or pseudoscience. Let’s not forget that intellect and technology are not substitutes for integrity.
Never stop learning: Many excellent books have been written about decision-making, such as Thinking, Fast and Slow (Kahneman), Simple Heuristics That Make Us Smart (Gigerenzer et al.), Risk Assessment and Decision Analysis (Fenton and Neil) and Experimental and Quasi-Experimental Designs (Shadish et al.). I’ve listed a number of books and resources on various topics that I’ve found helpful here.
United, we stand…
Marketing research has shown itself to be resilient to wide-ranging economic, social, and technological changes over the years. One of its historical strengths has been the ability to attract unconventional, forward-looking types of people. If marketing research can continue to be adaptable, without sacrificing rigor, it will remain on its feet for years to come.
Our business requires dynamic teams of people with diverse skills and experience, perhaps more so than most occupations. There is no unicorn marketing researcher who can do it all and there never has been, and new specializations are continually emerging. More than ever, marketing researchers must be self-directed in their professional development and take advantage of MOOCs, webinars and other online and F2F learning opportunities. And, as noted, there plenty of very readable books that cover subjects of relevance to our business in detail.
Not a small part of the growing need for self-direction is driven by high employee turnover and the untidy structure of 21st century MR:
Courtesy of David McCallum, Gordon & McCallum
There is a growing need to bring in more technical specialists from computer science and statistics but integrating them into the MR business is not always straightforward. The classroom is simply not enough and, moreover, the curricula in these fields bear little connection to the social and behavioral sciences that are the bedrock of marketing and marketing research. The learning curve has also become quite steep in both disciplines because there are now so many analytic techniques and programming languages to master. Fresh graduates can take a while to come up to speed though, to be fair, our practices are not beyond criticism and we should listen to their opinions and ideas.
Over the course of our careers we need to be jacks of all trades and masters of more than one. There is no single skills profile best suited for everyone, of course, nor is there one mix of skills profiles optimal for every organization. The needs of the market also change with time but core business skills and scientific thinking never lose their value and are portable. It’s also important to remember that there still is no substitute for experience.
Though science is often mystified in the news and entertainment media, the essence of the scientific method does not require advanced mathematics or technical skills of any kind. It does require critical thinking but this can be acquired in a multitude of ways. Other kinds of knowledge and skills are important too but scientific thinking is fundamental to sound marketing research and can act as a bridge among researchers who are specialists in very dissimilar areas. Together with the core business skills, it helps us be both generalists and specialists, and this will prevent MR from becoming a collection of salespeople with only a narrow knowledge of our business and little understanding of clients needs.
Kevin Gray is a marketing scientist who has been in marketing research for more than 25 years. His background covers dozens of product and service categories and over 50 countries. Kevin began his marketing research career on the client side in New York, and he has broad experience with the A-Z of marketing research. This includes advanced analytics and new product development for Nielsen Customized (CR) and Research International. He founded his consultancy, Cannon Gray, in 2008 and works with marketing research agencies, consultants, ad agencies and clients located in many regions of the world. His chief focus is on providing marketing science and analytic support to enhance decision making. He’s a strong believer in taking advantage of new research tools and data to their fullest…but without letting the tools and data become the ends rather than the means.
Kevin would like to thank David McCallum for his many helpful comments and suggestions on a draft of this article. David is managing partner of Gordon & McCallum, a business consultancy that specializes in advising the market research sector. A professional statistician by training, he a Board director of the Australian Market & Social Research Society (AMSRS) and was formerly Global Managing Director of Consumer Research for the Nielsen Company.