1. Clear Seas Research
  2. SIS International Research
  3. RN_ESOMAR_WP_GBook_480_60
  4. TopBanner480x60_Greenbook

What Do Marketing Scientists Really Do?

In this short article I’d like to draw back the curtain for a moment and let you peer inside the sometimes mystifying world of the Marketing Scientist.

marketer-scientist

By Kevin Gray

One of the things I like most about Marketing Research is its diversity.  On the job, all of us wear many hats.  What’s more, Marketing Research is no longer a cottage industry and has developed into a large global business, and we come from many different ethnic, religious and national backgrounds.  In terms of education we also hail from all over the map, including literature, linguistics, philosophy and the social sciences.  Our educational backgrounds often have given us just a fleeting glimpse of statistics and research methodology and to many of us “analytics” is arcane.  It can mean different things to different Marketing Researchers and the role of a Marketing Scientist also varies from company to company.

In this short article I’d like to draw back the curtain for a moment and let you peer inside the sometimes mystifying world of the Marketing Scientist.  Let’s imagine a hard case scenario.  Imagine yourself in the position of a Marketing Scientist who is given data and asked to “find something interesting for my client.”  While this sort of request might strike even someone new to our business as unprofessional and unreasonable, on more occasions that I care to remember this has happened to me.

What to do under these circumstances?  First, some fairly obvious questions come to mind apart from confirming budget and timing:

  1. What do the data mean?  You’ve been given a long questionnaire and data map but it could take ages to go through and make sense of them.  Not clearly understanding what the data mean can lead to a lot of rework later on or, even worse, misleading or incorrect results and recommendations.
  2. How clean are the data?  This is something you’ll have to ascertain yourself and you should never assume data are error-free.
  3. Who will be using the results, and how and when will they be used?
  4. What are the client’s expectations?

None of these questions can be answered automatically by software and each requires human expertise.  The last two questions are most critical and it is at this point that many wrong turns are taken because incorrect assumptions are made.  This can lead to a lot of agony (and cost!) downstream.  “Getting to know” the data and data cleaning can be accomplished in tandem, through an iterative process.  Recoding data is usually done at this stage.  This can be laborious and time-consuming but is usually essential because many questions or data fields have a large number of categories that must be re-grouped or combined so they are easier to interpret or because of small base sizes.  Rating scales may also need to be reversed so that higher numbers mean more positive scores than lower numbers.  There is occasional glory in analytics but also a lot of grunt work!

SPSS, Excel and CSV are lingua franca for data files but the software you will be using may require some other format.  Not infrequently, statisticians need only a small part of the original data file and must create one or more data files for analytic purposes.  Sometimes DP can do this for you but normally there are decisions the person analyzing the data is in the best position to make.

Once beyond these first critical steps, you’ll now need to think concretely about how to analyze the data.  In some instances this is clear from your understanding of the client’s needs and the data themselves but often it is not, as in our hard case illustration.  Perhaps surprisingly, in some respects analytics is becoming more difficult precisely because we have more options than ever before.  (See Analytics Revolution and Why Survey? for brief overviews of analytics.)  Frequently, several techniques are combined in the analysis.  Moreover, many statistical terms such as “regression” and “Bayesian” are quite generic and refer to broad families of methods, and your research exec or the client may have suggested a technique without really understanding what it is.  Many distinctions that seem like geeky minutiae to those with limited statistical background actually are very consequential but tricky to communicate.  This takes practice and is an important skill a Marketing Science person must acquire.

Generally speaking, you should keep the analysis as simple as possible and be solutions-driven rather than technique-driven.  When considering various approaches ask yourself “How will this choice affect my client’s decisions?  Will they be able to communicate the results to their internal clients?”  Don’t use a method just because you are comfortable with it and don’t try to show off your mathematical virtuosity.  That said, running huge numbers of cross tabs and letting significance tests (more or less) do your thinking for you is commonplace but dicey practice.

This is a lengthy topic but, in short, significance testing (and computer algorithms generally) can only suggest rough cutoffs for deciding what is “important” and what isn’t from a business standpoint.  Furthermore, significance testing is only concerned with sampling error and as a rule assumes simple random sampling, which is seldom in fact used in survey research.  Significance tests also are not independent and Type I error (“false positives”) rapidly accumulates when many tests are conducted on the same data.  In Data Mining and Predictive Analytics sampling is often less problematic but the data may contain millions of records and miniscule differences flagged as highly statistically significant.  What’s more, if the data represent an entire population – records for all customers for example – inferential statistics are meaningless.  On the other hand, abandoning significance testing altogether is unwise; sometimes it is helpful in cutting through the clutter and how to use it is a case-by-case decision.

Eminent statistician George Box had many wise words over the course of his long and distinguished career and “all models are wrong, but some are useful” is one his most quoted pieces of advice.  Analytics requires many choices and we usually will never know what mechanism or mechanisms gave rise to the data we are analyzing.  Often several models will provide equivalent “fit” to the data but suggest different courses of action, and the choice among them may dramatically affect the client’s decisions.  While model comparison indices such as the BIC and AIC or other heuristics can help narrow down the range of plausible models, as with significance testing, they cannot provide THE answer.  We need to roll up our sleeves and think.

“Correlation is not Causation” is currently a buzz-phase in the business media.  (Ironic, given that conspiracy theories flourish in many news outlets!)  An association may support or suggest a hypothesis but it does not prove a causal relationship.  Why does this matter?  When prediction rather than explanation is really what is necessary we sometimes can lighten up a bit and rely on semi-automated methods popular in Data Mining and Predictive Analytics.  However, while these techniques often excel at prediction, they frequently yield results that are hard to interpret.  Being able to spot potentially high-spending customers, for example, by itself may be insufficient.  Lacking insights into why they and similar customers behave the way they do will make it more difficult to design marketing programs that will work in practice.  Also, many decision makers are understandably distrustful of “black box” solutions.

There are many decisions to make in analytics and it is only possible to mention a handful of the most typical ones here.  To the extent possible a Marketing Scientist should be involved early in the design of the research.  That will reduce the headaches described at the beginning of the article!  In some situations, though, Marketing Scientists can become involved too early in the process and the discussion veers off towards methodological details before the key business concerns have been sorted out.  This also is to be avoided.

I should stress that I’ve only given a glimpse of Marketing Science, which is much broader and more varied than the foregoing might suggest.  One small request before we draw the curtain closed and get back to work: please do not just give your Marketing Scientists some data and ask them to find something interesting for your client!  You are all part of a team so please interact with them proactively and provide them as much background and feedback as you can.  Try to understand what your client really needs – which is not always what they request – and work backwards into the methodology.  This is a better way to do research and a better way to do business.

Share
You can leave a response, or trackback from your own site.

4 Responses to “What Do Marketing Scientists Really Do?”

  1. Beau Martin says:

    February 12th, 2014 at 2:29 pm

    @Kevin: Always interesting to see how others work.

    I’d suggest though the purpose of a marketing scientist is to set up and analyze experiments.

    The above looks more like a data miner, someone simply sifting through data with the idea there must be value there. This is in fact the folley of big data (i.e. that just because there is data it must be interesting.)

    Iin my experience the only time those that are running research ever show up with data and ask me to help them “find something interesting” is when the project is in deep trouble – that is to keep the project from outright failure.

    Bailing out bad research is should be only a small fraction of the work any marketing scientist does on a day to day basis. If you’re spending more than a small fraction of your time bailing out bad research it might be time to spend time educating and engaging with those that are setting up the research on a day to day basis.

    In general, I love your posts, but apart from the picture above, I think this article sells the marketing scientist short.

  2. Kevin Gray says:

    February 14th, 2014 at 5:05 pm

    Hi Beau, “I should stress that I’ve only given a glimpse of Marketing Science, which is much broader and more varied than the foregoing might suggest” might not have been emphatic enough! Indeed, there is a great deal more to Marketing Science than sketched out in the article, such as developing new methodologies. However, I decided to keep concentrate on basic elements of the role because I am concerned that Marketing Researchers nowadays very often lack a sound understanding of the basics, including much of the grunt work and simple decisions that are part of data analysis. Part of this is I feel is due to newcomers having to learn too much too quickly. There is also the misunderstanding that the role can now largely be automated through software thanks to media hype and sales claims. We’re a long way from “AI” yet!

  3. Beau Martin says:

    February 14th, 2014 at 11:15 pm

    While sales hype is certainly well beyond the capability of many tools, I think the same is true for many organizations as well. It sounds like you may deal with a number of them. The best run research firms have DP organizations to manage the data rather than having the most expensive person on their staff do the work.

    We might just work in really different environments, but the real advantage of a marketing scientist as I see it isn’t in data prep or even new methodologies. The real advantage of marketing sciences is in the frameworks for setting up and solving problems the best marketing scientists bring to performing research.

    The unfortunate truth however is that far too many marketing scientists are ex and out of demand social sciences professors that were unable to earn tenure in their fields. While they may be expert in the fine points of their social sciences fields all too often they have no business consulting to business and lack constructive frameworks to do so.

    Most of those organizations I believe you are concerned about are filled with people with degrees in fields like comparative literature, religion, English and sociology, not because they are good at research, or even trained to be so, but because they are cheap. What they often do well however is write presentations, and often, entertain clients.

    These are the types of firms, and frankly the types of clients, that thrive in qualitative backrooms because they are catered.

    Marketing sciences is way more than, in fact is not, backroom catering for the quantitatively inept, which is what I get the sense you are trying to make it.

    While many and maybe even most of those that work in marketing sciences today probably are not at the top of their field the one thing they should all know how to do is set up and solve good research problems.

    This setting of good research problems, especially experiments, always has to occur before the research is fielded, not after. It is not possible to conduct “science” or anything that approaches the “scientific method” after the fact.

    Just like there is a difference between discrete choice and conjoint, methods that both rely upon experimental designs but which are in fact quite different in their aims as well as their sophistication, a difference exists between a marketing scientist and a simple research statistician.

    If one is not setting up and resolving research questions through experiments, that reduces the role one plays from that of a Marketing Scientist to simply a research statistician. Here and above what I believe we’re seeing is a description of a research statistician.

    Let’s keep the message clear and the hype down: research statisticans build models and run significance tests after the fact. Marketing scienctists set up and run experiments and models in the research through data collection and supply conceptual frameworks to do so.

  4. Kevin Gray says:

    February 14th, 2014 at 11:47 pm

    Certainly marketing scientists should generally be involved early in the process of designing the research, as I pointed out. It is also true that some “marketing science” is carried out in the back room by DP – essentially factory set ups – with predictable results. Many methods, such as “conjoint” or “SEM” are sold as if they were brands, again with predictable results. Not a small amount of marketing research is sold and conducted by still inexperienced people who, however bright or well-intentioned, are in over their heads or by “veterans” who in reality are neither marketers nor researchers. Small wonder if there is growth in DIY or if our industry is under assault by software vendors!

Leave a Reply

*

%d bloggers like this: