Protecting the Science of Implicit Research Technology
Consistent with this theme, I found the conference this year to be increasingly focused on new methods burgeoning from the behavioral sciences compared to previous years. In terms of favorites, I found Jeremy Sack’s keynote from @LRWonline, which indirectly promoted implicit research methods through the theme of “brands are stereotypes,” to be very well delivered and (hopefully) impactful on the room full of insights professionals. The keynote and the presence of many behavioral science based vendors on the trade floor served as a sign that interest in these methods is on the rise.
And yet, as we continue to see the interest and application of implicit research technology rise in the industry, the innovators and developers of implicit research methods will need to play an increasingly important role in “protecting the science” behind these methods in order to realize more broad-scale adoption of these techniques. In fact, I will argue below that protecting the science is actually a prerequisite for “translating insight into confident business decisions”.
To illustrate, imagine we’re back the early days of survey sampling. According to @visioncritical, modern day market research survey sampling methodologies were established by Gallup during the presidential election of 1936, when his firm accurately predicted the outcome of the election using a sample of 50,000 voters, while the Literary Digest failed to predict the outcome with over 2 million responses.
You can imagine the excitement the industry must have felt over this new advancement: “this will transform the industry”, “our cost savings will enable any sized business to conduct predictive polling”, “new insight on what people want delivered at a price you can afford” and so on. The key to Gallup’s advantage was the basic survey science principle of representativeness. In addition to advancing polling research methods, Gallup also used scientific principles in the development of ad-testing procedures.
Now imagine, in those days, that new players were rushing to market with pseudo-scientific sampling methods, that didn’t adhere to the basic scientific principles of representative sampling or experimental design. For example, imagine that researchers were developing ad-testing methods that used sampling but didn’t use random assignment into groups. The insights coming out of those studies, would no doubt have been delivered with great panache and inspiring emotion, touting the benefits and “predictive validity” of new sampling methods, and inspiring confident business decisions as a result!
Scientifically Validated Implicit Research Methodology is Essential
And yet, over time, if critical business decisions were being made on those less than scientific sampling approaches, executives would gradually begin to see that the predictions weren’t as accurate as they were claimed to be and their businesses would suffer as result.
Suffering businesses, who could trace their missteps to non-science based research results, would surely lose confidence in the providers of those “insights” and the market research industry would also suffer as a result. Fortunately, the scientific approach to sampling and experimental design ultimately won the day and created valid sampling standards for our industry.
Fast forward eight decades, our industry is facing an inflection point again. New implicit research technology has emerged that complements traditional methods and improve predictive validity (see Falk et. al, 2012 for an example). There is tremendous energy in the industry around these methods with practitioners claiming they can “tap the consumer subconscious”, “more accurately forecast future behavior” and “deliver new insights to confidently differentiate your brand like never before.”
Furthermore, it turns out that all of these claims are true. But they are true only if you adhere to the science behind the approaches. Thus, as an industry, in order to avoid a potential loss of confidence we need a clear understanding of the scientific principles behind implicit research methodologies.
In that spirit, I offer what I hope to be clear explanations of how some of the most common scientifically validated implicit research techniques work, as well as some examples of methods that are currently parading under the banner of implicit research while not meeting the scientific criteria.
There are at least 20 different implicit association type measurement procedures that have been developed and vetted in the behavioral science literature (Nosek et. al 2011) (this can be as “there is no need to make up your own non-scientific method”). In a series of upcoming blog posts I will detail the most well known procedures and contrast them to non-implicit techniques, revealing the advantages and disadvantages of each.
In the name of creating lasting value for our industry, let’s protect the science, in order to deliver transformational insights that produce confident business decisions.
