I have a 11-year old daughter named Sophie who is not motivated to action unless a deadline is looming impossibly near. Unfortunately, Tomorrow’s Deadline is also when I become motivated to enable her bad habits by pitching in to help. And so it seems that every year the science fair surprises both of us in a way that should only be reserved for things you never see coming, like flat tires and cavities.
Last year was a slight exception. We had an idea that my work experience in behavioral psychology could offer some efficiencies. On top of that, the father-daughter connection provided enough enthusiasm to begin discussing the experiment several weeks before the tri-fold poster was to be displayed on the cafeteria table.
The Girl Scout Cookie Experiment
The experiment Sophie ultimately decided to go with combined social psychology and Girl Scout Cookies. She wanted to know if people are more likely to buy Girl Scout Cookies from someone that looks like them.
Originally, the experiment was to be confined to race and color, but after we both got excited about the possibilities, we began riffing hypotheses about gender, religious beliefs, proximity, mutual friends, income level. And if perceived similarity does impact purchase, do some similarities make more of a difference than others? What if there are no similarities? What if there are multiple Girl Scouts?
Out of Time… and Options
As exciting as this was, the weeks slipped by and it was apparent we no longer had enough time or cookies to get a representative sample across the most interesting questions (not to mention a control set of neighbors). And as you might have guessed by now, we were now dangerously close to our deadline… again. So close, in fact, that we did not have time to do the door-to-door experiment after all.
Sophie and I discussed our only remaining options:
- A) Ask some parents we knew whether they would prefer to buy cookies from a black Girl Scout or white Girl Scout
- B) Do the volcano experiment… again
While we were arguing about the mess we were in, it occurred to me that while we were both really hoping to do the Girl Scout experiment, Option A felt a little like cheating. By asking parents what they would prefer, we would not be learning anything about what they would actually do.
Most parents would probably say that race doesn’t matter, or even that the Girl Scout doesn’t matter. After all, it’s really all about the Thin Mints! And while they may believe this, the truth is that they don’t know how these factors would influence their choice to buy or not buy cookies.
In our case, using these self-reported predictions would be like fabricating the results; we’d be taking a shortcut by not actually conducting a scientific experiment—a requirement to pass the Science Fair.
I was busy blaming the school for their bias against behavioral science, and rationalizing the results which I knew would be garbage when this thought struck me: Why do I work in an industry that spends billions of dollars justifying the same thing?
The Market Research Problem
While we desperately hold on to the archaic notion that our decisions are the result of conscious deliberation, easily recorded in a questionnaire, we intuitively know this is not the case. In my college psychology class, I don’t recall conducting any behavioral experiments where we asked people what they would do in a given context—we actually had to conduct the experiment.
Even a casual assessment of traditional market research suggests that something is broken.
Humans are simply neither able to accurately self-report their motives nor predict their own behavior. Scientific evidence demonstrates that decisions are heavily weighted by subjective emotion and impulse, yet we are hardwired to explain our behavior rationally—even to ourselves. This is the most insidious problem for market researchers, that humans can only confabulate stories that explain our behavior, which makes us notoriously unreliable in predicting it.
A Simple Solution
For me this lesson of the “Girl Scout Cookie Experiment” clarifies both the greatest problem and solution for the future of the market research:
- Our biggest problem is that the methodological (and economic) foundation of the industry relies on the myth of consumer self-understanding.
- The solution is simple, yet challenging: to ensure that the research we deliver to clients would pass a sixth grade science fair.
Ironically, our company was named one of the Top 50 most innovative market research firms in the 2015 GRIT report. This is partly because we conduct scaleable behavioral experiments rather than relying on surveys. This may qualify us as a “progressive” market research company, but I am proud that it also qualifies us as a “traditional” scientific research company.