I love Starbucks coffee. I’m a brand lover for sure – I make several stops per day for their delicious dark roasted coffees. But I’m not a fan of their latest taste test for their new instant coffee (VIA™) because it stacks the deck in favor of their new product.
This morning I promptly walked into my neighborhood Starbucks, and stepped up to the counter to take my taste test of the new ready brew instant coffee (there was a coupon for a free tall coffee waiting for me after the test). The task was to see if I could taste the difference between two blind samples of coffee, and to tell my Barista which one I liked better. Traditional instant coffee is known for being flat and flavorless on the tongue, and the new Starbucks VIA™ is pitched to be the opposite. One sample was the new Starbucks VIA™ instant coffee, and the second sample was Starbucks regular brewed coffee. This seemed like a simple straightforward comparison. I tasted away and lo and behold I did indeed like the instant coffee better.
But here’s the rub from a research standpoint…The instant coffee was either Starbucks Columbia or Starbucks Italian Roast and the brewed coffee was Starbucks Pike Place blend…ah, the research bias reveals itself. Pike Place is a mild (by Starbucks’ standards) blend and the Columbian and Italian Roast coffee is much bolder. Obviously this is poor experimental design. We’re not comparing apples to apples here. If we really wanted to see if people could tell the difference between instant and brewed coffee, we would want everything else in our experiment held constant – that is, we would want the beans across samples to be the same. Why not compare instant Italian Roast to brewed Italian Roast? If the beans differ across samples, how can we conclude whether it is the brewing method or the beans that is driving preferences between the two samples?
Of course, in a new campaign for instant coffee, you can’t have people walking away thinking the instant coffee tastes flat and flavorless. The new VIA™ product is far from it. But we also know from psychological science that preference judgments for one product are made relative to a comparison product. With that knowledge, it is not hard to imagine why the designers of the taste test would want something relatively mild as comparison coffee. So in this case, even if unintentional, it looks like Starbucks stacked the deck in favor of a successful VIA™ launch.
To be fair the taste test isn’t a real experiment, but rather just an effective awareness campaign for Starbucks’ new product. It is so effective in fact, that we’re out here blogging about it…
Here’s to good coffee and sound experimental design in your research!
Watch the Starbucks new ad below and let us know what you think of the coffee, campaign and the experimental design of the taste test:
[fve]http://youtu.be/bFvaXXFrg-E[/fve]