By: Kathi Stevens
Do you remember the Pepsi challenge? The Pepsi Challenge was a marketing promotion campaign started by Pepsi in 1975 that pitted Pepsi and Coke against each other. In a blinded taste test, consumers were asked to pick the cola that tasted the best. The real glory came when the loyal Coke drinker picked Pepsi. No one was more surprised than the loyal Coke drinker who would now change her ways and drink Pepsi forever more.
While this situation had minimal risk for the consumer, we know the same would not be true in a business setting. As a professional market researcher, you want to be prepared to challenge an alternative view. Going back to the Pepsi Challenge, a prepared participant would know that research suggests tasters will generally prefer the sweeter of the two beverages upon a single sip, even if they prefer a less sweet beverage over the course of an entire serving. The prepared participant would then have a credible defense if they were a Coke loyalist and chose Pepsi.
Market researchers face a similar dilemma as they are continually challenged to reconcile pre-determined evaluations of advertising, like copy testing, with secondary analytics such as market mix modeling. If a researcher is not prepared for an alternative view, they will lose credibility and bring into question their agency’s overall expertise.
Copy testing is an analysis used by many companies to evaluate the potential performance of an advertisement prior to airing. Marketing mix modeling evaluates the actual performance of a creative in market. If the two are not aligned showing the same results, the latter will most likely be questioned.
There are multiple approaches to align copy testing and marketing mix modeling. One can either explicitly model pre-test measures in the marketing mix or use pre-market test results to qualitatively inform mix models during development. Whatever approach is used, it is pretty clear that pre-market testing continues to play a role in setting expectations for in-market evaluation. A 2013 report by the CRE (Council for Research Excellence) entitled “The Current State of Marketing Mix Models” calls out the growing need for consideration of ad copy quality in mix results.
IRI has found that including copy quality in the modeling may lead to bias and therefore takes a qualitative approach to incorporating copy test results into marketing mix modeling insights. By taking a qualitative approach, the modeler has more flexibility to determine the validity of the copy test scores and how appropriate it is to directly compare them to creative measures in marketing mix models. When reviewing ad copy test results diagnostically for marketing mix modeling, here are some things to keep in mind:
- Seemingly minor changes can have significant impact on in-market performance. Work with your client/agency to understand how the tested ad copy was different from the aired ad copy.
- Tested results must be for the same duration/page size/type media.
- Ad copies tested early in development and in rough format are not comparable to a finished execution.
- Once an ad copy has been in market, any subsequent ad copy testing can be misleading because it may have already started to wear out.
- Be sure what was tested in the copy test is the same brand variant as in the creative that was executed. Often time companies change the product in the ad before it is aired or they test the ad on the broader parent brand vs. the brand variant featured in the creative.
Marketing mix modeling and copy testing can easily co-exist and should be used together to inform ad copy performance. A comprehensive Marketing Mix Model should qualitatively leverage both results to determine the relative in-market performance. Market researchers who are diligent about using pre-market testing in marketing mix results will find that clients are more receptive to the mix analysis findings and to the agency presenting these results. Even if you find your Coke is Pepsi, you will be prepared to understand why and not risk your credibility.
What do you think of the practice of aligning copy testing and market mix modeling?