Conjoint Analysis Myths

Sagittarius street lamp

The use of conjoint analysis methods has increased over the years in different industries, and as Chris Chapman, from Google, indicated in his presentation at the 2013 Sawtooth Software Conference, this family of techniques has been successfully used to:

  • Determine feature preference
  • Predict market share
  • Find unmet needs in product portfolios
  • Determine likely response from competitors

However, this has led to some misconceptions about conjoint analysis that need to be discussed to adjust expectations. Based on his experience, Chapman lists some of the conjoint analysis myths that clients have:

  • MARKET SHARE: Many users of conjoint expect to get an answer to how many people would buy a product. However, what conjoint analysis tells us is how many people prefer a product compared with other tested alternatives. To get market share predictions we need a model that includes not only preferences, but also information about awareness, distribution, marketing, channel effects, etc. Conjoint analysis provides an important piece of the puzzle, but more information is needed.
  • GOOD FEATURES/BAD FEATURES: Many conjoint studies are set up to identify features that drive purchase. Clients sometimes interpret the results as indicators of “good” features in absolute terms. However, conjoint analysis provides insights into the tradeoffs people make among most and least preferred features. Preferences are conditioned by the context which is defined by the tested features. There is no absolute. Results can change drastically if a new feature is included or another is excluded.
  • PRICING: Clients often expect conjoint to tell them what the prices should be. However, conjoint is better suited to learn about price sensitivity, not to set exact price points. Results are very much dependent on the price ranges tested.
  • NEW PRODUCT DEVELOPMENT: Conjoint analysis is often used to determine what product should be developed. However, the highest average preference for a particular product configuration may hide heterogeneity in preferences, so you may end up with a product that doesn’t appeal to anyone in particular. Preferences alone are not enough to make this type of decisions. We need to consider cost and competition. The best product configuration may be already offered by a competitor and better opportunities may be found in niche areas. In Chapman’s opinion, conjoint analysis is best to inform product lineup and more precise optimization requires more data and expertise.
  • RESPONDENT GROUPS: Clients sometimes perceive respondents as belonging to a particular group or type (e.g. brand preferrers) with implicit true states. The fact is that conjoint analysis results are presented in aggregated probabilities even when individual-level utilities are estimated, which means there is not a “true type.” We should look at these groups as people with “tendencies”(e.g. they are likely to prefer brand X).
  • STATISTICAL POWER: Looking for great statistical power, if budget permits, many clients prefer large samples. However, Chapman advocates for using smaller samples (ideally 400) to replicate the study using different conjoint methods (traditional, adaptive, menu-based, etc.) to compare and validate results and thus maximize interpretative power by dividing statistical power.
  • BETTER THAN INSTINCTS: Yes, the key is to learn from the conjoint data even when we make incorrect decisions. In cases in which the data sounds counterintuitive, we shouldn’t simply dismiss it. Going with the data in this case tend to result in losses that are modest in comparison to when wrong decisions are made based solely on instincts.

These issues should not prevent you from doing conjoint analysis, but having them in mind helps with expectation management and leads to a better use of the insights conjoint analysis provides.

Michaela Mora is the president of Relevant Insights, LLC, seasoned in custom market research with more than 20 years of experience in industries such online subscription services, software, entertainment, offline and online retailing, automotive, travel, hospitality, consumer packaged goods, non-profit, insurance, and beverage among others.


Speak Your Mind