An Introduction to Choice-Based Conjoint

October 14, 2010 By Leave a Comment

Choice-Based Conjoint is the most popular conjoint technique used in market research today, and for good reason. Yet, if you’re new to market research, it may be a relatively new concept to you. In order to help overcome this hurdle, Research Access contributor Nico Peruzzi has put together this extremely helpful introductory video, which in under 10 minutes, provides a recent history of choice-based conjoint, discusses things end-users of conjoint need to be aware of when planning a conjoint project, and introduces the latest conjoint method called Adaptive Choice-Based Conjoint.

Our thanks to Nico for this particularly valuable contribution benefiting those who are just beginning their market research education. For an overview of the Conjoint concept in general, as well as background information on a variety of other market research concepts and techniques, be sure to check ou the Resources section of Nico’s website.

Filed Under: Market Research, Video Tagged With: , ,

Highlights from the Sawtooth Software Conference 2010 – Day 4

October 11, 2010 By Leave a Comment

Day 4 is a half-day, but it had some very serious papers and great discussion – the marketing science community is alive and well here!

Modeling Demand Using Simple Methods: Joint Discrete/Continuous Modeling (Tom Eagle, Eagle Analytics of California)

  • Discussed 4 approaches volumetric modeling (find out not only “what”, but “how many” of something respondents prefer.
  • Examined Regression Models, Choice Models, Economic Models, and Joint Discrete/Continuous (D/C) Models, with a focus on the latter.
  • D/C Models estimate in two stages: first fit an allocation Multinomial Logit model, then fit a general linear volume model using predictions from stage 1 as independent variables.
  • Compared all methods on 4 relatively simple datasets.
  • Summary of comparisons: Choice Modeling performs as well or better than the joint D/C models.
  • Conclusions: Joint D/C volume modeling is a valid approach to modeling complex volume models.  Because all these techniques occur on the back-end, you can design your study the same way and try out different approaches once data is collected.

Recent Developments in PLS Modeling: An Application for Customer Loyalty and Retention (Stuart Drucker, Drucker Analytics)

  • This talk was about predictive analytics, specifically key driver analysis
  • The way NOT to solve this problem is using stepwise regression (TAKE NOTE: lots of people still do this!).
  • Discussed Partial Least Squares (PLS) Regression and PLS Structural Equation Modeling (PLS-SEM).  In these approaches, factor analysis is web with regression analysis into a unified (more confirmatory in the case of PLS-SEM) framework.  Better manages common issues including multicollinearity.
  • Conclusion: The decision of which model to accept is a philosophical one, depending on whether the ultimate focus is estimating a system of effects (including Customer Satisfaction and the Key Drivers) – use PLS-SEM, or if the focus is maximizing the explained variation of Customer Retention – use PLS.

A Head-to-Head Comparison of the Traditional (Top Down) Approach to Choice Modeling with a Proposed Bottom Up Approach (Don Marshall, TVG, Siu-Shing Chan, Univ. of Pennsylvania, and Joseph Curry, Sawtooth Technologies)

  • Huge effort involving lots of people to set up this experiment.
  • Based on Jordan Louviere’s recent assertion  (2009) that when using HB to measure preferences that we were just capturing respondent inconsistency and that we needed to stop modeling the way most people currently are.
  • Compared current gold standard (Hierarchical Bayes (HB)) where individual preferences are influenced by group averages (called the “Top-Down” approach), to the “Bottom-Up” approach that examines individual preferences independent of group averages.
  • In Top-Down, respondents see different choice sets and choice the best, with a dual-response none follow-up, and are shown fewer screens.
  • In Bottom-Up, respondents all see the same choice sets, they choice top choice, last choice, and whether all/some/none are acceptable, and are shown more screens.
  • Conclusions: while the design and analysis criteria for bottom-up continue to evolve and improve, this analysis provides no compelling reason to recommend bottom-up over top-down at this point.  Interview length and completion rates favor top-down.

HB-CBC, HB-Best-Worst or No HB at All? (Ralph Wirth, GfK Group) – Note: this paper won “best paper” for the conference.

  • Concerns have been raised regarding CBC-HB, and this paper used Monte Carlo Simulations and 4 real-world datasets to find out if the concerns were justified.
  • The Best-Worst idea in CBC is gaining interest – the idea is to show profiles and have respondents choose not only their most, but also their least preferred option.  Compared this approach to standard CBC-HB and also a Louviere approach which asked for most preferred, least preferred, and an in-between choice, and did NOT use HB for analysis.
  • Conclusions: no model was consistently superior based on fit.
    • The Louviere approach is worth considering when data conditions are good and/or the focus is on share prediction rather than prediction of individual choices.  Purely individual estimation makes it much simpler than HB approaches but seems detrimental when data conditions are sparse.
    • The HB approach has good overall performance also under sparse data conditions, there is no negative influence of individual-specific error variances, and the results suggest that the use of additional preference information from worst-choice leads to better estimations, as Best-Worst CBC-HB is consistently superior to standard CBC-HB.
    • Discussion: lots of opinions here, but some main take aways are that:
      • HB will not fall out of use anytime soon, as it appears to perform well under a number of situations.
      • We need to model (simulate) against real-world outcomes whenever possible.
      • Best-Worst CBC is emerging as something to keep exploring, however, there is potential to get into a lot of trouble during the analysis, and there is no commercial tool that provides a solution (other than doing completely custom-programmed analytics).

Conference Conclusions

  • If you are involved in doing conjoint analysis, or other varieties of research that seek to understand preferences and choice behavior, this conference is a must attend.  The top minds in the field are all here and they are pushing the boundaries to achieve better measurement of choice behavior.  The conference occurs every 18 months, and information can be found at www.sawtoothsoftware.com.
  • Note that Jordan Louviere, a key discussant at the conference, was the recipient of the AMA 2010 Parlin Marketing Research Award.  See his interview in the 9/30/10 edition of Marketing News.
  • Plus, the food was really good :)
Filed Under: Events, Market Research, Uncategorized Tagged With: , , ,

Highlights from the Sawtooth Software Conference 2010 – Day 3

October 8, 2010 By Leave a Comment

For the morning sessions, the main conference was joined by the attendees of the conjoint analysis in healthcare conference that is running here in parallel.

The Value of Conjoint Analysis in Healthcare for the Individual Patient (Liana Fraenkel, Yale University of Medicine, VA Connecticut Healthcare System)

  • Looked at conjoint as a way to elicit patient preferences in low certainty situations.
  • Research shows that eliciting patient preferences can have some positive effects.
  • Conjoint analysis is a natural fit, given the trade-off approach.
  • Used Adaptive Conjoint Analysis to provide individual-level interviews.
  • Also looked at using MaxDiff, although it appeared that some respondents had difficulty with the best-worst trade-off, therefore tried a best-only approach.
  • Pros = works at the individual patient level, can handle lots of info, can provide immediate feedback, trade-offs are like real life, discourages rating all features equally
  • Challenges = hard to get independent attributes, hard to specify levels of attributes, have ranges of levels, have dominant attributes, can be difficult for respondents.
  • Reality = patients so rarely are given choices that they don’t know how to react, MD buy-in, discordance between patient preferences and what MD things should happen.

Tailoring Treatment Based on Preference Values (Marsha Wittink, Univ. of Rochester School of Med, Univ. Of Pennsylvania School of Med)

  • Goal was to use conjoint to identify which attributes of treatments are most important to help design better interventions.  Focus here was on patients with depression.
  • Used Hierarchical Bayes to calculate individual level utilities, and also used latent profile analysis to look for unique groups.
  • Did find unique groups preferring different treatment modalities.
  • Plan to use data to assess whether these preferences are better predictors of treatment uptake than other demographics.  Could also use to tailor treatments.

Conjoint Design Effect on Respondent Engagement (Paul Johnson, Western Wats)

  • Looked at CBC tasks with 20 vs 30 cards, and also at Adaptive CBC – looked at placement of conjoint task before or after other survey questions.
  • No differences in the way respondents answered other questions based on type or order of conjoint.
  • Purchase intent higher with ACBC task – might put respondent more in the mood to buy.
  • Respondents did NOT speed through the rest of the survey after doing the conjoint first.
  • Time spend on conjoint task longer for ACBC.
  • Few to no differences in measures of consistency, hit rates, or error levels in model.
  • ACBC did best job of predicting a winning holdout concept.
  • Other benefits of ACBC: get explicit rules of non-compensatory rules made by respondents, get more stable model estimates with smaller N’s.

Sales Promotions in Conjoint Analysis (Marco Hoogerbrugge & Eline van der Gaast, SKIM Analytical)

  • Looked at the best ways to present price promotions for testing in conjoint analysis.
  • Ideas of ways to display:
    • Original (gross) price + % or $ off.
      • When model, need to assign levels to final price.
  • Original (gross) price NOT shown, promotion (net promoted price) only thing shown.
  • Original (gross) price + promotion price shown and highlighted.
    • Could model just net price, or keep original price as main effect, or include interactions between original and promotion price in model.
    • Modeling would benefit from data about purchase behavior – more external data.
    • Note that you’ll need to model differently based on which approach you take.

How Many Questions Should You Ask in CBC Studies? – Revisited Again (Jane Tang & Andrew Greenville, Vision Critical)

  • Past (and more recent) research
    • Johnson & Orme (1996): ask up to 20 tasks; in later tasks brand becomes less important, price more important, and more likely “none” choice.
    • Hoogerbrugge & van der Wagt (2006): increase in hit rate after 10-15 tasks is small; complexity of study more influences hit rate.
    • Markowitz & Cohen (2001): HB hit rates not greatly enhanced by increasing sample size; more choice sets better than more sample.
    • Suresh & Conklin (2010): complex survey design leads to lower respondent engagement; more complex attributes leads to choosing “none” more often and more price reversals.
    • Hauser, Gaskin & Ding (2009): Non-compensatory rules used when more time pressure, more products, and more familiarity with the category.
    • Current study looked at 3 conjoint tasks: 6 cards with 3 options, 15 cards with 3 options, and 15 cards with 5 options.
    • Conclusions:
      • Increasing the # of tasks gave limited improvement in prediction ability, but at cost of slight deterioration in sensitivity and consistency.
      • Simplifying behavior occurs in later tasks – more likely so when respondent familiar with the category.
      • Increasing complexity of task (showing 5 vs 3 options) doesn’t help anything.
      • The balance is this: sometimes, individual level models don’t converge when have small # of tasks, so need to ask more tasks to get precision, but as do so, reliability goes down – need a balance .

The Strategic Importance of Accuracy in Conjoint Design (Matthew Selove, USC, John Hauser, MIT Sloan)

  • Looked at what happens when we have “noise” in a sample versus “heterogeneity”
  • Noise leads to less differentiation in product decisions, whereas heterogeneity encourages differentiation.
  • Compared a “well” and “poorly” design conjoint study.
  • Poor designs lead to more noise, which leads to inconsistency in the conjoint task and less ability to validate to hold-outs.
  • Need to accurately estimate randomness.  If you have no hold-outs to validate against, Louvierre recommends tuning model to an exponent of 0.4.

Product Portfolio Evaluation Using Choice Modeling and Genetic Algorithms (Chris Chapman, PhD & James Alford, PhD, Microsoft)

  • With conjoint data, we know how to optimize a product, but what about a product line?
  • Took CBC and ACBC data, derived individual-level partworths using HB model, iterated using a genetic algorithm to fit many portfolio preference models, and inspected.
  • Took 1080 possible products (based on 9 attributes with 2-7 levels) and found that after 6-8 products in the portfolio there was no more increase in share of preference.
  • Also asked which products appeared in a large number of winning portfolios – found a couple “new” products this way.
  • Other findings: CBC had much noisier price data than did ACBC; ACBC has more stakeholder face validity, smaller sample sizes needed, and better respondent engagement.

The Impact of Covariates on HB Estimates (Keith Sentis & Valerie Geller, Pathfinder Strategies)

  • Note: this presentation and the next were probably the most controversial and shocking to most of the conference participants.
  • Method: estimate HB partworths with and without a covariate, compare the quality of the two sets of partworths; do this for several different covariates, one at a time.  Looked at 3 classes of covariates: demographic variables, category behavior variables, and attitudinal variables.  Measures of fit and predictive accuracy included: RLH, Hit Rate, Holdout Likelihood, and MAE (error).  Measures of partworth variability included: importance spread and standard deviation ratio.  Ran across 5 datasets.
  • Conclusion: NO lift in predictive accuracy by using covariates; did see some increase in partworth variability.
  • Discussion: should covariates be in our tool kits?  This paper says NO, but why might we want to include them?  Carefully chosen covariates can provide insights by subgroup.

Added Value through Covariates in HB Modeling (Peter Kurz, TNS Infratest Forschung GmbH, and Stefan Binner, bms marketing research + strategy)

  • Since HB assume one single multivariate normal population, can get “shrinkage” of respondents  toward the population mean, and therefore segment differences could be reduced.  So, how about adding covariates into the upper level model?
  • Method: looked at 10 commercial studies, 30,000+ conjoint interviews, B2B, B2C, worldwide; looked at natural (demographic), segmentation membership, and intention or past behavior data as covariates.
  • Conclusion: they ran all kinds of models, from standard HB to HB with covariates to Latent Class, etc., and HB with covariates did better only on 2 studies.
  • Recommendations: ensure sufficient sample size, use standard HB, use hold out tasks.
  • Discussion: If have clearly defined clusters, covariates could help; more heterogeneity can help with market share projections, line extension and optimization decisions, estimates if willingness to pay, and it helps IIA (red bus-blue bus problem).

Regarding these last two papers, everyone really wanted to believe the covariates could helpful, but the evidence argued against improved predictive accuracy by using covariates.

Filed Under: Events, Market Research, Uncategorized Tagged With: ,

Highlights from the Sawtooth Software Conference 2010 – Day 2

October 8, 2010 By Leave a Comment

The sun came out in Newport Beach on Wednesday, but with 10 presentations there was no time to play. Here are some highlights on each of the talks.

What Drives Me? Developing a Conjoint-Based Recommendation Engine for Individual Vehicle Consideration (Ely Dahan, UCLA, Claremont, Princeton)

  • Individual-level conjoint engages the respondent in an adaptive, engaging exercise with the goal of helping them gain personal insight and get a recommendation.
  • Used the example of a car recommendation engine in development for Edmunds.com.
  • Uses pre-existing market data, incorporated with individual choices.
  • Has good promise for the area of consumer search.

Analyzing Consumers’ Screening Rules by Means of Virtual Online Shops (Soren Scholz & Reinhold Decker, Bielefeld University, and Beate Sarnowski & Marie Schuir, Interrogare)

  • A virtual online shop was developed to help overcome some of the limitations of traditional conjoint approaches when studying complex purchase decisions.
  • Their tool picks up on non-compensatory decision making behavior (note that Adaptive Choice-based Conjoint does as well).
  • Good predictive validity (hits on hold-out tasks).
  • Can be combined with conjoint techniques to improve measurement.

The Success of Choice-Based Conjoint Design Among Respondents Making Lexicographic Choices (Keith Chrzan, John Zepp, & Joseph White, Maritz Research)

  • A lexicographic choice model involves a sequential choice process – like non-compensatory decision making (setting rules or cut-offs, must-haves and unacceptables).
  • Research shows between 20% and 66% of respondents use a lexicographic choice process.
  • Be careful with minimal overlap designs – designs with overlap can be more informative.
  • Best-Worst Discrete Choice Analysis (where not only a best is chosen from a consideration set, but a worst is, as well) helps deal with lexicographic responders.
  • The idea is that if someone is going to make a cut-off type of choice, we’re going to lose information about other attributes, so we need to do things to get them to answer about secondary attributes. Note that Adaptive Choice-Based Conjoint does this well.

Menu-Based Choice Modeling Using Traditional Tools (Bryan Orme, Sawtooth Software)

  • The idea of selecting from menus in a choice exercise makes sense given all the times in the real world that respondents do so (restaurants, configure a computer, cars, telecom/internet/phone bundles, single or multiple drug therapies).
  • These designs are custom built, and currently, are custom analyzed.
  • Counts analysis won’t cut it; volumetric allocation, serial cross-effects, and exhaustive alternatives models are being used.
  • Exhaustive alternatives shows promise – has the benefit of being a single model, but can have a large number of parameters to estimate. Be careful of overfitting.
  • Aggregate Logit interestingly does as well as HB in some tests.

Analyzing Pick n’ Mix Menus via Choice Analysis to Optimize the Client Portfolio (Chris Moore & Corrine Moy, GfK NOP)

  • Moving from showing respondents a portfolio of “fixed” products for the consumer to choose from to a portfolio of features that consumers can pick and choose from to design their own product.
  • Focus on identifying a set of features that should be offered, how consumers want to customize their product, the premium (if any) consumers are willing to pay for a customized product, and how to price each feature to simultaneously increase customer value/revenue/profit.
  • Serial cross-effects model appears to be very robust with excellent holdout validation.
  • It’s important to include LOTS of sample (think N=1000).
  • Be careful about overloading respondents with too long/difficult a task.

An Empirical Test of Bundling Techniques for Choice Modeling (Jack Horne, Silvo Lenart, Paul Donagher, & Bob Rayner, Market Strategies International)

  • Some are using the term “Build Your Own” for Menu-Based Choice.
  • The “Reservation Price” is David Bakken’s idea of the highest price an individual consumer is willing to pay for a given product.
  • Key assumptions for bundling research:
    • When offered individually, consumers will buy products that are less than or equal to their reservation price and not buy products that are greater than their reservation price.
    • When responding to fixed bundles, the reservation price for the entire bundle (not the individual component products) will determine purchase.
    • This bundle reservation price may or may not be the sum of the reservation prices for the individual items making up the bundle.
    • Single Brand BYO asks respondent to build product for a single brand; Market BYO does the same for multiple brands; Fixed Bundling is like discrete choice (show products – which one prefer).
    • These methods force respondents to react to different marketplaces, and they produce different results.
      • Single Brand BYO: have fewest products chosen, often times only one.
      • Market BYO: larger predicted take rates of individual products.
      • Fixed Bundles: price sensitivity curves are flatter; larger predicted take rates of individual products.
      • Revenue and product penetration are maximized using Marker BYO or Fixed Bundles
      • Single Brand BYO and Market BYO also reach those individuals interested in single products who may not be interested in purchasing Fixed Bundles. In this way, the BYO methods can compliment Fixed Bundles.
      • When there is no brand, Single Brand BYO may be a good alternative for measuring generic willingness-to-pay and take rates.

Anchoring Maximum Difference Scaling Against a Threshold: Dual Response and Direct Binary Responses (Kevin Lattery, Maritz Research)

  • Following up MaxDiff questions with additional questions to tease apart respondents who have the same item importance rankings but could have gotten there by different means.
  • Louviere’s Indirect Method asks a question after every MaxDiff screen: “Considering just the 4 features above, which of the following best describes your views about which features are very important for your ideal [product]?” Response set = All 4 of these features are very important, None of these 4 features are very important, Some are very important, some are not.
    • Awkward and additional respondent work; appears to be reintroducing some scale/disposition bias, and if results used in segmentation, could dominate the segmentation. The more items shown per task, the more likely the outcome will appear indeterminate.
    • Direct Method is asked only once at the end of the MaxDiff exercise: “Please tell us which of the features below are very important for you ideal [product]?” Response set = all MaxDiff items (or a couple pages with randomized sets if item list is very long.
      • Much quicker; respondents more critical (less items very important); some context dependency for which items are chosen.

Directing Product Improvements from Consumer Sensory Evaluations (Karen Buros, Radius Global Market Research)

  • Discussed the issue that consumer product evaluations on taste, smell, texture, color and other sensory perceptions lack specificity.
  • Penalty analysis is the traditional approach, however, it can yield conflicting directions for product improvement (e.g., overall flavor too strong, and overall flavor too weak)
  • Tried to understand product perceptions using minimally verbal scales (bi-polar), have respondents rate product on multiple dimensions and use Latent Class regression to derive respondent-level coefficients measuring the impact of attributes on purchase intent.
  • Issues = multicollinearity, sensory attribute interactions, lacking chemists on staff (had to opt for perceptions).
  • Jury still out on effectiveness.

Policy Implications on the Diffusion of Alternative Fuel Vehicles: An Agent-Based Modeling Approach (Rosanna Garcia, Northeastern University, and Ting Zhang (Xi’an Jiaotong University)

  • Used agent-based modeling to model the interactions between technology push, regulatory push and market pull on eco-innovation.
  • Used conjoint data (discrete choice) to gauge market pull.
  • Used NetLogo to do agent-based modeling (ABM).
  • Found the use of conjoint data a great way to instantiate and validate ABM.

The Impact of Respondents’ Physical Interaction with the Product on Adaptive Choice Results (Bob Goodwin, Lifetime Products)

  • Wanted to determine the potential impact of respondents’ physical interaction with the product on the precision of adaptive choice-based conjoint (ACBC) results.
  • Split-sample ACBC studies of folding banquet tables and chairs were conducted using online and mall-intercept field methods. Market simulation results were then validated using actual product sales and market share distributions.
  • For tables, “touch and feel” less important than previously thought. In-person concentrated more on leg style, online more on size/shape.
  • For chairs, mall “butt test” didn’t look that important; however, the “mesh” chair was impossible to validate as it had no sales data. Only very minor differences in attribute importances.
  • Concluded that cost-benefit for in-person tests not justified, as online method had less prediction error for both tables and chairs.
  • Note, however, that new, innovative, or unfamiliar products may need to be demonstrated in person.

Using Eye Tracking and Mouselab to Examine How Respondents Process Information in CBC (Martin Meissner, Soren Scholz, & Reinhold Decker, Biefeld University)

  • Asks whether adding eye-tracking or mouselab data to choice-based conjoint (CBC) could improve modeling.
  • It was somewhat challenging to match up eye tracking data directly to attributes. And mouselab introduces a different cognitive process, as the image of the attribute level only appeared when a respondent moused over it, and disappeared when not moused over.
  • Found that added eye tracking data did improve models, especially when the intensity of the tracking to each attribute was quantified and incorporated into the model.
  • Concluded that eye tracking data can be used to qualitatively analyze how respondents approach purchase decisions, cross-check and validate CBC utilities and importances, check whether the relevance of attributes and way of information processing changes during interviews, and significantly improve choice models.
  • Also learned that only a minority of respondents consistently apply simple decision heuristics, like focusing on price only, repeated measurements do not seriously harm information processing, warm-up tasks facilitate a more holistic evaluation of alternatives, and the design of the task affects respondent information acquisition behavior (more complexity changes behavior).
  • Issues = adding person-specific data can lead to overfitting the model. The cognitive processes are still somewhat unknown – need fMRIs to really tell. Eye tracking is currently expensive to implement and adds a lot of time to the task.
Filed Under: Events, Market Research Tagged With: , , , , ,

Some Highlights from the Sawtooth Software Conference 2010 – Day 1

October 6, 2010 By Leave a Comment

[Editor's Note: Research Access contributor Nico Peruzzi is on-site at the Sawtooth Software conference this week, and sends us this dispatch.]

The weather is pretty grey in Newport Beach, CA, but that’s good because we can focus on the talks.  Day 1 is workshop day, and here are some highlights of the two that I attended.

Adventures in Advanced CBC Applications

Presented by Bryan Orme, President of Sawtooth Software and David Lyon, Principal at Aurora Market Modeling, LLC, this workshop covered lots of different advanced applications of Choice-Based Conjoint (Discrete Choice Modeling).

Here are just a few highlights of this four-hour workshop:

  • Alternative-Specific Designs give the flexibility to show unique sets of attributes based on the context (e.g., if asking about choices for short-range travel, when “drive my car” is shown, “parking fee $8.00” is also shown.  However, when “ride the bus” is shown, “picks up every 20 min.” and “75 cents per one-way trip” are shown.  These designs make the exercise more realistic.
  • Prohibitions (restricting attribute levels that can be shown in combination) can be a red flag for the way you have set up your attributes and levels.  General rule: if the respondent will be confused by a combination, use a prohibition; however, if it’s simply “we don’t offer this combination”, then you don’t have to use a prohibition.  Remember that prohibitions reduce the efficiency of your design and you’d need more sample to overcome this issue.
  • Condition Pricing (varying the price points shown conditioned on the presence of a level of a certain attribute) only makes sense is levels of price are in some sense consistent across conditions: typically the same $ difference or same % difference).
  • Hierarchical Bayes is general the best analysis method for these scenarios – certainly better than Aggregate Logit – and this is true of even basic CBC analyses.
  • Summed Pricing extends the idea of conditional pricing and attaches incremental price changes to each level of each attribute.  It’s important to add +/- X% to summed prices so that price is not confounded with levels.  Doing so helps partial out the effects of each level; so that the utilities are not “conditional” on the price.  +/- 30% was suggested as the random variation to be applied.
  • Bundling was discussion, and it can get complicated, but the idea is to create a custom design that mimics the purchase process as much as possible.  Modeling and simulation in this case is custom.
  • Volumetric CBC asks respondents to declare a “quantity” of each choice that they would buy, versus making a “discrete” choice as to which one they would buy.  Volumetric CBC appears better at revealing near-term spikes in market share, whereas, discrete choice is better at showing a long-term equilibrium for a product (it is more stable over time).
  • “Evoked Set” CBC takes a reduced set of attribute levels forward into the task (e.g., start with 14 levels, but respondents will only answer about 7).  Takes some fancy data-processing work.
  • Build-Your-Own (BYO) is task where respondents see all attributes and choose which level of each that they prefer.  BYO can be a good “training task” for respondents, especially if they are going into a complex choice task.  BYO is built into Sawtooth Software’s Adaptive Choice-Based Conjoint.
  • Menu-Based Choice (MBC) appears to be what will become the newest addition to the conjoint family.  Design is very customized and respondents have an opportunity to choose various items from a “menu” to build their preferred product.  Counts analysis can work for now, but Hierarchical Bayes (HB) analyses will likely form the basis of models being developed to manage this type of data.  However, early research suggests that HB doesn’t give as much lift over Aggregate Logit in menu-based choice as it does in regular CBC.

Research for Solid Pricing Decisions

Presented by David Lyon, Principal at Aurora Market Modeling, LLC, this workshop presented an overview of commonly used pricing research.  The workshop was organized in terms of direct questioning techniques and trade-off methods.

Direct Questioning Techniques:

  • Willingness to Pay: “How much are you willing to pay for this?”.  At best, for a radically new product, gets you near the ballpark.  Don’t pre-list answer choices – wipes out upside possibility.  Plot % willing to pay at a certain price.  Overall, pretty weak technique.
  • Monadic Designs: split sample into groups and present different price to each.  Best to add a buy-response question (“Would you buy it?”, or better yet, a less-variable measure of purchase intent such as an intent scale or allocation/likelihood.  Use large samples and match cells carefully.
  • Sequential Monadic: ask initial, fully-disguised monadic question, then follow-up with “What about this price?”, “What about that price?”.  Sometimes down low to high price, or high to low, or “Gabor-Granger” (random order).  Problem = no way to disguise focus on price in follow-ups, and results in consistent over-estimation of price sensitivity = not realistic.  Huge biases here.
  • Van Westendorp Price Sensitivity Meter: four questions: At what price would you find the product… “too cheap”, “cheap”, “expensive”, “too expensive”?  Curve crossing analysis shown to be unrealistic; try plotting % of respondents who fall in the “normal” range (between cheap and expensive) against prices.  Ok for early exploration; view with skepticism.
  • Newton-Miller-Smith variation of Van West: add 2 questions: at [cheap price], likelihood to buy?, at [expensive price], ditto.  Translate likelihood scale into purchase probability, average probability curves over all respondents.  Problem = most of us can’t do translation to probabilities.

Trade-Off Techniques:

  • Ratings-Based Full Profile Conjoint: present profiles, have respondent rate or rank them.  Problem = systematically underestimates price sensitivity.  More concrete attributes like price are under predicted, whereas more emotionally laden attributes are over estimated.
  • Price-Only Choice-Models: allows showing different prices for different brands, allows different price utilities for different brands, and avoids systematic underestimation of price effects.  Think of each product’s price as a separate “attribute” in the conjoint sense.  Can fractionalize the design so each respondent does not need to do the whole design.  Usually modeled at aggregate level, but using HB would be better.  Issue – cannot simulate with products deleted from or added to the basic set we designed around.  Pretty face valid that we are testing price, but experience shows that price sensitivity is realistic.
  • Discrete-Choice Modeling: (Choice-Based Conjoint): a whole talk in and of itself, but here are a few highlights.  Add other attributes to price-only to provide more realism to the task and decrease bias toward oversensitivity.  Using Multinomial Logit introduces the “red bus – blue bus” problem (independence of irrelevant alternatives: IIA) – means that if we have 4 products with shares: 40%, 30%, 20%, and 10%, and say we cut the price for the first product so that its share increases to 50%, IIA takes the loss of 10% and distributes it proportionally away from the other 3 products (i.e., we now have, 50%, 25%, 16.7%, and 8.3%).  Logit does this without “thinking”, no matter what the data might say.  Use HB instead to Aggregate Logit to solve this problem.
Filed Under: Events, Market Research Tagged With: , , , , ,