Conjoint Analysis: After the Basics by Joseph Curry Scroll through the text or use the hyperlinks below to access the subsections of this article: Study Design Study Implementation Data Analysis The popularity of conjoint analysis over the last 10 years is due partly to the availability of easy- to-use conjoint software. But, if you've purchased the software you probably realize that "easy-to-use" is not the same as "easy-to-apply". Don't be intimidated. Applying conjoint analysis isn't difficult once you are aware of its assumptions and you know how to handle a few key issues. The following should help researchers who know the basics get ready to do their first conjoint studies. Back to top STUDY DESIGN Which Attributes? The most important determinant of a successful conjoint study is selecting the right attributes and levels. The attributes and levels you choose depend on your study objectives. If forecasting sales is your objective, you need to include all attributes that have a significant impact on the purchase decision. If you are trying to determine which features a product should have, you need include only those attributes on which you plan to take action. If you are studying pricing or brand equity, you need to include those attributes. In the event that your study objectives change after you are in the field, you probably won't have the right attributes for the new objectives. The solution is not to include every attribute you can think of, but to agree on the study objectives before fielding the study and then stick with them. How Many Attributes? The number of attributes to include in a study depends on several factors: the study objectives, the time allotted for the interview, the level of respondent involvement, and the form of conjoint analysis that you use to collect the data. The two most common forms of conjoint analysis are the card sort and Adaptive Conjoint Analysis (ACA). In a card sort, respondents are asked to rank or rate a set of concepts; each concept is printed on a card and is fully profiled on all of the attributes. As you increase the number of attributes included in your study, both the amount of information on a card and the number of cards per interview increase. Thus, with more than 6-8 attributes, respondents can become overloaded with information, and the reliability of their answers may diminish. When using a card sort, limit the number of attributes to eight or less. If you must include more than eight attributes, investigate the possibility of using a "bridging technique" , which can handle more attributes. Unfortunately, bridging techniques eliminate your ability to analyze data at the individual level. ACA shows respondents two concepts at a time in a "paired-comparison trade-off" format. Each concept is profiled on no more than five attributes (most likely two or three). Because ACA uses partial profiles and its computer interview asks respondents about only the most important and relevant attributes, an ACA study can include up to 30 attributes. With ACA, allow about 10 minutes of interview time for every five attributes you include; a 15-attribute interview would take about 30 minutes. Because ACA can handle up to 30 attributes, it's possible to ask respondents to evaluate more attributes than they would consider when making a purchase. This can cause the relative importance of attributes to be reduced. In pricing studies, it can lead to an artificial reduction in price sensitivity. For pricing studies, consider using a full-profile approach -- such as a card sort or choice-based conjoint -- or use ACA with five or fewer attributes. "Dual conjoint" using ACA is also a possibility if your pricing study requires a large number of attributes. Interactions Between Attributes Most conjoint studies account for main effects only, ignoring the impact of interactions between attributes. Such main-effects models work remarkably well in most cases; fortunately, few attributes have been found to interact. One important exception is brand and price: the shape of a respondent's price sensitivity curve often differs by brand. One way to handle such known (or suspected) interactions is to create a compound attribute made up of the two interacting attributes. A brand-price compound attribute might have levels such as Brand A at $150, Brand A at $200, Brand A at $250, Brand B at $150, and so on. You can also use choice-based conjoint analysis to handle -- or test for -- interactions. Using choice-based conjoint precludes the possibility of analyzing your results at the individual respondent level, however. Choice of Attribute Levels Attribute levels must be mutually exclusive and exhaustive. Each product that you plan to include in your analysis must be describable on one -- and only one -- level of each attribute. You cannot skip attributes or apply two levels of the same attribute to a single product. During analysis, you can interpolate between attribute levels but you cannot extrapolate beyond the end points. For example, if we were studying lawn mowers and included engine size as an attribute with levels of 2.0, 2.5 and 3.0 horse power, we could run an analysis that included mowers with 2.25 or 2.75 horse power engines but not ones with 1.5 or 3.5 horse power engines. To arrive at a proper set of attribute levels you must think ahead to the range of products you will want to include in your analysis. You should assess: (1) the time period that the analysis will span; (2) the range of products you might offer during that time period; and (3) the range of products your competition is likely to offer. Your range of attribute levels must be broad enough to cover all of the relevant scenarios. The best way to check that you have the right attributes and levels is to set up and run the market simulator on a set of test data before you field your study. Number of Attribute Levels For continuous attributes such as price, weight, or battery life you must decide how many levels your attributes should include. Having more levels results in a longer interview, but gives you finer resolution; fewer levels shortens the interview, but the increments between levels are coarser. However, researchers have found that as the number of levels increases the measured importance of an attribute increases as well. Consider, for example, two price attributes: one with three levels: $150, $200, and $250, and another with five: $150, $175, $200, $225, and $250. Both cover the same price range, but if you use the five-level attribute in your study, price will have more effect in the model than if you used the three-level attribute. Researchers have not yet determined the source or the magnitude of this effect; in the meantime, you can minimize its impact by assigning the same number of levels to all your attributes. It's unlikely that you can achieve this across the board (for example, some attributes are binary and others require many levels), but make them as uniform as possible. Back to top STUDY IMPLEMENTATION Sample Size Conjoint analysis can be applied to individuals, market segments or entire markets. At the individual level, for example, you could use conjoint analysis to predict which new home a home buyer would prefer. In this case a sample size of one would be sufficient. Similarly, conjoint studies of limited markets, such as the major oil companies, can be based on just 10 to 15 respondents. For market segments, conjoint results tend to stabilize after about 30 to 50 respondents; try to have at least 30 to 50 respondents per market segment you want to study. Typically, consumer studies with multiple segments have sample sizes ranging from 200 to 400 respondents. Interviewing Modality All forms of data collection are suitable for conjoint analysis except telephone surveys. With few exceptions, responding to conjoint questions over the telephone is too difficult for respondents. Interview Framework Conjoint interviews ask respondents to make tradeoffs between attributes. As the intended use for a product changes, respondents may make different tradeoffs between the same attributes. For example, in a study of sports utility vehicles, the trade-offs respondents make between ground clearance and cargo space will differ depending on whether the vehicle will be used primarily for shopping or driving off-road. It is important that respondents have this sort of framework in mind as they complete the interview and it is just as important that you know what that mind set is. You can handle this by either specifying a use occasion for respondents or eliciting a use occasion from them. Number of Cards The minimum number of cards (NC) each respondent needs to evaluate in a card sort is given by the formula: NC = NL - NA + 1 where NL is the total number of attribute levels in the study and NA is the total number of attributes in the study. If your study has five attributes with four levels each, the minimum number of cards would be (5x4) - 5 + 1, or 16. And, if you use only the minimum number, you cannot account for respondent error in evaluating the concepts. Therefore, it is generally recommended that you include 1.5 to 2 times the minimum number of cards in your card sort task. Pretesting In a conjoint study you want to be sure that respondents are correctly interpreting your attributes and levels. After pretest respondents have completed the interview, debrief them specifically about what each attribute meant to them. You also should make sure that the conjoint task is not too long or difficult; this will depend in large part on the subject of the interview and the respondent's involvement with the product category. Check whether your pretest interviews contain nonsensical pairings of attributes so that you can eliminate them. For example, most car studies would not pair "convertible" and "station wagon" in the same concept. Although conjoint analysis permits these types of pairings, respondents feel they are absurd and may take their task less seriously. In a card sort, eliminate nonsensical pairings by generating new sets of concept cards until you get a set with no illogical pairings. With ACA you can prohibit pairings of specific attributes automatically. Back to top DATA ANALYSIS Interpreting Utilities Respondents' utilities are useful in generating hypotheses for further analysis. Table 1 shows a hypothetical set of utilities for one respondent using our lawn mower example. The utilities are scaled using a standard format: the lowest level of each attribute is set to zero and the highest level across all attributes is set to 100. Note that the zero-level utilities indicate the least preferred level of each attribute; they do not mean that those least-preferred levels have no utility.   Respondent#1 UTILITIES Power 2.0 Horse Power 2.5 Horse Power 3.0 Horse Power 0 35 60 Cutting Width 18" 21" 24" 27" 0 24 53 100 Warranty 1 Year 3 Years 0 48 Brand A B C D 16 59 0 5 Price $150 $200 $250 80 35 0 Table 1: Utilities for One Respondent Interpreting utilities involves analysis of the gaps between utility levels; the absolute values of the utilities have no inherent meaning. For example it would be incorrect to say that Respondent #1 prefers a 3.0 horse power engine (60 utiles) over a 24" cutting width (53 utiles). However, we could say that since the gap between a 3.0 and 2.5 horse power engine is 25 utiles and the gap between a 24" and 18" cutting width is 53 utiles, Respondent #1 would prefer a mower with 2.5 horse power engine and a 24" cutting width over one with a 3.0 horse power engine and an 18" cutting width. Utilities can be valued only relative to other utilities because respondents in a conjoint interview are asked only relative questions about product concepts. Attribute Importance Analysts often use average utilities to compute the importance of each attribute. This is typically done by taking the difference between the lowest and highest average utility for each attribute (the range), adding these differences across all attributes to get a total, then dividing each attribute s difference by the total and multiplying by 100. Table 2 shows an example of this computation.   All respondents UTILITIES RANGE ATTRIBUTE IMPORTANCE (Impact on Purchase Decision) Power 2.0 H.P. 2.5 H.P. 3.0 H.P. 0 28 57 57 - 0 = 57 100 x 57/ (57+84+46+47+78) = 18.3% Cutting Width 18" 21" 24" 27" 0 34 61 84 84 - 0 = 84 100 x 84/ (57+84+46+47+78) = 26.9% Warranty 1 Year 3 Years 0 46 46 - 0 = 46 100 x 46/ (57+84+46+47+78) = 14.7% Brand A B C D 40 55 8 40 55 - 8 = 47 100 x 47/ (57+84+46+47+78) = 15.1% Price $150 $200 $250 78 43 0 78 - 0 = 78 100 x 78/ (57+84+46+47+78)=25.0% Table 2: Frequently Used, but Incorrect, Way of Measuring Attribute Importance Although this form of analysis appears valuable and informative, there are several problems inherent in it. Using the example in Table 2, importance is typically interpreted as follows: cutting width has the greatest impact on buyer preference (26.9%), followed by price (25.0%), power (18.3%), brand (15.1%), and warranty (14.7%). This statement is incomplete and can be misleading: If we changed the range of levels of an attribute, we would expect its range of utilities to change and therefore its relative importance. For example, if we tested prices from $150 to $200, the price utility range would have been just 35 utiles, making it the least important among the attributes. Thus, it is more precise to qualify the earlier statement of importance by saying that for the attributes and levels tested, cutting width has the greatest impact on buyers' preferences, followed by price, power, brand, and warranty. Another problem with this type of analysis is that attribute importance is often computed using average utilities as shown (incorrectly) in Table 2. The correct way to compute the importances is to compute them for each respondent and then average across respondents. Information Hidden by Average Utilities When interpreting utilities be aware that you can miss important information if you look only at average utilities. Table 3 expands on Table 2 by showing the utilities for two subgroups of respondents: those with gas-powered lawn mowers and those with electric-powered lawn mowers.     All Respondents Gasoline-Powered Electric-Powered UTILITIES UTILITIES UTILITIES Power 2.0 Horse Power 2.5 Horse Power 3.0 Horse Power 0 28 57 0 41 83 0 15 31 Cutting Width 18" 21" 24" 27" 0 34 61 84 0 37 59 86 0 31 63 82 Warranty 1 Year 3 Years 0 46 0 41 0 51 Brand A B C D 40 55 8 40 13 51 10 71 67 59 6 9 Price $150 $200 $250 78 43 0 91 56 0 65 30 0 Table 3: Average Utilities for All Respondents and Two Segments Notice that what appears to be indifference between Brand A and Brand D among all respondents turns out to be a strong preference for Brand D in the gasoline-powered segment and a strong preference for Brand A in the electric-powered segment. Also, notice the differences in relative importance for both power and price between the two segments. To avoid this sort of misinterpretation, analyze utility averages at the level of segmentation for which you will be making recommendations. In all instances, analysis of utility averages is best used for hypothesis generation. Market simulations, which are based on individual utilities, are a better and safer form of analysis. Conjoint Results are Relative -- Not Absolute Analysts generally use three types of models to simulate market behavior based on conjoint data: First Choice, Share of Preference, and Purchase Likelihood. Each model estimates buyer preference by combining respondent utilities with a set of competing products profiled on the study attributes. None of these models (nor any other model based on conjoint data alone) can support absolute statements such as: "Product A will get 8% market share" or "Product C will out sell Product B 2-to-1." Conjoint models yield only relative results and are best used for ranking alternative courses of action. For example, they can tell us that Product A is expected to be least preferred or that we should produce Product C rather than Product B. First Choice Models Overstate Preference First Choice models estimate preference behavior by adding up a respondent's utilities for each product in a simulation and then assuming the respondent chooses the product with the highest utility. First Choice models, however, tend to produce more extreme results than are observed in actual purchase situations; they overestimate preference for the most attractive products and underestimate it for those that are least attractive. This occurs because First Choice models make no allowance for buyer error. Share of Preference models, which are probabilistic, tend to produce more realistic estimates of purchase behavior. Accounting for Similarities Between Products Share of Preference models split a respondent's preference among all products included in the analysis. The share of preference assigned to a product is a function of the total utility for each product and also depends on the form of the model. One of the most commonly used models converts respondent utilities to preference shares using a logit transformation. Although logit models generally yield better estimates than First Choice models, they overestimate preference for similar products. This can be a problem if your product category has a number of "me-too" products. Use a Share of Preference model that corrects for product similarity. Preference Share is Not Market Share Preference shares from Share of Preference models look like market shares, but they are not market shares and you should not report them as such. For preference shares to equal market shares, all of the following conjoint assumptions would have to hold in your product category: All products have equal distribution. All products have equal advertising (in terms of both quality and exposure). Buyers are familiar with all product (including any new products that are part of the analysis); that is, they know all product features, prices and specifications. All products are at the peak of their life cycles; none is at the beginning or at the end. All attributes important in making a purchase decision have been included in the study. Buyers make completely rational purchases and do not buy on impulse. Buyers make one, and only one, purchase. Buyers have no brand loyalty and no reluctance to replace products they own. Preference equals behavior. Accounting for Market Growth or Shrinkage In estimating shares of preference for different market scenarios, conjoint analysis does not account for growth or shrinkage in a market. As you add, remove, or reconfigure products, shares of preference are redivided among products, but estimates of market size are not provided. Some analysts use the Purchase Likelihood model to provide market size estimates since purchase likelihoods go up or down as products are made more or less attractive. This is incorrect since respondents' statements of purchase likelihood cannot be taken literally - they are almost always overstated. What s more, Purchase Likelihood models are designed for analyzing single-product markets, product categories where First Choice and Share of Preference models break down. One way to estimate market growth or shrinkage is to ask specific questions about expected product purchases in your survey and combine these data with your conjoint results. Or you could explore the possibility of using choice-based conjoint analysis. Conjoint Provides Demand Side Information Only Finally, keep in mind that the results of a conjoint study give us only half the picture -- the demand side. There are also cost, capacity, and other implications of implementing a particular product strategy. Ultimately, decisions need to be based on an interplay between what a market wants and what a company can afford to offer. In this article I have summarized some of the key issues and assumptions you'll deal with as you get started with conjoint analysis. With experience you'll quickly become adept at applying this technique to a wide range of marketing issues. As a newcomer to conjoint, you'll also find it helpful to explore the topics covered in this article in greater depth. Our reference library contains many useful articles that we would be pleased to send you; contact us at info@sawtooth.com. Joseph Curry is President of Sawtooth Technologies, Evanston, Illinois. This article originally appeared in the Marketing Research: A Magazine of Management and Applications, published by the American Marketing Association; all rights are reserved. Back to top Back to Sawtooth News Index