      |
STUDY DESIGN
Which Attributes?
The most important determinant of a successful conjoint study is choosing the right attributes and levels. Their choice depends on your study objectives. If you are forecasting sales, you need to include all attributes that have a significant impact on the purchase decision. If you are determining which features a product should have, you need to include only those attributes on which you might take action. If you are setting prices or estimating brand equity, you need to include those attributes.
If your study objectives change after you are in the field, you probably won't have included the right attributes for the new objectives. The solution is not to include every attribute you can think of, but to agree on the study objectives before you go to the field and then stick with them.
How Many Attributes?
The number of attributes to include in a study depends on several factors: the study objectives, the time allotted for the interview, and the level of respondent involvement in the product category. It also depends on the form of conjoint analysis you are using, the three most common being: card sort, Adaptive Conjoint Analysis (ACA) and Choice-Based Conjoint (CBC).
In a card sort, respondents are asked to rank or rate a set of concepts. Each concept is printed on a card and is fully profiled on all of the attributes included in the study. As you increase the number of attributes in your study, you also add to the amount of information presented on each card and the number of cards per interview. If your study uses more than six to eight attributes, respondents may become overloaded with information, and the reliability of their responses could diminish. When using a card sort, you should limit the number of attributes to eight or fewer. If you must include more than eight attributes in a card sort, investigate an approach called "bridging." Note, though, that using bridging blocks you from analyzing data at the individual respondent level - a key advantage of using conjoint measurement.
In contrast to the card sort approach, ACA shows respondents two concepts at a time in a "paired-comparison trade-off" format. Each concept is partially profiled on two, three, four or five attributes. Because ACA uses an adaptive approach to data collection (partial profiles along with sets of questions that determine which attributes are most relevant to each respondent), ACA studies can include up to 30 attributes. ACA uses about 10 minutes of interviewing time for every five attributes you include, so the interview for a 15-attribute study would take about 30 minutes.
Because ACA can handle a large number of attributes, it's possible to ask respondents to consider a larger number of attributes than they would normally evaluate when making an actual purchase. This could lead to an unintended decrease in the value of the utilities for all the study attributes. In pricing studies, this "number of levels effect" leads to an artificial reduction in buyers' price sensitivity and an overestimation of their willingness to pay for individual features. As a result, you should consider using a full-profile approach - such as card sort or choice-based conjoint - for pricing studies. Or, use ACA and limit the number of attributes to five or fewer. If your pricing study requires a large numbers of attributes you may want to use "dual conjoint," a technique that combines CBC with ACA, or ACA with holdout concepts, for calibrating price sensitivities.
CBC, the last of the three approaches, shows respondents several product concepts at a time and asks them to choose the product they would most likely purchase. These choice tasks are repeated several times for each respondent. The number of attributes you can use with CBC has increased in the last few years. It is now possible, although not recommended, to include up to 30 attributes; most CBC studies include 6 to 10 attributes.
Interactions Between Attributes
Most conjoint studies account for main effects only, ignoring the impact of interactions between attributes. Such main-effects models work remarkably well in most cases because few attributes have been found to interact significantly. An important exception involves the brand and price attributes: the shape of a respondent's price sensitivity curve often differs across brands.
One way to handle such known (or suspected) interactions is to create a compound attribute made up of the two interacting attributes. For example, a compound attribute for brand and price might look like this:
Brand A at $150
Brand A at $200
Brand A at $250
Brand B at $150
An even better way to deal with interactions is to use CBC and randomly generate the choice tasks. The advantage of this approach is that it lets you test for all possible two-way interactions where, in contrast, the compound attribute approach assumes that you know, in advance, which attributes interact.
Choosing Attribute Levels
Within each attribute, the levels you specify must be mutually exclusive and exhaustive. You must be able to describe each of the products you want to model in terms of one - and only one - level of each attribute. You cannot skip attributes or apply two levels of the same attribute to any product.
During analysis, you can interpolate between attribute levels to specify untested levels, but you cannot extrapolate beyond the endpoints of the attribute. For example, if we were studying lawn mowers and included engine size as an attribute with levels of 2.0, 2.5 and 3.0 horsepower, we could run analyses that included mowers with 2.25 or 2.75 horsepower engines, but not ones with 1.5 or 3.5 horsepower engines.
To arrive at a proper set of attribute levels you must think ahead to the range of products you will want to include in your analysis. You should assess: (1) the time period that the analysis will span (is it the current market, the market a year from now, two years from now, etc.?); (2) the range of products you might offer during that time period; and (3) the range of products your competition is likely to offer over the same time period. The range of attribute levels you select must be broad enough to cover all of the relevant scenarios. The best way to check that you have chosen an adequate set of attributes and levels is to set up and run the market simulator on a set of test data before you field your study.
Number of Attribute Levels
For continuous attributes such as price, weight, or battery life you must decide how many levels your attributes should include. Having more levels results in a longer interview, but gives you finer resolution; fewer levels shortens the interview, but the increments between levels are coarser. In general, asking respondents to evaluate fewer levels within a specified range and interpolating between levels when doing the analysis leads to more accurate results.
Another important consideration in setting the number of levels for an attribute is the "number of levels" effect. Researchers have found that, as the number of levels increases, the measured importance of an attribute artificially increases as well. Consider, for example, two price attributes: one with three levels ($150, $200, and $250), and another with five levels ($150, $175, $200, $225, and $250). Both span the same price range, but if you use the five-level attribute in your study, price will have greater importance in the analysis than if you used the three-level version of the attribute. Researchers have yet to determine the source of this effect. In the meantime, you can minimize its impact by assigning, as much as practicable, the same number of levels to each attribute.
STUDY IMPLEMENTATION
Sample Size
Conjoint analysis can be conducted at the individual level, the segment level or the market level. For example, at the individual level you could use conjoint analysis (card sort or ACA, but not CBC) to predict which new home a homebuyer would prefer. In this case a sample size of one would be sufficient. Similarly, conjoint studies of limited-size markets, such as the major oil companies, can be based on just 10 to 15 respondents.
Typically, consumer studies use samples ranging from 150 to 1200 respondents. Card sort and ACA studies tend to be toward the lower end of this range, while CBC studies tend to be at the upper end. For conjoint studies where it's necessary to measure differences between segments, about 200 respondents are needed per segment.
Interviewing Modality
All forms of data collection are suitable for conjoint analysis - except telephone surveys. With few exceptions, responding to conjoint questions over the telephone is too difficult for respondents. For ACA, you must collect data by computer.
"Frame-of-Mind"
Conjoint interviews ask respondents to make tradeoffs between product attributes. As the intended use for a product changes, respondents are likely to make different tradeoffs between attributes. For example, in a study of sport utility vehicles, the tradeoffs respondents make between ground clearance and cargo space will differ depending on whether the SUV will be used for shopping or for driving off-road. It is important that respondents maintain a consistent frame-of-mind as they complete the interview - and that you know what that framework is. The best way to do this is by specifying a particular framework as part of the conjoint interview.
Length of the Conjoint Task
In a card sort, the minimum number of cards (NC) that each respondent needs to evaluate is given by the formula:
NC = NL - NA + 1
where NL is the total number of attribute levels in the study, and NA is the total number of attributes in the study. For example, if your study has five attributes with four levels each, the minimum number of cards would be (5 x 4) - 5 + 1, or 16. If you use only the minimum number of cards, you cannot account for respondent error in evaluating the concepts. Therefore, it is generally recommended that you include 1.5 times the minimum number of cards in a card sort task.
For ACA, the minimum number of questions (NQ) a respondent is asked is given by the formula:
NQ = 3(NL - NA - 1) - NL
This includes the questions leading up to the paired-comparison questions and the paired-comparison questions themselves. You can decrease the number of questions - and shorten the ACA interview - by reducing the number of paired-comparison questions ACA asks. A good way to do this is to set the number of pairs ACA can ask to its maximum value, then run through the interview and count the actual number of pairs ACA asks by enforcing the stopping rule for NQ given above. Cut that number in half and then set the number of pairs to that value for your study.
With CBC, the rule of thumb for the number of choice tasks respondents can handle is much simpler: 20. If you need to shorten the interview, you can cut this number in half at the expense of doubling the sample size to obtain the same confidence level.
Pretesting
Within a conjoint study you want to make sure that respondents are correctly interpreting your attributes and levels. Do this by debriefing pretest respondents to find out specifically what each attribute meant to them. Also make sure that the conjoint task is not too long or difficult.
During pretesting you should also check whether your interviews contain "nonsensical" attribute pairings; if so, eliminate them. For example, most car studies would not pair "convertible" and "sport utility vehicle" in the same concept. Although these sorts of pairings are permitted in conjoint interviews, respondents feel they are absurd and may take their task less seriously. In a card sort or a CBC interview of fixed design, eliminate nonsensical pairings by generating new sets of concept cards or tasks until you get ones with no illogical pairings. In an ACA interview or a CBC interview of random design, you can prohibit pairings of specific attributes.
DATA ANALYSIS
Interpreting Utilities
Interpreting respondent utilities is a useful way to generate hypotheses for more detailed analysis. Table 1 shows a set of utilities for one respondent for the earlier lawn mower example. The utilities are scaled using a standard format: the lowest level of each attribute is set to zero and the highest level across all attributes is set to 100. Note that the zero-level utilities indicate the least preferred level of each attribute; they do not mean that those least-preferred levels have no utility.
| Table 1: Utilities for One Respondent | ||
| Respondent#1 | UTILITIES | |
| Power | 2.0 Horse Power 2.5 Horse Power 3.0 Horse Power |
0 35 60 |
| Cutting Width | 18" 21" 24" 27" |
0 24 53 100 |
| Warranty | 1 Year 3 Years |
0 48 |
| Brand | Mow-Rite Chopper Cut Master Lawn King |
16 59 0 5 |
| Price | $150 $200 $250 |
80 35 0 |
Interpreting utilities involves analyzing the gaps between utility levels since the absolute values of the utilities have no meaning by themselves. For example, it would be incorrect to say that Respondent #1 prefers a 3.0 horsepower engine (60 utiles) over a 24" cutting width (53 utiles). But, we could say that, since the gap between a 3.0 and 2.5 horsepower engine is 25 utiles and the gap between a 24" and 18" cutting width is 53 utiles, Respondent #1 would prefer a mower with 2.5 horsepower engine and a 24" cutting width over one with a 3.0 horsepower engine and an 18" cutting width. Utilities can be valued only relative to other utilities because respondents in conjoint interviews are asked only relative questions about product concepts.
Attribute Importance
Analysts often use average utilities to compute the importance of each attribute. This is typically done by taking the difference between the lowest and highest average utility (the range) for each attribute, adding these differences across all attributes to get a total and then dividing each attribute's difference by the total and multiplying by 100. Table 2 shows an example of this computation.
| Table 2: Frequently Used, but Incorrect, Way of Measuring Attribute Importance | ||||
| UTILITIES (All Respondents) |
RANGE | ATTRIBUTE IMPORTANCE (Impact on Purchase Decision) |
||
| Power | 2.0 H.P. 2.5 H.P. 3.0 H.P. |
0 28 57 |
57 - 0 = 57 | 100 x 57/ (57+84+46+47+78) = 18.3% |
| Cutting Width | 18" 21" 24" 27" |
0 34 61 84 |
84 - 0 = 84 | 100 x 84/ (57+84+46+47+78) = 26.9% |
| Warranty | 1 Year 3 Years |
0 46 |
46 - 0 = 46 | 100 x 46/ (57+84+46+47+78) = 14.7% |
| Brand | Mow-Rite Chopper Cut Master Lawn King |
40 55 8 40 |
55 - 8 = 47 | 100 x 47/ (57+84+46+47+78) = 15.1% |
| Price | $150 $200 $250 |
78 43 0 |
78 - 0 = 78 | 100 x 78/ (57+84+46+47+78)=25.0% |
Although this form of analysis appears to be valuable and informative, there are pitfalls to using it. For example, in Table 2 "importance" typically would be interpreted as follows: cutting width has the greatest impact on buyer preference (26.9%), followed by price (25.0%), power (18.3%), brand (15.1%), and warranty (14.7%). This statement is incomplete and could be misleading: If we changed the range of levels for an attribute, we would expect its utility range - and its relative importance - to change. For example, if we had tested prices from $150 to $200, the price utility range would have been just 35 utiles, making price the least important among the attributes. It is more precise to qualify statements of importance by saying that for the attributes and levels tested, cutting width has the greatest impact on buyers' preferences, followed by price, power, brand, and warranty.
Another problem with this type of analysis is that attribute importance is often computed using average utilities as shown in Table 2; this is incorrect. The correct way to compute importance for a particular attribute is to compute the range for each respondent and then average across respondents.
Information Hidden by Average Utilities
When interpreting utilities be aware that you can miss important information if you look only at average utilities. Table 3 expands on Table 2 by showing the utilities for two equally-sized subgroups of respondents: those with gas-powered lawn mowers and those with electric-powered lawn mowers.
| Table 3: Average Utilities for All Respondents and Two Segments | ||||
| All Respondents | Gasoline-Powered | Electric-Powered | ||
| UTILITIES | UTILITIES | UTILITIES | ||
| Power | 2.0 Horse Power 2.5 Horse Power 3.0 Horse Power |
0 28 57 |
0 41 83 |
0 15 31 |
| Cutting Width | 18" 21" 24" 27" |
0 34 61 84 |
0 37 59 86 |
0 31 63 82 |
| Warranty | 1 Year 3 Years |
0 46 |
0 41 |
0 51 |
| Brand | Mow-Rite Chopper Cut Master Lawn King |
40 55 8 40 |
13 51 10 71 |
67 59 6 9 |
| Price | $150 $200 $250 |
78 43 0 |
91 56 0 |
65 30 0 |
Notice that what appears to be equal preference for Mow-Rite and Lawn King for the overall sample turns out to be a strong preference for Mow-Rite among gasoline-powered segment and a strong preference for Lawn King in the electric-powered segment. Also, notice the differences in relative importance for both power and price between the two segments. To avoid this sort of misinterpretation, examine utility averages at the level of segmentation for which you will be making recommendations.
In all instances, analysis of utility averages is best used to generate hypotheses, not draw conclusions. Market simulations, which are based on individual utilities, are a better and safer form of analysis.
Conjoint Results are Relative -- Not Absolute
There are four types of models analysts can use to simulate market behavior based on conjoint data: First Choice, Share of Preference, Randomized First Choice and Purchase Likelihood. Each model estimates buyer preference by using respondent utilities to estimate preferences for products profiled on the study attributes. None of the models (nor any other model based on conjoint data alone) can support absolute statements such as: "Product A will get 8% market share" or "Product C will out sell Product B 2-to-1." Conjoint models yield only relative results and are best used for ranking alternative courses of action. For example, they can tell us that Product A is preferred to Product B, or whether changing a level of a given attribute for Product A will increase the number of respondents who prefer it.
First Choice Models Overstate Preference
First Choice models simulate preference behavior by summing a respondent's utilities for each product and then selecting the product with the highest utility. Thus, First Choice models tend to produce more extreme results than are observed in actual purchase situations in that they overestimate preference for the most attractive products and underestimate it for products that are least attractive. This occurs because First Choice models make no allowance for buyer error or out-of-stock situations. Share of Preference models, which are probabilistic, tend to produce more realistic estimates of purchase behavior.
Accounting for Similarities Between Products
Share of Preference models simulate preference behavior by splitting a respondent's preference among all of the products included in the analysis. The share of preference assigned to each product is a function of the total utility for that product and also depends on which form of the model is being used. One of the most commonly used Share of Preference models uses the logit transformation to convert respondent utilities to preference shares. Although logit-based models generally yield better preference estimates than First Choice models, they overestimate preference for similar product, which is particularly a problem in product categories with a number of "me-too" products. In these instances, use either a Share of Preference model that corrects for product similarity or the Randomized First Choice model.
The Randomized First Choice model is an important extension of the First Choice model. For each respondent it adds variation to the respondent's utilities for each attribute level and also to the respondent's total utility for each product in the model. It then applies the First Choice rule to determine the respondent's product choice. It repeats this process numerous times for each respondent, drawing the variations at random from distributions that yield the best fit to a set of holdout concepts presented to the respondent during the interview. By averaging results of the repetitions across all respondents, the Randomized First Choice model produces "shares of choice" that compensate for product similarity better than Share of Preference models that correct for product similarity. In particular, the Randomized First Choice model avoids the awkward situation where, in correcting for product similarity, the Share of Preference model can actually increase the predicted share of a product that was made slightly less attractive.
Preference Shares and Shares of Choice are Not Market Shares
Preference shares from Share of Preference models and shares of choice from Randomized First Choice models look like market shares, but they are not market shares and you should not report them as such. For preference shares and shares of choice to equal market shares, all of the following conjoint assumptions would have to hold in the product category under consideration:
- All products have equal distribution.
- All products have equal advertising (in terms of both quality and exposure).
- Buyers are familiar with all products (including any new products that are part of the analysis); that is, they know each product's features, prices and specifications.
- All products are at the peak of their life cycles; none is at the beginning or at the end.
- All attributes important to the purchase decision have been included in the study.
- Buyers make completely rational purchases and do not buy on impulse.
- Buyers make one, and only one, purchase.
- Buyers have no brand loyalty and no reluctance to replace products they own.
- Preferences translate into purchase behavior.
Accounting for Market Growth or Shrinkage
In estimating shares of preference or shares of choice for different market scenarios, conjoint analysis does not account for changes in market size. As you add, remove, or reconfigure products, shares are redistributed among products, but market size does not vary.
Because purchase likelihood goes up or down as products are made more or less attractive, some researchers use the Purchase Likelihood model to provide market size estimates. This can be risky: respondents' stated purchase likelihoods cannot be taken literally - they are almost always overstated. What's more, Purchase Likelihood models are designed for analyzing single-product markets, where First Choice and Share of Preference models cannot accurately simulate market conditions.
Other researchers use CBC with the "I would not choose any of these" alternative added to each choice task to estimate changes in market size. This choice gives respondents the opportunity to say they would not purchase anything if they had to pick among the alternatives presented. However, use of the "I would not choose any of these" alternative has been extensively debated and the consensus, for now, is that it should not be used to estimate changes in market size.
The best way to estimate changes in market size is to ask specific questions about expected product purchases in your survey. You then combine these data with the results of your conjoint modeling.
Conjoint Provides Demand Side Information Only
Finally, keep in mind that the results of a conjoint study give us only half the picture - the demand side. There are also cost considerations, capacity constraints and other factors that influence which product strategy is best. Ultimately, decisions need to be based on an interplay between what the market wants and what a company is capable offering.
In this article we have reviewed many of the key issues and assumptions you'll deal with as you get started with conjoint analysis. With experience you'll quickly become adept at applying this technique to a wide range of marketing issues. As a newcomer to conjoint, you may also find it helpful to explore the topics covered in this article in greater depth. A comprehensive resource is the Conjoint Analysis Literature List Web site (http://www.geocities.com/melles99/melles_e.html), which includes a bibliography of related articles. The Sawtooth Software reference library (www.sawtoothsoftware.com) also contains many useful articles. If you need help getting started with a study, Sawtooth Technologies (www.sawtooth.com) is a resource for seminars and consultation services that are designed to get you up to speed quickly.
|
Technical Support | Contact Us | Search | Sitemap | Home Sawtooth
Technologies, Inc. 1500 Skokie Blvd., Suite 510, Northbrook, IL 60062 ©2003 Sawtooth Technologies, Inc. All rights reserved. |