51

International Journal of Innovation and Economics Development, vol. 4, issue 6, pp. 51-67, February 2019

International Journal of Innovation and Economic Development

ISSN 1849-7020 (Print)

ISSN 1849-7551 (Online)

URL: http://dx.doi.org/10.18775/ijied.1849-7551-7020.2015.46.2004 DOI: 10.18775/ijied.1849-7551-7020.2015.46.2004

Volume 4 Issue 6

February, 2019 Pages 51-67

Virtual Reality-Based Product

Representations in Conjoint Analysis

Jonas Jasper

Institute for Business-to-Busines Marketing, Münster, Germany

Abstract: Innovations are an important factor for companies to build and sustain a

competitive advantage. Even though companies generate up to 51 % of their income with

products and services launched less than three years ago, the failure rate of new product

development is still at around 90 %. One way to reduce the failure rate of innovations is to

conduct customers’ preference measurements in the early stages of new product

development processes. However, customers often display a substantial amount of

uncertainty when having to evaluate highly innovative products as they may not be able to

grasp its features and functionalities. Therefore, some empirical studies have already

compared different presentation forms in a conjoint setting. These studies aim to optimize

preference measurement results by exposing respondents to state-of-the-art product

descriptions in the new product development process. Based on a quantitative-empirical

analysis, this thesis contributes to this research vein by integrating virtual reality-based (VR)

product representations within a conjoint setting for the measurement of preferences in the

development process of a technically complex innovation. The results of this study point out

that VR offers the potential for early customer integration within the new product

development process.1

Keywords: Virtual Reality, New Product Development, Conjoint Analysis

1. Introduction

Innovations have become one source for companies to sustain a competitive edge and they

are essential for survival (Chuang 2015; Porter 2011; Backhaus et al. 2014a; Brockhoff 1999;

Christensen 1997). In times of global competition, a company's ability to introduce new

products to the market is a decisive factor in maintaining their competitive position, and it is

crucial to a company's long-term success (Carayannis et al. 2015). A study of the Centre for

European Economic Research points out that innovative companies made up to 51% of their

income with products and services launched less than three years ago (Rammer et al. 2014).

Also, the number of innovations across industries is increasing, while the time span over which

they are launched is shrinking (Carayannis et al. 2015). Companies aim to exploit time

advantages by launching new products faster than their competitors. Thus, companies start to

engage in a race to become pioneers when bringing innovations to the market. At the same

time, the failure rate of new product development can be up to 90 % (Hill et al. 2014). This

failure rate puts companies under pressure to pursue an efficient product development process

to remain competitive.

A problem within the new product development process is the division between developers'

and users' points of view (Daecke 2009). An advantage from a developer's point of view is not

1 The paper is based on the dissertation thesis of the author.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

52

in itself a sufficient condition when it comes to success in product development (Schmidt 2001;

Ernst 2002). What is important is the customer's2

Perception of the new product as being useful (Ogawa/Piller 2006). Thus, customers make the

final decision on the success or failure of a new product or service (Backhaus et al. 2014a;

Meffert et al. 2012).

But why do so many innovations fail in satisfying customers? An answer to this question is

found in von Hippel's (1986) framework of sticky information. For the development of

customer-oriented new products, there are two different types of information that need to be

aligned. On the one hand, there is a need for information, which refers to what users need.

On the other hand, there is solution information, which refers to how to build products. The

quality of the solution in conjunction with the significance of the needs it meets represents the

value of the innovation (Herstatt/von Hippel 1992). The problem is that need information and

solution information share two characteristics:

1. First, each has its respective location. By definition, users are in “possession” of the need

information, since their needs have to be satisfied. On the other side, the “owners” of the

solution information are the manufacturers and designers.

2. Second, both types of information are sticky, meaning that it is difficult both to translate

and to transfer them to the respective other location. In other words, it is difficult fo r

manufacturers to gather need information from customers, while at the same time

customers are not motivated and do not have the expertise to understand the solution

information of the companies (Toubia 2010).

This information transfer issue provides a fundamental challenge for new product

developments. If companies or manufacturers were able to grasp customer needs in detail and

with little effort, it would be easy for them to develop a product that fully satisfies these needs.

By the same token, if customers could become experts in the field of how to find a solution for

their needs, they would be able to develop new products on their own (Toubia 2010). Due to

this stickiness, companies pursue a variety of strategies for the sake of gathering relevant

need information from customers. This latter procedure refers to a customer-oriented new

product development process (NPD), which is intended to increase market acceptance of new

products and services (Ernst 2002; Narver et al. 2004; Helm/Steiner 2008; Backhaus et al.

2014a; Atuahene-Gima 2005). When aiming to make the development process more efficient,

obtaining input from customers in the early stages of new product development processes has

proven to be a promising strategy (Griffin/Hauser 1993). A systematic procedure for capturing

and incorporating customer needs and wants throughout the development of a new product or

service is thus highly relevant to reduce failure rates on innovations, especially in the context

of global competition and global markets (Füller/Matzler 2007; Barczak et al. 2009; Kim et al.

2011). Figuring out customers' preferences has become the “focal point”' (Ye et al. 2007, p.

1191) for most companies to ensure that their products become successful on the market.

However, asking customers what they want is not necessarily a strategy that is well-suited to

eliciting need information. Customers tend to mention needs that are served by corresponding

products already available on the market (Ulwick 2002; van Kleef et al. 2005).

This problem is particularly challenging when dealing with technical and highly innovative

products with which customers have no experience (van Kleef et al. 2005). Customers often

display a substantial amount of uncertainty when having to evaluate these types of product

(Hoeffler 2003). Highly innovative products require that customers change their behavior to

adapt to the product (Goldenberg et al. 1999). Therefore, one major difficulty when measuring

customer preferences results from customers' lack of knowledge about new products. When

confronted with and asked to state preferences about innovative products, customers may be

2 Includes potential customers.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

53

unable to give credible responses (van den Hende/Schoormans 2012). Reduced stability of

market research assessment and information with limited validity and accuracy are the result

(Shocker/Hall 1986; Durgee et al. 1998). When this happens, standard preference

measurement techniques fail (Hoeffler 2003; van Kleef et al. 2005). Therefore, marketing

scientists and practitioners have come up with new ideas to optimize preference measurement

techniques to avoid the above-mentioned problems.

When measuring customer preferences within the new product development process, a vast

amount of different techniques exist, whereby the conjoint analysis is generally accepted as

one of the most relevant methods for marketing research (Wittink et al. 1994; Helm et al.

2004; Steiner 2007). It provides insights into respondents' preferences for product attributes

and features and thereby lets companies guide the development process in the “right” direction

at an early stage. The conjoint analysis supports the identification of relevant features a new

product should have and enables determination of how it is priced. The underlying idea is that

respondents evaluate the overall desirability of a comprehensive product or concept, instead

of assessing different attributes and features independently (Hillig 2006; Orme 2006}. Such

an evaluation is made possible by developing some alternative product concepts (stimuli) that

consist of previously defined attributes and features (levels) and by having respondents

evaluate these features jointly (Sichtmann/Stingel 2007).

2. Literature Review

Research in the field of integrating varying presentation styles in a conjoint analysis started in

1981, shortly after Green/Srinivasan introduced the conjoint analysis in their well-known

paper. Since then, the research stream has continued to make progress, and conjoint settings

have been used in various contexts and with varying presentation styles. One reason for the

ongoing interest in this research vein is the steady emergence of new options to build more

advanced ways of stimuli presentation. Dahan/Srinivasan (2000) tested the effectiveness of

verbal, static and animated virtual prototypes against physical product concepts, to predict

market shares for bicycle pumps. While verbal stimuli predicted market shares well below

actual shares, both static and animated virtual prototypes closely matched actual market share

predictions, which in turn came from physical prototypes. In their paper “The Virtual

Customer”, Dahan/Hauser (2002) provide empirical insights and an overview of how customers

can be integrated into each stage of the product development process using six different web-

based interview techniques. The authors reveal that multimedia-propped product concepts

increase the efficiency and accuracy of the detection of respondents' preferences

(Dahan/Hauser 2002). In the most recent study conducted by Brusch (2009), textual,

multimedia and real prototypes of door handle systems are compared in a conjoint setting.

The author attested that multimedia-based product concepts yield the highest predictive

validity by exceeding both, real and verbal-based descriptions.

3. Research Goals and Conceptual Foundations One promising new path for early customer integration is the use of virtual realities for the

measurement of preferences in the development process of complex innovations (Backhaus et

al. 2014a; Söderman 2005; Kim et al. 2011). Virtual Reality (VR) is a technology of virtual

prototyping (VP), which stands for an easy-to-understand user interface that allows customers

to explore the functionality of new products and features interactively (Gausemeier et al.

2001). Virtual reality is a three-dimensional, computer-generated environment that allows

customers to interact with and to manipulate the product representation in real time. The

advantage of virtual realities is that they can be developed earlier in the new product

development process, more quickly and more cost-effectively than actual prototypes

(Backhaus et al. 2014a; Urban et al. 1997}. Also, virtual representations of highly complex

products aim to facilitate the transfer of information by presenting the features and benefits

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

54

of the new product to potential clients via a combination of visualizations, animations,

illustrations and texts (Gausemeier et al. 2011; Gausemeier et al. 2001).

Since virtual reality allows customers to immerse themselves in the virtual world, a

considerable amount of realism can be expected (Gausemeier et al. 2001; Ye et al. 2007;

Backhaus et al. 2014a). Even though previous studies have used multimedia-based stimuli, no

one so far has exposed respondents to state-of-the-art forms of virtual reality-based product

concepts that would allow full customer interaction and immersion.

This study has the aim to shed light on this stream of research. To such end, the study at hand

is guided by the following research question: Can preference measurements for a technical

and highly innovative product be improved by integrating virtual reality-based product

representations into a conjoint analysis?

The admittedly generic formulation of this research question intends to ensure that all different

facets are covered under its umbrella.

3.1 Object of Study

In line with the research gaps identified above, the study at hand aims to test the usefulness

of varying presentation styles within a conjoint analysis for a technical and highly innovative

product. Thus, the product should be in its infancy stage to ensure that customers are not

familiar with its functions and benefits. Also, it is essential that the product can be described

by some decision-relevant attributes and levels, which in turn can be represented by using

different presentation styles (Ernst 2001). Furthermore, the product should contain different

features, to ensure that the product has diverse utilities to customers.

At the time the study was run, a major German automotive headlight manufacturer had just

started the development of a new product within the leading edge technology network

Intelligent Technical Systems OstWestfalenLippe, which covers the requirements. The

company has an intelligent self-adjusting automotive headlight system in the concept phase,

which continuously analyzes the environment and relevant vehicle data for independently

controlling and adapting the optimal headlight settings for the illumination of the street in front

of the automotive. A similar system is not available on the market and respondents do not

have experience with it. Furthermore, the system is seen as containing a high degree of

complexity and is difficult to explain. As the se lf-adjusting headlight system builds on newly

developed technologies, it is highly innovative and is classified as technical innovation (Zahn

1995). Finally, automotive headlight systems are represented by different presentation forms

such as text, pictures, and virtual realities. For the manufacturer, the intelligent self-adjusting

automotive headlight system is a cost-intensive innovation project involving considerable

financial risk. Thus, the company has an interest in receiving feedback from a customer

assessment to steer the development project into the “right” direction at an early point in the

new product development process (Backhaus et al. 2014a)

3.2 Sample and Data Collection

Data are collected during one week in Germany at the world's biggest industrial trade fair. A

total of 367 respondents participated in the study. Of these, two respondents had to be

eliminated, as they did not complete all three tasks, which left a sample of 365 cases. These

365 respondents were divided as follows: 128 respondents conducted the assignment with

verbal stimuli, 126 with pictorial stimuli and 111 respondents are exposed to virtual-based

stimuli. To achieve a high level of realism with the virtual prototype, respondents sat on a

force-feedback motion platform, that is, a dynamic driving simulator, as shown in Figure 1, in

front of a 75''-LCD display, and interactively steered a virtual automotive at night (Backhaus

et al. 2014a, Backhaus et al. 2014b).

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

55

Even though the possibility of a sampling error was reduced by randomly assigning

respondents to groups, the composition of respondents was examined (Schmidt 2001, Stadie

1998). Otherwise, it is possible that differences between the groups could wrongly be ascribed

to the presentation style, even if they are the result of differing respondent profiles. Indeed,

more men than women participated in the study and about half the respondents had an

academic degree, which should be taken into consideration when interpreting the results.

Nevertheless, respondents show similar patterns regarding age, gender, educational

attainment and prior knowledge across the three groups.

Figure 1: Dynamic Driving Simulator

Source: Heinz Nixdorf Institute Paderborn

Additionally, groups' respective composition are compared concerning different personal

characteristics, which are used in the second and third part of the empirical study. Except for

the analytic cognitive style, the null-hypothesis of equal means cannot be rejected

(Bowerman/O´Connell 2007; Field 2009). Thus, respondents from the text group tend to be

slightly more analytical, compared to both other groups. It should be taken into consideration

when interpreting the results. For all other variables, the composition of respondents across

the three groups is homogenous. Overall, these findings confirm that the structure of

respondents across the three presentation styles is similar for all but the analytic cognitive

style, thereby qualifying the usage of a between-subject design (Schmidt 2001; Stadie 1998).

4. Results 4.1 Comparison of Validity Measures

Before examining the validity measures, a comparison is made if conjoint outcomes differ when

respondents see text, picture or virtual reality-based product concepts in conjoint analysis.

After all, if no differences are found between outputs of different presentation styles, further

investigation of the superiority of one presentation style over the others is less promising.

Hence, part-worth utilities and relative importance weights were compared between the three

groups using of an ANOVA and subsequently conducted Games-Howell-tests (Field 2009;

Stadie 1998; Schmidt 2001; deBont 1992}.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

56

4.2 Convergent Validity

4.2.1 Part-Worth Utilities

As a first indicator for assessing if differences exist between groups, a comparison of each of

the three groups' respective part-worth utilities follows (Stadie 1998). Respondents' part-

worth utilities are first calculated on an individual basis and then aggregated for each

respective group (i.e. text, picture, and VR-based stimuli).3 The resulting part-worth utilities

are shown in Figure 2.

Figure 2: Group Specific Part-Worth Utilities

Source: Own illustration

Overall, part-worth utilities between the groups of text, picture, and virtual reality-based

stimuli differ slightly. For the first attribute (user interaction), a required interaction for starting

the self-adjusting automotive headlight system is the least attractive choice across all three

groups. Respondents exposed to pictorial and verbal stimuli perceive the optional interaction

as the best one, while respondents from the VR group favor no interaction. This is a remarkable

finding, as user interaction is the attribute within the virtual reality in which most of the actual

interaction takes place. In other words, the empirical analysis shows that preference

measurement using VR-based stimuli differs significantly for the attribute calling for a great

deal of involvement on a respondent's part. Since one characteristic of a VR is the interactivity

component, this finding is a first indication that VR-based product concepts do indeed yield

diverging results.

3 Some studies such as Steiner (2007) and Stadie (1998) advise standardizing part-worth utilities before the

aggregation. However, standardization of part-worth utilities is only necessary if their absolute value

requires interpretation. For the study at hand, the relative size of the part-worth utilities is used, so no

standardization is required. Moreover, this standardization is only feasible if (1) a parameter estimation

model other than an OLS – such as a variance-based model – is applied, since the constant term cannot be

standardized; or (2) if an OLS model without a constant term is calculated. The latter approach, however,

results in less precise estimation results, because all information is included in the slope of the regression

line, even though much of the information “belongs” to the constant term.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

57

For the attribute adjustment scenario, respondents from the text and picture groups attach

the least utility to the option in which the automotive needs to stand in front of a wall/garage

to conduct the headlight adjustment. The second-to-least utility is attached to the option in

which the adjustment process is to take place on the rear of a car in front. The greatest utility

has a system that can perform the adjustment on the street in front of the automotive. Even

though respondents from the VR group also attest the level on the street to have the greatest

amount of utility, they rate the option on a wall slightly better than on the rear of the preceding

car.

Concerning the cut-off line, the marginal difference is attested between groups. While

respondents from the picture group favor the asymmetric z-shape, respondents from the text

and VR group prefer the L-shape. As the difference between all levels is rather low, this

indicates that this attribute is of minor importance for respondents' overall decision.

Finally, the price attribute shows the most likely pattern: for all three respondent groups, the

highest price (EUR 2,500) receives the least utility, followed by the average price (EUR 1,800)

while the lowest price (EUR 1,100) has the highest utility.

To test for significance, a one-way analysis of variance (ANOVA) followed by post-hoc tests

are conducted (Field 2009; Bowerman/O´Connell 2007}. Before calculating the one-way

ANOVA and subsequent post-hoc tests, three underlying assumptions are tested: first of all,

the samples of experimental units need to be randomly selected and must be independent of

one another. As shown in the study setup, this prerequisite holds. Secondly, the ANOVA

requires groups to be distributed normally (Stevens 2007; Bortz 2005; Everitt 1996). The

Kolmogorow-Smirnow and the Shapiro-Wilk test point out that this assumption is violated

(Field 2009). Finally, the ANOVA requires equal variances of each group, which can be assessed

using the Levene-test (Bowerman/O´Connell 2007). According to the Levene-test, the data at

hand violates the equal variance assumption. As the normal distribution and the equal variance

assumptions are violated, the Welch-test is conducted as an alternative to the one-way ANOVA

(Field 2009; Steiner 2007}.

When conducting subsequent multiple group comparisons (i.e. posthoc tests), alpha inflation4

can be an issue (Field, 2009). Alpha inflation is an increase in the probability of committing a

Type I error proportionate to the number of group comparisons (i.e. post-hoc tests) performed

(Field 2009). For instance, when comparing three groups, the probability of a Type I error

increases from 5 % to 14.3 %5. To avoid alpha inflation, the Bonferroni correction is applied

(McLaughlin/Sainani 2014; Field 2013; Bowerman/O´Connell 2007}. As two assumptions of

the ANOVA are violated, the “most powerful” (Field 2009, p. 374) post-hoc test is the Games-

Howell-test, as it not only accounts for alpha inflation, but remains valid even in the event that

the equal variance assumption is violated and sample sizes are unequal (Games et al. 1981;

Field 2013).6 Hence these parametric tests are calculated to compare the mean part-worth

utilities of all three groups.

4 also referred to as Familywise error rate or Experimentwise error rate (Field 2009). 5 Type I error when considering alpha inflation for a three group comparison at the 5 %-significance level:

1 - (1 - .05)3 = 14.3 %. 6 A robustness check was conducted in which the non-parametric Mann-Whitney-Test and Kolmogorov-

Smirnov-Test were performed (Ernst 2001; Sheskin 2011; Gastwirth et al. 2009). The results indicate that

respondents from the text-versus-picture-group additionally differ for the level On the street (p .01).

Furthermore, for the picture-versus-VR-group, significant differences were attested for the following

levels: interaction required (p .05) and Asymmetric Z-shape (p .1). For the sake of reservation,

conclusions are based on the more conservative results.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

58

The Welch-test indicates that statistical differences between groups can be attested for the

levels interaction optional and on the rear of a car in front. The subsequently conducted

Games-Howell-test points out that the text and VR group differ regarding the optional user

interaction at the .05 level while the picture and VR group differ at the 1 %-significance level.

Furthermore, for the variable on the rear of a car in front, the null hypothesis that the picture

and VR group are the same can be rejected in favor of the alternative hypothesis that means

differ between both groups at the .1 significance level. One possible explanation is that

respondents with virtual reality-based stimuli were able to see that the traffic ahead can

actually be disturbed and therefore attach less utility to this feature. Overall, groups diverge

across the above mentioned part-worth utilities, while no difference is attested between the

text and picture group.

4.2.2 Relative Importance

A second indicator for testing convergent validity is to compare the relative importance weights

between groups (Stadie 1998; Schmidt 2001}. Figure 3 depicts the average relative

importance across the three groups. While respondents from the text and picture group have

a similar focus, the emphasis respondents from the VR group place on these attributes

deviates. More specifically, the user interaction attribute has almost the same relative

importance for respondents of the text (42.22 %) and picture (41.72 %) group. However, the

group of respondents exposed to VR-stimuli perceived the attribute as less important (29.83

%). The VR group, in turn, placed greater importance on the adjustment scenario: 37.65 %

as compared to 20.14 % and 24.96 % for the text and picture group, respectively. As indicated

above, the cut-off line attribute has uniform relative importance across all groups, ranging

from 12.37 % for the text group to 13.45 % for the picture group, and up to 15.77 % for the

VR group. Finally, the price attribute has the least relative importance for the VR group, at

16.75 % compared to 19.86 % for the picture group and 25.27 % for the text group. The

statistical significance was evaluated using ANOVA and subsequent post-hoc tests (Field 2009).

The output of the significance levels of the group comparisons depicts Table 1.

Figure 3: Relative Attribute Importance across Groups

Source: own illustration

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

59

Table 1: Significance Levels for Mean Comparison of the Relative Importances

Source: Own illustration

In conjunction with the findings in Figure 3, the user interaction attribute differs significantly

between the text and the VR group (p .01) and between the picture and the VR group (p

.01). No significant differences were found between the text and picture group, as already

shown in Figure 3. For the adjustment scenario, the text and the picture group differ at the 5

%-significance levels, while the VR group differs from both other groups even at the 1 %-

significance level. For the relative importance of the cut-off line, no significant differences are

attested between groups. Finally, respondents perceived the price} attribute differently

between respondents from the text and picture group (p .05) and between text and VR group

(p .01). No significant deviations in the relative importance of the price attribute are detected

between the picture and the VR group. Overall, these results indicate that the three groups do

indeed diverge concerning the relative importance of the attributes. In other words, when

exposing respondents in a conjoint setting to different product presentation forms, the relative

importance they assign to each of these attributes varies.

4.3 Internal Validity

To address the question of which presentation style is superior, the coefficient of determination

(adjusted R2) was used as a first criterion (Backhaus et al. 2011; Bowerman/O´Connell 2007).

As an additional goodness-of-fit measure, the correlations between empirical input data and

estimated preference data across the three different groups (i.e. text, picture and VR) are

compared. As data for this study are on an interval scale and are converted into ordinal scales,

all three correlation coefficients, namely Pearson R, Spearman's rank correlation and Kendall's

Tau, can be estimated (Brusch 2009; Backhaus et al. 2011; Steiner 2007}. Table 2 depicts

the output.

Table 2: Internal Validity across Groups

Source: own illustration

First of all, the adjusted R2 is reasonably high, which means that respondents' answering

behavior can overall be regarded as consistent (Bowerman/O´Connell 2007). Secondly,

respondents exposed to textual stimuli have the lowest fit, where .74 of the total variation is

explained by the estimated OLS model. For respondents exposed to pictorial stimuli, the

adjusted R2 rises to almost .78. Finally, for respondents whose product explanations are based

on virtual reality, the adjusted R2 is over .81. Thus, the internal validity increases from text to

picture and peaks with virtual reality-based product concepts. The standard error decreases,

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

60

meaning that the variation across respondents is lower and the answering behavior is more

consistent.

As shown in Table 2 adjusted R2 between the text and the VR group differs at the .1 significance

level only. Furthermore, the correlation coefficients across all groups are also reasonably high,

meaning that respondents from all three groups show high consistencies in their answering

behavior. However, no significant differences are attested between all three groups between

Pearson, Spearman and Kendall's Tau. Hence, based on internal validity, only weak statements

are made as to which presentation style (i.e. text, picture or VR) leads to superior results since

no significant differences are attested to all but the adjusted R2 between the text and VR group.

4.4. Predictive Validity

One goal of preference measurements and conjoint studies in the narrow sense is to predict

actual decisions respondents would make in a real-life decision task (Dahan/Srinivasan 2000).

Therefore, predictive validity is tested across the three groups of presentation styles (Ernst

2001). As three hold-out cards are included in this study, it was possible to estimate the first-

choice hit rate as well as the first-second-choice hit rate (Steiner 2007). Table 3 exhibits the

results across all three respondent groups.

Table 3: Predictive Validity across Groups

Source: own illustration

While the first-choice hit rate is only slightly better for the VR-group, the first-second-choice

hit rate can predict actual decision behavior of the VR group better than in the text group by

7.36 percentage points. In comparison to respondents from the picture group, actual decisions

of the VR group can be predicted even better, by 10.47 percentage points. A chi-square test

was performed to test if the three groups differ at a significant level (Cerjak et al. 2010}. As

shown in Table 3, the picture and VR group differ significantly concerning the first-second-

choice hit rate (p .05). This result indicates that the conjoint study with embedded virtual

realities as stimuli is slightly better in predicting actual decision behavior of respondents

compared to a picture enriched conjoint setting.

4.5 Applicability Measures

The experience a respondent has of his or her interview (i.e. applicability measures) is

assessed as an additional validity criterion, as respondents' perceived interview experience

provides additional insights into the question of which presentation format leads to the higher

validity and thereby to better preference measurement results. Table 4 depicts the average

value of respondents' answers for all applicability measures and the respective significance

levels.

First, as shown in Table 4, respondents exposed to virtual reality-based stimuli were

significantly more satisfied with the amount of information provided as product description,

compared to respondents shown text or picture-based stimuli (p .01).

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

61

Table 4: Applicability Measures across Groups

Source: own illustration

Second, the VR group assessed the degree of realism significantly higher than either of the

other groups (p .01). These findings emphasize that the degree of realism increases towards

the more realistic end of presentation form alternatives. However, scholars have different

opinions regarding the question of whether or not a high degree of realism within a conjoint

study is favorable or not: while Srinivasan et al. (1997) advocate a high degree of realism to

get more viable conjoint results, Schrage (1993) and Appleyard (1977) state that highly

realistic product presentations can be too difficult to comprehend.

To test this question, respondents' decision difficulty is retrieved as a third applicability

measure. As can be seen, the perceived decision difficulty of the VR group is significantly lower

than for respondents of the text (p .01) and picture groups (p .1). This finding contradicts

the conclusion drawn by Cerjak et al. (2010) and Jaeger et al. (2001), that respondents

experience the same degree of difficulty when exposed to different presentation styles. It also

refutes Ernst (2001), who states that the degree of difficulty is higher, the more enriched

presentation formats become.

Spiegler (2011); Hoeffler/Ariely (1999) point out that respondents who feel uncertain about

their decisions or are overwhelmed by the evaluation task are likely to apply simple decision

rules instead of profound trade-off decisions, which in turn can result in less precise preference

measurements. As can be seen from Table 4, respondents of the VR group show a significantly

higher degree of certainty in their decision of placing the different product concepts in the

conjoint task than the text group (p .1) and the picture group (p .1). Contrary, the text

and picture groups almost experienced the same degree of certainty. This finding partially

stands in contrast to the results found by Jaeger et al. (2001), who could not attest any

significant differences between groups of presentation styles.

Toubia et al. (2012, p. 138) point out that “one key limitation of preference measurement

methods such as conjoint analysis is the potential lack of motivation experienced by

respondents”. To avoid lack of motivation, conjoint tasks should be enjoyable and diverse

(Helm et al. 2003; Cerjak et al. 2010; Steiner 2007). As shown in Table 4, groups do indeed

differ, even at the 1 %-significance level. The same conclusion is drawn for the picture and VR

groups (p .01). Similarly, respondents from the VR group perceived the conjoint task as

more diverse than the text group (p .01) and the picture group (p .01). However, for both

measures, no differences can be attested between the text and picture groups. It can be

concluded that when respondents are exposed to virtual reality-based product concepts, they

have more fun and perceive the task as more diverse. It is an important finding, as marketing

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

62

scholars have tried to increase respondents' level of enjoyment during a conjoint task in order

to optimize preference measurement results (Toubia 2012).

Respondents' enjoyment can be affected negatively, though, if the time it takes to conduct the

conjoint task is perceived as being too long (Ernst 2001; Day 1975; McDaniel et al. 1985}. To

account for this, the objective interview time it took respondents to conduct the conjoint task

is compared to the perceived interview time. As shown in Table 4, the actual time it took for

the text group to complete the conjoint task was 419 seconds. The picture group completed

the task in 451 seconds, and the conjoint task took respondents of the VR group an average

of 713 seconds to complete. The mean7 difference between the VR group and the text group

on the one hand, and VR group and the picture group, on the other hand, is significant at the

1 %-significance level. This finding is not surprising, as watching a virtual reality product

concept (i.e. the automotive headlight adjustment process) already takes some time.

What is indeed surprising, however, is what happens when additionally taking into

consideration the perceived interview length across groups: the results indicate that the group

presented with virtual reality-based product concepts perceived the interview as being

significantly shorter than the group that had only textual descriptions (p .1). In other words,

even though it objectively took respondents of the VR group 70 % longer to conduct the

conjoint task than respondents from the text group, they perceived the task as being

significantly shorter. The only aspect for which no significant differences are found across the

three groups was the amount respondents would need to be paid in order to take part in a

similar study in the future again.

4. Discussion and Implications for Further

Research This study aims to compare the effect on preference measurement results in a conjoint analysis

and validity criteria for respondents exposed to text, picture , and virtual reality-based

presentation styles.

First of all, the comparison of part-worth utilities across all three groups attests that for two

part-worth utilities, significant differences exist between the text and VR and between the

picture and VR groups.

Secondly, comparing the relative importance weights that respondents attach to each attribute

emphasizes that respondents in each group focus on different attributes. While the text and

picture group and the picture and VR group differ for two out of four attributes, the text and

VR group differ significantly for three out of four product attributes. It is interesting to note

that the relative importance of the price decreases as the presentation form moves towards

the more realistic end of the continuum. The price attribute becomes less important when

moving from text to picture and from picture to VR-based presentation forms. One possible

interpretation of this tendency is that respondents from the text group had no clear

understanding of the product and its features. Instead, when ranking different cards, they

relied on price as the decisive factor, as – in contrast to all other attributes – a lower price is

better than a higher price, due to its interval scale (Stevens 1946). By contrast, groups that

had a picture or a virtual reality as their information source were better able to capture the

functionality of the other attributes and relied less on price. An ordinary least square regression

is conducted between the relative importance of the price attribute and respondents' product

7 A robustness-check is conducted, where median instead of average values of the actual interview times

are compared to avoid that extreme outliers falsify the effect. The same results were found, thereby

supporting the conclusion as stated in the text.

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

63

understanding, with product understanding8, as the independent and relative importance of

the price as the dependent variable. The resulting beta-coefficient is negative (𝛽 = -.18) and

significant at the 1 %-level, which supports the conjecture that the relative importance of price

decreases as respondents' understanding of the product increases.

Shedding light on the question of which presentation style leads to superior results was

subsequently looked at. This is accomplished by comparing results between groups concerning

internal validity, predictive validity and different applicability measures. In particular, the

adjusted R2 for respondents of the VR group is higher than for the text group. However, no

significant differences are found regarding the additionally tested correlation coefficients, such

as Pearson, Spearman and Kendall's Tau. Thus, based on internal validity, only weak

conclusions can be drawn as to which presentation style shows outperforming results, a finding

also made by previous researchers. The same line of argumentation holds for predictive validity

between groups. Here, significant differences are only attested between the picture and the

VR group concerning to the first-second-choice hit rate, where the VR group outperformed

respondents from the picture group.

Finally, additional applicability measures were assessed to find an answer to the second

research question. As depicted in Table 4, the VR group showed significantly better results for

eight out of nine applicability measures than the text group and six out of nine than the picture

group. For instance, not only did respondents from this group perceive the presented stimuli

as more realistic; they also were certain of themselves when making their decisions than

respondents from either of the other groups. Furthermore, they had fewer difficulties in ranking

the product concepts, and they enjoyed the conjoint task more than respondents from the text

and picture group. Finally, even though it took respondents from the VR group longer to

conduct the conjoint task, they perceived it as less time-consuming than either of the other

groups did. These findings for the applicability measures indicate that conjoint results with

embedded virtual realities outperform both of the other presentation forms for this context. At

the same time, no differences are attested between respondents from the text and picture

group.

4.1 Limitations

Notwithstanding these findings, this study faces some limitations. For instance, only text,

picture, and virtual reality-based product concepts are compared. However, adding physical

prototypes could improve the results, as respondents' actual decision are consulted as a

measure of comparison (Dahan/Srinivasan 2000). As the object of study was in its infancy

stage, no physical prototype was available, which excluded the feasibility of adding physical

prototypes. Also, this study was limited in measuring the effect of varying presentation styles

on the validity of conjoint results only. However, as shown above, before deciding in favor of

or against a certain presentation style, an analysis should be conducted as to how respondents'

product understanding is influenced when exposing them to different presentation forms. A

final limitation is that this study implicitly assumes that respondents are all equal in their

capabilities to process information presented in different modalities. However, the cognitive

theory of multimedia learning (CTML) points out that individuals differ in their abilities to

process verbal, visual and VR-based information (Mayer 2005; Paivio 2013). While some

individuals are better able to deal with textual descriptions, others are more proficient at

dealing with visual information elements. These limitations should be taken into consideration

in further research.

8 Product understanding is measured by multi-item scales and verified by commonly used criteria

(Bergkvist/Rossiter 2007).

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

64

References

Adofu, I., and M. Abula."Domestic debt and the Nigerian economy." Current Research

Journal of Economic Theory 2.1 (2010): 22-26.

Appleyard, D. (1977): Understanding Professional Media: Issues, Theory and Research

Agenda, in Human Behaviour and Environment, ed. by I. Altman and J. F. Wohlwill, New

York, NY: Plenum Press. Crossref

Atuahene-Gima, K. (2005): Resolving the Capability: Rigidity Paradox in New Product

Innovation, Journal of Marketing, 69 (4), 61–83. Crossref

Backhaus, K., B. Erichson, W. Plinke, and R. Weiber (2011): Multivariate Analysemeth-

oden: Eine anwendungsorientierte Einführung, Berlin: Springer, 13 ed.

Backhaus, K., J. Jasper, K. Westhoff, J. Gausemeier, M. Grafe, and J. Stöcklein (2014a):

Virtual Reality Based Conjoint Analysis for Early Customer Integration in Industrial Product

Development, 8th International Conference on Digital Enterprise Technology - DET 2014

Disruptive Innovation in Manufacturing Engineering towards the 4th Industrial Revolution,

25 (0), 61–68.

Backhaus, K., J. Jasper, K. Westhoff, J. Gausemeier, M. Grafe, and J. Stöcklein (2014b):

VR-Basierte Conjoint-Analyse zur frühzeitigen Ermittlung von Kundenpräferenzen, IFF

Wissenschaftstage.

Barczak, G., A. Griffin, and K. B. Kahn (2009): Perspective: Trends and Drivers of Success

in NPD Practices: Results of the 2003 PDMA Best Practices Study, Journal of Product

Innovation Management, 26 (1), 3–23. Crossref

Bergkvist, L. and J. R. Rossiter (2007): The Predictive Validity of Multiple -Item Versus

Single-Item Measures of the Same Constructs, Journal of Marketing Research, 44 (2), 175–

184. Crossref

Bortz, J. (2005): Statistik für Human-und Sozialwissenschaftler, Heidelberg: Springer, 6

ed. Bowerman, B. L. and R. T. O’Connell (2007): Business Statistics in Practice, McGraw-

Hill Irwin, 4 ed.

Brockhoff, K. (1999): Forschung und Entwicklung, München: Oldenbourg. Crossref

Brusch, M. (2009): Präsentation der Stimuli bei der Conjointanalyse, in Conjointanalyse,

ed. by D. Baier and M. Brusch, Berlin: Springer, 83–98.

Carayannis, E., E. Samara, and Y. Bakouros (2015): Introduction to Technological

Innovation, in Innovation and Entrepreneurship, Springer, 1–26.

Cerjak, M., R. Haas, and D. Kovacic (2010): Brand Familiarity and Tasting in Conjoint

Analysis: An Experimental Study with Croatian Beer Consumers, British Food Journal, 112

(6), 561–579. Crossref

Christensen, C. M. (1997): The Innovator’s Dilemma: When New Technologies Cause Great

Firms to Fail, Boston, MA: Harvard Business School Press.

Chuang, F.-M., R. E. Morgan, and M. J. Robson (2015): Customer and Competitor In-

sights, New Product Development Competence, and New Product Creativity: Differential,

Integrative, and Substitution Effects, Journal of Product Innovation Management, 32 (2),

175–182. Crossref

Daecke, J. (2009): Nutzung virtueller Welten zur Kundenintegration in die

Neuproduktentwicklung, Wiesbaden: Springer. Crossref

Dahan, E. and J. R. Hauser (2002): The Virtual Customer, Journal of Product Innovation

Management, 19 (5), 332–353. Crossref

Dahan, E. and V. Srinivasan (2000): The Predictive Power of Internet-Based Product

Concept Testing Using Visual Depiction and Animation, Journal of Product Innovation

Management, 17 (2), 99–109. Crossref

Day, G. S. (1975): The Threats to Marketing Research, Journal of Marketing Research, 12

(4), 462–467. Crossref

De Bont, C. J. P. M. (1992): Consumer Evaluations of Early Product-Concepts, Delft

University, Delft.

Durgee, J. F., G. C. O’Connor, and Veryzer Jr, R. W. (1998): Using Mini-Concepts to Identify

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

65

Opportunities for Really New Product Functions, Journal of Consumer Marketing, 15 (6),

525–543. Crossref

Ernst, H. (2002): Management der Neuproduktentwicklung, in Handbuch

Produktionsmanagement: Strategieentwicklung - Produktplanung - Organisation -

Kontrolle, 2 ed., ed. by S. Albers and A. Herrmann, Wiesbaden: Gabler, 333–358. Crossref

Ernst, O. (2001): Multimediale versus Abstrakte Produktpräsentationsformen bei der

Adaptiven Conjoint-Analyse: Ein empirischer Vergleich, Frankfurt am Main: Lang.

Everitt, B. (1996): Making Sense of Statistics in Psychology: A Second-Level Course,

Oxford: Oxford University Press.

Field, A. (2009): Discovering Statistics Using SPSS, Los Angeles, CA: Sage, 3 ed.Field, A.

(2013): Discovering Statistics Using IBM SPSS Statistics, Los Angeles, CA: Sage, 4 ed.

Füller, J. and K. Matzler (2007): Virtual Product Experience and Customer Participation —

A Chance for Customer-Centred, Really New Products, Technovation, 27 (6–7), 378–387.

Games, P. A., H. J. Keselman, and J. C. Rogan (1981): Simultaneous Pairwise Multiple

Comparison Procedures for Means when Sample Sizes are Unequal, Psychological Bullet in,

90 (3), 594. Crossref

Gastwirth, J. L., Y. R. Gel, and W. Miao (2009): The Impact of Levene’s Test of Equality of

Variances on Statistical Theory and Practice, Statistical Science, 24 (3), 343–360. Crossref

Gausemeier, J., J. Berssenbrügge, M. Grafe, S. Kahl, and H. Wassmann (2011): Design

and VR/AR-Based Testing of Advanced Mechatronic Systems, in Virtual Reality &

Augmented Reality in Industry, ed. by D. Ma, X. Fan, J. Gausemeier, and M. Grafe, Berlin:

Springer, 1–37.

Gausemeier, J., P. Ebbesmeyer, and F. Kallmeyer (2001): Produktinnovation: Strategische

Planung und Entwicklung der Produkte von Morgen, München: Hanser.

Goldenberg, J., D. R. Lehmann, and D. Mazursky (1999): The Primacy of the Idea Itself as

a Predictor of New Product Success, Marketing Science Institute, 99–110.

Green, P. E. and V. Srinivasan (1978): Conjoint Analysis in Consumer Research: Issues

and Outlook, Journal of Consumer Research, 5 (2), 103–123. Crossref

Griffin, A. and J. R. Hauser (1993): The Voice of the Customer, Marketing Science, 12 (1),

1–27. Crossref

Helm, R., L. Manthey, A. Scholl, and M. Steiner (2003): Empirical Evaluation of Preference

Elicitation Techniques from Marketing and Decision Analysis, Friedrich-Schiller-Universität

Jena, Jena.

Helm, R., A. Scholl, L. Manthey, and M. Steiner (2004): Measuring Customer Preferences

in New Product Development: Comparing Compositional and Decompositional Methods,

International Journal of Product Development, 1 (1), 12–29. Crossref

Helm, R. and M. Steiner (2008): Präferenzmessung: Methodengestützte Entwicklung

zielgruppenspezifischer Produktinnovationen, Stuttgart: Kohlhammer.

Herstatt, C. and E. von Hippel (1992): From Experience: Developing New Product Concepts

via the Lead User Method: A Case Study in a “Low-Tech” Field, Journal of Product

Innovation Management, 9 (3), 213–221. Crossref

Hill, C. W. L., G. R. Jones, and M. A. Schilling (2014): Strategic Management: An Integrated

Approach, Stamford, CT: Cengage Learning, 11 ed.

Hillig, T. (2006): Verfahrensvarianten der Conjoint-Analyse zur Prognose von

Kaufentscheidungen: Eine Monte-Carlo-Simulation, Wiesbaden: Deutscher Universitäts-

Verlag.

Hoeffler, S. (2003): Measuring Preferences for Really New Products, Journal of Marketing

Research, 40 (4), 406–420. Crossref

Hoeffler, S. and D. Ariely (1999): Constructing Stable Preferences: A Look Into Dimensions

of Experience and Their Impact on Preference Stability, Journal of Consumer Psychology,

8 (2), 113–139. Crossref

Holbrook, M. B. and W. L. Moore (1981): Feature Interactions in Consumer Judgments of

Verbal versus Pictorial Presentations, Journal of Consumer Research, 8 (1), 103–113.

Crossref

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

66

Jaeger, S. R., D. Hedderley, and H. J. H. MacFie (2001): Methodological Issues in Con- joint

Analysis: A Case Study, European Journal of Marketing, 35 (11-12), 1217–1239. Crossref

Kim, C., C. Lee, M. R. Lehto, and M. H. Yun (2011): Evaluation of Customer Impressions

Using Virtual Prototypes in the Internet Environment, International Journal of Industrial

Ergonomics, 41 (2), 118–127. Crossref

Mayer, R. E. (2005): Cognitive Theory of Multimedia Learning, in The Cambridge Handbook

of Multimedia Learning, ed. by R. E. Mayer, Cambridge: Cambridge University Press, 31–

48. Crossref

McDaniel, S. W., P. Verille, and C. S. Madden (1985): The Threats to Marketing Re- search:

An Empirical Reappraisal, Journal of Marketing Research, 22 (1), 74–80. Crossref

McLaughlin, M. J. and K. L. Sainani (2014): Bonferroni, Holm, and Hochberg Corrections:

Fun Names, Serious Changes to P Values, PM&R, 6 (6), 544–546. Crossref

Meffert, H., C. Burmann, and M. Kirchgeorg (2012): Marketing: Grundlagen

Marktorientierter Unternehmensführung, Wiesbaden: Gabler, 11 ed.

Narver, J. C., S. F. Slater, and D. L. MacLachlan (2004): Responsive and Proactive Market

Orientation and New–Product Success, Journal of Product Innovation Management, 21 (5),

334–347. Crossref

Ogawa, S. and F. T. Piller (2006): Reducing the Risks of New Product Development, MIT

Sloan Management Review, 47 (2), 65.

Orme, B. K. (2006): Getting Started With Conjoint Analysis: Strategies for Product Design

and Pricing Research, Madison, WI: Research Publishers LLC, 2 ed.

Paivio, A. (2013): Imagery and Verbal Processes, Hillsdale, NJ: Psychology Press.Porter,

M. E. (2011): Competitive Advantage of Nations: Creating and Sustaining Superior

Performance, New York, NY: Simon and Schuster.

Rammer, C., B. Aschhoff, D. Crass, T. Doherr, M. Hud, C. Ko ̈ hler, B. Peters, T. Schubert,

and F. Schwiebacher (2015): Innovationsverhalten der deutschen Wirtschaft:

Indikatorenbericht zur Innovationserhebung 2014, Zentrum für Europäische

Wirtschaftsforschung, Mannheim.

Schmidt, T. (2001): Einsatzfelder für Multimediale Akzeptanzmessungen bei Technischen

Innovationen: Theoretischer Erklärungsansatz und Empirische Überprüfung, Münster: Lit.

Schrage, M. (1993): The Culture(s) of Prototyping, Design Management Journal, 4 (1), 55–

65.

Sheskin, D. J. (2011): Handbook of Parametric and Nonparametric Statistical Procedures,

Boca Raton: Chapman & Hall, 5 ed.

Shocker, A. D. and W. G. Hall (1986): Pretest Market Models: A Critical Evaluation, Journal

of Product Innovation Management, 3 (2), 86–107. Crossref

Sichtmann, C. and S. Stingel (2007): Limit Conjoint Analys is and Vickrey Auction as

Methods to Elicit Consumers’ Willingness-to-Pay: An Empirical Comparison, European

Journal of Marketing, 41 (11-12), 1359–1374. Crossref

Söderman, M. (2005): Virtual Reality in Product Evaluations with Potential Customers: An

Exploratory Study Comparing Virtual Reality with Conventional Product Representations,

Journal of Engineering Design, 16 (3), 311–328. Crossref

Spiegler, R. (2011): Bounded Rationality and Industrial Organization, New York, NY: Oxford

University Press. Crossref

Srinivasan, V., W. S. Lovejoy, and D. Beach (1997): Integrated Product Design for

Marketability and Manufacturing, Journal of Marketing Research, 34 (1), 154–163. Crossref

Stadie, E. (1998): Medial gestützte Limit-Conjoint-Analyse als Innovationstest für

Technologische Basisinnovationen: Eine explorative Analyse, Münster: Lit.

Steiner, M. (2007): Nachfragerorientierte Präferenzmessung, Wiesbaden: Springer.

Stevens, J. (2007): Intermediate Statistics: A Modern Approach, New York, NY: Lawrence

Erlbaum Associates, 3 ed.

Stevens, S. S. (1946): On the Theory of Scales of Measurement, Science, 103 (2684), 677–

680. Crossref

Toubia, O. (2010): New Product Development, in Handbook of Technology Management, 2

Jonas Jasper

Virtual Reality-Based Product Representations in Conjoint Analysis

67

ed., ed. by H. Bidgoli, Hoboken, NJ: Wiley, 953–963.

Toubia, O., de Jong, Martijn G., D. Stieger, and J. Fu ̈ller (2012): Measuring Consumer

Preferences Using Conjoint Poker, Marketing Science, 31 (1), 138–156. Crossref

Ulwick, A. W. (2002): Turn Customer Input into Innovation, Harvard Business Review, 80

(1), 91–98.

Urban, G. L., J. R. Hauser, W. J. Qualls, B. D. Weinberg, J. D. Bohlmann, and R. A. Chicos

(1997): Information Acceleration: Validation and Lessons from the Field, Journal of

Marketing Research, 34 (1), 143–153. Crossref

van den Hende, E. A. and J. P. L. Schoormans (2012): The Story is as Good as the Real

Thing: Early Customer Input on Product Applications of Radically New Technologies, Journal

of Product Innovation Management, 29 (4), 655–666. Crossref

van Kleef, E., van Trijp, Hans C. M., and P. Luning (2005): Consumer Research in the Early

Stages of New Product Development: A Critical Review of Methods and Techniques, Food

Quality and Preference, 16 (3), 181–201. Crossref

Wittink, D. R., M. Vriens, and W. Burhenne (1994): Commercial Use of Conjoint Analysis

in Europe: Results and Critical Reflections, International Journal of Research in Marketing,

11 (1), 41–52. Crossref

Ye, J., S. Badiyani, V. Raja, and T. Schlegel (2007): Applications of Virtual Reality in Product

Design Evaluation, in Human-Computer Interaction: HCI Applications and Services, ed. by

J. A. Jacko, New York, NY: Springer, 1190–1199. Crossref

Zahn, E., ed. (1995): Handbuch Technologiemanagement, Stuttgart: Schäffer-Poeschel.