supporting complex group decisions: a probabilistic multi-dimensional scaling approach
Post on 26-Jun-2016
215 Views
Preview:
TRANSCRIPT
Pergamon Mathl. Comput. Modelhg Vol. 21, No. 12, pp. 25-33, 1995
Copyright@1995 Efsevier Science Ltd Printed in Great Britain. All rights reserved
0895-7177(95)00089-5 0895-7177/95 $9.50 + 0.00
Supporting Complex Group Decisions: A Probabilistic Multi-Dimensional
Scaling Approach
R. F. EASLEY Dep~tment of ~~~nagement, College of Business Administratjon
University of Notre Dame, Notre Dame, IN 46556, U.S.A.
D. B. MACKAY Department of Marketing, School of Business
Indiana University, Bloomington, IN 47405, U.S.A.
Abstract-Groups often face complex decisions; decisions in which the decision alternatives are not clearly defined and the criteria for choosing an alternative are subject to dispute within the group. We present a Group Decision Support System that will use judgments from the group to visualize the decision problem in a probabilistic geometric space. In this geometric representation, actual decision alternatives and an ideal alternative-an artificial alternative that identifies the ideal solution to the group’s decision dilemma-are portrayed as distributions in a multi-dimensional space. Dispersions of the distributions measure the uncertainties of the decision process. The psychometric theory used to develop the probabilistic geometric repr~entation is described. Preliminary research is presented which demonstrates that geometric representations of this type hefp groups both to understand better the decision they face and to find better solutions.
Keywords-Probabilistic multi-dimensional scaling, Group decision support systems, Thurstone’s Law of Comparative Judgment, Ratio of quadratic forms distribution.
1. INTRODUCTION
Group decision making has been studied for many years. The Management Science literature has seen an extensive development of Multicriteria Decision h4aking methods, such as Multi- ple Objective Linear Programming, which have been extended to model group decisions (e.g., [l---3]). Multi-Attribute Utility Theory and the Analytic Hierarchy Process [4] have also engen- dered numerous approaches to decision modeling and have been extended to group situations. The Information Systems field has witnessed an explosion of research and development in Group Decision Support Systems: research and development that has often made use of models devel- oped in other disciplines.
One body of research that has yet to be tapped in this effort is the extensive psychometric theory concerned with modeling the process of comparative judgment, the basis of choice behav- ior that underlies most of these models. Thurstone’s Law of comparative Judgment [5], which posits that comparisons are made on the basis of draws from probability distributions represent- ing the decision alternatives on a uni-dimensional continuum, is perhaps the most established comparative judgment theory in psychometrics. Thurstone’s model has been reformulated to include ideal points IS], which represent subjects’ ideal decision alternatives, and extended to a multi-dimensional space 171. Probabilistic multi-dimensional scaling (MDS) methods based on this theory have been used in marketing, for example, to develop perceptual maps of products,
This research is based upon work supported by the National Science Foundation under Grant SES-9207765.
Typeset by &,&-‘&$
25
26 R. F. EASLEY AND D. B. MACKAY
in which multi-dimensional distributions are used to represent real products and ideal products.
The expected distance between a distribution representing a real product and a distribution rep-
resenting an ideal product is an inverse measure of consumers’ preference or utility for the real
product. The resulting product maps are used not by consumers but by companies, for example,
to position or reposition their products.
The inspiration for our Group Decision Support System (GDSS) is the idea that a group, while
making a decision, will benefit from being shown this type of geometric representation of the way
they perceive their choice problem. From initial studies, it appears that the visualization of
the group’s decision process will provide useful information and insight to the group members.
Our goal is to develop an interactive GDSS that will first obtain preference judgments (and
optionally, dissimilarity judgments) from group members and then estimate and display the
geometric representation which best captures the data obtained. This representation will consist
of a screen display of a multi-dimensional configuration in which the expected values of the
group’s ideal point and of the decision alternatives are plotted, along with the variance structure
of each point.
It appears, on the basis of preliminary research, that these representations have great potential
for increasing the group’s understanding of its own decision processes, for leading to better
decisions, and for providing greater understanding of the decisions reached. In this paper, we
first describe the theoretical development of the psychometric model that underlies our approach,
and then provide a brief overview of the proposed GDSS. We then present a case study that
illustrates in greater detail the workings of the GDSS and highlights some potential benefits of
its use.
2. DEVELOPMENT OF THE PROBABILISTIC MDS MODEL
In developing this GDSS, we have chosen to apply a probabilistic MDS model that has its roots
in mathematical psychology. We propose to use this approach to provide members of a group,
facing a discrete choice, with some insight into their own cognitive processes. In this section, we
review the development of this probabilistic MDS model.
A reasonable place to begin this account is with Thurstone’s Law of Comparative Judgment 151.
This theory is based on the assumption that a subject, when asked to express a choice for two
stimuli (Sr and Sz), compares the affective values of two random variables. In other words, if Xr
is the value drawn from the distribution of Sr, and likewise Xz for Ss, Sr is chosen over 5’s
if X1 > X2. Thurstone was not dogmatic about the type of distribution involved, though the
normal distribution has most commonly been used. He did distinguish, however, between various
cases based on distributional assumptions. His case I model, the most general expression of his
theory, was
p( - pj = Zij (ff” + fl: - 2rijaiQj) 1’21 (1)
where zij is the normal deviate corresponding to the proportion of times Si is judged to domi-
nate Sj, and rij is the correlation between Xi and Xj. His case III model is more constrained,
with the covariances rijoigj assumed to be equal. His case V model is further constrained, with
the random variables X, assumed to be independent and identically distributed.
The model used in this GDSS is based on Thurstone’s case III model, with all covariances
assumed to be zero. Though Thurstone developed his model to account for variance in sensory
judgments, it has been applied to many types of judgments where uncertainty exists about the
decision alternatives being compared. This uncertainty could be due to unfamiliarity with the
decision alternatives, and, for group judgments, to disagreement about the perception of the
decision alternatives. The judgments can be of many different types. In our case, we are most
interested in preference ratio judgments, judgments where a subject expresses not just a choice
between two alternatives, but a ratio indicating the strength of preference. Ratio judgments have
the advantage of being scale independent, facilitating the development of models that combine
Complex Group Decisions 27
ratio judgments with other types of judgments, such as dissimilarity judgments. For a succinct
description of Thurstone’s Theory, see [8, v9, pp. 237-2411.
The probabilistic uni-dimensional model of Thurstone was extended by Coombs, Greenburg
and Zinnes [6] to form a probabilistic, uni-dimensional unfolding model in which subjects and
decision alternatives are both represented as distributions in the same uni-dimensional space.
The subjects’ points are referred to as ideal points, representing the ideal alternatives of the
subjects, and the distances between the expected ideal point locations and the expected decision
alternative locations determine the preferences of the subjects, that is, the smaller the distance
between an ideal point and an alternative, the more desirable is that alternative to the subject.
Zinnes and Griggs [7], making use of Hefner’s [9] modeling of dissimilarity data, then extended
these developments to a multi-dimensional space. Later, Zinnes and MacKay [lo] generalized
the probabilistic multi-dimensional unfoiding model so that distinct variances could be estimated
for each point in an isotropic space. MacKay [ll] extended the model to an anisotropic space,
allowing distinct variances to be estimated on each dimension, and allowing the coordinates of
each point to be correlated. These models assumed independent sampling, where two samples
are drawn from the ideal point distribution, one for each of the two stimuli being considered. A
recent important extension [12] allows dependent sampling, where the ideal point distribution
is sampled only once when making a preference ratio judgment for two stimuli. Dependent
sampling seems to be a more plausible process when the stimuli are presented simliltaneously
to a subject, as they are in our GDSS. Unlike independent sampling, the dependent sampling
assumption makes it possible to estimate separately the variance structure of both stimuli and
ideal points. MacKay and Zinnes ]12] d emonstrate that sampling assumptions can have clear
effects on empirical results. The issue of dependent versus independent sampling is discussed at
greater length by Zinnes and Griggs [7].
The data we obtain from the group members consists of a set of preference ratio judgments
{rijk} = {dik/dij) for i = 1,. . . ,m subjects, and j, K = 1,. . . ,n stimuli, where the squared
distance between ideal point i and stimulus j is given by
d2”3 = 5 (q - Sj# 1 (2) 1=1
The set of preference ratios (rijk} is not required to be complete. If many alternatives are under
consideration, some comparisons may be skipped. Alternatively~ to increase accuracy, some pairs
may be replicated. The goal is to find, for both the stimuli and the group ideal point, good
estimates of the p-dimensional vectors of coordinates ~1,. . . , pa+1 and the p-by-p covariance
matrices Ci , . . . , &+I.
Continuing with the formulation of MacKay and Zinnes [12], we define dikn as the difference
between points i (the ideal point) and k on dimension n, then let
we can then write the preference ratio r@k as
where
2 d:k X’AX ‘Qk = c = XtBX * (4)
(5)
The v~ianc~covariance matrix Cijk depends on the sampling method used. For independent
samples,
(64
28 R. F. EASLEY AND D. B. MACKAY
while for dependent samples,
To obtain maximum likelihood estimates of the location and variance parameters, it is necessary
to derive the density function of rijk. Direct calculation of this density function is not practical
at present. However, it may be derived from the distribution function
F(?-‘)=P[g$e] = P (X’ (A - Br2) X) 5 0,
=P(X’GX)IO = P[-@~;~o] 3
(7)
where
G=A-Br2 (8)
w = V’L_‘X (9)
and
ci.ji,, = LL’. (10)
The variance-covariance matrix Eijk is defined according to the sampling method used, (6a)
or (6b), while V and H = Diag(hi,. . . , hzp) are the eigenvectors and eigenvalues, respectively,
of L’GL. The random vector W has the multivariate normal distribution N(hW, I) where
(11)
Thus, the distribution function of the ratio r2 can be expressed as an indefinite quadratic form
with coefficient matrix G. This can then be estimated, assuming unique eigenvalues, by following
the method of Imhof [13]. The density function of r2 is numerically estimated by taking central
differences of the distribution function. Differentiating to get the density function of T, the
likelihood function of this unfolding model is
L = n 2rijk f ($jk) (12)
Ennis and Johnson [14] have shown that the indefinite quadratic form used here can be ex-
pressed as a weighted doubly infinite sum of central F distribution functions. They do not,
however, report a computational advantage over the Imhof approach. Faster methods of estimat-
ing the density function of r are currently being investigated by the authors.
In [15], a model is presented which allows estimation of a single ideal point for a group of
subjects, using only preference ratio data. This development makes possible the development
of a probabilistic MDS model of group decision-making based on preference (and optionally
dissimilarity) data obtained from the group members.
Thurstone suggested that the variance structure made explicit with his approach could be
useful in modeling groups: “In the measurement of social attitudes of a group it is not only
the average affective value of a proposal or idea that is of significance but also the dispersion of
affective values within the group” [16, p. 2481. By explicitly modeling the means and variances
of the decision alternatives and the group’s ideal point, we hope to make use of Thurstone’s
insight and develop a GDSS that successfully disentangles the dispersion in group evaluations-
dispersion that may differ from alternative to alternative and attribute to attribute-from the
consensus of how the decision alternatives differ and what the ideal decision alternative is.
Complex Group Decisions
Figure I. Flowchart of the GUSS process.
29
3. A BRIEF OVERVIEW OF THE PROPOSED GDSS
The GDSS we are developing is described m the flowchart m Figure 1 and in [17]. Given a set of n decision alternatives loaded in the program as visual images or descriptions, in Step (I), preference ratio judgments are collected from the group members. A full set is obtained if the n(n - I)/2 judgments per group member is not judged to he too many, otherwise a partial set is obtained. This step involves the first of two user-int,erfaces, which are described in greater
detail below. In Step (a), the data are anafyzed and a geometric representation, or configuration, is estimated by maximum likelihood methods. This configuration specifies the locations of the alternatives and an ideal alternative for the group, as well as a variance structure for each. In Step (3), the group is shown t,his result, using graphical images such as those shown below. In Step (4), th e estimated configuration is discussed by the group, focusing, for example, on interpretation of dimensions, distances anti variance Irlagnit~ldes~ a,nd any implications of the above for the group decision. If a decision can be made, this is done and the process ends. as in Step (5). If the discussion leads to further clarification of some points or affects decision maker judgments, the process is repeated, as in Step (6).
4. A PRELIMINARY CASE STUDY
This section is devoted to describing a preliminary cast study. and to further elaborating
the GDSS, specifically those parts involved in Steps 1-J of the GDSS as described in Figure 1. In this case, only one cycle of the GDSS was Ijerformed, so discussion of Steps 5 and 6 is minimal.
The Otis Elevator case study involved a group of four people responsible for choosing one sheet metal shear from a set of five machines they had carefully evaluated. The machines were manufactured by the companies: Accurshear, Cincinnati. Niagara: Standard and Wysong. The group was composed of an industrial engineer from the office, an engineer from t,he shop floor, a supervisor, and a machine operator. The? latter two had never used a computer, but had no trouble with the data entry program. Each group member was individually interviewed to collect a full set of preference judgments for t,he five machines, and to discuss the decision. In the final GDSS, these judgments would be collected simultaneously from the group members.
In order to assure that group members understood the na,ture of the judgments they were asked to make in Step (l), the dat,a collection program started with a training session. In Figure 2, you see the standard data entry screen, eliciting in this case a prelimiilary training judgment. Throughout the program, only the mouse was used, to click on either the left or right frame to make judgments, or to click on buttons as directed. In Figure 2, the subject here has clicked on the image of the taller rectangle until the numeric ratio and the ratio indicated by the bars connecting the images are in agreement with the actual ratio of the heights of the two rectangles pictured. The frame around the left image indicates simply that it is t,he one with the higher value.
30 R. F. EASLEY AND D. B. MACKAY
The training program moves from simple exercises like this one to more ambiguous judgments, and finally to purely subjective judgments.
r Height Ratio ,
Which column is trller, and by how much?
Prsrs Enter to record your answer
I . ,,,,, .._. _i ,., __, ,I __ _.._ _.._ ._..” ..,, Figure 2. Data entry screen, initial training for preference ratio judgments.
To estimate the configurations (Step (2), Figure 1) the data were taken off-site. The output
was discussed later with the group. In the actual GDSS, the configuration would be estimated
on-site in real time. The four lower-half matrices in Table 1 contain the preference data obtained
from the Otis subjects. The preference ratios are read as columnrow, so the first subject, for
example, preferred the Cincinnati to the Standard by 4:1, and preferred it to the Wysong by 61.
The configuration itself is determined interpreting these same data as distance judgments, which
are read row:column. Thus the first subject would have Standard 4 times farther from his ideal
point than Cincinnati, and Wysong 6 times farther than Cincinnati.
Table 1. Raw preference ratio data for Otis sheet-metal shear decision.
A C N S W ACNSW ACNSW A C N S W
A
N 3 3 4 8 34 4 5 5
S 3 4 5 24 6 0.4 5 5 1 5 5 $
W 3 6 + 1 2 6 $ 4 4 3 4 1 5 5 $ +
In this case, we also collected scale-valued di~imilarity data. The four group members were
asked to pick the most dissimifar pair of machines from the ten possible pairs. The pair chosen
was automatically placed at the top of a IO-point scale. The group members were then asked
to drag and drop the rest of the pairs onto the same dissimilarity scale. They did not appear
to have trouble doing this, and a few spent some time carefully rearranging the pairs until they
were satisfied. These judgments, unlike the scale-independent preference ratio judgments, are
made on a common scale and can be easily compared to each other, thus helping to anchor the
configuration.
A standard deterministic MDS method [18] was used to obtain an initial estimate of the
locations of the five machines and an ideal point in a two-dimensional space. Then, the variance
in the data was distributed over the points, in a manner that does not distinguish dimensional
variances. The initial configuration calculated for the Otis data, with the variances constrained
to be the same for the five machines, is shown in Figure 3. The brackets are added here for illustration; they are not part of the configuration.
Complex Group Decisions
Figure 3. Initial configuration for the Otis data, variances constrained to be identi- cal. Letters indicate: Ideal choice, Accurshear, Cincinnati, Niagara, Standard, and Wysong.
31
The likelihood of this initial configuration was then calculated by extracting the parameter
estimates for the density function of each ratio of distances. The brackets in Figure 3 demarcate, for example, the dimensional differences between each pair of points involved in the ratio of the
distances between Accurshear and the ideal point and between Cincinnati and the ideal point.
The variances of these differences can be obtained by adding the standard deviations of each end
point. With these parameter estimates, we then obtained an estimate of the density function for
the distance judgments.
N I xx 0 C
W A
s
Figure 4. Configuration after one iteration of variance structure estimation.
The likelihood score was obtained by taking the product of all such density function values, or
by summing their logs. The log likelihood score of this configuration was approximately -250. The log likelihood function can be used as an objective function for standard optimization pro-
cedures. In the Otis trial, a grid-search method [19] was used to attempt to find a configuration with maximum likelihood. Following an Alternating Maximum Likelihood method [12], the vari- ance estimates were changed first in an effort to improve the likelihood score. The configuration
shown in Figure 4 was obtained after one cycle of the grid search method. The likelihood was improved to -198 by changing the variance estimates as shown. Note that the major axis of
32 R. F. EASLEY AND D. B. MACKAY
variance for the group’s ideal point had emerged at this point, and remained throughout the analysis. When convergence was reached in this variance optimization phase, another ensued in
which the locations were moved while the variances were fixed. The two phases were repeated
until no significant improvement in likelihood was achieved, and then a final phase followed in
which all variables were free.
In this manner, three visualizations were developed for presentation to the group, as part of
Step (3) of Figure 1. One of the three is shown in Figure 5. All three configurations were nearly
identical in terms of the estimated locations of the means of each distribution. Of the two not
shown here, one had the variances of each decision alternative’s distribution constrained to be
the same, and the other had a separate ideal point distribution estimated for each group member.
The insights from the discussion can all be understood in reference to Figure 5, so the others are
not shown here.
Figure 5. Otis group preference mapping with no constraints on variance structure.
The design of the geometric representation interface is of particular concern, since the best
method of conveying the meaning of a multivariate normal distribution to a nontechnical group
is not obvious. In these figures shown to the Otis group, the outermost ellipse is one standard
deviation from the mean, and the concentric ellipses within are meant to suggest density rather
than display it exactly. This display may be modified in the GDSS, but it appears to be effective,
since some insights offered by the group members clearly relied on their proper understanding of
the relative magnitude of the different variances portrayed.
In discussing the configurations (Step (4) of Figure l), it was clear that all of the group members
thought the Cincinnati was the best quality shear, with the Accurshear a close second. It was a
simple matter for them to determine that the horizontal axis of this configuration corresponded
to overall quality of the machines. It was interesting to note that the ideal point was not at the
extreme right. It turns out there was a simple reason for this: the Cincinnati was about 40%
more expensive than the Accurshear, and both of the engineers, who had budget responsibility,
wanted to get the Accurshear. The machine operator wanted the Cincinnati, and the supervisor
was between them. This was the major disagreement in the group, and was well captured by the
orientation of the major axis of the variance on the estimated ideal point.
The discussion of the vertical axis was less revealing, but seemed to relate to the age of the
companies, and in that sense to general reputation. Also, the Niagara machine was the only one examined using older, mechanical technology-the other four were hydraulic-which seemed to
account for Niagara’s isolation on the vertical axis.
During the group discussion of differences in magnitude of the variances on the different shears,
the shop floor engineer pointed out that they corresponded to the order in which they visited the
Complex Group Decisions 33
different machines over a six-week period. The Wysong, with the largest variance, was the one
they had seen first, and the Niagara the one they had seen most recently, with the others following
in between. He suggested that they should perhaps spend some time discussing the machines
they had seen earliest, for which there were large variances, to make sure they all remembered
them the same way. This is exactly the sort of insight into the group decision process that the
explicit modeling of variance makes possible. At this point, the group could have had such a
discussion and then repeated the process (as in Step (6) of Figure l), to see what might come
up in the next configuration. As it was, the final, unanimous decision of the group favored the
Accurshear, which is consistent with its ranking based on expected distance from the ideal point.
5. CONCLUSION
We have presented an interactive Group Decision Support System that uses probabilistic,
psychometrically-based geometric representations of the group’s perceptions to help them under-
stand their decision process and reach a decision. Based on a preliminary case study, we are
confident that the proposed GDSS can provide useful insights to groups facing difficult decisions.
It appears to be reasonable to expect group members, even those without a sophisticated under-
standing of probability distributions, to be able to understand and interpret the visual displays.
We are currently developing improvements to the estimation procedures involved. When the
GDSS is completed, further empirical studies will be done t,o assure that this is the case.
REFERENCES
2.
3.
4.
5. 6.
7.
10.
11.
12.
13.
14.
15.
16. 17.
18.
19.
R.E. Steuer, Multiple objective linear programming with interval criterion weights. Management Sczence 23 (3), 305-316 (1976). S. Zionts and J. Wallenius, Interactive multiple objective linear programming, Afanagement Sczence 29 (5), 519-529 (1983). M. Zeleny, An essay into a philosophy of MCDM: A way of thinking or another algorithm?, C’omplLters and Operations Research 19 (7), 563-566 (1992). T.L. Saaty, Group decision making and the AHP, In The Analytrc Hzerarchy Process: Applicatzons and Studies, (Edited by B.L. Golden, E.A. Wasil and P.T. Harker), pp. 59-67, Springer-Verlag. New York, (1989). L.L. Thurstone, A law of comparative judgment, Psychological Reuzew 34, 273-286 (1927). C.H. Coombs, M. Greenberg and J.L. Zinnes, A double law of comparative judgment for t,he analysis of preferential choice and similarities data, Psychometrzka 26, 165-171 (1961). J.L. Zinnes and P.A. Griggs, Probabilistic multidimensional unfolding analysis, Psychometrika 39, 327-350 (1974). Kotz-Johnson, Encyclopedia of Statistzcal Sczence, Vol. 9, John Wiley k Sons, New York, (1988). R.A. Hefner, Extensions of the law of comparative judgment to discrimlnable and multidimensional stimuli, Doctoral Dissertation. University of Michigan, (1958). J.L. Zmnes and D.B. MacKay. Probabilistic multidimensIonal analysis of preference ratlo Judgments. Com- mvnzcation and Cognitzon 20 (l), 17-44 (1987). D.B. MacKay, Probabilistic multidimensional scaling: An anisot,roplc model for distance judgments, Jollrnal of Mathematical Psychology 33 (2), 187-205 (1989). D.B. MacKay and J.L. Zinnes, Probabilistic multidimensional unfolding: An anisotropic model for preference ratio judgments, Working Paper, Indiana liniversity, (1994). J.P. Imhof, Computing the distribution of quadratic forms in normal variables, Bzometriku 48. 419-426 (1961). D.M. Ennis and N.L. Johnson, A general model for preferential and triadic choice m t,erms of central F distribution functions (unpublished working paper), (1992). D.B. MacKay, R.F. Easley and J.L. Zinnes. A single ideal pomt model for market structure analysis. .Joumal of Marketing Research (to appear). L.L. Thurstone, The prediction of choice, Psychometnka 10 (4), 237-253 (1945). R.F. Easley and D.B. MacKay, Visualization of complex group decisions, In 1994 Proceedzngs. Deczsion Science Institute, 1994 Annual Meeting, Honolulu, (1994). J.B. Kruskal, F.W. Young and J.B. Seery, How to Use KYST. a Very Flezzble Program to do Multidimen- sional Scaling and Unfolding, Bell Telephone Labs, Murray Hill, NJ, (1973). J.P. Chandler, STEPIT-Finds local minima of a smooth function of several parameters. Behavzoral Sczence 14, 81-82 (1969).
top related