Pergamon Mathl. Comput. Modelhg Vol. 21, No. 12, pp. 25-33, 1995

Copyright@1995 Efsevier Science Ltd Printed in Great Britain. All rights reserved

0895-7177(95)00089-5 0895-7177/95 $9.50 + 0.00

Supporting Complex Group Decisions: A Probabilistic Multi-Dimensional

Scaling Approach

R. F. EASLEY Dep~tment of ~~~nagement, College of Business Administratjon

University of Notre Dame, Notre Dame, IN 46556, U.S.A.

D. B. MACKAY Department of Marketing, School of Business

Indiana University, Bloomington, IN 47405, U.S.A.

Abstract-Groups often face complex decisions; decisions in which the decision alternatives are not clearly defined and the criteria for choosing an alternative are subject to dispute within the group. We present a Group Decision Support System that will use judgments from the group to visualize the decision problem in a probabilistic geometric space. In this geometric representation, actual decision alternatives and an ideal alternative-an artificial alternative that identifies the ideal solution to the group’s decision dilemma-are portrayed as distributions in a multi-dimensional space. Dispersions of the distributions measure the uncertainties of the decision process. The psychometric theory used to develop the probabilistic geometric repr~entation is described. Preliminary research is presented which demonstrates that geometric representations of this type hefp groups both to understand better the decision they face and to find better solutions.

Keywords-Probabilistic multi-dimensional scaling, Group decision support systems, Thurstone’s Law of Comparative Judgment, Ratio of quadratic forms distribution.

1. INTRODUCTION

Group decision making has been studied for many years. The Management Science literature has seen an extensive development of Multicriteria Decision h4aking methods, such as Multi- ple Objective Linear Programming, which have been extended to model group decisions (e.g., [l---3]). Multi-Attribute Utility Theory and the Analytic Hierarchy Process [4] have also engen- dered numerous approaches to decision modeling and have been extended to group situations. The Information Systems field has witnessed an explosion of research and development in Group Decision Support Systems: research and development that has often made use of models devel- oped in other disciplines.

One body of research that has yet to be tapped in this effort is the extensive psychometric theory concerned with modeling the process of comparative judgment, the basis of choice behav- ior that underlies most of these models. Thurstone’s Law of comparative Judgment [5], which posits that comparisons are made on the basis of draws from probability distributions represent- ing the decision alternatives on a uni-dimensional continuum, is perhaps the most established comparative judgment theory in psychometrics. Thurstone’s model has been reformulated to include ideal points IS], which represent subjects’ ideal decision alternatives, and extended to a multi-dimensional space 171. Probabilistic multi-dimensional scaling (MDS) methods based on this theory have been used in marketing, for example, to develop perceptual maps of products,

This research is based upon work supported by the National Science Foundation under Grant SES-9207765.

Typeset by &,&-‘&$

25

26 R. F. EASLEY AND D. B. MACKAY

in which multi-dimensional distributions are used to represent real products and ideal products.

The expected distance between a distribution representing a real product and a distribution rep-

resenting an ideal product is an inverse measure of consumers’ preference or utility for the real

product. The resulting product maps are used not by consumers but by companies, for example,

to position or reposition their products.

The inspiration for our Group Decision Support System (GDSS) is the idea that a group, while

making a decision, will benefit from being shown this type of geometric representation of the way

they perceive their choice problem. From initial studies, it appears that the visualization of

the group’s decision process will provide useful information and insight to the group members.

Our goal is to develop an interactive GDSS that will first obtain preference judgments (and

optionally, dissimilarity judgments) from group members and then estimate and display the

geometric representation which best captures the data obtained. This representation will consist

of a screen display of a multi-dimensional configuration in which the expected values of the

group’s ideal point and of the decision alternatives are plotted, along with the variance structure

of each point.

It appears, on the basis of preliminary research, that these representations have great potential

for increasing the group’s understanding of its own decision processes, for leading to better

decisions, and for providing greater understanding of the decisions reached. In this paper, we

first describe the theoretical development of the psychometric model that underlies our approach,

and then provide a brief overview of the proposed GDSS. We then present a case study that

illustrates in greater detail the workings of the GDSS and highlights some potential benefits of

its use.

2. DEVELOPMENT OF THE PROBABILISTIC MDS MODEL

In developing this GDSS, we have chosen to apply a probabilistic MDS model that has its roots

in mathematical psychology. We propose to use this approach to provide members of a group,

facing a discrete choice, with some insight into their own cognitive processes. In this section, we

review the development of this probabilistic MDS model.

A reasonable place to begin this account is with Thurstone’s Law of Comparative Judgment 151.

This theory is based on the assumption that a subject, when asked to express a choice for two

stimuli (Sr and Sz), compares the affective values of two random variables. In other words, if Xr

is the value drawn from the distribution of Sr, and likewise Xz for Ss, Sr is chosen over 5’s

if X1 > X2. Thurstone was not dogmatic about the type of distribution involved, though the

normal distribution has most commonly been used. He did distinguish, however, between various

cases based on distributional assumptions. His case I model, the most general expression of his

theory, was

p( - pj = Zij (ff” + fl: - 2rijaiQj) 1’21 (1)

where zij is the normal deviate corresponding to the proportion of times Si is judged to domi-

nate Sj, and rij is the correlation between Xi and Xj. His case III model is more constrained,

with the covariances rijoigj assumed to be equal. His case V model is further constrained, with

the random variables X, assumed to be independent and identically distributed.

The model used in this GDSS is based on Thurstone’s case III model, with all covariances

assumed to be zero. Though Thurstone developed his model to account for variance in sensory

judgments, it has been applied to many types of judgments where uncertainty exists about the

decision alternatives being compared. This uncertainty could be due to unfamiliarity with the

decision alternatives, and, for group judgments, to disagreement about the perception of the

decision alternatives. The judgments can be of many different types. In our case, we are most

interested in preference ratio judgments, judgments where a subject expresses not just a choice

between two alternatives, but a ratio indicating the strength of preference. Ratio judgments have

the advantage of being scale independent, facilitating the development of models that combine

Complex Group Decisions 27

ratio judgments with other types of judgments, such as dissimilarity judgments. For a succinct

description of Thurstone’s Theory, see [8, v9, pp. 237-2411.

The probabilistic uni-dimensional model of Thurstone was extended by Coombs, Greenburg

and Zinnes [6] to form a probabilistic, uni-dimensional unfolding model in which subjects and

decision alternatives are both represented as distributions in the same uni-dimensional space.

The subjects’ points are referred to as ideal points, representing the ideal alternatives of the

subjects, and the distances between the expected ideal point locations and the expected decision

alternative locations determine the preferences of the subjects, that is, the smaller the distance

between an ideal point and an alternative, the more desirable is that alternative to the subject.

Zinnes and Griggs [7], making use of Hefner’s [9] modeling of dissimilarity data, then extended

these developments to a multi-dimensional space. Later, Zinnes and MacKay [lo] generalized

the probabilistic multi-dimensional unfoiding model so that distinct variances could be estimated

for each point in an isotropic space. MacKay [ll] extended the model to an anisotropic space,

allowing distinct variances to be estimated on each dimension, and allowing the coordinates of

each point to be correlated. These models assumed independent sampling, where two samples

are drawn from the ideal point distribution, one for each of the two stimuli being considered. A

recent important extension [12] allows dependent sampling, where the ideal point distribution

is sampled only once when making a preference ratio judgment for two stimuli. Dependent

sampling seems to be a more plausible process when the stimuli are presented simliltaneously

to a subject, as they are in our GDSS. Unlike independent sampling, the dependent sampling

assumption makes it possible to estimate separately the variance structure of both stimuli and

ideal points. MacKay and Zinnes ]12] d emonstrate that sampling assumptions can have clear

effects on empirical results. The issue of dependent versus independent sampling is discussed at

greater length by Zinnes and Griggs [7].

The data we obtain from the group members consists of a set of preference ratio judgments

{rijk} = {dik/dij) for i = 1,. . . ,m subjects, and j, K = 1,. . . ,n stimuli, where the squared

distance between ideal point i and stimulus j is given by

d2”3 = 5 (q - Sj# 1 (2) 1=1

The set of preference ratios (rijk} is not required to be complete. If many alternatives are under

consideration, some comparisons may be skipped. Alternatively~ to increase accuracy, some pairs

may be replicated. The goal is to find, for both the stimuli and the group ideal point, good

estimates of the p-dimensional vectors of coordinates ~1,. . . , pa+1 and the p-by-p covariance

matrices Ci , . . . , &+I.

Continuing with the formulation of MacKay and Zinnes [12], we define dikn as the difference

between points i (the ideal point) and k on dimension n, then let

we can then write the preference ratio r@k as

where

2 d:k X’AX ‘Qk = c = XtBX * (4)

(5)

The v~ianc~covariance matrix Cijk depends on the sampling method used. For independent

samples,

(64

28 R. F. EASLEY AND D. B. MACKAY

while for dependent samples,

To obtain maximum likelihood estimates of the location and variance parameters, it is necessary

to derive the density function of rijk. Direct calculation of this density function is not practical

at present. However, it may be derived from the distribution function

F(?-‘)=P[g$e] = P (X’ (A - Br2) X) 5 0,

=P(X’GX)IO = P[-@~;~o] 3

(7)

where

G=A-Br2 (8)

w = V’L_‘X (9)

and

ci.ji,, = LL’. (10)

The variance-covariance matrix Eijk is defined according to the sampling method used, (6a)

or (6b), while V and H = Diag(hi,. . . , hzp) are the eigenvectors and eigenvalues, respectively,

of L’GL. The random vector W has the multivariate normal distribution N(hW, I) where

(11)

Thus, the distribution function of the ratio r2 can be expressed as an indefinite quadratic form

with coefficient matrix G. This can then be estimated, assuming unique eigenvalues, by following

the method of Imhof [13]. The density function of r2 is numerically estimated by taking central

differences of the distribution function. Differentiating to get the density function of T, the

likelihood function of this unfolding model is

L = n 2rijk f ($jk) (12)

Ennis and Johnson [14] have shown that the indefinite quadratic form used here can be ex-

pressed as a weighted doubly infinite sum of central F distribution functions. They do not,

however, report a computational advantage over the Imhof approach. Faster methods of estimat-

ing the density function of r are currently being investigated by the authors.

In [15], a model is presented which allows estimation of a single ideal point for a group of

subjects, using only preference ratio data. This development makes possible the development

of a probabilistic MDS model of group decision-making based on preference (and optionally

dissimilarity) data obtained from the group members.

Thurstone suggested that the variance structure made explicit with his approach could be

useful in modeling groups: “In the measurement of social attitudes of a group it is not only

the average affective value of a proposal or idea that is of significance but also the dispersion of

affective values within the group” [16, p. 2481. By explicitly modeling the means and variances

of the decision alternatives and the group’s ideal point, we hope to make use of Thurstone’s

insight and develop a GDSS that successfully disentangles the dispersion in group evaluations-

dispersion that may differ from alternative to alternative and attribute to attribute-from the

consensus of how the decision alternatives differ and what the ideal decision alternative is.

Complex Group Decisions

Figure I. Flowchart of the GUSS process.

29

3. A BRIEF OVERVIEW OF THE PROPOSED GDSS

The GDSS we are developing is described m the flowchart m Figure 1 and in [17]. Given a set of n decision alternatives loaded in the program as visual images or descriptions, in Step (I), preference ratio judgments are collected from the group members. A full set is obtained if the n(n - I)/2 judgments per group member is not judged to he too many, otherwise a partial set is obtained. This step involves the first of two user-int,erfaces, which are described in greater

detail below. In Step (a), the data are anafyzed and a geometric representation, or configuration, is estimated by maximum likelihood methods. This configuration specifies the locations of the alternatives and an ideal alternative for the group, as well as a variance structure for each. In Step (3), the group is shown t,his result, using graphical images such as those shown below. In Step (4), th e estimated configuration is discussed by the group, focusing, for example, on interpretation of dimensions, distances anti variance Irlagnit~ldes~ a,nd any implications of the above for the group decision. If a decision can be made, this is done and the process ends. as in Step (5). If the discussion leads to further clarification of some points or affects decision maker judgments, the process is repeated, as in Step (6).

4. A PRELIMINARY CASE STUDY

This section is devoted to describing a preliminary cast study. and to further elaborating

the GDSS, specifically those parts involved in Steps 1-J of the GDSS as described in Figure 1. In this case, only one cycle of the GDSS was Ijerformed, so discussion of Steps 5 and 6 is minimal.

The Otis Elevator case study involved a group of four people responsible for choosing one sheet metal shear from a set of five machines they had carefully evaluated. The machines were manufactured by the companies: Accurshear, Cincinnati. Niagara: Standard and Wysong. The group was composed of an industrial engineer from the office, an engineer from t,he shop floor, a supervisor, and a machine operator. The? latter two had never used a computer, but had no trouble with the data entry program. Each group member was individually interviewed to collect a full set of preference judgments for t,he five machines, and to discuss the decision. In the final GDSS, these judgments would be collected simultaneously from the group members.

In order to assure that group members understood the na,ture of the judgments they were asked to make in Step (l), the dat,a collection program started with a training session. In Figure 2, you see the standard data entry screen, eliciting in this case a prelimiilary training judgment. Throughout the program, only the mouse was used, to click on either the left or right frame to make judgments, or to click on buttons as directed. In Figure 2, the subject here has clicked on the image of the taller rectangle until the numeric ratio and the ratio indicated by the bars connecting the images are in agreement with the actual ratio of the heights of the two rectangles pictured. The frame around the left image indicates simply that it is t,he one with the higher value.

30 R. F. EASLEY AND D. B. MACKAY

The training program moves from simple exercises like this one to more ambiguous judgments, and finally to purely subjective judgments.

r Height Ratio ,

Which column is trller, and by how much?

Prsrs Enter to record your answer

I . ,,,,, .._. _i ,., __, ,I __ _.._ _.._ ._..” ..,, Figure 2. Data entry screen, initial training for preference ratio judgments.

To estimate the configurations (Step (2), Figure 1) the data were taken off-site. The output

was discussed later with the group. In the actual GDSS, the configuration would be estimated

on-site in real time. The four lower-half matrices in Table 1 contain the preference data obtained

from the Otis subjects. The preference ratios are read as columnrow, so the first subject, for

example, preferred the Cincinnati to the Standard by 4:1, and preferred it to the Wysong by 61.

The configuration itself is determined interpreting these same data as distance judgments, which

are read row:column. Thus the first subject would have Standard 4 times farther from his ideal

point than Cincinnati, and Wysong 6 times farther than Cincinnati.

Table 1. Raw preference ratio data for Otis sheet-metal shear decision.

A C N S W ACNSW ACNSW A C N S W

A

N 3 3 4 8 34 4 5 5

S 3 4 5 24 6 0.4 5 5 1 5 5 $

W 3 6 + 1 2 6 $ 4 4 3 4 1 5 5 $ +

In this case, we also collected scale-valued di~imilarity data. The four group members were

asked to pick the most dissimifar pair of machines from the ten possible pairs. The pair chosen

was automatically placed at the top of a IO-point scale. The group members were then asked

to drag and drop the rest of the pairs onto the same dissimilarity scale. They did not appear

to have trouble doing this, and a few spent some time carefully rearranging the pairs until they

were satisfied. These judgments, unlike the scale-independent preference ratio judgments, are

made on a common scale and can be easily compared to each other, thus helping to anchor the

configuration.

A standard deterministic MDS method [18] was used to obtain an initial estimate of the

locations of the five machines and an ideal point in a two-dimensional space. Then, the variance

in the data was distributed over the points, in a manner that does not distinguish dimensional

variances. The initial configuration calculated for the Otis data, with the variances constrained

to be the same for the five machines, is shown in Figure 3. The brackets are added here for illustration; they are not part of the configuration.

Complex Group Decisions

Figure 3. Initial configuration for the Otis data, variances constrained to be identi- cal. Letters indicate: Ideal choice, Accurshear, Cincinnati, Niagara, Standard, and Wysong.

31

The likelihood of this initial configuration was then calculated by extracting the parameter

estimates for the density function of each ratio of distances. The brackets in Figure 3 demarcate, for example, the dimensional differences between each pair of points involved in the ratio of the

distances between Accurshear and the ideal point and between Cincinnati and the ideal point.

The variances of these differences can be obtained by adding the standard deviations of each end

point. With these parameter estimates, we then obtained an estimate of the density function for

the distance judgments.

N I xx 0 C

W A

s

Figure 4. Configuration after one iteration of variance structure estimation.

The likelihood score was obtained by taking the product of all such density function values, or

by summing their logs. The log likelihood score of this configuration was approximately -250. The log likelihood function can be used as an objective function for standard optimization pro-

cedures. In the Otis trial, a grid-search method [19] was used to attempt to find a configuration with maximum likelihood. Following an Alternating Maximum Likelihood method [12], the vari- ance estimates were changed first in an effort to improve the likelihood score. The configuration

shown in Figure 4 was obtained after one cycle of the grid search method. The likelihood was improved to -198 by changing the variance estimates as shown. Note that the major axis of

32 R. F. EASLEY AND D. B. MACKAY

variance for the group’s ideal point had emerged at this point, and remained throughout the analysis. When convergence was reached in this variance optimization phase, another ensued in

which the locations were moved while the variances were fixed. The two phases were repeated

until no significant improvement in likelihood was achieved, and then a final phase followed in

which all variables were free.

In this manner, three visualizations were developed for presentation to the group, as part of

Step (3) of Figure 1. One of the three is shown in Figure 5. All three configurations were nearly

identical in terms of the estimated locations of the means of each distribution. Of the two not

shown here, one had the variances of each decision alternative’s distribution constrained to be

the same, and the other had a separate ideal point distribution estimated for each group member.

The insights from the discussion can all be understood in reference to Figure 5, so the others are

not shown here.

Figure 5. Otis group preference mapping with no constraints on variance structure.

The design of the geometric representation interface is of particular concern, since the best

method of conveying the meaning of a multivariate normal distribution to a nontechnical group

is not obvious. In these figures shown to the Otis group, the outermost ellipse is one standard

deviation from the mean, and the concentric ellipses within are meant to suggest density rather

than display it exactly. This display may be modified in the GDSS, but it appears to be effective,

since some insights offered by the group members clearly relied on their proper understanding of

the relative magnitude of the different variances portrayed.

In discussing the configurations (Step (4) of Figure l), it was clear that all of the group members

thought the Cincinnati was the best quality shear, with the Accurshear a close second. It was a

simple matter for them to determine that the horizontal axis of this configuration corresponded

to overall quality of the machines. It was interesting to note that the ideal point was not at the

extreme right. It turns out there was a simple reason for this: the Cincinnati was about 40%

more expensive than the Accurshear, and both of the engineers, who had budget responsibility,

wanted to get the Accurshear. The machine operator wanted the Cincinnati, and the supervisor

was between them. This was the major disagreement in the group, and was well captured by the

orientation of the major axis of the variance on the estimated ideal point.

The discussion of the vertical axis was less revealing, but seemed to relate to the age of the

companies, and in that sense to general reputation. Also, the Niagara machine was the only one examined using older, mechanical technology-the other four were hydraulic-which seemed to

account for Niagara’s isolation on the vertical axis.

During the group discussion of differences in magnitude of the variances on the different shears,

the shop floor engineer pointed out that they corresponded to the order in which they visited the

Complex Group Decisions 33

different machines over a six-week period. The Wysong, with the largest variance, was the one

they had seen first, and the Niagara the one they had seen most recently, with the others following

in between. He suggested that they should perhaps spend some time discussing the machines

they had seen earliest, for which there were large variances, to make sure they all remembered

them the same way. This is exactly the sort of insight into the group decision process that the

explicit modeling of variance makes possible. At this point, the group could have had such a

discussion and then repeated the process (as in Step (6) of Figure l), to see what might come

up in the next configuration. As it was, the final, unanimous decision of the group favored the

Accurshear, which is consistent with its ranking based on expected distance from the ideal point.

5. CONCLUSION

We have presented an interactive Group Decision Support System that uses probabilistic,

psychometrically-based geometric representations of the group’s perceptions to help them under-

stand their decision process and reach a decision. Based on a preliminary case study, we are

confident that the proposed GDSS can provide useful insights to groups facing difficult decisions.

It appears to be reasonable to expect group members, even those without a sophisticated under-

standing of probability distributions, to be able to understand and interpret the visual displays.

We are currently developing improvements to the estimation procedures involved. When the

GDSS is completed, further empirical studies will be done t,o assure that this is the case.

REFERENCES

2.

3.

4.

5. 6.

7.

10.

11.

12.

13.

14.

15.

16. 17.

18.

19.

R.E. Steuer, Multiple objective linear programming with interval criterion weights. Management Sczence 23 (3), 305-316 (1976). S. Zionts and J. Wallenius, Interactive multiple objective linear programming, Afanagement Sczence 29 (5), 519-529 (1983). M. Zeleny, An essay into a philosophy of MCDM: A way of thinking or another algorithm?, C’omplLters and Operations Research 19 (7), 563-566 (1992). T.L. Saaty, Group decision making and the AHP, In The Analytrc Hzerarchy Process: Applicatzons and Studies, (Edited by B.L. Golden, E.A. Wasil and P.T. Harker), pp. 59-67, Springer-Verlag. New York, (1989). L.L. Thurstone, A law of comparative judgment, Psychological Reuzew 34, 273-286 (1927). C.H. Coombs, M. Greenberg and J.L. Zinnes, A double law of comparative judgment for t,he analysis of preferential choice and similarities data, Psychometrzka 26, 165-171 (1961). J.L. Zinnes and P.A. Griggs, Probabilistic multidimensional unfolding analysis, Psychometrika 39, 327-350 (1974). Kotz-Johnson, Encyclopedia of Statistzcal Sczence, Vol. 9, John Wiley k Sons, New York, (1988). R.A. Hefner, Extensions of the law of comparative judgment to discrimlnable and multidimensional stimuli, Doctoral Dissertation. University of Michigan, (1958). J.L. Zmnes and D.B. MacKay. Probabilistic multidimensIonal analysis of preference ratlo Judgments. Com- mvnzcation and Cognitzon 20 (l), 17-44 (1987). D.B. MacKay, Probabilistic multidimensional scaling: An anisot,roplc model for distance judgments, Jollrnal of Mathematical Psychology 33 (2), 187-205 (1989). D.B. MacKay and J.L. Zinnes, Probabilistic multidimensional unfolding: An anisotropic model for preference ratio judgments, Working Paper, Indiana liniversity, (1994). J.P. Imhof, Computing the distribution of quadratic forms in normal variables, Bzometriku 48. 419-426 (1961). D.M. Ennis and N.L. Johnson, A general model for preferential and triadic choice m t,erms of central F distribution functions (unpublished working paper), (1992). D.B. MacKay, R.F. Easley and J.L. Zinnes. A single ideal pomt model for market structure analysis. .Joumal of Marketing Research (to appear). L.L. Thurstone, The prediction of choice, Psychometnka 10 (4), 237-253 (1945). R.F. Easley and D.B. MacKay, Visualization of complex group decisions, In 1994 Proceedzngs. Deczsion Science Institute, 1994 Annual Meeting, Honolulu, (1994). J.B. Kruskal, F.W. Young and J.B. Seery, How to Use KYST. a Very Flezzble Program to do Multidimen- sional Scaling and Unfolding, Bell Telephone Labs, Murray Hill, NJ, (1973). J.P. Chandler, STEPIT-Finds local minima of a smooth function of several parameters. Behavzoral Sczence 14, 81-82 (1969).