testing transitivity with a true and error model
DESCRIPTION
Testing Transitivity with a True and Error Model. Michael H. Birnbaum California State University, Fullerton. Testing Algebraic Models with Error-Filled Data. Models assume or imply formal properties such as transitivity: If A > B and B > C then A > C - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/1.jpg)
Testing Transitivity with a True and Error Model
Michael H. BirnbaumCalifornia State University,
Fullerton
![Page 2: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/2.jpg)
Testing Algebraic Models with Error-Filled Data
• Models assume or imply formal properties such as transitivity:
If A > B and B > C then A > C• But such properties may not hold if
data contain “error.”• And different people might have
different “true” preferences.
![Page 3: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/3.jpg)
Error Model Assumptions
• Each choice in an experiment has a true choice probability, p, and an error rate, e.
• The error rate is estimated from (and is the “reason” given for) inconsistency of response to the same choice by same person over repetitions
![Page 4: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/4.jpg)
One Choice, Two Repetitions
A B
A
B€
pe2
+ ( 1 − p )( 1 − e )2
p ( 1 − e ) e + ( 1 − p )( 1 − e ) e
p ( 1 − e ) e + ( 1 − p )( 1 − e ) e
€
p ( 1 − e )2
+ ( 1 − p ) e2
![Page 5: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/5.jpg)
Solution for e
• The proportion of preference reversals between repetitions allows an estimate of e.
• Both off-diagonal entries should be equal, and are equal to:
( 1 − e ) e
![Page 6: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/6.jpg)
Ex: Stochastic Dominance
: 05 tickets to win $12
05 tickets to win $14
90 tickets to win $96
B: 10 tickets to win $12
05 tickets to win $90
85 tickets to win $96
122 Undergrads: 59% repeated viols (BB) 28% Preference Reversals (AB or BA) Estimates: e = 0.19; p = 0.85170 Experts: 35% show 2 violations (BB) 31% Reversals (AB or BA) Estimates: e = 0.196; p = 0.50
![Page 7: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/7.jpg)
Testing Higher Properties• Extending this model to properties
relating 2, 3, or 4 choices:• Allow a different error rate on
each choice.• Estimate true probability for each
choice pattern. Different people can have different “true” patterns, which need not be transitive.
![Page 8: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/8.jpg)
New Studies of Transitivity
• Work currently under way testing transitivity under same conditions as used in tests of other decision properties.
• Participants view choices via the WWW, click button beside the gamble they would prefer to play.
![Page 9: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/9.jpg)
Some Recipes being Tested
• Tversky’s (1969) 5 gambles.• LS: Preds of Priority Heuristic• Starmer’s recipe• Additive Difference Model (regret;
majority rule)• Birnbaum, Patton, & Lott (1999) recipe.• Recipes based on Bleichrodt & Schmidt
context-dependent utility models.
![Page 10: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/10.jpg)
Replications of Tversky (1969) with Roman
Gutierez• First two studies used Tversky’s 5
gambles, but formatted with tickets instead of pie charts.
• Two studies with n = 417 and n = 327 with small or large prizes ($4.50 or $450)
• No pre-selection of participants.• Participants served in other risky DM
studies, prior to testing (~1 hr).
![Page 11: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/11.jpg)
Three of Tversky’s (1969) Gambles
• A = ($5.00, 0.29; $0, 0.79)• C = ($4.50, 0.38; $0, 0.62)• E = ($4.00, 0.46; $0, 0.54)Priority Heurisitc Predicts:A preferred to C; C preferred to E, and E preferred to A. Intransitive.
Tversky (1969) reported viol of WST
![Page 12: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/12.jpg)
Response Combinations
Notation (A, B) (B, C) (C, A)
000 A B C *
001 A B A
010 A C C
011 A C A
100 B B C
101 B B A
110 B C C
111 B C A *
![Page 13: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/13.jpg)
Weak Stochastic Transitivity
€
P ( A f B ) = P ( 000 ) + P ( 001 ) + P ( 010 ) + P ( 011 )
P ( B f C ) = P ( 000 ) + P ( 001 ) + P ( 100 ) + P ( 101 )
P ( C f A ) = P ( 000 ) + P ( 010 ) + P ( 100 ) + P ( 110 )
![Page 14: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/14.jpg)
WST Can be Violated even when Everyone is Perfectly
Transitive
€
P ( 001 ) = P ( 010 ) = P ( 100 ) =1
3
€
P ( A f B ) = P ( B f C ) = P ( C f A ) =2
3
![Page 15: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/15.jpg)
Triangle Inequality has similar problems:
• It is possible that everyone is transitive but WST is violated.
• It is possible that people are systematically intransitive and WST is satisfied.
• Possible that everyone is intransitive and triangle inequality is satisfied.
![Page 16: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/16.jpg)
Model for Transitivity
€
P ( 000 ) = p000
( 1 − e1
)( 1 − e2
)( 1 − e3
) + p001
( 1 − e1
)( 1 − e2
) e3
+
+ p010
( 1 − e1
) e2
( 1 − e3
) + p011
( 1 − e1
) e2e
3+
+ p100
e1
( 1 − e2
)( 1 − e3
) + p101
e1
( 1 − e2
) e3
+
+ p110
e1e
2( 1 − e
3) + p
111e
1e
2e
3
A similar expression is written for the other seven probabilities. These can in turn be expanded to predict the probabilities of showing each pattern repeatedly; i.e., up to six errors.
![Page 17: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/17.jpg)
Expand and Simplify• There are 8 X 8 data patterns in an
experiment with 2 repetitions.• However, most of these have very small
frequencies.• Examine probabilities of each of 8
repeated patterns.• Frequencies of showing each of 8
patterns in one replicate OR the other, but NOT both. Mutually exclusive, exhaustive partition.
![Page 18: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/18.jpg)
Tests of WSTPercentage Choosing Column >pr Row Gamble
Row A B C D E
A 73 77 80 85
B 30 68 79 79
C 16 29 74 78
D 11 16 24 63
E 13 17 15 33
![Page 19: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/19.jpg)
Patterns for A, C, EPattern Rep. 1 Rep 2 Both
000 14 28 5
001 18 25 15
010 23 38 1
011 12 5 3
100 24 33 7
101 5 6 1
110 301 256 220
111 19 25 5
Sum 416 416 257
![Page 20: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/20.jpg)
Pattern Both Rep 1 or 2Not both
Est p Pred Both
Pred 1 or 2 Not both
000 5 16 .03 8.1 8.6
001 15 6.5 .07 15.3 6.5
010 1 29.5 .00 4.7 37.2
011 3 5.5 .01 2.8 5.9
100 7 21.5 .03 7.8 26.0
101 1 4.5 .00 0.9 5.5
110 220 58.5 .85 196.6 67.6
111 5 17 .02 4.6 17.9
Sum 257 159 1 240.9 175.1
![Page 21: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/21.jpg)
Comments• Results are surprisingly transitive, unlike
Tversky’s data (est. 95% transitive).• Of those 115 who were perfectly reliable,
93 perfectly consistent with EV (p), 8 with opposite ($), and only 1 intransitive.
• Differences: no pre-test, selection;• Probability represented by # of tickets
(100 per urn), rather than by pies.• Participants have practice with variety of
gambles, & choices;• Tested via Computer.
![Page 22: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/22.jpg)
Pie Chart Format
![Page 23: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/23.jpg)
Pies: with or without Numerical probabilities
• 321 participants randomly assigned to same study; except probabilities displayed as pies (spinner), either with numerical probabilities displayed or without.
• Of 105 who were perfectly reliable, 84 were perfectly consistent with EV (prob), 13 with the opposite order ($); 1 consistent with LS.
![Page 24: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/24.jpg)
Lower Standards
• Look at AB,BC,CD,DE choices and EA choices only:
• 10 of 321 participants showed this pattern; all in the pies-only condition. Although the rate is low (6% of 160), association with condition is clear!
• By still lower standard used by Tversky: 75% agreement with above pattern: 37 people, 27 in pies-only condition.
![Page 25: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/25.jpg)
Tests of Lexicographic Semi-order and Additive Difference• LS implies no integration of
contrasts (additive difference model allow integration)
• Both LS and additive difference models imply no interactions between probability and consequences.
![Page 26: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/26.jpg)
Test of Interaction
R S Pies & p
Pies & No p
($7.25, .95; $1.25, .05)
($4.25, .95; $3.25, .05)
16 22
($7.25, .05; $1.25, .95)
($4.25, .05; $3.25, .95)
84 77
![Page 27: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/27.jpg)
Among the 37 Leniently classified as Intransitive
• Are those 37 who are 75% consistent with the LS in the 5 choices also approx. consistent with LS in tests of Interaction?
• No. 26 of these have all four choices in the pattern of interaction predicted by TAX and other utility models.
![Page 28: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/28.jpg)
Summary• Priority Heuristic model’s predicted
violations of transitivity are rare and rarely repeated when probability and prize information presented numerically.
• Violations of transitivity are still rare but more frequent when prob information presented only graphically.
• Evidence of Dimension Interaction violates PH and additive Difference models.
![Page 29: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/29.jpg)
Conclusions
• Violations of transitivity are probably not due to intransitive strategy (LS or additive difference model), but rather to a configural assimilation of the probability values, which are then used in a numerical utility model.
• We are still unable to produce the higher rates of intransitivity reported by Tversky and others.
![Page 30: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/30.jpg)
Transitivity Test: ADLTrial Choice % Response Pattern
RepsEst. parameters
00 01 10 11 p e
8 50 to win $10050 to win $20
50 to win $6050 to win $27
18 190 23 25 23 0.10
0.10
3 50 to win $6050 to win $27
50 to win $4550 to win $34
29 140 44 39 37 0.17
0.20
21 50 to win $4550 to win $34
50 to win $10050 to win $20
74 35 33 20 172
0.85
0.12
![Page 31: Testing Transitivity with a True and Error Model](https://reader035.vdocuments.us/reader035/viewer/2022070406/5681416e550346895dad5609/html5/thumbnails/31.jpg)
Results-ADLpattern Rep 1 Rep 2 Both
000 LPH 21 13 1
001 TAX 134 147 106
010 20 18 8
011 38 37 10
100 15 9 0
101 14 10 0
110 12 15 7
111 6 11 1
Sum 260 260 133