an open problem: characterizing random utility models of best-worst choice a.a. j. marley university...
Post on 20-Dec-2015
216 views
TRANSCRIPT
An Open Problem: Characterizing Random Utility Models of Best-Worst Choice
A. A. J. Marley
University of Victoria, Victoria
DIMACS Workshop on Polyhedral Combinatorics of Random Utility, May 2006
Research Supported by
• The Australian Research Council (ARC)
• The Natural Sciences and Engineering Research Council (Canada)
Summary
• An illustrative empirical study
• Definitions and theoretical examples
• Random utility models for best-worst choice
• Characterizing random utility models of best worst choice
• (Further theoretical results on Thurstonian models of best-worst choice)
Public Issues
• Preserving the environment• Quality medical care• Cost of living• Crime and public safety• National unity/Canada’s future• Level of taxation• Poverty in Canada• Drinking and driving• A quality food supply• The drug problem• Food safety
Question & Layout for Best-Worst Task (example shows 2 questions)
Question & Layout Used for First Best-Worst Task
Identify the public issue that causes you the most concern and the public issue that causes you the least concern in each of the following sets of issues.
Check only one issue for each of the Least and Most Columns.
LEAST ISSUE MOST LEAST ISSUE MOST
Preserving the environment Cost of living
Level of Taxation A quality food supply
Quality medical care Quality medical care
The drug problem Poverty in Canada
Fractional factorial design for choice sets (Best-Worst task)
Food safety 0 0 1 0 0 1 1 1 0 1 0 1 6
Preserving the environment 1 0 0 1 0 0 0 1 1 1 0 1 6
Cost of living 0 1 0 0 1 0 1 0 1 1 0 1 6
Level of taxation 1 0 1 0 0 0 1 0 1 1 1 0 6
National unity/Canada’s future 0 0 1 0 1 0 0 1 1 0 1 1 6
Crime & public safety 0 0 0 1 0 1 1 0 1 0 1 1 6
A quality food supply 0 1 1 1 0 0 0 0 0 1 1 1 6
Quality medical care 1 1 0 0 0 0 1 1 0 0 1 1 6
Drinking & driving 0 0 0 1 1 0 1 1 0 1 1 0 6
The drug problem 1 0 0 0 1 1 0 0 0 1 1 1 6
Poverty in Canada 0 1 0 0 0 1 0 1 1 1 1 0 6
# Issues in set 4 4 4 4 4 4 6 6 8 8 8 8
Issue 1 2 3 4 5 6 7 8 9 10 11 12# timespresent
Comparison of Most-Least totals Vs MNL estimates
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-1.000 -0.500 0.000 0.500 1.000 1.500
MNL Estimates
Mo
st-
Le
as
t T
ota
ls
Summary
• An illustrative empirical study
• Definitions and theoretical examples
• Random utility models for best-worst choice
• Characterizing random utility models of best worst choice
• (Further theoretical results on Thurstonian models of best-worst choice)
Consistent Random Ranking Models
And the Open Problem
Observations
• A consistent random ranking model on (all the subsets of) a set T has consistent margins
• Conversely, if a set of best-worst choice probabilities satisfy a random ranking model (next slide), then one can define a set of best and worst choice probabilities through the margins of the ranking probabilities
• Thus, the major task is to characterize a random ranking model on best-worst choice probabilities
Summary
• A major open problem is to characterize the class of (best, worst and) best-worst choice probabilities that satisfy a consistent random ranking model
• We have a characterization of the maxdiff model for best-worst choice. This model is the most frequently used in applications
• The maxdiff model (with its ‘natural’ extension to best and worst choices) is not compatible with a consistent random ranking model
Thank you for listening
Summary
• A major open problem is to characterize the class of consistent random ranking models of best, worst and best-worst choice
• We have a characterization of the maxdiff model for best-worst choice. This model is the most frequently used in applications
• The maxdiff model (with its ‘natural’ extension to best and worst choices) is not compatible with a consistent random ranking model
Thank you for listening