air travel forecast problem 1 objectives introduction to forecasting methods experience with delphi...
TRANSCRIPT
1Air Travel Forecast Problem
Air Travel Forecast Problem
Objectives• Introduction to forecasting methods• Experience with Delphi• Experience with consensus-seeking techniques• Strength/weaknesses of various methods
2Air Travel Forecast Problem
Methodology Tree for Forecasting
Causalmodels
Datamining
Statistical
Univariate
Theory-based
Data-based
Extrapolationmodels
Multivariate
Rule-basedforecasting
Unaidedjudgment
Judgmental
SelfOthers
Role playing(Simulatedinteraction)
Role No role
Conjointanalysis
Knowledgesource
Quantitativeanalogies
Unstructured Structured
Feedback No feedback
Predictionmarkets
DelphiDecom-position
Structuredanalogies
Methodology Tree for Forecastingforecastingpriciples.com
JSA-KCGSeptember 2005
Neuralnets
Expertsystems
Intentions/expectations
Judgmentalbootstrapping Segmentation
Linear Classification
Game theory
3Air Travel Forecast Problem
Techniques for Forecasting
Form groups of about 5 to 7 people, then use the:
Delphi procedureFirst estimate – individual and anonymous
Statistical summary – group
Group discussion (use consensus technique)
Second estimate – individual and anonymous
Statistical summary - group
Minutes12
3
20
2
3
40
4Air Travel Forecast Problem
Group Results
AccuracyRankings:(Round 2)
Group
1 2 3 4 5 Averageranks
Judgment
Bootstrapping
Segmentation
Causal model
Extrapolations
5Air Travel Forecast Problem
Discussion
Discuss Delphi
Expected results
When to use
Actual Results
Initial hypotheses
Results in Air Travel study
Calculation of your error score
Conclusions
6Air Travel Forecast Problem
DelphiAgreement among experts
Your resultsMore agreement among panelists on Round 1 _____No differences (Round 1 vs. 2) _____More agreement on Round 2 _____
Findings from literature: Typically more agreement on later rounds
Expected accuracy: Which do you expect to be closest to actual ranks?
Your opinionsRound 1 more accurate _____Round 2 more accurate _____No difference _____
Delphi improves accuracy vs. traditional meetingsgiven some expertise among panelists
7Air Travel Forecast Problem
Round 2: Previous Rankings vs. Your Rankings
Method
Average Ranking
MBA(21 groups)*
Adv. Mgmt.(28 groups)*
You
Judgment 2.2 2.4
Bootstrapping 3.2 2.9
Segmentation 2.2 2.0
Causal 2.6 2.9
Extrapolation 4.7 4.8
*Groups from U.S., Sweden, Norway, and Netherlands
8Air Travel Forecast Problem
Evidence-based Findings(“>” means “more accurate than”)
1. Objective methods > subjective: especially for large changes
2. Causal methods > naïve: especially for large changes
3. Bootstrapping > Judgment
4. Structured meetings > unstructured
No Yes
Sufficientobjective data
YesNo
YesNo
Large changes expected
Policy analysis
YesNo
Conflict among a fewdecision makers
Type ofknowledge
Policyanalysis
NoYes
Domain Self
YesNo
Time seriesCross-section
Type ofdata
Goodknowledge ofrelationships
Policyanalysis
No Yes
Gooddomain
knowledge
Yes No
YesNo
Large changes likely
Similarcases exist
YesNo
Judgmental methods Quantitative methods
YesNo
Delphi/Predictionmarkets
Judgmentalbootstrapping/Decomposition
Conjointanalysis
Intentions/expectations
Role playing(Simulatedinteraction/
Game theory)
Structuredanalogies
Expertsystems
Rule-basedforecasting
Extrapolation/Neural nets/Data mining
Causalmodels/
Segmentation
Quantitativeanalogies
Accuracyfeedback
Unaidedjudgment
NoYes
Selection Tree for Forecasting Methodsforecastingprinciples.com
JSA-KCGJanuary 2006
YesNo Use adjusted forecast
Several methods provide useful forecasts
Singlemethod Omitted
information?
Combine forecasts
Use unadjusted forecast
Using the Selection Tree
?
9
10Air Travel Forecast Problem
Rankings based on Evidence-based Findings
Method Rank Why?
Causal model 1.5
Objective and causalSegmentation 1.5
Extrapolation 3 Objective and naïve
Bootstrapping 4 Objective/subjective and causal
Judgment 5 Subjective and causal
Evidence summarized in Armstrong (1985), Long-Range Forecasting, and Armstrong (2001), Principles of Forecasting – see forecastingprinciples.com
11Air Travel Forecast Problem
Accuracy of the Different Methods of Forecasting U.S. Air Travel, 1963-1968
(Successive updating used)
Source: Armstrong & Grohman (1972) in full text at forecastingprinciples.com
Forecast HorizonYears (Number Ahead of Forecasts)
Mean Absolute Percentage Error*
Extrapolation Judgment Econometric
1 (6) 2 (5) 3 (4) 4 (3) 5 (2) 6 (1)
5.712.717.422.527.529.9
6.815.625.134.142.1
45.0**
4.26.87.39.86.20.7
Averages (21) 19.3 28.1 5.8
* The forecasts were lower than actual in nearly all cases.** Estimated
12Air Travel Forecast Problem
Average Error Scores*
Round 2
MBAs 7.4
Advanced Mgt. 7.5
Forecasting Experts 8.4
You
*Key: Best possible = 0No information (all ties) = 6Worst possible = 12
13Air Travel Forecast Problem
General Advice
• Beware of unaided judgment• Be conservative when uncertain – thus, use equal ranks
given uncertainty about most accurate method