using multi-attribute tradespace exploration for the...
TRANSCRIPT
seari.mit.edu © 2010 Massachusetts Institute of Technology 1
Using Multi-Attribute Tradespace Exploration for the Architecting and Design of Transportation Systems
Julia NickelFinal Presentation
(S.M. Engineering Systems)
February 25, 2010
seari.mit.edu © 2010 Massachusetts Institute of Technology 2
Motivation
In Engineering Systems we seek to develop generally applicable, domain-independent methods. The application of sophisticated decision and design methods such as MATE* across domains supports this research by uncovering commonalities and domain differences. The transportation domain was chosen because of its relative similarity to the aerospace domain, which helps to control for a number of factors.
2
*MATE = Multi-Attribute Tradespace Exploration (Ross et al. 2004)
seari.mit.edu © 2010 Massachusetts Institute of Technology 3
Thesis Structure
Chapter Content
1. Introduction Motivation, Research Questions…
2. Literature Review MATE, Transportation planning ,CBA
3. Implementation Issues in applying MATE within the Transportation Domain
Updated CSER 08 paper: Domain differences
4. Case study 1: Chicago Airport Express At length case study, MATE, CBA, political feasibility
5. Case study 2: Portuguese High-Speed Rail
MATE Set-up, local context
6. Discussion Answer 4 research questions, future work
7. Appendix Chicago data, model summaries, interesting tradespaces
3
seari.mit.edu © 2010 Massachusetts Institute of Technology 4
Thesis Structure
Chapter Content
1. Introduction Motivation, Research Questions…
2. Literature Review MATE, Transportation planning ,CBA
3. Implementation Issues in applying MATE within the Transportation Domain
Updated CSER 08 paper: Domain differences
4. Case study 1: Chicago Airport Express At length case study, MATE, CBA, political feasibility
5. Case study 2: Portuguese High-Speed Rail
MATE Set-up, local context
6. Discussion Answer 4 research questions, future work
7. Appendix Chicago data, model summaries, interesting tradespaces
4
seari.mit.edu © 2010 Massachusetts Institute of Technology 5
Research Questions
1. What design methods are used for transportation systems planning? What are their limitations? What alternative system analysis methods are available?
2. What implementation issues arise if MATE, as a systems analysis method developed within the aerospace domain, is applied to the transportation domain?
3. What methodological insights emerge through the application of Cost-Benefit Analysis and MATE, individually and complementarily?
4. What insights are gained from the application of MATE to two transportation case studies for both MATE and transportation planning?
5
seari.mit.edu © 2010 Massachusetts Institute of Technology 6
Research Question 1/4
What design methods are used for transportation systems planning? What are their limitations? What alternative system analysis methods are available?
6
seari.mit.edu © 2010 Massachusetts Institute of Technology 7
Research Question 1/4
What design methods are used for transportation systems planning? What are their limitations? What alternative system analysis methods are available?
Analytic toolsCost-Benefit AnalysisFinancial AnalysisEconomic Impact AnalysisEnvironmental Impact Analysis
Planning modelsRational planningSatisficingIncremental changeOrganizational processPolitical bargaining
References: de Neufville (1990), Luzzi (2001), Meyer and Miller (2001)
7
seari.mit.edu © 2010 Massachusetts Institute of Technology 8
Research Question 1/4
What design methods are used for transportation systems planning? What are their limitations? What alternative system analysis methods are available?
Analytic toolsCost-Benefit AnalysisFinancial AnalysisEconomic Impact AnalysisEnvironmental Impact Analysis
Planning modelsRational planningSatisficingIncremental changeOrganizational processPolitical bargaining
References: de Neufville (1990), Luzzi (2001), Meyer and Miller (2001)
8
seari.mit.edu © 2010 Massachusetts Institute of Technology 9
Research Question 2/4What implementation issues arise if MATE, as a systems
analysis method developed within the aerospace domain, is applied to the transportation domain?
9
Implementation issues encountered• A mission objective does not need to be an integral part of a project
definition• The complex, multi-layered stakeholder structure cannot be reduced to 1, 2,
or one aggregate stakeholder• Equity and value judgments cannot be excluded from the technical design
decision• Inheritance (hard and soft) bring with them stickiness of the status quo
(-> Prospect Theory)• Several cost types play an important role. A distinction needs to be made
between resource expenditure (money, time) and potentially non-reparable adverse effects that extend to third parties (harmful effects to life)
seari.mit.edu © 2010 Massachusetts Institute of Technology 10
Decision making processes that do not involve a mission objective
10
Economy Environment
Equity
Three E’s of political sustainability according to Fred Salvucci
1) Problem definition, analysis of options based on (marginal) changes to statusquo and unintended, adverse effects (often not fully evaluable).Example: dealing with inner-city congestion
2) Goal definition, recognition of competing objectives (some variation of 3 Es) and pledge to more or less reconcile them in whatever is done, actions filtered by what is politically acceptable to constituents
3) Mission focus: Stepping back to identify core mission, identification of solutions and evaluation criteria, evaluation, decision making, implementation
Challenge: Accurately identify implementation constraints on solutions (notoverly constrained so as to limit creativity, not under-constrained to generate
realistic solutions).
seari.mit.edu © 2010 Massachusetts Institute of Technology 11
Research Question 3/4
What methodological insights emerge through the application of Cost-Benefit Analysis and MATE, individually and complementarily?
11
seari.mit.edu © 2010 Massachusetts Institute of Technology 12
Research Question 3/4
What methodological insights emerge through the application of Cost-Benefit Analysis and MATE, individually and complementarily?
12
Chicago Airport Express Portuguese High-Speed RailCase studies
seari.mit.edu © 2010 Massachusetts Institute of Technology 13
Research Question 3/4
What methodological insights emerge through the application of Cost-Benefit Analysis and MATE, individually and complementarily?
13
Chicago Airport Express Portuguese High-Speed RailCase studies
Insights (MATE and CBA)Complementary methodsCBA: broad view, quantifies net benefits to society as a wholeMATE seeks to best meet decision makers’ expectations for a system
seari.mit.edu © 2010 Massachusetts Institute of Technology 14
Research Question 3/4
What methodological insights emerge through the application of Cost-Benefit Analysis and MATE, individually and complementarily?
14
Chicago Airport Express Portuguese High-Speed RailCase studies
Insights (MATE and CBA)Complementary methodsCBA: broad view, quantifies net benefits to society as a wholeMATE seeks to best meet decision makers’ expectations for a system
seari.mit.edu © 2010 Massachusetts Institute of Technology 15
Chicago Airport Express
seari.mit.edu © 2010 Massachusetts Institute of Technology 15
CTA Blue Line
Kennedy ExpresswayO’Hare
Loop
seari.mit.edu © 2010 Massachusetts Institute of Technology 16
Chicago Airport Express
seari.mit.edu © 2010 Massachusetts Institute of Technology 16
CTA Blue Line
Kennedy ExpresswayO’Hare
Loop
Cab fare ~ $45
seari.mit.edu © 2010 Massachusetts Institute of Technology 17
Chicago Airport Express
seari.mit.edu © 2010 Massachusetts Institute of Technology 17
CTA Blue Line
Kennedy ExpresswayO’Hare
Loop
seari.mit.edu © 2010 Massachusetts Institute of Technology 18
Chicago Case Study Overview
(1) description of the approach(2) an introduction(3) identification of stakeholders(4) elicitation of their attributes using both CBA and MATE(5) generation of alternative system concepts(6) calculation of Cost-Benefit Values for different options(7) calculation of aggregated Utility-Expense and plot of
tradespaces for different options(8) tradespace exploration(9) evaluation of the best design option(10) discussion of the shortcomings of CBA (discussed in
Chapter 2) (11) discussion of political feasibility and potential opposition(12) technical recommendations
18© 2010 Massachusetts Institute of Technologyseari.mit.edu
Introduction
Attribute elicitationArchitecture concepts generation
CBA
MATE
Technical evaluation
Political feasibility evaluation
seari.mit.edu © 2010 Massachusetts Institute of Technology 19
MATE and CBA Definitions
MATE:
CBA: Quantify and aggregate first-order (tangible) costs to society, discount over system lifetime, choose highest Net Present Value
19
(DV1, DV2, …DVm) (X1, X2, … Xn) U(X)
Technical and cost models
Utility models, based on stakeholder interviews
Expense
Utility Explore tradespace, present,
discuss, start over
seari.mit.edu © 2010 Massachusetts Institute of Technology 20
Approach
• 3 decision makers: City of Chicago, CTA, Private Operator
• Technical data from CTA, collected during summer internship 2008
• Missing data filled in by proxy assumptions• Semi-structured utility interviews with
decision makers elicited very different attributes -> grouped into hierarchy tree
20
seari.mit.edu © 2010 Massachusetts Institute of Technology 21
Hierarchy of Attributes
“Program” and portfolio decisions overlapdue to lack of continuous funding
How to deal with attributesat different levels?
Causal relationships for higher level goals not always well understood(no models available)
In thesis: decomposition(not ideal, ignores cross-reactions, “dummy” attributes)
21
StakeholdersCity of Chicago ■CTA ΘPrivate Operator ж
seari.mit.edu © 2010 Massachusetts Institute of Technology 22
Contractual Attributes
Contractual attributes (depend on negotiations between stakeholders)
•Freedom of Private Operator to make changes ж
Competition agreements with City, CTA ж
Length of concession contract ж Θ
Dispersion of (managerial) control ж Θ
Fare level growth per year ж
How to model? Solution in thesis for most part: modeling as design variables
22
Rationale: Outcome of negotiations unknown, enumerate many different possibilities of outcomes to see how they perform
StakeholdersCity of Chicago ■CTA ΘPrivate Operator ж
seari.mit.edu © 2010 Massachusetts Institute of Technology 23
Nr. Concepts Abbreviation
Description Rejected? Reason
0 Base case Base Minimal improvement to status quo infrastructure
yes Requirements specify minimal improvement from status quo
1 Direct service
Route 1a
Train solution, shared tracks with local train
yes Violates reliability requirement of 80% on time
Route 1b
yes Cannot be financed based on willingness to pay information from stakeholder interviews
2 Express service
Route 2a
Train solution, individual right-of-way no -
3 Bus Rapid Transit
BRT1 Rapid buses on Kennedy Expressway, no separate lane
yes Violates reliability requirement of 80% on time
BRT 2 Rapid buses on separate lane of Kennedy Expressway
no -
4 Blue Line Switch
BLS Rapid buses on separate lane on Kennedy Expressway replacing local
train, Airport Express on freed-up tracks from base case
no -
Screening Architecture Concepts
23
Three architecture concepts meet cost constraint and minimum reliability requirements -> further MATE and CBA analysis
seari.mit.edu © 2010 Massachusetts Institute of Technology 24
Rights-of-Way of Architecture Concepts
24
Route 2
seari.mit.edu © 2010 Massachusetts Institute of Technology 25
Rights-of-Way of Architecture Concepts
25
Route 2 Blue Line
Kennedy Expressway
seari.mit.edu © 2010 Massachusetts Institute of Technology 26
Cost-Benefit Analysis (CBA)
i - j)/ ( 1+d)t = NPV
B1 or C1: Capital cost, consisting of construction cost and vehicle acquisition, and (not applicable in the case study) income from sales of vehicles or infrastructure;B2 or C2: Increases C2 (or reduction B2 from discontinuing service) in operating cost, calculated for fuel cost, personnel (maintenance and operations) and vehicle replacement costs individually. B3 or C3: Reductions (B3) or increases (C3) in emissions, calculated for the emission types CO (Carbon monoxide), NOx (Nitrogen oxide), PM10 (Particulate Matter of of 10 micrometers or less), SOx (Sulfur Oxide) and VOC (Volatile Organic Compounds) individually; B4 or C4: Savings (B4) or increases (C4) in travel time to Blue Line riders, airport riders and drivers on the Kennedy Expressway, calculated for each group individually.
26
Notional
seari.mit.edu © 2010 Massachusetts Institute of Technology 27
CBA Results for Feasible Architecture Concepts (1)
In $ 2008 M Base
case
Route 2 BRT Blue Line Switch (excl.
former Blue Line)
d=7% 0 -97 -70 718
d=10% 0 170 -37 447
27© 2010 Massachusetts Institute of Technologyseari.mit.edu
Smaller is better! (Savings to society)
Discount rates based on ESD.71, de Neufville (gov’t vs. industry discount rate)
seari.mit.edu © 2010 Massachusetts Institute of Technology 28
CBA Results for Feasible Architecture Concepts (2)
In $ 2008 M Base
case
Route 2 BRT Blue Line Switch (excl.
former Blue Line)
d=7% 0 -97 -70 718
d=10% 0 170 -37 447
28© 2010 Massachusetts Institute of Technologyseari.mit.edu
Smaller is better! (Savings to society)
seari.mit.edu © 2010 Massachusetts Institute of Technology 29
CBA Results for Feasible Architecture Concepts (3)
In $ 2008 M Base
case
Route 2 BRT Blue Line Switch (excl.
former Blue Line)
d=7% 0 -97 -70 718
d=10% 0 170 -37 447
29© 2010 Massachusetts Institute of Technologyseari.mit.edu
BRT is the only robust design for varying discount rates (7-10%)
seari.mit.edu © 2010 Massachusetts Institute of Technology 30
Airport Express Model: Attributes and Design Variables (DVs)
Attributes DM Metric Weight
Expense
City's initial cost City $M 0.16
City's cost share City % 0.12
CTA initial cost CTA $M 0.2
Shared tracks with
Blue Line (BL)
CTA % BL
capacity
reduced
0.4
Operating costs PO $/day 0.1
Concession payment PO $M 0.2
Utility
QOS City Scale [1 to 5] 0.28
Span of service CTA Hrs/day 0.1
QOS_PO PO Scale [1 to 5] 0.2
Freedom to make
changes
PO Scale [1 to 5] 0.15
Competition
Agreements
PO Scale [1 to 5] 0.15
30
DVs Range Measure
Concept [1, 2, 3] Route 2, BRT, BLS
Fare level [10, 20] $
Frequency [5, 20] headway in min
Travel time [20, 30] min
Amenities [1,2,3,4,5] Qualitative Scale, 5
most amenities
Span of service [16, ...,24] hr/day
City cost share [10, 50] %
Freedom to make
changes
[1,2,3,4,5] Qualitative Scale, 5
most freedom
Competition
agreements
[1,2,3,4,5] Qualitative Scale, 5
most protection from
competition
CTA payment [0, 100] $M
Max design-space = 3,600,000,000
Based on previously captured value propositions
seari.mit.edu © 2010 Massachusetts Institute of Technology 31
City Tradespace (by Concept)
Route 2
BRT
BLS
31© 2010 Massachusetts Institute of Technologyseari.mit.edu
n=20,000
Red (Route 2) is the only concept planners had initially considered.
Quality of Service:Fare levelFrequencyTravel timeAmenitiesSpan of service
City cost share
City’s initial cost
Concept= BRT, Fare= $10Frequency= 8 min Travel time =27minAmenities = 1Span of service = 19 hrsCity Cost Share= 10%Comp agreements= 5Freed. to make changes =4CTA paym.= 15
seari.mit.edu © 2010 Massachusetts Institute of Technology 32
CTA Tradespace (by Concept)
Route 2
BRT
BLS
32© 2010 Massachusetts Institute of Technologyseari.mit.edu
n=20,000CTA payment
Maintenance time
CTA payment =0Span of service < 18 hrs
seari.mit.edu © 2010 Massachusetts Institute of Technology 33
Private Operator Tradespace (by Concept)
Route 2
BRT
BLS
33© 2010 Massachusetts Institute of Technologyseari.mit.edu
n=20,000
Red (Route 2) is the only concept planners had initially considered.
Operating costsConcession payment
•Freedom to make changes
•Competition agreements
•QOS_PO:Fare levelFrequencyTravel timeAmenitiesSpan of service
seari.mit.edu © 2010 Massachusetts Institute of Technology 34
•Freedom to make changes
•Competition agreements
•QOS_PO:Fare levelFrequencyTravel timeAmenitiesSpan of service
Private Operator Tradespace (by Concept)
Route 2
BRT
BLS
34© 2010 Massachusetts Institute of Technologyseari.mit.edu
n=20,000
Red (Route 2) is the only concept planners had initially considered.
If I relax maximum Operating Cost, more red designs are feasible
Operating costsConcession payment
seari.mit.edu © 2010 Massachusetts Institute of Technology 35
Utility for Three Decision Makers (vs. Operating Cost)
CTA’s and Private Operator’s utility aligned and correlate with lower Operating Costs (red).
CTA
35© 2010 Massachusetts Institute of Technologyseari.mit.edu
Best for all
infeasible
City UtilityQuality of Service:Fare levelFrequencyTravel timeAmenitiesSpan of service
PO Utility•Freedom to make changes
•Competition agreements
•QOS_PO:Fare levelFrequencyTravel timeAmenitiesSpan of service
CTA Utility
Maintenance timeOps Cost ($/day)
seari.mit.edu © 2010 Massachusetts Institute of Technology 36
Utility for Three Decision Makers (vs. Total Construction Cost)
CTA
36© 2010 Massachusetts Institute of Technologyseari.mit.edu
Best for all
PO Utility•Freedom to make changes
•Competition agreements
•QOS_PO:Fare levelFrequencyTravel timeAmenitiesSpan of service
City UtilityQuality of Service:Fare levelFrequencyTravel timeAmenitiesSpan of service
CTA Utility
Maintenance timeTCC ($B)
seari.mit.edu © 2010 Massachusetts Institute of Technology 37
Utility for Three Decision Makers (vs. Total Construction Cost)
CTA
37© 2010 Massachusetts Institute of Technologyseari.mit.edu
Best for all
Cheap design exists with high utility for PO and CTA
City UtilityQuality of Service:Fare levelFrequencyTravel timeAmenitiesSpan of service
PO Utility•Freedom to make changes
•Competition agreements
•QOS_PO:Fare levelFrequencyTravel timeAmenitiesSpan of service
Concept= BRTSpan of service < 18 hrsCTA payment =0City Cost Share= 30%Fare= high
TCC ($B)
seari.mit.edu © 2010 Massachusetts Institute of Technology 38
Fares: True Conflict of Interest
Private Operator
City
Lower fares (in $)
Higher fares
38© 2010 Massachusetts Institute of Technologyseari.mit.edu
seari.mit.edu © 2010 Massachusetts Institute of Technology 39
Fares: True Conflict of Interest
Private Operator
City
Lower fares
Higher fares
39© 2010 Massachusetts Institute of Technologyseari.mit.edu
… but also possibility for negotiation
seari.mit.edu © 2010 Massachusetts Institute of Technology 40
Insights from MATE and CBA
• Complementary methods• CBA: broad view, quantifies net benefits to society as a
whole, prescribes what should be cared about• MATE seeks to best meet decision makers’ expectations
for a system, can quantify intangible benefits• Issues with representation of constituents (value-based
interviewing)• Recommendation from case study: BRT
40
seari.mit.edu © 2010 Massachusetts Institute of Technology 41
Political Feasibility
• Dedicated BRT line under-uses resources(mitigation: use for toll)
• Prestige concerns of BRT (mitigation: marketing)• Employment generation and job losses (BLS: -50 + 120
= 70 net new jobs, but…)• Dispersion of investment: electricity generated
domestically, investments in vehicles and fuel leave state
• Uncertainty and scalability of operations
41
seari.mit.edu © 2010 Massachusetts Institute of Technology 42
Next Thesis*: Larger issue of political feasibility
• Tradeoff between political feasibility and technical optimality
• May limit solution space early on, without exploration of possible mitigation
• Hopefully: What factors increase political feasibility, and how can they be incorporated in the conceptual design phase?
42
*Candidate for S.M. in Political Science, August 2010
seari.mit.edu © 2010 Massachusetts Institute of Technology 43
Thank you.
Questions?
Contact: [email protected]
43
seari.mit.edu © 2010 Massachusetts Institute of Technology 44
References
44
Citations• de Neufville, R. (1990). Applied Systems Analysis, Engineering Planning and Technology
Management. New York, NY, McGraw-Hill.• Luzzi, J. (2001). The Rational Planning Model in Forest Planning: Planning in the Light of
Ambivalence. Ecosystem Workforce Working Papers, Ecosystem Workforce Program, University of Oregon.
• Meyer, M. and E. Miller (2001). Urban Transportation Planning. New York, NY, McGraw-Hill.• Ross, A.M., Hastings, D.E., Warmkessel, J.M., and Diller, N.P. (2004) “Multi-Attribute
Tradespace Exploration as a Front-End for Effective Space System Design.” AIAA Journal of Spacecraft and Rockets. Jan/Feb 2004.
Publications• Nickel, J., Using Multi-Attribute Tradespace Exploration for the Architecting and Design of
Transportation Systems, S.M. Thesis, Engineering Systems Division, MIT, Feb 2010. [pdf]• Nickel, J., Ross, A.M., and Rhodes, D.H., "Comparison of Project Evaluation Using Cost-Benefit
Analysis and Multi-Attribute Tradespace Exploration in the Transportation Domain," 2nd International Symposium on Engineering Systems, Cambridge, MA, Jun 2009. [pdf]
• Nickel, J., Ross, A.M., and Rhodes, D.H., "Trading Project Costs and Benefits in Multi-Attribute Tradespace Exploration," 7th Conference on Systems Engineering Research, LoughboroughUniversity, UK, Apr 2009. [pdf]
• Nickel, J., Ross, A.M., and Rhodes, D.H., “Cross-domain Comparison of Design Factors in System Design and Analysis of Space and Transportation Systems,” 6th Conference on Systems Engineering Research, Los Angeles, CA, Apr 2008. [pdf]
seari.mit.edu © 2010 Massachusetts Institute of Technology 45
Utility and Expense functions
Ux= wi*
Ex= wi*
• diminishing return function (γ= 1/2, δ=1/2) underlies utility and expense functions in the tradespaces
• weights wi are derived from weights distributes in interviews, normalized over the selected attributes
45© 2010 Massachusetts Institute of Technologyseari.mit.edu