economics, policy and political process presentation to lmu systems engineering leadership program...

Economics, Policy and Political Process

Presentation to LMU Systems Engineering Leadership Program

Marilee J. WheatonMarch 19, 2015

Marilee J. WheatonEducation:

Bachelor Degrees in Math and SpanishMagna Cum Laude, California Lutheran University

Graduate coursework in Mechanical Engineering (Thermal Fluids)

California State University, Northridge

Masters in Systems Engineering from USC Viterbi

Industrial and Systems Engineering Department, 1993

UCLA Executive Education ProgramAnderson School, 2001

PhD Program, Systems Architecting and Engineering, Astronautical Engineering Department, USC Viterbi

CMMI certified team member, Six Sigma Black Belt training

Marilee J. WheatonWork Experience:Started at Lockheed California Company in BurbankThe Aerospace Corporation, from 1980 to 1999 and 2002 to present

FFRDC for Space Systems, GSE&I, Architect-Engineer for Space SystemsCurrently Systems Engineering Fellow, Systems Engineering DivisionPreviously General Manager, Computer Systems and Systems Engineering Divisions, The Aerospace InstitutePrevious program office experience in Milstar, SDI Programs, Ground Systems programs (AFSCN, IC)

Industry experience at TRW Systems (now NGIS) from 1999 to 2002

Marilee J. WheatonTeaching Experience:USC Viterbi School of EngineeringOriginally taught CS 510, Software Engineering Economics, Fall 2003

Request by Dr. Barry Boehm, Director CSSE, who was on sabbatical

Then started teaching SAE 549 in 2004 through 2008

Share Dr. Rechtin’s vision for the importance of system architecting concepts and heuristics

Fall 2006 taught ISE 561, Advanced Engineering EconomicsFall 2008, Spring and Fall 2009, Co-Developer and Instructor, SAE 560, Economic Considerations for SAEFall 2010, Full circle to CS 510 againSpring 2011, Back to SAE 549Spring 2013, Back to SAE 560

Marilee J. WheatonProfessional Affiliations:Fellow, AIAA

Immediate Past Chair, Economics Technical CommitteeLeadership Team, Space 2009 through Space 2011 Conferences

Fellow Life Member, Society of Women Engineers

Past SWE LA President, National Life Membership Coordinator

Long time active member in Cost SocietiesInternational Society of Parametric Analysts (ISPA) and Society for Cost Estimating and Analysis (SCEA)Past Board Chair, Board Member and Conference Chair for ISPAMember, Space Systems Cost Analysis Group

Fellow, International Council on Systems Engineering (INCOSE)

Member, Corporate Advisory Board (CAB)Technical Program Chair, CSER 2011 and 2014

6

Management Relationships: The Political Process

• Why is this subject important?• Some reasons why an engineer might care:

– The political process determines budgets– The political process often sets time limits to

accomplish a project• And then doesn’t provide enough resources to

accomplish that project

– The political process often imposes regulations and constraints on designs

7


• The domain of the system architect in a project:

Technical

Management

Business

Political

People

People

People

People

Performance

Focus Is On:

Schedule

Cost

PoliticsTechnical PoliticalAwareness Awareness

Level of Level of

HIGH

HIGHlow

low

Operating domain of theSystem Architect

8


• The political system:– Not just formal political institutions (Congress &

White House)• Interagency rivalries• Intra-agency tensions• Dozens of lobbying groups• Influential external review groups• Powerful individuals both within and outside

government• And always, the media

9


• The political system:– Extremely complex interaction– Impossible to model quantitatively

• Too many variables• Most unquantifiable• Constant unpredictable change

– Confusing, sometimes chaotic– But -- determines budgetary funding levels

10


• Either enables engineering design process to go forward or imposes constraints:– Budget cuts– Schedule stretch outs– Technical reviews– Reporting requirements– Threat of outright cancellation

11


• Why so complex?– Power is very widely distributed in Washington– No single, clear-cut locus of authority to support

for long-term, expensive programs– Support must be cobbled together from grab-bag

of widely varying groups– Each may perceive program's worth very

differently; interests may diverge radically when pressure is on

12


• Coping skills for the modern design engineer:– First essential skill: ability to think in its terms– Must understand: political process logic system

is entirely different from the one in which scientists and engineers are trained

– D.C. uses logic of politics, which is rigorous - but:• Premises & rules profoundly different from scientific &

engineering logic• Will repeatedly arrive at different conclusions on basis

of same data

13


• Coping skills for the modern design engineer:– Scientific/engineering proof = firm assumptions +

accurate data + logical deduction– Political logic structured entirely differently

• Not logical proof• Based on negotiation, compromise & appearances

– Proof = "having the votes" • If so: program = worthy, useful and beneficial to the

nation• If not: no matter what its technological merits, will lose

out

14


• Focal point for the system architect’s study is the budget process

• The budget: the ultimate political value judgment– That's where the money is– No money means no program, regardless of need

or of the program’s technical merit– Remember that this value judgment repeated each

year

15


• Federal budget process best understood as struggle over political values because outcome embodies nation's current consensus (or its lack) as to the relative importance of its innumerable priorities

• Government is nothing like military or corporation– No "bottom line" instead, must decide what its

goal is

16


The facts of life:

• Not in priority order!

• All are “money = politics”#1 Politics, not technology, controls what

technology is allowed to achieve

#2 Cost rules

#3 A strong, coherent constituency is essential

#4 Technical problems become political problems

#5 The best engineering solutions are not necessarily the best political solutions

17

Smarter Buyer 1 Course Primary Learnings

Key topics – Wall Street Demands: How does influence of Wall Street impact

a government program?– Sector Demands: How do the aerospace/defense industry

sector financials impact individual program design and execution?

– Industry Bid/no-bid decision making and customer influence– Demands on the industry Program Manager: What does this

mean for the System Program Director (SPD)?• Take Aways for Future Reference

– The “X” chart: The key players in government/industry interaction and their roles

– The Financial Pyramid: financial measures important to industry– Smarter Buyer Reference Sheet: summary of key points we

heard from industry

18

Government Risk-Reward “Vee” Diagram

Top-Dow

n

Dem

ands and

Constraints Pr

ogra

m

Succ

ess

Con

trib

utio

ns

Time

Demands (Risk)Outcome(Reward)

Sp

ace

Dem

and

sW

arfi

gh

ter

and

Po

licy

Mak

erD

eman

ds

Pro

gra

m

Dem

and

sS

pace

Cap

abilities

Warfig

hter

Cap

abilities

Pro

gram

C

apab

ilities

USecAF/DNROPortfolio

PEO’s and NRO Director’s Space Portfolio

DoD/ ICBudget

Obligate all money

PortfolioSuccess

Mission Success

Space Budget

Program Requirements And Constraints

System Program Director

Space Contributions to Mission Area

PEO’s Space Portfolio

Congress

Taxpayers

19

SHAREHOLDERS

Industry Risk-Reward “Vee” Diagram

Top-Dow

n

Dem

ands and

Constraints Pr

ogra

m

Fina

ncia

lC

ontr

ibut

ions

Time

Demands (Risk)Financial Outcome(Reward)

Sec

tor

Dem

and

sW

all S

tree

t D

eman

ds

Pro

gra

m

Dem

and

sS

ector

Fin

ancials

Co

rpo

rate F

inan

cialsP

rog

ram

Fin

ancials

CEO targetsfor sectors

Sector targets for programs

Industry Program Manager

Corporate Financial Expectations

Sector PortfolioReturns

Corporate Earnings Stability and Predictability

Sector Expectations

Program FinancialPerformance Pressures

Program Financial Returns

Corporate Returns

Sector Returns

BOD

20

The “X” Diagram

Obligate all money

Demands (Risk) (Reward)

Program Demands

Program Capabilities

System Program Director

Program Req’ts And Constraints

Program Financials

Program Demands

Financial Outcome (Reward)Demands (Risk)

DoD/IC Budget Mission SuccessCongress

Taxpayers

Corporate Financials

Wall Street Demands

Corporate FinancialExpectations

Corporate Financial Results

BOT

SHAREHOLDERS

Industry Program ManagerProgram Financial Returns

Program Financial Performance Pressures

Space BudgetSpace Demands

SpaceCapabilities

Portfolio Success

Sector Financials

Sector Demands

Sector PortfolioReturnsSector Expectations

Warfighter and PolicyMakerDemands

Warfighter and PolicyMakerCapabilities

21

Financial Pyramid

• What are these financial parameters?

• How do these parameters relate to one another?

• What is the role of the CEO, Sector GM/VP, and Program Manager in their use?

• How can the government influence industry financial metrics?

• How do these financial metrics impact your program?

Orders

Sales

Earnings/Cash Flow

Share Value

$

This financial pyramid is the central link between Wall Street and industry through the CEO, VP/GM, and the Program Manager

Opportunities

22

Shared Financial Risk Management Metrics

PM

SPD

BusDev

VP/GMCEO

Return metric

Cash flow

Schedule

Hurdle rate

Return metric

Sales Growth

Cash flow

Cash flowAward fee

Acquisitions

Return metric

Sales Growth

Cash flow

Hurdle Rate

23

S M A R T E R B U Y E R R E F E R E N C E S H E E T In d u s tr y A s s e s s m e n t : N e c e s s a ry to u n d e rs ta n d c h a n g in g in d u s tr y e n v iro n m e n t

C o m p e t it iv e la n d s c a p e W a ll S tre e t ’s v ie w o f v ia b ility – th e s p a c e in d u s try a n d in d iv id u a l c o n tra c to r re tu rn s C o n tra c to r

R e c e n t w o n / lo s t re c o rd M a rg in s fo r p a r t ic u la r d iv is io n /g ro u p – R e tu rn B a c k lo g : W h a t is th e C o n tra c to r d o in g to d a y – re s o u rc e s a v a ila b le W h a t e ls e a re th e y b id d in g o n /t im in g o v e r la p S k ills a n d c o m p e te n c ie s – P a s t p e rfo rm a n c e , s u b c o n tra c to rs S tra te g ic p la n : to w h e re o r in to w h a t b u s in e s s d o th e y w a n t to m o v e , i.e . f ro m s u b

to p r im e ? R is k /R e tu rn : u n d e rs ta n d th e fa c to rs a n d re la t io n s h ip s to k e e p th is in b a la n c e

R is k a n d s u c c e s s fa c to rs d e f in e d a n d u n d e rs to o d – T e c h n ic a l, S c h e d u le , C o s t P ro g ra m re tu rn is c o m m e n s u ra te w ith r is k fo r C o n t ra c to r C o n s is te n c y b e tw e e n c o n tra c t T ’s a n d C ’s – a n d w h a t is to b e in c e n t iv iz e d

C o s t: b e a w a re o f th e m o tiv a t io n s b e h in d th e c o s t f ig u re s R e a lis t ic in d e p e n d e n t c o s t e s t im a te Is th e p ro p o s a l a b id - to -w in o r p r ic e - to -c o s t ( In c u m b e n t lo s e s 7 5 % o f th e t im e ) F o c u s o n v a lu e p ro p o s it io n o f C o n tra c to r /p ro p o s a l a n d n o t o v e r ly e m p h a s iz e d o n c o s t

C a s h F lo w : h e lp c o n s is te n t C o n tra c to r e a rn in g s M a k e re g u la r p a y m e n ts : b o th fo r s c h e d u le a n d a c tu a l re c e ip ts T im e d b e tw e e n f ro n t -e n d a n d b a c k -e n d : b e tw e e n p ro g re s s p a y m e n ts a n d in -o rb it

s u c c e s s S tru c tu re s a v in g s s o th a t C o n tra c to r is a b le to k e e p s o m e

P ro c e s s : In te g ra te th e G o v ’t a n d C o n tra c to r b u s in e s s p ro c e s s e s a n d c o m m u n ic a t io n In s u re o p t im a l re q u ire m e n ts u n d e rs ta n d in g a n d e v a lu a t io n - a d e q u a te t im e b e tw e e n d ra f t

a n d f in a l R F P a n d a ls o b e tw e e n th e R F P a n d P ro p o s a l S u b m it ta l O p tim a l t im in g b e tw e e n S o u rc e S e le c t io n a n d P ro g ra m s ta n d -u p to m in im iz e re s o u rc e s

u n d e r /n o n -e m p lo y e d C o m m u n ic a t io n o f te n a lo n g e n t ire p ro c e s s

M e tr ic s : U n d e rs ta n d m e tr ic s u s e d b y C o n tra c to r le v e ls o f m a n a g e m e n t

In te rn a l m e tr ic s C o n tra c to r u s e d to m e a s u re p ro g re s s o f p ro g ra m A c c e s s to d a ta ; h o w o f te n a n d b y w h a t v e h ic le in fo s h a re d R e a liz e th a t in te rn a l m e tr ic s c h a n g e m a y a ls o c h a n g e c o n tra c to r p e r fo rm a n c e

F e e : D e s ig n a n d m a in ta in a fe e s tru c tu r e th a t in c e n t iv e s th e r ig h t s u c c e s s g o a ls

S p re a d a m o n g b a s e , a w a rd a n d in c e n t iv e S p lit a m o n g te c h , s c h e d u le , c o s t a n d m a n a g e m e n t th a t is a lig n e d w ith g o a ls A d e q u a te p o o l a s a m o t iv a t io n in c e n t iv e fo r c o n tra c to r to re s p o n d to G o v e rn m e n t

c o n c e rn s U n e a rn e d p o r t io n w ith p o s s ib ility o f ro llo v e r

24

Top 10 IPA Team Finding Areas

1. Poor government cost baseline (e.g., awarding the acquisition contract based on less than government cost estimate)

2. Poor schedule baseline (e.g., awarding the acquisition contract based on a schedule shorter than government schedule estimate, “meet me at the pass” planning, not using technology on/off ramps effectively

3. Changes in major requirements after acquisition contract award

4. Poor government SPO technical baseline (e.g., at KDP B)

• Missing or poor SOO, TRD, WBS/SOW, CARD, Approved Acquisition Strategy

• Cutting corners during preparation to save time in getting on contract

• Using success-oriented plans (over promise/ under perform)

• Assuming that none of those problems that other programs have encountered will happen to this program

Source: S. Soderquist, Director, SMC Acquisition Center of Excellence (ACE), presented Jan 2007 Space Systems Cost Analysis Group (SSCAG) Meeting

25

Top 10 IPA Team Finding Areas (Cont.)

5. Poor contractor processes and poor implementation of those processesIMS/IMP, EVMS, engineering/qualification equipmentParts/box/subsystem/system testing, configuration control

6. Poor government oversight of contractor processes and testing

7. Program disruption due to problems in government decisionsTime required to provide data to independent teams, and lack

of timely access to decision makersTime required for RFP preparation and source selectionBudget cut drillsDifficulties in meeting obligation and expenditure standards,

resulting in OSD budget cuts8. Other system engineering shortfalls

Test and Evaluation planning, requirements decomposition and traceability, trades, interface planning

9. SPOs not applying the lessons learned and best practices derived from past program experience

10. Too few qualified people in the SPO and contractors

26

Tom Young Panel on NSS Acquisition

1) Cost #1, not mission success2) Unrealistic estimates = unrealistic

budgets = unexecutable programs3) Undisciplined system requirements 4) Government space acquisition

capabilities seriously eroded5) Industry failed to implement proven

management and engineering practices

26

27

Cost and Schedule Estimating

• Recognizes that best value is not necessarily lowest cost bid

• Government must place value on non-deliverables essential to mission success (Examples: SE, MA, QA,…)

– Then industry will also value them

– Exclude “name-that-tune-in-three-notes” bids

• Has a well-established Independent Cost Estimating (ICE) and program control function

• Budget program to 60 - 80% confidence, including a management reserve sized by risk– Expend reserves to execute unforeseen elements of baseline

program—not new requirements

28

Stability

• Stable, manageable baselines—requirements, budget, and schedule also include managing expectations– Manage necessary but unplanned changes

– Rigorous systems engineering process for assessing impact of new requirements

– New requirements must come with new funding

• Allows trade spaces vs. “cast-in-concrete” requirements– Capabilities, cost, and schedule

• Architectures that allow right-sized programs (can be executed in about 5 years)– Regulates appetite of user community

University of Southern California

Center for Systems and Software Engineering

Next Generation Systems and Software Cost Estimation

Barry Boehm, USC-CSSE



Next-Generation Measurement Challenges

• Emergent requirements– Example: Virtual global collaboration support systems– Need to manage early concurrent engineering

• Rapid change– In competitive threats, technology, organizations,

environment

• Net-centric systems of systems– Incomplete visibility and control of elements

• Always-on, never-fail systems– Need to balance agility and discipline



The Broadening Early Cone of Uncertainty (CU)

ConOps Specs/Plans IOC

• Need greater investments in narrowing CU– Mission, investment, legacy

analysis

– Competitive prototyping

– Concurrent engineering

– Associated estimation methods and management metrics

• Larger systems will often have subsystems with narrower CU’s

Global Interactive, Brownfield

Batch, Greenfield

Local Interactive,Some Legacy

18 February 2009©USC-CSSE 31

X8

X4

X2



32

COSYSMO

SizeDrivers

EffortMultipliers

Effort

Calibration

# Requirements# Interfaces# Scenarios# Algorithms

Volatility Factor

- Application factors-8 factors

- Team factors-6 factors

- Schedule driver WBS guided by ISO/IEC 15288

COSYSMO Operational Concept



Next-Generation Systems Challenges



environment





©USC-CSSE

3415 July 2008

Rapid Change Creates a Late Cone of Uncertainty– Need evolutionary/incremental vs. one-shot development

Feasibility

Concept of Operation

Rqts. Spec.

Plans and

Rqts.

Product Design

Product Design Spec.

Detail Design Spec.

Detail Design

Devel. and Test

Accepted Software

Phases and Milestones

RelativeCost Range x

4x

2x

1.25x

1.5x

0.25x

0.5x

0.67x

0.8x

Uncertainties in competition, technology, organizations,

mission priorities



Effects of IDPD on Number of Increments

0

20004000

60008000

10000

1200014000

1600018000

20000

1 2 3 4 5 6 7 8

Build

Cumulative KSLOC

0% productivity decline10% productivity decline15% productivity decline20% productivity decline

• Model relating productivity decline to number of builds needed to reach 8M SLOC Full Operational Capability

• Assumes Build 1 production of 2M SLOC @ 100 SLOC/PM

– 20000 PM/ 24 mo. = 833 developers

– Constant staff size for all builds

• Analysis varies the productivity decline per build

– Extremely important to determine the incremental development productivity decline (IDPD) factor per build

2M

8M

SLOC






environment





Further Attributes of Future Challenges

©USC-CSSE

37

Type Examples Pros Cons Cost Estimation

Systems of Systems

•Directed: Future Combat Systems

•Acknowledged: Missile Defense Agency

•Interoperability•Rapid Observe-Orient-Decide-Act (OODA) loop

•Often-conflicting partner priorities

•Change processing very complex

•Staged hybrid models•Systems engineering: COSYSMO•Multi-organization development costing

•Lead Systems integrator costing•Requirements volatility effects

•Integration&test: new cost drivers

Model-Driven

Development

•Business 4th-generation languages (4GLs)

•Vehicle-model driven development

•Cost savings•User-development advantages

•Fewer error sources

•Multi-model composition incapabilities

•Model extensions for special cases (platform-payload)

•Brownfield complexities•User-development V&V

•Models directives as 4GL source code

•Multi-model composition similar to COTS integration, Brownfield integration

Brownfield

•Legacy C4ISR System

•Net-Centric weapons platform

•Multicore-CPU upgrades

•Continuity of service

•Modernization of infrastructure

•Ease of maintenance

•Legacy re-engineering often complex

•Mega-refactoring often complex

•Models for legacy re-engineering, mega-refactoring

•Reuse model for refactored legacy

18 February 2009



Further Attributes of Future Challenges (Continued)

©USC-CSSE 38

Type Examples Pros Cons Cost Estimation

Ultrareliable Systems

•Safety-critical systems

•Security-critical systems

•High-performance real-time systems

•System resilence, survivability

•Service-oriented usage opportunities

•Conflicts among attribute objectives

•Compatibility with rapid change

•Cost model extensions for added assurance levels

•Change impact analysis models

Competitive

Prototyping

•Stealth vehicle fly-offs

•Agent-based RPV control

•Combinations of challenges

•Risk buy-down•Innovation modification

•In-depth exploration of alternatives

•Competitor evaluation often complex

•Higher up-front cost

•But generally good ROI

•Tech-leveling avoidance often complex

•Competition preparation, management costing

•Evaluation criteria, scenarios, testbeds

•Competitor budget estimation•Virtual, proof-of-principle, robust prototypes

18 February 2009



Net-Centric Systems of Systems Challenges

• Need for rapid adaptation to change– See first, understand first, act first, finish decisively

• Built-in authority-responsibility mismatches– Increasing as authority decreases through Directed,

Acknowledged, Collaborative, and Virtual SoS classes• Incompatible element management chains, legacy

constraints, architectures, service priorities, data, operational controls, standards, change priorities...

• High priority on leadership skills, collaboration incentives, negotiation support such as cost models– SoS variety and complexity makes compositional cost

models more helpful than one-size-fits-all models



October 2007 ©USC-CSSE 40

Comparison of Cost Model Parameters

Parameter Aspects COSYSMO COSOSIMO

Size drivers # of system requirements

# of system interfaces

# operational scenarios

# algorithms

# of SoS requirements

# of SoS interface protocols

# of constituent systems

# of constituent system organizations

# operational scenarios

“Product” characteristics Size/complexity

Requirements understanding

Architecture understanding

Level of service requirements

# of recursive levels in design

Migration complexity

Technology risk

#/ diversity of platforms/installations

Level of documentation

Size/complexity

Requirements understanding

Architecture understanding

Level of service requirements

Component system maturity and stability

Component system readiness

Process characteristics Process capability

Multi-site coordination

Tool support

Maturity of processes

Tool support

Cost/schedule compatibility

SoS risk resolution

People characteristics Stakeholder team cohesion

Personnel/team capability

Personnel experience/continuity

Stakeholder team cohesion

SoS team capability






environment





Always-on, never-fail systems• Consider using “weighted SLOC” as a productivity metric• Some SLOC are “heavier to move into place” than others

– And largely management uncontrollables– Examples: high values of COCOMO II cost drivers

• RELY: Required Software Reliability • DATA: Database Size• CPLX: Software Complexity• DOCU: Required Documentation• RUSE: Required Development for Future Reuse• TIME: Execution Time Constraint• STOR: Main Storage Constraint• SCED: Required Schedule Compression

• Provides way to compare productivities across projects– And to develop profiles of project classes

15 July 2008 ©USC-CSSE 42



Conclusions• Future trends imply need to concurrently address new

DoD estimation and management metrics challenges– Emergent requirements, rapid change, net-centric systems of

systems, ultrahigh assurance

• Need to work out cost drivers, estimating relationships for new phenomena– Incremental Development Productivity Decline (IDPD)

– ESLOC and milestone definitions

– Compositional approach for systems of systems

– NDI, model, and service composability

– Re-engineering, migration of legacy systems

– Ultra-reliable systems development

– Cost/schedule tradeoffs

• Need data for calibrating models





References Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”,

Systems Engineering 9(1), pp. 1-19, 2006. Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of

Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and

Software Engineering,” CrossTalk, October 2007, pp. 4-9.Boehm, B., Brown, A.W.. Clark, B., Madachy, R., Reifer, D., et al., Software Cost Estimation with COCOMO

II, Prentice Hall, 2000.Dahmann, J. (2007); “Systems of Systems Challenges for Systems Engineering”, Systems and

Software Technology Conference, June 2007.Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.

Department of Defense (DoD), Instruction 5000.2, Operation of the Defense Acquisition System, May 2003.

Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004.

Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006.Lane, J. and Boehm, B., “Modern Tools to Support DoD Software-Intensive System of Systems Cost

Estimation, DACS State of the Art Report, also Tech Report USC-CSSE-2007-716Lane, J., Valerdi, R., “Synthesizing System-of-Systems Concepts for Use in Cost Modeling,” Systems

Engineering, Vol. 10, No. 4, December 2007.Madachy, R., “Cost Model Comparison,” Proceedings 21st, COCOMO/SCM Forum, November, 2006,

http://csse.usc.edu/events/2006/CIIForum/pages/program.htmlMaier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-

284).Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software

Engineering Institute, 2006. Reifer, D., “Let the Numbers Do the Talking,” CrossTalk, March 2002, pp. 4-8.Valerdi, R, Systems Engineering Cost Estimation with COSYSMO, Wiley, 2009 (to appear)




List of Acronyms

AA Assessment and Assimilation

AAF Adaptation Adjustment Factor

AAM Adaptation Adjustment Modifier

COCOMO Constructive Cost Model

COSOSIMO Constructive System of Systems Integration Cost Model

COSYSMO Constructive Systems Engineering Cost Model

COTS Commercial Off-The-Shelf

CU Cone of Uncertainty

DCR Development Commitment Review

DoD Department of Defense

ECR Exploration Commitment Review

ESLOC Equivalent Source Lines of Code

EVMS Earned Value Management System

FCR Foundations Commitment Review

FDN Foundations, as in FDN Package

FED Feasibility Evidence Description

GD General Dynamics

GOTS Government Off-The-Shelf




List of Acronyms (continued)

ICM Incremental Commitment Model

IDPD Incremental Development Productivity Decline

IOC Initial Operational Capability

LCA Life Cycle Architecture

LCO Life Cycle Objectives

LMCO Lockheed Martin Corporation

LSI Lead System Integrator

MDA Model-Driven Architecture

NDA Non-Disclosure Agreement

NDI Non-Developmental Item

NGC Northrop Grumman Corporation

OC Operational Capability

OCR Operations Commitment Review

OO Object-Oriented

OODA Observe, Orient, Decide, Act

O&M Operations and Maintenance

PDR Preliminary Design Review

PM Program Manager




List of Acronyms (continued)

RFP Request for Proposal

SAIC Science Applications international Corporation

SLOC Source Lines of Code

SoS System of Systems

SoSE System of Systems Engineering

SRDR Software Resources Data Report

SSCM Systems and Software Cost Modeling

SU Software Understanding

SW Software

SwE Software Engineering

SysE Systems Engineering

Sys Engr Systems Engineer

S&SE Systems and Software Engineering

ToC Table of Contents

USD (AT&L) Under Secretary of Defense for Acquisition, Technology, and Logistics

VCR Validation Commitment Review

V&V Verification and Validation

WBS Work Breakdown Structure

economics, policy and political process presentation to lmu systems engineering leadership program...

Documents

systems engineering

systems engineering

computer systems

trw systems

software engineering

political processpresentation

political processwhy

budgetsthe political