Design and Development Risk Management

Dr. Dick Hayes, Evidence Based Research, Inc.

CDR. Steve Palmquist, USCG

George Thurmond, Georgia Tech

John Birkler, RAND

Bill Lese, Logicon

John Smith, Decisive Analytics

Cyrus Staniec, Logicon

Debora Donnellon, NIMA

Design and Development Risk Management Dimensions

•Product Service Market Risks–Cost, Schedule, and Quantity Tradeoffs–Expectations and Value

•Technical Risks–Maturity of Technology–Technical Advance Required–Interoperability

•Management Processes–Process Indicators–Personnel Indicators–Productivity Indicators

Management Processes Checklist

•Identify Warning Signs, Enable Timely Interventions• Process Indicators

–Changing goals - “lowering the bar”–Slipping milestones “stretch out”–ECP’s that raise costs–Parallel development efforts without strong oversight–Failure to establish and maintain top down, integrated vision–Progress report vagueness, lack of item resolution–Low levels of technical interaction–Weak Configuration Management

•Productivity Indicators During Development–“Healthy” Debugging Rate–Ease of Integration–Improving Test Results over Time

Management Processes Checklist (2 of 2)

• Personnel Indicators– Changes in key personnel– Leadership inexperienced in similar projects– Breadth of senior personnel knowledge– Corporate culture barriers

Metrics for Assessing Products and Services

•Explicit Metrics, Consistent over time, are essential

•Key Dimensions Include•Reliability•Ease of Maintenance•Ease of Use•Interoperability•Design Characteristics•Measures of Systems Performance•Measures of C4ISR Effectiveness•Measures of Force Effectiveness•Measures of Policy Effectiveness

•Ignoring any key dimension is likely to lead to problems

Design and DevelopmentRisk Management: Longer Term

•Creation of Specialized Training for C4ISR Acquisition Managers (IT acquisition is different from systems acquisition)

•Creation of a Community of C4ISR Acquisition Managers–Web site–Workshops–Annual Symposium–Common Literature and References

•Implement Virtual Collaboration Tools Using IT to Support C4ISR Acquisitions

–Faster–More Inclusive–Clearer Consensus–Cheaper but limited IT capability in community, work process change required, leadership needed.

•Life Cycle Tracking of Relevant Data–Program Data and Information–Simulations, Experiments, Tests

•Improved Contractual Flexibility–Internet Based–Pre-qualified–Specialized Services–Examples: GSA Schedules

Design and DevelopmentRisk Management: Longer Term (2 of 2)

Short Term Recommendations

• Increase Visibility & Quality of Application of the C4ISR Architecture Framework (V 2.0)

– formally applies to “all on-going & plannedC4ISR Architectures”– But lacks standards, quality control, and formal approval process

• Emplace a process to link across individual Architectures

Other Good Ideas

• Information Dissemination is Critical

– CROP vs. Drinking from A Firehose

– Subscription, Limited Push, Rich Pull desirable

– Experimentation is essential to determine best mix

• Co-evolution is essential

– Market risk element in JWIDs, ACTD’s, etc.

– Systems need CONOPS, Doctrine, Training, Maintenance, etc.

– Fools gold vs. golden nuggets

Existing Tools and Sources of Help

•Software Program Managers Networkwww.spmn.com

•Information Technology Resources Boardwww.itrb.gov

•Software Development CMM Software Acquisition CMM People CMM Architecture Tradeoff Analysis Method (ATAM)

www.sei.cmu.edu

•C4ISR Architecture Framework V2.0

•Joint C4ISR Analysis/Planning System (JCAPS)

•C4ISR Decision Support Center (DSC)