the process improvement journey of boeing …...page 1 john d. vu sepg 2005 -ieee the process...
TRANSCRIPT
Page 1 John D. VuSEPG 2005 -IEEE
The Process Improvement Journey Of
Boeing Information Services, Wichita
John VuTechnical Fellow
The Boeing Company
Page 2 John D. VuSEPG 2005 -IEEE
Boeing Information Services in Wichita
• Provide software to support Wichita division.• Focus on software design, architecture, application
development & maintenance, COTs integration, technology evaluation, selection, and transfer.
• Support all Boeing commercial aircraft and some military airplanes (KC135, KC 10, B52, E-3 AWACS etc.)
1992Initiate
Process Improvement
1995Assessed At Level 2
2001Assessed At Level 4
2004Assessed At Level 5
1993Assessed At Level 1
1997Assessed At Level 3
2002Transition to CMMI
Page 3 John D. VuSEPG 2005 -IEEE
Why Boeing Information Services, Wichita?Boeing Wichita is• Part of a company-wide improvement. • One of 72 organizations identified in an improvement strategy plan.• Pilot site for SW-CMM validation study (1991-1994)• Unique since it was not re-organized during merger.
– Management commitment is at all levels– Data collection is not disrupted– SEPG members rotational process is still active
• Leading software activities in Boeing– Major contribution to DCAC/MRM program– Key contribution to 3D Graphic design of airplane (CATIA)– Pilot site for several new technologies
• Activities & Lessons Learned are shared among organizations– Templates & Techniques are being used by many organizations
• First IS organization in Boeing to achieve SW-CMM level 4 in 2001
Page 4 John D. VuSEPG 2005 -IEEE
Process Improvement Results10 year study on process improvement
120 projects in Boeing Information Services in Wichita participated in the validation study of the SW-CMM between 1991-1994
Measurement baseline established in 1991 and re-established in 1996
Pilot site for CMMI Transition
Data collected and analyzed independently by Dr. Kay Nelson of University of Kansas
Page 5 John D. VuSEPG 2005 -IEEE
Process Improvement Context
Task
Software Process Improvement Plan
Organization’s Business Goals
CMMI Current Process Capability Maturity
Training
Reviews
Implementation
Measurement
Improvement Tasks
CMMI is only a guide
Business goals are key drivers
Plan is based on appraisal results
Business goals are key drivers
Measurements Repository
Data are used to verify im
provement results
Page 6 John D. VuSEPG 2005 -IEEE
Measurements Are Key To Success
Core Measurements:• Defects: Post & Pre-released• Estimates: Plan vs. Actual (Schedules, Efforts, Costs)• Cycle Time: Time to complete an activity• Customer Satisfaction: Monthly Survey• Employee Satisfaction: Bi-Annual survey• Number of management decisions based on metrics
Page 7 John D. VuSEPG 2005 -IEEE
It All Started With Project Estimates
• The utilization of historical data will improve project performance by reducing the variation in estimates
• Better estimates will improve project schedules• Better schedules will improve project management• Better project management will improve project
quality and reduce costs• Better project quality and reduced costs will improve
customer satisfaction• Satisfied customers will improve relationships• Better relationships will improve the business
Page 8 John D. VuSEPG 2005 -IEEE
Software Estimates(Actual vs Planned)
140%
Ove
r/Und
er P
erce
ntag
e
(Based on 120 projects in Boeing Information Systems)
Sche
dule
Var
iatio
n
- 148%
+ 22%
-125%
+26% +20%
- 24%
+12%
-18%
+4%
- 7%
Utilize Historical data for all project estimates
Ove
r est
imat
esU
nder
est
imat
es
Level 1 & 2 Level 3, 4 and 5
Page 9 John D. VuSEPG 2005 -IEEE
Establish Formal Gate Reviews
Req. Design Code Test Post-Release
Rew
ork
Effo
rt
Before Formal ReviewAfter Formal Review
8%
1%
12%
3%
19%
4%
Implementing Formal Review increased Design effort by 4%decreased Rework effort by 31%
Reduce 31%in rework
Cost: Benefit ratio is 4% : 31% or 1 : 7.75
Page 10 John D. VuSEPG 2005 -IEEE
Total Number Of Defects Per YearTo
tal N
umbe
r Of D
efec
ts
1997 1998 1999 2000 2001 2002 2003 2004
Level 3 level 4 level 5
1997 1998 1999 2000 2001 2002 2003 2004
Level 3 level 4 level 5
: Total Pre-released defects: Total Post-released defects
Page 11 John D. VuSEPG 2005 -IEEE
Defect Prevention Cost Savings
10%
- 20%
64%70%
81%77%
82%
1998 1999 2000 2001 2002 2003 2004
Level 3 Level 4 Level 5
Perc
enta
ge C
ost S
avin
gs P
er Y
ear
Page 12 John D. VuSEPG 2005 -IEEE
Increased Software Reuse = Reduced Costs
Perc
ent o
f Sof
twar
e re
use
80%
70%
60%
50%
40%
30%
20%
10%
0%
10%
25%
58%58%64%64%
Level 1 Level 2 Level 3 level 4 level 5Level 1 Level 2 Level 3 level 4 level 5
?
Page 13 John D. VuSEPG 2005 -IEEE
Increased Software Reuse = Reduced CostsPe
rcen
t of S
oftw
are
Reu
se
40%
35%
30%
25%
20%
15%
10%
5%
0%
CodeOthers
3%
10%
25%28%
15%
33%
Code reuse: No modificationOther reuse: Templates, Test cases etc.
36%
Level 2 Level 3 Level 4 Level 5
7%
Page 14 John D. VuSEPG 2005 -IEEE
Software Maintenance Cost Savings
Perc
ent o
f Cos
t Sav
ing
Per y
ear
100%
80%
60%
40%
20%
0%1997 1998 1999 2000 2001 2002 2003 2004
Level 3 level 4 level 5
1997 1998 1999 2000 2001 2002 2003 2004
Level 3 level 4 level 5
23%
: Average percentage of cost savings based on 1997 baseline
27%
56%
69% 67%
73% 70%
204% Increased Cost Savings
Page 15 John D. VuSEPG 2005 -IEEE
Cycle Time = Supported Hours Per Element
Num
ber o
f Hou
rs R
equi
red
To
supp
ort a
n El
emen
t
1.0
.8
.6
.4
.2
0.0
Element = Configuration Item
1997 1998 1999 2000 2001 2002 2003 2004
Level 3 Level 4 Level 5
1997 1998 1999 2000 2001 2002 2003 2004
Level 3 Level 4 Level 5
.77
.57.50
.34
.24 .26
.21 .23
: Average number of hour required to supported an Elements in maintenance
70% More Efficient
Page 16 John D. VuSEPG 2005 -IEEE
Cycle Time
100
80
60
40
20
0.0
1996 1997 1998 1999 2000 2001 2002 2003 2004
Ave
rage
day
s pe
r Cha
nge
Req
uest
per
mon
th
79.8
44.6 50.240.8
30.9 30.624.7
34.5
29.1
64% Faster response to Customer Change Request
Level 3 Level 4 Level 5
Page 17 John D. VuSEPG 2005 -IEEE
Flow Time Days Avoided(1996 Baseline)
1997 1998 1999 2000 2001 2002 2003 2004
Flow Time Days Avoided
Page 18 John D. VuSEPG 2005 -IEEE
Customer Satisfaction5.0
4.0
3.0
2.0
1.0
0.0
1997 1998 1999 2000 2001 2002 2003 2004
Ave
rage
Cus
tom
er S
atis
fact
ion
Inde
x ba
sed
on m
onth
ly s
urve
y
3.853.98
4.08 4.11 4.224.14 4.16
4.05
Level 3 Level 4 Level 5
Page 19 John D. VuSEPG 2005 -IEEE
Employee SatisfactionNumber of Employees Number of Employees
Extremely satisfied 10
Highly Satisfied 9
Very satisfied 8
Satisfied 7
Not Quite Satisfied 6
Neutral 5
Not excited About 4
Dissatisfied 3
Very Dissatisfied 2
Highly Dissatisfied 1
74%
Mean = 5.7
96%
Mean = 8.9
Before Process Improvement
AfterProcess Improvement
Satisfaction Level
Page 20 John D. VuSEPG 2005 -IEEE
Productivity = Less People - More Works
500
400
300
200
100
0
55% Reduction in Headcount
280 K
240K
200 K
180 K
160 K
140 K
100 K
80 K
60 K
40 K
20 K
0 KNum
ber o
f peo
ple
in W
ichi
ta In
form
atio
n Sy
stem
s
Num
ber O
f Sof
twar
e El
emen
ts S
uppo
rted
150% Increase in Statement Of Work
Element: Software Configuration Item
Level 3 Level 4 Level 5
Page 21 John D. VuSEPG 2005 -IEEE
Return On Investment
There is no perfect formula to calculate Return On Investment for Process Improvement. Different organizations use different methods. Our 10 year study indicated a significant return on investment when maturing from a Level 1 to Level 5 as calculated by the following formula:
ROI = 2740%
Benefit realization – Cost of Process improvementReturn On Investment =
Cost of Process ImprovementX 100%
Where:
Benefits Realization = Labor cost savings
Cost of Process Improvement = Cost of SEPG (Labor + SPI Tools + Training)
Page 22 John D. VuSEPG 2005 -IEEE
Benefit Cost RatioOur Benefit Cost Ratio is a measure of how much money is gained from following the CMMI improvement framework.Our 10 year study indicated a significant benefit cost ratio when maturing from Level 1 to Level 5.
Benefit Cost Ratio = Benefit Realization
Cost of Process Improvement
Benefit Cost Ratio = 28.5
Where:
Benefits Realization = Labor cost savings
Cost of Process Improvement = Cost of SEPG (Labor + SPI Tools + Training)
Page 23 John D. VuSEPG 2005 -IEEE
CMMI Transition
We found • No evidence of difficulty transitioning from SW-CMM to CMMI• CMMI makes engineering work more visible to management• The notion that CMMI Level 3 has many processes and is
difficult to implement is not true • Transition from CBA/IPI to SCAMPI is an improvement• Investment in process improvement can be (and should be)
explained in business terms
Process Improvement works
Page 24 John D. VuSEPG 2005 -IEEE
Capability Maturity ModelsBased on our 10 year study, we concluded that:
There is a systematic approach to improving an organization’s software and systems, and achieving business goals and objectives
There are stages of process maturity in which an organization can significantly improve its products and services by followinga recommended sequence
By following an evolutionary path of a well defined model the organization can continuously improve its products and services, and at the same time meet or exceed its business goalsand objectives