translating the business needs into an improvement programme using cmm: a practical experience...

Translating the business needs into an improvement programme using CMM:

a practical experience within PHILIPS TV

Guy Van HooveldThanks to Hans Aerts, Baudewyn Meersseman and Geert Acke

for some of the presentation material

High End TV, Guy Van Hooveld 2

• History of SPI within PHILIPS

• High End TV – why SPI ?

• Achievements

• Recommendations

• New challenges

• Conclusions


Evolution of Embedded Software: 4 Software Stages

– Software Introduction stage

– Feature Expansion stage

– Mastering Complexity stage

– Complex Systems stage

Hardware-dominated industry

Software-dominated industry


Historic Overview• 1987 - 1991: Phase 0: Early Awareness Creation

Nov ‘87: Software Reference Model of the new “Center for Software Technology” mentions the “project” view

‘89: “Managing the Software Process” by Watts Humphrey ‘91 - ‘92: Growing SPI awareness within Philips

• 1992 - 1996: Phase 1: SPI Infrastructure Creation Jun ‘92: First SPI Guidelines published Jun ‘93: First SPI Steering Committee meeting Dec ‘96: Uniform approach within Philips for CMM certification

• 1996 - now: Phase 2: Monitor SPI Results 1997:

SPI Steering Committee with a clear assignment and budget CTOs clearly showed commitment to SPI

Dec ‘00: More than 50% of all software staff works in sites with CMM level 2 or higher

2003: number of CMM level 3 organizations is growing




• Achievements

• Recommendations

• New challenges

• Conclusions


High End Market

High End products, High End expectations and targets


Business priorities

• Predictibility (shorter lead time, lower lead time and effort slip)

• Product quality (low field call rate)• Featuring (extended functionality)• Cost (reducing development cost, increasing

productivity)

• These priorities have to be translated into an improvement programme. Software is playing a key role.


Predictibility

• The world of consumer electronics is relying on introduction dates, selling seasons

• Being on time with the right product is paramount• Lead time, time slip are top priorities• Any delay is paid cash (market share, sales volume,

…)


Field call rate : what is acceptable?Field call rate : what is acceptable?

4 Cylinders +25.000 €

6 Cylinders + 40.000 €

8 Cylinders + 70.000 €

The higher the price, the higher the complexity, the higher the potential failure modes… ?


Expected quality in High End TV

• There is a correlation between product complexity (and size) and potential failure modes -> field call rate

• Does a BMW 7 user expect more problems than a BMW 3 user ?

• Does he/she accept more problems ?• Of course not :

QUALITY NEEDS ARE MUCH HIGHERin the HIGH END segment


Field call rate: what about this...?Field call rate: what about this...?

16Kb SW 200 €

64Kb SW/ 50 Hz.600 €

2Mb+ SW/100 Hz.1200 € to 8000 €

The higher the price, the higher the complexity, the higher the potential failure modes… ?


Evolution of Embedded Software: 4 Software Stages

– Software Introduction stage

– Feature Expansion stage

– Mastering Complexity stage

– Complex Systems stage

Hardware-dominated industry

Software-dominated industry


Software Performance Improvement

SW Introduction

Feature Expansion

Mastering Complexity

Open Systems

Process

Organization

Product

People

Technology

Architecture


Improvement Requirements per Stage

SW

Introduction Feature

Expansion Mastering

Complexity Open

Systems

Organization • Team: 1-3 • Single room

• Skill group • Team: 3-12

• Multi-site team, • Organization wide

• Organization wide, • Alliances

People • Electronic engineers

+ SW generalists + SW specialists + Architects

+ SW purchasers + SM managers

Technology • 4-8 bit µP • Assembler, C,

PL/M

• Req. Specs • 8-16 bit µP, • C

• Req./Design • 16-32 bit µP • C, C++, • RTK, components

• OS, • external applications

(DBMS, GUI, internet)

Architecture • None • Recycling of

product code • Proprietary

(closed) • Open Systems, • Standards

Process • Product

testing, • Basic CM

• CMM ≥ 2 • CMM ≥ 3 • Supplier Agreement

management

Evolution in Embedded SW Development


How much software does a shaver contain ?

Telephone exchange:1975: 256 Kbytes1990: 80 Mbytes

High-end TV:1985: 8 Kbytes2000: 2 Mbytes

Shaver:2000: 4 Kbytes2015: ?????


Software Growth

1

10

100

1000

10000

19

85

198

7

19

89

19

91

19

93

19

95

19

97

19

99

20

01

Year

So

ftw

are

Siz

e (

KB

yte

s)

TV VCR Double every 2 years




• Achievements

• Recommendations

• New challenges

• Conclusions


Evolution time slip

CR / SR Timeslip

8%

46%

6%6%

0%0%

10%8%

42%

0%

10%

0%

6%

0%0%5%

0% 0%

0%

10%

20%

30%

40%

50%

Sep-98 Dec-98 Mar-99 Jun-99 Sep-99 Dec-99 Mar-00 Jun-00 Sep-00 Dec-00 Mar-01 Jun-01 Sep-01 Dec-01

Release date

% T

imes

lip

CR timeslip Target

01/2001

Decision making made possible: predictibility is also predicting delay.Peak in 2000 ? Product was not on the critical path and had the appropriate quality.The potential delay was predictable and has been taken into account.


Evolution post release defects density

Post-release defect density (faults/100 kbytes)

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

Jan-99 Apr-99 Jul-99 Oct-99 Jan-00 Apr-00 Jul-00 Oct-00 Jan-01 Apr-01 Jul-01 Oct-01 Jan-02

fau

lts/

kby

tes


Software quality (measured with FCR)

• 1998: top on the pareto list• 2000: major improvement visible• 2002: disappearing from the pareto• 2003: very few small incidents reappearing• 2004: ? (new platform)


Correlation pre-post release defectsPre Vs Post release defects

0

50

100

150

200

250

300

350

400

0 200 400 600 800 1000 1200

Pre-release defects

Po

st-r

elea

se d

efec

ts

We could expect a linear correlation, but reality shows that the number of post releasedefects is in fact more or less constant while complexity is growing.


Some achievements based on facts and figures

Results of SPI and of a platform approach:– Number of clusters (types of products) released has tripled

since 1998– Software product quality has significantly increased (field

calls reduced by a factor 10)

despite the following facts – Software size and complexity have quadrupled since 1998– No significant software headcount growth in High End TV

since 1999




• Achievements

• Recommendations

• New challenges

• Conclusions


CMM priorities – step 1what to do first

• Basic processes• Seldom an issue (most organizations very quickly master those KPAs)• Configuration management• Requirements management• Basic planning and tracking (not necessarily CMM compliant)• Subcontract management (basic) if applicable


CMM priorities – step 2Increase control• Basic leverage

– Process quality• QA

– High level supportive quality officer– Escalation procedure– Issues tracked to closure– Subcontract management (extended) if applicable

• Estimates– Better planning and tracking (extended)

– Product quality (related to PE and PR KPAs)• Functional test team installed very early• Peer reviewing at all levels• Architecture• Guidelines (coding rules, design templates, …)


CMM priorities – step 3Mature organization

• Extended processes : • OPF-OPD – fully integrated strategy• ISM – PP/PT at a higher level: improve proactivity• IC – look outside software -> project management• TP – formalized

• Start thinking of CMM L4 and L5• Defects prevention first (focus on product quality)• Introduce statistical process control


CMM prioritiesWhat to do in parallel ?• To achieve the maturity levels, focus has to be put

on:– Improving the capabilities of the organization

• Install a SEPG group• Hiring higher level people• Organize training• Focus on interdepartmental awareness• Integrate in overall improvement plan• Adapt the organization structure

• To realize product strategy• Define and deploy future proof concept and architecture• Adapt the organizational model accordingly




• Achievements

• Recommendations

• New challenges

• Conclusions


New challenges• Major new challenges for software:

– Cooperation between product development and IC related SW development (rely more on suppliers)

– “Digital” (huge increase of software size/complexity)• Leading to problems like:

– Customer-supplier relationship and software ownership– Customer-specific and common components in one architecture– Main software/DSP software integration

(technical integration of stacks and organizational cooperation between two existing large SW organizations with their own systems/structures/culture)

• Requiring solutions in:– Programme definition– Architecture– Process– Programme execution


New challenges (continued)

• Need to keep product quality under control• -> Process performance needs to be quantified and

analyzed: first candidates are Peer Reviews and Testing(improving the quality of defects filters at phase transitions)

• New kinds of testing are being deployed/extended• Home testing• Field testing• Stress testing• User perception




• Achievements

• Recommendations

• New challenges

• Conclusions


Software Performance Improvement

SW Introduction

Feature Expansion

Mastering Complexity

Open Systems

Process

Organization

Product

People

Technology

Architecture


What is the key to implement change ?

• Stepwise approach to address business needs progressively

• Know what is wrong• Know what has to be changed first• Need to “Measure”, “Detect”• CMM L2, L3

• Relying on QA (quality officer)• Product quality information

• CMM L4, L5• Apply statistical process control

• Also focus on the other deployment axes


Different aspects are relatedHigh

Architecture and Technology level

Organization, Staff, and

Process level

problematic inefficient

high risk target

HighLow


Balanced Software Performance Improvements

High

Architecture and Technology level

Organization, Staff, and

Process level

problematic inefficient

high risk target

HighLow

SPISPI

translating the business needs into an improvement programme using cmm: a practical experience...

Documents

software process

software technology

software staff

evolution of embedded

history of spi

philips high end tv

showedcommitmentto spi

spi guidelines