oxford software engineering software engineering services & consultancy

26
OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy Metrics in the context of the CMM/SPICE SPIN-UK, 29 September 1998

Upload: taya

Post on 12-Feb-2016

51 views

Category:

Documents


0 download

DESCRIPTION

OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy. Metrics in the context of the CMM/SPICE SPIN-UK, 29 September 1998. OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy. Some views on measurement… - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Metricsin the context of the CMM/SPICE

SPIN-UK, 29 September 1998

Page 2: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Some views on measurement…• “What is not measurable, make measurable.”- Galileo• “When you can measure what you are speaking about and express it in

numbers you know something about it; but when you cannot measure it, when you cannot express it in numbers your knowledge is of a meagre and unsatisfactory kind.” - Lord Kelvin

• “…This remark [above] has been quoted with an approval it does not altogether deserve - it does not, for example, do justice to the work of Darwin or Pasteur, to name only two of Kelvin’s contemporaries.” - G.U.Yule (statistician)

• “You can’t control what you can’t measure.” - DeMarco• “Metrics is crap.” - Dijkstra

Page 3: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

What is measurement?

Page 4: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

real world

Page 5: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

Formal relational systemmeasurement

real world mathematical world

Page 6: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

Formal relational system

Results

measurement

mathematics and

statistics

real world mathematical world

Page 7: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

Relevantempirical

information

Formal relational system

Results

measurement

interpretation

mathematics and

statistics

real world mathematical world

Page 8: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

Relevantempirical

information

Formal relational system

Results

measurement

interpretation

mathematics and

statistics

decisions and

actions

real world mathematical world

Page 9: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Characteristics...

• good measurement should not (in general) conflict with empirical insights

• sound measurement systems reflect the level of understanding of those things that are measured

• good measurement encourages improved understanding

Page 10: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Empirical relational system

Relevantempirical

information

Formal relational system

Results

refinedmeasurement

more

sophisticated

interpretation

mathematics and

statistics

betterdecisions

and actions

real world mathematical world

Page 11: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Measurement - state of practice…

• can be self serving, but increasing awareness of role

• often not well founded, but effort is invested

• data collected but often not valued (even when useful)

Page 12: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Effective measurement require an appreciation of...

• measurement fundamentals– how do we measure?

• context– what do we need to measure?– what can we measure?

• soft (human) factors– why should I measure/be measured?

Page 13: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Measurement within the Capability Maturity Model:

Page 14: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Initial

Repeatable

Defined

Managed

OptimizingThe Capability Maturity Model

Organizational abilities are described as five levelsof increasing capability

Page 15: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Defined

maturity level key process areas

activitiescommitment

abilityverification

M & A

common features

measurement and analysis

in each KPA from levels 2 to 5

ISM

TPPR

OPD

IGC

SPE

OPF

Page 16: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Measurement within the CMM...

• is a common feature of every key process area at each described maturity level

• measurement and analysis common feature is least well described– SEI recognize need for organizations to derive their own measures– SEI say nothing about how to measure

• in practice we tend to find measurement and analysis is the last common feature to be satisfied for any given KPA– can use this as an informal assessment ‘short cut’.

Page 17: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

...Measurement within the CMM...

• When investigating measurement capability look for...

– a complementary view of quality throughout the organization

– recognition of why data is collected, and understanding of definitions and analysis

– data verification (accuracy known) and measures validated

– information used to make decisions and direct actions

Page 18: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

...Measurement within the CMM

• Level 4

– measure products and process quantitatively• Software Quality Management• Quantitative Process Management

– aspires to production engineering (cf project engineering)

– still controversial

Page 19: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Measurement within SPICE (ISO 15504)

Page 20: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

The SPICE Architecture

ORG

ENG

SUPPPROJ

CUS

Proc 4

Proc 2Proc 1

Proc 4Eng. 3

Develop S/W Design

Proc 5 54210

Practice 1Practice 2

Practice 3Eng 3.4 Establish

traceability

ProcessCapability

Welldefined

Page 21: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

A comparison of the CMM and SPICE

• SPICE...

– is more complex• this is a good thing and a bad thing

– it may be unrealistic– but does require thought

– is structured by process• ...not capability - so may make arbitrary target setting more difficult• allows all processes to be addressed to some extent (c.f. CMM’s SPE)

– has process capability defined by generic processes

Page 22: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Measurement within SPICE...

• Measures required throughout (like CMM)...

– Generic Practices (define process capability of every process)

• level 2 - plan process (I.e. estimation)

• level 2 - track with measurement

– note - “use of measurement implies measures have been defined and selected and data has been collected”

• level 3 - use well defined data

• level 4 - measurement oriented (like CMM level 4)

Page 23: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

...Measurement within SPICE...

• ...Measures required throughout (like CMM)

– Processes

• engineering 2.2 hints at ISO9126

• project 2.5 and 7.2 - size, effort,cost schedule resources risks

• project 5.2 - define quality metrics

• organization 2.9 - define process measures

• organization 2.12 - set performance expectations

• organization 3.6 - define measures of impact

Page 24: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

...Measurement within SPICE

• introduced at low capability level (like CMM)

• states expectation of what measurement requires (unlike CMM)– (even at low capability level)

• minimal guidance (examples) on what to measure (less than CMM)

• implies a distinct capability to define and assess measures

Page 25: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

Summary

• Measurement

• within the Capability Maturity Model

• within SPICE (ISO 15504)

Page 26: OXFORD SOFTWARE ENGINEERING Software Engineering Services & Consultancy

OXFORD SOFTWARE ENGINEERINGSoftware Engineering Services & Consultancy

[email protected]

+44 (0) 1993 700878

O X F O R DS O F T W A R E E N G I N E E R I N G

L I M I T E Dwww.osel.co.uk