is your metrics programme out of control
DESCRIPTION
TRANSCRIPT
![Page 1: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/1.jpg)
IS YOUR METRICS PROGRAMME OUT OF CONTROL?
Ally Gill MBCS CITP, UKSMA Conference -‐ Nov 8 2012
![Page 2: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/2.jpg)
“It is really just as bad technique to make a measurement more accurately than is necessary as it is to make it not accurately enough”
Arthur David Ritchie -‐ ScienKfic Method: An Inquiry into the Character and Validity of Natural Law (1923)
2
![Page 3: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/3.jpg)
“Just because you can measure it, it doesn’t mean you should”
Disgruntled employees the world over...
3
![Page 4: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/4.jpg)
4 4
OBJECTIVES
q Why do Metrics Programmes go out of control? q How can we recognise the symptoms? q What can we do to cure them? q Can we prevent the problems in the first place?
![Page 5: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/5.jpg)
IN THE BEGINNING
5
![Page 6: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/6.jpg)
6
IN THE BEGINNING
6
q An execuKve launches an Improvement IniKaKve using the next Silver Bullet e.g. CMMI
q A “guru” is assigned to manage the measurement programme q CMMI becomes the holy grail for measurement q Demands for data go viral because of a command and control mentality
q Chaos ensues
![Page 7: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/7.jpg)
7
AND THE RESULT OFTEN IS...
7
![Page 8: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/8.jpg)
THE GURUS HAVE TAKEN OVER THE ASYLUM
8
![Page 9: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/9.jpg)
9 9
THE GURUS HAVE TAKEN OVER THE ASYLUM q Management fails to set objecKves against which data should be collected and accept what the metrics gurus tell them they should be doing
q Everything and anything that can be measured is measured, regardless of how useful the measures are
q Measurement becomes a self fulfilling business in its own right
![Page 10: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/10.jpg)
10 10
THE GURUS HAVE TAKEN OVER THE ASYLUM q Management loses sight of the extent of the demands and costs of the data collecKon programme
q Measurement gurus hide behind the CMMI but misinterpret it q New measures are constantly added but none are reKred q Data is collected long acer anyone remembers why it was collected in the first place
![Page 11: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/11.jpg)
11 11
THE GURUS HAVE TAKEN OVER THE ASYLUM q Senior Management must retain control over the programme and regularly review the data being collected against the business objecKves
q Measures must be constantly reviewed in respect of their value and outdated measures reKred
q Management acKons and decisions based on measures must be taken and be made transparent to the organisaKon
![Page 12: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/12.jpg)
DEATH BY A THOUSAND DATA REQUESTS
12
![Page 13: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/13.jpg)
13 13
DEATH BY A THOUSAND DATA REQUESTS q Incessant demands are made to record and report data, ocen the same data in different formats, through different tools and to different people
q Data providers report the data they expect the requestors to want to see
![Page 14: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/14.jpg)
14 14
DEATH BY A THOUSAND DATA REQUESTS q Project managers and staff find increasingly ingenious ways to create “the right numbers” rather than focusing on delivery
q So much data is being collected, consolidated and reported that no-‐one can actually find Kme to analyse the informaKon and take appropriate acKons
q Metrics are seen as a unnecessary evil and the organisaKon loses any value from its metrics programme
![Page 15: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/15.jpg)
15 15
DEATH BY A THOUSAND DATA REQUESTS q Automate data collecKon methods as much as possible q Eliminate redundancies between reporKng systems q Review the cost of data collecKon systems on a regular basis and monitor the number of teams collecKng data
q Listen to the data providers and act accordingly
![Page 16: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/16.jpg)
METRICS FOR METRICS SAKE
![Page 17: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/17.jpg)
17 17
METRICS FOR METRICS SAKE q Management make decisions based on percepKon rather than facts
q Different programme/projects reviews focus on a single set of numbers -‐ usually financial and schedule based -‐ regardless of the real focus of the review board
17
![Page 18: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/18.jpg)
18 18
METRICS FOR METRICS SAKE q The cost of collecKng, storing and analysing the data outweighs any financial benefit or value from using the data to improve the business
q The feedback loop is broken and those dealing with the metrics data fail to report back to those who provide it
q The governance system may have lost it’s way and needs to be brought back into line
![Page 19: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/19.jpg)
19 19
METRICS FOR METRICS SAKE q If you don’t have the capacity to perform proper analysis of the data you collect -‐ stop collecKng it
q If you can’t explain how and why specific measures are recorded -‐ then stop recording them
q Just because you’ve always measured something, it doesn’t mean you will always have to
q If trends are stable and outliers are few and far between you may assume that a process is stable and focus on something else
q Review the governance process
![Page 20: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/20.jpg)
GARBAGE IN, GARBAGE OUT
20
![Page 21: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/21.jpg)
21 21
GARBAGE IN, GARBAGE OUT q Data providers report the data they expect the requestors to want to see
q Metrics data is misinterpreted differently by different groups of people
q Heads of departments enter into compeKKon with each other and set arbitrary targets to be met
![Page 22: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/22.jpg)
22 22
GARBAGE IN, GARBAGE OUT q There are mad flurries of PM/PMO acKvity when monthly status reports are due
q Data from last month looks remarkably similar to the data from the previous 6 months
q The numbers are always “spot-‐on” q Departments always show small period on period improvement in their reporKng
![Page 23: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/23.jpg)
23 23
GARBAGE IN, GARBAGE OUT q Check different types of data -‐ not just financial and schedule -‐ to ensure that all the numbers match the prevailing environmental condiKons
q Stop using data to compare unlike systems q Use data to improve the system as a whole not just the units within the system
![Page 24: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/24.jpg)
KEEP IN CONTROL
24
![Page 25: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/25.jpg)
25 25
KEEP IN CONTROL q Management need to manage their metrics programme before it starts to take control
q Only collect and analyse measures which will add value to the business
q Discard metrics which no longer add value q Don’t hide behind CMMI to suit a hidden agenda q Don’t use the numbers to beat up your people q Do use numbers to make the system work bejer
![Page 26: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/26.jpg)
26 26
KEEP IN CONTROL q Watch out for dysfuncKonal behaviours and take correcKve acKon as early as possible -‐ start with yourself!
q Data must be treated holisKcally not in isolaKon of other environmental factors
q Listen to staff -‐ if people are complaining about the volume of data requests they may well have a valid issue
q Validate metrics by other management techniques like “management by walking around”
![Page 27: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/27.jpg)
START OFF ON THE RIGHT FOOT
27
![Page 28: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/28.jpg)
28 28
START OFF ON THE RIGHT FOOT q MANAGEMENT
q Align your metrics to your business goals and objecKves q Review your measurement programme on a regular basis q Ensure data collected in used to improve the business
![Page 29: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/29.jpg)
29 29
START OFF ON THE RIGHT FOOT q METRICS GURUS
q Measures are not the core business they are enablers to bejer business
q Consider all stakeholder viewpoints, and don’t assume that management understand what you do any more than anyone else
q If staff are struggling to meet your data demands, there’s probably a very good reason -‐ find out what it is and fix it
q Don’t hide behind CMMI and insist it’s a “CMMI compliance thing”
![Page 30: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/30.jpg)
30 30
START OFF ON THE RIGHT FOOT q PROJECT MANAGERS & TEAMS
q Push back when duplicate or redundant data demands come your way
q Don’t make up the numbers -‐ find a bejer way to deal with the problem
q If you don’t get visibility of how the data is being used to improve your daily work find out why
![Page 31: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/31.jpg)
“...management by numerical goal is an attempt to manage without knowledge of what to do, and in fact is usually management by fear.”
Chapter 2, Page 76
31
![Page 32: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/32.jpg)
? 32
![Page 33: Is Your Metrics Programme Out of Control](https://reader034.vdocuments.us/reader034/viewer/2022051818/54964065b479593d4d8b4f2b/html5/thumbnails/33.jpg)
IS YOUR METRICS PROGRAMME OUT OF CONTROL?
Ally Gill MBCS CITP, UKSMA Conference -‐ Nov 8 2012
CONTACT DETAILS""email – [email protected]"web - www.allygill.co.uk"twitter - @allygill"
33