bio presentation f9

International Conference OnSoftware Testing Analysis & Review

May 16-20, 2005Orlando, FL USA

F9

5/20/2005 11:15 AM

PEANUTS & CRACKERJACKS:WHAT BASEBALL TAUGHT ME

ABOUT METRICS

Robert SabourinAmiBug.com Inc

BIOPRESENTATION

Robert Sabourin Robert Sabourin has over 20 years of management experience, leading teams of software development professionals. A well-respected member of the software engineering community, he has managed, trained, mentored, and coached hundreds of top professionals in the field and frequently writes and speaks at conferences on software engineering, SQA, testing, management, and internationalization. The author of I am a Bug!, the popular software testing children’s book, Robert is an adjunct professor of Software Engineering at McGill University.

AmiBug.Com, Inc.© Robert Sabourin, 20059-Mar-05 Slide 1

Peanuts & Crackerjack®

What baseball taught me about testing metrics.

Robert SabourinPresident & Principal consultant

AmiBug.Com, Inc.Montreal, Canada

[email protected]


“Peanuts & Crackerjack®

• Overview– Learning, Metaphors– The Players– The Coaches– The Fans– The Line Count– The Score Board– The Standings– The Big Picture


Learning and Metaphors

• Teaching• Learning• Retaining• Applying knowledge• Share experiences• Don’t get carried away …


The Players

• Baseball cards• Name & Position• Achievements

– History & highlights– Statistics & context

• Character• Gum


Base on BallsStrike outs

Hits by type

Speed of pitch

Stolen bases

Fielding ErrorsDefensive

Innings pitched

ERAPitching

Base on BallsStrike outs

HitsBatting


K-LOCsFunction points

Bugs by type

Estimate/Actual

Estimate/Actual

Agility – ContextAll defensive

Dev/Rework

Defect DensityDevelopment

Bug isolation %Show stoppers

BugsTesting


The Players

Load testerSluggerSecurity testerPinch hitterSubject MatterPinch runnerExplorerOutfieldersTest AutomationInfieldersScripted TesterRelief PitcherTest AnalystStarting Pitcher

Testing RolesBaseball Roles


The PlayersHall of Fame

Meyers

Beizer

Collard

Gelperin


The Coaches

• Team building– Hiring winners– Player Development– Farm club, Training

• Management– Strategy, Tactics, Implementation– Progress and status

• Feedback– Encouragement– Redirect


The Coaches

• Direct play by play– Swing away– Hold– Bunt– Steal a base– Sacrifice fly


The Test Lead

• Team building– Hiring winners– Tester Development– Training

• Management– Strategy, Tactics, Implementation– Progress and status

• Feedback– Breakfast of champions


The Fans

• Baseball Fans– Watch games– Buy the tickets– Eat the food– Exhibit loyalty– Provide feedback– Important stakeholders


The Baseball Fan

• Feedback– Anxiety– Excitement– Stress


The Baseball Fan

• Feedback– Applaud Progress– Cheer success– Boo the other team


Raving Fans of Testers

• Developers• Database Admin• Managers


Raving Fans of Testers

• Customers• End Users• Business Stakeholders


Baseball Line Count

• Batter up– Strikes– Balls– Outs


Testing Line Count

• Generally– Time– Bugs

– Priority, Severity• Scripted

– Confidence– Pass Fail

• Exploratory– Opportunities– On charter time %


UrgentSevere

UrgentNot Severe

Not UrgentSevere

Not UrgentNot Severe

Bug Sweet Spot


Baseball Scoreboard

• Game Progress– When– Runs– Running total


Baseball Scoreboard

• Feedback– Big, Visible, Obvious– Focus of attention– Great place for Ads!


Baseball Scoreboard

• Team Effectiveness– Meaningful– Statistics– Running totals


Testing Scoreboard

• Progress in the Project– When in project? Phase– Bugs counts by severity, priority– Objectives identified, elaborated– Exploratory Test charters identified– Tests run, passed, failed, blocked– Test Project Statistics– Testing effort done, effort to go– Running totals (of all counts)


Testing ScoreboardTest Objective Chunk Elaborated Go A B C D E F G H I J Latest Runs

Overall Stability OS001 NO FAIL FAIL PASS FAIL FAIL 3Overall Stability OS002 YES FAIL FAIL FAIL 1Overall Stability OS003 NO FAIL FAIL 0Overall Stability OS004 NO FAIL FAIL 0Overall Stability OS005 YES FAIL PASS PASS 1

Typical End User Usage Scenarios TE001 YES FAIL PASS PASS 1Typical End User Usage Scenarios TE002 YES FAIL FAIL PASS PASS 2Typical End User Usage Scenarios TE003 NO FAIL FAIL 0Typical End User Usage Scenarios TE004 NO FAIL FAIL 0Typical End User Usage Scenarios TE005 YES FAIL FAIL FAIL 1Typical End User Usage Scenarios TE006 NO FAIL FAIL 0

Typical Administrator Usage Scenarios TA001 YES FAIL FAIL FAIL 1Typical Administrator Usage Scenarios TA002 NO FAIL FAIL 0Typical Administrator Usage Scenarios TA003 YES FAIL FAIL FAIL 1Typical Administrator Usage Scenarios TA004 YES FAIL PASS PASS 1Typical Administrator Usage Scenarios TA005 NO FAIL FAIL 0Typical Administrator Usage Scenarios TA006 NO FAIL FAIL 0Typical Administrator Usage Scenarios TA007 NO FAIL FAIL 0Typical Administrator Usage Scenarios TA008 NO FAIL FAIL 0

Basic Operation of all Functions BO001 YES FAIL PASS PASS PASS 2

Data Integrity DI001 NO FAIL FAIL 0Data Integrity DI002 YES FAIL FAIL PASS PASS 2Data Integrity DI003 NO FAIL FAIL 0Data Integrity DI004 YES FAIL PASS FAIL PASS FAIL FAIL 4Data Integrity DI005 YES FAIL PASS PASS 1Data Integrity DI006 NO FAIL FAIL 0

Build Chunk Execution Status


Testing Scoreboard

Statistics Total PercentNumber System Test Chunks 26 100%Number of System Test Chunks Elaborated 12 46%Number of System Test Chunks To Elaborate 14 54%Number of System Test Chunks PASS 7 27%Number of System Test Chunks FAIL 19 73%


Testing Scoreboard

• Feedback– Big, Visible, Obvious– Focus of attention– Great place for Ads!


Testing Scoreboard

0

50

100

150

200

250

01/1

1/98

01/1

8/98

01/2

5/98

02/0

1/98

02/0

8/98

02/1

5/98

02/2

2/98

03/0

1/98

03/0

8/98

03/1

5/98

03/2

2/98

03/2

9/98

04/0

5/98

04/1

2/98

04/1

9/98

04/2

6/98

05/0

3/98

05/1

0/98

05/1

7/98

05/2

4/98

05/3

1/98

06/0

7/98

06/1

4/98

06/2

1/98

06/2

8/98

07/0

5/98

07/1

2/98

07/1

9/98

07/2

6/98

08/0

2/98

08/0

9/98

08/1

6/98

08/2

3/98

08/3

0/98

09/0

6/98

09/1

3/98

09/2

0/98

09/2

7/98

10/0

4/98

10/1

1/98

10/1

8/98

10/2

5/98

11/0

1/98

11/0

8/98

11/1

5/98

11/2

2/98

11/2

9/98

12/0

6/98

12/1

3/98

01/1

4/99

01/1

8/99

01/2

5/99

02/0

1/99

02/0

8/99

Open P1 & P2 Bugs Bugs Pending validation (Corrigé)

Trend ChartOpen Bugs


Testing Scoreboard

Trend ChartOpen Bugs By Type


Baseball Standings

• How is the team doing?– Games played– Games at hand– Total Wins– Total Loses– Relative placement– Division standing– League standing– Attendance


Testing Project Standings

• How is the project doing?– Progress toward goals

– On Time – Quality – Budget– Comparing all projects

– To go / Done effort– Estimation accuracy

– Work breakdown– Effort distributions


Baseball Big Picture

• Is the franchise healthy?– Revenues– Expenses– Profit– Loss– Contracts– Media– Civic responsibility– Capital


Baseball Big Picture

• How is the franchise doing?– Value– External factors– Supply– Demand– Opportunity– Image– Brand– Revenue


Testing Big Picture

• Is the company healthy?– Revenues– Expenses– Profit– Loss– Contracts– Media– Civic responsibility– Capital


The Big Picture

• How is testing adding value to the organization?

– More sales revenue– Fewer support calls– Lower development cost– Competitive time to market– Best of breed technology– Accountable and auditable!– Attracts top talent, Fun team– All know how testing ties to corporate values


Thank You

Questions?

bio presentation f9

Documents