william krebs – ibm laurie williams, lucas layman, nc state
DESCRIPTION
Extreme Programming Evaluation Framework XP:EF. William Krebs – IBM Laurie Williams, Lucas Layman, NC State. Bill Krebs Software Transformation Consultant IBM Corporate: QSE Durham, NC USA Phone: 919 486-0238 | T/L: 526-0238 [email protected]. Evaluation Framework. Approach: - PowerPoint PPT PresentationTRANSCRIPT
Quality through Software Engineering
April, 2007 © 2007 IBM CorporationBH
Extreme Programming Evaluation FrameworkXP:EF
William Krebs – IBMLaurie Williams, Lucas Layman, NC State
Bill KrebsSoftware Transformation ConsultantIBM Corporate: QSEDurham, NC USAPhone: 919 486-0238 | T/L: [email protected]
2
Evaluation FrameworkEvaluation Framework
X P -c fC on text F ac to rs
X P -amA d h eren ce M etrics
X P -r mR esu lts M etrics
X P :E FX P E va lu a tion F ram ew ork
Goals:• Concrete• Easy to measure by small team• Unbiased
Approach:• Include objective• Include subjective
• Leading indicator• Teaches Team
• They cross check each other
3
XP:EF
| April 2007 © 2007 IBM CorporationBH
Context Factors (IBM Study)
Small team (7-10) Co-located Web development (toolkit) Supplier and customer
distributed (US and overseas)
Examined one release “old” (low XP) to the next “new” (more XP)
4
Planning PracticesPlanning Practices
1313 Stories AddedStories Added 1 1 Story RemovedStory Removed 60 60 Total StoriesTotal Stories 5 5 Month releaseMonth release 1 1 Week iterationsWeek iterations
+ + Shodan chartShodan chart
Shodan Survey
Planning
Cust Access
Short Rel
Standups
70 80 90 100
5
Testing PracticesTesting Practices
Shodan Survey
Cust Acceptnce
Auto Tests
Test First Design
40 60 80 100
46% Test Coverage46% Test Coverage (Alpha Set)(Alpha Set)
11% Test Frequency11% Test Frequency 0.45 Test to Story Ratio0.45 Test to Story Ratio 0.42 Test LOC / Src 0.42 Test LOC / Src
LOCLOC
Manual Customer Manual Customer Acceptance TestsAcceptance Tests
No Supplier No Supplier Acceptance TestsAcceptance Tests
+ Shodan at left+ Shodan at left
6
Coding PracticesCoding Practices
Pair Prog
Cont Integration
Collective Ownership
Coding Standards
Sustain Pace
Simple Design
Refactorng
Metaphor
40 60 80 100
48% Pairing Frequency48% Pairing Frequency
+ + Survey at leftSurvey at left
Be aware of pairing patterns:
Expert / ExpertExpert / NoviceNovice / Novice
7
IBM: XP-Outcome IBM: XP-Outcome MeasuresMeasures
XP Result Metric XP Result Metric Old Old New New CommentsComments
Response to Customer Response to Customer ChangeChange(Ratio (user stories in (Ratio (user stories in + out) /total)+ out) /total)
NANA 0.230.23 No basis for No basis for comparison, but comparison, but shows flexabilityshows flexability
Pre-release QualityPre-release Quality(test defects/KLOEC of (test defects/KLOEC of
code)code)
1.01.0 0.500.50 Improved!Improved!
Post-release QualityPost-release Quality(released defects/KLOEC (released defects/KLOEC
of code)of code)
1.01.0 0.610.61 Improved!Improved!
Productivity (stories / PM)Productivity (stories / PM)Relative KLOEC / PM Relative KLOEC / PM
1.01.01.01.0
1.341.341.71.7
Improved ?Improved ?Hard to measureHard to measure
Customer SatisfactionCustomer Satisfaction NANA High High GoodGood
Morale (via survey)Morale (via survey) 1.0 1.0 1.111.11 Significant? Other Significant? Other variables?variables?Moved from 57% to 72% of full use of all XP practices
8
XP:EF
| April 2007 © 2007 IBM CorporationBH
References
XP:EF spreadsheet– xpef_14.xls by William Krebs.
Concept paper– Laurie Williams, William Krebs, Lucas Layman, and Annie I. Antón,
“Toward a Framework for Evaluating Extreme Programming,” Proceedings of the 8th International Conference on Empirical Assessment in Software Engineering (EASE ’04), Edinburgh, Scotland, pp. 11-20
– http://www4.ncsu.edu/~lmlayma2/papers/WKL04.pdf