tate sustainablesoftware

7/30/2019 Tate SustainableSoftware

http://slidepdf.com/reader/full/tate-sustainablesoftware 1/258



Praise fo r Kevin Tate's

Sustainable Software Development

"Over the years I have seen the software development pendulum swing

from one extreme to the other, as deficiencies in 'best practices' at one end

of the spectrum spawned a new set of 'best practices' at the opposite end.

Kevin Tate's book has finally brought the pendulum to a screeching halt,

right about dead center. This book provides a balanced and practical guide

to what's important ifyour goalis to develop softwarethat lasts."

—MaryPoppendieck, Poppendieck.LLC.

Authorof"Lean Software Development"

"1) In this very practical and accessible book interspersed with real-world

examples and personal opinions, Kevin has distilled his years of developing

quality software into a set of principles and practices that have been proven

to work. If you are thinking of introducing an agile development environ

ment (ADE) into your organization or of improving the one you already

have, this book willhelp you clearlyunderstand the benefits of a sustainable

ADE, establish the practices to make it happen and coach you through the

follow-up required to change the culture of your organization to make sure

the changes take hold.

I am currently faced with exactly this challengeand this book has already

givenme several ideas I am looking forward to tryingout.

2) In an industry plagued with missed deadlines despite long overtime

hours, this book offers a refreshing alternative: a set of guiding principles

and simplepractices to followthat allowyouto get the job done byworking

smarter, not harder. Drawing on the author's extensive experience develop

ing quality software, the book clearlyexplains the principles behind a sus

tainable agile development environment, why it works, the practices to

make it happen and the follow through required to turn these practices intohabits."

—Peter Schoeler, TechnicalDirector, ArtificialMind&Movement



"It's a familiar scene—the schedule's tight, people are putting in heroic efforts

to get everythingdone, then at the last minute a change request comes in that

wipes out the gainsyou had finally managed to make in meeting your ship date.

Looks like it's pizza at your desk for the weekend again! An unfortunate situa

tion to be in but a pattern that repeats itselfall too often. "Sustainable Software

Development"offershope to break this cycle. It showshowa changein mindset

can free you from the tyranny of unrealistic expectations and brings develop

ment realities out onto the table for everyone to see. By following these tech

niques you will be able to define and manage a software development

environment that willwork for the long haul."

—Kevin Picott



Sustainable

Software

Development



The Agile Software Development Series

Alistair Cockburn and Jim Highsmith, Series Editors

Agile software development centers on four values identified in the AgileAlliances

Manifesto:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

The development ofAgile software requires innovation and responsiveness, based on generating

and sharing knowledge within a development team and with the customer. Agile software

developers draw onthe

strengths of customers, users, and developers, findingjust

enoughprocess to balance quality and agility.

The books in The Agile Software Development Series focus on sharing the experiences ofsuch Agile developers. Individual books address individual techniques (such as Use Cases),

group techniques (such as collaborative decision making), and proven solutions to differentproblems from a variety of organizational cultures. The result is a core ofAgilebest practices

that will enrich your experience and improve your work.

Titles in th e Series:

Alistair Cockburn, SurvivingObject-OrientedProjects, ISBN0-201-49834-0

Alistair Cockburn, WritingEffective UseCases, ISBN 0-201-70225-8Lars Mathiassen, Jan Pries-Heje, and Ojelanki Ngwenyama,ImprovingSoftwareOrganizations: FromPrinciples to Practice, ISBN0-201-75820-2

Alistair Cockburn, AgileSoftwareDevelopment, ISBN0-201-69969-9

Jim Highsmith,AgileSoftwareDevelopmentEcosystems, ISBN0-201-76043-6

Steve Adolph, Paul Bramble, Alistair Cockburn, and AndyPols,PatternsforEffectiveUseCases, ISBN 0-201-72184-8

Anne Mette Jonassen Hass, ConfigurationManagement Principles and Practice, ISBN 0-321-11766-2

DSDM Consortium and Jennifer Stapleton, DSDM, SecondEdition:BusinessFocusedDevelopment, ISBN 0-321-11224-5

MaryPoppendieck and TomPoppendieck,LeanSoftware Development: AnAgile Toolkit,ISBN 0-321-15078-3

Craig Larman, Agileand IterativeDevelopment:AManager's Guide, ISBN0-131-11155-8

Jim Highsmith,Agile ProjectManagement:CreatingInnovativeProducts, ISBN0-321-21977-5

For more information visit www.awprofessional.com/series/agile



Sustainable

Software

Development

An Agile Perspective

Kevin Tate

AAddison-Wesley

Upper Saddle River,NJ • Boston •

Indianapolis • San Francisco • New York • Toronto

Montreal • London • Munich • Paris • Madrid

Cape Town • Sydney • Tokyo • Singapore • MexicoCity



The author and publisher have taken care in the preparation of this book, but make no expressed or implied war

ranty of any kind and assumeno responsibility for errors or omissions.No liabilityis assumed for incidental or con

sequential damages in connectionwith or arisingout of the useof the informationor programs contained herein.

The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales,

whichmayinclude electronic versions and/or customcoversand content particular to your business, training goals,marketingfocus,and branding interests.For more information,pleasecontact:

U. S.Corporate and Government Sales

(800) 382-3419

[email protected]

For salesoutside the U. S.,please contact:

International Sales

[email protected]

Visitus on the Web:www.awprofessional.com

Library ofCongress Cataloging-in-Publication Data:

Tate, KevinSustainablesoftwaredevelopment: and agileperspective/ KevinTate.

p. cm .

Includes bibliographical references.

ISBN 0-321-28608-1

1.Computer software—Development. I. Title

QA76.76.D47T382005

005.1—dc22

2005019120

Copyright© 2006Pearson Education, Inc.

All rights reserved. Printedin theUnited States ofAmerica. Thispublication isprotected bycopyright, andpermissionmust be obtainedfromthe publisherprior to anyprohibited reproduction, storage in a retrieval system, ortransmission in anyform or byanymeans, electronic, mechanical, photocopying, recording, or likewise. Forinfor

mation regarding permissions,write to:

Pearson Education, Inc.

Rightsand Contracts Department

One Lake Street

Upper SaddleRiver, NJ07458

ISBN 0-321-28608-1

Textprinted in theUnitedStates on recycled paperatR R Donnelley inCrawfordsville, Indiana

First printing, October, 2005



To my grandparents, Harris and Alberta Tate,whowere always thereforme.



Contents

About theAuthor xvii

Foreword xix

Introduction xxii i

Chapter 1: Sustainable Software Development 1

SustainableDevelopment 2

ChemicalManufacturingand Sustainable Development 3

Continual Improvement: The AcceleratorButton 7

The Genius of theAND versus theTyranny oftheOR 10

Summary 12

Chapter 2: Unsustainable Software Development anditsCauses 13

Technical Debt and the Flywheel 15

The Perils ofJumping in Place 16

The Causes of Unsustainable Development 17

ProjectStresses 17

UserRequirements 17

ExternalDependencies 18

Competition 18

Disruptive Technologies 18

DisruptiveBusinessModels 19

CostManagement 20

Project Controls 21

Collaboration 21

Methodology 21

Expertise 22

DecisionMaking 23

Leadership 23

Culture 23

Simplicity andReliability 23

Summary 24



Chapter3: The Principles ofSustainable Software Development 27

WhyPrinciplesAreMore Important Than Practices 29

Applying thePrinciples ofSustainable Development 31

ContinualRefinement oftheProduct andProject Practices 31

A Working Product atAll Times 32A ContinualEmphasis onDesign 33

Valuing Defect Prevention over Defect Detection 33

Culture, byDescriptive Words and Phrases 36

Disciplined 36

Responsible 37

Leadership 37

Visionary andTactical 38

Shared Sense ofUrgency 38

Highly Collaborative 38Complementary Talents andSkills 39

Continually Improving andLearning 39

Change Tolerant 39

Risk-Aware 39

Fun 40

Summary 40

Chapter 4: Working Product 41

PRACTICE 1: No "BrokenWindows" 49

PRACTICE 2: Be Uncompromising about Defects 50

How toAvoidaDefectBacklog 51

PutMore Effort intoDefect Prevention Than Bugfixing 52

Being Pragmatic aboutDefects 52

PRACTICE 3: "BarelySufficient" Documentation 53

PRACTICE 4: Continuous Integration 56

PRACTICE 5: NightlyBuilds 56

PayAttention toBuild Times 57

PRACTICE 6: Prototyping 59Throwaway Prototypes 61

"TracerBullets" 62

PRACTICE 7:Don't NeglectPerformance 62

PRACTICE 8: ZeroTolerancefor Memory and ResourceLeaks 63

PRACTICE 9: Coding Standards and Guidelines 65

CONTENTS



PRACTICE 10:Adopt Standards (Concentrate on Your

Value-Add) 67

PRACTICE 11: Internationalize from Day One 68

PRACTICE 12: Isolate Platform Dependencies 69

Summary 70

Chapter 5: DefectPrevention 73

The Role of Quality Assurance 76

PRACTICE 1:Ruthless Testing 78

Unit Tests: Test-Driven Development 80

Integration Tests 81

SystemTests 83

Record andPlayback 84

Visual Validation Testing 86

Performance Testing 87

Resource-Usage Testing 88

Regression Testing 88

Usability Testing 89

User Verification Testing 89

Resources Required forRuthless Testing 90

PRACTICE 2: Use Available Tools 92

Compiler 92

Source CodeAnalyzer 93

Test CoverageAnalysis 94

ExecutionProfiler 94

EventLogging 95

Source Code Documentation 95

Memory andResource LeakDetection 95

ConfigurationManagement 96

PRACTICE3: Pair Programming and CodeReviews 98

PRACTICE 4: LightweightRoot-Cause Analysis 100

Summary 104

Chapter6: Emphasis onDesign 107

PRACTICE 1:Design Vision 112

PRACTICE 2: Guiding Principles 112

AHypotheticalExample ofGuiding Principles 113

A Real-WorldExample ofGuiding Principles 114

CONTENTS



PRACTICE 3: SimpleDesign 115

PRACTICE 4: Refactoring 117

PRACTICE 5: Design Patterns 119

PRACTICE 6: Frequent Rapid Design Meetings 122

PRACTICE 7: Commitment to Rearchitecture 123PRACTICE 8:Design for Reuse 128

Summary 130

Chapter 7: ContinualRefinement 133

PRACTICE 1: Iterative Development 135

Collaboration andIterativeDevelopment 137

Velocity 138

Iteration 0 138

PRACTICE 2: ReleasePlanning 139PRACTICE 3: Daily Standup Meetings 141

PRACTICE 4: Retrospectives 142

PRACTICE 5: Coachingand TeamDevelopment 144

Coaching 145

Understanding theBusiness 145

Professional Development 148

PRACTICE 6:Make KeyMetricsVisible 148

Summary 150

Chapter 8: Culture Change and SustainableDevelopment 151

Making Change Happen 152

Change Factors and Enablers 155

Change Factors 156

Change Enablers 158

Start withWhat You Can Control and Influence 162

Avoid Transition Plans 162

Turning Unsustainable into Sustainable Development 163Where toBegin 164

SustainableDevelopmentfor NewProjects 166

Some Other Ideas 167

Identify Key ArchitectureBattles 167

GetYourDefects Under Control 167

CONTENTS



Collaboration with Users 168

Training 168

Group Refactoring Exercises 168

The Work Environment 169

Change Examples 170

TheIntroduction ofa ControversialPractice 170

IntroducingAgileDevelopment Into aMature

Software Organization 171

Summary 176

Appendix 1: Practice Summary 177

Appendix2: Extreme Programming and SustainableSoftware Development 183

The Four Values ofXP 183

XP's Twelve Practices 184

The Strengths of XP 185

XP is Extreme 186

Technical Excellence 186

Collaboration 186

Customer Collaboration 187

The Weaknesses ofXP 187XP Targets Software Developers, andNobody Else 188

ProjectManagement 189

Scalability 189

PairProgramming 189

The 40-Hour Week 190

The On-Site Customer 190

Self-Organization andLeadership 191

SimpleDesign 192

Common Misconceptions ofExtreme Programming 192"NoDesign" 192

uNo Discipline" 193

Other AgileDevelopment Methods 193

Summary 194

CONTENTS



Appendix 3: Sustainable SoftwareDevelopment andtheCMM 195

The CapabilityMaturityModel (CMM) 196

Agile Developmentand the CMM 197

Summary 200

Appendix 4: Recommended Reading 201

Chapters 1 and2: Sustainable SoftwareDevelopment,

Unsustainable SoftwareDevelopment 201

Chapter3:ThePrinciples ofSustainable Development 203

Chapter 4:WorkingProduct 204

Chapter 5: DefectPrevention 205

Test-Driven Development 205

Automated Testing (General) 205

Usability Testing 205

Other 206

Chapter 6:Emphasison Design 206

DesignPatterns 206

Design Patterns, Agile Development, and Simple Design 207

Refactoring 207

Other 208

Chapter 7: ContinualRefinement 208

Chapter 8:MakingCulture ChangeHappen 210

Conclusion 211

References 215

Index 219

CONTENTS



Acknowledgments

Tosay that I nowhave anappreciation for the amount ofworkthatgoes into

writing a book is an understatement. People like Pierre Berton who can

crank out enjoyable book after enjoyable book are a wonder, especially

when they can still cling to a normal life that features a job, family, friends,

and hobbies. I have a great job andworkwith fun and smart peopleand,it

goes without saying, I treasuremy family above all else, so I guess I neg

lectedhobbiesand keeping in touchwithfriends, soI amlooking forward to

gettingback to a more normal lifestyle.

I couldn't havewritten this book without the support and patienceofmywife,ClaudiaMorawetz, and mytwokids,Andrea and Heidi. Their will

ingness to giveme a few hours on weeknightsand manyweekends to work

on thisbook arewhatmade thisproject feasible. Special thanksare alsodue

to Jim Highsmith. I had the pleasure to work with Jim intermittendy

through the last fewyears, and inmanyways I creditJimwith the inspiration

to write this book. Jim always provided timely and useful feedback and

insights, and he was particularlygood at helpingme see the positiveside of

someof the negativereviews receivedas the book took shape.

There weremany reviewers who contributed to this book. I've appreciated all the feedback,whether positiveor negative. Kevin Picott, TomPop

pendieck, and Jim Markham in particular spent a great deal of time

providing mewith detailed, constructive input that was instrumental in clar

ifying the text and its presentation. Paul Petralia, the editor of this book,

helped guide me through all the usual first-time author misunderstandings

and problems, and MichelleVincentikept everythingmoving.

Fd alsolike to thank all my co-workers past and present. I've learned

something from each of you. The NetWare connectivity team applied the

principles of sustainable development which led to my rapid acceptance ofagile development: Richard Sniderman, Martin Wolf, Larry Philps, Tom

Kelly, Andrzej Szyszkowski, and Sam Vuong. Today, Fm still learning at

Alias Systems and Fd like to thank DavidWexlerfor giving me permission

and support to write this book, and Peter Mehlstaeublerfor sponsoring the

introduction of agiledevelopment atAlias. Fd especially liketo thank allthe



people who became actively involved as leaders in our change to an agile

way of working: MarkDavies (who took the first plunge with his team),

Milan Sreckovic, Kevin Picott, AniaLipka, Lisa Fung, John Gross, Gerry

Hawkins, Lynn Miller, Tom Wujec, John Schrag, Jey Michael, Theresa

Edwards, StevenWatanabe, Ian Ameline,Thomas White, Paul Tokarchuk,Martin Watt, Chris Gailey, MoiraMinoughan, BrianKramer, Cory Mogk,

Jim Craighead, Paul Breslin, Thomas Heermann, Al Lopez, LarryPhilps,

ChristinaElder,Jerome Maillot, andmanyothers.

I would alsoliketo takethisopportunityto thank the teacherswhohave

influencedand inspiredme. In highschool,Mr.Ens (math) taught mewhat

mental discipline was and Mr. Haight (history) was probably the best all-

around teacher Fve ever had. In university, Herb Yang,Jean-Paul Tremblay,

and mythesissupervisor, Ze-NianLi,wereallinfluential.

I am glad this project isover and I promise my family and friends to not

take on another big project—at least for awhile!

ACKNOWLEDGMENTS



About the Author

Kevin Tate is Chief Product Architect at Alias Systems Corp., a leading

innovator in 3D computer graphics software, custom development, and

training solutions for the film and video, games, web, interactive media,

automotive, industrial design, education, and visualization markets. At

Alias, his role encompasses development, methodology, product architec

ture, and technology strategy. He hasmorethan20years' experience in the

software development industry. Kevin is a dedicated cyclist, canoeist, and

lover of the outdoors. He lives in Toronto, Canada with his wife and two

children.



Foreword

Sustainable Software Development encompasses what I consider the two

critical concepts in agile development—building adaptable teams and

building adaptableproducts.Using theseconcepts enables teams to deliver

value to their customers in the short term and sustain the capability to

deliver additional value over time. You can have an incredibly talented agile

team—put themtoworkon a clunky, poorly designed, ineffectively tested,

spaghetti coded, legacysystem—and itwillnot be able to deliversustainable

value to customers. Conversely, you canhave a superb product in all those

dimensions—and the wrong team can turn it into a poor, unsustainableproduct, very quickly. SustainableDevelopment helps usthinkclearly about

the continuing delivery of customer value, from botha team anda product

perspective—iteration byiteration,product release byproduct release.

Three themes dominate Kevin s book—sustainability, balance, andcon

text. Sustainability means developing the capability to deliver customer

value today and tomorrow. Agility istheartofbalancing—short term versus

long term, anticipation versus adaptation, ceremony versus informality. And

finally, because every project team and every project is different, onemust

understand thespecific context ofeach project inorder toadapt practices tothat context.

Sustainable development delivers value to the customer today and

keeps thecost ofchange low inorder to deliver value in the future. In agilecircles sustainability has been applied topeople, as inmaintaining a sustainablepaceofwork. But products arealsosustainable—or not. I remembera

company whose product had an 18-month QA cycle—they reached that

"unsustainable" position a litde at a time, but the cumulative effect ofneglectwasnon-responsiveness to theirmarket.

Kevin identifies four principles of sustainability—working product,design emphasis, continual refinement, and defect prevention. In each of

these areas hethen aggregates a number ofpractices intoa coherent whole.For example, the chapter on design provides a vision of design and thenshows how a number ofpractices (not usually found aggregated together)contribute to that design vision. Practices in this chapter include guiding



principles, design patterns,refactoring, simple design, reuse, re-architecture,

and rapid designmeetings.

Sustainability dependson the art of balancing. Agility canbe definedas

the art of balancing flexibility and structure, anticipation (planning) and

adaptation. I use the term art because there is no silver bullet—no one

answer. Toillustratethis idea of balancing, Kevin uses a juggling analogy of

keeping fourballsin the air—the elements of continual refinement, working

product, designemphasis, and defectprevention.

The journey fromagile novice to agile artist requires first learning agile

practices, and then learning how to adapt them and balance one with

another. In Chapter6, "Emphasis on Design," Kevin shows us how to bal

ance the anticipation practices of guiding principles and design patterns

withthe adaptation practices of simple design and refactoring. If traditional

methodologies can be accused of overemphasizing anticipation at the

expense of adaptation, then agile methodologies canbe accused of overem

phasizing the reverse. Kevin's perspective helps practitioners find amiddle

ground—just enough, but not toomuch anticipation, combined with effec

tive adaptation. Wouldn't it bewonderful if software development was sim

ply one way or the other? But it's not. It's not simple. It requires artistic

balanceandKevin gives usaninsight intohisownbalancing artistry.

Finally, weneed to understand the factors that helpusmakebalancing

decisions. These factors provide the context for making project decisions

and adapting agile practices. In their book, Shared Voyage: Learning and

Unlearning from Remarkable Projects, Alexander Laufer, Todd Post, and

Edward Hoffman relate storiesfrom several remarkable projects including

the Advance Composition Explorer project at NASA and the Joint Air-to-

Surface Standoff Missile project for the U.S. Air Force. This remarkably

agile-like bookshows how agile principles can be applied to aerospace proj

ects. Oneofthethree prime objectives ofthestudy thatresulted inthisbook

was, "to enhance awareness to the different contexts surrounding different

projects." HowtheNASA andAir Force teams applied practices andprinciples was context specific. Kevin provides uswith key insights intoadapt

ing agile software development practices to specific situations, to specificcon t ex t s .

Kevin was practicing agile techniques before they acquired the agile

moniker, and his lengthy experience in software product development

shines throughin thisbook. Aggregating practices into his four categories

FOREWORD



provides great benefit and the material is immeasurably strengthened by

Kevin's reallife,practical experiences that manifest themselves through his

fascinating stories,tips, and examples.

I met Kevin about five years agowhenhewas instrumentalin bringing

agile practices to his company. It turned out to be one of my favorite agileenablement experiences, due in great measure to Kevin's leadership of the

change process. Chapter 7, "Continual Refinement," delves into change

practices fromthe perspective of a practitionerwhohasled theway.

There are two categories of authors in the agile movement. First, there

are those like Kent Beck whose groundbreaking Extreme Programming

Explained helped launch the movement into our everyday consciousness.

Second, there are those like Mike Cohn whose wonderful book UserStories

Applied extends our understanding of key principles and practices. Kevin's

book lies in this second category—taking things we thought weknew and

expanding them in new ways thatmake us think, "Wow, I wish I'd thought

of that." To paraphrase LukeHohmann {Journey of the Software Profes

sional), enjoy your journeywitha uniquesoftware professional.

Jim Highsmith

Flagstaff,Arizona

May 2005

FOREWORD



negative statement, it implies that programmers can't write good software

and don't payattention to architecture, and that buildersdo.Thisis a slight

to professional programmers, on the one hand,while beingoverly generous

to builders on the other.

I ama bigbeliever in thepowerofmetaphor,

especially in the contextoftechnology. Sometimes, when faced witha highly technical term or feature

to develop, it is incredibly valuable to use a metaphor that ties the term or

feature into the realworld through a simplecomparison that everyonecan

understand. This is used in Extreme Programming, not to mention in many

other technical fieldssuch as usability analysis.

My real objection to Weinberg's Law is that it implies that a valid

metaphor for software development is architectural engineering (i.e., build

ing a building). It is astonishing howmany people, from users to managers

to software developers, believe this isavalid metaphor for software develop

ment when asked. Certainly, it is tempting to think that software is like

building a building, and frankly what software developer, team leader, or

manager wouldn't want to use the inspiration ofFrank Lloyd Wright or the

Figure 1-1

These buildings show different architectural styles and through their style and aesthetics also demonstrate thetemptation to use buildings as ametaphor for software development. Left image ©Kathy McCallum, right

image ©Steve Lovegrove. Imagesfrom BigStockPhoto.com.

INTRODUCTION



Searstower in Chicagoas a metaphor for hisor herwork?Well-architected

buildings are certainly interesting to lookat, andexceptional buildings helpdefine cities.

Why isn't building a building a valid metaphor for software develop

ment? The short answer to thisquestion is that buildings are largely static

and software is not.Buildings arevery expensive tomodify after they have

beenbuilt,and indeedthe entireprocess usedto design and construct the

building isintended tomaximize the chances thatthebuilding will meet the

customer's requirements and will, within some margin of error, staywithin

thedesired budget. It'seasy to identifywhen abuilding iscomplete, but suc

cessful software never is. Software is cheap to modify, and any successful

piece of software isvirtually guaranteed to change over its lifetime. Actually,

the software industry has trained its users intobelieving thatmajor modifi

cations arepossible, if not inevitable andexpected.

Good software developers realize that theirprograms canbemodified

to add just about any unanticipated feature—of course, there are limits.

Changing aweb browser into adatabase ora scientific visualization applicationintoaword processor might involve huge rewrites, but the key thing is

thatgood software programs with good developers can do just about anythingtheywant. "It's onlysoftware" is a common mantra.

Buildings, on the other hand, are another matter. Ask an architect to

make an unanticipated change and chances are she'll reply that the onlyrecourse is to tear the building down and start over. Ask an architect to

design a building that will handle all the likely configurations and he'llrefuse theproject. This isbecause there are only a limited number ofpossible changes thearchitect and customers can make to the building over time.In software, unanticipated features are highly likely because software is

developed in a highly complex, hard to understand, technical, and con-

standy evolving environment. After five years of use andmodification, the

source code for a successful software program is often completely unrecognizable from its original form, while a successful building after five years isvirtually untouched.

These differences alone should beenough to convince you thatsoftwareisnot like "building a building" at all. And yet, the methodology taught inschool to software developers, themethodology used inmost software companies, andthemethodology described inthebulk of today's software literature is exacdy the same basic methodology, that is, used in buildingconstruction and other forms of engineering; the so-called "waterfall"

INTRODUCTION



Why does the metaphor we use to think about software development

matter? Metaphors help us analyze a problem from a unique and common

perspective. Good metaphorshelp groupsof peoplegrasp a complex con

cept or problem easily and clearly to understand what they are trying to

build. In the software industry, we have been using the wrong metaphor,

largely unconsciously, and have adopted practices from engineering that

reinforce thismetaphor.

Too many development teams develop software products asif theywere

building a building. They place toomuch emphasis on features of the build

ing and trying to completely understand its requirements before it is built.

The temptation is to finish the project or product and then to stand back

and admire the creation. Inevitably, however, unanticipated changes come

along in the form ofuser requests, bug fixes, and changes to othersoftware

or hardware that the program is dependent upon. The result is that over

time, doors get added in odd places, walls get thinner and are moved

around, cables getrun throughelevator shafts, andholes getadded betweenfloors or rooms. Amess accumulates that becomes increasingly hard tomanage, and the greater the mess, the easier the temptation to add to it. Add in

competitive pressures and a constantly changing ecosystem of underlying

technology (operating systems, web browsers, databases, UI toolkits, etc.)and chaos ensues.

INTRODUCTION

Figure 1-2

Nature provides many

useful metaphors for

sof tware and sof t

ware architecture that

accurately reflect the

nature of software

development. Suc

cessfu l software is

like a cora l reef

because it inhabits a

complex and continu

allyevolving ecosys

tem of both

complementary and

competing technolo

gies and software.

Image©Jason Sah.Image from BigStock-

Photo.com.



opposed to anemphasis on the documentsasthe guarantor of success. Byits

nature, this relies on the people doing the work to be self-disciplined and

professional asopposed to thinkingthat a processwith its mandatory docu

mentationapplied to that groupwillprovide the necessary results. This is a

roundabout wayof statingthat agile developmentand leanmethods emphasize flexibility and collaboration over stifling bureaucracy to minimize

wastedeffort.Andthisiswhyagile and leanpracticesareasmuchabout cul

ture change as they are about practices.

Where This Book Fits

I wanted towrite this book because I see a need for it. Quite a bit of the agile

development literature focuses on project management practices that pro

mote agility. Projectmanagement practices are,of course, vital. But just as

vital is the realization that in order to be agile, your softwareitself and the

methods you use to develop it had better support your ability to be agile.

You can't be agile if yoursoftware is britde and is difficult to change with

too few safeguards in place to enhance confidence in the changes. Software

project management anddevelopment practices andsoftware mustgohand

i n hand .

I also feelthat toomany software projects focus exclusively on features

andbugfixing andnotenough onexcellence. Software quality isan issue for

our industry, but wetoo easily accept it as inevitable and somehow a result

of the need for constant change. But as software becomes increasingly

important in our daily lives, issues like security and quality are becoming

more urgent. I think it'stime we find ways to achieve features AND quality,

not justview themasexclusive to eachother.

AsI seeit, thisbook falls into abit of an awkward spot.Whilethisbook

is heavily influenced by the agile development community, and Extreme

Programming (XP) inparticular, agile inmy opinion leaves toomuch to the

imagination of the reader/listener, and this leads to many misconceptions.Some of these misconceptions are dangerous and very false—for example,

that in XP there is no software design and that teams who practice agile

development are somehow less disciplined. What I have tried to do in this

book is expand on the existing agile literature by drawing on my work

INTRODUCTION



of an elegant design whena pragmatic approach ("ifitworks, don't change

it") is often in the better long-terminterestsof the project.I also have seen

manyteamswhofocuson creating the best possiblesoftwareor,evenworse,

spendingan endlessamountof timewritingrequirements and design docu

mentsoftenwithNO software andlosing

theirfocus

onwhatevery project's

sole aimshould be: shipping!

Thisbook is for software developers, managers, and users who believe

in self-discipline, professionalism, and in shipping software products with

high quality that last. Especially for those who are looking for ways to

embrace change and increase efficiency, while continually improving them

selves, their teams, and their organizations.

INTRODUCTION



asdefects and escalations reportedbycustomers. The software isboth brittle

and fragile as a result of factors suchas over- (or under-) design, a code first

then fix defects later {code-then-fix) mentality, toomanydependenciesbetween

codemodules, thelackof safeguards suchasautomated tests, and supposedly

temporary patches orworkarounds that arenever addressed. Theseareproj

ectsthatareunknowingly practicing unsustainable development.

In unsustainable development, teams are primarily reactive to changes

in their ecosystem. By and large, these teams are caughtin a vicious cycle of

reacting to events andworking harderand longer hours akin to beingon a

treadmill orwalking up a down escalator. Theresultis a projectdeath spiral,

where the rapidity of descent depends on the amount of complexity faced

bythe teamanditsprinciples andpractices anddiscipline.

In sustainable development, teams areableto ho. proactive about changes

in their ecosystem. Their ability to be proactive isenabledby their attention

to doing thework thatisofthehighest value to customers withhigh quality

and reliability and an eye towardcontinual improvement despiteincreasing

complexity. These teams areina virtuous cycle, where themore team isable

to improve themselves andhow they work together, the greater theirability

to dealwith increasingcomplexity and change.

Underlying sustainable development is a mindset that the team is in it

for the long haul. The team adopts and fosters principles and practices that

help them continually increase their efficiency, so that as the project gets

larger and more complex and customer demands increase, the team can

continue at the same pacewhile keeping quality highand sanity intact. They

do this bycontinually minimizing complexity, revisiting theirplans, andpay

ing attention tothehealth oftheir software and itsability to support change.

Sustainable Development

Sustainable development is amindset (principles) and an accompanying set

ofpractices thatenable a team to achieve and maintain anoptimal development pace indefinitely. Note that optimal doesn't mean fastest—that would

be pure coding, suchasfor a prototype.

Sustainable development isabout efficiency andbalancing theneeds of

the short and long term. It means doing justthe right amount ofworkto

SUSTAINABLE SOFTWARE DEVELOPMENT



WORKING HARDER

Actua l Per formance

-T ime

Ef for t

Time Spent Improving l

Time SpentWorking

-T ime

Capability

-Time

WORKING SMARTER

Ac tua l Pe r formance

-T ime

Effor t

A

Time Spent Improving

Time SpentWorking

-T ime

Capability

-T ime

fewer people available, the less the ability oftheplant to produce output.And so it goes.

Parking LotManagers

I believe there aretoomany people in the software industry, managersespecially, who judge the morale or productivity of their company by

howmanyhours their employees work on evenings andweekends onaregular basis. I call these people parking lotmanagers because they're

often proud of the fact that their company's parking lot is still full at

midnight and onweekends. However, very few ofthese managers realize that full parking lot effort is not sustainable, thatworking harder

may bevalid in the short-term when a concerted effort is required, but

itis definitely notin the best long-term interests ofthecompany.

CHEMICAL MANUFACTURING AND SUSTAINABLE DEVELOPMENT

Figure1-2

Working harder

(more hours) results

in decliningcapability

over time.Working

smarter, with anemphasis on contin

ual improvement,

leads to increasing

capability. From

[Repenningand Ster-

man2001].



adoptionor a paradigm shift is about to occur, willyouneed to completely

replace the factory.

Eachof the following questions examines someof the parallels between

chemicalmanufacturingplants and softwaredevelopment.

How manydevelopers viewwriting test code,building testability intotheircode,

enhancing a test framework, or cleaning up their product's build system as

nonessential to theirwork and hence something that lower paid people should

be doing?

Somehow,manydevelopers haveamindsetthat non-featureworkisnot

part of their job.Thisisnot only preposterous but also extremely dangerous

and is akin to a mechanic at a chemical plant believing the basicupkeep of

his tools andworkshop isnot part ofhis job.If developers don't write auto

mated tests, then chances are the culture of the company is focused on

defect detection, not defect prevention. As shown in Chapter 5, this is the

worstpossible situation a software company canget itselfinto because it is

extremely expensive andwasteful, withmany defects being found by cus

tomers (which is the mostexpensive scenario of all). Likewise, if developers

don't personally payattention to theinfrastructural details oftheirproducts

(such asbuild and configuration management systems), it is alltoo easy for

problems to creep inthateventually impact key performance indicators that

also impact developer's productivity. Examples arebuild times, code bloat,

source tree organization, and improper dependencies between build units

creeping into the product.

How many managers refer to non-feature work as a fax that is unessential to

productdevelopment?

If a company is in a feature warwith its competition, the company's

management needs to understand how the decisions and statements they

make ultimately impact the long-term success of the company. Managers

must recognize that they have a key role in leading the attitude of theorganization. In this case, management, or even in many cases software

developers themselves, needto realize thatthe only taxon the organization

is their attitude about non-feature work. As in chemical manufacturing

plants, itiscounterintuitive thatongoing investment incontinual improvement

SUSTAINABLE SOFTWARE DEVELOPMENT



Summary

Developing software is a complex undertaking that isperformed in anenvi

ronmentof constant change and uncertainty. In the Introduction, I likened

this to a coral reef, not only because of the complexity of the softwareecosystem andtheneedfor constant change, but also because ofthe fragility

of existence. It is very hard to build or inhabit a software ecosystem that

thrives over the long term.

Very litde software iswritten once, installed, and then never changed

over the course of its lifetime. And yet, the most prevalent development

practices usedin the industry treatchange asan afterthought. Competition,

the changing ecosystem, andthe fact that users (and society in general) are

becoming increasingly reliant on software, ensure that the software must

change and evolve over time. The resulting combination of increasing complexity, need forchange, anddesire to control costs isavolatile onebecause

very few software organizations and teams areequipped with themindset,

discipline, and practices to bothmanage and respond to complexity and

change.

The answer to all the stresses placed on software organizations and

teams lies in sustainable development, which is the ability to maintain an

optimal pace of development indefinitely. In sustainable development,

teams are able tobe proactive about changes in theirecosystem. Theirabil

ityto beproactive is enabled by their attention to doing thework thatisofthe highest value to customers with high quality and reliability and an eye

toward continual improvement despite increasing complexity. These teams

are in a virtuous cycle, where the more the team is ableto improve them

selves and how they work together, the greater their ability to deal with

increasingcomplexity and change.

The next chapter describes unsustainable development and its causes.

Thisis important to understand before considering howto achieve sustain-

ability.

12 • SUSTAINABLE SOFTWARE DEVELOPMENT



Chapter 2

Unsustainable

Software

Developmentand its Causes

Unsustainable development, asdepictedin Figure 2-1, isan ail-too common

situation today in the software industry. Most software teams place too

much of a short-term emphasis on feature development and fixing defects

and do not pay enough attention to the health of the underlying software.

The result is software with a high cost of change, that is increasinglyunmaintainable, and where every change has the risk of destabilizing theproduct.

In unsustainable development, teams tend to spendan ever-increasing

amount of time fixing defects and stabilizing the software. Features still get

developed, but less time is available for feature development due to the

amount of timerequired to stabilize the software, whichincreases with each

new release. This results in teams tending to become change-adverse while

they are stabilizing theproductbecause each change increases the risk that

something will break. Teams tend to desire freezing of requirements asearlyaspossible so they can getsome work completed. This reduces theirability

to respond to customer requests and changes to underlying hardware or

software because theyare spending toomuchtimeon stabilization,which is

wastedeffort,and not enoughonnewwork.



Figure2-1Unsustainable devel

opment is character

izedbya constantly

increasing cost of

change to the soft

ware. The usual evi

dence ofa high cost

of change is a con

stantly increasing

number of defects.

Eachchange adds

complexityand

uncove r s o r c a us e s

defects that require

more changes, and

this complexityleads

to a declining abilityto respond to cus

tomer requests and

changes to the

ecosystem.

• 14 •

Ability to Respond

Defec ts

Cost of Change

T lme -

The symptoms of a short-term oudook are only evident oncea thresh

old has been crossed. Thisis the pointwhere the developmentteamstarts to

spendsomuch time trying to keep upwith the defect backlog that theyarenoticeably unable to spend enough time to develop new features. It may

take only a few months or many years to reach this point.Many develop

ment teams maynot even be aware howbad things are: Release cyclesmight

becomedeath marches, customers, management, and sales force complaints

might getlouder, anddevelopers themselves will startblaming theproblems

onmanagement ("ifonly they had listened tome...") and other developers

("they produce toomuch buggy code..."). It may takea while, but morale

will eventually suffer then plummet; in extreme cases, peoplewill losetheir

jobs through being fired, layoffs, or outsourcing, and customers will losepatiencewith the product and company.

Luckily, the extremeis seldom reached. Unluckily, most project teams

live in a gray area thatisnot sustainable development but is uncomfortably

close to unsustainability. These projects have not reached the threshold

where the problem is obvious; developers feel like they are barely keeping

up or are making slow progress butwith toomuch stress. This isthe software development death spiral, wherethe onlyvariable betweenunsustain

ableprojects is the rapidity ofthe descent towardobvious unsustainability.

Unsustainable Development and the Boiled Frog

If you areworking ona project onthe unsustainable development path,you may not beaware of it. Think ofa frog and a pot ofwater. If you

UNSUSTAINABLE SOFTWARE DEVELOPMENT AND ITS CAUSES



The Perils of Jumping in Place

One strong piece of evidence for technical debt in the software industry is

the number of products that must be completely rebuilt in order to keep

up with competitive products or to respond to a major technologychange. Completely rewriting a product requires a massive investment in

effort and time, and the result is most often a loss of market share while

the next generation (replacement) product is built. These companies are

building new products tojumpinplace, to address the same marketwith a

similar product! Even if the product is truly better, the loss of market

share and customer mindshare is not easily overcome, especially since

existing competitors will have gotten stronger and new competitors

entered the market during the time required to build the next generation

product.An extreme example of losingmarket share when jumping in place is

best dubbed the Osborne effect after Osborne Computer, Inc. Osborne

produced one of the first portable computers in the 1980s.It was the sizeof

a small suitcaseand could barelyfit under an airplane seat, yet it revolution

ized personal computers at the time because it came preinstalled with use

ful software and was portable to some degree, something that is taken for

granted today, largely thanks to Osborne. Not many users took advantage

of the portability of the computers due to their size and weight, but it

didn't matter because of the value of the system and its preinstalled software. Osborne had a successful computer on the market that was selling

well. Then, for some reason, it decided to build a replacement rather than

refining its current product. Osborne compounded its mistake by prean-

nouncing the next generation of its computer with a larger screen, faster

processor, etc. This was a huge mistake; customers stopped buying

Osborne computers to wait for the new generation, cash got tight, the new

product fellbehind schedule, and soonOsborne Computer was in Chapter

11bankruptcy.

Bankruptcyis the extremeend of a spectrum of lost market share.Lostmarket share in any form requireseffort to get back to the former position,

and this is wasted effort that technology companies must be diligent to

avoid.

16 • UNSUSTAINABLE SOFTWARE DEVELOPMENT AND ITS CAUSES



The Causes of Unsustainable

Development

Whyisunsustainabledevelopmentso commonin the softwareindustry?It's

becausesoftwaredevelopmentis a complexundertaking. Figure2-2depicts

someof the factors that contribute to the complexity. There are a number of

project stresses, which you can't control, that are tryingto push your project

toward unsustainability. Your onlydefense is to employa number of things

that you can control, or at least influence, to counteract the stresses and

keep the project on a sustainablepath. Get one or more of thesewrong and

your project is going to start into the death spiral. The project stresses are

going to be different for every project, and over time the stresses are bound

to change.Therefore, constant diligence is required on yourpart to keep the

balance.

Project Stresses

User Requirements

User requirements continually evolveand change. It is virtually impossible

to predict what users are going to require more than some small number of

weeks to months in t he future.

Project Stresses

User Requirements

External Dependencies

Competition

DisruptiveTechnologies

Disruptive Business Models

Cost Management

Project Controls

Collaboration

Methodology

Expertise

Decision Making

Leadership

Culture

Simplicity

THE CAUSES OF UNSUSTAINABLE DEVELOPMENT

Figure2-2

Software projectsare subjected to a

number of project

stresses that can' t

be controlled. These

stresses ar e a co n

stantly changingset

of forces that push

your projecttowards

unsustainability. You

can employ a number

of defenses that you

can control or influ

ence to try and keep

the projecton a sus

tainable path.

17 •



External Dependencies

Today's software depends on operatingsystems, libraries,third-party com

ponents, etc.Software usually also dependson computerhardwareor hard

ware interfaces (especially if it's embedded). The steady advance of

technology means that changes to these interfaces are a given. Managing

these changesis a challenge and requiresconstant diligence and foresight. A

change to anyof thesemight set a software teamback for days, weeks, or

months as the new versionof the operating system, compiler, devicedriver,

libraries, web browser component, etc. is integrated and tested. And for

many software teams, especially where it is impossible to dictate the end

user's computingenvironment, the teammust support the old andthe new

interfaces. This adds to development complexity and to the risk that the

softwarewillbreak in unexpected ways.

Competition

It is relatively easyto setup a softwareorganizationtodaybecause the barri

ers to entry are low. All that is needed are some bright people, a few com

puters, and a connectionto the Internet to get started.

Unfortunately, what many companieslack is a workable business plan.

They do not havea clearidea of their true market worth because they over

estimate or do not know their market size, and they almost always suffer

from an inability to adequately sell and market their work. They may be

good at gettinginitialfundingandmaking someinitialsales, but most soft

ware organizations struggle to stay in business. Their very existence and

sometimes more innovative products make it difficult for other companies

to thrive.

Disruptive Technologies

The constant advance of technology means there is a need for continual

innovation,or at least a need to keep up. Given that products need tomeet

users' demands over the long term, it is tempting to think that in everycase

it makes perfect sense to always do only what your customer wants you to

18 • UNSUSTAINABLE SOFTWARE DEVELOPMENT AND ITS CAUSES



Project Controls

Collaboration

Working effectively with other people in your organization and your customers is a crucial aspect of software development. If people are off doing

their own thing, then the project as a whole suffers. Good things happen

when people cancooperativelywork together toward a commonobjective.

Methodology

The development methodology, the principles (mindset) and practices

(project planning, design, testing, coding, etc.) you use, has a dramatic

impact on the success of a project. Methodology directly influences effi

ciency, change tolerance, responsiveness, and softwarequality.

Asdescribed in the Introduction, an important factor is that we use the

wrong metaphor for software development (that software development is

like building a building). Because the metaphor is incorrect, teams either

approach software development using plan-driven development, because

that is what is taught to them in school as the approach used by engineers,

or,ifthat doesn'twork, theyusevariants ofanad-hoccode-then-fix approach.

Software teams who approach software development with litde or no

discipline commonlyemploy code-then-fix. Sometimes this is a reaction to

practices that do not work for their project, and sometimes it is a lack of

focuson technicalexcellence. Theseteamsspendmost of their timewriting

software and verylittle time on planningand analysis. Software developed

thiswayis fragile and britde andwill resist change because evenifgoodcod

ing practices are followed, the lackof a set of sound practiceswill lead to a

great dealofdead code and thrashing.

Plan-driven development is derived fromengineering as it is applied to

the design and constructionof bridges and buildings. It is best typified by

the ISO-9000setof standards. The plan-drivenapproach iswhat is taught inschools to software developersbecause there is the mistakenbelief that all

the complexities in software and the target ecosystem can be understood

before any code iswritten. This is rarelytrue over the short term and defi

nitely false over the longer term.

TH E CAUSES OF UNSUSTAINABLE DEVELOPMENT



unsustainabledevelopmentencompasses a largegrayareaof decliningcapa

bilitywith a variablepace of decline.Asa consequence,unsustainabledevel

opment may not be immediately evident, evento the people on the project.

Unsustainable development results from a complex ecosystem of con

tinual changeand stresses that are largely out of the control of an organization. Therefore, in order to achieve sustainability, the organization must

recognize its stressesand continuallyadjust its defensesand copingmecha

nisms.

The next chapter returns to the theme of sustainability. It starts by

describing the difference betweenprinciples andpractices and thenoutlines

some of the keyprinciples that lead to sustainability. These principles are

used to frame the practices described in the rest of the book, because sus

tainable development beginswith principles (mindset) and uses practices

that reinforce them.

SUMMARY • 25



length of time. Not just their code either—with personnel turnover there

comes an increasingly largebodyof code that nobodyon the current team

wr o t e .

Its unrealistic to expect the education system to duplicate real-world

conditions. This is because the education system needs to teach students

their craft, which must be taught in a controlled environment so students

aren't overwhelmed. But knowinga craft isonlyone part ofwhat isneeded.

In order to provide a bridge from the academic to the practicalworlds,we

need a mindset that can be used to create a culture of software development.

This should be the culture of sustainable development so that teams can

achieve and sustaina highpaceof development over the long term, despite

all the surrounding complexity, without heroics.Otherwise, it's too easyto

burn out on the profession.

Without the right mindset and culture, too many people try and fail

with the methods theyare taught. Theythen fallback on ad-hocvariants of

the traditional linear approaches because the linearityprovides a degree of

comfort that progress is being madewhile alsobeing familiar. But by being

ad-hoc, there is either goingto be too litde structure (whichis code-then-fix

development) or too much, which applies too much bureaucracy in an

attempt to keep the process under control. And bureaucracy tends to stifle

innovation, flexibility, and change tolerance—the very properties that a

dynamicand changingenvironmentrequires.

The bulk of this book, and most books on software development, arefilled with practicesand rules. Practices are too often treated as templates

for success, when the reality is that practices must be tuned to the situation

and team. And rulesare a result of bureaucracythat is put in place to try to

make software development more predictable. Predictability is often

achieved, but at the cost of innovation and the ability to respond to cus

tomer requests. If your teamimplementsa givenset of practices but doesn't

have the right mindset or culture that supports the practices, you're not

going to get far. You will be better off than doing purely ad-hoc software

development, where youwillbe goodbut not great.

Before your team talks about development practices you need to first

focus on the principles you are going to follow. These principles should

guide you every day in making tradeoffs and in fostering the right mindset

and culture for your team and organization. Thepractices will follow from

the principles you choose to adopt.

28 • THE PRINCIPLES OF SUSTAINABLE SOFTWARE DEVELOPMENT



Why Principles Are More ImportantThan Practices

The difference between principles and practices is analogous to the differ

encebetween vision and features in a project. Unsuccessful projects startby

defining a set of features that whenput togetherare surmised to meetsome

userneed.Theseprojects are technology looking for amarket, and the odds

of their success are low. Successful projects, on the other hand, start with a

clear user needand a vision ofhowtomeetthat need. The vision is simple

and canbeusedto develop aninitial setof features and a prototypethat can

be put in front of customers for feedback. The feedback isused to refine the

feature set,which is then put in front of customers for more feedback, and

soon.This is iterative development, asit is called in agile software develop

ment. The following are key aspects in the vision-oriented approach to

product development:

• A user need and vision so it is clearwhat is being built

• Rapid refinement to adapt to change

• A close relationshipwith users so theycanprovide timelyfeedback

• Continual learning to refine the product so it best meets user needs

while avoidingunnecessaryfeatures

Too many people are looking for a magic set of rules or list of practices

that define the silver bullet of software development. Our education and

upbringing reinforce this desire by using rules and step-by-step proce

dures to help us solve problems. We run into trouble, however, because we

rarely recognize the role of complexity in the application of rules. Rules

work best when the problem is simple but break down as complexity

increases. But software development, and product development in gen

eral, is not simple.

In sustainable software development, the principles are like vision andthe practices are like features. Principles definewhat you want to accom

plish and the practices howyouwant to go about it. Toomuch of the mantra

in softwaredevelopment is around setsofpractices.Wehear "followthis set

ofpractices" or "if you aren't following these practices,youaren't using this

WHY PRINC IP LE S ARE MORE IMPORTANT THAN PRACTICES • 29



technique." If only itwere that simple! These practice-oriented approaches

put the cart before the horse by defining the features before the vision.

While there isno doubt that in every case thepractices arevaluable and can

be learned from, ifyoudon't knowwhat youare tryingto achieve, then your

efforts are going to be misdirected.The problemwith practices and rules in the face of complexity, as in

software development, is that it is too easy (and human) to lose sight of

whereyouaregoing andwhatyouwantto accomplish, getboggeddown in

the details, and flail around. The most common response to flailing around

is to applybureaucracy, to define a setofstrict rulesand milestones that can

not be deviatedfrom. But bureaucracy stifles innovation and is most often

misguided, leading to unintended side effects. It is also inflexible and

change-intolerant.

Some practice-oriented approaches provide a simple set of practices

that are intended to lead to sustainable development. But, by stating their

approach as a set of hard-and-fast rules, theymislead people into believing

they are complete, when in fact they are not. Good teams apply these prac

tices and can achieve success,but the chance of failure is still high because

of the complexities inherent in software development and the people who

developit. Also, bystatingthat the practicesmust be adhered to, a different

sort of bureaucracy or elitism develops where people say silly things like: "I

won't work more than 40 hours this week because that is one of our prac

tices," even though there is a critical deadline in the near future, or "We

never design aspects of our architecture because we only design what we

need today," eventhough it isplainlyobvious that a bit of extra work today

would make tomorrow's work, that everyone recognizesneeds to be done,

an order ofmagnitude easier.

Great teams recognize that what they are trying to achieve is more

important than how theyachieveit. Theylisten to the practices that are pro

posed to them, ask questions to understand what they are trying to accom

plish, and then apply them as needed. Great teams see through the strict

rule-based fagade and are successful because they see rules as a solid foun

dation to build on. They work from a clear user need (high user value

through features and high quality)and a vision (the principles) and continu

ally learn and refine their approach (the practices) as their project pro

gresses. This is what underlies sustainable development. To summarize,

there are three elements to sustainable development:




A Continual Emphasis on Design

Design is a key differentiator between products. Both good design andbaddesign are immediately obvious and can heavily influence purchasing deci

sions, but themost important aspect ofdesign quality in terms ofsustainable

development isontheproductivity ofthedevelopment team. Design deci

sions andchanges aremade every day andbyevery member ofthe team, and

theteam needs to allow itself tomake design changes as appropriate throughout the project. Traditional software development emphasizes trying to

design the software to anticipate every possible future use atthebeginningoftheproject orpriorto thewriting ofany code. This does notwork. Agile

software development advocates designing only what you need today while

not making sillydecisions that close the door on future enhancements. Con

trary towhat some people believe, agile development actually requires alot

ofdesign, perhaps even more than traditional approaches.

Acontinual emphasis on design is required forsustainable development

because gooddesign and sustainable design practices extendthe life of the

product by keeping the implementation in a healthy state that enhances

maintainability and changeability while ensuring thereisa simple long-term

design vision.

Valuing Defect Prevention over Defect Detection

Defect detection involves trying to find and fix defects after changes have

beenmergedinto the production software. Defectprevention, on the other

hand, emphasizes catching defects before the changes are merged into the

production software.

The key symptoms of a defect detection culture are an undue reliance

on defect tracking systems, manual testing efforts, and large numbers of

defects that reach customers.

In the typical defect detection organization, organizations invest a great

deal in a QA or testing organization. Theyhave elaborate defect trackingsystems in which members of the organization spend coundesshours sort

ing, prioritizing, and assigning defects for fixing. And yet regressions

(defects fixed or features introduced in previous versions of the product

which no longer work) are still introduced in new product versions as

APPLYING THE PRINCIPLES OF SUSTAINABLE DEVELOPMENT • 33



shape, weight, color, and texture,and that you are ableto focus on any

one ballwhile keeping all four in focus.

The larger balls that represent features and bugfixing are com

pletelydifferent inshape and weightfrom the small balls.The complexityof the task ofjugglingthe four small ballsand two large balls usually

leads teams to juggle onlya few balls because that's much easier.

The team may juggle just the large balls. This leads either to the

purely ad-hoc code-then-fix approach to software development or

bureaucracy, consisting of milestones, rules, and mandatory docu

ments. Unfortunately, these approaches ultimately lead to unsustain

able development.

Or the team might juggle one or more of the small balls so they

don't have to deal with the large balls. They'll produce awesome

designs or have a working product or continually refactor their code,

but theywon't get anything done.And if theymanageto ship and have

customers, chances are the customers won't be satisfied.

Great software teams are able to juggle all six balls. They under

stand that juggling the small balls comes first because they are more

APPLYING THE PRINCIPLES OF SUSTAINABLE DEVELOPMENT

Figure 3-1

Software develop

ment is like juggling

the four balls of sus

tainable development

at th e same time as

the much larger balls

that represent fea

tures and bugfixing.

Instead of just jug

gling the two large

balls, sustainable

software development

recognizes the genius

of th e AND an d

results inthe ability to

juggle all six balls at

the same time, eventhough they are dif

f er en t s izes and

weights.

35



predictable, and this makes the task of juggling the large balls mucheasier. This ability to accomplish the genius of the AND in the face of

the tyranny ofthe OR leads to sustainable development while capturing

the complexities required to do so.

Culture, by Descriptive Words

and Phrases

Sustainable software development requires a certain kindof culture. While

it is hard to completely describe the desired culture, it is possible to help

focus onwhatthat culture isbyusing some descriptivewordsand phrases.

Disciplined

Theteammustbe highly disciplined in itsapproach andgoalfocused. Disci

pline iswhatthe team uses to ensure that all theworkis doneproperly and

with no shortcuts. For example, discipline helps teams ensure that their

work is really donewhen theysayit is.

Unfortunately, agile development has been labeled as being undisciplined by some. Yet any team, regardless of its practices, will not succeed

unless itisdisciplined. WhatI suspect ishappening isthat thereisconfusion

about the difference between discipline and ceremony. Ceremony in this

case refers to the required activities thatmustbe performed in a project: for

example, a requirements document before a certain milestone, milestones

suchasa feature freeze, weekly statusmeetings, designdocuments,or man

datedbeta programs. Agile teams minimize ceremony in favor ofkeeping to

the task at hand and producing results (a working product), while more

involved processes emphasize ceremony and documentation over theresults. In many cases, unfortunately, people mistakenly believe that unless

documents are createdthere isno discipline. This is a mistaken beliefhow

ever, because these documentsare frequendywastedeffort and are no guar

antee that a prescribedprocess isbeingfollowed.




Responsible

The team must beresponsible for itswork and notplay thepartofavictim.Victims goalongwith anythingand don't rockthe boat, then absolve them

selves of any responsibility so they can lay the blame elsewhere. Thereare

actually very few situations in life where you are truly a victim. In most

cases, thevictim is just as susceptible to his orherown line of thinking andunwillingness to takecharge ofthesituation as they areto the situation. For

example, if a decision ismade thata responsible person knows iswrong,thatperson doesn't just goalong with it,heor she ensures thateveryone isaware ofthe consequences sothatthedecision iseither made with the right

dataor is changed. Victims, on the otherhand, go along with the decision

and then say, "I was told to...," even though they knew it was thewrongthing.

Responsible people have courage to take onany situation anddonotget

swept upbyevents. They trustothers todothebest they can andarealways

diligent to ensure thatnothing falls between the cracks. They don't just say,"thatcan'tbe done." They explain why andthenhelp uncover the real prob

lemand provide a viablealternative.

Leadership

The team has multiple leaders whowork together, and the leaders are not

justthemanagers, team leads, or coaches. See Chapter8 for moreideason

leadership in sustainable software development.

I alsobelieve that an attempt to build a successful culture is enhanced

by the presence of at least one level 5 leader [Collins 2001]. These are the

individuals who recognize that the greatest success iswhen they can help

others succeed andgive them theopportunity to receive the accolades they

deserve, who view success as a team accomplishment and failure as a per

sonalresponsibility, not anopportunity to apportionblameon others.These

people are fewand far betweenand areunfortunately rarelypromoted into

seniormanagement because theyoftenlackthe charisma, high-volume talk

ing, and massive ego and self-confidence; the traits most often associated

with gettingahead in the corporateworld.

CULTURE, BY DESCRIPTIVE WORDS AND PHRASES • 37



Visionary and Tactical

Effective teams strive for the optimal balance between being visionary

(thinking about the long term) and tactical (focusing on today's problems),

andthisbalance changes over the course ofthe project. In the early phases

of a project there should be a heavier emphasis on being visionary, andin

latterphases the emphasis should be placed on tactics related to finishing

theproject. Projects fail when there istoomuch emphasis onbeing visionary

lateinthedevelopment cycle orwhen there istoomuch emphasis on tactical

issues too early in the project. Great teams are able to balance the two at

each moment in the project.

Shared Sense of Urgency

In sustainable development it is importantthat everyone on the teamshares

the same sense ofurgency to complete theproject. If thebusiness peopleare

worried about the competition and the developers are more concerned

about the perfect architecture, failure willresult.

A sharedsense of urgency is also themost importantfactorin building

teams. It isn't compatible personalities or team building exercises or even

necessarily mutual respect that makethe best teams, it's a shared goal or

aggressive but achievable deadline that the team uses as a rallying cryfor

getting the project done.

Highly Collaborative

Collaboration is a vital success factor in sustainable development. Good

teamsare able to work together,embracethe differencesbetween person

alities and roles, and strive for a successful project. They also recognize

that there are clear roles for each of them to play. Businesspeople don't

write software and programmers don't write business plans. Collaboration should be like breathing; it is always there and is done almost sub

consciously. And it should be done in person (as opposed to e-mail or

voicemail) asmuchaspossible because that is the most effective formof

collaboration.




Complementary Talents and Skills

Great cultures recognize talents and skills as separate from each other. Too

many companies are focused on skill alone and recruit only for skills. But

skills can be taught, whereas talent can't. Software development isaperfectexample because people who aretalented atcomplex problem solving make

excellent programmers. Excellent programmers can learn new program

ming languages ina short period oftime andwill over time outperform their

peers. Andyet, mostjobadsin ourindustry still lookforacertain numberof

years of experience programming in a particular language when what they

should be looking forarepeople who aretalented problem solvers.

Continually Improving and Learning

Continual improvement andlearning isavital element ofsustainable develop

ment. Great peopleand teams continually learn and refine their projects and

how they approach the projects. They use short time increments (many times

peryear) toensure theirpace ishigh andlightweight anduse techniques tomon

itorprogress, keep themselves on track, andintroduce changes asrequired.

Change Tolerant

Great teams understand that change is necessary and desirable. They also

understand that change must sometimes be made in the face of conflicting

or incomplete information. However, they also realize that just because a

decisionismade at one point in timedoes not meanthat it can't be or won't

be modified asnew information is available. Hence, being change tolerant

also means that sometimes some extra work must be done at the current

timesothat changetoleranceis possiblein the future aswell.

Risk-Aware

Risk isa constant element ofallprojects. Effective teams arealways aware of

risk, are able to differentiate between real and perceived risk, and are not

CULTURE, BY DESCRIPTIVE WORDS AND PHRASES



afraid to make decisions with the best information possible in the face of

risk. They are also constandy looking for ways to reduce oralleviate risk.

Fun

You can't always choose what you do, but you can choose howyou do it

[Lundin et al2000]. Goodpeople and teams recognize this. They find ways

to makework more fun for their co-workers and customers.

Summary

Mindset and culturearemoreimportant than practices. Unfortunately, cul

ture is not something that can be turned on and off like a light. Luckily,

there isa spectrum ofpossible ideal cultures. Whatmight workin oneenvi

ronment and with one set of team members will not necessarilywork in a

different environment with different people.

Creating the idealculture takes a lot of hardwork, and it takespeople

dedicatedto making it happen—good people. The vastmajority of people

want to succeedandhavethe toolsrequiredto do so, if theyareput into the

rightenvironment. Theremustbe a shared sense ofpurposeandvision, and

thisiswhy theremustbe aneffort made to getting thepeoplewiththe right

attributes onto the team and others off the team, or on the bus as in Good

to Great [Collins 2001]. It's how the people in the team apply themselves

to creating the necessary environment that separates the poor teamfrom

the average from the good and from the great team. Thereis an element of

luck and being in the right place at the right time with the right people.

Applying the principles and practices outlined in this book and thinking

about yourdesiredcultureshouldget yourteamwellon the wayto a thriv

ing, positive culture.

The rest of thisbook describessomeof the recommended practicesthat

support the fourprinciples of sustainable development. There is a chapter

for each principle, and each chapter explains in detail the rationale behind

theprincipleandthepractices that help tomakeit real.




Chapter 4

Working

Product

Aworking product atalltimes should be the ultimate goal for everysoftware

team. The further a product is away from beingina working state and the

longer it is inthatstate, thegreaterthetimerequired toget theproduct backto a working state.

Software products need to be in aworking state at all times. This could

be called virtually shippable because the product could be shipped after a

short period ofverification testing and stabilization, the shorter the better. A

workingproduct doesnot meanthat the product isfeature completeor that

all features are done; rather, that it can be shown to or given to a customer

for use with the minimum possible number of unexplainable errors. The

closera team can get to continually keeping its product in a working state,

the better th e chances that its efforts can be maintained over t ime becauseof the level of discipline required and alsobecausehaving a workingprod

uct imparts greater flexibility and agility.

Why Working Product and Not

Working Software?

I use the term working product and not the more common phrase

working software deliberately. This is because from the customers'

point of view, they rarely receive just software. Software is almostalways accompanied by explicit deliverables such as documentation

plus supporting infrastructure like a website for information, updates,

and support. There are also implicit deliverables such as an expecta

tion that there has been testing before the customer sees the product.

Hence, I use the term working product to encompass all the explicit



In order to keep a productin aworking state, teams needto focus on

the quality of everything they do. Achieving this level of product qualityrequires a great deal ofdiscipline and focus onthepartofthe development

team: focus onthe quality oftheproduct and the discipline to ensure that

every software modification passes an extensive suite of automated tests, or

it is removed from the product untilit does pass the tests (see Ruthless Test

ing in Chapter 5). Whenever a product is in a non-working state and

changes are still being made to it, the development team is literally flying

blind and technical debtisaccumulating; a debt that will have tobe repaidlater. This situation is a recipe for disaster because the cost of fixing prob

lems increases with time, as does theprobability thatmultiple problems willshowup in the finalproduct and be found at customersites.

The Need to Address Quality Promptly

If yourcargets a flat tire, you know thatifyou keep on driving without

fixing the flat, the rim will need replacing, and then the axle, andthen

various body parts as they are ground down through contact with the

pavement.Eventually, yourcaris going to needmajorrepairs, andeven

when you do get it repaired, it will never be the same. Most people

wouldn't even consider the option of not changing a flat tire on their

car. Yet with software teams the severityof a problem and its conse

quences are not so obvious, so the temptation is there to just leave it

even if it leadsto more problems down the road.

In Lean Software Development [Poppendieck and Poppendieck

2003] the analogy is drawn with an assembly line used ina manufac

turing plant. Ifa problem is found withwhat is being produced on the

assembly line, a big flashing light goes off and production stops until

the problem is fixed. Software teams may hate having to stop their

work, but addressing quality concerns promptly is a critical part of

keeping a product in a working state while minimizing the effort

required to keep it inthat state.

A useful conversation to have witha product team is around the ques

tion What would it takefor ustoship ourproduct every day toour customers?

In this caseship couldmeanprovidingdaily updatesor a completenewver

sionof the software to install. The purpose of answering this question is to

WORK ING PRODUCT • 43



of ongoing accomplishment and ability to be proactive not reactive (i.e., in

control). Achieving aworking product isnot extra work ifitisdone every day!

Ship at Unpredictable Times

One ofthe chief advantages of having aworking product is being able

to shipat unpredictable times. I saw a project a while ago where the

requirement came into shipa Japanese version of the shipping prod

uct.A large deal witha softwaredistributor wasat stake; the distributor

needed to get the Japanese version within a month orthe opportunity

would bemissed foranotheryear.

The problem is that team members were in the middle of a devel

opmentcycle for the newversion ofthe product. One ofthe keyarchi

tectural changes they had made for the new version was to

internationalize the product because they anticipated the need for aJapanese version. However, because the architecture had changed

quite a bit from whatwas currently shipping, they could not portthe

changes back to the shipping version. In a typical development team,

this situation would cause a great deal of panic (and amissed opportu

nity). Luckily, this wasn'ta typical team.Their solution to the problem

was to take the softwarethey wereworking on, turn offallthenew fea-

turesso the software was functionally identical to the shipping version,

then complete the localization. The end result was a Japanese version

of the product in less thantwoweeks; the English to Japanese transla

tionandtestingtook moretime thanthe engineering work!

This is one example of the power of having a working product.

Having a working product put this team in control of its situation and

allowed the flexibility andconfidence to shipwhen itwanted.

BeWary of Artificial Milestones

One of the teams I worked with had a functionality freeze milestone

where all new features for the release had to be integrated into the

productbeforethe functionality freeze date.

The intention ofthe milestone was good (largely to ensure thatthe

product could be documented with the features in a stable state and to

ensuretherewas adequate time to test and stabilize the final product

before shipping). However, for most features, the functionality freeze

came too early in the schedule. As a result, developers would often

WORKING PRODUCT • 47



postpone bugfixing until after the milestone, or they would check in

incomplete features and sometimes justcodestubs so thatthe feature

would appear to be there and could then be fixed up during the bugfix

ing phase! Also, team leaders would encourage developers to post

pone bugfixing until after the milestone so the features could be

completed.

Hence, with artificial milestones (as with many practices), it is

important to watch out for unintended side effects. In the case of the

functionality freeze milestone, the emphasis on all development work

beforethe milestonewason getting features intothe code base, not on

ensuringthat the softwarestill functioned.

On this product, the software would often be in an unshippable

state for as much as six or nine months out of every year! Although it

was tempting for teams to think they'd made terrific progressat the

milestone ("Just look at all the features we checked in!") all hell would

break loose after the milestone as the team would gallantlytry to get

the software backto a shippable state, with the end resultthat many

defects couldn't be fixed due to a lack of time.

Even worse, many of the features would have to be removed or

havethe user interface disabled just to get the softwareworkingagain.

A greatdeal of effortwas wasted, not only in fixing all the defects long

aftertheywere first introduced and getting the softwareworking again,

but inall the features that were implemented but nevershipped.

Due to the chaos, each product release had slightly lowerquality

than the release before it. The end resultwas a gradual degradation ofthe software's quality overtime and a rapid accumulation of an ever-

increasing backlog of defects. This in turn led to longerbugfixdeath

marches in every release, where even though the team was working

hardto fix all the bugs, itnevercaughtup. Becausethere were so many

bugs, the feature freeze milestonewas forced earlier formore stabiliza

tion time atthe end ofthe release cycle.And, of course, the pressure to

develop new features neverwent away.

The development team literally felt trapped because it had to

choose between spending additional time bugfixing and less time

developing features, or focus on developing new features and fixingfewer defects.Theywerealso spendingan incredible amount oftime in

theirdefect tracking system sorting, prioritizing, andanalyzing defects.

This is part of a classic software development death spiral described in

Chapter 2, one that stifles the ability of the team to be innovative and

agile.

48 • WORKING PRODUCT



Imagine That Your Customer Can See YourBroken Windows

I learned the importance of having no broken windows early in my

career. The company Iwasworking for atthetime depended onasoft

ware product that had been written by another company that went

bankrupt. Luckily, wewere able to getthe source code sothatwe didn't

have to develop a similar product ourselves. Iwas given the responsi

bility of keeping this software working.

This source code was littered with comments like:

// Ugly workaround to fix the case where the index

/ / i s negative. Be sure to fix this in version 3!!

//

At first, I found it frustrating to read these comments and to fix problemsthat had neverbeenproperly resolved inthe first place. Butaftera

while, I developed a sense of empathy for the original developers. No

doubtthey had beenunder a great deal of time pressure, and Isuspect

they would have been embarrassed ifthey knewthat one of their cus

tomers had to deal with this code.

This experience taught me that while being under extreme time

pressure maybe a valid reason for leaving a broken window, thereare

novalid reasonsto leave itthere foranything morethan a few days.

PRACTICE 2: Be Uncompromising

about Defects

It is vital that you adopt an uncompromising attitude toward defects. The

goal isto avoid a defect backlog, where there is a listof opendefects that is

too largefor the teamto dealwithimmediately. Defectbacklogs are evil and

must be avoided because:

• Defect backlogs burden teams withrepetitive searching, sorting, and

categorizing tasks that waste time and take away from productive

work.

• A backlog, by definition, means thatmore defects are being carried

around than can be reasonably dealt with. The number of defects

• 50 • WORKING PRODUCT



The Value of Being Uncompromising

I have worked on and with quite a few teams thatwereuncompromis

ing with defects. The common thread in all these teams was that the

number of known defects in our product was constant to slightly

decreasingovertime, even asweadded newfeatures and rewrote large

sectionsof the code. Every team used different techniques and prac

ticesto accomplish this.What mattered wasthe mindset.

Put More Effort into Defect Prevention Than Bugfixing

The most important point about being ruthless about defects is that you

need toputmore effort into preventing defects thanintofixing them. Defect

detection is stillvital;it's just that defect prevention is evenmore important.

Being ruthless about defects andtrying to avoid theaccumulation ofaback

logisone thing, butifyour product doesn't have safeguards in place topre

ventdefects from getting toyourQAdepartment or customers, thenyouare

still going to get a backlog, no matter how fast you fix and classify your

defects. Defectprevention isdescribed in detailinChapter5.

Use a DefectTracking Database, but Don't Build

It Yourself!

Being ruthless about defects, of course, assumes your team is using a

defect tracking system. If you don't have one, there are many good

commercial, free, and open source systems that can be downloaded

and installed pretty quickly if you don't want to purchase one.A good

example is bugzilla (http://www.bugzilla.org/). Because there are so

many decent defect systems available today, this is a perfect example

of a tool that should never be developed internally. Focus on what

makesyour product unique and on shipping it!

Being Pragmatic about Defects

Ifyou arestarting aprojectwith known technical debtor are concerned that

your product quality may slip, there are a few things you can do:




• Insert bug fix-only iterations between feature development itera

tions. If productquality is slipping, you'd be surprised byhowmany

users appreciate a stable product with a few focused features.

• Setsomequality-related exitcriteria foryouriterations. Then, set the

rulethat the team can't call an iteration complete untilthe exitcriteria are met. An example exit criterion might be "There can be no

'stop ship' defects in the product and no known regressions."

Another might be "There can be no known open and unresolved

defects" (to reinforce the need to be ruthless). These exit criteria add

someextra discipline to iterative development throughhaving a clear

goal.

• A variant on the previous point is to let the team decide (the team

includes the users or user representatives!) whenthe quality is good

enough to move on to the next iteration.

Theseideasmightcausesomepeopleto balk,but I thinkyouneedto do

what is best for your project.Yourgoal is to ship repeatedly, and with the

least amountofwastedeffort. I haveused or seeneachof the abovesugges

tions on various projects, and theywork. However, you do need to be care

ful, becauseyoushouldnot need to regularly rely on thesenor use themfor

anyextendedperiod of time;disciplined iterative development aloneshould

be sufficient.

PRACTICE 3: "Barely Sufficient"

Documenta t ion

In order for teams to concentrateon a working product, theyshouldmini

mize the amount of time spent writingdocuments that are not part of the

product. Examples of this type of document are requirements and design

documents1.

1. Internal documentation is distinct from the externaldocumentation that is included with the

software. External documentation such as helpand feature documentation is required bycustomersandshouldbe considered as features. Theintentof thispractice istominimizethe internaldocumentation only.

PRACTICE 3: "BARELY SUFFICIENT" DOCUMENTATION • 53



Barely sufficient documentation is another important part of agile

development. Alltoooften, teams forget why theyproduce documentation:

to produceaworking product. In toomany cases, documentation is required

beforea project canproceedto the next step.This introducesa linearity to

development thatisunhealthy andunrealistic. Many people losesight ofthefact that what trulymatters in anydevelopment project is the process,espe

cially the conversations and decisions requiredto produce a usefulproduct.

Documents are secondary to collaboration and learningand are a result of

the process, not its focus.

This practiceis intended to encourage teamsto focus on whatmatters

themost:theirworking products.If youhavebeeninvolved in or observeda

project where the team spent months or years trying to understand user

requirements andwritingdocuments without everproducing a product (or

evenwriting any code at all), you'll understand the reasoning behind thisrule! Customers don't know what they want until they see it; documents

don't help customers at all.

Minimal Documentation and Code Comments

Some people carry the drive for minimal documentation too far. One

example is code comments, where some individuals claim that code

comments are superfluous because (a) when automated tests are

present, code documentation adds very little and (b) most code comments are extra work that don't add value .

I think this is a dangerous attitude because good comments aid

the understandability of code. Many of the best programmers I know

write the comments first and describe the logic of what they are doing

and then they write the code. So, while it's true that comments that

state the obvious aren'tvaluable, it's important to not dismiss all com

ments.

Someof the barely sufficientforms of key documentation are:

• The use of index cards for feature description in cycle planning.

Index cards are used to describe features, to record the essential

facts, and encourage teammembers to discuss the details. Theyalso

give team members something tangible to hold in their hands and

quicklymovearound during planning sessions.




• Thecollection offeature cards for a project should serve as adequate

documentation of requirements. It may be necessary in exceptional

cases to provide additional documentation, but these should be truly

exceptional circumstances not the norm.

• Designyour software collaboratively in front of a whiteboard. Great

discussions almost always result thathighlight problems and alterna

tiveapproaches. Then, simply takea pictureof the whiteboardwhen

done andplacethe image on awebpage.

• Analternative to drawing a design onawhiteboardis to use a sketch

ing tool. There are some good inexpensive tools available, such as

SketchBookPro fromAlias (http://www.alias.com).

• Putdesign details likerelationships andcode rationale with thesource

code (where it canbe easily keptup to date) and extract them using a

tool like JavaDoc or doxygen. This is a good practice because it

encourages collaboration and decisionmaking over document writ

ing in addition to encouraging teams to get on to developing their

product.

For detailson designing software in an agile development context, refer

to the sectionSimple Design inChapter6 onDesign Emphasis.

The examples given above of barely sufficient documentation are by

necessity briefand far fromexhaustive. Anentirebook [Ambler andJeffries

2002] has been written on this topic, and interested readers should refer to

i t for fur ther information.

The Dangers of Excessive Documentation

I can think of two prominentexamples of teams I haveworked inthat

did not understand the practice of minimal documentation.

One casewas early in my career where Iworked atacompany with

a heavyweight development process. There was a rigorous process

where for every feature we had to produce, in series: a requirements

document, a functional specification, a design document, and writtentest plans. We had a large set of bookshelves where all these docu

ments were kept. Review meetings were held for each document. It

was amazing that we got any work done! We moved very slowlyand

wasted a lot of time writing, reviewing, and pushing documents

around.

PRACTICE 3: "BARELY SUFFICIENT" DOCUMENTATION • 55



And some of our priorities were rather bizarre: I remember one

meeting where there was adebate about whether we should use round

bubbles or ellipses in design documents. And I'm sureyou can guess

how much these documents were consulted after we'd moved on to

other features...!

Another example I can think ofwas a teamthatwas charged with

building a next-generation version of an existing product. There was a

desire to build something different thatwould leapfrog the competition.

This team spent six months trying to identify user requirements and

designing what a next-generation product should look like architec

turally. The result? A team leader, two senior software developers, two

otherdevelopers, auserrepresentative, and a high-profile customerpro

duced quite a few documents and no code, notevena prototype, before

the project was cancelled! That wasa wasted opportunity, and a ton of

wastedeffort, andthe needfor the new product is still there today.

PRACTICE 4: Continuous Integration

An importantteamdiscipline isto continuously integrate changes together.

Frequent integration helps to ensure that modules that must fit together

will,and alsothat the product continuesto workwith all the changes. Many

developers have the bad habit of checking out a number of files and not

checking them in again until their work is done, often days or weeks later.

Developers shouldintegrate theirworkevery day, or evenbetter,manytimes

per day. Thisgradualintroductionof changes ensures that integrationprob

lems or regressions arecaughtearly, in additionto naturally allowing multi

pledevelopers toworkonthesame section of the source code. Of course, in

order to catchproblems it is important that your teamhas automatedtests

inplaceto helpcatch integration problems asexplained inChapter5.

PRACTICE 5: Nightly Builds

In order for yourteam to ensure that yoursoftware isworking, it shouldbe

completely rebuilt from scratch oneormore times per day. Thebuild should

include asmany automated tests as possible to catch integration problems




class A {

protected:

c l a s s B *m_Bptr ;

TIP Timestamp the start and end ofall builds.

Agood practice to follow is to timestamp the start and end of

all complete builds and keep a log. Then, write a simple script

to catch large increases in build time. If possible, produce a

simple chart that can then be put on a web page and quickly

viewed. It's always easier to glance at a chart than to dig

through a log file!

The Entire Team Owns the Product Build

Itshould be part ofthe team's mindset that it is everyone'sresponsibil

ityto pay attention to the product build. Unfortunately, some develop

ers I haveencountered feel that cheaper labor should be hired to look

after the builds. This attitude is wrong and unprofessional. It is wrong

because the cheaper labor won't knowthe code and won't be able to fix

any problems when they're found—only the actual developers can do

so. It is unprofessional,because hiring someone else to do partofyourjob is like a mechanic hiringsomeone else to lookafter his tools...!

PRACTICE 6: Prototyping

Making a conscious effort to prototypesolutions to risky problemshelps to

increase the chance of havinga working product. Prototypes are an inex

pensive way to try out ideas so that as many issues as possible are understood before the real implementation ismade. There are two main classesof

prototypes that you can use to positively impactyour product. The first is

the true prototype, where a test implementation is used to understand a

problem before it is implemented for real. The second is the notion of

"tracer bullets" [Hunt and Thomas 2000].

PRACTICE 6: PROTOTYPING • 59



coupleof days. Isuspect we savedourselves morethantwo months of

engineering effort!

Throwaway Prototypes

A throwaway prototype is an excellent tool to evaluate risky features and

develop better time estimates for a feature. As explained in the "iterative

development" section, the elimination of riskas early aspossible in a project

is often a critical project success factor. Doing a quickprototype in a small

number of iterations is often an excellent way to acquire a much better

understanding of the problem and the actual risk. Also, by developing the

prototype as a throwaway, usually in a copyor branch of the product or a

simpleprototype application, it is easy to ensurethat the product isstillin aworkingstatewhen the real solutionto theproblemis implemented.

Quite simply, a prototype is a quick and dirty implementation that

exploreskey target areas of a problem. Prototypes should not be production

ready, nor should they provide a complete implementation of the desired

solution. They might even be implemented in a scripting language like

Python or Ruby to save time. The bottom line with prototypes is that they

should be used as a point of reference onlywhen implementing the com

plete solution to a problem.

One of the downsides ofprototypes is that youoften haveto showthemto customers and management in order to demonstrate progress. One of the

conclusions your audience could draw, if you are not careful, is that the

problem is solvedand done when, in fact, all you have is a rough proof of

concept. If you do show a prototype to customers, be sure they know what

you are showing them before you show it to them to avoid situations where

your audience thinks the problem is solved.Youcould alsomake it obvious

through roughvisualoutput that clearly doesnot lookdone.

Agood analogy to use is car design. The carmanufacturers all regularly

produce concept cars to try out ideas or demonstrate a proof of concept.

These carsare one-offs that could be turned into a production versionfairly

quickly, but they are definitely not ready for shipping to customers. Like

wise, manyproduct designers ensurethat if theyareshowing a digitalproto

typesuchasa rendered imageto their clients that the image doesnot make

the product lookdone. If yoususpectthere isgoing to bepressureto use the

PRACTICE 6: PROTOTYPING • 61



prototype for the shipping product, it might be better to consider using

tracer bullets instead.

"Tracer Bullets"

"Tracer Bullets" are a deliberate approach that uses a prototype that is

intended from the outsetto gradually turn into the final solution[Hunt and

Thomas2000].The analogy is combat: Imagineyouhave to shoot at and hit

a target in complete darkness. The brute force approachwouldbe to point

yourgun in the right directionand hope for the best.However, amoreeffec

tiveapproach is to use glowing rounds calledtracer bullets.Youstart by fir

ing a few tracer bullets and then correctingyour aim until you can fire real

bullets to destroy the target. In the real world, tracer bullets are every few

bullets in a clip. Hence, the software analogy is that if you're uncertainyou're pointing in the right direction, start by building a viable prototype

and keep adding to it until your customer says, "That's it!" This is really a

microcosmof agiledevelopment,where multiple short iterations are used to

continually add to a prototype. The difference is that agile development is

applied to a collection of features (awhole product) and tracer bullets can

be used on one feature, problem, or area of risk.

PRACTICE 7: Don' t

Neglect Performance

Performance is one topic that generates passionate discussions in software

development.Somepeople feelthat code clarityis more important and that

you should get the code clarity right first and then optimizethe 1 to 3 per

cent of code that needs it. Others feelthat you should code for performance

first, because if you don't, your code will alwaysbe slow.

Personally, I get scared when a development team or developer says"we'llworryabout performance later." I have seen too manyprojectswhere

the teamfelt the need to get the features in beforeworrying about perform

anceled tomonthsor years of rework. Codeclaritydoes comefirst,but not at

the expenseofperformance. This is another casewhereyouneed to embrace

the genius of the AND: You must be concerned with performance AND




and they can alsobe exploited by hackers to compromise your customer's

computers.

I wish it were possible to say that this is an easy practice to follow.

Unfortunately, the reality is that even ifyouuse available tools (see Chapter

5 on DefectPrevention),leakscan stillhappen. Sometimes, lowerlevelAPIs

that your application usesleakmemory. There isn't muchyou can do about

those except to report the problemto the vendor. And whileit might seem

ideal to adopt a policywhereyoualways freeeveryallocatedresource, some

times youcan't.Themostfrustrating example is ifyouhavea largeapplica

tion that allocates large amounts of memory. Because operating systems

don't provide a fast exit path for applications, if youwere to explicidyfree

up all allocatedmemorywhen the application stops, it may take many sec

onds for the application to actually go away, which can be frustrating to

users.Therefore, to get the appearanceof a fast exit, it is common practice

to not bother freeing up memory at all, quit, and let the operating system

worry about it. This wouldn't be a problem except that when you use a

memory leak detection tool, allof the memory and resources that were not

explicidy freed on exit are reported as leaks, which makes legitimate leaks

virtually impossible to detect...!

All I can advise is to at least make sure you exercise due diligence—

don't just ignore trying to find leaks. The tools available today have APIs

that you can hook up to your application to start and stop leak detection.

These can be veryuseful.

No Memory Leak Can Be Ignored

A common misconception about memory leaks is that only the big

ones matter. Unfortunately, this is not true; a leakas small as one byte

of memory is enough to set up a fence in the application's memory

pool thatwill ensurethe system cannotreusethat memory.One of my

favoriteexamples of this problem happened a few years ago...

I had just startedata companyworking on a productdeveloped in

C++ where it turned out the team had been very lax about memory

leaks. Less thana month intothe job Iabsent-mindedly leftthe applica

tion running atthe end ofthe workday. When Icame in the next morn

ing, Idiscovered that the application hadcrashedsometime during the

night. I wasn't sure itwas a problem, thinkingthat perhaps the crash

was caused by backups or some such thing, so that night I left it run-




ning again with the same result. Curious, Ithen wrote a simple script to

monitor thesystemmemory (using a system call) so Icould log mem

oryusagewhile running the application. What Idiscovered was normal

memory usage in the form of spikes as the program allocated and

properly freed up large chunks of memory, but overall the memory

consumed increased at a constant pace. The culprit turned out to be a

verysmall leak of4 bytesthatunfortunately washappening every time

th e cursor blinked!

Our users had been complaining that they had to constantly save

their work because the application crashed at unpredictable times

(sometimes unfortunately during a save operation). Once we under

stood the number of leaks in the product, we undertook a concerted

effort to remove the leaks and released a version of the software with

nonew features, just all the leaks fixed. I insisted we dothis despitethe

braying of ourmarketing department.

That release was one of the most satisfying releases I have ever

done, because our customers gave the release rave reviews and

applauded us for not adding features and just fixing theirmajor com

plaint—stability. And afteranother release, our marketing department

realized that although they hadn'tbeenable to say anything newabout

the product, their jobwas now much easier becausewe had gained a

reputation of listening to our customers,which is very powerful mar

keting indeed.

PRACTICE 9: Coding Standards

and Guidelines

Make sureyour team talks about what good code isandwhat coding prac

tices teammembers like and dislike. Put themdown on paper or in a web

page andgeteveryone on the team to agree to them. Don't gethungup on

silly formatting issues likewhether curly bracesshouldappear on the sameline or next, just get everyone to agree thatwhen they modify a file, they

should follow the same formatting conventions. Having consistent coding

conventions simply makes code easier to read. Plus, there is a hugebenefit

to having the team discuss good code and bad code; they can learn from

eachotherandunderstandeachother's viewpoints.

PRACTICE 9: CODING STANDARDS AND GUIDELINES



XMLis another example of the powerof adoptingstandards. The use

ofXML hasproliferated quickly because ofthe ease ofuseofthe language

and its supporting libraries. The ready availability of XML libraries and

parsers means that today itisvirtually impossible to justify designing apro

prietary text-based file format andwasting time writing a parser forit.Thisismostdefinitely a goodthingbecause it means that onemorepieceof the

standardbuilding blocks required byvirtually allprograms has been taken

careof.XML frees developers toworkon theirvalue-add.

PRACTICE 11: Internationalize

from Day One

Get in the habit of ensuring yourproducts are internationalized3 from day

one.There is no extra overheadonce youunderstand what is required. The

bottom line is that the largest amountof economic growth globally in the

next decadeis projected to take placeoutside of English-speaking United

States. The implication should be obvious: Your company needs to think

globally, and so should you.

This practice applies to the need for aworking product because of the

discipline required. The process of continual localization as the product is

changed is a discipline that,while sometimes burdensome, also requires discipline onthepartoftheproject team andthe skills learned through thisdis

cipline transfer to otherareas ofkeeping the product in aworking state.

Internationalization and aWorking Product

Internationalization mayseem like anodd practice to include with the

principle of working software. I viewthis type of issue as part of the

required mindset. You need to think about the critical things for

the long-term successofyour project thatare trivial todoatthe start of

3. Internationalization istheactofmaking software capable of displaying itsuserinterface inmultiplelanguages. Localization isthetranslation ofthetextstrings intoother languages. Softwaredevelopers internationalize their software, while localization isusually done byathird-party translation servicemanagedby the project team.

WORKING PRODUCT



the project but ifattempted later will severely destabilize it for a long

period of time.

I have seen too many projectswhere internationalization was not

considered until several yearsintothe project. In each case, the archi

tecture hadbecome complicated enoughthat the workwas considered

extremely risky, and the project team continually argued against it ortried to suggest band-aid solutions orworkarounds suchas onlylocal

izing one small part of the product. Yet, for most ofthese projects, the

business reality was that since the product was sold worldwide, it

neededto havea multilingual interface. Eventually, the business inter

ests won;the softwarechangeswereas disruptive as predicted andthe

resultwasa product that took a great deal of time to get backto a sta

bleworking state.

PRACTICE 12: Isolate

Platform Dependencies

If your codemust support multiple platforms, a clean separation between

platform-specific code and all the other code is essential to have software

that can be easily kept in a working state. Some examples of aplatform are

operating systems, graphics subsystems, runtime environments, libraries,

anduserinterface toolkits. Thebestway to isolate platform dependencies isusually through an abstract interface (such as a Fagade design pattern) so

that the majority of the software is isolated from the platform-dependent

code. Otherwise, software can become littered with conditional code that

makes the codehard to maintain, debug, and port to other platforms in the

future. If platform dependencies are important to your project, take the

timeto do them right.The effortwillpayoff.

Beware of Copyand Paste PortingI have seen a number of very poor ports of code, especially to a new

operating system. The problem isthatthe quickest wayto do a port is

to simplycopyand paste the code, thenmake the changes required forthe new platform.

PRACTICE 12: I SOLATE PLATFORM DEPENDENCIES • 69



The most extreme example I've seen of copy and paste porting

was a project wherethe programmers duplicated the contents of entire

source files, even ifall thatwas different forthe new platformwas a few

lines of code! The result was extra effort, since bug fixes were often

made inonlyone copyof the code, or features wereonlyadded for one

platform. Itwas alsohard to knowwhatwas actually platform-specific.The resultwas moretime spent coding andtesting. In the end, we had

to completely redothe port, and the second porting effort was harder

than th e first!

Admittedly,the previousexample is an extreme one. Buteven on a

smaller scale, copy and paste can still be a major impediment to keep

ing a product ina working state. For example, I remember one project

where a product was first written for UNIX, then ported to Windows,

and then to the Mac. Becausethe portswere not done atthe same time,

therewere far too manycaseswherethreecopiesof the same few lines

of codewereduplicated oneafterthe other. The codewas hard to read,

maintain, and keep working. We had to do a major cleanup, which

probably cost as much as the total of the original ports. All this effort

could havebeen saved ifthe portshad beendone properly, by isolating

the platform dependenciesbehindsuitable abstractinterfaces.

Summary

Softwareproducts should always be in a shippable state.This doesnot mean

a feature completestate.Aworkingproduct gives teams the maximumpos

sible flexibility and agility while minimizing wasted effort. The longer a

product isin a nonworking state, the greaterthe effortrequiredto return it

to aworking state. If problems areneglected too long,it maybe impossible

to return the product to aworking state.

This chapter has outlined a number of practices related to keeping a

product in a working state.Bynecessity, there are references to practices in

other chapters because all the principles and practices described in thisbook reinforce each other. Subsequent chapters cover the principles of

defect prevention, design, and continuous refinement. Defect preventionis

required for working software because teams must have automated, not

effort-wasting manual, methods to keep their products in a working state.




Likewise, in a complex environment of continual change, the design prac

tices employed by the teammust reinforce agility and changeability so that

allhelldoes not break loose every time a change ismade. The principle of

continuous refinement is also vital forworking software, because theproject

management practices used by the team must provide a lightweight andflexible way to continually monitor the project at frequent intervals to gauge

progress and introduce the necessary changes to keep the project on track,

and the product working.

SUMMARY



Defect

Prevent ion

Afocus on defect prevention overdefect detection isanimportant principle

of sustainable development. Defect detection follows what is probably the

most common method used today to develop software: the code-then-fix

mindset, where features are developed, testing is done (by users, a testinggroup, or QualityAssurance), and then defects are fixed. In this approach,

there is a noticeable time lag between when defects are introduced and

when they areactually fixed. Defect prevention bycontrast follows the code-

then-fix mindset, where features are developed using automated tests and

related practices that catch the vast majority of defects when theyareintro

duced so they canbe fixed immediately when itis cheapest to do so. Apro

file that illustrates the difference between a defect prevention cultureand a

defect detectionculture is shown inFigure5-1.

Cheaper or Easier?

I've had some developers tell me, "It's cheaper to fix defects after

they're found than itis to prevent them." These developers are delud

ing themselves and confusing cheaper with easier: It certainly seems

easier to concentrate on getting features outthe door, then worrying

about fixing them up later. But what these people miss is that cus

tomers find many of the defects, and that is extremely expensive for

your company and for your customers. Even more important, however,is that having to fix defects later causes the team to lose control of the

project, and this lossofcontrol directly impacts theability to be agile.

Defect prevention in a chemical manufacturing plant is taking the

pumps offline periodically to perform preventive maintenance. Pumps

thatare regularly maintained are less prone to breaking down, and by

pro-actively maintaining their equipment, the plant employees areable



74

1 i

Number

)f Defects

Found /\

' '

Developers QA Customers

A

Number

o f De f ec t s n^v

Found

'"\Developers QA Customers

Figure5-1In a culture predominantly oriented around defect detection (left), themain burden forfinding defects is placed ona quality assurance (ortesting) organization, and quite a few defectsareshipped to customers. However, in a defect prevention culture (right) the largest numbersofdefects are found attheearliest possible moment, before the product reaches quality

assurance, andveryfewdefectsare shippedto customers.

to avoid running from one crisis to another. They are in control ofthe

situation, not victims of circumstance. Adefect prevention mindset insoftware development should also result in being more in control than

purelyresponding to events.

Defect prevention has real return on investment. Let's assume that

thesame number ofdesign andcoding errorsare produced ina defect

detection and prevention culture. In a defect detection culture, the

majority of defects are found through manual testing and after code

hasbeen integrated into theproduct. On theother hand, in a defect pre

vention culture, the emphasis is on finding defects before integration

into the product. The predominance of low-value manual testing and

elaborate defect tracking systems in defect detection have a very realcost, as doesthefact that there is a time delay between when a defect

is introduced and when it is fixed. By contrast, ina defect prevention

environment, the result of thorough automated testing, high-value

manual testing, and fewer defects that reach customers more than off

sets the costof spending extra time in team collaboration and writing

DEFECT PREVENTION



automated tests. An elaborate financial model could bedeveloped, but

hopefully a quick mental exercise should do.

Unfortunately, most software organizations emphasize defect detection and

dovery litde intheway ofdefect prevention. There are some obvious symptoms of a defect detection oriented culture:

• Developers relyon testers/Quality Assurance/users to finddefects in

their products. Developers work on the software for a while, then

they "toss it over the wall" to testers/QA, who toss it back when

problems are found.

• A noticeable number of regressions are found by people (users or

QA), where features that already existed stopworking as expected

when newchanges aremade. Regressions area strong indicator that

thereisnot enough testing and also that theproductmay need refac-

toring due to unexpected interdependencies, another indicator of

technical debt.

• People believe that many tasks, including testing, are "beneath"

highly paid programmers and are best done by lower paid help, so

the developers canwork on features (i.e., write code).While the fea

tures get done in the short term, the products thesepeoplework on

get on the unsustainable development curve and quickly become

more expensive to maintain and enhance over time.

• A backlog of defects that is carried over from releaseto releaseof the

product. Defect backlogsare an indicator that technical debt is accu

mulating in the product, and they should be avoided at all costs.

Once a team is faced with a backlog, the ongoing pressures to

develop more features mean that the backlog isgoing to stayandwill

almostalways continue to grow.

• Project teammembers spend inordinate amounts of time prioritiz

ing,sorting,and viewing defect-related information. Defectbacklogs

are a huge drain on the timeavailable to a project team.The lesstime

spent doingthe administrative choresin defecttracking, the better.• An emphasis on the importance of bug tracking tools as strategic

tools. Bug tracking tools are important, but they aren't strategic

to your success. There are plenty of perfectly adequate systems

available.

DEFECT PREVENTION • 75



Another way to look at the type of culture isto ask your team the following

question:

How would you prefer tofind out about a serious defect?

(a) From your customer.(b) From yourQA department.

(c) Find it yourself.

In a defect prevention culture, your answer would be (c); in a defect detec

tion culture it would be (b), with (a) as an outcome . . .

In a defect detection culture, developers have abdicated the responsi

bility of testing and product quality to other groups, usually QA. This is

wrong, because developers control theprocessesandpracticesused topro

ducethe product andbecausecontrol is essential to agility.The primary difference between a defect detection and defect preven

tion culture is the attitude of developers and management toward testing

(once again, mindset). Developersmustbe expected to do everything in their

powerto preventdefects and to catchthembeforethe software reaches QA

and customers. It shouldbe apointofpride that customers do not encounter

defects and that defects found by QA are fixed as rapidlyas possible. This

does not mean that developers spend the bulk of their time doingmanual

testing, but it doesmeanthat developers need to lookat the taskof software

development asmorethan justwriting code.Thepractices in thischapter areintended to help build a defect prevention culture, which is an important

part ofdeveloping a professional attitudeto software development.

The Role of Quality Assurance

Emphasizing defectpreventiondoesnot eliminatethe need for a QA organ

ization.Rather, the role ofQualityAssurance is completely different than it

is in a predominandy defect detection oriented culture. GoodQApeople

areparanoidaboutensuring thequality oftheproduct beforeit reaches cus

tomers and are conscientious about their job. The role of QA should be to

ensure that new features work in ways that customers expect them to and

that the product asawhole is capable ofmeeting customer s needs.

76 • DEFECT PREVENTION



In a culture thatemphasizes defect detection, QAs actual ability to fulfill its role is significantly impacted by technical debt in the product. Astechnical debt accumulates, QA will spend an ever-increasing amount of

time regression testing, because asnewfeatures areadded,previously pres

ent features will break, and the product overall will begin to behave inunpredictable ways. Regression testing often consists of mindless sweeps

through the product where each feature, button, or menu item is tested.

Regression testing is not realistic because it does not represent how cus

tomers use the product; it is too low-level. Hence, real customer problems

are rarely found throughmindless regression testing. In addition, because

the QAteamis taskedwithmanaging large numbers of defects, teammem

berswill spendan increasing amount of their time prioritizing and sorting

defects, andthiswill detract from the time they have available for testing.

Theend result ofQAs spending toomuch time on low-level testing and

being relied upon too much for defect detection is that defects in new fea

tures and serious workflow problems aregoing to be shippedto and found

bycustomers.Asshown inFigure 5-2,this isthemost expensive scenario for

fixing these problems because ofthe time delay between when the defect is

introduced and fixed. Unfortunately, the cost of defects is a hidden cost in

software organizations because defects are accepted aspart of the process,

6 .5

Before Coding

Coding

60 -100

15

ITest Post-Ship

THE ROLE OF QUALITY ASSURANCE

Figure 5-2

Th e relative cost

of fixinga defect

increases the greater

the t ime between

when the defect was

first introduced and

when it is fixed. Themost expensive sce

nario is when a defect

is found at a cu s

tomer s it e and then

requires fixing

[Pressman 1992].

77



soit isvirtually impossible to calculate the truecostofdefects, thoughsome

have tried [Rothman2000].However,it should be easyto agree that:

• The costof fixing a defectcan't possiblydecrease. Fewdevelopers are

able to return to a sectionofcodewithout takingextra time to understand the code and the problem again.

• Defects found by customers are the most expensive becauseof the

extra costs ofsupport,problem reporting and tracking, andthe need

to createand test patches,amongother things.

Because of the demands placed on the QA team in a defect detection

culture, there is either a constant inability to proactively test new features

and workflows, or there is a constant demand to increase the size of the QA

team to dealwith the load.Unfortunately, neither solution is sustainable.In a culturethat emphasizes defection prevention, QAis ableto concen

trate onwhat is important: the fitness of newfeatures and the overall prod

uct. In this case, far fewer defects are shipped in the product to customers.

Theonlywayto enableaQAorganization to focusonwhatisimportant isfor

developers to proactivelyembrace defect prevention practices.

PRACTICE 1: Ruthless TestingRuthless testing is all about developers doing absolutelyeverythingin their

powerto ensurethat their software isasheavily testedasit canbe beforethe

software reaches people, both QA and customers.

Ruthless testing is one of the most crucial practices of sustainable

developmentandif thereis onepractice to focus on, thisis it. Ruthless test

ing is primarily about automated testing. Developers shouldnotdo anyof

the following:

• Spend the bulk of their time testing features the way QA or users

would.

• Turn themselves into mindlessbutton pushers in the hopes of finding

problems.

• Spendtimewriting testing plans or testing scripts (i.e., pushthisbut

ton, then do that) that they or someone else would use to test a




feature. This should remind readers of the need for minimal docu

mentation and practicingthe art of doingonlythose tasksthat are of

highvalue. Test scripts may be useful inaQAorganization to docu

ment testsbut theyhave limited usefulness to developers.

Advocate the hiring of lower paidworkers such as co-ops or interns

to perform testing tasks.

All oftheabove would be awaste of time, effort, andmoney.

Developershave access to a testingresource thatis excellent at repeat-

edlyperforming mind-numbinglydull tasks: a computer! Tests canbewrit

ten ascode, keeping the taskof software development as a coding exercise

with more codeproduced to complete a feature, but usually more codeded

icated to testing than to implementing the feature. Developers still have to

do some user-level testing, just as theydo today. The difference is that as

developers add and change functionality and fix defects, they put in place

safeguards in the form of automated tests. These automated tests are then

run asoftenas possible to give the developers confidence that theirproduct

isbehaving asexpectedandkeeping the expectedbehavior over time.

There are multiple types, or levels, of testing that can (and should) be

utilized in Ruthless Testing as shown in Figure 5-3.Eachof these types has

varyingdegrees of effectiveness,which is described inmore detail in the fol

lowing sections.

WorkingProduct?

WorkingCode?

User Awa re Te s t s

Usability User Verification

Visual Val idat ion

System Performance

Resource Usage

IntegrationRegression

Unit

Code AwareTes t s

Least

Potent ia l

fo r

Automation

Greatest

PRACTICE 1: RUTHLESS TESTING

Figure 5-3

The differenttypes,

or levels, of tests that

should be utilized in

ruthless testing. The

types of tests are

organizedby how

aware they are of user

tasks and code imple

mentation. User-

awareness indicates

how close the tests

areto replicatingexact tasks that users

perform. Code-aware

ness indicates th e

degree of knowledge

th e tests have with

the actual implemen

tation of the product.

79



Test automation is a critical part of ruthless testing. Code-aware tests

have the greatest potential for automation and provide the greatest return

for the least amount of effort.The user awaretests of usability, user verifica

tion, and visual validation are rarely automated because each of these tests

requires human effort for thedesign ofthetests andcareful observation and

analysis of the test results. However, while code-aware tests can determine

whetherthe codeisworking asexpected, theydo not necessarily ensurethat

the product works asusers expect. Hence, ruthless testing requires a strat

egythatmixes both code-level and user-level tests.

Unit Tests: Test-Driven Development

Unittestsarethe lowest level of tests youcancreatefor software becausethey

arewritten with explicit knowledge of the software's classes, methods, and

functions. Theyessentially consist of calling methodsor functions to ensure

that the results are as intended. Test-driven development means writing the

unit testsfornewcodebeforewriting thecodeandthen ensuringthat the tests

always pass whenever new changes are made [Astels 2003] [Beck 2002a]

[Husted and Massol2003] [Link and Frolich 2003]. In test-driven develop

ment,testsarerun every timethedeveloper buildsthe software. The resultof

thebuild iseithera greenlight(all testspassed) or a red light (atest failed).

Test-driven development is a powerful way to develop software. It

shouldnot be viewed asoptional,and it is a crucialpart of achieving sustain-

ability. Someof the benefits of test-driven developmentare:

• Code with tests can be modified and refactored with a great deal of

confidencebecause errors aredetected immediatelyby the tests.This

is important not only for the developer who originally wrote the

code,but alsofor developers unfamiliarwith the code.

• The tests ac t as invaluable documentation of the behavior of the soft

ware. If a change is made to one section of code that changes the

behavior expected byanother section ofcode,thenthe testswill fail.

Thisisvitalto preventing defects in existing features.

• Developers who are very goodat test-first development report that

they spend significandy less time debugging their software; some

even report zero debugging time!

DEFECT PREVENTION



TIP UseMock Objects!

If you deal with databases, network interfaces, user interfaces,

or external applications you should learn about Mock Objects

[Thomas and Hunt 2002] [http://www.mockobjects.com].Mock

objects are essentially a design and testing technique that let you

simulate an interface so thatyou can test your code in isolation

while having control over the types of events and errors that

reach your code.

TIP BeCreative When Implementing Automated Testing.

Automated tests shouldn't always mean manually specifying

calls and expected return values and error conditions as inJUnit.

In many cases, it is possible towrite a program to generate tests

for you. For example, suppose you have a piece of software (a

method, class, or component) that accepts a string of flags justlike the typical UNIX utility (e.g., grep, awk, sed).You could write

a program that called the code with the flags in all possible

orders.This typeof test isextremely thorough and bound to pay

off. I have seen cases where libraries that have been in use for

yearswith notests have tests addedto theminthiswayandseri

ous problemsare uncovered, insome cases fixing problemsthat

hadbeen long outstanding andthoughtto be unreproducible!

Integration Tests

Integration tests test the interface exposed bya collectionof classes that are

linkedor groupedinto a library/module/component. Unlike unit tests, they

do not understand the implementation of the library. Hence, integration

testscanbe a usefulmechanism to documentandtest theexpectedbehavior

of a library.

Figure5-4 illustrates the difference between unit and integration tests.

In thisexample, theunit testsfor eachclass in thelibrarytest the class's public and private interfaces, while the library's integration tests only test the

collective public interface of the library. Hence, integration tests can be

considered optional because they are a subset of the collective unit tests.

However, integration tests can still be useful because they are separately

PRACTICE 1: RUTHLES S TESTING



Figure 5-4

Integrationtests are

different from unit

tests. Inthis example,

a librarycalled

Omega is made up

of three classes. Each

class has its own unit

tests, and the library

has a separate set of

integrationtests.

82

LibraryOmega IntegrationTests

Library Omega

Class A Class B Class C

Class A

Unit Tes t s

Class B

Unit Tests

Class C

Unit Tests

.

maintained and created from the unit tests, especially if the programmers

who depend on the libraryuse the integration tests to documenthow their

codeexpects the library to behave. Thetestsdon't needto be complete; theyonlyneed to test the expectedbehavior. Two situations where integrations

tests aremost usefularewherea libraryisused inmore than one application

(or program) and when a third-party library is used.

One exampleofwhen to use integration tests iswhen you are dealing

with a third-party library or anylibrary that the product imports in a pre

compiledform.What youneedisa testharnessand a set of tests that realis

tically represent how you are using that library, the methods you call, and

the behavioryouexpectwhenyou callthosemethods. Youoften don't need

to test all the interfacesexposedby the library, just the onesyou depend on.Then, as new versionsof the library are shipped to you, all you need to do is

plug the newversioninto yourtest harnessand run the tests.Themore por

tions of your softwarearchitecturethat you can treat as third-party libraries

and test in this way, the more robust the interfaces your architecture will

have.Having sound interfaces willgive you a great deal of flexibility in the

future sothat at anytimeyoucouldchooseto replacean implementation of

one of these interfaceswithout disrupting the entire system. This is a huge

benefit and speaks to the need to havewell-designed interfaces that are eas

ily tested.

The Importance of Automated Testing

Ifirst learned the vital importance ofautomated testing on a team that I

worked on that produced networking software. My team had its own

DEFECT PREVENTION



live test network, and every time we made achange toour software, wewould push the latest software out to the test networkand run a num

ber oftests. Someofthetests were manual ones, butthe most effec

tive tests were the automated integration tests. Our automated tests

ran 24 hours a day, 7 days a week because we found that sometimes

problems would not show up until after a large period of time had

elapsed, sometimes days. I can remember many instances where

changes that Imade looked fine during a review but actually contained

subtle flaws that only showed up after a number of hours or days ofrepeated stress testing.

One particular episode I remember on the effectiveness of auto

mated tests involved a very large software application company, best

leftun-named. They found whattheythought wasa fault in ournetwork

protocol stacks and escalated it as a stop-ship problem. When we

received the report, we were puzzled becausewe knew our automated

tests covered the case in question. After a discussion, we found out

thatthey thought itwas our problem becausetheircodeworkedon our

competitor's protocol stacks. Theywerevery stubborn and refused to

admit the problem was in their code, so we gota binary of their soft

ware, installed it in our test network, and then ran it over the weekend

under a special debugger with a trigger for it to stop when the error

condition wewere looking for occurred. Monday morning wecame in,

captured the offendingassembly code, called them and said,"We don't

have your source code, but you should be able to find itatthe given

address and reproduce it using our test suite." We sent them the testsuite, andthey found that the problem actually did occuron the other

operating systems butwith different symptoms.Then they found that

usershadreported the problem to them, butthey had never been able

to reproduce it!

System Tests

System tests are automated tests that cover the functioning of the complete,integrated system or product. System tests know nothing aboutthestructure

ofthe software, onlyaboutwhattheproductisintendedtodofromthe user's

standpoint. System tests are intended to mimic the actions a userwill take,

such as mouse movements andkeyboard input, sothat these inputs can befed into theapplication inthedesired order and with thedesired timing.

PRACTICE 1: RUTHLESS TEST ING • 83



System tests are the most difficult type oftest to implementwell This isbecause in orderto be useful overthe longterm,the system test capabilities

must bedesigned into the software. Also, it is exceptionally difficult to addsystem test capabilities to existing software. For new software, the extra

design effort isworth itsincewell-thought-out system tests canbeextremelyeffective andcan significandy contribute totheproductivity ofthedevelop

ment team byreducing thedependence onpeople tomanually testthe soft

ware and also by providing an additional mechanism to prevent defects

reaching people (i.e., QAand customers).

Design testability intoyour product. Toomany teams relyonexternal

test harnesses ormanual testingby people andlearn to regretnot invest

ing the extra effort. The complexities of modern computer applications

make it difficult to create automated system tests. Modern applications

contain complexities such as multiple threads of execution, clients andservers that exchange unsynchronized data, multiple input devices, and

user interfaces with hundreds of interface elements. The result is highly

nondeterministic behavior that, without built-in testability, will lead to

unreproducible problems.

Record and Playback

Because ofthe complexity of automating system-level testing, mostsoftware

organizations choose to leave system testing to its QA organization (i.e.,

people). However, this isawasted opportunity because most software prod

ucts, especially new ones, could greatly benefit from a record and playback

architecture.

Many applications have a built-in logging capability. As the application

runs, it records the actions it is taking in a log file. Log files can be useful

when debugging a problem. In most cases, however, logging is only

intendedfortheprogrammers. Imagine ifitwerepossible to readthelogfile

back into the application and have the application perform all the operations recordedin the log file in the same sequence and at the sametime to

produce thesame result aswhen therecordingwas made. Thatisrecord andplayback. With such architecture, recreating problems anddebugging them

becomes almost trivial [Ronsse et al 2003].




Record and playback is equally effective in deterministic andnondeter

ministic applications1. One example ofa highly nondeterministic application where record/playback is widely employed is in the computer gamingindustry. In computer games, game companies invest agreat deal ofeffort increating special-purpose game engines. These engines take thegame terrain,levels, characters, props, etc. and render them in real-time in response to theinputs ofone ormore users. Game engines are complex pieces of softwarethat must consider multiple inputs (such as in amultiplayer game) wherecharacters in the gamecan interactwitheachother and the contentsof the

scene, and these interactions have a notion of time, space, and sequence;interactions aredifferentdependingonwherea characterisrelative to other

characters at any given time. Because of these complexities, many gamecompanies have by necessity designed record/playback into their game

engines so they can quickly diagnose problems thatwithout record/play

back would bevirtually impossible to reproduce and fix. In some 3Dgameengines the playback even allows the use of a completely different camera

viewpoint, which is like being a neutral third-party observer to the game

play. Many useful articles can be found written about these game engines on

game developer web sites suchashttp://www.gamasutra.com.

TheAutomation ofUser Interface Testing

One ofthemost frequently cited complexities in system testing istesting a product's user interface. One way to test applications with any

type of user interface is through a record and playback architecture

where all userinput iscaptured in the recording and can beplayed back

(another is to use mock objects, which are described above). With a

well-structured application, this mightallow the tests to be run inbatch

mode without the user interface, though this is not necessarily

required. However, even if you don't have a record and playback archi

tecture, there are still some useful tests that can be automated.

1.Deterministic programs are those where there isa single line ofexecution that, when providedwith some inputs, will always produce thesame output. Nondeterministic programs have multipleexecution paths thatinteract witheach other inacomplex way depending onsome time-basedinputs.

PRACTICE 1: RUTHLESS TESTING • 85



It is possible to use unit/integration tests on the application logic"underneath" the user interface. In this case, the most effectiveway to

testisto start byseparating theapplication logic from the implementa

tion of the user interface [Astels 2003], such as through use of theModel-View-Controller pattern [Gamma etal1995]. In terms oftesting,

given this type ofseparation of logic, itisvery easy to implement unitand integration tests for thenon-UI parts ofthe software. Manual test

ing ofthe Ul will still be needed, but this would bevisual validation test

ing. The more extensively the underlying application logic istested, the

more the visual validation can focus on the uniqueaspects of the user

interface, suchas ensuring thatUl elements are displayed correctly.

Visual Validation Testing

Visual applications (those thatproduce animage, movie, or allow theuserto

interactwith their data in real time) are the most complex to produce auto

matedtestsfor. Therearemany cases wherea personmustlookat the visual

output or interact with the program in some way (such as in a computer

game) to ensure that the application actually produces the outputs that it

should. For example, ina2D graphics application, the only feasible way to

ensure that a circle created bytheprogram, especially if it is anti-aliased2, is

really a circle,is to lookat it.It is important to ensure thattheamount ofvisual verification required

isatanoptimalminimal level. This isonly possible byautomating thecode-

aware tests asmuch aspossible to ensure that focus canbeput ontheverifi

cation, not say testing for regressions. This can be restated as: Use time

spenttestingbypeople wisely; don't getpeople todo tests that a computer

could do.

It shouldbe noted that some aspects of visual validation could also be

automated. If thevisual output canbe capturedasimages, a databaseofcor

rect images (as determined bya person) canbe used to ensure that imagescreated with the new version of software are the same as the correct images,

or different because theyshould be.

2.Anti-aliasing uses shading tocreate theappearance ofasmooth line onthe screen.

DEFECT PREVENTION



Don't Forget about the Economics

of AutomatedTesting

Remember that it costs time (and money) to write automated tests.Some people claim that writing an automated test the first time costs

between 3 and 30 times more than running a test manually. Hence,don't try to automate everything, especially ifit involves building up a

large infrastructure outside oftheactual code that makes up your prod

uctand is used bycustomers. Concentrate ontests thathave the high

est value or the highest costs to run manually. And if you can, build

testability intoyour software.

Performance Testing

Every pieceof software shouldhave some performance goals. Performance

testsensurethat yourapplication achieves or exceeds the desired perform

ance characteristics. Performance could be measured in one or all of unit

(e.g., to test the performance of frequendy called methods or functions),

integration, or system tests.The most common wayto do performancetest

ing is by usingsystem timers in your tests and either logging the results so

theycanbe compared or graphedagainst previous test runsor bycreating a

test failure if the performance is not within some expected range. Alternatively, there are quite a few commercial and open source performance test

ing applications available. My current favorite is Shark, which is a tool

available fromApplefor developmentonOSX.

Run-Time Monitoring

Software should be built with testability in mind. One useful but under

utilized idea is to build in run-time monitors. A run-time monitor is a

piece of codethat can be enabled or disabled and displays or saves alog of critical events (e.g., messages, input/output, memory usage).

Run-timemonitors augment automatedtests, because sometimes it is

useful to get live data while the application is running (even atacustomer site).




In my experience, it's best to carefully choose a small number ofuseful monitors thatare chosen for the application area. For example, if

you are concerned with real-time performance, then a useful monitorwould beone that does simple run-time performance analysis and tim

ing so that performance problems can be isolated quickly.

Resource-Usage Testing

Resource usage testing ensures that your application is using desirable

amounts of memory, CPU, time, disk bandwidth, network bandwidth, for

example. Resource usage tests canbe valuable to catch problems early and

without a personhaving to explicidy monitor them. Eachprojectwill have

its own set of resources that are more important to monitor than others.

Some of the factors are:

• The type ofapplication. Aweb-based application uses completely dif

ferent resources than a desktop application. Theymight both use a

database, but the desktop application isgoing to use nativeoperating

system resources, while the web-based application is going to use

resources on the web server and network bandwidth to the client.

• The operating system. Each operating system has its own set of

resources that areavailable to applications. For example, inMicrosoft

Windows it isimportantto payattentionto graphics contextsbecauseiftheyareleaked bythe application, the applicationmay crashwhenit

triesto create anewgraphics contextwhennomoreareavailable.

• Theprogramming language. Languages suchasC andC++arenotori

ous for ensuring that programmers must carefully manage their

memory allocations. If theydon't,memory leaks mayresult (memory

leaks are described in detail below).

Regression Testing

Regression tests are tests written to prevent existing features or bug fixes

from beingbrokenwhen new code changes aremade. Theyaren't a differ

ent kind of test, because these tests should be built into unit, integration,

and system tests. However, they are included as a reminder: Besureto add

DEFECT PREVENTION



tests when you fix a defect—there is nothing more frustrating than fixing adefect more than once!

Usability Testing

Usability testing is different from all the other types of testing already discussed because it cannot be automated. However, for any application with a

user interface, usability testing is a critical partof sustainable development

and should not be considered optional unless the interface is brain-dead

simple, and even then I'd still recommend it.

Usability testing involves observing real users in realistic conditions

using the product in a controlled environment. In a usability test [Rubin

1994] [Dumas and Redish 1999], careful attention must be made to not

guide or biasusers. Users are asked to perform a predefined taskand think

outloudwhile they use thesystem with more than oneperson observing and

taking notes. The reason several peopleshouldobserve the testsis to ensure

there isno biasor abilityto ignoreissues that arose. Whendonewell, usabil

ity testing will quickly uncover interface and interaction flaws that won't

normally be uncovered untilaftertheproductships. Hence, doing usability

testingasearlyaspossible is definitely a flavor ofbeingruthless.

Usability Testing Is Humbling!

Ican thinkof a number of occasions where Ihave observed a usability

test of a feature I implemented. Asan observer in these tests, it isvery

tempting to just jump upand demonstrate the "correct" wayto use the

feature, butthatwould defeat the purpose ofthe testing! Asa program

mer, it is easy to understand a feature and how to use it. However, I

have seen very few cases where a feature can be deemed usablewith

out some form of alteration.

User Verification Testing

Userverification testing is typically thelast type of testing that isperformed

before aproduct isinstalled and actually used bycustomers. It isn't necessaryforall products andwill depend onthecustomer. Forexample, government




agencies often demand verification testing, as do customers who use products where a product failure could result inthe loss ofa life, injury, or dam

age. This testing involves real users performing real work onthesoftware in

conditions that are as close as possible to how the end-users will use the

product, often in parallel with the live system. In applications such asmission-critical software, this testing couldtakeweeks tomonthsdue to the

highcostof failure afterthe software is installed.

Large Customers Need AutomatedTests, Too

Iwas fortunate to workwitha customer a numberof years ago that did

thorough verification testing before installing any new piece of soft

ware. This customerwas a large insurance firm with a complex com

puting environment. I think the company was visionary in its use ofautomated testing, becauseitdeveloped a comprehensivesuite of tests

and a separate test environment on an isolated network. Before the

company installed any software in its main network, it would install it

on its test network and run its automated tests for as many hours (or

days) as it feltwas necessary.

I enjoyed working with this customer because this company

always gave my team prompt feedback on problems it found withour

product. Every time the company found a failure, we made sure that

our tests weremodified to prevent the same problem from occurring

again. This led to an excellent working relationship. Becauseourautomated test suite caught problems before they reached this customer

and we had a proven ability to respond, this customer was not inter

ested inworkingwithanyoneelse.

Resources Required for Ruthless Testing

Ruthless testing is onlyeffectiveif tests are easy to run andresults canbe

obtained and analyzed quickly. Ruthless testing may require additionalcomputing resources and effort, depending on the size and complexity of

the product. Extra investment is required when it is impossible for the

teamto quickly and easily run and analyze testson its own. You may need

to also invest in some centralized or shared fast computers, although an

investment in distributed testing might be advisable, where spare CPU




PRACTICE 2: Use Available Tools

A large number of tools are available to software developers that can helpwith defect prevention. Some of the available tools offer easily automated

ways ofdetecting common coding errors. Othertools help a team communicate better through offering automated ways to document code and code

structure; better communication helps prevent defectsby making it easier

for the team to understand code and its structure before attempting to

change or enhance it.

There are some obvious tools, and some that aren't as obvious, as out

lined in the following paragraphs. In most cases, these toolsareveryeasyto

setup anduse, andmany canbe integrated intoyourproductbuildformax

imum effectiveness.

Teams that take the time to find the right tools for their projects andthen integratetheminto theirdaily workflow aredefinitely on the right track

when it comes to defectprevention. Better tools are becoming available all

the time,and it is important for a teamto knowwhat tools are available and

how they could help.

Compiler

A compiler is the perfectexample ofa tool that virtually everyone uses,but

that isnot always as effectively asit couldbe. Compilers are really good at

detecting common coding problems. Unfortunately, somepotentially seri

ous problems actually show up aswarnings, and the problemwithwarn

ings is that they are too easy to ignore. You should understand the

warnings that your compiler produces, prioritize the warnings, and, if at

allpossible, turn many of the warnings into errors. Then, make sure your

code compiles without the warnings you care about. The extra flags won't

add toyourcompile time; even ifallthe compiler doesiscatchwhatwould

have been a defect once a month, it's still worth the effort. That's one

defect a month that is caughtbefore your product reaches QA and your

c u s t om e r s .




Source Code Analyzer

A source code analyzer is a tool that detects common coding errors that

require more sophisticated analysis than that performed by a compiler.

These tools are obviously language dependent and typically come with adefault set of errors they look for, but they can also be customized soyoucanignorecertainerrors or evenadd yourownrules. Here isaveryshortlist

ofcommon problems thata source code analyzer could catch foryou.

• If youare using exception handling, it's almost always a bad idea to

have an empty catch block, since itmeans that if the exception does

occur, you have no control over how it is dealt with.

• Code that has probably been copied and pasted. It's obviously

impossible to performan exhaustive search for all duplicated code,

but duplicatedcode is always bad because it means that bugs in the

duplicated code need to be fixed in more than one place; what

shouldbe a single defectand fix almost always turns intomany.

• Short global variable names. Global variables are rarely desirable,

but if you have to use them, they should at least havemeaningful

names ...!

• Switch statements that don't have a default case.

• Deeply nested if statements and method complexity. Complexityis

usuallya measure of howmuch nesting there isof statements.

• An object'sconstructor method calling a virtualmethod. The problem in this caseis that the object could be partially initialized when

the method is called, which could lead to an unknown state.

Source code analysis tools are a pretty recent development. One of the

better tools that I have used is PMD for Java. PMD is open source

(http://pmd.sourceforge.net) and requires less thana day to setup;mostof

the work involves turning off checking for undesirable rules. I sincerely

hope that more open source projects are started to cover other common

programming languagesand development platformsand APIs, since everylanguageand API has good and bad uses.

If you are not using Java, PMD also comes with a tool called CPD

(copy-paste code detector). CPD workswithJava, C++, and C and iswell

worth looking into because it is surprisingly goodat finding code that has

PRACTICE 2: USE AVAILABLE TOOLS • 93



been simply copied and pasted. Copying and pasting code isa lazy shortcutthat developers often resort towhen they are, for example, creating a new

classwith a method almost identical to an already existing class. Copying

code is never a good thing because if there is a bug in the copied code,

invariably the bugwill get fixed in only one place. CPDwill help ensurecode that should obviously have been refactored in the first place never

makes it into the product code.

Test Coverage Analysis

A tool you should consider using is one that measures how manysource

code statementsare "covered" by your tests.Thismeasureof coverage can

be used to help add tests to get more coverage and even to remove tests,

which can be necessary whenyour tests take too long to run.

Test Coverage: Handle with Care!

Test coverage is an aid to your testing efforts and should never be a

focal point. It is a metric that is most usefulto ensure all areas of the

code receive at least some desired level of testing coverage. Beyond

that, the value is dubious inmy experience.

A development team at a large well-known software company

decided a fewyearsagoto focus onachieving 100 percent codecover

age in its tests. They actually did it, but found that their product was

unreliable and unstable. The reason? In order to achieve the 100 per

cent coverage, the developers removed error checking for boundary

conditions!

Execution Profiler

Anexecution profiler isatoolthat helps pinpointperformance problems incode. Typically, these tools give youa report of the methods/procedures/

functions in yourcode that are takingthe most timein a givenrun. Youcan

use these reports to examine themethods in question anddetermine if there

are anyobvious performance issues.




Execution profilers are often availablein an IDE or as commercial tools

or open sourcesoftware.Examplesof excellentcommercial tools are VTune

from IntelandSharkfrom Apple.

Event Logging

Event logging isadiagnostic toolthatlets you put "print" statements inyour

code to record a log of execution. Instead ofwriting your own, pick up an

opensourcelogger, ifyoucan,suchaslog4j (Java) or log4cplus (C++).

Source Code Documentation

Source code documentation tools typically extract documentation from

your sourcecode and generatea set ofweb pages. Some tools alsoproduce

class hierarchy diagrams and searchable indexes that let you find method

and variable names quickly. Probably the best-known source documenta

tion tool isJavaDocs, which is available aspart of the Java language. If you

aren't usingJava, or need to produce documentation for many languages,

there are manyopen source projects (I recommend doxygen), which you

should evaluatebefore attempting to buildyourown.

Memory and Resource Leak Detection

Memory leaks resultwhen a developer allocates memory for an objectbut

doesn't free it, and sheisusinga language and/or run-time environment that

has no memorymanagement (such asC and C++). I use the term resource

leakto refer to the related case when the developer uses a system resource

(such as a graphics context) and stops using it without releasing it back to

the system. Bothof these types of leaks result in serious defects and unpre

dictable applicationbehavior that customersdefinitely notice.

TIP A Simple LeakDetection Method

Onevery simple leakdetection methodyou can use is to rede

fine the memory and resource allocation and free calls with




your own in your debug build. All your redefined versions

need to do is increment or decrement a counter. Then, in your

unit tests you could simply run all the tests and then print the

value of the counter. If it isn't zero, you've probably got a leak

somewhere that you could then track down witha real detect ion tool.

Luckily, there are good tools available to help detect memory and

resourceleaks. SomeIDEs havebasicmemory leakdetectionbuilt in, so all

you need to do is turn on a compiler flag, and there are also commercially

available tools suchasPurifyandBoundschecker.

Configuration Management

Allassets developed for andusedin a project shouldbe storedin a configu

rationmanagement (CM) tool. Having a versioned copyof allyourchanges

will help prevent defects, because the good tools help the arduous and

error-prone taskofmerging newcodein with the existing code, especially

when there are multipledevelopers workingon the samepiece of code. A

good CM tool should allow a team to do basic version control (check-in,

check-out, labeling or "stamping" the current version of all files in a prod

uct, branchingof source trees, etc.) overa network and also provide some

form ofmerging changestogether.

The Value of CM

This practice seems completely obvious, but it is surprising howmany

software teams manage versions of their software simply by taking

copies and not using a CM tool.

This example is admittedly extreme, but one of my friends worked

at a company where the developers kept copies of their product ontheir local disk with only occasional attempts to keep their copies in

sync with each other. One of the most distressing stories my friend

related was when a customer received a new version of the product

from one of the developers. The customer found a problem in the

DEFECT PREVENTION



product and reported it, butthis caused panic because the developer's

hard drive had crashed and the backups were incomplete. The develop

ment team hadto try and piecetogether sourcecode thatwas as close

as possible to what the customer had installed, largely working from

memory of the changes made. They eventually fixed the customer's

problem, butwhat took them weeks should have taken only minutes.The distressing part of the story is thatafterthis crisis, the team con

tinueddeveloping as they hadbefore!

Cost isnot an acceptable excuse for neglecting to usea CMtool.There

are manygood CM tools available free or via open source such as CVS. If

you have themoney, itisalso agoodideatobuyhardware foryourCMdata

basethat provides some form of robustness, especially in case ofhard drivefailures.

CMtoolsareextremely easy to set up and use, and a goodCMtoolwillsave youatonof time. Thereis absolutely no reason tonot extensively usea

CM tool!

The Promise of Open Source

What if all the software teams in the world contributed to the creation

of tools that are meaningful to software developers, especially for

defect prevention? Commercial software vendors have by and large

done a poor job of creating the toolsthat software developers requireto be more efficient and prevent defects. Further, too many software

companies invest effort inthe creation of their own internal tools, usu

ally because they have a "tailored" environment (or think theydo), but

also due to a lack of knowledge ofwhat's available oreven an inability

to contribute because of legal reasons (the work most developers do

during work hours isthe property ofthecompany).

Open Source tools like junit (and all its derivatives), pmd, log4j,

cvs, bugzilla, and Eclipse all showthe potential of open source for cre

ating amore powerful software infrastructure for the industry. There is

anawful lotof duplicated effortgoing into proprietary tools,andif evenhalf ofthat effort were put into building a set of easy-to-use tools, wewould all be better off.




PRACTICE 3: Pair Programming

and Code Reviews

An important practice to prevent defects is to ensure that someone else

looks at new codebefore it ischecked in. This is true whether it's a bug fixor

apartofa feature and regardless oftheamount ofcode changed, even ifit's

only a single line! Themain reasons to have someone else lookat newcode

are to ensure that:

• More than one person understands the code. XP righdy calls this

"collective code ownership." If more than one person understands

the code, it means that there are manypeople who understand the

overall behavior of the system. This isveryimportant to defectpre

vention, because anyaddedknowledge in the teamofoverall system

behavior is a good thing and will help everyone on the teamthink

aheadandbe abletomore accurately predict wherefuture problems

might occurorwhere newchanges conflictwithor duplicate already

existing code.

• Sharedcodeunderstanding alsomeans that it iseasierfor teammem

bers to passtheir knowledge on to someone else, such as new team

members. It also prevents people being"locked" into onesection of

the codeor being a single point of failure because theyare the only

oneswhounderstand it. This is a bad thing;not onlyfor the individ

ual's careerbut also for the company (whathappens when the key

person get sick?).

• Ruthless testing is employed. Part of the job of the secondperson is

to think about the tests and ensure that the tests cover all the impor

tant cases. Inmany ways, the reviewer canlookatthe "fitness" ofthe

solution justbyknowing what theuserproblem isandexamining the

tests to ensure theymeet the desired criteria.

• The newcodeis not an ugly workaroundand doesnot duplicateany

alreadyexistingcode.• The new code does not contain any obvious logic errors.

There are two primary ways to ensure someone else looks at code

changes: pairprogramming andcode reviews. Unfortunately, most software




texts emphasize code reviews, but the optimal approach should actuallyconcentrate on pair programming, with code reviews as an adjunct

or outlet.

The primary advantage ofa code review isthat it gets anumber ofpeople together to discuss some code. Learning happens and good ideas are

exchanged. However, code reviews are actually very poor at finding logicerrorsunless aninordinate amount oftimeisspentinthe actual reviews, and

this extra time isinmy opinion hardto justify unless you arewriting soft

warewherethereis the potential of a catastrophic effect if the software fails

(such as a person being killed). Theproblem with finding logic errors isthat

thebesttime to find them iswhile thecode isbeing written; when presented

witha mass of code as in a code review the logic and order behind the cre

ation ofthecodeisno longer evident. Hence, pairprogramming.

Pair programming is simple. Twoprogrammerswork at the same com

puter on the sameproblem at the sametimewith one monitor, keyboard,

and computer.Pair programmingismuchmore effective than code reviews

at finding logic errors and preventing defects because both programmers

will bring their sharedand differing perspectives to the problem. In fact, all

the benefitsof havingsomeone elselook at code before it is checked in are

maximized with pair programming. Additionally, pair programming is an

excellent means to introducenew team members to the software by pairing

themwith amoreexperienced team member. There isnobetterway tolearn

a software system than to help solve realproblems in it.

The onlyproblem with pair programming is that it is a very intense

experience that can be stressful and possibly even demotivating for one or

bothpeople in thepair. It isn't for everyone andit shouldn't, inmyopinion,

be viewed as something that is done exclusively all day, every day, by every

member of a software team.

Thebenefits of pair programming are clear. Butwhatisn't talked about,

except by those who use pair programming as an excuse to not useXP or

agile development, aresome ofthedrawbacks ofpairprogramming. Forthe

"observer" in the pair, it can be annoying watching someone else type and

makethe same typing mistakes over(andover) again, typetoo slowly, jumparound toomuch, andinshortnothave "control." Fortheperson typing, it

canbe frustrating to have someone interrupt your train of thought ormake

suggestions that are on a wildly different line of thinking. More serious

drawbacks of pair programming show up when there is a personality

PRACTICE 3: PAIR PROGRAMMING AND CODE REVIEW • 99



conflict in the team. Although thesesituations are relatively rare, theycan

not be ignored because theycanbe very disruptive to the team.

I think the optimal approach to ensuring that someone else looks at

every software change is to usea mix ofpair programming and lightweight

code reviews. Pair programming should be used as much as possible, but

withan available oudet toworkindependendywithjointcode reviewswhen

required. The keythingis to keepthe code reviews lightweight usingsome

of the followingsuggestions:

• It is critical to followthe "continuous integration" practice outlined

inChapter4 onWorking Software. Changes shouldbe smalland eas

ily described. Don't wait days or weeks or months before having

code reviewed, because that will require a great deal of effort to

review and minimize the benefits of the review.

• Given small changes for review, exchange them frequendy in e-mail

or have someone drop byfor a quickwalkthrough.

• Avoidneedless documentation and "ceremony" for the review. Keep

it simpleenough to ensure that the benefits are gained.

• Buy a projector for your team and hold group reviewsessions with

critical code and its associated tests. Try to keep these fun and

focused on benefitssuch aslearning,not on trying to find "mistakes"

and "errors." Relyon the tests to find the major errors, so your team

can derive the most benefit.

PRACTICE 4: Lightweight

Root-Cause Analysis

Use regular lightweight root-cause analysis sessions to understand what

caused a defect and how to prevent it in the future. Root-causeanalysis is an

attempt to understand the real cause of a problem and to prevent similarproblems in the future. It's important to stress lightweight; the goal is to

learn and moveon so that you canminimize the number of defects in your

product andhenceminimize theamountof timespent fixing defects overall.

Make sure your team and your users know the difference between a

defect that must be fixed and one that doesn't or probably never will get




fixed. You can'tafford toletdefects hang around because any form ofdefect

backlog, evena short-term one, is a drain on a team's focus and time.

Youshould also considerhaving a special won't fix state for defects in

the following categories:

• The cost to fix the defect faroutweighs the benefit to the user. If the

user benefit is high, then you should seriously consider rebranding

thebugfix asa feature andtreatit assuch inplanning.

• Thebenefit to the userisvery lowor questionable.

• The defect onlyoccurs throughsome bizarre sequence of events that

is virtually impossible.

Markthesedefects won't fix and review thelistperiodically (perhaps oncea

year). Makesure that the databaseentry recordsyour reasoning, and if that

reasoning no longer applies, you might want to consider fixing the defect

after all.

Well-run root-cause analysis sessions are an extremely efficient mecha

nismto get a team to collectivelyunderstand typicaldefectsin a product and

alsoto build collective ownershipfor preventing thesedefects in the future.

The goal is to be super efficient time-wiseand to understand the answers to

three questions:

1. What caused this defect? If possible, record the cause of the defect

in your defect-tracking database. Hopefully, you can do this updatelive during the root-cause analysis session itself. The goal is not to

assign blame; the only reason to answer this questionis so you can

answer the followingquestions:

2. What mechanism, had it been in place, would have caught this

defect? This question helps categorize what is required to catch

defects beforetheyget to customers. For example, woulda testhave

caughtthe defect?More usertesting? Morecommunication between

users and developers?A simpleassertionfailurein the code?

3. How can we prevent this defect or this class of defect in the future?Thebest possible methodto prevent defects from recurring again is

towrite a testor seriesof testsand add themto yourtest suite.Fixing

defects is not an optimal use of time, and the greater the amountof

timeyour team spends preventing defects,the more timeyou'll have

to develop the features your customersneed.

PRACTICE 4: LIGHTWEIGHT ROOT-CAUSE ANALYSIS • 101 •



Of thesequestions, 3 isthemostimportant.Theonlyreasonyouanswer

the first two is so you can answer 3. This is counter to what someconsider

standard doctrine for root-cause analysis—that the answers to the first two

questions are where the effort should be spent. I strongly disagree. The

valuein root-cause analysis isin preventingother defectsand inpromotingashared understanding in the team about what can be done to prevent

defects. Quite often there are alsousefuldiscussions about what constitutes

good code and bad code.Bynot apportioning blame and focusing on pre

vention,youareputting the emphasison the positiveaspectsof the exercise,

in particular the aspectsthat emphasize team learningand collaboration.

It is important to have a simpleand efficientprocess for the root-cause

sessions. This ensures the process is repeatable, easily understood, easily

changed, and therefore lightweight. A simple process might be something

like:

• Have a computer in the sessionwith your bug tracking system run

ning so changescan bemade liveduring the session.

• The root-cause session is not just for developers. Users and other

team members should be present. Even if they get bored during

some of the more technical conversations, their presence helps keep

everyone motivated to keep those conversations short. Also, most

softwareprojectswillfind that a surprisingnumber of defectswould

have been prevented through better communication. Having every

one present will help ensure that there is shared recognition of the

communicationproblems so that everyonecanwork toward commu

nicating more effectively.

• Start by reviewing the priority of all the newly reported defects.

Users should have the final say on the priority. Determine which

defects aregoingtobe fixed before thenext session and assign them.

Somedefects maynot be defects at all but feature requests. Other

defects may require major effort to fix and so are effectively features.

Convert these to feature cards so they can be prioritized during

upcoming iteration planning meetings. Some defects will be fixed

later bywork that is already underway. These canbe kept as a back

log;hopefully, thesewillbe the onlydefectsin yourbacklog.

• Answerthe questions. Remember that the benefit of this exerciseis

the collaboration and the process, not the documentation of the




whether theyareQA, a testinggroup,or customers. Oneof themost critical

practices for sustainable development, ruthless testing, is described in this

chapter. Ruthless testing is a challenge to teams to go beyond test-driven

development and to build testability into their software.When software has

testability built in, then everyone, fromdevelopers to QAto customers, benefits. Anothercritical practiceis that of ensuring that someone else looksat

yourworkbeforeit isadded to theproduct.Thiscaneitherbe doneviapair

programming or code reviews, though pair programming is preferred

because problems are caughtimmediately.

Theotherprinciples described in this bookreinforce defect prevention.

A focus on aworkingproduct isimportantto ensurethat the teamdoesnot

spendallits time fixing broken tests. Likewise, an emphasis on design helps

defect prevention because one of the design goals should be building in

testability. Also, well-designed software is easier to test because it is well

structured with clean interfaces that can be easily identified and tested.

Finally, continual refinement supports defect prevention bygiving the team

a process whereit can regularly and frequently review and refine their auto

mated testingcoverage and processes.

SUMMARY • 105



Chapter 6

Emphasis

on Design

Design matters! Well-designed software is obvious to users and to the pro

grammers whoworkon it.Well-designed software is useful and easyto use,

maintain, extend, and understand.Andwhencompetition is intense,designis often th e difference between winner and loser.

In order to achieve sustainable software development, software must be

designed to support and enhance changeability. This is because agility is

possible only with the ability to easily modify and extend the software.

Imagine that youare taskedwithcreating a setofplayground equipment. If

the equipment you design requires awelding torchand hammer to reconfig

ure,you are in a ton of trouble ifyourcustomerasks for changes to the con

figuration. On the other hand, ifyourequipment isdesigned to bemodular

and is put together with some bolts and a wrench, you can respond toalmost any customer request. Software is unfortunately morecomplex than

playground equipment because it is never leftin any one arrangement, it is

continuallychanging and evolving.

In sustainable development, you cannot afford to have a product that

hasa high costofchange because your inability toeasilychange your prod

uctwilldictatea slow pace ofdevelopment thatwillleave youin continual

catch-up mode. Good designpracticesareoneofyour most important tools

to help controlthe cost ofchange.

Design clearly has a critical role in software development, regardless ofthemethodology used. Design is also the hardest aspect of software devel

opmentto get rightbecause of factors such as schedule pressures, the peo

ple on the project, and continual change. As a result, designs fall across a

spectrum that ranges between beingcw<?rdesigned and underdesigned, with

an unfortunate number ofprojects at either end.

107



Overdesignedprojectsare typically the result of overzealous application

of the engineeringapproach to softwaredevelopment.This is the traditional

waterfall top-down design method, which is still the most widely taught

approach to software development. Top-down design advocates thorough

designbefore coding,which of course involves understanding requirementsbefore design.This iswhatmostbuilding planners use:Theyhavea stable of

mathematics, standard questions to ask the customer, and building codes

that help themmake the right decisionsbefore they actuallybegin construc

tion. Unfortunately,when top-down designis applied to software, the result

ismost often overdesignand a high cost of change.That is, if any software is

produced at all—manyof these projects turn into exercises that produce

stacks of documents but no code.

The other extreme to top-down designis ad-hoc code-then-fix develop

ment, which is just as bad. In this case, there is litde or no design, andchangeis just ashard becausethe code is a tangledmessor hard to compre

hend. Underdesigned software is painful to work with. It is also unfortu

nately a great form of job security, because one of the symptoms of

underdesigned software is the fact that the code is dividedinto the exclusive

domains of various individuals. These heroes can crank ou t new features,

but heavenhelp you if theyeverleavethe project.

In order to achieve sustainable development, you can't rely on top-

downor bottom-up development. What is required is a middle ground so

you can capture the goodpoints of each: Youwant to have the discipline

and ability to think about gooddesign whileat the sametimeproceedingas

rapidly aspossible.

Extreme Programming [Beck 2004] outlines evolutionary or emergent

design. Thismethod relies on simple design (only design what you need),

refactoring (disciplined code changes), and test-first development (to ensure

that the behavior stays asoriginally intended—see Chapter5). Evolutionary

design works because ofa tight feedback loopwithendusers ofthe software:

The idea is to get the software into their hands as earlyaspossibleand then

evolve the design as requirements change. However, evolutionary design can

lead to unintended oscillation, where the same code is repeatedly changed

over andoverbecausesufficient thoughtwasnot put into the design:

One ofthe inherent dangers ofanyform of iterative development isconfus

ing iteration with oscillation. Good iterative design involves a successive

108 • EMPHASIS ON DESIGN



convergence on a workable, adaptable design. Poor iterative design in

volves flailing around randomly searching for a solution—and mistaking

this oscillation for iteration. [Highsmith 2005]

For sustainable development, you need to understand how to balance

thegoodaspects ofup-frontdesign andvarious design techniques withevo

lutionary design while avoiding the pitfalls of each. With up-front design

youneed to avoidoverdesigning the solution byerringon the sideof simple

design whileat the sametimeavoiding the documentation trap by focusing

on producing software not documents. And you have to avoid design and

code oscillation by thinking ahead and having a design vision and guiding

principles. The design vision and guiding principles are explained as prac

tices below.

Anotherwayto think aboutup-frontand evolutionary design isthrough

the understanding of good design and gooddesign evolution. Practitioners

ofup-frontdesign concentrate on understandingwhatmakes agooddesign,

while practitioners of evolutionary design focus on understanding designs

that resultfromevolutionandhowtheyevolved:

Ifyou'd like tobecome a better software designer, studying the evolution

of great software designs willbemore valuable than studying the great

designs themselves. Forit is in theevolution thattherealwisdom lies. The

structures that resultfrom theevolution can help you, but without know

ing why they were evolved into a design, you re more likely tomisapply

them orover-engineer with them onyour next project. [Kerievsky 2004]

The reality, however, is that design is hard because youneed to under

stand good design AND good design evolution. You can't understand

design evolution if you can't recognize good design, and design is pointless

without evolution because evolution is inevitable. Andgood design without

evolution is pointless, otherwise the tendency would be to design for the

sake of design. Hence, design has a yin (good design) and a yang (good

design evolution) and this I think explains why it is so hard to get right.Good design doesn't justhappen, it requires hardwork and thought plus

experience and knowledge. However, I also believe thatgooddesign doesn't

just emerge froma single individual, no matterhowbrilliant. Good design

requires collaboration, because collaboration provides support as the team

EMPHASIS ON DESIGN • 109



collectively works toward their goal while balancing the yin and yang of

design.

Design in sustainablesoftwaredevelopmentmeans creating an environ

ment where decisionsare continuallymadewithin a framework that empha

sizes change and designing for change, while ensuring consistency in the

decisions. Designwork is recognized asbeing crucial and something that is

done everydayby every memberof the teamin ascollaborative a fashion as

possible. The challenge is to ensure that the time spent on design tasks is

appropriate for what is being designedand that the effort spent document

ing the design is appropriate for the conditions of the project and is mini

mized to the greatest extent possible.

Other elementsof designin sustainablesoftwaredevelopment are:

• Agile development emphasizes having working software not compre

hensive documentation. In many ways, the agilemovement is a reac

tion to the almostcompulsive behavior that requires the production

of binder upon binder of designdocumentation. This type of docu

mentation is rarelykept up to datewith the code, and if it is, then the

amount of effortrequired to keep it up to date frequendy exceedsor

equalsthe effortrequired tomodifythe actualcode.

• Binders of documentation don't solve customer's problems, working

products do. Hence, agile development stresses the workingproduct

and lightweight ways of documentingthe designwork that the team

naturallydoes everyday.

• The requirement for documentation increases with the sizeandcom

plexity of the project orwhere there isdistributed development. If you

can't get allthe developers inone roomon a regularbasis, thenmore

documentation is required—with the goal, as always, of keeping the

documentation assimpleas possible.

• The primary value of design work is the process of doing the design

itself, not the documentation of the design. Documentation increases

comprehension andthe ability to communicate aboutthe design, but

reading design documentation isnowhere nearaseffective asactuallyparticipatingin the designprocessitself.

• Collaboration andface-to-face communication are a keypart of design

in agile development. The challenge is finding the right media to

enhance communication and collaboration. For collaborative design




PRACTICE 1: Design Vision

From a project managementstandpoint, iterativedevelopment is reallyhard

ifyoudon't havea vision. Hence,just asyouneed a vision for the project as

a whole,you should havea vision for your designaswell, evenif all you aredoingis adding to an already existing codebase.A designvision, or concep

tual design, is an overall picture of the pieces of your software: what they

are, what each does, and how they interoperate to solve the task at hand.

Without a designvision, oscillation will likely result because the team will

not have a clear idea of how the software is and should be structured.

The design vision can be a simple hand-drawn annotated sketch on a

whiteboard or a UML diagram.My personal bias is toward a hand-drawn

UML diagram on a whiteboard that is kept up to date. However, it doesn't

matter what form the designvisionis captured in, aslong asit exists.

PRACTICE 2: Guiding Principles

Guiding principles are a short list of statements that specifyoverall design

goalsand requirements. Their intent isnot to specify the designbut to help

team members make daily design decisions that are consistent with the

design intent. Some people also refer to them aspillars because they never

change, or are not changed in major ways. They are literally the ideas that

everything is built upon.

Guiding principles can be thought of as the visionfor the design.How

ever,where the visiondescribeswhat the project is about, the guiding prin

ciples describe how the project should be built. As with the vision, the

guiding principles need to be understood and agreed to by everyoneon the

team, and theyshould be created before the start of the project. Ideally, they

should be placed in a visible place where everyone can see them and even

pinned up at everyone's desk.

Twoimportant and distinctguidingprinciples are recommended (others arepossible):

• Engineering guiding principles. These describe how the project is

implemented in technical terms that the project team's developers

can understand.




through everydesigndecisionmade while the project is developed and help

ensure tha t the end archi tecture has the desired characteristics. The last

guiding principle is a statement that willmake the developers think: They

should avoid cutting corners and take the time to do things right.

The user experience guidingprinciplesmight be:

• Simple:Users of the existing systemmust be able to use the new sys

tem in less than 5 minutes

• Users can generate their own reports with a fewminutes of up-front

training and no ongoingsupport

• No glitcheswhen the existingsystemis replaced

These statements are tangible and measurable and can be used by the

product team to make daily decisions. Each of these principles stresses theimportance of a clean transitionto the new system in a differentway. Their

presence will help the team with their daily design work by focusing on

usability issues that willeasethe transitionto thenewsystem.

The vision provides the teamwith a clear statement of what they are

building and why. The guiding principles tell the team how the product

shouldbe built,withtheuserexperience guiding principles describingwhat

the user's experience should be like, and the engineering guiding principles

describing the "internal" characteristics of the implementation. In this case,

the guiding principles describe the importantcharacteristics of the newsystem that are required to address the deficiencies of the existing system

(which ishard to use, learn, extend, and maintain).

A Real-World Example of Guiding Principles

Adobe's Eve (http://opensource.adobe.com) provides an interesting real-

world example of guiding principles. Eve consistsof a language that is used

to specifyuser interfaces and a cross-platform layout engine. Although the

authors do not use the termguidingprinciples,theydo refer to the goalsand

requirements of the project:




Goals:

1. Makeit easierto specify andmodify the layoutof a human interface.

2. Have a singledefinition of the interfacefor allplatforms.

3. Have a single definition of the interfacefor alllanguages.

Requirements:

1. Must allowpiecemeal incorporation into an application, freeing the

entirehuman interfacefrombeingconvertedinonepass.

2. Generate layouts asgood or better than those generated byhand.

Thesegoalsand requirements areguidingprinciples. Theywere devel

oped before any codewaswritten and therefore serve as the design vision.

They are simple statements that serve as daily reminders for the develop

ment team as they make design decisions and tradeoffs. For example, the

first requirement ensures that the team creates an architecture that allows

existing applications to be converted to using Eve piecemeal. This must

have been a challenge to the developersbecause it implicidy dictates that

Eve be efficient and as lightweight as possible so that an application being

converted does not dramaticallyincreasein sizeor decreasein speed.

PRACTICE 3: Simple Design

Simpledesignmeans you should onlydesignand codewhat you know you

need while always striving for simplicity. In the pure definition of simple

design (such as in Extreme Programming)the goalwould be to onlydesign

what you need immediately. The problem with this purist approach is that it

canlead to oscillation, where a bodyof code is continually refactored tomeet

the current requirementsbecause there wasno designor planning ahead. If

you know that something is going to be required, then you should design it

and build it evenif thismeansthe usermayonlyseea limitedimmediateben

efit.But youneed to keep the designassimpleaspossible.Add the interfaceslater that youdon't immediately need, for example. Likewise, if a problem is

understood well enough to know that its solutioncan be found in a design

pattern, then the pattern should be used because patterns work and are well

understood. But patterns aren't perfect (see the Design Pattern practice

below), and againyouneed to always think about simplicity first.

PRACTICE 3: SIMPLE DESIGN • 115



Simple Design and a HotkeyManager

Simple design, when taken in its pure form (design only what you

need immediately), can be a dangerous practice. I observed a project

where a number offeatures required a hotkeymanager that mapped a

user-defined keyboard hotkey sequence into an application com

mand. Unfortunately, the first two developers who added commands

that had hotkeys associated with them did it by hardcoding the

hotkeys. Their code worked, but a third developer had to add a new

command and realized that the first two simple hotkey implementa

tions conflicted with the changes they needed to make. The result

was a major and messy unplanned refactoring exercise that se t the

project back.

Everyone on this team recognized they neededa hotkeymanager.

Some simpleupfrontdesignworkcouldhavemade the total amount ofwork much less. Admittedly, the addition of the first hotkey-based

command would have been harder because th e architecture would

have to have been built before the feature. But this work could have

beenscheduled as an explicit feature (it's goodto have visibility inthe

team that architecture work is being done),and itwould havemadethe

additionof allsubsequent commands trivial.

Scanning ahead is required. Think about evolving a simple but

solid hotkey manager over time, not trying to design the ultimate

hotkey manager immediately. Thisgivesyou the most amount of flexi

bility withminimal effort.

TIP Making architecture workvisible to users.

In agile development, it is sometimes hard to get architecture

work recognized. The cards are user-centric features, as they

should be, and users can too easilydismiss architecturework if

it doesn't havea recognizedbenefit.

One effective way I've seen to deal with this problem is to

attach architecture cards to user feature cards. The architecture

cards not only document what architecture and design work

needs to be put in place in order to enable the user feature, but

they also document the benefits (from a user's perspective) of

the work.

116 • EMPHAS IS ON DESIGN



In the hotkeycaseabove, a benefitof having awell-designed

hotkey manager would be that it makes the addition of subse

quent commands trivial. If the users can't see or understand a

benefit to architecture work, then perhaps the work isn't neces

sary after all.. .

Some readers may be uncomfortable with the gray area surrounding

being certain that supporting architecture and future extensibility are

required. I think it's important to accept that you're going to get it wrong

sometimes and over- or underdesign in some instances. That's software

development. Fix the problem as soon as you can, learn from the experi

ence, and move on. Teamwork, knowledge, experience, and collaboration

are vital to make the right decision.The more peoplemake designdecisions

with others and use each other as soundingboards for ideas, and the more

people learn from each other, and the greater the collective experience and

knowledge in the team, the greater the chance that the correct decisions are

going to be reached.

PRACTICE 4: Refactoring

Refactoring is theprocess of changing a software system in such a way

that it does not alter the externalbehaviorof the codeyet improves itsinternal structure. It is a disciplined way to clean up code that mini

mizes the chances of introducing bugs. In essence when you refactor

you are improving the design of the code after it has been written.

[Fowler 1999]

Refactoring involves changing software in a disciplined way so that the

structure of the code is changed but the code's behavior remains

unchanged. Refactoringis distinguishedfrommerelyrewritingcode (which

developers do all the time) by the discipline required. Refactoring is performed in a disciplined, step-by-step manner with tests in place to catch

problems as you proceed. Rewriting is performed in a more ad-hoc

delete/replace manner. Refactoring will help keep your code clean; rewrit

ing maynot.

PRACTICE 4: REFACTORING • 117



We Need the Vocabulary and Discipline

of Refactoring

Overthe last few years the term refactoring has entered the common

vernacular of software developers and managers. However, I am concerned that the term is now too commonly used because I don't see

enough people doing true disciplined refactoring. What has me con

cerned is that too many people are confusing generic code cleanups

(rewriting) with refactoring. We are losing the value of the vocabulary

and the discipline of refactoring.

Recently in a job interview, the person I was interviewing men

tioned that he was being taught refactoring in a university course.

When Iasked which refactoring methods he used, he lookedat me like

Ihadtwo heads. Aftera bit of discussion, itturns out the course did not

expose the students to anyof the refactoring terminologyand patternssuch as Extract Method, Extract Superclass, etc. Hence, the instructor

was using the term refactoring ina generic way that had no meaning.

And even worse, the students were "refactoring" without having any

automated tests! Tothe students inthis course, and to many others in

the industry, refactorhas becomea synonym for rewrite.

Butwe need the vocabulary of refactoring! Having a developertell

me she refactored the code is as descriptive as her telling me she had

an amazing lunch. However, if a developer tells me he extracted a

superclass froma numberof classes, Icanimmediatelyform an image

in my head of what the new class structure is and what the commonbehavior must be like. Icouldthen applythis knowledge when I lookat

the code later. Similarly, if she describes her lunchas an eggplant, yel

low and red pepper, and portobello mushroom sandwich with a salad

on the side, Ican get hungrythinkingabout it.

And we need the discipline of refactoring! Randomly rewriting

code does not mean that t he code is easier to understand or that the

design that results is better.The result could be a mess, and it could

lead to oscillation, as the same code is continually rewritten and

"cleaned up." Refactoring, on the other hand, should lead to better

design because of the steps involved, with automated tests in placeto

catch problems as the refactoring proceeds and to ensure that the

behaviorof the resultingchanges is correct.

The vocabulary of refactoring provides the power to clearly

describe what we mean. The discipline of refactoring helps us avoid

random code rewrites that lead to bad code and incomprehensible




designs. Let's not lose the value of the vocabulary and discipline and

challenge each other to refactor, not rewrite!

The quality of communication is higher with refactoring than with

rewriting. Thisisbecausewith refactoring thepersonwhomadethe changescan describe the changes through the type of refactoring that was per

formed. For example, a developercouldsay"In this changeI did an 'extract

method' refactoring, removing largely duplicatedcode frommethodsX, Y,

and Z." The type of refactoring (extractmethod) immediately conveys to

any other developer an immediate and rough feeling of the changes that

weremade and the complexityof the work requiredwithout havingto look

at the code. Without the refactoring discipline, then, the only recourse for

other developers to understand the complexityof the change is to actually

look at the code before and after the changesweremade.The level of confidence is higher with refactoring than with rewriting.

This is mostly because of the discipline required; with refactoring you are

discouraged from trying to make multiple changes at the same time and

rather to make changes in small, safe steps that can each be verified

through tests. However, when rewriting code, it is too easy to approach a

problem with wild abandon, which often results in a worse state or no

improvement.

PRACTICE 5: Design Patterns

Design patterns [Gamma et al 1995] are an invaluable resource for software

developers. They are essentially a catalogue of solutions to frequendy

encountered software development problems. Unfortunately, many devel

opers havenever heard of designpatterns, and others feelthat simpledesign

means they should not use patterns because using a design pattern seems

like up-front design. But if you are trying to solve a known problem, whyreinvent the wheel? The most common design patterns were not just

invented, they were observed by some clever and experienced software

developers as solutions that worked time and again in different software

projects and over time. Patterns are known to work, they are tested, and

they have a known behavior.

PRACTICE 5: DESIGN PATTERNS • 119



The value of design patterns as a common vocabulary should not be

underestimated. For example, I was in a design discussion one daywith my

team. One of my teammates started describing a solution in a message-

passing systemwhere a message was passed to a number of classesin turn

until one of the classes was able to deal with the message. This design provided the greatest amount of flexibility because it allowed the message-

handling classes to changeindependent of the messagesources.There was a

lot of confusionwithin the team about how this would work in detail, until I

realizedthat whatwasbeing proposed was the Chain of Responsibility pat

tern. As soon as I pointed this out, the tone of the conversation completely

changed. The confusion disappeared because everyone knew what this

pattern did. The discussion then switched to how the pattern could be

extended to meet the needs of prioritizedmessages and other detailed, not

philosophical, problems. Hence, our common vocabulary of a design pattern as a higher level concept allowed us to have a productive and focused

conversationon solvingthe real problems.

Design patterns can be abused just like any other practice. But just

because theyhaveproblems doesn'tmeantheyshouldbe ignored. Theyare

too valuable to be ignored, aslongas you understandwhat can gowrong.

Themost commonly citedproblemswithdesign patterns are:

• They are often overzealously used. Manyexamples can be found on

the Internet,in software development papers,or in booksofprojects

whereit seems like every lineof code cleverly employs some design

pattern. The softwarethat results from such projects is often unnec

essarily hard to understand and modifydue to the complex relation

ships between classes. These projects have sacrificed the rule of

simpledesignfor being clever.

• Design patterns are often usedtosolveproblems where a much simpler

solution is possible. If you use design patterns a lot, you'll find that

they're extremely reliable. They solve problems. However, once you

get to the point where you know patterns will work the majority of

the time, it's often too easy to think about how a design pattern can

be used for all problems and forget that sometimes a more ad-hoc

solution that involves a small bit of code is even better because it's

simplerl The mistake in this caseis thinking of design patterns before

simplicity.




• There are too many different ways tocode apattern. Since design pat

ternsonlyprovidean oudinefor implementation, it is still possible to

haveimplementations ofpatternsthat aretoo complex.

• Theuseofdesign patterns canleadtoanunhealthy emphasis onup-front

design. I have beenonprojects where people spent toomuch time tryingto analyze a problem sotheycould tryto identify whatdesign patterns

wouldapply. Ifyouaren'tsure,don'tuseapattern. Simplicity, asalways,

shouldbetheprimary goal. Chances are, ifthereisapattern,thereitwill

emerge.Youcanalways refactortoitlater, asin [Kerievsky 2004].

• If you don't know design patterns, the code can appear to be overly

complex. This really can't be helpedexceptthrough education.How

ever, it is a very real problem when new developers join a project

where patterns are used.

The Expected Behavior of Design Patterns

One of the problemswith design patterns is that once you are used to

them, you expect a certain behavior. I havebeen surprisedon occasion

when I look at some code and think, "Ahh, this is implementing the xx

pattern,"when in fact it is very close to the pattern but has slightly dif

ferent behavior.

I have encountered a number o f cases where the mistaken identifi

cationof a patternhas led to wasted time when tryingto find a problem

or understand the code. I highly recommend documenting the codewith the name of the pattern. Also, ensure that everyone on a team

understands patterns.

To be consistent with simple design, you don't need to implement

all the interfaces ifyou don't need them, andthe simple knowledgethat

the code is indeed a pattern means that other developers can quickly

add the interfaces they need at a later time with full confidence that the

expected behaviorof the patternwill not be altered.

In summary,design patterns are valuable and they shouldn't be ignored.

They do have problems, but as pointed out in the SimpleDesign practice,

teamwork, experience, knowledge, andcollaboration arethe bestwaytomini

mizethe risksin softwaredesignwhilemaximizing thepotentialreturns.It also

helpsto recognize thatmaking mistakes ishumanandpartof software develop

ment. Fixyourmistakesas quicklyaspossible,learnfrom them, andmoveon.

PRACTICE 5: DESIGN PATTERNS • 121



PRACTICE 6: Frequent Rapid

Design Meetings

Therapiddesign meeting isanimportantmechanism for a teamto ensureit is

having frequent design discussions. The output of thesemeetings is a shared

understandingof the design issues and a lightweight artifactsuchas a picture

ofawhiteboarddiagram. Thisisadeparturefrommoreformaldesignmeeting

wherethe output isa document. Thedesiredoutput of a rapid designmeeting

issharedunderstanding, learning, andcollaboration. Thisisa recognition that

the source code and automated tests should be where the designis ultimately

documented, supported by the most recent design illustrations, and the

emphasisshouldbe on the finishedproduct not the designdocumentation.

There are two types of designmeetings that teams can use: the design

discussion and the design review. A design discussion is when a number of

teammembers get together to discussdesigndecisionsof anykind. A design

review is an opportunity for a team to analyze its current architecture and

discuss short and long-term changesthat need to be made.

Designdiscussions shouldhappenmore frequendythan designreviews.

Most designdiscussions are going to be purely ad-hoc, when an issue comes

up, a number of people get together to discuss it. If ad-hoc discussions

aren't taking place on your project, then you have a serious collaboration

problem. You may find it useful to introduce a regularly scheduled (e.g.,

everyFridayfrom3-5) designdiscussion so the teamcan do a deeper review

ofone particular areaof the design,either what has been completed or what

needs to be added. These discussions are rarelya waste of time, because they

givethe entire team a chance to learn and collaborate.

Design reviewmeetingsmight be held after everyNth iteration, or they

might be added to the agendain a retrospective.Aswith a retrospective, the

goal of the design reviewis not to assign blame but to learn and provide a

positive future direction for the design. Typically, the design review would

consist of answering some basic questions about the product's design as it

currendy stands, such as:

• What areas of the current design are workingwell?

• What areas of the current design are not working well and should be

refactored or rewritten?

• Are there areasof the code that are too tightly coupled?




time be less than trying to bend the current architecture in the required

direction. Therefore, teams (and their management) need to be committed

to understandingwhenrearchitecture isrequiredandbe committed tomak

ing it happen. Using the chemical plant analogy fromChapter 1, thiswork

should be looked upon as preventive maintenance [md] e.g., taking your

pumps offlinefor work that helps them last longer.

Someof the common reasonsfor considering rearchitecture work are:

• A team has just releasedthe firstversion of a product. Shipping a ver

sion one of a product is the most difficult task any team can under

take, because team members are most often learning about the

product's ecosystem as theyare developing.They are bound to make

at least one, and most likely quite a few, design decisions that will

ultimatelylimit long-termsustainability.

• A product waswritten for a singleplatform (operating system, data

base, networking protocol, etc.) and it needs to be ported to a new

platform. These ports are an excellent opportunity to introduce

greater abstraction to hide the underlying platform-dependent

implementations. When done well, this type of abstraction should

simplifythe application and make it easier to modifyin the future.

• There have been changes to the underlying technology such as sys

tem apis or third-party libraries that the product depends on. Exam

plesmight be new input or displaydevices, or perhaps new hardware

or programming models.

• A single-threaded application needs to be multi-threaded. If the goal

is to maximize the usage ofmultiple processors, single-threaded soft

ware most often needs to be rewritten to break up its tasks into sepa

rable units of computation. Without this rework, usually the only

work that can be done is to optimizeloops or small sections of code.

This type ofoptimizationwillnot maximize useof additional proces

sors.

• It is natural that the team will understand the problem domain and

ecosystem better over time. Often, the current knowledge can be

used to further simplifythe product.

• The roadmap for the product has changed and in upcoming releases

the architecture is goingto have to support originally unanticipated

workflows. This case is tricky, since you need to practice simple




design and not build unnecessary architecture, but sometimes there

are some simple changes that will make these future changes much

easier.

• A section of the code is particularly defect prone, is difficultto mod

ify, and/or excessively coupled to other sectionsof the architecture.

When considering whether rearchitecture is required, it is vital to

remember the underlying tenets of sustainabledevelopment. Your product

is going to last for a long time—hopefullyit willlast far longer than you can

imagine. Consider the consequences of notmaking the necessary decision

when the cost islow (i.e., earlyin time)versusbeingforced to do evenmore

work in the future. It is vital that the decision to rearchitect or not should be

made consciously by the team. The most dangerous scenario is when the

need to rearchitect is ignored bythe teamor not discussed due to timepres

sures, external factors such as overwhelming customer requests or unrealis

tic expectations, or a laissez-faire attitude. Sustainability is often at stake,

and the team must have the discipline, and guts, to confront these issues

head-on.

Rearchitecture: The Good, the Bad, and the Ugly

In my experience, the most glaring need for rearchitecture workis just

afterthe completion of the first version of a product. There are manyreasons for this, but I think the most common is that development

teams, no matter how experienced, are bound to make some tradeoffs

and mistakes that will cost them over the long-term. I have strong

memories of two projects in particular, one good, and the other bad

(and ugly).

The Bad and Ugly

Thiswas a product thatwas built ineveryteam's worst-casescenario:

unrealistic deadlines, milestones that were repeatedly missed, and

attempts to bring the project back intoschedule by adding more peo

ple,getting peopleto work overtime, etc.

The product had an excellent early design vision that was well

modularized. To enforce the modularization of the architecture, the

build system had beenmodified to ensure thatdependencies werenot

PRACTICE 7: COMMITMENT TO REARCHITECTURE • 125



PRACTICE 8: Design for Reuse

The topic of reusablesoftwarehas somehowbecome the antithesisof agility,

especiallyamongmanyadvocatesof emergent design. And yet, the best way

to enhance responsivenessto newopportunities is to have something to startfrom. If you have to start from scratch or are continually reinventing key

algorithms, you arewasting effort, effort that should be spent concentrating

on the newaspectsofyourproject thatwillhelp you differentiatefromwhat

has alreadybeen done elsewhere.

Reusable sof tware has fewer defects and cleaner interfaces than non-

reusable software. Once a piece of software is used but not duplicated in

more than one place, it is going to be tested twice, and likely in different

ways. This makes the software more robust. When coupled with auto

mated tests of the interfaces for each use, the resulting software has amuch greater chanceof being extremelylowin defects than if it were used

only once.

The Business Value o f Reusable Software

For many projects, reusable software directly speaks to business value.

Iwas once involved ina project wherewe identified a market opportu

nity, but we had to deliver in less than six months. Ifwe didn't deliver

on time, the opportunity was gone. Luckily, we had two criticalreusable software components. Their existence made it possible to

deliverwith minimal risk, and on time. As Itold our executives later, if

we had not had reusable software, we wouldn't have even been able to

discuss the possibilityof delivering the project.

Reusable software doesn't emerge. Although reuse can be arrived at

through refactoring, there are often some key architectural decisions that

need to bemadeasearlyaspossibleand carried through the project tomake

subsequent workmucheasier and less prone to needingto be redone. Forexample, when extensibility is important, it's wise to design the plug-in

architecture early and then use it heavily, ideally so that allnewfeatures are

developed asplug-ins. Without this up-front design and architecturework,

the riskis that the system mightbe extensible, but not in a sustainable way.

• 128 • EMPHASIS ON DESIGN



Reuse is vital to sustainability because it minimizes wasted effort and

allows new initiatives to be launched quickly. Unfortunately, it is only

enabled by sound designand codingpracticesbecauseyou need to under

stand how to balance reuse againstwasted effort.

I suspect the reaction to reuse stems from projects where software is

overdesigned to make it reusable (which conflicts with simple design),

resulting in a complex, defect-laden mess. Thisiswheredesign intelligence

comes in :

• If youaren't surea pieceofsoftware is goingto be reused,at leastdo

everything you can to minimize coupling and dependencies with

other software modules, ensure there are automated tests for the

interfaces, and make it separately buildable (this is just common

sense). This at least makes it easier for the next developer to get

started andmakethe softwaretruly reusable.

• If youknowthe software definitely won't be reused,don't use that as

an excuseto ignoregooddesignpractices! (See the previouspoint.)

• Ifyouknow that there isahigh (>80%) likelihood of reuse, doit!But

keep the design and interfaces simple so that extension is possible

whenthe code really is reused. Don't overdesign andwaste efforton

aspects of the implementation youdon't immediately needandaren'tcertain of .

One of the largest problems and most important reasons to havereusable software isto eliminate duplicated code. It's tempting to justcopy

andpaste a section of code, andalthough there may be a savings in effort in

the short term, over the long term therewill be a large amount of wasted

effort. Duplicated code notonly makes thesize oftheprogram larger, it also

adds to complexity and is a common source of error, where a problem isfixedin one copyof the codebut not another.

Reusable software also eases replaceability. If some portionofthe archi

tecture needs to be replaced, it is always easier to takeout one sectionand

change its interfaces as required than it is to tryand replace the entire system. If thereareno interface changes required, even better, because thenat

least you can reuse the automated tests onthe interface to ensure your new

implementation has the same behavior.

PRACTICE 8: DESIGN FOR REUSE • 129



The notion of completely componentized software has been around

since the introduction ofObject-Oriented Programming. The notion ofsoft

ware ICs (i.e., softwarechips that can be wired together,much like chips on

a printed circuitboard) [Cox1986] comes from the success of product line

manufacturing. The automotive industry, for example, has reduced costs

and defects by using as many common components (body panels, wind

shield wipers, motors, seats, instruments, etc.) as possiblewithin a line of

related vehicles.

Unfortunately, the idealof fully componentizedreuse as in manufactur

ing still largely eludes the software industry. I think this is because of the

constant changein the software ecosystem and the levelof overallcomplex

ity. However, despitethe complexity, it isstillpossible to attainahighdegree

of reuse, and this aidssustainabilityby allowingteams to be more responsive

to opportunities and to avoid duplicating effort. One ofmyhopesfor open

source software is that over time we can dramatically cut down on dupli

cated effort within the industry and allow a greater amount of common

infrastructure to be developed and enhanced over time.

Summary

Design is vital to sustainable development because in order for software to

lastoverthe long term, it mustbewell designed. The problem, however, is

that design is hard. There is a dilemma that every project faces and that is

knowing when to design up-front andhowmuch effort to invest in up-front

design versus letting thedesign emerge. Gooddesign practices should stress

finding a balance between up-front andemergent design andemphasizing a

collaborative design process thatisnotbuilt around heavy requirements for

documentation of the design.

I believe it is important to startwitha vision and guiding principles for

the design. These arethe goalposts that team members mustconsider every

day as they make tradeoffs. From there, the design practices should stressdesign iteration, constant change, design visibility, and collaboration. Mis

takes are inevitably made, where software is over- or underdesigned. Soft

ware teams should rely on a focus on simplicity and a desire to avoid

duplicated effort plus teamwork, knowledge, experience, and collaboration




to reduce the number of mistakes and their severity. It is vital that design

mistakesare fixedasquicklyas possible,the appropriate lessonsare learned,

and then the teammoveson. It is alsocriticalthat the team has the guts and

instincts to make difficult decisions, such as to rearchitect before the cost of

change becomes too high.

SUMMARY • 131



lightweight aspossibleso that the focus is placed on achievingresults,while

the tight feedback loopwith customers encourages teams to do onlywhat is

most usefulto the customers, whichhelpsminimizewasted effort.

The main advantage of agile methods is that they help teams manage

uncertainty and changeearlyand throughout the project.However,whileall

agileprojects combineaspects of anticipationand adaptation, poorlyimple

mented agilemethods can result in sloppyplanning and reactive thinking

[Highsmith2004b].Sloppy planningresults whenteamsignoreinformation

they already know or should knowwith minimal effort. Reactive thinking

resultswhen too much emphasis is placed on the team'sability to adapt to

change. The resultcouldbe a projectwhere lots of changes are beingmade

but the amount of actual progress is low. Hence, the challengewith agile

methods is to relyon both adaptability and anticipationand to havea team

that knowswhen and how to applyeach,and inwhatmeasure.

Traditional methods of software development rely too heavily on

upfront planning and anticipation and virtually ignore adaptation. These

approaches stress attempting to understandalluser requirements and pro

ducinga detaileddesign beforewriting anysoftware.

Agilemethods arenot theexact inverse of traditionalmethods; the com

mon agile statement "we have more confidence inour ability to adapt than

our abilityto predict the future" shouldnot be anexcuseto ignoreanticipa

tion. Instead, in agile methods there must be an understanding of good

enough anticipation. Forexample, spending a few days atthebeginning ofa

project understanding user requirements or spending anhourevery day dis

cussing software design areworthy activities because they cancut down on

reworkand even, if done right, increase the ability of the team to adapt to

change. Thelatter topic, especially asit applies to design, is dealt within

greaterdetail inChapter6onDesign Emphasis.

The needto involve both adaptation and anticipation in agile methods

is similarto the need to considerboth the short and long terms. In software

projects, if too much emphasis is placed on the short-term, then software

quality will suffer because the team will becontinually reacting to crises and

notbeing proactive. This will result ina continual degradation ofthe abilityto adapt to change as described inChapter 2.Equally as dangerous iswhentoomuch emphasis is placed on the long term (i.e., anticipation); these are

the projects that are continually at risk of never shipping because they are

unableto dealwith changing user requirements through their emphasis on

planning and design.

134 • CONTINUAL REFINEMENT



Agile Practices Are Still UsefulWhen Adaptation

Is Not Required

There aresome projects whereagility seems less important, wherethe

temptation exists to stick with traditional approaches. Examples arewherea software defectmight lead to a loss of life or injury orthe fail

ure of an expensive piece of equipment such as in medical or aero

space applications, or in projects where milestones aretightlydefined

andmust beapproached ina given order. However, even inthese proj

ects there is value in lightweight collaborative planning and project

tracking as described inthis chapter, because virtually all projects can

benefit from the steady rhythm and discipline of regular planning and

tracking available in iterative development. Teams don't need to make

changes to their plan in every iteration, buthaving the ability to change

ifneeded could bethe difference between asuccessful or failed project.In my pre-agile days Iwas a believer in Microsoft Project and its

Gantt charts. However, despite repeated attempts, Ineverproduced a

project schedule that stayed valid for more than a few weeks. If I didn't

makean effortto create a new scheduleeveryfewweeks, then Iwould

proceed blind and trust my instincts, and because creating a new

schedule was so much work, Iwas probably flying blind more often

than not. This iswhere a lightweight planning and tracking method as

advocated byagile development would have been extremely useful.

PRACTICE 1: Iterative Development

Iterative development canbe thought ofasplan a little, do a little, learn alit

tle. Iterative development isa core practice ofsustainable software development because it gives the team a lightweight way to make measurable

progress and to get feedback from customers as frequendy and as early as

possible. Getting feedback as early as possible iscritical to ensuring that the

systemmeets user requirements, that it is assimpleas possible, and that thereisminimal wasted effort. Minimizing wasted effort (features that users don't

care about, features thatare built inunusableways, etc.) isanimportant business consideration because wasted effort has a cost and is inefficient.

Iterative development is amazingly simple. The schedule is broken upinto a number of short equal-length iterations (2 to 6weeks). At the end of

PRACTICE 1: ITERATIVE DEVELOPMENT • 135



Iterative Development Is a Good Techniquefor Open-Ended Research Projects

This is because research projects are almost always characterized by

critical decisionsas to whetherto continue orhalt exploration of a possible path to a solution. If a relatively short iteration time(2to4weeks)

is used, then the iteration reviews become an excellent forum to review

the current progress and make continue/haltdecisions.

This canbe contrasted to managing a research project withoutthis

sense of time. In this case, it is oftentoo easyto continue down "blind

alleys" too long. Ofcourse, iterations can oftenonlybe planned one at

a time since there is minimal insight into future problems. However,

since many false avenues are stopped earlier than they might other

wise be, research projects often produce results faster with iterative

development than without!

Collaboration and Iterative Development

Iterativedevelopment ismost effective when there isbroad participationin

iterationplanning.At the beginningof each iteration, the entire team (busi

ness people, technical staff, and customers) has a short and intense iteration

planning session. In this planning session the team analyzes and sorts the

current set of desired features into an iteration plan based on risk (do therisky things early) and priority (determined by the customer). Tradeoffs are

madethroughconversations withinthe teamduringthe planningsession.

Cross-functional collaboration is important to sustainable development

because there is no hiding risk from anyof the participants. Hence, active

riskmanagement is encouraged.Also, because itisa team exercise, everyone

involved understands the current product status, what still needs to be

accomplished, and alsowhat canbe accomplished in the timeavailable. And

because the planningis focused on providing value to customers realtrade

offscan be made, for example, betweennew features and making existing

features work better fo r customers.

Iterative development is a humbling (in a good sense!) experience for

the entire team. It's always easy to get enthused about a particular feature,

but you can only introduce it once it is compared to the other features

already in the plan.Collaboration helps to ensure that the righttradeoffs are

PRACTICE 1: ITERATIVE DEVELOPMENT • 137 •



PRACTICE 3: Daily Standup Meetings

Hold a standup meeting the same time every day that is attended by the

entire team—including businesspeople. The meeting should take an

absolute maximum of 15 minutes (standing helps encourage this). The dailystandupmeeting is importantfor sustainable development because it helps

to get issues out in the open each day, which helps to ensure that people

aren't blocked for long periods of time and that everyone on the team is

aware ofwhateveryone else isdoing. Thetraditional alternative is typically a

weekly or monthly status meeting, and the problem with these meetings is

that issues areoften raised toolate,that they're too long, andthatthe critical

topics don't get raised.

In a daily standupmeeting, eachpersonanswers threequestions:

• What did you do yesterday?

• What are yougoingto do today?

• Are there anyobstacles to yourprogress?

Keep track of time and limit discussion to ensure that the meeting fin

ishes quickly. Invariably, issues come up during the meeting that can be

dealtwith immediatelyafter the scrumby thosewhoare interested.

Standupmeetings arehighly effective at gathering status,gettingimpor

tant issues out into the open, and encouraging collaboration among the

team.Themosteffectivewayto solve problemswhenmore than oneperson

needs to be involved is face-to-face rather than in e-mail; the standup meet

ing, and the collaboration that happens immediately after, is an aid to the

criticalissuesbeing dealt with face-to-face immediately.

It is alsoimportant that dailystandups lead to solving the problems of

themomentimmediately after the standup is finished. Weekly statusreport

meetings are not nearly as effective because manyproblems are no longer

immediate and the delayis a potentialkillerof teammomentum. Likewise,

e-mail can have a similar effect because there is a disconnect between th e

raisingof the problem and anypossiblesolution.Of course, there are always

impromptu meetings, and some problems will still require these, but the

existence of the daily standup means that team members know there is a

forumwheretheycanget these issues out in theopen easily.

PRACTICE 3: DAILY STANDUP MEETINGS • 141



Daily Standup Meetings andDiscipline

As with any type of meeting, daily standups require discipline. Time

discipline and topic discipline are the hardest things to master with

dailystandup meetings.

One team Iworked with had daily standupsthat lasted morethan

an hour! It's too easyto want to dive into a topic during the standup;

someone has to suggest that people who are interested can get

together after the standup meeting. It makes senseto have someone in

charge ofthe meeting to getit started and to truncate discussions that

are clearly goingto take too long.

It'salso hard to know how much to say when it's your turn in the

standup meeting. I've seen people that default to saying too little, so

that some of the obstaclesaren't broughtup, andalso people who say

too much so that too muchdetail

is outlined. Again, this is where itmakes sense to have someone in charge of the meeting, to both

prompt with questions for more detail and also to provide feedback to

people after the standup ifthey provide too muchdetail.

PRACTICE 4: Retrospectives

Retrospectives [Kerth 2001] area critical part of sustainable software devel

opment. In order forcontinual refinement to be effective, the teammust be

able to continually learn and adjust not only its project {what the team is

building) but also themethods members use {how the teamworks).

Teams should hold retrospective meetings at regular intervals during

theirprojects. Ideally, thiswould be at the end of every iteration or the end

of every secondor third iteration. The purposes of these retrospectives are

to provide a forum where the teams canopenly question theircurrentdevel

opment processes andprogress, learn from theirmistakes and failures, and

decidewhat changes theyneed tomaketo proceed effectively.

One ofthekey aspects ofa retrospective isthatyoumust layout groundrulesin advance. It isvitalthat a retrospective be a positive experiencethat

focuses on learning and not on laying blame. Start by assuming that every

one on the team is doing the best he or she can,with the recognition that

thereis always room for improvement foreverymember ofthe team. Ideally,




The duration of retrospectives is a variablethat must be carefullymoni

tored to ensure they do not become a burdensome overhead. Frequent

checkpoint retrospectivesthat are held during a project should take an hour

or less,while retrospectives held at the end of a major releasemight take a

day or more, depending on how elaborate a retrospective is required.

Traditional Methods: Plan a Lot, Do a Lot,

Learn a Little

Traditional methods of software development are best termed plan a

lot, do a lot, learn a little. Great effort is spent before any software is

written to understandthe user's requirements and design the system.

Then it is built, usuallywith little or no customer feedback.

Customer feedback usually doesn't occur until a beta version isreleased to select customers. But beta releases are almost always too

late to do anything about customer-reported issues or requests

because they're usually late in the release cycle. Because customer

feedback is late inthe release cycle, there is a high chance of wasted

effort as features that aren't critical to customers are buil t or features

are built in ways that aren't usable. And because customer requests

often lead to riskychanges thatwould jeopardize the eventual release,

they often get moved into the next release, when in most cases the

changeswill be less useful, not to mention late!

Usually, teams are so busy building their products and trying tomeet theirdeadlines thatthey holda post-mortem atthe completionof

the project. Butalong the way, they have missed many opportunities

forcourse correction and process changes.

PRACTICE 5: Coaching

and Team DevelopmentCoaching and team development are importantfor continual refinement to

help the team achieve the bestpossible results while collaborating together

and with customers. Coaching is an interpersonal skill that helps people

provide eachotherwith real-time feedback sotheycando theirworkmore




effectively. Real-time feedback is important for continual refinement

because it ensures problems are dealt with as quickly as possible. Team

development (understanding the business and professional development)

are equally important because the more the team understands the business

andhascurrent skills, the better itwill be at both anticipating and initiating

changesin its ecosystem.

Coaching

Coaching is an important interpersonal skill for every memberof the team.

Coaching is essentially providing individuals (people who work for you,

people you report to, and people youworkwith), teams,and customers the

real-time feedback they need to become more effective and continually

improve. Coaching helps minimize friction and negative tensions whilehelpingpeoplework better together and focus on the desired result.

Understanding the Business

Understandingyour company's business is an importantelementof sustain

able development. The knowledge gained through this understanding is

vitalin anticipating and understandingchanges to the ecosystem your proj

ect is dependent on. While some would argue that software developersshould focus on writing code, I believethat this too often results in undesir

able side effects, such as:

• Team members who don't understand what is valuable to their cus

tomers andwhy, even when told. This is often because they do not

understand how the customersuse the product and hence what is

important to them.

• People who cannot listen to customers and understand what they are

really asking for. Understanding what the customer is trying to do is

critical because users most often ask fo r alterations to their current

workflow only, whenperhaps the problemis theworkflow itself.

• People whodon't understand what provides value to customers. Some

times,work that provides value to customers ismundane, but it must

be done.

PRACTICE 5: COACHING AND TEAM DEVELOPMENT • 145



• People who don't understand the economics of their business. How

does your company makeitsmoney? Sometimes, customers ask for

majorfeatures. If youalways listento your customers, then theywill

get everything for free. Your company needs to stay in business,

though, and it may be more desirable to sell the customer a newproduct or add on to an existingproduct.

• Incompanies with multiple products, especially where there is overlap

ping functionality, itiscommon forpeople oneach team towant toadd

allfeatures their customers askfor to their product when infact only

oneoftheproducts may needit.Also,wheremultiple products serve

the needsofa single customerormarketsegment,theremaybe detri

mental competition between the product teams; these teams need a

major wake-up call to realize that the competition they need to be

concerned with is outside their company!

• People who don't understand the economics of their markets. Too

many technically brilliant products are conceived to address what

may be a sexy or obvious opportunity, which unfortunately is not

viable economically due to a small target market or unrealistic pric

ing. Other problems maybe misunderstanding the technical sophis

tication of users and building a solution that is too complex or too

simple for them. Or, developers may not understand the economic

reality of their customers where, for example, end-users may not

value a softwareproduct at a realistic level because they (the end-

user) are faced with massive costs in other areas that dwarf any

potential for software.

• Teams that don't pay attention to their development processes. I have

seentoo manyprojects, where due to a whole rangeof factors such as

poor planning and design, sloppy programming practices, a lackof risk

analysis, defectbacklogs, etc.,a project teamis required that is too large

for thesize ofthetargetmarket. Theseareprojects that arequickly can

celled, because the investment does not justify the returns: i.e., why

shoulda company invest in a projectwith lowerreturns than ifitwasto

take themoneyandput it in a bank account to collectinterest?

• If the onlyvalue youprovide to an organization iswriting code, then

your job can be outsourced (i.e.,worked on by cheaper labor).

You need to understand your company's markets, users, competition,

and financials. Youwon't always have access to all this information down to




the finest level ofdetail, but you should have enough access to askintelligent

questions or be able to ask questions to get someinformation. Don't look

upon it as "management's" responsibilityto giveyou this information;effec

tive communication goes both ways. People in management are almost

always too immersed in the details of the business to realize what type ofinformation youneed.

Mylargestobservation of peoplewhowork inmost companies todayis

that they are too isolatedfrom customers. Try to visit your customers or at

least ask peoplewho do visit customers to write up reports or give a short

talk about what they learned.Bringyour customers to visityou as often as

you can.

Direct Customer Contact HelpsCreate

a Balanced View

Recently, one of my co-workers went to a trade show and attended a

user forum for the product he works on. He went in thinking he had

better bring a bulletproof jacket because all he had seen were defect

reportsand postings on user forums. He camebackcompletely rejuve

natedandvery positive.Sure, there were some unhappycustomers but

even they said, "Hey, I know I complain about your product, but it

rocks. I couldn't do without it!".

Projectteams often get a distorted view of their work ifthey don't

have enough direct customer contact. This is because it is human

nature to want to report problems and to say nothing when things are

goingwell, and customers are human. Since most problem reports are

done through impersonal e-mail orweb forms, it is also human nature

for many peopleto be less diplomatic than they should be—problems

are frustrating after all. If all a product team sees are the defects that

customers reportand negative e-mails, then the team is likelylackinga

balanced view of what customers really think.

One of the things I have little tolerance for is a software team that insistsit needs to completely rewrite the product. Too many developers don't

understand the company'sneed to stayin business. The problem, of course,

is that it's always easier to write code than to read it, especially when you

didn't write it. And it's no fun just fixing bugs in someoneelse's code. But

few companies in the software industry can afford to start again from

PRACTICE 5: COACH ING AND TEAM DEVELOPMENT • 147



scratch. That's why teams should be encouraged to rewrite parts of their

products as required and aggressively and constantly refactor as recom

mendedinChapter6.Thealternative isnot pretty:Whilea product isbeing

rewritten, your competition is developing newfeatures whileyou are effec

tively standing still. Evenif you build in new features, you are still going tobe behind simplybecauseof the lostmarket share. People who get this are

highlytreasured byorganizations (or at least should be!).

Professional Development

Professionaldevelopment is important for both personal and project rea

sons [Hunt and Thomas 2000] [Hohmann 1996]. From a project stand

point, teamsthat are knowledgeable in asmanyareas aspossible, eventhose

that are seemingly unrelated to the current project, have the best chanceofsucceeding at sustainable development.The reason for this is simple:These

teams have a varied set of skillsand knowledge, a rich tool chest if you will,

that can be drawn from and appliedwithmaximaleffect.Professionaldevel

opment is vital to ensure that the team has all the skills it needs to under

stand its ecosystemofmarkets, competition, and underlying technologies.

At a personal level, your skills are vital to your team's success and to

yours. It is your responsibility to ensure you are continually improving in

your profession. You can't rely on anyone else to tell you to take courses or

read or learn. You have to be proactive.You can't blame anyone else if yourcareer does not progress as you would like. Read, take training courses,

experiment with newprogramminglanguagesand approaches to problems,

and above all learn, collaborate, mentor, and teach.

PRACTICE 6: Make Key

Metrics Visible

Everyproject should collectsomekeymetrics and findways to make them

visible to the project team.Visibility might be achieved by a simpleweb

page or maybe even a big visible chart on a whiteboard or by the water

cooler.The reasonmetrics are helpful is that they can, if they are presented

in the right way, help the team focus on keeping important aspects of its




project under control. Metrics helpthe team track itsprogress on a regular

basis andget early warning signs when problems arebeginning to develop.

Entire bookshavebeenwritten on the subjectofmetrics, and I think some

timespeople get carried away. A small set of simple metrics should suffice,

andif their collection and display canbe automated, then your teamwill bein goodshapeto continually monitor and improve its progress.

Anoverlooked aspectofmetrics is that theyenhance learning. Bymak

ing them visible, they help drive meaningful discussions about the trends

and how to change or improve the trends. If done in a positive way, these

conversations help drivethe collaborative problemsolving that is invaluable

to the team.Of course, ifmetrics are implemented in a negative way, suchas

by tracking individual instead of team metrics, then you can expect the

opposite effect.

Someexample metrics might be:

• Charts of velocity andfeature points remaining (asdescribed above).

• A chart thatshowsthestatus ofall system tests, andhowlong they took

to run. This helps to catch system and performance problems early

and prevents caseswhere the systemtests don't run.

• A graph that shows the amount of time required for nightly build(s).

This helps to keep the builds assmall and fast aspossible,which is a

critical issue to ensure developers are asproductive aspossible: Slow

builds mean lower productivity.

• A graph that shows the number of outstanding (notfixed) defects, by

severity. Outstanding defects are a keyindicator of sustainabledevel

opment. Teams cannot carry around large backlogs of defects and

should fix as they go.

• A graph that shows the number of incoming defects, by severity over

time. This is an indicator of how well tested the product was. High

incoming rates might lead teams to set aside more time for fixing

defects in the short term, or perhaps help point to areas of the prod

uct that need refactoring/replacement.

• A graph thatshows thenumberoffixed defects perweekovertime.

• A companion to theprevious twographs (incoming andfixed defects) is

a graph thatshows the net gain. The net gain is essentially incoming

minus fixed, which should alwaysbe around 0 and as close as possi

ble to a flat l ine.

PRACTICE 6: MAKE KEY METRICS VISIBLE • 149 •



A chart that tracks the amount of source code versus test code. This

gives team members visible evidence that they are adding enough

tests as their source code grows.

A chart that showsdefect density, in terms of number of defects (on

average) per thousand lines of code.

Summary

Continual refinement is an importantprinciplefor sustainabledevelopment

because it enables teams to both adapt to and anticipate changeinwhatthey

are doing (their project) and how they are doing it. Teamsbreak down their

projects into short, manageableiterations.The advantage of breaking down

work in thisway is that complexityis much easier to manage in small slices,

and immediate feedback fromusersprovides the teamswith a powerfulway

to changedirections if theyneed to. In addition, teams employregular retro

spectives so that they can talk about their projects in an open and honest

way. This allows them to make adjustments to their processes, which is

equallyimportant to sustainability becausecontinual learningand improve

ment help ensure the projectwilllastover the long term.

• 150 • CONTINUAL REFINEMENT



Chapter 8

Culture Change

and Sustainable

Development

It is highlylikelythat in order for yourorganization or teamto achieve sus

tainabledevelopment, a newproduct development cultureis required.I feel

this chapter is necessary because I have seen too many situations where

enthusiastic people with great ideas are stymied, consciously or uncon

sciously, by their organizations.

Too few people consider how to introduce change to their colleagues

andmanagement and howtomake the changeslast. So-calledchangeagents

almost always have a great deal of enthusiasm, but they often take on theentire burden of change themselves, hit a stumbling block or opposition,

and then do one ofmanythings:retreat into a shell,lash out, get burned out,

or find another job. Manyorganizations, usually unknowingly, have a built-

in immune systemthat can quickly stiflechange.But evenin these situations

change is not impossible if you understand the dynamics of the people in

the organization.

Where changeis required, there is the significant challenge of achieving

lasting change. It's easyfor people to try out something new, but as soon as

something goes wrong, the most common reaction is to revert to what wasknown to work in the past—even if it doesn't really work. Lasting change

requires changes to what is measured, recognized, rewarded, and rein

forced. For example, many companies recognize people who work long

hours and who are independent project heroes, whereas in a sustainable

development culture, it would be more valuable to recognize people who

151



find innovativeways to achieve technical collaboration on their projectsand

make the people around themmore effective.

Achieving lastingculture change ishard. It takes a lot ofdetermination,

vision, andpersistence. Principles andpractices aren't goodenoughbecause

people defineculture.And peopleare complex and unique. Therefore, it is

unrealistic to expectthat a sustainable software development culture canbe

so clearlydefined that it isidenticalin everyinstance. In fact, the opposite is

true;weshouldexpectsustainable development culturesto be different sim

ply because of the people involved, and these differences will manifest

themselves throughvariations in howpractices are applied.

There are two change scenarios of interest in this chapter: sustainable

development on anewprojectand turning aproject that ison the unsustain

able path into sustainable. These situations are obviously unique and

require variations in approach. This chapter is therefore broken up into

three parts:the common topicofmaking change happen, followed byadis

cussion of the unique aspectsof the two scenarios.

Understanding

The primary challenge inattempting culture change is in understand

ing the dynamics of each uniquesituationand the people, culture,and

environment, then using this knowledge to determine how change can

be first started and then maintained ina highly collaborativeway.

Cynics caneasilyread a chapter like this and come awaywith the

impression that leadership and introducing change involves manipula

tion. But you needto applya filter as you read. Think about the impor

tance of truly understanding the situation and about how critical

positive collaboration and engaging others is. You often have to help

people understand theircurrent situation andwhatthe possibilities are,

and you have to help influence the change, but you can't do it alone.

Forget aboutmanipulation andcommand-control; they won't work!

Making Change Happen

Manygood books are available on how tomakechangehappen in anorgan

ization. For reference, I've included someofmyfavorites in Appendix 4.

152 • CULTURE CHANGE AND SUSTAINABLE DEVELOPMENT



Iwish itwerepossible to providea template formaking change happen.

Something along the lines of: If you're a programmer and want to change

your organization to sustainable development, do this, then that, say this,

etc.Mypersonal experiencewith leadingchangehasbeen that everyorgani

zationis different, that this isparticularlytrue of softwareorganizations,andthat softwareorganizations are especially difficult to change. The problem I

think stemsfrommanyfactors,including:

• Softwarepeople comefrom many backgrounds. I'veworkedwithpeople

who have degrees in engineering, physics, mathematics, astrophysics,

computer science,psychology, sociology, English,music,business,and

accounting.Many of my co-workershavehad multiple degrees;some

haveevenhad no degrees.

• In many organizations people who understand amazinglylitde about

softwaredevelopment, sometimes appallingly so,wind up managing

the organization. Note: I'm not necessarilysayingthis is a problem. I

haveworked for somemanagerswho are excellent at knowing what

they don't know and who to ask—although I've also worked for

managers who don't, which is a problem. It's difficult to find good

managers who love both business and software development. Usu

ally, people with a sales or financial background and/or people who

have written a few lines of code sometime in the past are the ones

who wind up in control because they tend to be more extroverted

and/or charismatic than most technical people. Management is only

a problem when it isbad or incompetent.

• Many organizations do not discussor try to consciously foster their

culture and development practices or pay attention to key people

issues such as professional development and leadership. Instead,

they focus on projects, schedules, and results.While there isno ques

tion that results are important, if the bottom line is the main focus

then there is a problem.

• There are a lot of incredibly bright people in software organiza

tions. On just about any topic of importance there will be no short

age of opinions. This can be both good and bad depending on the

topic at hand, and it can lead to situationswhere creating consensus

is difficult or even impossible. However, being bright in a problem-

solving sense doesn't necessarily translate into being a good com

municator. Many people in software organizations are good talkers

MAK ING CHANGE HAPPEN • 153



but not listeners, and some are not very good at expressing their

opinions in a respectful way. Hence,whilehavinga group of bright

people may seem like a good thing, when it comes to making

change happen, sometimes the fact that everyone has an opinion

and that some people are poor at expressing their opinions meansthat resistance to change can be quite high. Or, resistance might

just seemhigh because people are usually very good at expressing

displeasurebut not asproactiveas theyshould be at providing pos

itive feedback.

• Software education lacks standards and a poor understanding of

what the basicsare.Mostpeople learnmore about softwaredevelop

ment on the job than theydo in schoolbecause onlyin the workplace

are they confronted with the task of maintaining a million lines of

code for ten years or joiningprojects where a large amount of codealreadyexists and the people who wrote it are no longer available. I

also know people who have graduated from top universities in com

puter science who have been well schooled in theory such as NP

complete problems but haven't taken a course in compilers or com

puter construction!

• People don't like being told how to work. They want to be able to

choose the best approach based on learning. Hence, the challenge

with achieving culture change is finding ways to achieve complete

buy-in and then helping and providing adviceas required.• People are different.Consider this basic list of character traits: intro

vert, extravert, opinionated, stubborn, consensus-builder, excitable,

emotional, and restrained. Softwaredevelopers tend to be introverts.

Salesand marketing people tend to be extraverts. Add age asa factor

aswell:Older and more experienced workers tend to be more jaded

by past experiences (or perhaps they are justmore relaxed).

• Organizations are complex simply because of the mix of different

people, for better and worse. What will work in one organization

likelywill not work in another.• Complacency. Wealmostalways underestimate the effort required to

introduce change into an organization. Complacency is often at the

heart of failedchangeeffortsbecause ifyour co-workersdon't under

stand the need for change theywon't activelyparticipate.

• It's hard to admit that you have to change, too. We tend to generalize

when things get uncomfortable. Driving a car is a perfect example.




When asked, weconsistently saythat there are toomany jerkson the

road and that other drivers causethe problems. And yet that clearly

can't be th e case because someone has to be the other driver. Few

people admit that they need to improve their driving skills when

actuallyweshould allbe continually striving to getbetter.• Try this experiment with your co-workers. Have all of them cover

their eyes and askpeople to raise their handsif theybelieve theyare

open to change. Then, with eyes still covered, ask people to raise

their hands if they believe other people are unwilling to change.

What youwillinvariablyseeis that virtuallyeveryhand willbe raised

in response to both questions! The implication should be obvious:

that whileyoumay think you are open to change, the impressionyou

give to other people may be different. Change has to start with you

and it isyourresponsibility to find waysto engageothers.

None of these factors is catastrophic. They're reallyjust the people factors

you need to keep in mind when considering how to introduce change into

your organization. People, and the culture they have built for themselves,

are probably the largest variable between organizations.People and culture

are also the largest potential barriers to change. Youhave to recognize that

any change challengesthe comforts that havebeen built up over time.

Change Factors and Enablers

In order to create change, I believe the following change factors must be

present:

• Leadership

• A sense of urgency

• Executive support

And the following change enablers enable the overall changeeffort:

• Persistence

• Training

• Cont inual "wins"

CHANGE FAC TORS AND ENABLERS • 155



Figure8-1The reaction people

haveto any change

initiative is most likely

going to take the

form of a bell curve(ornormal)distribu

tion. Some people are

going to be apathetic

or be openly resist

ant, and others are

going to be enthusi

astic and wantto get

to it. Most reactions

are goingto fall

somewhere in

between, and the fac

tors that are going toinfluence whether th e

bulkof the peopleare

going to resist or

enthusiastically sup

portthe changes are

whether there is a

sense of urgency that

everyone can identify

with, the degree of

executive support,

and th e involvement

of key leaders at

every level inthe

organization.

• 156 •

#

People

"What we do

today works

fine"

Degree of

Acceptance

Factors:

Leadership

Sense of urgency

Executive support

"Let's do it!"

• Positive reinforcement of desired behaviors

• Communication, vision,and strategy

Each of the above topicsis coveredin detailbelow.

In my experience, you're going to get acceptance or resistance in vary

ing degrees that usually resembles a bell curve, as illustrated in Figure8-1.

The point isthat it'svery rarethat youcanconvince everyone right away, butyou can influence how the changes will be perceived and accepted by the

majorityof people.Andmost importandy, youcan't manipulatepeople, but

you can engagethem.

Change Factors

Leadership

Leadership, for somereason,is a term that manypeople shy awayfrom. Per

haps this is because of the negativeconnotation: that leaders are the senior

managers in the company and that leadership is hence top-down. Certainly,

manymanagementbooks arewritten that way, with the need for changecom

ing from the CEO or senior management, and various techniques such as

CULTURE CHANGE AND SUSTAINABLE DEVELOPMENT



strategic goal setting and coaching are employed to get the organization on

board and steer the right course. The other extreme is the bottom-up

approach, wherechange isinitiated at thevery lowest level of theorganization

andthenpercolates up throughthe hierarchy. ThisisoftenhowExtremePro

gramming gets started in an organization, with a software team tryingit outand then the practices beingadopted byother teams. Thecommonthread in

both these extremes isthatno single person canmake change happen.

In order for anychangeto succeed, theremustbe leadership atall levels.

Change starts with individual leaders. These are people who realize that

change has to start with what they can control: themselves. They start

change by first occupying a new space themselves, then finding waysto get

other people (their co-workers, their boss, their team, and their subordi

nates) to want to occupy that space with them, and then helping them to do

so. By this definition, there are leaders at many levels in an organization,

from the programmerwho prods her co-workersand team leader into using

automated tests, the team leader or coach who helps his team achieve its

best work and ship a product, the product managerwhoworks with users to

ensure they understand the importance of their input, to the CEO who

leverages the expertise of her teams to set the business goals for the organi

zation.The trick inmaking changehappen isfindingasmanyleaders aspos

sible and at asmany levelsof the organizationsas possible, developing and

empowering them, and getting them actively involvedin the changeprocess.

Sense of Urgency

Changewon't happen unlesspeople understand and agreewith the need for

change. One of the most important tasks before trying to initiate change is

to develop a message that clearly and emphatically states the need for

change. The newway can't just be better; it has tomake a positivedifference

to everyone in the organization and its customers, and by extension to the

company's bottom line.

Know Your Audience

One of the key aspects of developing a sense of urgency is having a

message that can be tuned to the audience. Executives and senior

CHANGE FACTORS AND ENABLERS • 157



managers need to have a completely different understanding of the

need for change thandevelopers orQA people, for example. Yourabil

ity to communicate to the various audiences will help determine how

much importance will be placed onthe changeinitiative.

Executive Support

Changecanhappen without executive support, but the more executivesup

port there is, the easier the changeeffortwillbe. An executive is reallyany

one who controls the ability to spend money and helps steer company

policy—thehigher in the organization, the better. Executives can help free

up funds and people's timefor training; they can alsohelp with communica

tion and rewarding desired behaviors. A word of warning: It may also be

necessary to convinceexecutivesthat they need to change, too—for exam

ple, if their styleis overbearingor if they are unable to collaborate.

Change Enablers

Persistence

With any change, there are going to be peaks and valleys in effort, perform

ance, and results.You have to go in knowing that these are going to occur

and that youwillneed continual effortover a longperiod of time. The peaks

are going to be periods of timewhere it will be easy to relax and relish suc

cess toomuch, whenwhat youreallyneed is a new set ofgoals that push you

on to even greater accomplishments.The valleys are by far the most trying

periods, where it seemslike everyoneis ignoring the messageor reverting to

their old ways. It is completely natural for people to revert to the familiar

during trying periods because the familiar ways are going to be more com

fortable than new approaches. A common valleyin a change initiative is at

the beginning (remember the chemicalmanufacturing example in Chapter1), where there is a dip in performance as the team deals with all the same

problems using new approaches. These are the times where leaders must

lead, to ensure that the organization doesn't get too comfortable when

everything is going well and overreact when it's not.




Team Sports and Persistence

Competitive team sports are an example of the need for persistence.

Great leaders in sport (coaches andplayers) are those who help their

teams keep their composure during games so that they are always

mentally in the game.

Ifthe team iswinning, itmust not let upor its opponentwill come

back. Ifthe team is losing, itneeds to nevergive upandnot get so des

perateto make a playthat playerscommit mentalmistakes. Likewise, if

a player commits an error, he or she can't afford to sulk or layblame;

instead that player needs to admit the mistake, learn from it, and be

ready forthe next opportunity. Andcelebration of success is important,

but not ifit leads to a false sense of security.

The most physically gifted team does not always win. Too many

teams build a lead then relax. Orthey celebratea victory prematurely.

Orthey let one or two losses ina rowerodetheir confidence.Orthey

commit stupid penalties at crucial times. Great teams win because

they're persistent, and they don't let the peaks or the valleys affect

them through being disciplined.

Training

Training is often vital to help people learn new skills and understand new

methods.It canbe donebybringingconsultantsin (externaltraining)or usingyourownpeople(internal training). Eachtypeof training hasits ownadvan

tagesand disadvantages, but the keyfactoris that whoever leadsthe training

hasto havecredibilitywiththe audience. Externalconsultantsshouldbe care

fully selected to ensurethatyourorganization ismaximizing its investment by

having thebest expertswith the deepestexpertise (andcredibility) possible.

Internal training mayseem like a luxurybecause it requires that people

havetimeto prepare the coursematerial.However, inmyexperience, it canbe

powerful and effective, especially when one of the organization's respected

leaders puts together appropriate materialwhere he or she can pass on important experiences or diveinto a topic that is relevantto the organization.

Training topics should be carefully chosen. To introduce sustainable

development into an organization, I recommend a minimum of training on

agile project management, hands-on technical training for technical staff,




plus some kind of executive training to give an overview of the methods

being introduced and the business rationalebehind them.

The Value o f External Consultants

External consultants are often invaluable to assist with training and

establishing a sense of urgency. They bring experiences from other

companies, andthese experienceshelpthem communicate successful

strategies employed.

Also, sometimes what an external consultant says will hold more

weightthan an internal employee.Often,havingan externaland neutral

voice ask a hard question that has beenasked a million times before by

employees is enough to shake people out of their sense of compla

cency. Hence, especially for larger organizations, I highly recommend

that you identify and hire an experienced consultant who will havecredibility in your organization.

Continual "Wins"

The most important thing to aimfor is to produce continual "wins." These

canbe in theformofnew tests, new tools, changes in defecttrends, success

ful projects, or whatever ismeaningful to yourcompany. Theyareparticu

larly vital in the early days of change and should be targeted as soon aspossible, mosdy to ensurethat peopleseethat change ispossible and that it

is recognized (via communication). You cannot expect change to happen

quickly, and so you must use the continual wins to reinforce success and

keep everyone aware of thefact that progress isbeingmade.

Positive Reinforcement of Desired Behaviors

Change requires new behaviors that must replace existing behaviors. For

example, here are some of the common behavior changes required when

introducing sustainabledevelopment to an organization:

• Project leaders must be able to create an environment where every

one participates in iteration planning. In many organizations the role




of the project leader is to identifythe tasksthat must be completed,

collect estimates from the developers, create a schedule, and then

track progress against the schedule.

• Refactoringand rewritingmust be fully supported byeveryonein the

organization.

• People must learn to resist the pressure to get features done. If the

product is not in a working state, what's the point in completing the

feature and adding to the problem?

• In many organizations, executives or product managers have the

mos t c on tac t with customers and teams are i nsul at ed f rom the

"demands" of customers. It's almost always best to set up direct and

frequent collaboration with customers.

• Developers must write tests and work at defect prevention. If devel

opers are still rewarded for the number of hours theywork and fea

tures theychurn out, regardless ofquality, theywillnot embrace their

role in defect prevention, and no culture changewill result because

they will still look to a testing or QA organization or customers to

find problems.

• People must collaborate. If heroes are rewardedfor saving projects

and actingas prima donnas, then collaboration willbe lessimportant

than being a hero.

Communication

Lots of positive andwell-timed communication is a critical changeenabler.

At everycompanyI haveworked at, communication hasbeen identifiedby

employees as one of the principal shortcomingsof the organizationor man

agement team. I think this is a reflectionof the fact that you can never com

municate enough.

The purpose of communicationduring changeis to keep people excited

and in tunewith the importance of the changes.It ispossibleto communicate

too much, which iswhy it isvery important to think about the timingof yourcommunication. Definitely talk about the continual wins the team has

achievedand recognizethat over time thingswillbegin to look drier and less

exciting.Those are the points where youmayneed to set a new set ofgoalsor

get somenew people involved or find somenewwaysto get the messageout.




Start with What You Can Control

and Influence

Ametaphor for how to get started in turning around a project is catchingup

on e-mail after an extended absence such as a vacation. The way I deal with

this situation is I start with what I can control, which is the e-mail I receive

the day I return. This ensures that the backlogdoesn't get anyworsewhile I

work on older messages. I will first look for recent high-priority messages

and dealwith those. Then, as time permits, I will start by deleting (without

reading) messages that are clearly junk or out of date. I will then look for

mail threads (i.e., multiple messages with the same subject) and start by

reading the most recent message on the thread. In most cases, I can then

delete all the older messages and be caught up on the subject. By the time

I've finished thisprocess, veryfewmessages willbe left, and lifewillbe back

to normal becauseI'll be caughtup.

Movingto a culture ofsustainabilityisobviouslynot the samenor is it as

simple asdealing withan e-mail backlog. However, ifyouareableto recog

nizethat youneedto startwithwhatyoucancontrol,knowwhereyouwant

to end up, and haveanunderstanding of the stepsyoucan take to get your

self out of the current situation, then the change scenariobecomes more

plausible. Youcan't try to solve everything at once, and you need to think

about the problem in understandable increments that keep you on the

desired path. Changeinitiatives willfailif they attempt to boiltheocean or

lack focusor a clearsenseof firststeps and overallvision.

Think of yourorganization as a supertanker; it is big and takes a long

time to change direction. Giventhat youknowwhere youwant to end up,

recognize that you can't make the entire turn at once. Instead, what are the

things you can do at any point in time that nudge the supertanker in the

desired direction?

Avoid Transition Plans

In my opinion, one of the most common mistakes made when introducing

change is to use a transition plan. In my experience, they don't work. The

problem with transitionplans is it is commonto rationalizereasons for why




the transition plancannot be followed, such as a recession or a critical proj

ect or feature that must be completed first. Typically, transition plans are

something that canbeput inplacelater oraresomehow optional. Latermay

never come. There must be a strong desire for change, and change often

means taking a riskor beingaggressive. Transition plansare rarely risky oraggressive.

An analogy that one ofmy colleagues cameup with is losingweight. In

order to loseweight, you can't talk about the need to loseweight and the

need to change your lifestyle by eatingbetter and exercising regularly. Suc

cessfulweight lossis doing it,not talkingabout it, starting immediately not at

some time in the future or by phasing in better food andmore exercise.

Turning Unsustainableinto Sustainable Development

If you are working on a project that is on the unsustainable development

curve, the most important thing you must recognize is that you can't just

"blow up" your current processes and practices. That would most likely

lead to disaster. Next, you need to recognize that achieving sustainable

development is going to take time, and probably more time than you think.

Youalsohave to recognize that you are going to have to make compromisesand can't strive for perfection; you likelywon't ever have the architecture

you'd likeor have tests in place for everysinglelineofcode.Just think about

getting continuallybetter over time, in tinyincrementsif need be.

If you can, start with a small team. Then, use the successes and knowl

edge gained in that team to get other teams involved. You might want to

consider moving people between teams if you can. Newpeople added to a

highlyefficientteamwillbe able to learn the new principles quickly. How

ever, you need to be careful when moving people from the "model" team

to other teams because they can easily get buried in the day-to-day realityof their new team unless they are strong, persistent leaders/coaches or can

be moved in pairs or sub-teams. You might want to think of these sub-

teams as a "tiger team", that can be added to any project and lead change

by example. But these people must not think of themselves as better or

superior.

TURNING UNSUSTAINABLE INTO SUSTAINABLE DEVELOPMENT • 163



Unsustainable projects, or those on the path to unsustainability, are

likely going to have a few key problems that are going to need targeting.

Most likely, thesewillincludeaspects suchas:

• Legacy codethatispoorly structuredandhasmany

brokenwindows.• Inadequate or nonexistent test coverage.

• Attitudes that result in the use of phrases like "ship to QA" or

"throw it over the wallfor testing." These are indicators of a defect

detection culture, where developers are not doing enough testing

themselves.

• Entrenched development practices and attitudes that are counter to

what is required.Thelongerpractices havebeenused, theharder it is

to change them and the easierit is for people to go back to them at

the first signof trouble or stress.

• Defect rates that are unsustainable and increasing.

• Bureaucratic project managementpractices that emphasize planning

and tracking progress againstthe plan.

• A lack of understanding of the current development problems and

visibility into theirmagnitude.

Where to Begin

Assuming that you have buy-in to change and have decided on your

approach (training,consultants, etc.), where do you begin? Start by analyz

ing the mindset ofyour organizationand mapping it against the four princi

ples: continual refinement,working product, defect prevention, and design

emphasis. Giventhat yourearlygoalisgoingto be buildingwins,the follow

ing practicescan be used to produce results quickly:

• You can get an errant project back on track quickly using the agile

projectmanagement practicessimplyby beginning iterativedevelop

ment, trackingyourvelocity, usingdailystandupmeetings, and retro

spectives.

• Start ruthless testing by emphasizing unit tests for new code and, if

you can, by putting safeguards in place such as system level tests to

catch regressions sooner.

• 164 • CULTURE CHANGE AND SUSTAINABLE DEVELOPMENT



Introduce simpledesign and refactoring.

Ifyou aren'talready doing so, make sure your product isbuiltnightly

(completely, from scratch) and that you are using a configuration

management system.

Pay attention to your design practices. If you do a lot of up-front

design currently, don't stopdoing design because you don't thinkit's

agile. Don'tgethungup onsimple design andemergent design. Start

with frequent rapiddesignmeetingsand architectural reviews.

Make sureyouhave a defect tracking system inplaceandthat youare

rigorously tracking defects. Then, start collecting somemetrics to

find out what kind of defect trend and backlog you have. This will

helpplanyournext steps to helpget defects under control.

As you become familiarwith the new practices, use your retrospec

tives as a way to review the practices from this book (or any other

good source) and introduce newpracticesas you seefit.

The Three Phases of Change

There are three phases that a change initiative goes through. Each

phase has its unique challenges and pitfalls. It helps to be aware of

which phase you think your change initiative is inso the team can have

realistic expectations and goals.

1. In the initial phase, the challenge is to get started and to

develop the first wins. A common failure in this stage is to

have expectations that are too highandthen to give up when

they aren't met. Enthusiasm has to be high in at least a core

group of people to make a difference. It may be a good ideato

form a guiding council to channel the initial enthusiasm.

2. In the second phase, the challenge is to work through the

inevitable peaks and valleys of enthusiasm and results. A

common failure in this stage is to assume the job is done,

wrap everything up, and then a few months later realize that

too many old behaviors are coming back.

3. The third phase is one of ongoing maintenance. The maingoal

in this phase is to simply ensure that the keystone principles

and practices are in place andthat there is continual learning

and improvement. Probably the most common failure in this

TURNING UNSUSTAINABLE INTO SUSTAINABLE DEVELOPMENT • 165



stage isto lose discipline ona keystone practice suchas retro

spectives so that opportunities to learn and improve are

missed.

Sustainable Development

for New Projects

In many ways, introducing sustainable development principles and prac

tices for a new project is going to be easier than on an establishedproject.

Teams on new projectsare almostalways determined to do things the right

way. It's goingto be possible, forexample, to employruthlesstestingbyhav

inggood test coverage and building testabilityinto your project.However,newprojectsareoftenlaunchedwith unrealisticexpectations,

or they are too ambitious.If you're going to start with new practices, I rec

ommend you start with the keystone practices from Appendix 1 and then

use your retrospectives to tune and add to your processes over time.

Another problem with new projects is that people are often trying out

new practices for the first time. I have found that it's vital to have the team

talk about its practices in an openway, especially through the initial phases.

For example, I worked with one team where the team embraced simple

design,but a fewdevelopers had read aXP book and thought simpledesignmeant no up-front design. This resulted in some of the code being refac-

tored repeatedly and some obvious features not being planned for. The

result was chaotic until the team realized there was a problem and members

were able to communicate openly about what they wanted, so they intro

ducedweekly designmeetings until the designsetded down.

Probably the most dangerous problem with new projects is that their

newness rapidly fades. New projects often face immense pressure in that

they need to produce a feature-complete project in as short a timeframe as

possible. If there are established competitors, this can be an uphill batde.There are going to be more valleys than peaks, and a great deal of persist

ence and discipline isgoing to be requiredto stickwith and refinethe prac

tices when the temptation is going to be to just crank out the features

without consideringwhat typeof quality tradeoffs need to bemade.




Some Other Ideas

Every projectand situationisgoingto be different, soit isimpossible to pro

vide a step-by-step solution to your problem. I've tried to outline someof

the common approaches I'veusedor seen applied to various situations, particularly when dealing with a legacy product that was not developed using

sustainabletechniques.

Identify Key Architecture Battles

Do an architectural analysis, where you identify thingssuch as poor struc

ture and areas with a large number of defects. Every time you modify

these sections of code, evenif onlyfor aminor change, considerdoingsomerefactoringwork in addition to justmakingthe changes, evenif thismeans

setting aside extra time tomake the changes. Besure to add tests at least for

the new code, and if you can, identifythe keyinterfaces and add tests for

them.

In your iteration planning sessions,write up somecards for the refactor

ingwork.Users,product managers,andmanagers arein myexperiencevery

supportiveof refactoring work if theyare involved in the processofhelping

to set overallpriorities and understand, at least at a high level, the purpose

of the refactoringwork. The visibility that this gives to the architecture of

the product isgoodfor the entire team,whileat the sametimeensuringthat

architectural issues are weighed against the feature work that users are

demanding.

Get Your Defects Under Control

Hopefully, you should now be convinced that defect backlogs are a bad

thing. Introduce somevisible metrics, and possibly an ambitious goal such

as HP's "lOx reduction goal" [Grady 1997] to get people focused on the

problem. Then, workhard toward a culture builtaround defect prevention

and working software.

SOME OTHER IDEAS • 167



Collaboration with Users

Formany organizations, oneofthemost difficult stepsin introducing agile

development isgetting timely feedback from users. This isan ideal problem

for seniormanagement in the company to get involved with, to set up thenecessary usercontacts and also toget everyone in the organization focused

on the need for greater user contact.

Training

Training is a crucial part of moving from an unsustainable to sustainable

development culture. Training needs to be considered for the entire team,

ideally at the same time. In the early stages, using an externalsource (con

sultant or training organization) for training is usually required, because

evenif someoneinside the companysays the samethings as the consultant,

the consultant will often be listened to more, simply because he or she is

from outside. However, once the initial training is complete, it is highly

desirable to set up an internal trainingprogram,where people inside the

company aregiven the opportunityto trainotherson topicstheyare knowl

edgeable in.

Group Refactoring Exercises

An unorthodox idea that I've seen work is a competitive refactoring exer

cise. Pick a pieceof code that hasmanybroken windowsin it. Then, get a

bunch of people together in a conference room and do a pair program

mingexercise to refactorthe code.Considerpairing people togetherwho

don't worktogethertoomuch.Usea projector to review the solutionsthat

each pair came up with during the day and discuss the pros and cons of

each. Optionally, an award couldbe given for the solutionthe group likes

the best. This exercise is time-consuming, but it helps everyone under

stand the difference betweengood code and bad, and the result can often

be appliedin the live codethe next day. Also, the discussions are valuable

for everyone, and thewhole process canbe fun with anunderlying thread

of competition.




The Work Environment

Onefactor thatisoften overlooked when introducing change isthe impactof the physical space the team works in. If you possibly can, change the

physical space that the team operates in to facilitate collaboration andreduce unnecessary interruptions. Sometimes, simple changes can make

huge differences in terms of productivity, and the highly collaborative

nature of agile development demands a work environment that does not

inhibit collaboration and free communication. Too many organizations are

penny wise, pound foolish and skimp on equipment that will increase pro

ductivity simply because the equipment is considered too expensive. Here

are some ideas to consider:

• Build a project room for the teamor converta meetingroom into a

project room. Cover the walls of the room with whiteboards and

corkboards, and thinkof this space asthe locationwhereanyone can

go to get an immediate ideaof the progressof the project.The Apple

II project, for example,had a war room that gave everymember of

the companyinstant insightinto the project [Lynn and Reilly 2002].

• Makesure there are lotsofwhiteboardsreadily available and installa

projector so the team can hold productive team code and architec

ture reviews. Magnetic whiteboards are also an ideal place to keep

the index cardsfor iterationand release planning.

• Do people who run automated tests have fast computers? Com

puters are so inexpensive now that there should be no excuse to

not ensuring that people have adequate computing power readily

available.

• Do you need to buy dedicated computers for running automated

tests?

• Do you use the right tools for your project?

• Are you using a good versioncontrol system?

• Are there SDKs, compilers, or development environments that

would make your team more efficient?• If the desks available to the team inhibit collaboration through hav

ing partitionsthat are too highor are not largeenoughfor pair pro

gramming, find a way to buy somenew furniture that removes these

problems.

SOME OTHER IDEAS



and software development processes. Attention was needed in all three

areas, but I believe that often process changes have the highest return, so

that iswhere I decided to focus initially.

There was a general recognition that the software development

processes were not what theyshouldbe. Somepeoplewere experimentingwith different methods, but not with any noticeable success overall. There

had been a fewattemptsto introducenewprocesses, but theyhad all failed.

I spent a bit of time lookingat eachone and talkingto the peoplewho had

been involved. The one attempt that left the biggest impressionon mewas

one where the proposed process was meticulously documented with dia

gramsand loadsof text. It wasessentially a proposal to add more bureau

cracy, with mandatory documents, sign-offs, and decision-making groups.

Nobody reallybought into it, for obvious reasons.

I wasworking withawoman fromour HRgroup, and wedecided that

wewanted to bring in outside trainingandfocusinitiallyon project manage

ment. We started by obtaining an approved budget for some project man

agement training.We spent a fewweeks talking to consultantsand various

traininggroups. Allbut one of the groupswe spoke to offeredwhat I'll call

traditionaltrainingin waterfall methods: Gantt charts,PERTcharts,project

tracking, etc.TheoneexceptionwasJimHighsmith. At the time,I had been

reading anythingI could get myhands on related to softwaredevelopment

and development processes, from the CMM to agile. The books that had

gottenme the most excited were the ExtremeProgramming books [Beck

2004] [Auer and Miller2001] [Jeffries et al2000],Jim'sAdaptiveSoftware

Development [Highsmith 1999] book, and the Scrum book [Schwaber and

Beedle2001].

Initially, I wasn't content with any of the groups we spoke to about

training. The traditional project management groups made me uncomfort

ablebecause, basedonmyprevious work experience, I hadminimal confi

dence that what theytaughtworked, except perhaps for buildingbridges.

I wasintriguedby agile development but didn't feel that anyof themethods

I had read about were complete. However, I realized that what really

resonatedwith me about agile development were its principles (see www

.agilemanifesto.org) andIwas able to connectmostof these principles tomy

previous job. We hada really good talkwith JimHighsmith, andhe agreed

to tailora course forus that provided a balanced view across whatI viewed

at the time as projectmanagement and technical practices, with practices

combined from ASD, XP, and Scrum.




Thenextstepwas followup. One thingI always worryaboutwithtrain

ingcourses isthatit'seasy forpeople to getenthused during the course, but

the moment they get back to their desks and are confronted with their

e-mail, voicemail, feature lists, bug backlog, etc. that reality can quickly

overwhelm them. Later,they'llsaytheyneverhad a chanceto try out anyofthe new ideas.I alsoworryabout situationswherea selectfewpeople attend

a course, come back enthusiastic, and then are beaten down by the pes

simism of those who didn't attend the course. Therefore, as a followup, I

decided to form an agileguidingcouncil that consistedof the more enthusi

astic course participants. The purpose of the group was to get asmanypeo

ple as possible as quickly as possible involved in leading the change and

decidingwhat our next stepswere.

The people in the guiding council are the ones who really made the

changehappen. Wemet weekly for the first year, then bi-weekly, and then

sporadically for about another year after that. The council decided to bring

in more training, so we got some more budget approved. This round of

trainingwould be a fewone-daycourses taught byJim Highsmith and also

some technical training that we thought should focus on test-first develop

ment, simple design, and refactoring.The council worked in subteams that

did things likecreatinga visionstatement and somenifty posters and sourc-

ing and organizing the training. We also spent a lot of time talking about

next steps, challenges, and successes. For the technical training, we chose

Object Mentor, which eventually also ran some other courses for us. Jim

Highsmith alsovisitedus for a dayonce amonth or soto talk tovariouspeo

ple and groups. He definitely helped lend credibility to the initiative,

because he could speakfromhis experiences at other companies.

Our first real success came about two months after the initial training.

One of the team leaders had left the course and essentially thought "Hmm,

that was interesting. I wonder if the companyis going to do anythingabout

it?" Then, through his participation in the council and with some encour

agement from his peers on the council,he decided to try it out with his team.

His teamwas enthusiasticabout the Extreme Programming books, somem

bers built out a quick projectwith test-first development, etc. The quality

was amazing aswas the speed that the results were achieved. It definitely

turned someheads and got more people interested.

Shortlyafter the firstsuccess, the technical trainingstarted payingdivi

dends. Someof the programmers had been adding unit tests to their new




code, and as these tests started failing when other programmers modified

the code, more people started noticing. The interesting thing is that our

products prior to the training did havesome automated tests.We had a unit

test framework, for example, but it was onlyused by a few individuals and

onlysporadically. One ofmyfavoritestories is that wehad what were essen

tially system-level automated tests for one of our products. But everyone

ignored the tests because they always failed, and theyhadn't been updated

inalongtime. One day, oneof theprogrammers called ameeting withallthe

teamleadersfor the product and, in a veryniceway, outlinedwhythe tests

shouldbe taken seriously! It wasa turningpoint for that project.

Therehavebeenmany interesting lessons learned to date.For example,

we didn't really understand the importance of the physical workspace to

agile development. Pair programming turned out to be impossible onmost

ofour desks because theyaretoo small fortwopeople tositsideby side. For

the curious, wedon't mandate pair programming, andmost of the people

whodopairup only do sofor an houror two at a time. Another thing that

was lacking was enough whiteboards. I'll never forget one day when I

receivedan e-mailsayingthat wewould haveto erase the whiteboards when

customers were touring the building so theywouldn't see anything confi

dential. My reply was a shortNO, andluckily Imanaged to convince people

thatproductivity was more important thancustomers seeing ourbuilding!

A few years later, I can safely say we have achieved lasting change. It

isn't uniform across all teams, because there is still a divide between our new

products developed since agile was introduced and those that existedbefore. We have rapidly developed one amazing newproduct, and we're

developing othernew products using the same methods. We have builtup

several newareas nowwithnew desks, andwehave happily converted many

of our former formal sit-down meeting rooms into agile project rooms

where thewalls are plastered withmagnetic whiteboards and corkboards.

Asa result, a numberof large tables have been relegated to thebasement to

collect dustsowecanstandup andmove around.

The challenges for our existing projects have been quite large. The

teams have adopted asmany agile practices as theycan, and they've addedmany new automated tests. But the amount of codethat isn't covered by

automated tests is quite large, andinmany cases not enough attention was

paidin thepastto issues such as code duplication anddesign, sothe teams

arestill paying offaccumulated technical debt. Still, huge strides have been

CHANGE EXAMPLES • 175



made and the teams are motivated to continually improve. I hope that even

tuallywe can close the gap with our other teams.

Summary

Implementing sustainable development in most organizations is going to

require a changein the development culture.This chapter is an introduction

to some techniques that can be used while outlining change factors and

enablers that must be thought of. Achieving a culture change is hard and

requiresmore effort and persistence than most people anticipate.Hence, it

is important to considerwhat youwant to achieve and how you are goingto

achieve it .




Appendix 1

Practice Summary

This appendix contains a summary of all the principles and practices out

linedin thisbook.Themain purpose ofthisappendix isto helpunderstand

how criticaleach of the practices described in this book is to overallsuccess

and to helpget startedon a newproject. In particular, I wouldliketo high

light the keystone practices, which are the practices that make the largest

contributiontowardachieving the overall principles or mindsetfor sustain

able development.

Keystone Species ina Natural Ecosystem

One of my favorite books in the area of natural biology is called

Krakatau: The Destruction andReassembly of an Island Ecosystem

[Thornton 1996].OnAugust 27,1883, the island of Krakatau was vir

tually obliterated by a volcanic explosion. The island was literally

reduced to a lifeless rock. This book is about Dr. Ian Thornton's visits

overmanyyears to the island and hisobservations asthe island slowly

and gradually came back to life from virtually nothing to a thriving and

complete ecosystem.

One aspect of his analysis that I find fascinating is his identifica

tionof keystone species. These are the species of plants and animals

that, iftheydisappeared, would cause theentire ecosystem toapart, or,

iftheywere absent, then theecosystem would notdevelop asitshould.

In

the caseof Krakatau, oneofthese species wasthe fig tree.It

is fascinating to read how, oncethe fig treebecame established onthe island,

the paceof changeon the island accelerated.

I thinkthenotion of a keystone species inan ecosystem carries over to

theidea ofkeystone practices forsustainable software development. As I've

177



178

outlined throughout this book, the complexityof the ecosystem that soft

ware inhabits dictatesamindset or principles that lead to successin both the

short and long term. And criticalto the desiredmindset are a set of practices

that act as keystones for the approach to software development. Hence,

while it is inevitablethat practices changeover time and that the exact set ofpractices employed isgoing to depend on the team and the circumstances of

its project, certain practices are going to change more than others, while

others form the foundation that all the other practices are built around. If it

were possible to start from a clean slate (like Krakatau), then given the

mindset required for sustainability as a starting condition, some practices

are going to be obvious startingpoints.

Keystone practices areidentified withthe icon: ^^

Working Product

1. No "Broken Windows"

2. BeUncompromising

about Defects

3. "Barely Sufficient"

Documentation

4. Continuous Integration

5. NightlyBuilds

6. Prototyping

7. Don't Neglect

Performance

APPENDIX 1

Bad code should never be tolerated because one

mess leads to another, and another, and another,

Forsustainability, defect backlogsmust be avoidedat

allcosts, startingfromthe earlydays ofa project.

Although this practice is important, I'veworkedon

teams that producedlots ofdocumentation and still

got a goodproductto market. Aslongas the focus is

on producing a working product and not the docu

ments,andas longas the majority ofeffortgoes intothe product and notthe documents, then sustainabil

itycan still result.

Teams that delay integration are simplydelaying the

timewhentheywill haveto dealwiththe inevitable

problems. Themoreoftenintegration occurs, the

faster problemsare uncovered.

Having a nightly build that works is the first level of

reassurance that the product isworking.

Prototyping is a simple and effective wayto reduce

riskand better understand a problem before having

to solve it live.

Performance is one ofthose aspects ofa product that

can't beneglected for long. You can probably get by



8. Zero Tolerance fo r

Memoryand Resource

Leaks

9. CodingStandards

and Guidelines

10. Adopt Standards

(Concentrate on Your

Value-Add)

11 . Internationalize From

DayOne

12 . Isolate Platform

Dependencies

Defect Prevention

1. Ruthless Testing

without paying attentionto performancefor a while

but at some point you're going to haveto invest in it.

This is a keystone practice only ifyou are using a pro

gramming language where programmers are respon

sible for memory and resource management.

Howbadlyyou need standards and guidelines is

going to depend on the experience level ofthe team

and the degree ofsimilarity inthe working habits of

team members. Coding standards and guidelines can

helpmanyteams haveusefuldiscussions about good

and bad code.

You can reinventthe wheeland still complete a proj

ect, but you're stillwasting effort.This is one of those

mindsets that becomes more importantwithexperience, as you learnto use what is available to you as

longas it is goodenoughand stiflethe temptationto

buildit yourself.

Many projectsdon't care about supporting multiple

languages. However, for those projects that do, there

are simplethingsyou can do inthe earlydays ofthe

project that will save wasted effort later.

If portability between platforms is a concernfor yourproject, makesure youisolatethe dependencies,preferably through platform-independent abstrac

tions so that platform issues are not exposedto the

majority ofthe code. Ifplatform-specific code is

strewn everywhere, then the code is harder to main

tain and moreerror-prone,and this makes it hardto

keepthe productina working state.

Most projects rely toomuch onmanual testingandhaving peopleperform repetitive tasks that should be

automated and performed on a regular basis bycomputers.

PRACTICE SUMMARY 179



180

2. Use Available Tools

3. Pair Programming andCode Reviews

4. Lightweight Root-

CauseAnalysis

Design Emphasis

1. DesignVision

2. Guiding Principles

3. Simple Design

4. Refactoring

5. Design Patterns

APPENDIX 1

Thereare tools available that, ifused wisely, can

boost productivity. Teams should use the tools that

are available to them to help prevent problems while

constantly being on the lookout for new tools.

This practice helps catch defects while increasing theunderstanding ofteam members inthe code they are

collectivelyworking on. However, the danger is that

these not get out of control, which happens ifpair

programming is mandated or code reviews are dis

tributed too widely.

This can be an extremelyvaluablepractice, although

itcan also get out of control. The team needs to care

fully use this one.

Having a simpleand easilyunderstood designvision

helps the team makedaily design decisions that fit

within the long-term goals ofthe project.Sustainabil

itycan becompromised ifthe team does not consider

the midto longterm intheir dailydecisions.

Guiding principles reinforce the designvisionand

helpthe team keep principles that are importantto

sustainabilityin mind.

Simplicity is vital for sustainable development

because simplicity tends to makesoftwareeasier to

modify and less brittle over time.

Continual disciplined refactoring, withan eyetoward

cleaning upcodewhile maintaining the desired

behavior, is importantfor sustainabilitybecause

it allows the team to evolve its software with

confidence.

Design patterns are a tool that everydeveloper

should be aware of and understand when to use them

and howto use them properly. Attheir best, they can

savea great dealofeffort. Attheirworst, they can

contribute to making softwareoverly complexand

brittle.



6. Frequent Rapid

DesignMeetings

7. Commitment to

Re-Architecture

8. Designfor Reuse

Continual Refinement

1. Iterative Development

2. ReleasePlanning

Collaboration about design, as frequently as possible,

is where design starts. Having a forum to talkabout

good design and good code and not just problems

helps everyone on the team.

(Especially for largedevelopmentefforts)

Acommitment to re-architecture is a keystone prac

tice on projects that involve large teams and large

amounts of code that must change over a number of

years. Onthese larger projects, beingwilling to make

difficult decisions to re-architect portions of the sys

tem can make the differencebetween sustainability

and unsustainability.

(Especially for largedevelopment efforts)

Designingfor reuse is a keystone practice on projectsthat involve large teams and large amounts of code

that change overa number ofyears. This is because

the costs ofduplicated effortare magnified the

greater the effort, not only in initial design and devel

opment time but also inongoing maintenance as

changes are madethat must be carefully made in

multipleplaces.

Iterative development is a keyunderpinning of sus

tainable development. By breaking the projectdown

intosmall manageablechunks of time, the team is

ableto reduce risk, continually monitor its progress,and makecourse correctionsas required—even ifno

changes are ever made to the scheduled features.

(For most projects)

Release planning is an important aspect of projects

that are going to take more than some number ofmonthsorare released to customers ona regular

basis. Forthese projects,having a clear ideaofwhat

is going to be ina release contributes to sustainabilitybecause it helpsteams plan architectural changes

PRACTICE SUMMARY 181



Appendix 2

Ext reme

Programming

and Sustainable

Software

Development

The purpose of this appendix is to describe the strengths and weaknessesof

Extreme Programming (XP) because XP is the most visible of the agilemethods. The messageof this appendix should be that XP is good, but that

when it comes to achieving sustainable development,XP's values and prac

tices are a good start but are not sufficient.

The Four Values of XP

1. Communication.

Clear and effective communication is required from customers (to

communicate their needs), developers (to describewhat is possible),

andmanagers (tohelp set expectations)to make an effective project.

183



2. Simplicity.

A product shouldonlybe as complex asis actually required for the

problem at hand. Overly complex products havemany subde and

unnecessary problems.

3. Feedback.Proper feedback stopsproblems as early aspossible in the develop

ment process.

4. Courage.

The team must have the courage to take risks and try new ideas

and approaches. Timely feedback willhighlightproblems as earlyas

possible.

XP's Twelve Practices

XPs twelvepractices are derived from the four values.

1. Test Driven Development.

Unit tests written before and during coding allow rapid change by

providing confidencethat errors aren't creeping into the product.

2 . Small Release s.

Small releaseshelp ensure that the developers and customers are able

to provide each other with continuous feedback.

3. Refactoring.

Refactoringallows the designto be continuallyenhanced and simpli

fied over timeby reducing redundancy and complexity.

4. Simple Design.

Reducing complexityis just as important as coding.

5. PlanningGame.

Only plan enough to get started and refinethe plan after every release.

6. Pair Programming.

All production code is written by two people collaborating at the

keyboard. This enables collective code ownership and helps to catch

codingproblems asearlyas possible.

7. Onsite Customer.

An onsite customer ensures that feedback is nearly instantaneous so

no time iswasted waiting.

• 184 • APPENDIX 2



8. System Metaphor.

A common real-world metaphor that is understood by customers,

developers, and managers isused for elementsof the project.

9. Collective Code Ownership.

Collective code ownership helps ensure that no one teammember isthe critical path or sole repositoryof knowledge on a particular sec

tion of code.

10. Continuous Integration.

Changes are placed into the shared version control systemas early

and often aspossibleso that everydeveloperisworkingfrom the lat

est version of the software. This helps to ensure that problems are

caught as early as possible.

11. Coding Conventions.

Consistent coding conventions aid communication within the team

and reduce the amount of time required to learn a new section of

code.

12 . Sustainable Pace.

Work at a consistent pace so that consistent and accurate estimates

can be made. Use a 40-hour work week t o ensur e there is also a bal

ance between home and work.

The Strengths of XPThe main strength of XP is that it is designed to appeal direcdy to soft

ware developers. The XP practices,and the emphasis on coachingand self-

organizedteamsaremost likelya reaction to softwaredevelopment cultures

that stressprojectmanagement and bureaucracy as ameans to achieve suc

cessful projects.XP is likely alsoa reactionto bad managers who tell teams

what to do, when, and how to do it.

XP hasintroduced a healthydebatewithinthe software industry. Tradi

tional engineering methods place decision-making power primarily in thehands of managers through bureaucracy, tracking plans, unnecessary mile

stones, and documentation as a tracked deliverable. XP counters bureau

cracy by placing the decision-making power in the hands of the software

development team and the software's users. Instead, by emphasizing value

to customers through working software not documentation, practices that

EXTREME PROGRAMMING AND SUSTA INABLE SOFTWARE DEVELOPMENT • 185



emphasize technical excellence, collaboration between developers, and

direct customer contact, XPissteering the industry more toward producing

high-quality andhigh-value results forcustomers, which isexacdywherethe

emphasis should be.

XP is Extreme

One of the appeals ofXP isthat it isextreme, especially comparedto tradi

tional software development practices. Pair programming, test-driven

development, refactoring, the planninggame, etc. are all completely out of

the normal realm of experiencefor traditional softwareteams. As a result,

XP has received a lot of attentionand has gottenpeople to think and ques

tion their development methods. The resulting debates havebeen good forthe software industry.

Technical Excellence

XPs focus on technical excellence is particularly important. Although it's

hard to identify core practices, a case could be made that they are simple

design, refactoring, and test-driven development. Each of these practices is

simple yet powerful and helps to emphasize achieving results as quickly aspossible so that users can provide feedback and changesmade in response

to the user comments. Likewise, continuous integration helps teams focus

on keeping their softwareintegrated and working through unit tests and the

goal of frequent and smallreleases.

Collaboration

There is no doubt that XP gets teamsworking together, intimately. Tightcollaboration within teams is a vital aspect of success, and by emphasizing

pair programming, collective ownership, self-organizing teams, and having

an on-site customer focuses teams on collaborating. The result is exhilarat

ing and fun for those involved, and the results are often phenomenal.

Greater collaborationhelps to ensure that commonproblems such asdupli

cated code and effort, unmaintainable code, and independendy developed

186 • APPENDIX 2



modules unable to integrate with each other. The integration problem is

particularly acute; I have seen projects where two developers working in the

same cubicle produce modules that can't be integrated, even when integra

tion was a key design issue!

XP's collaborative practices help to avoid the hero syndrome. We allknow the heroes; they're the ones who work long hours to get the critical

features into the software. At many companies, they're rewarded by the

company for doing so, and there is a general recognition that the company

would be in deep trouble if they left. Myexperiencewith heroes (and I will

confess that in my first job in industry I wasone of them) is that they either

leave behind a swath of destruction in the code, or they definitely leave

behind code that can't be maintained by anyone else because nobody else

knows the code the way they do. While it is true that the best programmers

do produce more [Demarco and Lister 1999], often considerably more,

there is still plenty of room through collaborationto keep the output of the

best programmers high while making the entire team into the heroes

through high-valueand high-qualityresults.

Customer Collaboration

The most important aspect of XP's collaborative practices is the emphasis

on collaboration with customers . Because customers are involved in the

planning of iterations and prioritization, and because there are regular

releases of software to customers, the team is able to get timely feedback

that helps to ensure teammembersare always doingwhat ismost important

to the customer. Thus, XP maximizes value to the customer while minimiz

ing the amount of effort required to create that value through decreasing

effort wasted on nonessential work.

The Weaknesses of XP

Manyof theweaknesses ofXP stemfromthe factthat it isoftenpresented in

a semi-religious, almostcult-like way. This is the view that in order to prac

ticeXP, you need to utilize all the practices and stickwith them to the letter,

with no alternative if you can't use them or want to add to them. There are

EXTREME PROGRAMMING AND SUSTAINABLE SOFTWARE DEVELOPMENT • 187



some in the XP communitywho recognize this problem, so hopefully this

view will moderate over time.

XP is also subject to problems of interpretation. People who read the

XP books or speak to XP teams can easily misinterpret the practices,

overzealously apply them, and then decide that sinceone or two of the practices don't work, therefore XP doesn't, when in fact the XP mindset does

work, and the XP practices around test-first development, refactoring, and

simple design in particular are absolutelybrilliant. People who dismissXP

as a whole are missing out on somevaluable practices. Unfortunately, you

don't have to l ook too h ard o n the In terne t to find articles that reflect the

failed XP experience.

XP Targets Software Developers, and Nobody Else

XP is too narrowly targeted at software developers. Successful software

products are developedby teams,and this is stating the obvious,but teams

include more than just softwaredevelopers.Documentation, business peo

ple, Quality Assurance (QA), and usability are glaring omissions. Often,

theseother functions usepractices that are incompatiblewithXP and then

strugglewith experimentalpractices, or because there is somuch emphasis

on the software these other functions are minimized -[md] to the possible

peril of the project.

The role of the manager is particularly abused in XP. I know quite a

fewmanagerswho have read Extreme Programming Explained [Beck2004]

and been turned offon the whole thing becausemanagersare cast in the role

of the bad guy. While it can't be disputed that somemanagers are the bad

guys, I believe that most managers want success and are a part of that

success, not a hindrance to it. Perhaps the labelmanager is the problem.XP

disdains bureaucrats (who tell what to do, when, and how) and while it

is true that bureaucrats are not desired, leaders who know how to set up

a collaborative, empowering environment are definitely required. Some

people call this role the coach, and while that does capture the essence of

this style of leadership, I believe that coaching is justone attributeof a suc

cessful leader.

188 • APPENDIX 2



Project Management

XP describesa project planningparadigmthat is usefulfor a small teamof

software developers. Given XP's focus on software developers, this

shouldn't be a surprise,nor shouldit be viewed asparticularly bad becausethe practicesare usefulfor some teams. However, the lackof a more rigor

ous attention on project management has led agilepractitioners to propose

hybrid agile techniques such as Scrum with XP [Schwaber and Beedle

2001] or ASD with XP [Highsmith 1999] and why Jim Highsmith, for

example, wrote his excellent book Agile Project Management [Highsmith

2004a].

Scalability

XPwas developed for the C3project for Chrysler. It featured a smallproject

team with a single customer. The XP practices alone are hard to scale to

larger teams because (amongmany things) they do not offer any solutions to

collaboration between teams, or asmentioned above, do not address any of

the issuesfaced by non-software developers in the team.

Another aspect of scalability is a result of the fact that XP is primarily

targeted at software developers. Because XP typically starts in software

teams, change is attempted in a purely bottom-up way that can be trans

ferred to other software teams but not through an entire organizationwith

out extra practices.

Pair Programming

Pair programming is presented as something that software developers

absolutelymust do. Yet,pair programming ishard:Either you are typing or

you are watching someone else type, often poorly or slowly. The reader of

the XP books is left with the impression that developers should pair pro

gram allday,everyday. While pair programming can be veryuseful and pro

ductive, its use should be limited to what the people on the team are

comfortable with. It certainly shouldn't be mandated.




What gets lost in the pair programmingdebate is that the intent of pair

programming is to encourage collaboration between developers and to

ensure that problems are caught as the code is typed. Essentially, by forcing

pair programming,youforcedevelopers to collaboratewith each other with

the goal of building a team that collaboratesby default, and in person notthrough e-mail or some other contrived means. There are other ways to

achievethis samelevelof collaboration, and it's much healthier to encourage

a range of collaborativepractices than to mandate a singleone.

The 40-Hour Week

The other practice that is often overzealously applied is the 40-hour work

week. I have read articles where XP teams were criticized because, insteadof pushing to get a product out the door, theyhave insistedon workingno

overtime,and hence lack anysenseof urgency. I alsohave spoken to people

who claim to be usingXPbut work80-hourweeks on a regularbasis, which

also isn't good.

The purpose of the 40-hour workweek is to get teams to work hours

that allow themto staysaneandhavea lifeoutsideworkoverthe longterm,

measured in manymonths to years. This sense of balance is important to

maintain a sustainablepace of development. The reality is that extra hours

are going to be required on occasion, so the emphasis should be on the

work/lifebalance overthemidto longtermwitha rationalaverage number

ofhoursperweekworked. Reasonable working hoursand avoiding burnout

are a result of the focus on sustainable development, not the focus. Thus,

sustainable development requires principles and practices that target sus

tainabilityand increasing capability overtime.

The On-Site Customer

On-site customers are great if you canget them, or if you can work on your

customer site where your customer is readily available. But what if your

product has thousands of customers? Tens of thousands? Hundreds of

thousands? What if your product sellsto customers inmany diverse vertical

market segments,where eachmarket has unique requirements? The bottom

190 • APPENDIX 2



line is that in many cases it is impossible to have a true on-site customer.

Manyteamswho tryXP failbecausetheyare unable to get over this hurdle,

becauseXP and all the agiledevelopment techniqueswork best when there

is a timely feedback loop between the customer and development team.

Unfortunately, this is a rich enough topic for another book, but there arealternativesthat are different from havingan on-site customer but equally as

useful.Just soyou aren't left hanging,here are a fewideas:

• Learn about the field of usability or, even better, hire a usability

expert or team. Usability professionals have a rich and diverse num

ber ofwaysto test softwarewith users and get the required feedback

that agile teams need.

• Use the Internet to ship your software on a regular basis (daily or

weekly) and get feedback from your customers so they can provide

you with instant feedback and you can solve their problems in an

optimal time. More on this in Chapter 4 onWorkingSoftware.

• Have a customer representative on your team. This person is a user

of the product who spends a great dealof his or her time talking to

andworkingwith customerson real problems. These representatives

funnel the feedback from customers into the team and give team

members the feedback they need when theyneed it.

Self-Organization and Leadership

Manyteamsget carriedaway withself-organization and forget that there are

often situationswherepeoplewith specific roles are required.For example,

it almostalways makes sense to havea project leader to dealwith the daily

coordination and management of the project. Good project leaders (or

coaches) are able to nudge teamsforward, while helping them to make the

right decisions and know the right decision when theysee it. Pure democ

racydoesn't always work, and sometimes somebodyneeds to step forward

and make a tough decisionwhere there is no clear consensus,simplyin the

interest of keeping the project moving forward.

A related topic is that there is a general trend to shy awayfrom the term

leadership in XP.As explained in Chapter 8, leadership is a critical success

factor in successful projects and is required at all levelsof an organization.




"No Discipline"

Manypeoplefeel that XP teamsarenot disciplined. Actually, XP requiresa

great dealof discipline. Themisunderstanding comes in through the defini

tion of discipline. Typically, those who think XP teams are not disciplinedthink of discipline asprocess compliant, where the processdictatesthe pro

duction of documents in a carefully staged order of analysis then coding.

These people use the absence of the documents created in traditional devel

opment (e.g., requirements documents, designdocuments) as evidence that

agile teamsare undisciplined. There arealsofeelings ofdiscomfortthat agile

teams do not put any effort into understanding user requirementsor do not

do design,when in fact theydo. The differenceisthat with agileapproaches,

the emphasis is on the software, not documents, and making a decision as

late aspossible.

Discipline is more than just documents or rules; it is a function of the

people in the team. Agiledevelopment provides a simple model that can be

used to give a project a regular heartbeat with a constant understanding of

progress and learning. It is up to the people in the teams to take advantage

of the information they have. This is really no different than in traditional

development:Disciplineis up to the peopleon the teamand not an attribute

of the methodology.

Other Agile Development Methods

Manypeople have only heard of XP or think that agile development isXP.

This is a pity, because there are other agile methods that have a lot to offer.

In an environment where a team emphasizes agility and continual improve

ment in capabilitythrough regular retrospectives and experimentationwith

methods, the other agilemethods are an ideal source of ideas and practices.

Myfavorites are:

• Adaptive Software Development [Highsmith 1999] and Agile Pro

jectManagement [Highsmith 2004a]

• LeanDevelopment [Poppendieck and Poppendieck2003].Leandevel

opment is inspired by leanmanufacturing [WomackandJones 1996].

• Feature-Driven Development [Palmer and Felsing 2002].




Crystal [Cockburn 2001] [Cockburn2004].

SCRUM [Schwaber and Beedle 2001].

Dynamic Systems DevelopmentMethod (DSDM) [Stapleton2003].

DSDM introduces a healthybusinessfocus to agiletechniques.

AgileDevelopmentEcosystems [Highsmith 2002] contains a good

summaryof agiledevelopment and agilemethods.

Summary

Don't be deceived by the fact that there is more text in this appendix dedi

cated to the weaknesses ofXP than to its strengths. The strengths more than

balance out the weaknesses, and aswith any softwaredevelopmentmethod,

it is easy to poke holes in the implementation. The practices of XP are an

excellent placeto start,but XPshouldnot beviewed asa completemethod

ology.

Themost criticalelementthat ismissingwithXP isthe notion ofcontin

uallearningand improvement in themethodsusedby the team. Everyteam

is different because it's made up of different people and faces a different

ecosystem. Hence, teamsneed to place an emphasison continual improve

ment of their methods through regular retrospectives and methodology

changes that are consistent with the principles (mindset) of sustainable

development. Thisgives themamorehealthybalancebetweensoftwareand

non-software and shouldencourage them to continually look for newideas.

If yourteamisusing XP, I highly recommend that youtryoutnewpractices

asrequiredwhilestaying true to the values. XP unfortunatelygetstoo much

attentionin the software community. Thisisboth goodand bad:goodin the

sensethat XP has causeda largenumber ofdevelopersto question the tradi

tional methods of softwaredevelopment, and bad in the sense that there is

more to agiledevelopment than XP.There are quite a fewother agilemeth

ods, and each offers different agile practices or viewpoints that should be

consulted and incorporated as a team continuallylooks for new sources of

ideas and inspiration.

194 • APPENDIX 2



Appendix 3

Sustainable Software

Development

and the CMM

The misconception among many commercial software developers is that

process discipline in software development (such astheCMM) is incompat

iblewith fast-moving developmentprocesses such asXPA similar miscon

ception among many process-oriented people—CMM or otherwise—is

that developing software quickly is tantamount tochaos. If these twoviews

persist, they willkeep excellent development teamsfrom realizing the ben

efits ofstructuredprocess improvement, andlikewise keep larger organizationsfrom looking atalternative developmentmethods. [Glazer 2001]

TheCapabilityMaturityModel (CMM) was developed bythe Software Engi

neeringInstitute as awayto describethematurity of a software organization's

development practices. Peoplewhopractice agile development view theCMM

as being the opposite of agile: ponderous, cumbersome, and antiquated.

Process-oriented people look upon agile methodsasbeingundisciplined and

chaotic. However, from the standpoint of sustainable development, there is

actuallya great deal to learn from and draw on from both agiledevelopmentand theCMM.Aspointedout in the quote above, peoplewho dismiss agile are

missing thebenefitsofsimpleand powerfuldevelopmentmethods,and people

who dismiss the CMM are limiting their ability to implementorganizational

processimprovements. In thisAppendix, I willbriefly describethe CMMand

then showhowthe CMMand agiledevelopmentcomplementeachother.

195



The Capability Maturity

Model (CMM)

The Software CMM [Paulk et al 1993] is a five-level model that describes

good engineering and managementpractices and prescribes improvement

priorities for softwareorganizations [Paulk 2001]. The intent of the CMM

is to provide areas of focus (calledKeyProcess Areas or KPAs)for software

organizations that have proven in many software organizations to lead to

excellence in software development. The Software Engineering Institute

also provides standard guidelines for assessment, so that a third-party

assessorcan evaluatean organization and produce a report that outlines the

maturity of the organization's processes. The combination of the CMM

model and objective third-party assessments means that an organization

can at any time have an understanding of its current capabilities plus the

types of activities it must focus on to move to the next level. Hence, the

CMMis intended to provide a generalframework for planning and meas

uringthe capability of a software organization. The CMM doesnot specify

or mandate actualdevelopment processes, it merely specifies what to do in

general terms.

The five maturity levels oftheCMMand the KPAs associatedwith each

level ofmaturity are shown in Table A3-1. CMM level 1 organizations are

characterized by chaotic processes that require heroism to complete soft

ware projects.At the other extreme,CMMlevel5 organizations have com

petence in all the KPAs, established processes, and are continually

improving their processes.

Because the CMMdoesnot specify the actualprocessesthat should be

used in software development, CMM 5 maturity does not necessarily

equate with software quality or sustainability. It usually does, but that is

more the resultof themethodsput in placeby the peoplewho designedthe

processes and their hard work. Also, in my experience at any large com

pany, there is a great deal of variability in the quality of work between

teams. Hence, CMM 5 is not a guarantee that the traditional code-then-fix

method not used. I have worked with CMM 5 companies as subcontrac

tors, and myexperienceis that while the majorityof them produced excel

lent work, there were somegroups that franklyshocked mewith the poor

quality of the work they produced.

196 • APPENDIX 3



Table A3-1.The Five Maturity Levelsof the CMM andthe KPAs AssociatedwithEach Level of Maturity

Level Focus

5 Continual process

Optimizing improvement

KeyProcess Areas

Defect Prevention

Technology ChangeManagementProcess ChangeManagement

4 Productandprocess Quantitative Process Management

Managed quality SoftwareQuality Management

3 Engineering processes Organization Process Focus

Defined and organizational Organization Process Definition

support Training Program

Integrated Software Management

Software Product Engineering

IntergroupCoordinationPeer Reviews

2 Projectmanagement

Repeatable processes

Requirements Management

Software Project Planning

Software ProjectTracking &Oversight

Software Subcontract Management

SoftwareQuality Assurance

SoftwareConfiguration Management

1

Initial

Competentpeopleandheroics

Agile Development and the CMM

Agiledevelopment and the CMMcan co-exist. Sincethe CMMspecifies the

what (in terms of the KPAs) and ismethodologyneutral, agiledevelopment

methods can be used as the method (the how) to achieve much of CMM

process maturity. Also, if you've gotten this far in this book, it should be

obvious that the claimthat agile development doesnot require discipline is

verymuch false. Agiledevelopmentis hardwork and requiresa great dealof

discipline.

SUSTA INABLE SOFTWARE DEVELOPMENT AND THE CMM 197



The practices of extreme programming havebeen contrastedwith the

CMM KPAs in various articles [Glazer 2001] [Paulk 2001] [Bandaru2004].

If you are interested, I highly recommend reading them. The conclusion

reachedby all these sources is that the extremeprogramming practices do

offer a solution to most of the KPAs. Of the eighteenKPAs, six are largelyaddressed in XP, and five are partially addressed. In this book, I have con

sciously added practices such as configuration management, professional

development, and businessknowledgebecause they are implied in the agile

literature, but required for sustainable development. As a bonus, they hap

pen to also be called out as KPAs. Likewise, the principles of continual

refinement and defect prevention turn out to be level5 KPAs, and are thus

of extremelyhigh value to softwareorganizations.

What I find particularly intriguing with agile development and the

CMM is the role of the agilemindset in the implementation of developmentprocesses. As I have personally discovered, an organization can only be as

agileasits least agilepart.Furthermore, sincesoftwareteamsmust by their

nature be cross-functional, if one functional part of the team is not agile,

then the entire teamwillfeelthe effects. Here are someexamplesthat I have

personally seen:

• Product managementand/or engineeringmanagementthat wants to

completely specify all the features that make up the next software

releasebefore anycodeiswritten. Sometimes it is reallyhard for people to accept that they can't predict what the next release will con

tain, even if that release date is a year out or more. If these feature

lists are followed religiously, there isno room for agility.

• A marketing department that needs a large amount of lead time to

write the marketing material for the next release. This means it will

be hard to add features that are important to users late in a release

cycle,and this limitsagility.

• A documentation group that is used to working on documentation

after allthe features are done. If it takes a long time to write the doc

umentation, then documentation is going to continually lag develop

ment and slow it down.

• A software testing team that is heavily relied upon by developers to

find defects (i.e., a defect detection culture). Invariably, the testing

group will become a botdeneck and will also be frustrated by the

poor quality of the product given to them to test.

198 • APPENDIX 3



• Bizarremanufacturing policies that dictate,for example, that physi

calCDsmustbe produced for all releases, that aminimum inventory

must be maintained, and that everynew releasemust involve new

CDs for example, electronic distribution and the Internet are not uti

lizedor are thought of assecondary to the CDs.Agile productsmustshipregularly; if eachshipmentcostsmoney, itmaybe easierto ques

tion the frequent releases than to question the silly practices that

cause the high costs . . .

Each of above examples illustrates the problem with process improve

ment in an organization. If you choose to become an agileorganization, then

agilecan start in softwareteams,but it had better spread, or overallthe pos

itive effects of agiledevelopment willnot be asnoticeable as they should be.

This iswhere the principlesof agiledevelopmentcomein because theymust

be applied across the entire organization. The practices will largely differ

depending on the function (i.e., software developers will have different

practices than documentation people or marketingpeople), but themindset

must be similar to maximizeoverallagility.

The topic of an agileenterprise isone that could easilyoccupy a book of

its own. As an exercise to interested readers, I would recommend that you

read through the principles in the Agile Manifesto (www.agilemanifesto

.org) and reflect on the role of each in all the functional areas of your soft

ware teams. In terms of the principles of sustainabilityoutlined in this book

and some of the select practices, I believethat there is a great deal of appli

cability to CMMmaturity:

• Continual refinement in how the teamworks through frequent retro

spectives and tuning of processes is a level5 KPA.

• Having a working product every day, even if it is not functionally

complete, makes Software Quality Management (a level 4 KPA)

attainable. The goals of SQM are to plan the SQM goals (in this case:

it works), make them measurable, and to quantify and manage the

goals.

• Defect prevention is a level5 KPA.

• The notion of minimaldocumentation appliesto many of the KPAs.

However, most notable is how it applies to Organization Process

Definition (a level 3 KPA). There's nothing wrong with creating a

diagram and short description of your agile process as is done in

SUSTAINABLE SOFTWARE DEVELOPMENT AND THE CMM



[Highsmith 2004a] and accompanying that with a list of best prac

tices and lessons learned. Such a documentwould be useful in a large

organization, especially ifit'scontinually updatedandit'sonly aslong

as it needs to be. A 50-page document can often convey the same

information asa500-page document. Besides, in the agile way, what'swrongwith leaving some things open to experimentation as long as

teams are usingthe agile principles?

Summary

TheCMM and agile development aren't opposites. Thereis value to under

standing the CMM and applying it to an organization's process improvement initiatives, even if the organization wishes to be agile. The CMM

outlinesbest practices that are not covered by agile development, such as

the areas ofmanagement andprocess management, and thesearevitalto the

overall organization. Agile development provides principles, or mindset,

plus additionalpractices that areequally important to avoida bureaucratic

approach.Hence, the CMMand agile can coexist.

200 • APPENDIX 3



[Collins and Porras 2002] Collins, Jim, and Porras, Jerry I., Builtto Last:

SuccessfulHabitsofVisionary Companies, HarperBusiness, 2002.

There are elementsof thisbook that I likea great deal, and others that I

find dated today. Myfeelings about thisbook are perhaps jaded by an

experience at a former company where a great deal of effort was putinto formulating the companyvision and mission, but it didn't do one

bit of good because ofnumerous bad decisions...

[Economist 2004]Make itSimple:A Survey of Information Technology, The

Economist, October 2004.

If you can find a copyof this article, it iswellworth reading. The article

discusseswhat the author feels to be the largest problem with technol

ogy today: the need for simplicity. It will be particularly interesting to

read this article in a few years,since the article mentions some up-and-comingtechnologies,manyofwhich are likelyto failor be unable to live

up to the hype.

[Norman 1999] Norman, Donald A., The Invisible Computer, MIT Press,

1999.

This is oneofmyfavorite books on usabilitybecause itmakesyou think

about simplicity. It describes the classic Crossing the Chasm [Moore

2002] problem in terms of usability and places usability on an equal

footingin termsof importancewith technologyand marketing.

[Repenning and Sterman 2001] Repenning, Nelson P. and Sterman, John

D., Nobody Ever Gets Credit for Fixing Problems That NeverHappened:

Creating and Sustaining Process Improvement, California Management

Review,Vol. 43 No. 4, Summer2001.

I was thrilled when I first found this paper because even though it is a

study of chemicalmanufacturing plants and has nothing to dowith soft

ware development, it speaks to many of the issues faced by today's soft

ware organizations.

202 • APPENDIX 4



Chapter 3: The Principles

of Sustainable Development

[Boehmand Turner 2004]

Boehm,Barry and

Turner,Richard,

Balancing

Agility andDiscipline, AddisonWesley, 2004.

In my opinion, this book should be titled Balancing Agility andCere

mony. This is because the mistaken impression that somepeople have

drawnfromthe tide is that agility is theopposite ofdiscipline. And that

ismostemphatically incorrect.Onceyougetpast that point, this isactu

ally an excellent book that describes a model for understanding and

contrasting various approaches to softwaredevelopment and the chal

lenges facedby differentsoftware projects.

[Buckingham and Coffman 1999] Buckingham, Marcusand Coffman, Curt,

First, Break all theRules, Simon & Schuster, 1999.

This is an excellent book that focuses on the difference between talents

and skills. Skills can be learned, and, while some talents can be

improved, they are largelyborn not made. The book contains a number

of interesting chapters on topics such as helping people identify their

talents and how to put people in positions where they can best utilize

their talents.

[DemarcoandLister 1999] Demarco,Tomand Lister, Timothy, Peopleware:

Productive Projects and Teams, 2ndEdition, DorsetHousePublishing Com

pany, 1999.

This is a softwaredevelopmentclassic and ishighly recommended.

[Lundin et al 2000] Lundin, Stephen C, Paul, Harry, and Christensen,

John,Fish!ARemarkable Way toImproveMorale andBoostResults, Hyperion, 2000.

Thisbook takesonlya fewminutes to readand it istimewellspent.The

main theme of the book is that you can't always choosewhat youwork

on, but you can choosehow you do it. The differenceis critical, and the

more people who understand this message, the better the chance that

themodernworkplace couldbe transformed fromdrudgery to fun andinspiring.

RECOMMENDED READING • 203



[Lynn and Reilly 2002] Lynn, Gary S.,Reilly, Richard P.,Blockbusters: The

FiveKeys toDeveloping GREATNewProducts, HarperBusiness,2002.

Although this book is not about sustainabledevelopment or even soft

ware development, it is a goodbook about innovative product develop

ment. There are quite a fewparallelsbetween the methods described inthis book and agile development methods: particularly about the need

for iterative development (which the authors call lickety-stick innova

tion), collaboration, and physical spaces that enhancecollaboration and

communication.

Chapter 4: Working Product

[Amblerand Jeffries 2002] Ambler, ScottW., Jeffries, Ron,AgileModeling

Wiley, 2002.

This is the best book I've seen on the topic of lightweight documenta

tion. It is packedwith ideason how to capture requirements,specifica

tions, design, etc.

[Beck 2004] Beck, Kent, Extreme Programming Explained, 2nd Edition,

Addison-Wesley, 2004.

Kent Beck does an excellent job of introducing the motivation behindcontinuous integration, nighdy builds, and prototypes. The Extreme

Programmingliterature is in general filled with many useful ideas and

practices.

[Hunt and Thomas 2000] Hunt, Andrew and Thomas, David, The Prag

matic Programmer, AddisonWesley, 2000.

This is an excellent book for software developers, and I highly recom

mend it. Someof the practices described in this book are derived from

the pragmatic programmer.

204 • APPENDIX 4



Chapter 5: Defect Prevention

[Beck 2004] Beck, Kent, Extreme Programming Explained, 2nd Edition,

Addison-Wesley, 2004.

Extreme programming is the methodology that introduced test-drivendevelopment and pair programming. These practices are very well

described inKentBeck's bookthat introduced Extreme Programming.

Test-Driven Development

[Astels 2003] Astels, David, Test Driven Development: A Practical Guide,

PrenticeHall PTR, 2003.

I think this is one of the best books currendy available on test-drivendevelopment. The author provides an excellent overview of different

aspects of test-drivendevelopment.I especially like the chapters on the

testing of user interfaces and mock objects.

[Beck2002a] Beck,Kent, Test Driven Development: By Example, Addison-

WesleyProfessional, 2002.

[Husted and Massol 2003] Husted, Ted and Massol, Vincent, JUnit in

Action, Manning Publications, 2003.

Automated Testing (General)

www.testing.com : I highlyrecommend this web site by BrianMarick. The

pageon Classic TestingMistakes in particularishighly recommended.

[Marick 1995] Marick, Brian,The Craft ofSoftware Testing Prentice Hall,

1995.

Usability Testing

[DumasandRedish 1999] Dumas,JosephS.andRedish, JaniceC,A Practi

calGuide to Usability Testing Intellect,Ltd. (UK), 1999.

[Rubin 1994] Rubin, Jeffrey, Handbook of Usability Testing: How to Plan,

Design, andConduct Effective Tests, Wiley, 1994.




[Fowler 2002] Fowler,Martin, Patterns ofEnterprise Architecture, Addison-

Wesley,2002.

Another usefulbook of design patterns forusewiththeweb, databases,

se rve r s, e t c.

Design Patterns, Agile Development, and Simple Design

[Evans 2003]Evans,Eric,Domain-Driven Design, Addison-Wesley, 2003.

This is one of the best software design books in my opinion. It fits

extremely well with agile development and an emphasis on simple

design within a design vision as described in the Emphasis on Design

chapter, though the terminologyis different.

[Kerievsky 2004] Kerievsky, Joshua, Refactoring to Patterns, Addison-


Anexcellentbook that dealswith the hard-core codingproblems ofbal

ancing the up-front use of designpatterns versushaving them emerge

over time through refactoring.

[Martin 2002] Martin, Robert C, Agile Software Development, Principles,

Patterns, and Practices, Prentice Hall, 2002.

This book is unfortunately misnamed. The value of this book is inunderstanding the drawbacks of various designpatterns and in general

how to employdesignpatterns in an agile development context.Highly

recommended.

Refactoring

[Fowler 1999] Fowler,Martin,Refactoring: Improving theDesign ofExisting

Code, Addison-Wesley, 1999.

The bible of refactoring.Enough said!




Other

[AmblerandJeffries 2002] Ambler, ScottW., Jeffries, Ron,Agile Modeling

Wiley,2002.

Thisbook containsa rich descriptionof somelightweightideasfor documenting designs.

[Armitage 2004] Armitage, John, Are Agile Methods Good for Design?,

IEEE Transactions,Jan/Feb 2004.

This is an excellent paper on user interface design for projects that

employ agiledevelopment.

Chapter 7: Continual Refinement

[Cockburn 2004] Cockburn, Alistair, Crystal Clear: A Human-Powered

MethodologyforSmall Teams, Addison-Wesley Professional,2004.

AlistairCockburn providesuseful and unique insights into the require

ments of softwaredevelopment done by small teams.

[Cohn 2004] Cohn, Mike,User Stories Applied: ForAgileSoftware Develop

ment,Addison-WesleyProfessional,2004.

I highly recommend this book. It is packed with practical insights into

how to gather requirements and write effective user stories for agile

development.

[Cohn 2005] Cohn, Mike,A Regular Heartbeat: TheBenefits ofFixedItera

tion Lengths, 27 January 2005, Cutter ConsortiumAgile Project Manage

ment e-MailAdvisor,e-mail: [email protected].

A short but useful article that describes the benefit of a regular heart

beat for projects.

[Crane 2002] Crane, Thomas G., TheHeartofCoachingFTA Press, 2002.

This isone ofmyfavoritebooks on coaching.

208 • APPENDIX 4



[Highsmith 2004a] Highsmith, Jim,Agile ProjectManagement, 2004.

This is the definitive workon agile project management and an excel

lent source bookof ideas and insight into agile methods.

[Kerth 2001] Kerth, Norm, Project Retrospectives: A Handbook for TeamReviews, Dorset House, 2001.

Although this book describes multi-day retrospectives that are usually

held at the end of a project, there aremanyexcellent ideas in the book

that canbe easily adaptedfor shorter, moreagile-appropriate retrospec

tives. This is the referenceyoumust haveif you are interested in effec

tive retrospectives.

[Larman 2003] Larman, Craig,Agile and Iterative Development: A Man

ager's Guide, Addison-WesleyProfessional,2003.

The valueof this book is that it iswritten formanagersand executives to

help them understand, support, and apply agile development. This is

the only book that I'm aware of that directly discusses the business

valueof agiledevelopment, sincebusinessvalueis an important consid

eration for the target audience.

[Poppendieck and Poppendieck 2003] Poppendieck, Mary and Poppen

dieck, Tom, Lean Software Development, Addison-Wesley Professional,

2003.

Lean development takes a unique approach to agile development:

identifying the simplest set of practices possible that lead to effective

software development. If you have heard of lean manufacturing, then

this book will be of great interest because the thinking and rationale

are similar.

[Schwaber and Beedle2001] Schwaber, Ken and Beedle,Mike,AgileSoft

ware Development withSCRUM, Prentice Hall, 2001.

SCRUM gets a lot of attention because it is a simple agile project man

agement technique.




Chapter 8: Making Culture

Change Happen

[Beck2002b] Beck,Kent,XPandCulture Change, Cutter ITJournal, Cutter

Consortium, Oct 2002.

Kent Beckdiscussesculture changerequired for Extreme Programming

from the standpoint of the various stakeholders, especially developers

and managers.Youshould be able to get a copyof this publication from

www . c u t t e r . c om .

[Cockburn 2001] Cockburn, Alistair, Agile Software Development, Addi

son-Wesley, 2001.

Manyfactors with the physicalwork environment and the impact it canhaveon agiledevelopment are described in this book.

[Crane 2002] Crane, ThomasG., TheHeart ofCoaching FTAPress, 2002.

This is one of my favorite books on coaching. There is a very useful

chapter on culture changeand the role that individualand team coach

ingplaysin achieving culture change.

[Kotter 1996] Kotter, John P., Leading Change, Harvard Business School

Press, 1996.

This book describes the problems that are commonly faced when

attempting culture change in an organization. The value of the book is

in the strategythat is outlined for achievingchange.

[Lindval et al2004] MikaelLindval,Dirk Muthig, Aldo Dagnino, Christina

Wallin, Michael Stupperich, David Kiefer, John May, and Tuomo Kahko-

nen, Agile Software Development in Large Organizations, IEEE Computer,

December 2004.

The authors of this paper outline a number of good points related to

attempting to introduce agiledevelopment into large organizations such

asMotorola, Nokia, and DaimlerChrysler.

210 • APPENDIX 4



Conclusion

I believe today (because I am always learningand adapting) that the princi

ples oudined in this book are keyingredients to attainingsustainability. At

the veryleast, teamswho are willing to take on the challengeof sustainabil

itywillhavean excellentheadstart on their journeybyadoptingthemindset

outlined and carefullyconsideringand adopting eachof the practiceswith a

goal toward continual improvement.

I havetried to developand follow a fewkeythemesin thisbook:

• Software development is a complex undertaking. The software

industryneeds to mature, but in order to do so, we need to match the

development methods we use with the complexity of the software

technologyecosystem,which forcesthe need for continual change.

• The methods that are still predominandy taught in school and

employed in industry are code-then-fix, or the waterfall method.

These methods are inadequate to the task because they are change-

adverse and are either too lax {code-then-fix) or impose too much cer

emony and process (waterfall).

• The principles and practices employedin this book are directly from

agile development. Agile methods are intended to provide just

enough ceremonyand process to allowteamsto get their job done in

a lightweight manner. Unfortunately, some people associate agile

development with being not disciplinedwhen in fact agile develop

ment requires a great deal of discipline, as should be obvious when

reading this and other agilebooks.

• Agile development requires specific project management practices

and an emphasis on technical excellence and collaboration. It is too

easy to apply agile project management practices such as iterative

development and forget that the state of the softwarebeingworked

• 211 •



on (e.g., maintainability, design and architecture, etc.) and whether

the development team includes users (and/or user representatives)

and business peoplethat collaborate on the projectalso greatly influ

ences agility.

Although mybackground is one where my focus is on producing

shrink-wrapped software products, a product could be any kind of

software: awebsite,IT system such asa databasebackend, firmware,

consulting project, etc.

Too many projects can't cope with the complexity of the software

development undertaking. The result is unsustainable development,

where teams are largely only able to respond to changes in their

ecosystem. They have littlecontrol andarealmost always in catch-up

mode. Themost effort goes toward adding features and fixing bugs.

These projects respond to complexity with complex solutions that

are unreliable (i.e., buggy), britde (break easily), and unable to sup

port future change.

In sustainable development, teams are able to be proactive about

changes intheirecosystem. Theirability tobe proactive isenabled by

their attention to doing theworkthat isof the highest value to cus

tomerswithhighquality and reliability and an eyetoward continual

improvementdespite increasing complexity.

Sustainable development requires acknowledging the need for

change. This means adopting a different mindset and being uncom

promising in fourkey areas: aworking product every day, continual

refinement, defect prevention over detection, and an emphasis on

lightweight but continual design. Sustainable development results,

whereit ispossible to be proactive about change.

The first principle of sustainable development is having a working

product every day. Aworking product, even if it is not functionally

complete, gives teams flexibility and allows them to be much more

responsive to changes in their ecosystem (user requirements, com

petitivethreats, sales opportunities, etc.) and proactivewhen oppor

tunities arise. This flexibility is crucialin sustainability.

The second principle of sustainable development isdefect prevention.

Defectprevention isa change inmindsetfor virtually allteams, where

insteadof usingthe code-then-fixmentality to development, the team

does everything in its power to prevent defects reaching customers.

212 • CONCLUSION



Central to defect prevention is the practice of ruthless testing, where

computers are relied upon to do the repetitive and boring testing

tasks that peopledo today. This allows people to concentrate on the

more creative aspects of testing, such as testing organizations using

the product in realistic ways asthe productis developed.

• The third principle of sustainable development is design emphasis.

Designmatters, and good designis required to ensure that the prod

uct is designed to promote future maintainability and modifiability,

both of which are crucial elements in sustainable development.

Teams alsoneed to understand how to designwhat theyareworking

on. Simpledesign,refactoring,and designpatterns allplayimportant

rolesin design. Everyproject and teamhas to find the right balance

betweeniterativedesignand up-frontdesign. When theydo up-front

design, however, it isnot done in the traditionalwaythat isheavyon

documentation, but rather through lightweight collaborative design

sessions and the use of design patterns. Design experience and

domain expertise is vital in ensuring that teams do not rely on up

front design and design patterns too much because there should

always be a bias toward iterative design.

• The final principle of sustainable development is continual refine

ment. Continual refinementappliesto how the project isplanned and

tracked through iterative development. Equally important is that

continual refinement is required for how the project is approached,

so that the team can continually enhance its development processes

and collaboration.

• The discipline of iterative development is still useful in applications

where continual change is not necessary, possible, or desired. These

are projects such as software used in medical analysis (where people

can get hurt if there is a defect) or projectswith set milestones that

must be met in a certain order. This is because even if there is litde

changein the iteration plans, the simple discipline of being able to

plan and track progress ishighlyvaluable.

• Because culture change is most often required to achieve the necessarymindset, I have included a chapter as a starter toward achieving

the change.

I hope you, the reader, find this book useful.

CONCLUSION • 213



References

[Ambler and Jeffries 2002] Ambler, ScottW.Jeffries, Ron,AgileModeling

Wiley, 2002. Seealsowww.agilemodeling.com and www.agiledata.org.

[Armitage2004] Armitage,John, AreAgileMethods Goodfor Design?, IEEE

Transactions, Jan/Feb 2004.

[Astels2003] Astels,David, Test Driven Development:A PracticalGuide,Prentice Hall PTR, 2003.

[Auer and Miller 2001] Auer, Ken, and Miller,Roy, Extreme Programming

Applied, Addison-WesleyProfessional, 2001.

[Bandaru2004] Bandaru, V., Understanding XP: A CMM Perspecitive, Devel-

operlQ, Oct 2004.

[Beck2002a] Beck, Kent, Test Driven Development:ByExample, Addison-


[Beck2002b] Beck, Kent,XPandCulture Change, Cutter ITJournal, Cutter

Consortium, Oct 2002.

[Beck2004] Beck,Kent, Extreme Programming Explained, 2nd Edition,

Addison-Wesley,2004.[Boehmand Turner 2004] Boehm,Barry, and Turner,Richard,Balancing

AgilityandDiscipline, Addison-Wesley, 2004.

[Buckingham and Coffman 1999] Buckingham,Marcus, and Coffman, Curt,

First, BreakAll theRules,Simon & Schuster, 1999.

[Christensen 2003] Clayton M. Christensen, TheInnovator's Dilemma,

HarperBusiness, 2003.

[ChristensenandRaynor2003] Christensen,ClaytonM., andRaynor,MichaelE., TheInnovator'sSolution, Harvard Business School Press, 2003.

[Cockburn2001]Cockburn,Alistair,AgileSoftwareDevelopment, Addison-Wesley,2001.

[Cockburn 2004] Cockburn, Alistair,Crystal Clear:A Human-Powered

Methodology forSmallTeams, Addison-Wesley Professional, 2004.[Cohn2004]Cohn,Mike,User Stories Applied: ForAgileSoftware Develop

ment, Addison-WesleyProfessional, 2004.

[Cohn 2005] Cohn, Mike,A Regular Heartbeat: TheBenefits ofFixedIteration

Lengths, 27January 2005,Cutter ConsortiumAgileProject Managemente-MailAdvisor, e-mail:[email protected].

• 215 •



[Collins2001] Collins,Jim, GoodtoGreat, HarperBusiness, 2001.

[Collins and Porras 2002]Collins,Jim, and Porras, Jerry I.,BuilttoLast:

SuccessfulHabitsof Visionary Companies, HarperBusiness, 2002.

[Cox 1986]Cox,B.J.,Object-Oriented Programming:An Evolutionary

Approach, Addison-Wesley, 1986.

[Crane 2002] Crane, Thomas G., TheHeartof Coaching FTAPress, 2002.[Demarco and Lister 1999]Demarco,Tom, and Lister,Timothy,Peopleware:

Productive Projects andTeams, 2nd Edition, Dorset House, 1999.

[Dumas andRedish1999]Dumas,Joseph S.,and Redish,JaniceC,A Practical

GuidetoUsability Testing Intellect, Ltd. (UK), 1999.

[Economist 2004]MakeitSimple:A Survey ofInformation Technology, The

Economist, October 2004.

[Evans2003] Evans,Eric,Domain-Driven Design, Addison-Wesley, 2003.

[Fowler 1999]Fowler,Martin,Refactoring: Improving theDesignofExisting

Code, Addison-Wesley, 1999.

[Fowler: IsDesign Dead?] Fowler,Martin, IsDesign Dead?, http://www

jiiartinfowler.com/articles/designDead.html.[Fowler2002]Fowler, Martin,Patterns ofEnterpriseArchitecture, Addison-

Wesley, 2002.

[Gamma et al 1995]Gamma, Erich, Helm, Richard,Johnson, Ralph, and

Vlissides, John, Design Patterns, Addison-Wesley Professional,1995.[Glazer 2001] Glazer,Hillel,Dispelling theProcessMyth: Havinga Process

DoesNotMeanSacrificing AgilityorCreativity, Crosstalk:The Journal of

DefenseSoftwareEngineering, November2001.

[Grady 1997]Grady,Robert B.,SuccessfulSoftwareProcess Improvement,

PrenticeHall PTR, 1997.

[Highsmith 1999]Highsmith,Jim,Adaptive Software Development:A Collaborative Approach toManaging Complex Systems, Dorset House, 1999.

[Highsmith 2002]Highsmith,Jim,AgileDevelopment Ecosystems, Addison-

Wesley, 2002.

[Highsmith 2004a] Highsmith,Jim,AgileProjectManagement, 2004.

[Highsmith 2004b] Highsmith,Jim, Planning andScanning 9 December 2004,

Cutter ConsortiumAgileProjectManagemente-MailAdvisor, e-mail:

[email protected].

[Highsmith 2005]Highsmith,Jim, The Limits ofEvolutionary Design, 6January2005, Cutter Consortium AgileProject Management e-MailAdvisor,

e-mail: [email protected].

[Hohmann 1996]Hohmann, Luke, Thejourney of theSoftware Professional:TheSociology ofSoftware Development, Prentice Hall PTR, 1996.

[Hunt and Thomas 2000] Hunt, Andrew, and Thomas, David, ThePragmatic

Programmer, Addison-Wesley, 2000.

[Husted and Massol2003] Husted, Ted, and Massol, Vincent, JUnit inAction,

Manning Publications, 2003.

216 • REFERENCES



[Jeffrieset al2000]Jeffries,Ron,AnnAnderson, ChetHendrickson,andRonaldE.Jeffries, Extreme Programming Installed, Addison-WesleyProfessional, 2000.

[Kerievsky 2004] Kerievsky, Joshua,Refactoring toPatterns, Addison-WesleyProfessional, 2004.

[Kerth2001]Kerth, Norm, Project Retrospectives: AHandbookfor TeamReviews, Dorset House, 2001.

[Kotter 1996]Kotter,John P.,Leading Change, HarvardBusiness SchoolPress,1996.

[Larman 2003] Larman, Craig,AgileandIterative Development:AManagers

Guide, Addison-WesleyProfessional,2003.

[Lindvalet al2004] Lindvall,Mikael,Muthig,Dirk, Dagnino,Aldo,Wallin,Christina, Stupperich, Michael,Kiefer, David,May, John, and Kahkonen,

Tuomo,AgileSoftware Development inLarge Organizations, IEEE Com

puter, December 2004.

[Link and Frolich 2003] Link, Johannes, and Frolich,Peter,UnitTesting in

]ava, Morgan Kaufmann, 2003.[Lundin et al2000] Lundin, StephenC, Paul, Harry,andChristensen,John,

Fish!A Remarkable Way to Improve Morale andBoostResults, Hyperion,

2000 .

[Lynnand Reilly 2002] Lynn,Gary S., and Reilly, RichardP.,Blockbusters: The

FiveKeys toDeveloping GREATNewProducts, HarperBusiness,2002.

[Marick 1995]Marick, Brian,TheCraft ofSoftware Testing Prentice-Hall,

1995.See alsowww.testing.com.

[Martin 2002] Martin, Robert C,AgileSoftware Development, Principles,

Patterns, and Practices, Prentice Hall, 2002.

[McConnell 1996]McConnell,Steve,Rapid Development,

MicrosoftPress,1996.

[McGrenere 2000] McGrenere, Joanna, Bloat: TheObjective andSubjective

Dimensions, Department of Computer Science,Universityof Toronto,

CHI 2000.

[Moore 2002] Moore, Geoffrey A., Crossing theChasm:Marketing andSelling

Disruptive Products toMainstream Customers, HarperBusiness,2002.

[Norman 1999] Norman, Donald A., TheInvisible Computer, MIT Press, 1999.

[PalmerandFelsing2002]Palmer, StephenR.andFelsing, John M.,A PracticalGuide toFeature-Driven Development, PrenticeHall PTR,2002.

[Paulk et al 1993] Paulk, M.C., Curtis, B., Chrissis,M.B., and Weber, C.V.,

CapabilityMaturityModel, Version 1.1, IEEESoftware, Vol. 10 No. 4(July 1993),pp. 18-27.

[Paulk2001]Paulk,MarkC, Extreme Programming from aCMM Perspective,XP Universe, July 2001.

[Poppendieck andPoppendieck2003]Poppendieck,Mary, andPoppendieck,Tom,Lean Software Development, Addison-Wesley Professional,2003.

REFERENCES • 217



[Pressman 1992]Pressman, RogerS.,Software Engineering A Practitioner's

Approach, 3rd Edition, McGrawHill, New York, 1992.

[Repenning and Sterman 2001] Repenning,Nelson P., and Sterman,John D.,

Nobody Ever Gets CreditforFixing Problems ThatNever Happened: Creat

ingandSustaining Process Improvement, CaliforniaManagement Review,

Vol.43 No. 4, Summer2001.[Ronsse et al 2003] Ronsse, Michiel, De Boschere, Koen, Christiaens, Mark,

Chassinde Kergommeaux, Jacques, and Kranzmuller, Dieter,Record/

Replay forNondeterministicProgram Executions, Communicationsofthe ACM,September 2003Volume46, Number 9.

[Rothman2000]Rothman,Johanna, WhatDoesItCost You toFixaDefect?

AndWhy Should You Care?,www.catapulse.com, October 2000.[Rubin 1994] Rubin,Jeffrey, Handbook ofUsability Testing:HowtoPlan,

Design, andConduct Effective Tests, Wiley, 1994.

[Schwaber and Beedle 2001] Schwaber, Ken,and Beedle,Mike,AgileSoftwareDevelopmentwithSCRUM, Prentice Hall, 2001.

[Stapleton2003]Stapleton, Jennifer, DSDM: Business FocusedDevelopment,Pearson Education, 2003.Seealsowww.dsdm.org.

[Thomasand Hunt 2002]Thomas,Dave,and Hunt, Andy,Mock Objects,IEEE Software,May/June 2002.

[Thornton 1996] Thornton, Ian,Krakatau: The Destruction andReassembly of

anIsland Ecosystem, Harvard University Press, 1996.

[WomackandJones 1996]Womack, JamesP.,andJones,DanielT, Lean

Thinking: Banish Waste andCreate Wealth in Your Corporation, Simonand Schuster, 1996.

218 • REFERENCES



adaptation, 134

adaptive softwaredevelopment, 193

Adobe's Eve, 114

agility, 41,211

capabilitymaturity model

and, 195

consistency vs., 170

continual refinement and, 134

design and, 110

development methods for, 193

Extreme Programming and, 183

manifesto on, 172

AND/OR thinking, 10

Apple

Shark, 87

war rooms, 169

Apple iPod, 24

architecture

identifying batdes over, 167

rearchitecture and, 123

record and playback, 84

user visibilityof, 116

barely sufficient documentation, 53

Boundschecker, 96

bug tracking tools, 75

bugzilla,52

builds

metrics for, 148nighdy, 56

release planning and, 139

time taken by,57

timestamping, 59

bureaucracy, 30

burnout, 9

Index

business models

disruptive, 19

business plans, 18

capability

workingharder vs.working

smarter and, 3

capabilitymaturity model

(CMM), 195

ceremony, 100

discipline vs., 36

change, 12

agents, 151

aversion to, 13

continual refinement and, 133

cultural, 151

designingfor, 107, 111

disruptive technologies and, 18

effecting,152

enablers of, 158

examples of, 170

factors for, 156

jumping in place and, 16

need for, 212

phases of, 165

react ions to, 156

resistance to , 154

tolerance of, 39

transition plans and, 162chemicalmanufacturing, sustainable

development in, 3,5

coaching, 144

Cobol, 11

code

copyand paste porting, 69

219



• 220 •

code {cont.)

documentation of source, 95

metrics for, 150

ownership of, 98

platform dependencies and, 69

reviews, 98source code analyzersand, 93

code comments, 54

code-aware tests, 80

code-then-fixmentality, 2,28,211

defect prevention and, 73

working product and, 42

coding standards and guidelines,65

collaboration, 38

asproject control, 21

cross-functional, 137

design and, 110

in Extreme Programming, 186

in software design,55

iterative development and, 137

positive reinforcement of, 161

with users, 168

Collins,Jim, 15,40

command-control, 152

commodities, 67

communication, 119,161

competition, 16,18

compilers, defect prevention and, 92

complacency,154

complementarity of talents/skills, 39

complexity,12

componentized software,130

configurationmanagement,57,96

consistency, 170

constructor methods, 93consultants, 160

continual improvement, 7,9,11

continuous integration, 56

copy and paste porting, 69,93

cornerstone tests, 91

cost management, 20

INDEX

CPD, 93

Creativity. Seealsoinnovation, 195

for retrospectives, 142

in automated testing, 81

culture, organizational, 23,36

changing, 151,212defect detection oriented, 75

of software development, 28

work environment in, 169

c u s t om e r s

collaboration with, 168

contact with, 147

defect reporting by,44

disruptive technologies and, 18

feedback from, 29,31

in Extreme Programming, 187

mindshare of, 16

on-site, 190

positivereinforcement of contact

with, 161

understanding, 145

value to, 3

dailyshipping, 43

data structures, 63

debugging

milestones and, 47


decisionmaking, 23

defects

avoidingbacklogsof,51

backlogsof,75

build times and, 57

cost of fixing and time of

discovery,73designing for reuse and, 128

detection of and cost of

change, 13

detection vs. prevention of, 8

metrics for, 148,167

milestones and, 47



no brokenwindowspractice

and, 49

practices for preventing, 179

pragmatic practices for, 52

preventing, 73,212

preventing from reachingcustomers, 42

preventing vs. detecting, 31,

33,73

prioritizing, 101

programming and code reviews

and, 98

quality assurance and, 73,76


root-cause analysisand, 100

ruthless testing for,78

time spent fixing, 13

tools for, 92

tracking, 52

uncompromising attitude

towards, 50

won't fix, 101


Dell Computers, 19

design, 107,213

bottom-up, 108

discussions, 122

emphasis on, 33

extreme, 108


for reuse, 128

for testability,84

frequent meetingsin, 122

guiding principles in, 112-113

patterns in, 119

practices for, 180


refactoring and, 117

reviews, 122

simplicity in, 115

top-down, 108

vision in, 112

deterministic programs, 85

discipline, 36,142


disruptive technologies, 18

diversity, 153documentation

barely sufficient,53

code reviews and, 100

dangers of excessive,55

design and, 110

in plan-driven development, 22

source code, 95

updating, 44

doxygen,55

dynamic systemsdevelopment

method (DSDM), 194

Eclipse, 45

education, 154,211

emergent design, 108.Seealso

design, 195

engineeringapproach to software

development, 27

engineering, guiding principles

for, 112-113

errors, defect prevention and, 93

Eve, 114

event logging,95

evolutionary design, 108.Seealso

design, 195

exception handling, 93

execution profilers, 94

exit criteria, 53

expertise, 22external dependencies, 18

Extreme Programming

design in, 192

misconceptions of, 192

practices for, 184

strengths of, 185

INDEX • 221



222

Extreme Programming (cont.)

values of, 183

weaknesses of, 187

Extreme Programming (Beck),108

Extreme Programming (XP), 183

feature cards, 55,136

feature-drivendevelopment,193

features

AND/OR thinking about, 10

continual improvement and, 7,

9,11

documenting,54

metrics for, 148

non-feature work and, 8

planning development of,44

simplicityand reliability and,23

time spent developing, 13

vision and, 29

feedback

from users, 29,31

in continual refinement, 142

iterative developmentand, 135

retrospectives and, 142

flexibility, 41

flywheel metaphor, 15

fun, 40

functionalityfreezes, 47

game engines, 85

global variables, 93

Good to Great (Collins), 40

guiding councils, 174

hardware, 63

Highsmith, Jim, 109,172hotkey managers, 116

Hunt, Andrew, 49

IBM, 67

improvement

continual, 8,39

INDEX

working harder vs.working

smarter and, 6

innovat ion

bureaucracy and, 30

pace of, 1

integrationcontinuous, 100

tests for, 81,85

integration, continuous, 56

interfaces

designing for reuse and, 128

external dependencies and, 18

integration testing, 81,85

internationalization of products, 68

In ternet

as disruptive technology,19

iPod,24

ISO-9000,21

i terations

bug fix-only, 53

design reviewsand, 122

iterative development, 135,213

Java, PMD for, 93

JavaDoc, 55

JavaDocs, 95

jumping in place, 16

Kerievsky, Joshua, 109

keystone practices, 177.Seealso

practices, 195

leadership, 23

change and, 156

culture of, 37Extreme Programming

and, 191

lean development, 193

Lean SoftwareDevelopment

(Poppendieck), 43

learning, continuous, 29,39



libraries, integration tests

and, 81

Linux, 20

loggingcapability, 84

logging, event, 95

logic errors, 99

management

attitude leadership and, 8

change and, 152

parking lot managers in, 5

project, 171

support for change and, 158

manipulation, 152

market share, 16

markets, understanding, 146

meetings

daily standup, 141

design, 122

status report, 141

memory leaks, 63

methodology,21

metrics, 148

for defects, 167

Microsoft

business model of, 20

dailyshipping at, 46

Office97,23

Windows, testing and, 88

milestones, artificial, 47

mindset

continual improvement and, 8

for sustainable development, 2

software development culture

and, 27

mock objects, 81

model-view-controllerpattern, 86

morale

working harder vs.working

smarter and, 4

multi-threading, 124

nested statements, 93

nightly builds, 56

no broken windows, 49

nondeterministic programs, 85

Norton Anti-Virus, 46

object-oriented programming, 130

objects,mock, 81

Office97,23

open source software, 20,45

defect prevention and, 97

fixes to, 67

Osborne Computer, Inc., 16

Osborne effect, 16

outsourcing, 146

overdesigning, 2,108,129

pair programming, 189

Palm PDA, 24

parking lot managers, 5

performance, 62

metrics, 148

testing, 87

persistance, 158

pillars of design, 112-113

plan-driven development, 21

release planning in, 139

sloppy planning and, 134

tradition methods and, 144

transition plans in, 162

platform dependencies, 69


PMD, 93

positive reinforcement, 160

practices, 28,177

attention to performance, 62

barely sufficient

documentation, 53

coding standards/guidelines, 65

continual refinement of, 31

continuous integration, 56

INDEX 223



• 224 •

practices {cont.)

defect prevention, 78

for continual refinement, 135

for Extreme Programming, 184

internationalization of

products, 68nighdy builds, 56

no brokenwindows, 49

principles vs., 28

prototyping, 59

standards adoption, 67

uncompromising attitude toward

defects, 50

zero tolerance for

memory/resource leaks,63

Pragmatic Programmer (Hunt,

Thomas), 49

principles, practicesvs.,28

product

continual refinement of, 32

working, at all times,31-32

product development, 29

product differentiation, 33

professional development, 148

project controls, 21

project management, 189

training for, 171

project stresses, 17

prototyping, 59


Purify, 96

Qt,45

quality assurance (QA)

defect prevention vs.detectionand, 33

importance of prompt, 43

in defect prevention, 73,76

reactive thinking, 134

record and playback,84

INDEX

refactoring, 117

group exercises in, 168

supporting, 161

refinement, continual, 133,213

coaching/team development

in, 144

dailystandup meetings in, 141

iterative development in, 135

metrics in, 148

practices for, 181

release planning in, 139

retrospectives in, 142

regression tests, 88

regressions, 51,75

reinforcement, positive, 160

reinvention of companies, 3releases

death spiral and, 14

defect backlogs and, 75


planning, 139

reliability, 23

Renerware, 46

Repenning, Nelson P., 4

replaceability, 129

research projects, iterative

development for, 137

resource leaks, 63, 88

detecting, 95

resource-usage tests, 88

responsibility, 37

retrospectives, 142

reuse, designing for, 128

ripple effects, 44

risk

awareness of, 39

prototyping and, 59

roadmaps, 124

root-cause analysis,100

rules, 28

run-time monitoring, 87



scalability, 189

scheduling

artificial milestones and, 47

self-organization, 191

Shark, 87

shippingat unpredictable times, 47

daily, 45

simplicity,23

in design, 115

SketchBook Pro, 55

sketching tools, 55

skills,complementary,39

software development. Seealso

design, 195

agilemethods for, 193capabilitymaturity model

and, 195

culture of, 28

death spiral in, 13

engineering approach to, 27

Extreme Programming in, 183

iterative, 135,213

manifesto for agile, 172

methodology of, 21

plan-driven development, 21

waterfall approach to, 27

SoftwareEngineering

Institute, 195

software ICs, 130

source code analyzers,93

standards

adopting, 67

for coding, 65

standup meetings, 141

Sterman, John D., 4

sustainable development, 212

advantages of, 3

applying principles of, 31

as juggling,34

beginning, 163

capabilitymaturitymodel

and, 195

chemicalmanufacturing example

of, 3,5

Coboland, 11

culture change and, 151definition of, 1

designin, 107

example of, 10

feedback and, 31

for new projects, 166

mindset behind, 2

practices and, 29

principles of, 27

vision and, 29,31

switch statements, 93

system tests, 83

talents, complementary,39

t e a m s

developing/coaching, 144

new projects and, 166

proactive, 2,212

self-organizationof, 191

technical debt, 15,77

testing, 44.Seealsoquality assurance

(QA),44,195

automated, 81-82,87,90

cornerstone, 91

cost of, 74

equipment for, 169

in defect prevention, 78

integration, 81, 85

performance, 87

positive reinforcement of, 161


record and playback, 84

regression, 88

resource-usage, 88

resources for, 90

system, 83

INDEX • 225 •