contributors to sw project failure - survey, conjecture, and preachy advice this presentation has...

Contributors to SW Project Contributors to SW Project Failure - Survey, Conjecture, Failure - Survey, Conjecture, and Preachy Adviceand Preachy AdviceThis presentation has been given to 46 This presentation has been given to 46 companies and organizations, in 8 companies and organizations, in 8 countries. No visible effect on the world countries. No visible effect on the world economy has been noted.economy has been noted.

Michael Buckley - Faculty, Computer Science and EngineeringMichael Buckley - Faculty, Computer Science and Engineering

State Univ. of NY at Buffalo – mikeb @ cse.buffalo.eduState Univ. of NY at Buffalo – mikeb @ cse.buffalo.edu

25 years as a consultant dealing with paying (and non-paying) 25 years as a consultant dealing with paying (and non-paying) customers.customers.

How Do Software-based How Do Software-based Projects Fail?Projects Fail?

Unhappy Customer – do not get what Unhappy Customer – do not get what they expect or expect what they getthey expect or expect what they get– Ran out of time and $$Ran out of time and $$– Unhappy user Unhappy user

Safety compromised, including death Safety compromised, including death Technically inadequate or over-Technically inadequate or over-

adequateadequate Does not contribute to the company Does not contribute to the company

business casebusiness case Menace to Society – Menace to Society – student favoritestudent favorite

Preachy advice to students Preachy advice to students #1#1

Never lose the customer’s point of Never lose the customer’s point of viewview

Never lose the user’s point of viewNever lose the user’s point of view Never lose the business point of viewNever lose the business point of view Never lose the team-member point of Never lose the team-member point of

viewview

Sometimes we risk Sometimes we risk scheduleschedule

85% of SW projects are either late or 85% of SW projects are either late or delivered under-spec.delivered under-spec.

source: SEI source: SEI Web SiteWeb Site

Microsoft no longer names their Microsoft no longer names their products for the release year. Um, why?products for the release year. Um, why?

source: my source: my computercomputer

Sometimes we risk Sometimes we risk budgetbudget

If the entire budget for the Denver If the entire budget for the Denver Airport Automated Baggage System Airport Automated Baggage System had been converted to cash, it could had been converted to cash, it could have paid wages for a manual have paid wages for a manual system for 1000 years.system for 1000 years.

source: Modern Materials Handling Magazinesource: Modern Materials Handling Magazine

Magazine?

... we lose money... we lose money ATMs in Mexico City were processing debits ATMs in Mexico City were processing debits

as credits, but only at the as credits, but only at the user account leveluser account level (i.e. bank and machine settlements were (i.e. bank and machine settlements were accurate)accurate)

– lines at the ATMs sometimes totaled 1200 peoplelines at the ATMs sometimes totaled 1200 people– ATM use increased 400%ATM use increased 400%– users managed to keep it an underground secret users managed to keep it an underground secret

for 2 weeksfor 2 weeks– misappropriated credits totaled $4 million, a small misappropriated credits totaled $4 million, a small

percentage was never recoveredpercentage was never recoveredsource: I was there

A Classic Error

what they wrote:what they wrote:

if (A = B)if (A = B)

B = C;B = C;

return;return;

what they wanted:what they wanted:if (A == B)if (A == B) {{

B = C;B = C;return;return;}}

what they got:what they got:A = B;A = B;if (A == 1)if (A == 1) {{ B = C;B = C; }} return;return;

#include <stdio.h> main($,_,__)char*__;{return !0<$?$<3?main(-79,-13,__+main(-

87,1-_,main( 86,0,__+1)+__)):1,$<_?main($+1,_,__):3,main(-94,-27+$,__)&&$==2?_<13?main(2,_+1,"%s %d %d\\n"):9:16:$<0?$<-72?main(_,$,"@n'+,#'/*{}w+/w#cdnr/+,{}r/*de}+,/*{*+,/w{%+,/w#q#\n+,/#{l+,/n{n+,/+#n+,/#;#q#n+,/+k#;*+,/'r :'d*'3,}{w+K w'K:'+}e#';dq#'l q#'+d'K#\!/+k#;q#'r}eKK#}w'r}eKK{nl]'/#;#q#n'){)#}w'){){nl]'/+#n';d}rw' i;# ){nl]!/n{n#';\ r{#w'r nc{nl]'/#{l,+'K {rw' iK{;[{nl]'/w#q#n'wk nw' iwk{KK{nl]!/w{%'l##w#' i; :\{nl]'/*{q#'ld;r'}{nlwb!/*de}'c ;;{nl'-{}rw]'/+,}##'*}#nc,',#nw]'/+kd'+e}+;#'rdq#\w! nr'/ ') }+}{rl#'{n' ')# }'+}##(!!/"):$<-50?_==*__?putchar(31[__]):main(-65,_,__+1):main((*__=='/')+$,_,__+1):0<$?main(2,2,"%s"):*__=='/'||main(0,main(-61,*__,"!ek;dc i@bK'(q)-[w]*%n+r3#l,{}:\nuwloca-O;m .vpbks,fxntdCeghiry"),__+1);}

Sometimes we are too Sometimes we are too eager to technologizeeager to technologize

The Puget Sound ferry system had been The Puget Sound ferry system had been using mechanical and hydraulic controls using mechanical and hydraulic controls since 1875. After converting to computers since 1875. After converting to computers and electronics, it began bumping docks, and electronics, it began bumping docks, switching from forward to reverse by itself, switching from forward to reverse by itself, injuring passengers, and dumping cars, injuring passengers, and dumping cars, and caused more damage in its next 90 and caused more damage in its next 90 days than in its previous 100 years.days than in its previous 100 years.

source: source: Software WoesSoftware Woes

Sometimes we risk Sometimes we risk functionfunction

Long distance services for every state east Long distance services for every state east of the Mississippi went down for two days of the Mississippi went down for two days due to a software change made 1 day due to a software change made 1 day before shipping a switching computer, that before shipping a switching computer, that had just passed 19 weeks of tests.had just passed 19 weeks of tests.

The change was made to the machine code, The change was made to the machine code, in 1s and 0s, without benefit of source code in 1s and 0s, without benefit of source code and compilation.and compilation.

The manager had to appear before a The manager had to appear before a Congressional committee (Committee on Congressional committee (Committee on Common Sense?)Common Sense?)

source: this guy was later my boss

Killer AppsKiller Apps The Therac 25 cancer radiation system killed 4 The Therac 25 cancer radiation system killed 4

patients on the table, due to a software error patients on the table, due to a software error that misjudged the appropriate dosage level that misjudged the appropriate dosage level based on the position of a mechanical filter.based on the position of a mechanical filter.– The company first regarded it as a user issue, The company first regarded it as a user issue,

because it was precipitated by an odd start-up because it was precipitated by an odd start-up sequencesequence

A fuel-saving computer on the DC-10 A fuel-saving computer on the DC-10 attempted to save fuel by cutting the engines attempted to save fuel by cutting the engines during landing. during landing. – The pilot restarted manually in time.The pilot restarted manually in time.

source: Software Woessource: Software Woes

Computer and human Computer and human interactioninteraction

At Chernobyl, technicians tried to run an At Chernobyl, technicians tried to run an experiment to feed power to backup experiment to feed power to backup generators, as the system was shutting down generators, as the system was shutting down (combining two operations into one (combining two operations into one sequence). sequence). – Technicians needed to send control rods into the Technicians needed to send control rods into the

reactor core (usually a “run” process), during reactor core (usually a “run” process), during shutdown. shutdown.

– The control system was not programmed to do The control system was not programmed to do anything during shutdown, except shut down.anything during shutdown, except shut down.

– Control rods did not respond, power spiraled, the Control rods did not respond, power spiraled, the core overheated.core overheated.

source: Inviting Disaster

Killer AppsKiller Apps An intravenous medication pump ran dry and An intravenous medication pump ran dry and

injected air into the patient. injected air into the patient. – the design RELIED ON manual interventionthe design RELIED ON manual intervention

A digital display combined the name of one A digital display combined the name of one patient with medication information from patient with medication information from another. Staff thought they had mis-another. Staff thought they had mis-medicated the patients.medicated the patients.

A heart monitor was designed to sound an A heart monitor was designed to sound an alarm when the patient’s heart rate dropped alarm when the patient’s heart rate dropped below a certain threshold, but shut off when below a certain threshold, but shut off when the heart stopped, thereby missing entirely the heart stopped, thereby missing entirely any cardiac arrest.any cardiac arrest.

Technology MisappliedTechnology Misapplied

The first robot to kill a human happened 25 The first robot to kill a human happened 25 years ago, and 4 times since (in industry).years ago, and 4 times since (in industry).– robots are primarily softwarerobots are primarily software

GM’s first attempt at robotic assembly GM’s first attempt at robotic assembly resulted in 100% robot failure within 40 hrs. resulted in 100% robot failure within 40 hrs.

The solution?The solution? – ““As much of a failure as a bug.”As much of a failure as a bug.”

source: Scientific American “The Mechanization of source: Scientific American “The Mechanization of Work”Work”

Asimo Tricked Into Falling Down Stairs

Survey of Working Software Survey of Working Software DevelopersDevelopers

““Give the top ten reasons why software projects Give the top ten reasons why software projects fail”fail”

472 students, representing 104 companies.472 students, representing 104 companies. 28% management28% management 61% contributing staff61% contributing staff 11% full time students11% full time students No item was included that did not have 5+ mentions No item was included that did not have 5+ mentions

in the survey.in the survey. Categories include People & Management, Process, Categories include People & Management, Process,

and Styleand Style Only Only slightlyslightly listed in order of impact. listed in order of impact.

References: “Code Complete”, Steve McConnell, References: “Code Complete”, Steve McConnell, Microsoft PressMicrosoft Press

People / ManagementPeople / Management

#1 - Territoriality, on the part #1 - Territoriality, on the part of all participants, at all levels, of all participants, at all levels, in all categories.in all categories.

examples: examples: ““marketing was always interfering”marketing was always interfering”““the customer kept asking for more”the customer kept asking for more”““my engineers kept trying new things”my engineers kept trying new things”““everyone wanted it their own wayeveryone wanted it their own way

preachy advice to students preachy advice to students #2#2

““Disagree, concede, compromise, Disagree, concede, compromise, but build the best system but build the best system possible.”possible.”

People / ManagementPeople / Management Weak Personnel and Problem Weak Personnel and Problem

Employees (see McConnell) - spoiling Employees (see McConnell) - spoiling an otherwise jelled team - includes an otherwise jelled team - includes management above the team.management above the team.

Reducing a person’s contribution, by Reducing a person’s contribution, by not managing the whole person – not managing the whole person – treating a person as “not a person” treating a person as “not a person” because he/she’s at workbecause he/she’s at work

- 15% of injuries treated at a Bethlehem Steel - 15% of injuries treated at a Bethlehem Steel plant were from kicking and punching time plant were from kicking and punching time clocks.clocks.

source: Trouble in source: Trouble in BethlehemBethlehem


Team member burnout.Team member burnout. Undermined Motivation (McConnell) - Undermined Motivation (McConnell) -

– Office GossipOffice Gossip– Office PoliticsOffice Politics

Unrealistic Expectations and Wishful Unrealistic Expectations and Wishful Thinking (McConnell) - planning to Thinking (McConnell) - planning to catch up later.catch up later.

Lack of project sponsorship from above Lack of project sponsorship from above and buy-in from below (McConnell)and buy-in from below (McConnell)

preachy advice to students preachy advice to students #3#3

““Why do we choose the jobs that we Why do we choose the jobs that we do?”do?”


Inadequate, uninformed budgets and Inadequate, uninformed budgets and schedules – schedules –

– There is a minimum, but no maximum, of There is a minimum, but no maximum, of what a project will take. Admit to that what a project will take. Admit to that minimum. Set up realistic groundrules, minimum. Set up realistic groundrules, track and control the project from track and control the project from beginning to end.beginning to end.

– Stupid estimatesStupid estimates– Knowingly omitting tasks from the budget Knowingly omitting tasks from the budget

and scheduleand schedule

No contingency plan - starting out with No contingency plan - starting out with 60 hr. weeks.60 hr. weeks.


Using unpaid overtime in the Using unpaid overtime in the schedule baseline.schedule baseline.

Insufficient basic management - Insufficient basic management - risk management, planning, risk management, planning, reporting, and controls reporting, and controls (McConnell) - cross your fingers (McConnell) - cross your fingers and wait for surprises. and wait for surprises.

Abandonment of planning Abandonment of planning under pressure (McConnell).under pressure (McConnell).


Abandonment of common sense Abandonment of common sense under pressure - testing, under pressure - testing, quality assurance, personal quality assurance, personal hygiene.hygiene.

Hero based projects –Hero based projects –– Reliance on one person’s mania Reliance on one person’s mania

and drive.and drive.– The truck factor.The truck factor.

The The WrongWrong/Right List/Right List

The survey included many anecdotes – The survey included many anecdotes –

RankRank / Responsibilities / Responsibilities My wayMy way / Our way / Our way Arbitrary DeadlinesArbitrary Deadlines / Reality / Reality ThreateningThreatening / Teaching / Teaching Time ClocksTime Clocks / Autonomy / Autonomy Employee of the MonthEmployee of the Month / Team- / Team-

oriented Rewardsoriented Rewards

The The WrongWrong/Right List/Right List Experts Experts / Generalists/ Generalists Temporal bonusesTemporal bonuses / Bonuses based / Bonuses based

on performanceon performance E-mailE-mail / Face to face / Face to face No personal callsNo personal calls / Home at work & / Home at work &

Work at HomeWork at Home One big goalOne big goal / Incremental goals / Incremental goals Bureaucracy - don’t use that credit Bureaucracy - don’t use that credit

cardcard / Find a way / Find a way Copied softwareCopied software / Legal software / Legal software

The The WrongWrong/Right List/Right List

1-person Jobs (Heroes)1-person Jobs (Heroes) / Teams / Teams (Leaders)(Leaders)

Casual OvertimeCasual Overtime / Compensation / Compensation SarcasmSarcasm / Honesty / Honesty Calculated DeceptionCalculated Deception / Honesty / Honesty Management by IntimidationManagement by Intimidation / /

Delegation of ResponsibilityDelegation of Responsibility Annual Performance Reviews – Annual Performance Reviews –

ambushambush / Monthly Performance Reviews / Monthly Performance Reviews

The The WrongWrong/Right List/Right List Mandatory OvertimeMandatory Overtime / Goal Oriented / Goal Oriented

- interesting work, realistic schedules, - interesting work, realistic schedules, or else or else staff it correctlystaff it correctly

Status MeetingsStatus Meetings / Issues Meetings / Issues Meetings Hands-off Hands-off / Hands-On/ Hands-On Misplaced RecognitionMisplaced Recognition / Knowing your / Knowing your

teamteam 5-O’clock walk-arounds5-O’clock walk-arounds / All-day walk / All-day walk

aroundsarounds Management by ClichéManagement by Cliché / Management / Management

of Realityof Reality

ProcessProcess

% of time spent% of time spent

Spec Analysis Design Code Integration Test Maintenance

The Dream Curve

ProcessProcess Omission of Lifecycle Steps - Omission of Lifecycle Steps -

– Lack of a requirements specification.Lack of a requirements specification.– Inadequate (or omission of) DesignInadequate (or omission of) Design– Lack of a test plan.Lack of a test plan.– Coding too quickly. Designing at the Coding too quickly. Designing at the

terminal.terminal.

Overspecification and too-rigid Overspecification and too-rigid specification and design.specification and design.

– Some details are better decided Some details are better decided incrementally.incrementally.

– Designing for flexibility is different Designing for flexibility is different than designing for function.than designing for function.

ProcessProcess

Requirements creep, feature Requirements creep, feature creep, developer gold-plating creep, developer gold-plating (McConnell).(McConnell).

Lack of a Software Engineering Lack of a Software Engineering subculture -subculture -

– Engineering methodsEngineering methods– Tracking, traceability, and controlTracking, traceability, and control– EducationEducation

Lack of non-execution-based tests.Lack of non-execution-based tests. Miscommunication of intent - in Miscommunication of intent - in

requirements and design.requirements and design.

ProcessProcess

Lack of Coding Standards.Lack of Coding Standards. Out of date, inaccurate Out of date, inaccurate

documentation - requirements documentation - requirements specifications and test plans.specifications and test plans.

No Version Control or any No Version Control or any enforced record keeping.enforced record keeping.

– During development.During development.– During maintenance.During maintenance.

ProcessProcess

No Configuration Management No Configuration Management of delivered product.of delivered product.

Same people design and code. Same people design and code. Same people code and test.Same people code and test.

Over-application of technologyOver-application of technology– the Technology Scale (?) asks the Technology Scale (?) asks

“Could an animal do this cheaper? “Could an animal do this cheaper? Then how about a motor?”Then how about a motor?”

StyleStyle

Unneeded Complexity - in Unneeded Complexity - in design and implementation.design and implementation.

Misinterpretation of test results Misinterpretation of test results - incomplete regarded as - incomplete regarded as exhaustive.exhaustive.

Beige Cosmos Produces Red FacesBy JOHN NOBLE WILFORD

Few gave a thought to the color of the universe until two months ago, when astronomers at Johns Hopkins University ran calculations through a spectrum of color schemes and concluded that on average the universe is pale turquoise, or just a shade greener.

It is a pleasingly serene color, which made the front pages of newspapers and the TV news. But reality, it turns out, is not so vivid. The universe is really beige. Get used to it.

"We got it wrong," the astronomers, Dr. Karl Glazebrook and Dr. Ivan Baldry, announced yesterday. They said they had been led astray by a flaw in their computer software.

StyleStyle

Cleverness in favor of clarity.Cleverness in favor of clarity. Idiosyncratic style – Idiosyncratic style –

preachy advice to students preachy advice to students (I’ve lost count)(I’ve lost count)

““design and program for an design and program for an audience, to agreed standards.”audience, to agreed standards.”

““standards are guidelines”standards are guidelines”

A division of GrummanA division of Grumman

““There will be no comments in the There will be no comments in the source code. Code should be self-source code. Code should be self-documenting”documenting”

get a sense of perspectiveget a sense of perspectiveGo-to’s, pointers, and issues of clarityGo-to’s, pointers, and issues of clarity

Go-tos are not to be “avoided at all costs”. It is, instead, serpentine code that needs to be avoided. Simplicity and clarity should override most other design decisions. A go-to, in particular, is a powerful tool when used as a direct, no-nonsense jump under well-stated conditions, and can very closely follow problem-space behavior if used with some planning and forethought (Ada, a language designed from scratch by smart French people, contains a goto keyword). On the other hand, the indirection of a pointer tends to be a computer-space construct, that is often confusing and - if honesty should prevail - unnecessary (Java, the latest geek programming language, does not allow the use of pointers).

... Eastman Kodak’s coding ... Eastman Kodak’s coding standardsstandards

StyleStyle Not matching the “solution space” to Not matching the “solution space” to

the “problem space”the “problem space”– Reliance on computer-centric constructs to Reliance on computer-centric constructs to

match real world events.match real world events.– Synchronous modeling of asynchronous Synchronous modeling of asynchronous

processes.processes.

e.g. Using the pattern name instead of what e.g. Using the pattern name instead of what it’s modeling.... it’s modeling.... a program that tracked used a program that tracked used car parts, had four Model-View-Controllers, car parts, had four Model-View-Controllers,

all named Model_View_Controller.all named Model_View_Controller.

StyleStyle

Reliance on esoteric technologies Reliance on esoteric technologies and CASE tools. You should suffer and CASE tools. You should suffer through enough projects without through enough projects without them to understand their need them to understand their need and place.and place.

Extending a system’s design and Extending a system’s design and architecture beyond its intent - architecture beyond its intent - know when to start over.know when to start over.

Keeping prototypes as part of the Keeping prototypes as part of the end systemend system

StyleStyle Comments that reflect what is Comments that reflect what is

done rather than what is done rather than what is intended.intended.

Residual, inaccurate comments.Residual, inaccurate comments. Leaving commenting until the Leaving commenting until the

end.end.

my favorite, actual comments:#define ON 1#define OFF 1 // set to the same as ON to

// avoid confusion

StyleStyle

Indirect Addressing - pointers, Indirect Addressing - pointers, convoluted array indices.convoluted array indices.

Nesting greater than 3 levels. Nesting greater than 3 levels. Tightly coupled modules.Tightly coupled modules. Incohesive modules.Incohesive modules.

StyleStyle

Freely written global variables.Freely written global variables. Weakly typed variables - Weakly typed variables -

operations and comparisons done operations and comparisons done on variables of incompatible type.on variables of incompatible type.

Code not understood by the author Code not understood by the author - done by trial and error.- done by trial and error.

Coding for speed and size to the Coding for speed and size to the exclusion of architectural design.exclusion of architectural design.

SummarySummary

People/Management: People will largely People/Management: People will largely perform as expected, unless self-worth perform as expected, unless self-worth is damaged.is damaged.

Process: a Methodology is a necessity.Process: a Methodology is a necessity.

Style: Over-technically complex Style: Over-technically complex solutions are not required and yield solutions are not required and yield poor results.poor results.

contributors to sw project failure - survey, conjecture, and preachy advice this presentation has...

Documents