Adopting Agile Software Project Management Practices: Success Factors, Changes Required, and Challenges

Subhas C. Misra, Ph.D.

*History and EvolutionHeavily Process BasedCode-and-fix methodStage-wise methodWaterfall methodSpiral method

Less Emphasis on ProcessAgile Software Development

*Stage-wise Model

*Waterfall Model

*Spiral Model

*Small releases

Developing software difficult to predictCustomers prefer pieces of software incrementally in shorter time scalesPossibility of missing deadlines is lessEasier to respond to new requirementsSource: http://www.stanford.edu/class/cs193d/handouts/15-xp.pdf

*Agile Software Project ManagementEmerging discipline.Initially proposed and promoted by a group of seventeen software professionals who practice a set of lightweight methods, and share a common set of values of software development.They consolidated their thoughts, and defined these methods as agile. The Manifesto for Agile Software Development was released in early 2001.Gained importance among the software professionals primarily in the last few years.

*Agile ManifestoIndividuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

*PrinciplesOur highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project.Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Continuous attention to technical excellence and good design enhances agility. Simplicity--the art of maximizing the amount of work not done--is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

*ConcernsWhat are the factors that will influence the success of projects that want to adopt agile software practices? What are the important changes required for adopting agile software practices in projects practicing traditional plan-driven software development? Can we rank them according to their level of importance?What are the most important challenges/risks that projects may encounter for adopting agile software practices? Can we rank them according to their level of importance?

*Success Factors: A Conceptual Framework

*Dependent Variable: SuccessMeasuresReduced delivery schedulesIncreased return on investment (ROI)Increased ability to meet with the current customer requirementsIncreased flexibility to meet with the changing customer requirementsImproved business processesConsolidated Success Measure

*What we didPhase I Identification of the concerns. Formulation of the research questions.Construction of the theoretical frameworks. Development of hypotheses based on the research questions.Phase IICollection of data.Phase IIIAnalysis of the data collected as part of Phase II.

*MethodologyResearch designSurvey techniqueQuestionnaire designIdentifying the respondentsPre-testing the questionnaire

*

IdentityResponse percent1Computer related (IS/MIS/DP/Hardware/Software/Telecommunications)32.5%2Banking/insurance10.2%3Real estate1.2%4Business supplies/services1.2%5Education/research5.1%6Entertainment/media/publishing1.2%7Hospitality0%8Medical/health care2.4%9Government2%10Engineering/construction0.8%11Consulting29.8%12Legal services0%13Manufacturing/distribution2%14Consumer retail/wholesale0.8%15Non-profit/membership organization0%16Electrical machines0.4%17Aerospace2.4%18Others7.9%

*

Number of employees in respondents organizationsResponse percent1Less than 1011.8%210-205.9%321-409.8%441-1007.9%5101-50010.2%6501-100027.6%7Greater than 100026.8%

Number of employees in respondents teamsResponse percent1Less than 519.4%25-1033.3%311-2026.2%421-4010.3%5Greater than 4010.3%

*

RoleResponse percent1Functional Manager7.9%2Project Manager17.7%3Team Leader18.9%4Developer/Tester29.5%5Other26%

Duration developing software using ASDResponse percent1Less than 1 year19.6%21-3 years30%33-5 years26.4%4Greater than 5 years24%

*Degree of practice of the Agile principles (snapshot only )

Strongly Disagree(1)SomewhatDisagree(2)Neither Disagree nor Agree (3)SomewhatAgree (4)Strongly Agree (5)Not Applicable or Dont Know (X)ResponseAverageWe give high priority to satisfying customers through early and continuous delivery of valuable software2%2%5%12%77%2%4.63We welcome changing requirements, even late during development3%7%9%37%42%2%4.11We deliver working software more frequently, from couple of weeks to couple of months, with a preference to a shorter timescale3%2%5%19%69%2%4.52Our business people and developers work together daily (very closely) throughout the project4%10%6%29%49%2%4.12We build projects around motivated individuals. We give them the environment and trust them to get the job done.2%6%8%31%51%2%4.24We emphasize more on face-to-face communication for conveying information to and within the development team.1%3%8%23%63%2%4.46

*Data Analysis: Success Factors

VariablesCorrelation coefficientSignificanceInd1 (Customer Satisfaction).140.036Ind2 (Customer Collaboration).232.001Ind3 (Customer Commitment).253.001Ind4 (Decision Time).254.000Ind5 (Team Distribution).074.171Ind6 (Team Size).097.107Ind7 (Corporate Culture).241.001Ind8 (Planning).075.168Ind9 (Control).301.000Ind10 (Technical Competency).102.096Ind11 (Personal Characteristics).180.010Ind12 (Communication & Negotiation).104.092Ind13 (Societal Culture).381.000Ind14 (Training & Learning).283.000

*Revised Success Factors Framework

*Linear Multiple Regressionln (Success) = 0.940 +0.044(Ind 13) + 0.031(Ind 3) +0.020(Ind 9) +0.037(Ind 14)Ind 3 Customer CommitmentInd 9 ControlInd 13 Societal CultureInd 14 Training and Learning

*Linear Multiple Regression

PredictorsRegression Model1Ind13, Ind14D1 = 2.343 + .255 (Ind13) + .197 (Ind14) 2Ind14, Ind9, Ind2Log (D2) = .626 + .098 (Ind14) + .048 (Ind9) + .052 (Ind2)3Ind3, Ind7Log (D3) = 1.232 + .036 (Ind3) + .038 (Ind7)4Ind3D4 = 4.24 + .013 (Ind3) 5Ind4, Ind7, Ind3Log (D5) = .722 + .079 (Ind4) + .052 (Ind7) + .033 (Ind3)

Reduced delivery schedules. D1Increased return on investment (ROI). D2Increased ability to meet with the current customer requirements. D3Increased flexibility to meet with the changing customer requirements. D4Improved business processes. D5

Customer Collaboration Ind2Customer Commitment Ind3Decision Time Ind4Corporate culture Ind7Control Ind9Societal Culture Ind13Training and Learning Ind14

*Transitioning Traditional Software Project Management Practices into Agile

ManagementOrganizationalPersonnelProcess

ManagementOrganizationalPersonnelProcess

Process Changes

Plan-Driven(Native Attribute Types)

Agile(Native Attribute Types)

Changes Required

Challenges/Risks

*The changes required for adopting agile practices in plan-driven projects, and the associated challenges/risks

Changes in organizational culture- From policy and procedure based to freedom of development and management by team members.- From individually assigned roles to that of team-work.- From solitary development attitudes to working with different teams.- From no competency requirements of team composition to minimum competency requirements.- From non-customer-centric to customer-centric development.

Changes in Management Style- From command-and-control management to leadership-and-collaboration.- From authoritative to collaborative and pluralist decision making.

Changes in Knowledge Management Strategies- From heavy documentation-based to tacit knowledge management.

Changes in Development Processes- From heavily process-centric to short, iterative, test-driven, people-centric development.- From standards compliance and measurement driven development to development under uncertainty.- From contract-compliant to change-tolerant development.- From lifecycle-based development to feature-driven evolutionary and iterative development.

Developer resistanceDeveloper perceptions of micromanagementDeveloper perceptions of freedomDistributed developmentProductivity differences between team membersDecrease in productivity during transitionOverzealous teamsTester resistanceUpper management resistanceHuman Resources resistanceVariability in subsystems and teamsDifferences in lifecyclesProblems with incorporating agile in legacy systemsDifferences in development processesDifferences in performance measurements and benchmarksConformance with traditional process standardsDifferences in management attitude towards project successProblems with team-size scalability.Problems with selecting the right agile methodology.

Negative Effect

Changes Required

Challenges/Risks

*Data Analysis: ChangesDefinitions of Variables

RankVariablesNMeanStd. Deviation1C21654.351.012C11654.190.903C41654.190.934C31653.861.18

Changes in organizational culture (From Plan Driven to Agile) C1Changes in management style (From Plan Driven to Agile) C2Changes in knowledge management strategy (From Plan Driven to Agile) C3Changes in development processes (From Plan Driven to Agile) C4

*t-test results: Critical ChangesOne sample t-testAll change items are significantly different from 3 (Neutral)Pair-wise t-testC1 (changes in organizational culture) and C4 (changes in development processes) do not have statistically different mean valuesDiscussionMost critical change: Changes in management style Remaining significant changes: Change in organizational culture, Change in development process, and Changes in knowledge management strategies Similar level of importance: Change in organizational culture, and Change in development process

*Data Analysis: Changes

RankVariablesNMeanStd. DeviationC1 (Changes in organizational culture) 1C1.31654.560.932C1.51654.381.133C1.21654.251.084C1.11654.041.325C1.41653.741.38C2 (Changes in management style)1C2.11654.461.112C2.21644.231.11C3 (Changes in knowledge management strategy) 1C31653.861.18C4 (Changes in development processes) 1C4.11654.611.022C4.41654.391.053C4.31654.121.314C4.21643.641.36

*t-test results: Critical Sub-changesOne sample t-testAll passed t test: all sub-items are significantPair-wise t-testChanges in organizational culture: Following sub-changes are of equal importanceFrom non-customer centric to customer centricFrom individually assigned roles to that of team workRest all are ranked

*Data Analysis: Challenges

*t-test results: Critical ChallengesOne sample t-test11 challenge items are significantly different from 3 (Neutral)8 challenge items are not significantly different from 3Pair wise t-testOnly Chal1 (Resistance from developers) and Chal9 (Upper management resistance) have statistically different mean valuesRest all pairs do not have statistically different mean valuesDiscussionUpper management resistance is the most critical challengeRemaining challenges are similar

*Additional Success Factors, Changes, and ChallengesAdditional Success Factors: Learning from failureTiming issuesOther team characteristicsUse of toolsAdditional Changes:Changes in customer attitudeChanges in personal characteristicsAdditional Challenges:No significant challenge categories identified

*ConclusionsOpportunities: GlitteringChallenges: Several crucialNot a ready-made solution.Need a balanceIdentify the home-grounds where plan-driven and agile fit respectively.Concerns of importance:success factorschangeschallenges.Research in Progress

********************************