tuning our agile process using systems engineering mike day [email protected] director of...
TRANSCRIPT
![Page 1: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/1.jpg)
Tuning our Agile ProcessUsing Systems Engineering
Mike Day
Director of Software Quality
January 27, 2011
![Page 2: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/2.jpg)
Who Am I…
BackgroundUndergraduate Degrees in Business Administration, Finance, Economics
Industries include Manufacturing, Non-Profit, Health Care, Telco
Masters Degree in Information Systems
Industries include Telco, Open Source, Enterprise and SaaS development
Practicing Agile for 10+ yearsConsulting/Agile Coaching - late 90s/early 2000s
Ping Identity Agile Quality Team development
PassionsSystems Dynamics, Data Mining and Modeling, and the Growth and Leadership of Highly Performant Teams
![Page 3: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/3.jpg)
Who is Ping Identity
THE leader in Secure Internet Single Sign-OnPingFederate – On-Premise, Enterprise SSO solution
PingConnect – On-Demand SSO solution for SaaS and the Cloud
~ 600 Customers Worldwide – growing at 1+ per day
> 40% of Fortune 100 Companies
100+ Global Partners
Founded in 2002 – Privately OwnedStrong Cash position – Debt Free
Consistently Meeting or Exceeding Financial Goals
Headquartered in Denver
Offices in Boston, Istanbul, London, Moscow & Vancouver
Worldwide sales and services
![Page 4: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/4.jpg)
Define:Systems Engineering
Systems Engineering an interdisciplinary field of engineering that focuses on how complex engineering projects should be designed and managed over the life cycle of the project. Issues such as logistics, the coordination of different teams, and automatic control of machinery become more difficult when dealing with large, complex projects. Systems engineering deals with work-processes and tools to handle such projects, and it overlaps with both technical and human-centered disciplines such as control engineering, industrial engineering, organizational studies, and project management.
System Dynamics an approach to understanding the behaviour of complex systems over time. It deals with internal feedback loops and time delays that affect the behaviour of the entire system
![Page 5: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/5.jpg)
Agile @ Ping
Plan
Releases
Product
Owners
ReleasesIterateIterate
Customers
Define
System
Product
Planning
V2
V1
?Accept? ?
Inception Inception Transition Transition
IterateIterateIterateIterate IterateIterate IterateIterate
FeaturesFeatures FeaturesFeatures FeaturesFeatures FeaturesFeatures
IterateIterate
IterateIterate
IterateIterate
IterateIterate
Hardened FeaturesHardened FeaturesIntegrated ProductIntegrated ProductRolloutRollout
• Business analysis thinking-throughoutBusiness analysis thinking-throughout• Requirements Elaboration throughoutRequirements Elaboration throughout• Requirements Matrix-built progressivelyRequirements Matrix-built progressively• Supplemental SpecificationSupplemental Specification• In-Flight Agile Dashboard drives feature priority throughoutIn-Flight Agile Dashboard drives feature priority throughout
Elaboration and ConstructionElaboration and Construction
QueueQueue
![Page 6: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/6.jpg)
Agile @ Ping
• 2006: 26 Releases
• 2007: 28 Releases
• 2008: 22 Releases
• 2009: 40 Releases
• 2010: 39 Releases
• & Low Field Defects (94% DDP)
• Agile From Initial Dev Team Staffing 7 Years • Distributed Agile: Teams/Individuals in 4 Countries• Fixed Timeboxes < 3% slips ~12 days/7 years
FlywheelExpectations
![Page 7: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/7.jpg)
Agile Pain Points… Throughout the Years
Management and MitigationQuality DebtRisk
Team Development and DynamicsVelocityDelivery to a time-boxed cadenceCommitment to Quality
Systems Engineering
![Page 8: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/8.jpg)
Quality Debt Management
Key Drivers for AnalysisDelays in Addressing DefectsFeature Development vs. Defect CorrectionTeams are DifferentMinimum Quality Criteria
Data Mining and Modeling Addressing ConcernsEngineering DashboardsQuality Debt PaintballReliability Growth ModelingDefect Lifetime AnalysisDefect Detection Percentage
![Page 9: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/9.jpg)
Engineering Dashboard
![Page 10: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/10.jpg)
Quality Debt Paintball
![Page 11: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/11.jpg)
Reliability Growth Modeling
![Page 12: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/12.jpg)
Defect Lifetime Analysis
82%
![Page 13: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/13.jpg)
Defect Detection Percentage
162 Total Field Escapes since July 2007
![Page 14: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/14.jpg)
Risk Management and Mitigation
Key Drivers for AnalysisExtremely Feature Rich Product OfferingExecution of More with LessTeams are DifferentMinimum Quality Criteria
Data Mining and Modeling Addressing ConcernsUnderstanding the Product LongtailRisk Mitigation ModelingCode Change TreeMaps
![Page 15: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/15.jpg)
Understanding the Product Long tail
![Page 16: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/16.jpg)
Risk Mitigation Modeling
Risk Calculcation Table
Ping FactorsCustomer Risk
(Impact of Failure)Quality Risk
(Probability of Failure)Complexity Dev
Whitneys / Wus'
Complexity SQE
Whitneys% Testing Complete
Weighted Risk
RemainingUsage
FrequencyBusiness Criticality
Defect Prone Areas
Changed Areas
Code Complexity
TOTAL RISK
Weights Weights 10 3 5 10 3
Info Only Not Used in Calculations
Used in Risk Calucation as Multiplier
Based on Matrix Completion
SQE Complexity / % Testing Remaining
Scoring to be provided by TPM
Scoring to be provided by TPM
Analysis of Prior Defect History and Defects from the Field
Perforce Change List analysis, TreeMap and Structure 101
Cyclomatic Complexity and Unit Test Code Coverage Statistics
Customer Risk
Quality Risk
Features / Risks Addressed Reference ID IDP and SP Adapter 3 2 0% 200% 3 3 5 2 2 39 51 3978Reference ID Attribute Serialization using JSON 3 6 10% 540% 1 3 1 5 2 19 61 6258.6Reference ID Implement Client Cert Auth 4 3 0% 300% 1 1 1 2 1 13 28 1092Reference ID Modify SP Adapter Advanced Configuration 3 3 0% 300% 1 1 1 2 1 13 28 1092Reference ID Pass authentication policy to the Idp application via query/post rather than REF 3 3 0% 300% 3 3 3 3 3 39 54 6318Reference ID Sample Code 3 0Support Sharepoint 2010 3 3 0% 300% 5 5 1 2 1 65 28 5460Sharepoint 2010 Interop 3 0Core Provisioning Logging to DB 3 1 0% 100% 3 1 1 1 1 33 18 594Configurable XML Signature Algorithm 3 1 0% 100% 3 1 1 1 1 33 18 594Google Package Google OpenID Idp Integration Kit 3 7 0% 700% 5 5 4 4 1 65 63 28665SFDC Customer Portal Connector 3 3 0% 300% 1 1 1 2 1 13 28 1092SFDC Partner Portal Connector 3 3 0% 300% 1 1 1 1 1 13 18 702SFDC CRM Connector 3 3 0% 300% 1 1 1 2 1 13 28 1092SFDC IdP Adapter 3 3 0% 300% 3 3 1 3 1 39 38 4446SFDC Support CRM SAML 3 4 0% 400% 1 1 1 2 1 13 28 1456SFDC Verify Idp Adapter with Protals 3 2 0% 200% 1 1 1 2 1 13 28 728Provisioning SDK 4 5 0% 500% 2 2 3 2 2 26 41 5330Social Networking Adapter 3 1 0% 100% 3 3 1 1 1 39 18 702
![Page 17: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/17.jpg)
Risk Mitigation Modeling (Cont.)
![Page 18: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/18.jpg)
Code Change TreeMaps
![Page 19: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/19.jpg)
Understanding and Improving Team Velocity
Key Drivers for AnalysisEnterprise Software Development != SaaS DevelopmentExecuting more FrequentlyTeams are DifferentMinimum Quality Criteria
Data Mining and Modeling Addressing ConcernsEngineering DashboardComplexity Point Modeling
![Page 20: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/20.jpg)
Engineering Dashboard
![Page 21: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/21.jpg)
Complexity Point Modeling
![Page 22: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/22.jpg)
Complexity Point Modeling
![Page 23: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/23.jpg)
What’s Next… Systems Engineering @ Ping
Analysis of Adoption of Product OfferingsFrequencyDevelopment to First Adoption Temporal MapUnderstanding Cost/BenefitModeling ROIDefining Level of Effort on Incremental Work
Test Asset ManagementConsolidation and Reporting of all Automated TestsBringing All Tests under Single Management UmbrellaMonitoring Test Effectiveness and Coverage
![Page 24: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/24.jpg)
Wrap it Up…
![Page 25: Tuning our Agile Process Using Systems Engineering Mike Day mday@pingidentity.com Director of Software Quality January 27, 2011](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e055503460f94af1d7a/html5/thumbnails/25.jpg)