agile load testing in the real world
TRANSCRIPT
1Webinar
Presentswith
Agile Load Testing in the Real World
2
Continuous Load Testing Realized
TODAY’S PRESENTERS
Lee Barnes: Utopia Solutions, Founder & CTOMike Ostenberg: SOASTA, Dir. Systems EngineeringBrad Johnson: SOASTA, VP Product
Agenda• A case study in driving a performance culture with agile delivery• A platform for delivering continuous load testing
Questions: Submit in chat box during event
3
© Utopia Solutions
Agile Load Testing in the Real World
Lee Barnes, CTOUtopia Solutions@USI_LeeBarnes
© Utopia Solutions5
Key Questions
What needs to change?
How do we make those changes?
Why don’t traditional performance testing models fit nicely with agile?
© Utopia Solutions6
Case Study
Background
• Large global manufacturer• Performance Testing CoE in operation for ~10 yrs• Seen as a huge value to the enterprise• Began moving toward agile a few years ago…
and the luster of the CoE began to dull…
© Utopia Solutions7
Traditional Performance Testing Model
Provides Consistency in…•Resources / Skills•Processes•Deliverable Quality•Technology
Request Test
ProjectTeam
Center ofExcellence
Perform Discovery
Develop & Execute
Deliver ResultsEvaluate Results
Info / Environment
© Utopia Solutions8
• One size fits all• Strict entry criteria• Performed late in SDLC• One-way communication of
results
Service Characteristics
CoE in the Waterfall Years
Performance Testing
Too late!
© Utopia Solutions9
Agile Development
Deploy
Where does performance testing fit?
© Utopia Solutions10
Early Attempts on Agile Projects
Agile
Performance Testing
Same Process…
Same Risk!
© Utopia Solutions11
Early Attempts on Agile Projects
Agile
PerformanceTesting
PerformanceTesting
PerformanceTesting
PerformanceTesting
CoE Approach•Rigid•4 week test cycle
Agile Approach•Flexible•2 week sprints
+ =
© Utopia Solutions
Back to the drawing board…
© Utopia Solutions13
Early Cycle / Ongoing
Performance Analysis
Establish Objectives
Ultimate Goal…
Late Cycle Performance
Testing Shift Left
© Utopia Solutions14
Identify Gaps
What’s stopping us from analyzing performance continuously?
• Performance objectives weren’t part of user stories
• Agile teams lacked performance engineering expertise
• Performance testing knowledge and tools were locked in the CoE
• APM tools / skills were non-existent
© Utopia Solutions15
Disclaimers
It was not easy.
The changes discussed on the next few slides were implemented over a two year duration.
It was not painless.
© Utopia Solutions16
Performance Testing in an Agile Environment
Build performance objectives in to user stories
Make performance everyone’s job
Create a role to coordinate performance activities
CoE performance test architects become mentors
Begin developing a performance focused culture
© Utopia Solutions17
Performance Testing in an Agile Environment
Map business and end user requirements onto system components
Create both single and multi user tests
Benchmark in development / QA environment
Look for trends in measurements vs. absolutes
Use service virtualization to stub out 3rd party or yet to be developed components
Execute unit performance tests as part of continuous integration
Implement Unit / Component Performance Testing
© Utopia Solutions18
Waterfall Agile
Responsibility for Performance
Performance CoE Everyone
Role of CoE End of cycle PT Provide mentorship to agile teamPre-deployment system PT
Role of development
Throw system over wallWait for results
Continuous focus on performance
Tools Traditional load test tools Traditional load test toolsAPM toolsService virtualizationContinuous integration
Environment Dedicated / production-like Development / desktop
Deliverable Boiler plate results delivered just prior to release
Continuous feed of actionable data
Performance Testing in an Agile Environment
Transformation SummaryWaterfall Agile
Responsibility for Performance
Performance CoE
Role of CoE End of cycle PT
Role of development
Throw system over wallWait for results
Tools Traditional load test tools
Environment Dedicated / production-like
Deliverable Boiler plate results delivered just prior to release
© Utopia Solutions19
Summary
Key takeaways…•Agile performance testing is not more / quicker performance tests•Success requires an shift in people, process and technology
20
Mike Ostenberg
21
Direct Questions?Lee BarnesFounder and CTOEmail: [email protected]
twitter.com/USI_LeeBarnes
linkedin.com/in/leebarnes
Blogwww.RaiseYourMQ.com
Thanks & Q&A
Contact SOASTA:www.soasta.com/cloudtest/[email protected] us:
twitter.com/cloudtesttwitter.com/cloudtest
facebook.com/cloudtestfacebook.com/cloudtest
Knowledge Center•White Papers•Webinar Recordings•Case Studies
CloudLink Community•Support•Tutorials•Video
RESOURCES
Contact Utopiawww.utopiasolutions.com630.375.6610
www.SOASTA.com www.UtopiaSolutions.com
22
www.SOASTA.com