the test manager’s role in agile: balancing the old and the new
TRANSCRIPT
W13
Test Management
10/15/2014 3:00:00 PM
The Test Manager’s Role in
Agile: Balancing the Old and the
New
Presented by:
Mary Thorn
ChannelAdvisor
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Mary Thorn
ChannelAdvisor A QA director at ChannelAdvisor in Morrisville, NC, Mary Thorn has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than seventeen years of experience in healthcare, HR, financial, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She is a strong leader in agile testing methodologies and has direct experience leading teams through agile adoption and beyond.
The Test Manager’s Role in Agile
Balancing the Old and the New
By Mary Thorn
Director of QA at ChannelAdvisor in Morrisville, North Carolina,
A broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques.
During her more than fifteen years of experience in healthcare, HR, agriculture, and SaaS-based products,
Mary has held manager and contributor level positions in software development organizations.
She has a strong interest in agile testing methodologies and direct experience leading agile teams through Scrum adoption & beyond.
About Mary
Mission & Vision
3-Pillars Strategy
Converting Strategy to Roadmaps (Backlogs)
"Selling" the Roadmaps –
Metrics & Assessments
Agile Champion:
Continuous Improvement
Outline
Mission
The QA team members are functioning as an equal part of the overall Scrum Team
They are Subject Matter Experts (SMEs) in the area of the application.
They are technically competent.
QA members are accountable, committed, trust their teammates, driven to results, and don’t fear conflict.
They are empowered to be change agents.
Vision
3 Pillars
Development & Test Automation
Software Testing
Cross Functional Practices
What does success look like?
Roadmap
Transparency around areas where improvement is needed within QA, especially around areas of automation.
It helps define what success looks like for the QA personnel for the year, and it helps define individual objectives around this.
Defines the strategy discussed above.
It actually makes you look like you know what you are doing.
Roadmap
Your QA team(s) or Scrum team
You as a leader (your experience, instincts, etc.)
Your organizational directives
Retro
Roadmap – Who inputs to them?
What are examples of what would be in a roadmap?
Build automation framework
Automate X smoke test
Automate X regression test
Define/implement performance testing strategy
Train/implement BDD
Train/implement exploratory testing
Write manual regression test cases for x
Create QA Standard Operating Procedure document
Migrate away from Quality Center/ALM and save the company$100k/year(BEST roadmap item ever)
Roadmap cont.
I have one now what?
Once you have your roadmap, you need to work with your product owner organization to create stories, and get them prioritized and injected across your entire team’s product backlogs. Make sure that you put all of your work in a “business context”, explaining what it will do for “them” or the “customer”.
Another approach is to allocate a specific percentage of each backlog to this sort of work.
If you do not have a global agreement of some kind, you have to allocate your work to the product backlogs on a situation-by-situation basis.
I have one now what?
Roadmap for Sale
Cross Cutting Strategy
The team has to get it done, do it incrementally, measure it, and make adjustments along the way.
QA Manager should collaborate with your team in the execution phase
This gives them ownership and accountability, and creates trust. Without team buy-in and the QA Manager’s leadership of the strategy, it is no more than writing on a piece of paper.
Strategy - Implementation
Measures something only if you plan to act on the results
Measures only what can be measured
Measures at the correct level and in the correct units
Is easy to collect
What makes a Metric “good”?
All the information they need to make decisions, and no more
Information at the level of detail they can use
Information at the scope they care about
Information pertaining to the time frame they care about
What do your Stakeholders want?
Name
Question being Answered
Basis of Measurement
Level & Usage
Expected Trend
When to Use It
When to Stop Using It
How to Game It
Warnings
Metrics Checklist
Defects – Sev 1/Sev trend
0
10
20
30
40
50
60
Total Sev 1 &2 per month hotrolled
Total Sev 1 &2 per month
Bug Score by Team
Unit Tests– a test that has no dependencies (do our objects do the right thing, are they convenient to work with?) Integration Tests– a test that has only one dependency and tests one interaction (usually, does our code work against code that we can’t change?) UI Tests– a test that tests the UI and work flow.
Unit = 12830
(10% Increase)
Int = 1071
(2% increase)
UI= 671
(5% change)
Automation Pyramid
Forrester Agile Testing Tool
Forrester Agile Testing Tool
0.00
1.00
2.00
3.00
4.00
5.00
Testing behavior
Agile testing organization
Agile testing practices Automation
Agile adoption
Agile Assessment Tool
Agile Assessment Tool
0%
25%
50%
75%
100%
Product Health
Release Health
Sprint Health
Team Health
Debt and Done Health
Product Quality
Affirm and reinforce lean / agile practices
Measure results, not output
Follow trends not numbers
Belong to a small set of metrics / diagnostics
Provide fuel for meaningful conversation
Diagnose & improve the processes that produce business value
Provide feedback on a frequent and regular basis
Are easy to collect
Good Agile Metrics
Agile Champion
Doneness/Policies
Removing Impediments
• Test Data Strategy
• Environments Strategy
• Continuous Integration Strategy
• Defect Cleanup Strategy
Removing Impediments
• Discovery Retro
• Quarterly QA Retro
• Technical Debt Retro
• Post Prod Sev 1 or 2 Retro
Retrospectives
Servant leader shares power, puts the needs of others first and helps people develop and perform as highly as possible.
Servant Leadership
Project Management – Scrum Masters
Architecture and Business Analysis
Development
Testing
UX Design
Product Owners
DevOps or Technical Operational Team(s)
Documentation
Customer Facing Operational Team(s)
Relationship building
Hiring
Hiring
First Who, Then What: Get the right people on the bus, then figure out where to go. Finding the right people and trying them out in different positions.
Hire by traits in order of:
1. Attitude
2. Aptitude
3. Skillset
NO ROCKSTARS NO HEROES
Hiring
People Management
Trainings – All 3 Pillars
Automation
Testing Practices
Team(IE Five Dysfunctions)
Agile
10% time
Management and Leadership
Continuous Improvement
Wrapping it Up