agile testing practice
TRANSCRIPT
Test manager, Program and project
manager
11+ years’ delivery experience
6+ years’ agile experienceMary Jiang
姜 融
About you?
Managers, testers, ScrumMasters…?
How long has your team used agile
principles/practices?
Which kind of testing your team are
working on?
Group Discussion (10 minutes)
What is your biggest concern regarding agile testing?
Or
What interests you most about agile testing?
(Image provided by Wikimedia, reproduced here
under a Creative Commons license.)
Agile context –Manifesto
Individuals and interactionsover processes and tools
Responding to changeover following a plan
Customer collaborationover contract negotiation
Working software
over comprehensive documentation
What is agile testing?
A software testing practice that follows the principles of agile software development
Involves all members of a cross-functional agile team, with special expertise contributed by testers
Testing from the customer perspective as early as possible
Includes everything beyond unit and component level testing
Does not just mean testing on agile project
What does agile testing mean?
Challenge
Continuous Change
Less documentation
Keep everyone on the same page
Face to customer
More testing effort
Opportunity
Everyone owns quality
Earn business value earlier
Break the boundaries
Involve the customer
Quick feedback
Whole team approach
Scrum Team commitment to quality
Scrum Team responsible for testing
Product owner responsible for acceptance testing
Development team responsible for all level’s testing
Agile testing mind-set
Don’t be afraid
Enjoy learning new skills and taking on new challenges
Not limited in the areas you can test
No judgment
Results-oriented
Collaborative
Passionate
Ten principles for agile testers
Provide continuous feedback
Deliver value to the customer
Enable “face-to-face” communication
Have courage
Keep it simple
Practice continuous improvement
Respond to change
Self-organize
Focus on people
Enjoy
-Lisa Crispin and Janet Gregory
Change doesn’t come easy
Be patient
Let them feel pain
Build your credibility
Work on your own professional development
Beware of the quality police mentality
Typical activities
for agile testing
• PROJECT INITIATION AND
PRE-PLANNING
• SPRINT N-1 AND SPRINT N
• CLEAN UP AND DELIVERY
Testing activities in project initiation
and pre-planning
Definition of “Done”
Release planning
Test planning
Preparation for visibility
Team commitment
“
”
Nothing is done until it is tested
FAMOUS SAYING IN AGILE
Definition of “Done”
Acceptance criteria
Agile tester through release planning
A big picture view
Sizing stories from the testing perspective
Testing considerations while prioritizing
What is in scope?
Sample big picture
Web Svc
SIEBEL SVCEIM LOAD CTI
UCM
REMEDY TOOLS
IVR
ESP
Portal
Siebel
UI
Alarm Etc
Web Interaction
Agent Ticket Application
Phone
Phone
W/S
Test
Perf
Load
Tech
UCM
Ubiquity
Portal
Web Interaction
GDC
US
Lightweight test plans
Lightweight documents
Test matrix
Test spreadsheet
A white board
Online tools
Preparation for visibility
Tracking test tasks and status
Communicating testing results/bugs
Release metrics
Test status
Traceability/coverage
Defect status
Consideration of Bugs
Is it a bug or is it a new feature?
Use a Defect Tracking System?
Which bugs need to be logged?
When to fix bugs?
Rules for dealing with bugs
Team commitment
Definition of “Done”
Ensure enough involvement
Communication channels
Agreement for visibility
Collaboration in a distribution environment
Testing activities in Sprint N - 1 and
Sprint
Sprint planning
Driving Requirement
Driving development
Collaborate with developers
Facilitate communication
Completing testing tasks
Regression Tests
Sprint Metrics
Demo
Retrospective
What will testers do during Sprint Planning?
Learn more details
Considering all viewpoints
Think about testability
Write task cards for testing
Ensure testing effort is considered in estimation
Driving requirement
High level tests and examples
Flowchart diagram
Test/verification points
Test driven requirements
Q&A with customer/representative
High level tests review
Sample for high level flowchart
Places BidPurchases
Item
Creates
Account
Searches
listings for
item
Creates an
AuctionShips Item
Buyer
Seller
User
Q&A and review
Collaboration tools to track Q&A and review
comments
Online and offline review with
customer/business representative
Driving Development
Start Simple
Add complexity
Risk analysis
Coding and testing progress together
Focus on one story at a time
Facilitate communication
Daily Stand-up
Bridge the gap between business and
development
Collaboration tools for distributed teams
Understand and sacrifice a little time
Completing Testing Tasks
Proactive - test as early as possible
Address the testing crunch
Anyone can do testing
Non-functional testing
Regression Tests
Automation will be preferred
Keep the build “Green”
Keep the build quick
Building a Regression Suite
Checking the “Big Picture”
Sprint metrics
Measuring progress Burn Down Chart
Is testing done?
Coverage Unit Test
Code
Test coverage
Defect Status
Samples for metrics –
Traceability/Coverage
CategoryPriority
TC Covered Defects TC NON-Covered Defects Total
# % # %
4-Business Critical 25 60.98% 16 39.02% 41
3-High 34 58.62% 24 41.38% 58
2-Intermediate 28 62.22% 17 37.78% 45
1-Low 19 76.00% 6 24.00% 25
Total 106 62.72% 63 37.28% 169
Samples for metrics – Defect Status
0
50
100
150
200
250
300
350
400
450
500
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Total Bugs
Total Fixed Bugs Ignore Reopen Situation(Forcast)
Incremeantal Won't Fixed Bugs
Total Fixed Bugs With Reopen Situation(Forcast)
Sprint Review (demo)
Conduct the demo
Listen to the customers
Ask questions if possible
Do not forget the “big picture”
Sprint Retrospective
Start, stop, continue
Identify test-related obstacles
Focus on a few action items for the next sprint
Clean Up and Delivery
Planning enough time for testing
Testing release candidate
Testing on a staging environment
Final non-functional testing
Integration with external applications
Product data conversion and load
Installation testing
Communication
Support customer testing
System Testing
Planning in advance
Test environment(QA/testing and staging)
Final testing including non-functional testing
Cooperation with third-party teams
ETL/data conversion
Communication constantly
Installation testing
Defect trend
What if it’s not ready?
Was the build broken in the last day?
Any show stopper bugs not detected until final
functional testing?
Any new non-functional issues found?
Any test data problems?
Integration issue?
User Acceptance Testing
High level plan and scope
Support and collaborate with customer
Get feedback from customer
Understand how customers view products
Know yourself
Change your testing thinking
Values are more important than process and
practice
Existing testing skills are useful
Build you own credibility
Agile Testing Quadrants
Functional Tests
Examples
Story Tests
Prototype
Simulations
Exploratory Testing
Scenarios
Usability testing
User Acceptance Testing
Alpha/Beta
Unit Tests
Component Tests
Performance & Load testing
Security Testing
“ility” testing
Application under test
Business View
Technology View
Su
pp
ort
ing
th
e t
ea
m Critiq
ue
pro
du
ct
Business supporting test
Support Team
Right tools to elicit requirements
From big picture to details
Driving development with business – facing testing
Critique product
Demonstrate applications to stakeholders to get early feedback
Use scenarios and workflow to test from end to end
Business supporting testers’ role
As a bridge between business and development
Required skills and capability
Domain knowledge
Solid testing skills and technical skills as supplement
Critical and logic thinking
Soft skills
Personalities
Automation skills
Technical Supporting Test
Support Team
Unit tests
Component Tests
Continuous build
Critique product
Performance/load testing
Security testing
“ility” testing
Technical supporting testers’ role
Driving development and non-functional needs
Required skills and capability
Solid testing and technical skills
Architecture level understanding as preferred
Developing and Programming skills
Requirement and results analysis skills