agile testing introduction
TRANSCRIPT
Agenda
Agile Recap
Agile Testing Overview
Agile Testing Game
Agile QA Common Approach
Test Strategy in Agile Environment
Advanced Problems For Discussion
Summary Activity
IMT Confidential 2
Agile Recap
IMT Confidential 3
Agile Recap
IMT Confidential 4
Testing terminology – Unit testing
• Automatic test on code level run every night or after new code is added – to verify no bugs
introduced
– Integration testing
• Testing interactions between different modules of the system
– Feature testing • Performed by a QA when a feature is finished implemented
– System testing
• To verify whether the system as a whole meets the specified requirements
– ExploratoryTesting • Exploratory testing is simultaneous learning, test design, and test execution.
– Regression Testing
• Test to detect side effects from changes to the system, i.e after bug fix
– Retesting
• Verification of bug fix
– Compatibility testing • Verification of compatibility between AMX applications and the different OS, office applications, browsers
and databases etc that we support
5 thinking levels of Tester. -----------------------------------
You don't believe it until you see it. You don't believe everything that you see.
Agile Testing Overview
• Agile testing is a software testing practice that follows the principles of agile software development and involves all members of a cross-functional agile team.
• It is a set of activities intended to ensure that products satisfy customer requirements in a systematic, reliable fashion.
• In SCRUM (agile) QA is the responsibility of everyone in R&D, not only the Test team. QA is all the activities we do to ensure correct quality during development of new products.
IMT Confidential 6
Agile Testing Overview
• Agile testing focuses
on testing iteratively,
as often as stable
code base is
available, until quality
is achieved from end
customer’s
perspective.
IMT Confidential 7
Agile Testing Overview
11/19/2015
IMT Confidential 8
Agile Testing Overview
Agile Testing Traditional SQA
Agile Testers are integrated, full-fledged
members of dev team and participate in
planning, estimating and all team
activities.
Testing is performed by a separate
organization or group reporting to SQA
Manager.
Agile testers work hand-in-hand with
development and product manager.
SQA teams may work with other groups
but with limited interaction and well-
establishment boundaries.
Business requirements are written a
piece at a time to accommodate
changing business needs.
SQA receive detailed business
requirements and schedules up front.
Throughout the release additional test
scenarios are uncovered through
exploratory testing.
When development is presumed
complete, the application is delivered to
SQA for testing
IMT Confidential 9
Agile Testing Overview Agile Testing Traditional SQA
Agile testing is continuous and
feedback is provided during all stages
of development.
SQA executes a complete cycle of test
and reports defects back to
development.
Agile testing follows a fluid, continuous
process with defects fixed as they are
found.
When testing is complete,
development fixes defects and
delivers another release to SQA
Agile tester are expected to radiate
information and provide complete
visibility into all test-related activities for
the benefit of the larger group.
Traditional SQA teams often keep test
design, implementation and progress
specifics within the SQA site.
Quality software is always ready to
be delivered.
The process of test and fix repeats
until time runs out.
IMT Confidential 10
Agile Testing Overview
IMT Confidential 11
• Agile Testing Mindset
Agile Testing Overview
IMT Confidential 12
• Exploratory Testing in Agile Environment
Agile Testing Overview
IMT Confidential 13
• Exploratory Testing in Agile Environment
Factor Scripted Exploratory
Domain Knowledge Lack of domain knowledge
can be overcome during test
design
Requiring sufficient domain
knowledge
System Complexity Allows for careful design of
test scripts
Relies on skill set of tester
Level of Doc Require good doc to be
supplied
Just assumes good domain
knowledge
Timeframes and Dealines Significant lead-in time
required
Little or no lead-in time
required
Efficiency
? ?
Coverage ? ?
Verification ? ?
Acceptable Risk Level ? ?
Reproducticibility ? ?
Agile Testing Overview
IMT Confidential 14
• Exploratory Testing in Agile Environment
Agile Testing Overview
• Challenges in Agile Testing – Short Iterations & Releases
• 1-week, 2-week, 3-4 week release cycles, even shorter feedback cycles
• Validate new “user stories”, ensure old functionality continues to work, communicate bugs, fix
them and validate again.
– Less time for documentation
• Need tools/formats for documentation that cover essence of the test but are lightweight
– Dealing with ‘Slacker’ and ‘Asker’
– Is the number of defects reported a good metric to measure testing
effectiveness?
– Is automation required only when regression testing is needed?
– How to manage Time zone difference?
– How to cope and deal with changes of requirement
IMT Confidential 15
Agile QA Game
IMT Confidential 16
Agile Testing Game
• Building IMT Agile Testing Manifesto
IMT Confidential 17
Candidate Final
Duplicate Not Clear
Agile QA Common Approach
IMT Confidential 18
Agile QA Common Approach
Responsible for Test Preparation
Take part in requirement elaboration
Attend Sprint planning meeting
Collaborate well with Product Owner to clarify user
stories/PBI and the concept of tests
Prioritize all tasks
Estimate time for testing features/backlogs
Identify and mitigate risks
19
Agile QA Common Approach
Responsible for Test Execution
Attend Daily Stand up Meeting
Design Test cases
Automate acceptance test cases wherever possible
Perform regression, exploratory & acceptance testing
Write/update & execute acceptance criteria
Identify the gap between what the customer expects
and what the team delivers
20
Agile QA Common Approach
Responsible for Test Evaluation Be part of Sprint review & Retrospective meeting
Responsible for evaluating test deliverables
Provide continuous feedback to development team
21
Agile Quality Assurance Strategy
• Test strategy in agile development
IMT Confidential 22
Test Strategy in agile
• We start our QA activities at Release planning stage
• Release planning – Create test plans
• High level test planning
• budgeting enough time
– Participate in story sizing
– Typical Test plan will include • Scope of testing
• New functionalities which are being tested
• Types/ level of testing based on the complexity of the features being tested
• Performance and load testing
• Infrastructure consideration (Test environment/software)
• Risks/ mitigation plans
• Resourcing
• Milestones and deliverables
Test Strategy in agile…
• Each Sprint – Participate in sprint planning
– Estimate tasks(QA input)
– Write test cases using stories(QA)
• High level test cases before coding begins- guide dev
• Detailed test cases written during coding
– Unit/ Integration tests(Dev) • Automatic test on code level run every night or after new code is added. Bugs found are fixed immediately
– Pair tests(Dev + QA) • Performed when a part of a feature is finished implemented. Bugs found are fixed immediately
– Feature tests(QA) • Performed when a feature is finished implemented. Errors found are logged in main project in bug system
and prioritized during next sprint planning. After all prioritized bugs are fixed, the feature is ready for System
Test at the end of the project
– Reviews (stories, req, test cases with customer, programmer)
• Increase collaboration and communications
Test Strategy in agile…
• Each Sprint contd…
– Test automation
• Automate new functional test cases
• Run automated regression tests
– Perform non functional testing (load, security, usability etc)
– Demo to the stakeholders
Test Strategy in agile…
• Hardening sprint
– A hardening sprint can be used for bug fixes in previous sprints. Bugs that are prioritized will be considered here
– The end game(System test)
Every member in the team is involved in this.
• Mock deploy on staging
• Smoke test on staging
• Perform non functional testing (load, security, usability etc)
• Complete regression test
• Perform UAT tests
• Other tests
– Compatibility(DB upgrade etc)
– Installation
– Localization etc
Test Strategy in agile…
• Release – Define Exit criteria
• Testing activities (bug severity, open bugs, coverage levels, metrics
etc)
• Product Artifacts (user guide, installation guides, other necessary
documentation are in place)
– Participate in retrospectives
• Start
• Stop
• Continue
Iteration metrics
To enhance predictability…
– Measuring progress
• Burn down charts
• Estimated vs. Actual time for tasks
– Defect metrics
• Root cause analysis
• Results from defect tracking system
– Traceability metrics
• Test coverage (story vs. test cases)
• Code coverage
Advanced Problems
• Agile Testing Interview Questions
– What is Agile Testing and how is it different to traditional
waterfall or the V model?
– What is your approach when requirements change
continuously?
– What are good characteristics of an Agile Tester / QA?
– What are the two key factors when working as a QA in an
Agile team?
– What are the three main roles in Scrum?
IMT Confidential 29
Advanced Problems For Discussion
IMT Confidential 30
Advanced Problems For Discussion
• Impact Analysis
IMT Confidential 31
Advanced Problems For Discussion
• IMT Projects lesson learnt
– Xfinity Home Security
– Precyse
– Precyse Automation
– TabSquare
– …………………………………
IMT Confidential 32
Summary Activity
• Using mind map to draw the map like below
to summary your knowlegde of Agile Testing
IMT Confidential 33
Summary Activity • An example of mind map about Agile Testing Day
IMT Confidential 34
Conclusion By Image
11/19/2015
IMT Confidential 35
Main References
IMT Confidential 36
http://www.istqb.org/
An Le, Tester Mindset
in Agile Scrum Model
http://jp.imt-soft.com/Blogs/Agile-Scrum/Tester-Mindset-in-
Agile-Scrum-Model
Main References
IMT Confidential 37
Nguyen Vu Hung, Tran Son Hai and et al, Tài
liệu Hướng dẫn Scrum dịch từ bản gốc được
cung cấp bởi Ken Schwaber và Jeff Sutherland,
Main References
IMT Confidential 38
Thank You
IMT Confidential 39
Questions
IMT Confidential 40
• Lazada Story Question:
• Recap Question: List the
pros and cons of Agile
Testing in Outsource
Project?