Session 7:The Lone Tester in an Agile World

Michael Larsen

Michael Larsen,Senior Tester

Senior Tester, Sidereel.com@mkltesthead

Producer & Commentator: TWiST PodcastSpeaker: PNSQC, CAST, STAREASTFacilitator: Weekend Testing AmericasContributor: How to Reduce the Cost of Software TestingBlog: TESTHEAD (http://mkltesthead.com)Articles: ST/QA, StickyMinds, The Testing Planet, Tea Time with Testers

About Me (cont.)

– 18 Years Software Testing Experience– Mostly with Traditional Development

Approaches– Usually Sole Tester for Projects or Companies– Made the switch to Agile development in

January, 2011– Close enough to the Pain to Remember It!

Traditional Test WorkflowReview requirements document

Review a software specification

Write out test cases based on software spec

Create detailed scripted test procedures

Wait for a build to arrive so you can test

Execute your scripted tests

Re-run tests that were added to cover bugs that were found in previous builds

Determine after enough testing or the specific release date if software is ready to ship

The Lone Tester in the Traditional World

– Finding issues in requirements and design documents

– Verifying/validating software meets the spec

– Running tests at given milestone points to see if the product break

Problems With Traditional Testing Approach

– Requirements are never complete – Spec often misses the mark– Product "meets the spec”, but isn't really what the

customer wants– Separate development and test cycles– Software thrown "over the wall" to be tested and

then thrown back to be reworked– Waiting for the product to be viable again to test

Is There Another Way?!

What is Agile?

– Build system in smaller chunks

– Business requirements discussed along with product development

– Build what is actually wanted

– Develop and test smaller bits of functionality more frequently

– Delivering product that meets real business needs

The Agile Manifesto

The Agile Approach• Small slices (Stories)

• Testing from the beginning of the project

• The whole team responsible for quality

• Test Driven Development used to help design code

• Tests fail first, build code to pass tests, refactor code to be more efficient, integrate with other code

• Continuous build and integration each time code is checked in

Is Your Team "agile"?

– is the team using a test-driven approach to development?

– are stakeholders actively involved in the development process?

– does the team produce high-quality software at regular intervals?

– is the team highly collaborative and self-organizing?

– is the team actively improving the process of how they work?

A Typical Story

– Developed in coordination with the product owner

– Develop the story's acceptance criteria along with it

Test Driven Development

– Developers often working in paired teams• Focus on Red,

Green, Refactor• Repeat the process

if necessary• When the story has

reached the state of DONE, move on to the next story

TDD is Not Testing

– ???

– TDD is a design process for development

– Helps to cover regression tests and unit tests on all code segments

– No build goes out that is not "all green" (all unit and regression tests pass)

The Whole Team is Responsible for Quality

– TDD

– Acceptance testing

– Automated testing throughout the entire cycle.

– Provide continuous feedback to the development team

???

Lots of "Testing"...

But where are the "Testers"?

Are Testers Needed?

– Some view Agile as a "post tester world”

– Testing is done by development, product owners, and customers

– Is there a need for dedicated testers?

Testers Needed and Valued

– Testing is integrated at all levels

– Quality is seen as a team effort

– Traditional Testing role (Quality Police) not required

– Testing are being done at many different levels

– Model of dedicated tester doing all the testing has changed

Agile Testing Requires Variety of Skills

– Domain knowledge about the system under test

– Technical competency to interact effectively with development team

Testing in Thin Slices

– Testing focused on if story meets customer requirements

– Exploration around stories to guide and provide feedback for team

– Automating acceptance tests to run in the future along with new features being developed

“Agile Testing is…”• According to Bret Pettichord:

• “Headlights of the project – where are you now? Where are you headed?”

• “Provide information to the team – allowing the team to make informed decisions”

• “Find and inform team of bugs”– A “bug” is anything that could bug a user –

testers don’t make the final call

• “Testers do not "assure" quality – the team does (or doesn’t)”

• “Testing not a game of “gotcha” – set goals, rather then focus on mistakes”

Challenges for Agile Testers

– Requirements and Acceptance Criteria specific to single story

– Testing as early as possible– Acceptance Tests developed before the software is

developed– Automated tests run against code every time a

build is performed– Regression testing requirements significantly

increased

Single Tester in an Agile Development Team

– Testers have to adapt to a different expectation and needs

– Plenty of room for "testing" but we need to broaden our toolkit

My Story: Sidereel

Sidereel: Pre-Tester

– Prior to 2011, no dedicated tester– All developers actively use TDD– 4 Stage deployment method put into place• developer machines / demo / staging / production

– Each step had additional people looking at the product

– Testing done at each level by product owners, content developers, support staff and execs

– Worked well, but still plenty of issues discovered after the fact

Sidereel: Post Tester

– In 2010, they decided that having a dedicated tester would be a benefit• Testers can provide many more directed efforts against

the code because of training and focus

• Product owners and other staff members can apply some time to testing, but they have other jobs

• Testers can bring all of their time to testing an application and continue to apply numerous approaches

How Can We Use Testers? Step Outside of the Obvious

• Tester verifies bug fixes

• Tester confirms user stories

Good start, but there is a lot more we can do!

Augment The Testing Already Happening

– TDD is meant to prevent defects

– TDD will help keep mistakes out, but not all of them

– Testers operate from a different mindset• Use Exploratory testing techniques• Aspects development team might not have considered• Look for missing story criteria

Acceptance Testing: Great Place for Testers

– Have testers implement Active and Automated Acceptance Testing

– Not essential for testers to know how to code, but helpful

– Developing acceptance test requirements and potential automated tests at the start of the iteration

Where to Look for Acceptance Tests

• Backlog• Current user stories• Current bugs

Automating Acceptance Tests

– Variety of Tools to Help This Process• Fitnesse• Selenium/

WebDriver• Capybara• Cucumber• Other testing

frameworks

Benefits

– Share results with the development team

– Help provide information about implementation

– Help the team get to DONE • DONE from a development perspective• DONE from a testing perspective

Disadvantages

– Passive "checking" as opposed to Active Testing• Inquiring about the results we get• Learning based on results and exploring new avenues

– Tests can quickly become stale

– Consistent upkeep required

Tester as Anthropologist

– Observe the development team

– Look for feedback of actual users

– Work with content developers

– Examine workflows– Discover the

underlying and unspoken "culture”

Other Benefits

– Testers help with planning and future work

– Testers determine testability requirements

– Testers clarify stories and validate them early

– Testers can help identify when a story is "too big"

Advocate for the Customer

– Understand the customer needs

– Review and relate to the business needs

– Review and relate to development practices

– The tester can straddle all of these

Being The Lone Tester

– Requires communication– Willingness to stretch– Pair with programmers to help make automation a

reality– Participate with planning and standups with

development team– Become a domain expert about the customers

needs

Tips to Thrive as a Lone Tester

– Focus on Communication – Learn the Expectations of Your Development Team – Stop Thinking in Terms of Us vs. Them – Learn How to Work with the Development Tools of Your

Organization – Plan for Pairing Sessions with Developers and Domain Experts

• Tester/Developer• Tester/Support• Tester/Designer• Tester/Customer

– Cultivate Relationships with Mentors – Develop Your Craft

Use Context-driven Principles– The value of any practice depends on its context. – There are good practices in context, but there are no best– practices.– People, working together, are the most important part of any

project’s context.– Projects unfold over time in ways that are often not predictable.– The product is a solution. If the problem isn’t solved, the product

doesn’t work.– Good software testing is a challenging intellectual process.– Only through judgment and skill, exercised cooperatively

throughout the entire project, are we able to do the right things at the right times to effectively test our products.

Secret Memo for Lone Testers: You're Not Really ALONE

– You have many allies– Developers are actively testing

as they develop code– Customer actively informs you

and guides you about features– Support team gives feedback

about pain points– Designers give feedback about

usability and user experience– Every one of these people

does testing, it's not ALL ON YOU!

References• Ambler, Scott (2010). "Agile Testing and Quality Strategies: Discipline over Rhetoric".

(http://www.ambysoft.com/essays/agileTesting.html)

• Crispin, Lisa (2003). "XP Testing Without XP: Taking Advantage of Agile Testing Practices" (http://www.methodsandtools.com/archive/archive.php?id=2)

• Hendrickson, Elizabeth (2008). "Driving Development with Tests: ATDD and TDD" (http://testobsessed.com/wp-content/uploads/2011/04/atddexample.pdf)

• Larsen, Michael (2011). "The Challenges of the Lone Tester: Learn to Thrive" (http://www.softwaretestpro.com/Item/5174/The-Challenges-of-the-Lone-Tester-Learn-to-Thrive)

• Parkinson, Shane (2008). "Agile Methods and Software Testing" (http://agiletesting.com.au/agile-methodology/agile-methods-and-software-testing/)

Questions???