Weaning a Legacy Platform from Traditional Offshore QA

January 31, 2012

The Agile Quality Ideal

•Unit Tests

•Integration Tests

•Story/Acceptance Tests

•The development team “owns” quality

The Agile Quality Ideal – Where is QA?

Traditional QA Responsibilities:

• Verifying functionality

• Verifying bug fixes

• Regression Testing

Nearly obsolete?

The Agile Quality Ideal – A possible role for QA

Why was offshore QA a good idea in the past?

Little or no test coverage

Fear of refactoring

Code rot

Changes introduce bugs

Copious regression testing needed

Why was offshore QA a good idea in the past?

Manual regression testing

Offshore QA

•3X cheaper

•Overnight feedback

Why is offshore QA an impediment now?

• Development team produces functionality

• QA team assures quality

Why is it so hard to abandon the offshore QA model?

• Low test coverage and brittle code are realities – They don’t change overnight

• All-hands-on-deck cleanup effort is impractical – Business must go on

Practical steps to wean platform from offshore QA

Three-step plan:1. Establish development team as a

Quality organization

2. Establish development team as a “Quality Assurance” organization

3. Find a new value-adding role for offshore QA staff

Constant improvement of code cleanliness / test coverage

a. Opportunistic legacy rescue

b. Scheduled/budgeted cleanup

c. Culture of bug prevention

Opportunistic Legacy Rescue: The Hippocratic Oath

First, do no harm

Opportunistic Legacy Rescue: The Boy Scout Principle

Always leave the campsite cleaner than how you found it.

The legacy rescue catch-22

The legacy code change algorithm

From Michael Feathers, Working Effectively with Legacy Code

1. Identify change points

2. Find test points

3. Break dependencies (safe refactoring)

4. Write tests

5. Make changes and refactor

Opportunistic Legacy Rescue: Not enough

• Business expects a certain velocity

• Large-scale refactoring is expensive

• Can’t absorb within context of story development

Budget for cleanup

• One pair dedicated to large-scale refactoring and code cleanup

• Team prioritizes cleanup items to tackle

• Pair addresses issues one-by-one

Creating a culture of bug prevention

• Scrum Kanban

• Newly-introduced bug goes to top of priority queue

• Bug backlog stays in control

• Bugs become painful

Feeling the pain

• Developers: Increase efforts toward bug prevention

• Business owners: Support bug prevention efforts (e.g. refactoring)

Practical steps to wean platform from offshore QA

Three-step plan:1. Establish development team as a

Quality organization

2. Establish development team as a “Quality Assurance” organization

3. Find a new value-adding role for offshore QA staff

Adoption of “QA-type” responsibilities by the development team

a. QA lead in team room

b. Co-owned Selenium test suite

c. “QA hats” for developers

QA lead in team room

• Owns QA process

• Provides direction to QA team

• Provides instant, face-to-face feedback to developers

• Diffuses us vs. them conflicts

• Collaborates with developers in writing Selenium tests

• Coordinates exploratory testing

Selenium test suite

• Co-owned by Development and QA

• Developers responsible for adding tests as per their judgment

• Selenium tests run on CI server

• Failures require immediate attention

Developers put on “QA hats”

1. Peer verification2. Exploratory testing

Peer verification

• In workflow, between DEV Completion and Business Verification

Exploratory testing by team

• ½ hour per week

Benefits of developers wearing QA hats

• Developers assume ownership of quality

• Quality gaps become more visible to developers

• Team develops mindset toward quality

• Detect bugs that would otherwise have gone offshore

• Offshore team’s load is lightened

Practical steps to wean platform from offshore QA

Three-step plan:1. Establish development team as a

Quality organization

2. Establish development team as a “Quality Assurance” organization

3. Find a new value-adding role for offshore QA staff

But we like our offshore QA team

• Long-standing relationship

• Domain knowledge

• Inexpensive

• We want to make their current role irrelevant, not make them irrelevant

• How can we leverage them to provide value in our new Agile world?

Re-training offshore QA team to take on value-adding roles

a. Second line of defense

b. Performance testing

c. “User-centric” regression tests

Second line of defense

Writing, maintaining and executing performance tests

Writing and maintaining automated “user-centric” regression tests

The offshore QA challenge

Update your skills or become obsolete

So how are we doing?

Size of offshore QA team Before November 2010: 16

November 2010 – June 2011: 8

Since June 2011: 4

Unit test coverage June 2011: 18%

January 2012: 36%

Production high/critical bugs created vs. resolved

All high/critical bugs created vs. resolved

Q & A

שו'ת

Making Web Applications Accessible37