全民QA

Ethan Huang

An Integrated Development and Testing Lifecycle

黄方 (Ethan Huang)

博克软件（杭州）有限公司

It’s about DEVELOPMENT

Tester A:

• Look at that bug; it’s pretty straightforward that the functionality doesn’t match our test case. Why can’t somebody do a quick smoke test before checking in the code?

Developer A:

• Well, yes I agree that’s a bug. But we didn’t have enough time, you know, the schedule is tough, we did as much verifications as we could before we checked the code in; but we didn’t have enough time to cover that functionality. It’s great that the testing team found that bug, we can fix it later.

A real conversation

Tester B (Test Lead) :

• But that costs a lot, we spent a whole day to manually execute all the functional test cases and found at least 5 obvious bugs. They could be identified even without looking at the test cases. Now we need another day for regression test after your team gets them fixed.

Developer B (Develop Lead) :

• But that’s the reality, isn’t it? It’s normal to have bugs. We cannot avoid delivering bugs together with the code. That’s why we have a testing team.

A real conversation

Find A BUG

1 h 2 h 4 h 2 h

1 d

1 h2 d1 d0.5 d

AT LEAST 1 week

Fix This Bug

SmokeTesting

GenerateA Dev Build

Push To Test

Bug Verification

RegressionTesting

Push To Staging

UATPush To Production

Rework/Cost

Bad quality

Team silos

And this conversations is happening everywhere on this planet…

Why can’t we do things right the first time?

What does

Do?

The REALITY

• 6 Developers : 1 Tester

• Non-structured narrative requirements

• Huge amount of legacy functionalities

• Few tests automated

• 13 weeks to deliver

The PROBLEM

After the planning the whole team was feeling upset because of the predictable BAD QUALITY

Team did root cause analysis

Team did root cause analysis

• 1 Tester cannot complete all testing work

• We might have to shrink testing phase

• Big, complicated features - long Dev cycle needed to deliver one feature

• Huge Regression Testing effort needed to cover legacy features as well

• Has no Requirements details , only mockups

• Don’t know what details to implement/write test cases

• Lots of dependencies – hard to test

80%

20%

• 1 Tester cannot complete all testing work

• We might have to shrink testing phase

• Big, complicated features - long Dev cycle needed to deliver one feature

• Huge Regression Testing effort needed to cover legacy features as well

• Has no Requirements details , only mockups

• Don’t know what details to implement/write test cases

• Lots of dependencies – hard to test

Team did root cause analysis - voted

Team decisions before kicking off

• Break the team silos – Team Wide Testing

• Do things right the first time – Create fewer bugs

Team

• Developers to be involved into all QA activities

• Let the only Tester organize the whole team

Process

• We don’t do waterfall

• We don’t do small waterfalls iteratively either

A new V Model

Activities

• Represent Requirement using UAT Cases

• Write Automation Tests before development

• Test Driven Development

• CCR + Local Verification

• Check-In, CI + Continuous Automated Testing

• Daily Verification/Daily Demo

• Do UAT every Iteration

Break feature down to small stories

Every single story is a development cycle

Every Sprint delivers some stories

Two Quality Gates

• Represent Requirement using UAT Cases

• Write Automation Tests before development

• Test Driven Development

• CCR + Local Verification - Quality Gate 1

• Check-In, CI + Continuous Automation Testing

• Daily Verification/Daily Demo - Quality Gate 2

• Do UAT every Iteration

What we achieved

12 Sprints for Development,

1 Sprint for Testing

What we achieved

We didn’t complete all the User Stories

But for those stories we delivered,the client couldn't find even ONE BUG

Summary

A new Team Model integrates Developers and Testers

A new Lifecycle Model integrates Development and Testing

New Development activities driven by Tests