what is an agile tester': henrik kniberg @ colombo agile conference 2014

72
Author Father Agile & Lean coach www.crisp.se Consultant Henrik Kniberg [email protected] @HenrikKniberg What is an Agile Tester? Colombo Agile Conf, June 2014

Upload: colombocampscommunity

Post on 20-Aug-2015

952 views

Category:

Software


7 download

TRANSCRIPT

Page 1: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Author

Father

Agile & Lean coach

www.crisp.se

Consultant Henrik Kniberg [email protected]

@HenrikKniberg

What is an Agile Tester? Colombo Agile Conf, June 2014

Page 2: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

00:22

Agile tester mindset

Henrik Kniberg

Page 3: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Page 4: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Page 5: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Page 6: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Case study: Game development company

Concept pres.

Resource planning

Graphics design

Sound design Dev Test &

deploy 1m

4h 6m

8

Game backlog

1w 6m 6m

15

Design-ready games

12

Production-ready games

1m 3w 3m 3w 1d (1m+2m)

Actual work: 3 months Time to market: 25 months

Page 7: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Before

Concept pres.

Resource planning

Graphics design

Sound design Dev Integr. &

deploy 1m

4h 6m

8

Game backlog

1w 6m 6m

15

Design-ready games

12

Production-ready games

1m 3w 3m (1m+2m)

3w 1d

Cross-functional game teams

Game team (graphics, sound, dev,

test, deploy, etc)

Time to market: 3-4 months

After

Actual work: 3 months Time to market: 25 months

7 times faster

Better games

Less Planning

More fun

Page 8: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Cross-functional teams

Henrik Kniberg 9

Dave Joe Lisa

Dave

Joe

Lisa January February March April May June July

6 months

3 months

Release

Release

We’re alot faster!

I’m a bit slower

We’re slow! I’m fast!

Page 9: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Test phase Test team

Tester role

Page 10: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Req Code Test

Test phase

Page 11: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Test team

Programmers Testers

We own quality

Page 12: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Dev team

Tester

Henrik Kniberg

Tester role I own

quality

Page 13: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Cross-functional development team

QA = Quality Assurance = Quality Assistance

Role Competency We own quality

Page 14: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Roles

Henrik Kniberg

Backend Test

DB

GUI Design

Page 15: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Competencies

Henrik Kniberg

Design

Backend

DB

GUI

Test

Page 16: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Cross functional team Doesn’t mean everyone has to know everything

Henrik Kniberg

DB Test Web Java Domain CM Lisa

Joe

Fred

Jenny

David

Erik

Product Backlog

Skills Needed to implement Top X backlog items

I’m good at Test!

I can do Java, but I’m not so good at it.

I don’t know CM at all. But I’m

willing to learn!

I won’t even go near a

database!

Knows a bit about many things

Knows a lot about one thing

Page 17: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Deliver, measure, adjust continuously

How do you know that your product works?

Henrik Kniberg

ensure

1. Understand the problem

2. Iterate until you’ve solved it

Who are the stakeholders?

What need do they have, that we want to solve?

How will we know when we’ve solved

it?

How will we know if we’re moving in the right direction?

Minimize the distance to MVP

Page 18: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

00:22

Typical activities

Henrik Kniberg

Page 19: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Make sure backlog items are testable & valuable

Henrik Kniberg

As a buyer I want to save my shopping cart so that I can continue shopping later

How to demo: 1)  Enter store 2)  Put a book in shopping cart 3)  Press ”save cart” 4)  Leave store, and enter it again 5)  Check that the book is in my

cart

Page 20: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Find defects early!

Henrik Kniberg Age of defect

$

Code

Cost of defect

Page 21: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Developers

Testers

Developers & Testers

Meeting room

Sit with the developers

Page 22: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Shorten the feedback loop

Henrik Kniberg

# of handoffs

0

1 2

3 4

5

Length of feedback cycle minutes hours days weeks months years

Maker User 1 2 3

People (# of handoffs)

Time (Feedback delay)

Page 23: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Push for Continuous Delivery

Henrik Kniberg

Test & integrate

Deploy to staging

Deploy to prod

Manual test

Manual Code & commit

Build Automatic

Page 24: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Create a shared vocabulary

Henrik Kniberg

Unit Test?

Quality? Integration test?

Acceptance test? Technical

debt?

Page 25: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Set working agreements for test automation Each user story has at least

one black-box acceptance test

Almost all code is covered by SOME kind

of test (AT or UT)

We know which code isn’t covered and have a good reason for it

Henrik Kniberg

AT and UT complement each other •  Acceptance tests: coupled to UI, decoupled from internal design •  Unit tests: coupled to internal design, decoupled from UI

Complex code has unit tests

Page 26: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Do & teach exploratory testing

Henrik Kniberg

! ?

Page 27: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Done includes ”no added technical debt”

Henrik Kniberg

Backlog Ready for dev

Ready for production

- No added tech debt

In progress

Page 28: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Example: Test automation backlog

•  Change skin •  Security alert •  Transaction history  •  Block account  •  Add new user  •  Sort query results   •  Deposit cash   •  Validate transfer  

Henrik Kniberg

Step 1: Decide what needs to be tested

Page 29: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Step 2: Classify each test

Henrik Kniberg

Test case

Change skin

Security alert

Transaction history

Block account

Add new user

Sort query results

Deposit cash

Validate transfer

Risk

Manual Test Cost

Automation Cost

low 0.5 hrs high

high 1 hrs high

med 3 hrs low

high 5 hrs low

low 0.5 hrs low

med 2 hrs medium

high 1.5 hrs low

high 3 hrs medium

Pay every time Pay once

Page 30: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Step 3: Sort the list

Henrik Kniberg

Test case Risk Manual Test Cost

Automation Cost

Block account high 5 hrs low

Validate transfer high 3 hrs medium

Transaction history med 3 hrs low

Sort query results med 2 hrs medium

Deposit cash high 1.5 hrs low

Security alert high 1 hr high

Add new user low 0.5 hrs low

Change skin low 0.5 hrs high

Automate first!

Automate later

Don’t bother automating

Page 31: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Example: Tech backlog

Henrik Kniberg

Product backlog

Tech backlog

Page 32: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Reserve X% of team capacity for the tech backlog

Henrik Kniberg

Tech backlog

Product backlog

Sprint 80%

20%

Page 33: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

00:22

Example: Spotify

Henrik Kniberg

Page 34: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Play Everywhere! Like a magical music player in which

you’ve bought every song in the world!

Page 35: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

>400 people in tech

Henrik Kniberg 36

Stockholm

Gothenburg

New York

San Francisco

Page 36: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

> 50 squads

Henrik Kniberg

Page 37: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Autonomous Squad Cross-functional, co-located, self-organizing team

Page 38: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

39

Page 39: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Squads are grouped into Tribes

Henrik Kniberg

Tribe Tribe Tribe

Tribe Tribe Tribe

Page 40: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Tribe Tribe

Chapter

Chapter

Chapter

Chapter Guild

Each Tribe is a lightweight matrix focused on delivery Vertical = Delivery. Horizontal = knowledge sharing & personal development

Page 41: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Reality is messy

Henrik Kniberg

Page 42: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Decoupling to enable frequent releases

Henrik Kniberg

Feature squads

Client App squad

!#?

Page 43: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Self-service model

Henrik Kniberg

Client App squads

IOS Android Desktop Web

Feature squads

Infrastructure squads

Enable & support

Enable & support

Enable & support

Page 44: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Release trains & Feature toggles

Page 45: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Failure Recovery is more important than Failure Avoidance

Henrik Kniberg

Failure Recovery Failure Avoidance

Page 46: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

“Limited Blast Radius” via decoupled architecture

Henrik Kniberg

Page 47: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

”Limited Blast Radius” via gradual rollout

Henrik Kniberg

Page 48: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Trust > Control 100% control = 0% motion

Henrik Kniberg

If everything’s under control, you’re going too slow!

- Mario Andretti

Page 49: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

Analyze data

Narrative & Prototypes & Metrics

Build MVP

Deploy Tweak

“Radio you can save!”

A/B stats

Idea/Problem

“Follow your favorite artist”

Page 50: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

00:22

Example: Big Government Project

Henrik Kniberg

Page 51: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Team structure - before

Henrik Kniberg 52

3 Development

teams

Test team

Requirements analyst team

?

% #

!

!

?

% #

!

!

Page 52: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 53

Next 10 features Ideas Features Development System

test User

acceptance test

Production

Page 53: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 54

Next 10 features

Ready for sys test

Dev in progress

Sys test progress

Team swimlanes

Page 54: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 55

Dev Team 1 Dev Team 2 Dev Team 3

Next 10 features Dev

in progress Ready for

sys test Team swimlanes

Page 55: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 56

”Daily cocktail party” 9:15 – 10:15

Page 56: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 57

Feature team 1 Feature team 2 Feature team 3

9:30 – 9:45 9:15 – 9:30 9:30 – 9:45

9:45 – 10:00

Test sync

Requirements sync

Dev sync

9:45 – 10:00

Project sync

10:00 – 10:15

Page 57: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 58

Count cards

Velocity per week

Example: Measuring velocity by counting cards

Page 58: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 60

60

Total # of delivered features

Week

Example: Release planning using a burnup chart All of these will be done

Some of these will be done,

but not all

None of these will be done

Page 59: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 61

”Oh no, bottleneck in System Test!

FLOW

Page 60: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Tech stories

Henrik Kniberg 62

Next 5 tech stories

Next 10 features

Page 61: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg

63

Bottleneck

”Let’s stop building new

features”

”... and focus on test automation!”

Page 62: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 64

Everyone doing tech

stories

Page 63: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 65

Top 3 recurring bugs

Page 64: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 66

Definition of ”ready for

development”

Definition of ”ready for

system test”

Page 65: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 67 Henrik Kniberg 67

Page 66: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Henrik Kniberg 68

Test Fix %&@#!

Release

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Release

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Test at end:

Test

%&@#!

Fix

Fix Time saved!

Test

Test continuously:

Before: Test at end

Now: Test continuously

Page 67: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Bug fixing process

Henrik Kniberg 69

Bug found!

More important than any of the current top 30?

No

Write sticky-note, find developer,

fix now!

Yes

Replace one of the other

top 30 bugs with this one

Yes

Ignore it

No

Don’t log it. Fix it NOW!

Blocker?

Page 68: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Three input queues

Henrik Kniberg 70

Next 5 tech stories

Next 10 features

Next 5 lower priority bugs

Page 69: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

00:22

Wrapup

Henrik Kniberg

Page 70: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

What is an Agile Tester?

Henrik Kniberg

•  An agile team member with testing expertise •  Helps the team become quality-aware •  ... and learn how to deliver better stuff

Page 71: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Mindset

Henrik Kniberg

Quality Assurance

Quality Assistance

Manual test

Functional test

Requirements

Automatic test

Exploratory test

Customer needs

Late involvement

Early involvement

Long feedback loop

Short feedback loop

Find defects Prevent defects

Page 72: What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

Agile is a direction, not a place

Henrik Kniberg

The important thing isn’t how you work.

The important thing is how you improve the way you work!