Testing in a Velocity Culture

Utah QA User’s GroupJanuary 22, 2013

John EsserDir. of Engineering Productivity & Agile Development

Ancestry.com

Ancestry.com is the world’s largest online family history resource with more than 2.2 million members, 10 billion records, and 5 PB genealogical data.

Ancestry.com can help you discover your roots and tell your unique family story.

Rapidly developing and releasing new features and capabilities is today’s key business driver.

“Quality is secondary to speed, and probably lower than that.”

-Facebook

Ancestry.com Evolution to Agility

Agile Boot Up (Scrum)

Enterprise Agile Framework

Agile Architecture Standards

Continuous Delivery Adoption

Business Agility

2 ½ year period (2010 – present)

Agilityis the constant,

incremental flowof business value realization.

Continuous Delivery

is consistently and reliablyreleasing business value incrementsfastthrough automated build, test, configuration and deployment.

True agility drives toward a velocity driven culture.

If testing slows you down…

…then quality is NOT free. It is actually costing you a lot.

Anything that doesn’t contribute to finding the right product is waste.-Lean Startup

And that includes testing.

Testing in a Velocity Culture

• Rapid feature delivery (days not weeks).• Rapid innovation/feedback.• Testing cannot slow you down.• Testing must enable productivity.• Limited resources—actually you never have

enough people or time.

How do you win?

We are in a “no win” situation.

“I don’t believe in a no-win situation” – Capt. James T. Kirk

Kobayashi Maru

To win we must“change” the rules

“Cheat”

The Ten Commandmentsof Testing

#1 - Nuture “quality is everyone’s responsibility” attitude.

No longer be identified as Quality Assurance.

QA could be “Quality Assistance” (Atlassian)

#2 – Devs Shalt Test

“Continuous Testing”

#3 – Devs Shalt Write All Automated Tests

(or, almost all)

Examples

• Google• Facebook• Thoughtworks• Atlassian• Microsoft (moving this direction)

#4 - Regression tests shalt be automated

If regressions aren’t automated this is a waste of time.

#5 – Software Engineers in Test (SET) Shalt Enable Devs to Test

If you are testing features or code then…?

SET = Software Engineer in Test

• Do not test product features or test the code.• Concentrate on getting testing into the

workflow of the developer.• Focus on enabling automation.• Make code testable.• Write tools, frameworks, infrastructure.• Focus on making the tests run faster and

produce better diagnostics.

#6 - Test Engineers Shalt Enable Devs to Test

Test Engineer

• Focus on user concerns/perspective• Looks for weak points & holes from a holistic

viewpoint.• Does not do “menial” things; does not run

regressions• Wide latitude in role; requires a variety of skills

and perspectives.• Technical; understand architectures,

technologies, assess risk, etc.

#7 - Thou shalt avoid browser-based automated testing

#8 – TEs shalt think in terms of productivity

“People began talking about productivity instead of testing and quality. Productivity is our job, testing and quality are the job of everyone involved in development. This means that developers own testing and developers own quality. The productivity team is responsible for enabling development to nail those two things.”

-Patrick Copeland, Google

#9 – Leverage users and user-like testing earlier and more frequently.

Internal dog-foodingFrequent customer betasCrowd-sourcing (uTest)

#10 - Thou shalt have high impact

Experience report @ Ancestry.com

A lean startup experimentBonnie Bingham, Test Engineer

Social Apps

Story of a New Team

• No code, seriously• We decided to use node.js• How can Testing be successful at this point?• A GREAT dev team– Included in everything– Build code every day, fix bugs

• Insert and evangelize Lean Startup principles

Lean Startup & Testing

• Validated Learning from users• Find early adopters• Get out of the building• Avoid waste• Kill a feature• Fail• Summary

As a testing professional, can you adapt to testing in a velocity culture?

Thank You.

Q & A