INDIVIDUELLE SOFTWAREINDIVIDUELLE SOFTWARE

QUALITY FROM THE PERSPECTIVE OF A PRODUCT OWNERFROM 6 WEEKS TO ONE-DAY RELEASE PHASES

AGENDA

Project context and stakeholders

The PO role and quality

The journey

Lessons learned -criteria for product owners

INDIVIDUELLE SOFTWARE

INTRODUCTIONS

INTRODUCTIONS

• I am…– A tester, head of testing, a product owner

• Bredex is– A software development company– With a strong focus on quality

• Own open source tool• Multi-tool strategy• Agile testing• Scalable test automation

Training

Development

Testing

THE PROJECT CONTEXT

Internal stakeholders(managers)

Community

Paying external customers

Internal customers

Development team for the product

THE PROJECT CONTEXT: QUALITY DESIRES

Internal stakeholders(managers)

Paying external customers

Internal customers

Development team for the productUsefulness for our own tests

New featuresBug fixesSpeed of valueMigratabilityMinimal regressions

Particularly quick value deliverySupportSpecific / individual featuresNo regressions

Alignment withgoalsHappy customers

SupportQuick bug fixes / patchesNew featuresExtensibility

Community

THE PROJECT CONTEXT: CORE MESSAGE

There are a lot of people to make happy.

Who often want very different things.

Now.

THE PROJECT CONTEXT: CORE MESSAGE

There are a lot of people to make happy.

Who often want very different things.

Now.

And because it’s open source, most of them are not paying.

And it's hard to identify them.

WHOLE TEAM QUALITY AND THE PO

• Everyone is responsible for quality– Every Dev-Team member helps and supports– (more on this later)

• What does the Product Owner do?– Testing?– Other things?

THE ROLE OF THE PRODUCT OWNER

• Prioritising requirements• Creating user stories and acceptance criteria• Being available for questions

THE ROLE OF THE PRODUCT OWNER

• Prioritising requirements• Creating user stories and acceptance criteria• Being available for questions

• Regular delivery of high quality*, valuable* releases

Photo from FreeDigitalImages.net tigger11th

*highly context- and stakeholder dependent

INDIVIDUELLE SOFTWAREINDIVIDUELLE SOFTWARE

A RELEASE JOURNEY THROUGH THE EYES OFA PRODUCT OWNER

HOW IT ALL BEGAN

STAIRWAY TO DOOM

No automation

Very slow release phases

No chance of hot fixes or respins

Aim for perfect quality (testing everything)

FROM THE PO PERSPECTIVE…

• Regularity: Non-existent• Quality: High aims, not necessarily attained• Value: Unsure

GETTING MORE FEEDBACK

INTRODUCING CONTINUOUS INTEGRATION

• Learning to build• Learning about build pipelines and test machines• Increasing test automation• Looking at test results daily and reacting to them

FROM THE PO PERSPECTIVE…

• Regularity: Non-existent• Quality: Better feeling• Value: Unsure

RELEASE CANDIDATES

Alex‘s law: release testing and fixing will expand to fill all available time

RELEASE CANDIDATES

• Idea: – Checklists for steps up to release (RC0-RC4)– Gradually fewer commits and fixes– Stable release

• Expected benefit:– Can move on to other development while final stages of release

are taking place

• Actual effect:– Still 6 week release phases, with more multi-tasking & branching

FROM THE PO PERSPECTIVE…

• Regularity: Non-existent• Quality: Same feeling, just slower• Value: Unsure

CONTINUOUS INTERNAL DELIVERY

CONTINUOUS INTERNAL DELIVERY

• Triggered by desire to know even more about quality• Extreme dog-fooding

– Testers work with new version on daily basis– New version used in nightly tests

• Switch to feature branches to ensure clean master– Big-bang integration, but fewer daily risks– Exploratory testing on all branches, automated tests on master

FROM THE PO PERSPECTIVE…

• Regularity: Attainable• Quality: Good feeling• Value: Unsure

BETA RELEASES

BETA RELEASES

• After a sprint with some cool new features to test out• Specific user groups / careful wording on download• Test our quality, test the value

FROM THE PO PERSPECTIVE…

• Regularity: As often as we want• Quality: Good feeling, and testable• Value: Testable

FROM BETA TO OFFICIAL

FROM BETA TO OFFICIAL

1-day release phase

0-day release phase

Product owner

wants to release

- Checklist for things we forget / familiar problems- Focus on relevance for release- Practised in releasing- Willing to take a couple of risks- Time can be increased if the situation demands it

INDIVIDUELLE SOFTWARE

QUALITY FROM THE PO PERSPECTIVE ANDTAKEAWAYS

WHAT ARE MY AIMS FOR QUALITY?

Quality of the development

Quality of the release itself

QUALITY OF DEVELOPMENT

Deliverability

Suitability

Correctness

Finishability

Assessability

Quality attributes Users only receive value when it is delivered.

Correctly implemented isn’t enough.

SUITABILITY

Suitability

Understandwhy

Use examples

Sprint review

Mini-demos

DELIVERABILITY

Finishability: the feature is “done” in a finite length of timeand can be released

Photo from FreeDigitalImages.net ddpavumba

Whole team testingLimit WIP,focus on done

Done=dev, test, docFeature Branches

What can we get done today?

DELIVERABILITY

• Typical product owner question when can I release?

• How can we know?– Exploratory testing as a part of the story– Automated UI Tests as separate stories– Git and Gerrit– Partially automated test result analysis

Assessability: the team receives quick feedback about quality

QUALITY OF THE RELEASE

Known correctness Speed

QUALITY OF THE RELEASE

Focus on finishability

Greatly reduce test

phase

Focus on assessibility

Magic sauce…

THE MAGIC SAUCE IS TRUST

THE TOUGHEST DECISION AS A PRODUCT OWNER…

We have been doingtesting continually

We‘ve done a final overview test

We‘ve checked ourfamiliar problems

We‘re using this mostdays

There will still beissues – known and

unknown

Is the bigger risknow lateness or

correctness?

We can solveproblems

quickly

No reliance on 100% clearinformation

TRUST

• Create trust in the software– Enough automation– Enough exploratory testing during the sprint– 1 day of test charters before an official release– Lightweight release checklists

• Use betas / specific customers to find unknown risks

• Retrospect when something unexpected happens

• I’d rather fix it quickly than wait and test too long*

* Except when it comes to airplanes …. ;-)

TAKEAWAYS

• Product Owners have additional quality criteria

• Context is important! Consider your full spectrum of quality criteria for your context

• Act iteratively and focus on getting better

• Remember what we’re doing it for: delivering value to customers regularly