testing in devops: ukstar conference
TRANSCRIPT
Test value first before correctness
27-28 February 2017LONDON
Laurent Py
@[email protected]://hiptest.net
PLAN
2
The match: quality VS speed
Step 1: Testing the idea
Step 2: Testing first increment in production
Step 3: General availability of the feature
Benefits of a progressive approach to quality
27-28 February 2017LONDON
QUALITY VS SPEED
WE HAVE SPEED
4
Between 5 to 10 deployments/day
Our DevOps
teamDevs/testers
Ops
Product manager
Users
Marketing
Support
4
WHAT ABOUT THE USER?
5
Speed of development feedback and learning
Value VS Quality
5
27-28 February 2017LONDON
STEP 1: TESTING THE IDEA
8
• Use personae • Capture the expected behavior with BDD • Define business assumptions
Benefits
Support
Devs/testeurs
Marketing
Ops
Product manager
WHAT VALUE DO WE WANT TO DELIVER?
8
9
• Create a shared understanding with examples• Based on a common business language• Definition of stop• When tests are automated → living
documentation
BDD AT A GLANCE
9
1111
27-28 February 2017LONDON
STEP 2: TESTING FIRST INCREMENT IN PRODUCTION
13
1. After step 1, we have our definition of success
2. We develop a first increment, do ET
3. We deploy it in production
FEATURE IS DEVELOPED
13
14
• Based on usage- NB users, clicks- NB requests…
• Impact on- Sign-up- Revenue- Retention…
LET’S MEASURE THE IMPACT
14
15
EXAMPLES OF METRICS
15
Notification feature• Usage
- NB clicks NB of notifications- NB of users
• Business: none
Test refactoring• Usage
- NB of refactoring actions- NB of impacts
• Business: Retention
16
LIVE CHAT TO GET REAL TIME FEEDBACK
16
1717
27-28 February 2017LONDON
STEP 3: FEATURE FULLY AVAILABLE
19
1. The impact has been validated
2. Investment on performance and non regression (checks are fully automated)
3. Feature made available to all users with a switch
FEATURE BECOMES FULLY AVAILABLE
19
20
Appsignal, Pingdom, Scalingo et Logmatic
MONITORING PERFORMANCE AND BUSINESS
20
2121
27-28 February 2017LONDON
CONCLUSION
23
•Test the value first• Invest progressively on quality•Create intimacy with the users•Create a suitable environment for experimentations
KEY TAKEAWAYS
23
Thank you!
27-28 February 2017LONDON
https://hiptest.net