![Page 1: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/1.jpg)
AD31 DevOps Engineering 10:00 AM
AD31 -‐ Using Component Testing for Ultra-‐Fast Builds
Presented by:
Timothy Cochran Thoughtworks
Brought to you by:
888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ https://agiledevopswest.techwell.com/
![Page 2: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/2.jpg)
Timothy Cochran Tim Cochrancurrently works for ThoughtWorks NYC, and has been full stack developer and architect for over 15 years. Working on everything from large distributed enterprise projects to small NGO visualization apps. He recently has been helping companies with digital transformation, moving towards continuous delivery and building a DevOps culture. He also is an automated testing zealot, practices TDD almost exclusively, he builds test strategy using the right combinations of unit, E2E and functional testing for clients.
![Page 3: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/3.jpg)
11
Utilizing Component Testing for Ultra fast Builds
©ThoughtWorks 2019 Commercial in Confidence
![Page 4: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/4.jpg)
GLOBAL SOFTWARE CONSULTANCY
2©ThoughtWorks 2019 Commercial in Confidence
![Page 5: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/5.jpg)
Tech Radar
ThoughtMobile
ThoughtWorks Incorporated
1993
2000
2005
1996
2004
2006
2008
2010
2014
2016, 2017, 2018
2018
500+ ThoughtWorkers
ThoughtWorks University
ThoughtWorks was one of the first
companies to run distributed agile
projects successfully
3000+ ThoughtWorkers
5000+ThoughtWorkers
2001
free, open-source software
Top 10 contributor in open source
![Page 6: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/6.jpg)
Why Automated Testing
4
Cost of QA
Productivity
Functional & Non-Functional Problems
User Sentiment
![Page 7: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/7.jpg)
The Problem
5
![Page 8: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/8.jpg)
Software Design
6
![Page 9: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/9.jpg)
Software Design
7
![Page 10: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/10.jpg)
8
Black Box
![Page 11: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/11.jpg)
9
E2E tests
Black Box
![Page 12: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/12.jpg)
10
Download code Compile Linting / Unit Test Code Coverage Upload Artifact
2 min 5 min 10 min 20 min 5 min
Build / Test pipeline (30 mins)
Prepare E2E tests
5 min 2 hour
E2E / Regression test pipeline (2 hours 5 mins)
Deployment pipeline (15 mins)
Copy binaries Restart Services Health Checks
5 min 5 mins3 mins
Schema Changes
1 min
![Page 13: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/13.jpg)
11
Download code Compile Linting / Unit Test Code Coverage Upload Artifact
2 min 5 min 10 min 20 min 5 min
Build / Test pipeline (30 mins)
Prepare
5 min
E2E / Regression test pipeline (35 mins)
Deployment pipeline (14 mins)
Copy binaries Restart Services Health Checks
5 min 5 mins3 mins
Schema Changes
1 min
30 mins
E2E tests
![Page 14: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/14.jpg)
Anti-patterns
12
Unit test - Deploy - Regression are separate pipelines, owned by different teams
Flakey tests, Fails one in 5 tests
Build is never green, Deploying with failing tests
Environments that don’t reflect Prod, Software still breaks in production
Broken Environments, Tests aren’t run because env is down or misconfigured
![Page 15: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/15.jpg)
13
![Page 16: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/16.jpg)
UI
Service
Unit
![Page 17: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/17.jpg)
Component Testing
15
![Page 18: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/18.jpg)
Software Design
16
![Page 19: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/19.jpg)
17
![Page 20: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/20.jpg)
18
E2E tests
![Page 21: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/21.jpg)
Component Testing
19
tests
tests
tests teststests
![Page 22: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/22.jpg)
Types of Components
20
A code module
A data pipeline
Third party service
UI Component
Microservice
![Page 23: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/23.jpg)
21
What goes wrong?
Dev Prod-Like Prod
Integration with other systems
Edge Data Cases
Misunderstanding of Requirements
More edge cases
User unsatisfied
Misconfigured
NFR problems
![Page 24: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/24.jpg)
22
E2E tests Component / Service
Edge Cases
Functionality (Acceptance)
Integration between components
Critical Path
Configuration
![Page 25: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/25.jpg)
23
What is a Component?
Code
DB IO External APIs
![Page 26: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/26.jpg)
A microservice?
24
![Page 27: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/27.jpg)
25
What is a Component? A service
Code
DB IO External APIs
Service
CartService
OrderService
AddToCartService
PostReviewService
![Page 28: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/28.jpg)
Domain Model
26
![Page 29: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/29.jpg)
How to test? - Out of Process
27
Component
Test
Stub dependencies using service virtualization
![Page 30: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/30.jpg)
How to test? - In Process
28
Component
Test
Stub dependencies using service virtualization
Or use in-memory stubs
![Page 31: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/31.jpg)
29
I use subcutaneous test to mean a test that operates just under the UI of an application.
![Page 32: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/32.jpg)
Contrived Example
30
![Page 33: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/33.jpg)
Contrived Example
31
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
![Page 34: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/34.jpg)
32
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
Test
![Page 35: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/35.jpg)
33
~1000ms: headless, locally
![Page 36: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/36.jpg)
34
UIReact
BFFNode.JS
ChargeServiceNode.JS
OtherServiceNode.JS
http
http
Test
![Page 37: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/37.jpg)
35
~50ms: locally
![Page 38: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/38.jpg)
36
How to test?
Advantage Disadvantage
In Process: Executed through Code
Fastest, Deterministic Doesn’t tests all the code (e.g. network, controller)
Out of Process: Executed through the API
Fast, Deterministic
Forces API based ecosystem
Retests boiler plate code, network layer
![Page 39: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/39.jpg)
Unit test
Unit test
Unit test
Unit test
37
Checkout Build
2 min 3 min
3 min
Linting
3 min
Test Coverage
5 min
In-Process Tests
5 min Deploy App
7 mins
Time for dev feedback - 22 mins
5 mins
Unit test
Out of Process tests
5 mins
E2E
![Page 40: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/40.jpg)
How to find entry points?
38
Understand what your application does, what capabilities does it provide?
Where is the complexity in your domain?
Talk to your engineers and business
Domain Driven Design (DDD) techniques is good way, establish the domain events, bounded contexts, entities, behaviour, values.
![Page 41: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/41.jpg)
39
UI testing
API
![Page 42: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/42.jpg)
40
UI testing
Service virtualization
![Page 43: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/43.jpg)
41
UI testing
Component test
![Page 44: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/44.jpg)
42
UI testinghttps://blog.pragmatists.com/genuine-guide-to-testing-react-redux-applications-6f3265c11f63
![Page 45: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/45.jpg)
43
E2E tests
![Page 46: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/46.jpg)
44
Culture
Quality is everyone’s responsibility
Cross functional teams
Deliberately design your application to be testable,
Think about testing from day one
![Page 47: AD31%!UsingComponent!Testingfor Ultra%Fast!Builds! · Why Automated Testing 4 Cost of QA Productivity Functional & Non-Functional Problems User Sentiment. The Problem 5. Software](https://reader035.vdocuments.us/reader035/viewer/2022081407/6051c6a9c7b1e866e90ae765/html5/thumbnails/47.jpg)