parasoft testing anything, any time with containerized service virtualization
TRANSCRIPT
Testing anything, any time with containerized service virtualization Leveraging on-demand virtual test environments
for earlier, better and faster testing
Robert Schrijvers & Rix Groenboom
Who are we Joint work:
• Rix Groenboom (Parasoft) • Manager EMEA Solution Architects • R&D with Universities (Groningen, Bamberg) • Implementing testing solutions for technology, finance, telecom and government
organisations
• Robert Schrijvers • Independent Consult • Java specialist with focus on performance • Working together with Parasoft since 2012 • Working for various banks in the Netherlands
Agenda • Modern times and need for Virtualization
• Virtualization per phase • Development • Test • Acceptance • Pre-production
• Requirements & implementation • Functionality (Dev) • Deployment (Ops)
• Wrap up
Agenda • Modern times and need for Virtualization
• Virtualization per phase • Development • Test • Acceptance • Pre-production
• Requirements & implementation • Functionality (Dev) • Deployment (Ops)
• Wrap up
Once upon a time …
Business Logic
Data
Client A Client B q Large applica4ons q Few Development teams q Few Test teams q Few Opera4ons team q Few releases q Few Dependencies q Life was easy … but slow
Batch processes
Modern times
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
Components: Dev -> Tst -> Acc -> Prod
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
D
T
A
P
DevOps teams • Many teams
• Independent
• Different cycle speed
• Test often • Test early • Test automatically
“Modern 4mes”
Decoupling by virtualization • Prevent teams slowing down by other teams
• Prevent skipping tests in early stages
• Extend test cases
• Depends on stage in release cycle • Various performance profiles • Various deployment types • Various transports (protocols) • Various payloads (messages-types)
Agenda • Modern times and need for Virtualization
• Virtualization per phase • Development • Test • Acceptance • Pre-production
• Requirements & implementation • Functionality (Dev) • Deployment (Ops)
• Wrap up
Development
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P
Development
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P
Virtualize N,O
Development
B API
HTTP D
T
A
P
Virtualize N,O
• Virtualize on local machine
• Unit test
• Functional tests
• Fully decoupled
• Fully independent
Test
Data NoSQL
A API
B API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P C API
Test
Data NoSQL
A API
B API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P C API
Virtualize P,X,Y,Z,
Third Party
Test
Data NoSQL
B API N
SVC
O SVC
Y BA
TIBCO
HTTP TIBCO
MQ
PROP
SOAP SOAP
JSON
PROP
PROP
PROP
D
T
A
P
Virtualize P,X,Y,Z,
Third Party
CD pipeline
Test
Test • Choose “cut-off”
• Deploy AUT
• Deploy Neigbours
• Virtualize • On Server • Configured environment • Supports custom transport • Supports custom payload
• Functional tests
• Environment choice • Test chain
Data NoSQL
B API N
SVC
O SVC
Y BA
TIBCO
HTTP TIBCO
MQ
PROP
SOAP SOAP
JSON
PROP
PROP
PROP
D T
A P
Virtualize P,X,Y,Z,
Third Party
Test
Acceptance • Focus on non-functional requirements
• Component central • DevOps team responsibility • Capacity • Stability • Performance • Resilience
Acceptance
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P
Acceptance
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
D
T
A
P
Virtualize N,O
Test data
Acceptance
B API
TIBCO
HTTP SOAP
JSON
D
T
A
P
Virtualize N,O
CD pipeline
Test Test data
• Reconfigure • Slow response • No response
• Virtualize • Reconfigurable • Performant • Stable • Representative data
set
Pre production • Does not fit in pure DevOps
• IMHO mandatory
• AKA “staging”
• Production like as possible • Sizing • Configuration • Data
• “Tomorrows production”
Pre-production
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Z BA
Y BA
Data RDBMS
Data Prop
Third Party SVC
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
P’
P
P
P
P
P
P
P P
P: Produc-on version
P’: New version
P
Pre-production
Data NoSQL
A API
B API
C API
M SVC
N SVC P
SVC
O SVC
X BA
Y BA
Data RDBMS
Data Prop
HTTP
TIBCO
HTTP TIBCO
HTTP
MQ
MQ
PROP
SOAP
SOAP
JSON
JSON
SOAP
PROP
PROP
PROP
P’
P
P
P
P
P
P P
P: Produc-on version
P’: New version
Virtualize Z,
Third Party
Test
Test
Test
P
Agenda • Modern times and need for Virtualization
• Virtualization per phase • Development • Test • Acceptance • Pre-production
• Requirements & implementation • Functionality (Dev) • Deployment (Ops)
• Wrap up
Requirements gathering (1) Based on these user-stories:
• Local and Server deployment
• Multiple Transport protocols
• Multiple Message payloads
• Flexible Data-set configuration
• Run-time deployment
• Run-time data provisioning
Let’s make a “Separation of concerns” between DEV & OPS
Requirements gathering (2) DEV (building Virtualizations): • Message responders: XML, JSON, EDI, SWIFT, Text… • Logic: Synchronous, Async, Stateful, “batch” • Data: File, CSV, SQL, NoSQL OPS (running Virtualizations): • Protocol listeners: HTTP, JMS, MQ TCP/IP, File, … • Deployment Descriptions: Endpoints, Queues, JNDI • Performance profiles: Delay, Time Out, Errors
• Some part of the Logic (functionality) • Some part of the Deployment (transport)
Deployment strategies Two main deployment strategies:
• Central: Client / Server model • Repository containing Implementations (PVA-files) • Environment Manager to control provisioning • Optional Load Balancing with multiple servers
• De-central: Container or Cloud model • API driven launch of local servers • GIT based deployment of Implementation (PVA files)
Towards completely scripted construction
Central Deployment (client / server)
Virtualize SME
Environment Manager
Stage/Test Server
Production Server(s) Failover Server(s)
Desktop
Application Under Test
Towards scripted deployment • Docker & Java & Tomcat
• Set protocol jars & Deploy Virtualize (war)
• Static implementation: • Deploy PVAs (model) from GIT • Provision test-data
• Dynamic implementation: • Build PVA files on the fly (using API) • Example:
• Given “customer with 2 credit cards”
• When “deleting one card”
• Then “one card should be left”
Towards scripted deployment
B API
TIBCO
HTTP SOAP
JSON
D
T
A
P
Virtualize N,O
(deployment descrip4on)
CD pipeline
Test
Logic&Data N,O
• Pipeline: • Deploy SUT (eg B API) • Deploy Virtualize (docker) • Initialize N,O placeholders
• Test-execution: • Populate N, O (“Given”) • Execute (“when/then”) • (and repeat)
Deployment summary • Central server and central deployment:
• “Waterfall”: Central “test data team”
• Central server and de-central deployment: • “Agile”: teams have control over Virtual Assets
• De-central Server and central deployment • For 3rd party interfaces / complex backends
• De-central server and de-central deployment • “DevOps”: Deploy Virtual Assets as part of pipeline
1. “library of PVA’s” and provisioning
2. “on the fly creation” (given-when-then)
Agenda • Modern times and need for Virtualization
• Virtualization per phase • Development • Test • Acceptance • Pre-production
• Requirements & implementation • Functionality (Dev) • Deployment (Ops)
• Wrap up
Wrap up • Virtualization decouples your components
• Decoupling oils the DevOps / CD pipeline
• Virtualization requires: • Various deployment models
• Local, Server, Docker, Cloud
• Ready to use transports and payloads • Extensibility where needed • Dynamic reconfigurable (provisioning) • Scalable to use in performance test • Integration in CD pipeline (APIs)