eurostar - testing as a service model
TRANSCRIPT
Today’s webinar is presented by Jonathon Wright and he will discuss Testing as a Service - Models.
Welcome to the EuroSTAR November
Webinar Series
This webinar is due to start at 2pm. Make sure you stick around at the end for the Q&A session and continue the conversation with the speaker on Twitter after the show!
@esconfs
#esconfswww.eurostarconferences.com
Testing as a Service - Models
Jonathon Wright, Director, Testing, Quality, AssuranceHitachi Consulting, Global
Jonathon Wright has over 15 years of commercial automation experience with a number of international organisations. Jonathon also contributed to the best-selling book ‘Experiences of Test Automation: Case Studies of Software Test Automation (2012)’ and a number of upcoming books on test automation and
API testing in the cloud. He is an active blogger on ‘Test Automation as a Service’ (TaaaS.net) as well as presenting at various international testing conferences
Gartner (London) / STARWest (California) / Fusion (Sydney) / ANZTB (Melbourne) / BCS SIGIST (London)
@esconfs#esconfs www.eurostarconferences.com
Q&A
If you have a question, the control panel on the right has a Questions box. All questions will be answered at the end of the presentation.
The GoToWebinar attendee interface is made up of two parts. The Viewer Window shows the presenter’s screen. The Control Panel is where attendees can interact and ask questions.
The webinar will be recorded and you will be notified via email when it is available.
@esconfs#esconfs www.eurostarconferences.com
How can I get the slides?
Slides and all materials will be posted on the EuroSTAR website as well as emailed to you.
@esconfs#esconfs www.eurostarconferences.com
http://www.slideshare.net/jonathon_Wright
Join the conversation on Twitter
#esconfs
@esconfs
@Jonathon_Wright
Continue the conversation with the speaker on Twitter after the show!
@esconfs#esconfs www.eurostarconferences.com
@jonathon_wright
#esconfs www.eurostarconferences.com
Better TaaS
5th November 2014
Testing as a Service - ModelsJonathon Wright, Director, Testing, Quality, Assurance
7
@jonathon_wright
#esconfs www.eurostarconferences.com
Hitachi - a global industry leader.Society Changes, Hitachi Transforms It.
$93.4
Billion947Companies
320,000+Employees
$3.4
Billion
Hitachi ranks 78th on the 2014 FORTUNE Global 500®.
© Hitachi Europe Ltd. 2014. All rights reserved.
8
@jonathon_wright
#esconfs www.eurostarconferences.com
Hitachi – in Europe
Information & Telecommunication Solutions Distribution Solutions
Railway Solutions Smart Cities
Service
IT Service, Data Centre Solutions, Consulting
System Solutions
Large-scale systems, Highly reliable systems
Finger Vein Authentication
Healthcare
Hardware, Software
Rolling stock + maintenance
Electrical components
Signaling / Train control systems
Nuclear Power Business
Mining Equipment
Platforms
Smart eco cities
Big Data Infrastructure
Energy Consumption
@jonathon_wright
#esconfs
Water & Natural Resources
9
@jonathon_wright
#esconfs www.eurostarconferences.com
Hitachi – Smart Cities - Copenhagen
Effecting the future home of EuroSTAR (2000, 2005, 2010) Copenhagen Cleantech Cluster
City
Admin
Mobility
"Green and Comfortable
Transportation"
"Lower energy consumption"
- City of bikes
- ITS
- Mobility Management
- Copenhagen Connecting
- Buildings
- Transport
- Procurement
- Street Lighting
Energy
"Carbon-neutral city"
- Smart meter by 2020
- Wind turbines
- Biomass in power plant
- Smart Cities
@jonathon_wright
#esconfs
Carbon Neutrality
by 2025
10
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing industry is changing forever
Intel© 14nm SoC in 2013
Does ‘Testing as a Function’ still have a future?
11
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing as a Function
Legacy Methodologies
Software Development Lifecycle
Hardware Analysis
Testing
Process
MethodologyBusiness
Software Development Lifecycle (SDLC) Model
12
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing industry is not ready
Support for business intelligence powered by testing ‘small data’
✔
Hitachi Content Platform Solution
During deployment of
the Hitachi Content
Platform Solution
70TB of satellite
information was
ingested into the
solution platform with
a daily growth rate of
50GB/day.
Business
Intelligence
Platforms
Business
Intelligence
Infrastructure
* http://www.hds.com/assets/pdf/hitachi-storage-solutions-at-work-nasa.pdf
13
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing as a Activity
Future Methodologies
Solution Delivery Lifecycle
Application
Portfolio
Business
IntegrationEnterprise
Management
Solution Delivery Lifecycle integration (SDLCi) Model
14
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing industry needs new models
Global challenges are not suitable for ‘Testing as a Function’ silos
Complex eco-
infrastructures
intelligent
machine-to-
machine inter-
connections
empowered by
‘Big Data’
* http://www.libelium.com/libelium-smart-world-infographic-smart-cities-internet-of-things/
15
@jonathon_wright
#esconfs www.eurostarconferences.com
New Model for Testing*
Model Based Testing (MBT)
*Paul Gerrard, 2014, ‘New Model for Testing’ discussion paper published, 6th August 2014
New Model for Testing
16
@jonathon_wright
#esconfs www.eurostarconferences.com
Model Based Design (MBD)
Advanced Modelling Simulation Technologies
Watershed management
Pipe-network Management
Prediction of water pollution
Pipe-network analysis
© Hitachi Europe Ltd. 2014. All rights reserved.
Models can be analysed, visualized, and optimized
Models can provide assurance through, analysis, simulation and prediction
* http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi
17
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
@jonathon_wright
#esconfsTesting as a Service - Models
https://leanpub.com/taas
18
@jonathon_wright
#esconfs www.eurostarconferences.com*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014
BUILD DIGITAL BLUEPRINT + ROADMAP
Why ‘Digital Enterprise’
‘Digitalised’ businesses are able to better adapt and exploit opportunity
@jonathon_wright
#esconfsTesting in the Digital Enterprise
19
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Business Delivery Management
Model Based Design (MBD)
Testing as a Service - Models
Business Lifecycle Management
Problem
Business Abstraction Layer
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)
Business Delivery Management - ‘As A Service’ layer
Test Abstraction Layer – Test Solution Delivery
Business Delivery Management - ‘Testing as a Service’ layer
Portfolio Lifecycle Management
Solution Lifecycle Management
Application Lifecycle Management
Test Abstraction Layer – Test Solution Delivery
Solution
20
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
• Business Lifecycle Management
– Business on a Page (BoaP) level abstraction or encapsulation representing the core
businesses differentiators such as agility, visibility and flexibility
• Portfolio Lifecycle Management
– number of solutions that are represented by a logical grouping as part of a Business Portfolio
or split by Business Domain or Workstream
• Solution Lifecycle Management
– covers logical groupings of applications that make up the solution landscape
• Application Lifecycle Management
– refers to a single application instance
Business Delivery Management
Business Lifecycle Management
21
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Business Abstraction Layer
Business Abstraction Layer
Model Based Design (MBD)
Testing as a Service - Models
Business Solution Discovery Business Solution Delivery
Business
on a Page
Business Level
Answers
Business Level
Questions
Business Level
Stories
Business
Requirements
Business Process
Design
Business Process
Modelling
Business Process
Validation
Business Process
Intelligence
Business Process
Scenarios
Business Process
Components
Business Process
Tests
Business Process
Data
Business Process
Transactions
Business User
Stories
Business Level
Keywords
Business Non-
Functional (NFRs)
Business Epic
Themes
Business Definition Layer
Business
Acceptance Criteria
Business Definition
Language
Business
Acceptance Tests
Business Domain
Knowledge
22
@jonathon_wright
#esconfs www.eurostarconferences.com
Business Delivery Management
Model Based Design (MBD)
Business Abstraction Layer
Business Solution Discovery Business Solution Delivery
Business Definition Layer
Test Abstraction Layer
Test Solution Discovery Test Solution Delivery
Test Definition Layer
Problem
Business Delivery Management (‘As A Service’ Abstraction Layer)
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)
*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
Solution
Logistics
23
@jonathon_wright
#esconfs www.eurostarconferences.com
Why ‘As A Service’
• Testing as a Service (TaaS)
• Software as a Service (SaaS)
• Platform as a Service (PaaS)
• Infrastructure as a Service (IaaS)
• Hardware as a Service (HaaS)
Testing in the Wild - Autonomous Vehicles
@jonathon_wright
#esconfs *Jonathon Wright, Top Gear Track, Test Results: 1:47.63, http://youtu.be/P8JnQV0nXrc, 31st October 2014
Performance Test Results: 1:47.63
24
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Problem
Business
Customer
SolutionProblem
Definition
Business
Intelligence
(CIO, COO, CTO)
Solution
Decommission
Social
Intelligence
Business
Innovation
Social
Innovation
‘As A Service’ - Model
Model Based Design (MBD)
Testing as a Service - Models
Business Problem, Idea or Challenge (cost, benefit, time to market)
Business Solution Delivery Quality (cheap, good, fast)
‘As A Service’
CustomerEnd-Users
25
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011
Business Delivery Management
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLCi)
Testing as a Service - Models
Portfolio Lifecycle Management
Business Lifecycle Management
Solution Lifecycle ManagementBusiness Delivery Management
AdviceProblem
Business Solution
Discovery
Business Solution
Delivery
26
@jonathon_wright
#esconfs www.eurostarconferences.com
Why ‘Testing As A Service’
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Testing in the Wild – Near Field
Communications (NFC)
• US Census Bureau* world population
counter 7.2 billion people on the planet
• GSMA Intelligence* over 7.2 billion mobile
devices operating in the world right now
• CNET* over 250 million machine-to-machine
connections
* http://www.cnet.com/news/there-are-now-more-gadgets-on-earth-than-people/
Testing as a Service - Models
27
@jonathon_wright
#esconfs www.eurostarconferences.com*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Model Based Design
Testing as a Service - Models
Shift Left
Shift Up
Shift Down
Shift Right
Test
in
Dev
Business
Acceptance
Testing
Test
Ops
User
Acceptance
Testing
Social
Enterprise
Solution Delivery Lifecycle integration (SDLi)
Continuous, Integration, Build & Deployment (CIBD)
Business
Enterprise
(CIO, COO, CTO)
Testing as a Service - Model
28
@jonathon_wright
#esconfs www.eurostarconferences.com
• Test Platform as a Service (TPaaS)
– Social Enterprise Collaboration
– Actionable Business Insight
– Enterprise Gamification
– Global Testing Marketplace
– All-Channel Customer Experience• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Test Platform as a Service (TPaaS) Model
29
@jonathon_wright
#esconfs www.eurostarconferences.com
Social Enterprise Collaboration
• Delivering true business value – Not just driving process efficiency
– Across the SDLCi with cross-functional domain tools
– Providing instant actionable business insight
• Supporting individual, team, organizational, and enterprise collaboration
– Integration with the Global Testing Marketplace (GTM) platform
– Produce high-quality reusable business assets in the cloud
– Capture specialist technical and cross-domain knowledge (i.e. DIDO)
– Game mechanics to encourage collaboration (i.e. Enterprise Gamification)
– Produce design patterns that allow business rules to be codified in business-specific meta-languages (i.e. BPMNv2.2, xPDL, XAML)
• Managing software that gets deployed everywhere
– All-Channel Customer Experience
*Jonathon Wright, 2014, ‘Agile Portfolio Management in the Cloud’ presented at Unicom, London, 20th March 2014
Test Platform as a Service (TPaaS) Model
Embracing Agile Portfolio Management in the Cloud
30
@jonathon_wright
#esconfs www.eurostarconferences.com* Huw Price, 2014, Grid-Tools.com, https://www.grid-tools.com/solutions/agile-designer/ , 3rd November, 2014
Business Delivery Value Chain - SDLCi
Requirement <-> Traceability <-> Use Cases <-> Traceability <-> Test Case <-> Traceability <-> Defect
Agile Designer
HP ServiceVirtualization
DataMaker
HP QCHP ALMSelenium
AgileBoards
• Test Cases• Virtual Data• Automation
• Find• Allocate• Synthesize• Data On Demand
• Requirements• Test Cases• Test Data• Defects• Automation
• Story Boards• Work Allocation• Work Tracking• Story Points
• Use Cases• Requirements• Complexity• Test Cases• Test Data• Virtual Data• Automation• Expected Results
• Backlog
Development
Functional
Integration
System Testing
Performance
Acceptance
Developers
Testers
Business
Test Platform as a Service (TPaaS) Model
S
D
L
C
i
31
@jonathon_wright
#esconfs www.eurostarconferences.com
Actionable Business Insight powered by ‘Small Data’
Funnel Virtualisation
BusinessInsight
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
DataData
Data
Data
DataData
* Llyr Jones, 2014, Grid-Tools.com, https://twitter.com/GridTools/status/487264150578790400 , 10th July 2014
Test Platform as a Service (TPaaS) Model
32
@jonathon_wright
#esconfs www.eurostarconferences.com
Enterprise Gamification
Service Thinking (Service as a Experience) Business Model
G
A
M
I
F
I
C
AT
I
O
N
E
N
T
E
R
P
R
I
S
E
Shift Left
Shift Up
Shift Down
Shift Right
Test
in
Dev
Business
Acceptance
Testing
Test
Ops
User
Acceptance
Testing
Social
Enterprise
Test Platform as a Service (TPaaS) Model
* http://enterprise-gamification.com/attachments/article/233/Improving%20Software%20Quality%20with%20Gamification%20-%20Full%20Paper.pdf
Business
Enterprise
(CIO, COO, CTO)
33
@jonathon_wright
#esconfs www.eurostarconferences.com
Enterprise Gamification
Multi-sided Metrics
Test Platform as a Service (TPaaS) Model
34
@jonathon_wright
#esconfs www.eurostarconferences.com
Global Testing Marketplace
Become a Testing Rock Star!
*http://blog.utest.com/2014/11/04/utest-platform-preview-new-dashboard-for-testers-on-paid-projects , 4th Nov 2014
Test Platform as a Service (TPaaS) Model
@jonathon_wright
#esconfs
35
@jonathon_wright
#esconfs www.eurostarconferences.com
Global Testing Marketplace
Improved Communication, Collaboration and Mobility
*Jonathon Wright, 2012, ‘Test Automation as a Service’ at Fusion, Sydney, 10th September 2012
Test Platform as a Service (TPaaS) Model
Test©
36
@jonathon_wright
#esconfs www.eurostarconferences.com
Global Testing Marketplace
Instant Scalability, Flexibility and Availability
@jonathon_wright
#esconfs *Jonathon Wright, 2014, ‘Solution Lifecycle Management in the Cloud’ at Unicom, London, 27th February 2014
Test Platform as a Service (TPaaS) Model
G
L
O
B
A
L
M
A
R
K
E
T
P
L
A
C
E
37
@jonathon_wright
#esconfs www.eurostarconferences.com
All-Channel Customer Experience
The digital journey for transformation or convergence
@jonathon_wright
#esconfs
S
O
C
I
A
L
I
N
T
E
L
L
I
G
E
N
C
E
@jonathon_wright
#esconfs Test Platform as a Service (TPaaS) Model
38
@jonathon_wright
#esconfs www.eurostarconferences.com
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
– H(API)y Testing in the Cloud
– Test First Delivery
– Building bridges in the Cloud
– Cloud 2.0 beyond legacy
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Test Infrastructure as a Service (TIaaS) Model
39
@jonathon_wright
#esconfs www.eurostarconferences.com
Ha(API)y Testing in the Cloud
• Build Bridges in the Cloud
– Use Cloud Maps to Learn, Discover and Model endpoint(s)
– Create Bridges (VNet to VNet) “If you build it, they will test” (IYBITWT)
– Embrace Test Infrastructure as a Service (TIaaS) in the Hybrid Cloud (TQA™ Lab)
• Avoid Legacy Cloud
– Heterogeneous Test Infrastructure as a Service (TIaaS) that are open, flexible and extensible
– Continuous Build, Integration & Delivery in YourTestCloud
– Embrace Community Test Clouds (Collaborate/Share Recipes)
• First Day Delivery
– Enable Day-zero Performance, Penetration & Security Testing as a Service
• Support for powerful Test Automation as a Service (TAaaS)
– UI Testing that is Scalable and Cost-Efficient
– API Testing that is Repeatable, Reliable and Fast
Unlock true delivery innovation
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
40
@jonathon_wright
#esconfs www.eurostarconferences.com
Building Bridges in the Cloud
“If you build it, they will test” (IYBITWT)
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
41
@jonathon_wright
#esconfs www.eurostarconferences.com
TQA™ Lab
Building Bridges in the Cloud
HP ALM (SaaS)HP PC (SaaS)HP PC
(Controller)
HP PC
(Load Generators)
HP PC
(Load Generators)
Hybrid Cloud
Public Cloud
HP PC
(Load Generators)
Private Cloud (Performance)
HP PC
(MOFW)
Solution Under Test (SUT)Client X
HP CodedFT
Private Cloud (Automation)
Agile Designer
Solution Under Test (SUT)Client X
VN
et-
to-V
Net
HP SV/NV
(SaaS)
HP Agile Manager
(SaaS)
HP StormRunner
(SaaS)
HP ALI (SaaS)
* http://h30499.www3.hp.com/t5/Unified-Functional-Testing/UFT-12-Test-Execution-in-the-Cloud-vs-Local-VMM/td-p/6571480
Cores: 72
RAM: 150gb
Nodes: 144=
SV (Servers) = 50
SV (Designers) = 60
PC (Generators) = 15k
UFT (API) = 100
UFT (Runtime) = 1440
Test Infrastructure as a Service (TIaaS) Model
D
R
D
R
42
@jonathon_wright
#esconfs www.eurostarconferences.com
Cloud 2.0
Moving beyond the legacy cloud
Test Infrastructure as a Service (TIaaS) Model
select <IaaS>
select <PaaS>
select <SaaS>
select <TaaS>
select <TaaaS>
Cloud Recipe for Solution Under Test
Cloud Recipe (IFTTT)
CloudSystem Automation
YourTestCloud.com
“Heterogeneous test cloud that are open, flexible and extensible”
Testing Cloud
If <Solution Under Test>
Then <Build Cloud 2.0>
Testing Cloud
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
43
@jonathon_wright
#esconfs www.eurostarconferences.com
Test First Delivery
Service / Network Virtualisation as part of Test Driven Delivery integration (TDDi)
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
44
@jonathon_wright
#esconfs www.eurostarconferences.com
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
– Test Automation in the Cloud
– AMMi© Assessment Model
– First Day Automation
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Test Automation as a Service (TAaaS) Model
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
45
@jonathon_wright
#esconfs www.eurostarconferences.comhttp://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf
Test Automation as a Service (TAaaS) Model
Test Automation in the Cloud: Case Study
46
@jonathon_wright
#esconfs www.eurostarconferences.com
AMMi© Assessment Model
The digital journey toward technical delivery enlightenment
AMMi Phase Target Criteria
Level 0 Accelerating Automated Test Lifecycle Methodology (ATLM)
Level 1 Traditional Framework Driven (Gen 1-5)
Level 2 Managed Processes are planned, performed, measured, and controlled
Level 3 Sustaining Automation process defined and validated against international
standards (ISO-29119)
Level 4 Quantified Predictability of automation process performance
Level 5 Optimising Automation process variation and statistical predictability
* Source: AMMi®(http://www.AMMi.org.uk) + Test Automation Patterns (http://TestAutomationPatterns.wikispaces.com)
Test Automation as a Service (TAaaS) Model
47
@jonathon_wright
#esconfs www.eurostarconferences.com
First Day Automation: Test Automation Experiences Book
* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences/, ISBN-10: 0321754069, 9th January 2012
Test Automation as a Service (TAaaS) Model
48
@jonathon_wright
#esconfs www.eurostarconferences.comFunctional Testing as a Service (FTaaS) Model
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service
– Shift everywhere (UP, DOWN, LEFT, RIGHT)
– Continuous, Integration, Build & Delivery
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
49
@jonathon_wright
#esconfs www.eurostarconferences.com
Shift Everywhere
Test in Business, Test in Dev, Test Ops, Test in the Wild
Functional Testing as a Service (FTaaS) Model
Test in Business Test in Dev Test in the Wild
50
@jonathon_wright
#esconfs www.eurostarconferences.com
Functional Testing in UFT 12 and beyond (i.e. CodedFT)
* https://www.packtpub.com/application-development/advanced-qtp-115-test-engineers-cookbook
@jonathon_wright
#esconfsFunctional Testing as a Service (FTaaS) Model
51
@jonathon_wright
#esconfs www.eurostarconferences.com
Continuous, Integration, Build & Delivery
Accelerate time to market
@jonathon_wright
#esconfsFunctional Testing as a Service (FTaaS) Model
52
@jonathon_wright
#esconfs www.eurostarconferences.com
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service
– Performance Testing in the Cloud
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Performance Testing as a Service (PTaaS) Model
53
@jonathon_wright
#esconfs www.eurostarconferences.com
Performance Testing in the Cloud
PTaaS – Microsoft – Visual Studio Team Foundation Service
Performance Testing as a Service (PTaaS) Model
54
@jonathon_wright
#esconfs www.eurostarconferences.com
Performance Testing in the Cloud
@jonathon_wright#esconfs Performance Testing as a Service (PTaaS) Model
PTaaS – HP - StormRunner (TrueAPI/TrueClient)
55
@jonathon_wright
#esconfs www.eurostarconferences.comMobile Testing as a Service (MTaaS) Model
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service
– Mobile Testing in the Cloud
– Continuous, Mobile, Build & Delivery
– High Volume Mobile (HVMAT)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
56
@jonathon_wright
#esconfs www.eurostarconferences.com
Mobile Testing in the Cloud
@jonathon_wright#esconfs
Real Mobile Devices (PerfectoMobile) Test Studio for iOS (Telerik)
* Telerik.com, http://www.telerik.com/mobile-testing, 4th November 2014
Mobile Testing as a Service (MTaaS) Model
57
@jonathon_wright
#esconfs www.eurostarconferences.com
Continuous, Mobile, Build & Delivery
MobileCloud™
@jonathon_wright#esconfs
* PerfectoMobile.com, http://www.perfectomobile.com/resources/papers/shift-app-testing-left-paper, 4th November 2014
Performance Testing as a Service (PTaaS) Model
58
@jonathon_wright
#esconfs www.eurostarconferences.com
High Volume Mobile Automated Testing (HVMAT)
On demand in the cloud featuring Network Virtualisation
*Jonathon Wright, 2013, ‘Test Automation in the Cloud’ for EuroSTAR, Gothenburg, Sweden, 23rd July 2013
Mobile Testing as a Service (MTaaS) Model
59
@jonathon_wright
#esconfs www.eurostarconferences.com
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service
– Security Testing in the Cloud
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Security Testing as a Service (STaaS) Model
60
@jonathon_wright
#esconfs www.eurostarconferences.com
Security Testing in the Cloud
How to prevent epistemic/systemic failure in the cloud from geo-realistic DoS attacks? Application
Security
@jonathon_wright
#esconfs
Infrastructure
Testing
Secure Coding
Mobile Security
Database Security
Information
Security
Our services identify
configuration and
deployment flaws which
may be introduced
when integrating mobile
solutions into an
enterprise environment.
We also perform
manual security testing
and analysis of smart
phone and tablet based
apps. Typically this will
encompass both the
client application and
server side services
supporting the app.
*John Denneny, MD, http://www.pentest.co.uk/whitepapers.html , 5th November 2014
Security Testing as a Service (STaaS) Model
Mobile Security External Infrastructure Security Assessment
Internal Infrastructure Security Assessment
Mobile Application Security Assessment
Firewall Security Assessment
Server Security Assessment
Laptop Security Assessment
Workstation Security Assessment
VOIP Security Assessment
VPN Security Assessment
Wireless Security Assessment
Active Directory Security Assessment
Citrix Security Assessment
Database Security Assessment Service (DSAS)
61
@jonathon_wright
#esconfs www.eurostarconferences.com
Testing as a Service – Models - Summary
BusinessAssurance
BusinessQuality
BusinessTesting
Business Assurance provides the visibility and
transparency throughout the solution delivery lifecycle• Digital Enterprise
• Business on a Page
• Business Delivery Management
• Business Intelligence (Actionable Business Insight)
• Business Innovation (Enterprise Gamification)
Business Quality allows enforceable quality across the
entire solution delivery lifecycle owned by everyone• Business Lifecycle Management
• Business Solution Discovery / Delivery
• Business Enterprise Collaboration
• Business Acceptance Testing
Business Testing
• Testing as a Service - Models
• Solution Delivery Lifecycle integration (SLDCi)
• Test First Delivery / First Day Testing
• Continious, Integration, Build & Delivery
• Shift Everywhere (UP/DOWN/LEFT/RIGHT)
• All Channel Customer Experience
62
@jonathon_wright
#esconfs www.eurostarconferences.com
Hitachi ConsultingHitachi’s Global Management Consulting & IT Services Business.
$740
MillionFY13 Revenue
6,500experienced &
dedicated
professionals
Better Approach. Better Results. Better Experience.
$10 Billion
results
© Hitachi Europe Ltd. 2014. All rights reserved.
@jonathon_wright
#esconfs www.eurostarconferences.com
Questions?
@esconfs#esconfs www.eurostarconferences.com
http://testhuddle.com/forums/topic/testing-as-a-service-models/
EuroSTAR Webinar Archive
Access almost 70 on-demand software testing webinars by the world's leading testing experts. Topics include….
Agile, Automation, Test Management, Process Improvements, Risk-based Testing, Performance Testing, Test Documentation, Cloud Testing, Session-based Testing, Weekend Testers
Visit the archive!
http://www.eurostarconferences.com/community/member/webinar-archive
@esconfs#esconfs www.eurostarconferences.com
The EuroSTAR Community.Are you a member?
Did you know we have an online resource area with almost …
600 software testing presentations, 70 topical webinars, over 40 videos and podcasts and 25 expert eBooks
Do you know that membership is completely free and you can access these premium
testing resources anytime?
Join today! http://www.eurostarconferences.com/community/
member
@esconfs#esconfs www.eurostarconferences.com