reinventing application performance testing with service virtualization

Welcome! Reinventing Application Performance Testing with Service Virtualization

August 2016

© 2016 CA. ALL RIGHTS RESERVED. ALL TRADEMARKS, TRADE NAMES, SERVICE MARKS AND LOGOS REFERENCED HEREIN BELONG TO THEIR RESPECTIVE COMPANIES. NO UNAUTHORIZED USE, COPYING OR DISTRIBUTION PERMITTED. PLEASE NOTE THAT THIS PRESENTATION IS FOR YOUR INFORMATIONAL PURPOSES ONLY DOES NOT SERVE AS ANY TECHNICAL USER GUIDES OR DOCUMENTATION. THERE IS NO DIRECT OR INDIRECT WARRANTY PROVIDED PERTAINING THE PRODUCTS AND ACTUAL USE MAY VARY FROM THE ILLUSTRATIONS PROVIDED.

Original presentation made by Nathan Devoll. Recorded version available at: https://www.brighttalk.com/webcast/9305/214105?=s

https://www.brighttalk.com/webcast/9305/214105?=s

2 © 2016 CA. ALL RIGHTS RESERVED.

Agenda

PERSPECTIVE

CHALLENGES WITH TRADITIONAL APPROACHES

REINVENTING PERFORMANCE TESTING WITH CA TECHNOL0GIES

ADVANTAGES AND REAL WORLD EXAMPLES

1

2

3

4


Businesses no longer have the luxury of time. Rising customer expectations, competitive threats, and increased consumer choice have turned rapid delivery and iteration of software applications into a competitive differentiator.

Optimizing the delivery value chain has become the new means to better business outcomes.

TechRadar™: Continuous Software Delivery, Q2 2015, Forrester Research, May 1, 2015

“

”2014 Vanson Bourne study commissioned by CA

UNPRECEDENTED

94%of executives face increased pressure to release appsmore quickly

“The Need for SPEED


DevTestOps

ideaAGILE DEVELOPMENT AGILE OPERATIONS

Establish a flawless customer experience with apps optimized for performance

Agile aligned testing supported with advanced technology

Eliminate constraints and dependencies across the SDLC

CUSTOMER EXPERIENCE AGILE TESTING

Agility across all 3 increases speed, reliability and efficiency

For more read the article “Getting in between Dev and Ops” on CA Highlights.

https://blogs.ca.com/2016/04/07/getting-in-between-dev-and-ops/?intcmp=searchresultclick&resultnum=3

https://blogs.ca.com/2016/04/07/getting-in-between-dev-and-ops/?intcmp=searchresultclick&resultnum=3


Application Performance is as Vital as Function

SpeedThis is taking to long.

40%

Source: Ankit Oberoi. 5 Reasons Visitors Leave Your Website. Website Magazine 2014

3rd Party IntegrationWhy Doesn’t this Work?

mobile subscribers, including 76% of the

smartphone owners, used location-based check-in.

16.7 Million

Source: Comscore

ComplexityWhere is the Problem?

of developers performance test their code before it is

pushed to productionSource: www.apmdigest.com, 15 Top Factors That Impact Application Performance. April 2013

of users will abandon your app if it takes more than three seconds to load.

< 5%

http://www.apmdigest.com/


Why Do Apps Perform Poorly in Production?

Not all Components are Tested1

Dev/Test Environments Don’t Mirror Production2

Systems and Data are not Available for Testing3


Reinventing Performance Testing

Test Design and Coverage – Optimize from the Start!1

Test at the Component Level3

Make Test Environments as “Life-Like” as Possible4

Utilize a Test Data Management Solution 5

Leverage Service Virtualization and Drive Testing at the API Layer2


CA Agile Requirements Designer - Align requirements and optimize test cases

Import User Stories from HP ALM or CA Agile Central to Automatically Create, Visualize and Optimize Test Cases

Automated Test Case GenerationAuto Test Generation Using Existing Tests

Test Design and Coverage – Optimize from the Start!


Leverage Service Virtualization and Drive Testing at the API Layer

-

ESB

Mainframe System ofRecord

ERP SaaS/API

Simulate unavailable systems across the software development lifecycle (SDLC), allowing developers, testers, integration, and performance teams to work in parallel.

I have everything I need, when I need it!“

http://www.ca.com/us/products/ca-service-virtualization.html?intcmp=headernav



Decompose Performance Expectations “Into the Lifecycle”

Verify

0.3s

Lookup

0.5s

Response

1.2sApplication

SLA:2.1 Seconds


Decomposing an SLA into Performance Expectations at a Component Level

Test 1

Component Performance Testing Against VSE

Response

1.2s

Verify

Lookup

Application

0.9s

Virtual Service Environment (VSE)

Virtual Service


Make Test Environments as “Life-Like” as Possible

What’s the Difference Between Stubs, Mocks, and Service Virtualization?


Don’t Skimp on the Data. Surround and Capture!


CA Test Data Manager - Virtual data is generated on the basis of a message definition and injected into a deployed virtual service.

http://www.ca.com/us/products/test-data-management.html


+ Linking Performance Monitoring and Testing


Performance Testing in a Box

Innovate Faster while Delivering a High Quality Experience

Use the same test cases created and optimized for functional testing

Simulate backend load by running test thru an API

Simulate life like test environments Generate purpose built test data Control outlier conditions Adjust think times

Save Millions in infrastructure costs

and 3rd party fees

Detect Defects earlier in the SDLC

Eliminate capacity constraints

for performance testing

Reduce delayswith high-performance

masking and subsetting, and automated data discovery

Application “X”

Test Cases

Thousands oftransactions

Agent


Deliver Innovations… Faster! TIMEWeek 4Week 2Week 1

xWeek 6Week 3

DEV 1 DEV 2 DEV 3DEV 1 DEV 2 DEV 3

DEV 1 DEV 2 DEV 3

DEV 1 DEV 2 DEV 3DEV 1 DEV 2 DEV 3

DEV 1 DEV 2 DEV 3

App 3App 2App 1

Week 7Week 5

Sequential

ParallelApp 3App 2App 1

AccelerateApplication

Delivery

Simulate needed systems … end-to-end

Create virtual services … easily

Increase developer productivity

1 2 3

Cut of Set up time for dev environments (from 42 days to 3)

Reduced Time-to-Market by for key applications.

http://bcove.me/4rx4wt6x30 Days

15%http://bit.ly/SV-TelefonicaChile

http://bit.ly/SV-TelefonicaChile


99% reduction in testing times-------------------

25% reduction in software defects

BusinessAutotrader sells advertising that allows private and trade buyers and sellers to exchange vehicles. More than 18 million people use Auto- Trader.com each month.

ChallengeTo meet customer demand for new online features and accessible across multiple devices, Autotrader needed to release software updates on a weekly basis. Testing, however, had become complex, costly and time-consuming.SolutionCA Service Virtualization enables Autotrader to virtualize and test new services and apps on a multitude of devices, while simulating both apps working normally and while experiencing performance issues. CA Services completed implementation and training in just six weeks.

Autotrader Avoids $300,000 in Testing

Costs with CA Service Virtualization

Autotrader is improving the customer experience with faster time-to-market.

“The CA Technologies team blew the competition away with their

knowledge and passion.”


Qualica Supports Rapid and High Quality Parallel Development with CA Service Virtualization

CA Service Virtualization has become the nucleus of parallel development, and being able to design a process that builds in quality has improved the efficiency of test processes.

“CA Service Virtualization allows us to reduce the total amount

of time spent on development.”

Sharing knowledge and expertise made

easier------------------------

Creates high-quality systems in a shorter

timeframe

BusinessQualica Inc. handles information systems for Komatsu. As part of the IT Holdings Group, it provides business systems, individually developed software packages and cloud services, primarily for the manufacturing, logistics and service industries.ChallengeQualica needed to create a new development infrastructure that guarantees quality and speed.SolutionCA Service Virtualization virtualizes services to simulate the partner system’s behavior and data; it automates testing, which helps automate user interfaces on mobile devices; thirdly, the load test function not only places a load on the application being tested, but also simulates loading on the applications that it is linked to, and provides a virtual response.


CA Service Virtualization: Real ValueForrester Total Economic Impact (TEI) Study for CA Service Virtualization

Based on three-year, risk-adjusted financial model for the composite organization

“So without Service Virtualization, as highly integrated as our systems are, there is no way we could have speeded things up and moved to an Agile methodology.”— Senior manager, software engineering

“In order to do performance testing required, we were able to virtualize a lot of back-end systems and hardware simulation, avoiding $1.8 million.”— AVP, quality assurance

The Total Economic Impact™ Of CA Service Virtualization, a commissioned November 2015 study conducted by Forrester Consulting on behalf of CA Technologies

Service Virtualization can help improve software quality, speed up time-to-market, and reduce costs.

ROI273%

NPV$4.30M

PAYBACK3.2 mos.

IMPROVED RELIABILITYIdentifying 150 defects earlier in QA per release

IMPROVED PRODUCTIVITY640 hours of savings in Dev/QA processes per release


CA Continuous Delivery Portfolio

Development OperationsTestPlanning Deploy

CA Agile Requirements Designer

CA Test Data Manager

CA Continuous Application Insight

CA Application Test

CA Release Automation

Agile Ops

Agile Central

CA Service Virtualization

CA Release Automation Standard + Continuous Delivery Edition

APIM APM

http://www.ca.com/us/products/ca-agile-requirements-designer.html

http://www.ca.com/us/products/ca-agile-requirements-designer.html

http://www.ca.com/us/products/test-data-management.html

http://www.ca.com/us/products/ca-application-test.html

http://www.ca.com/us/products/ca-application-test.html

http://www.ca.com/us/products/ca-agile-central.html




http://www.ca.com/us/products/ca-release-automation.html

http://www.ca.com/us/products/api-management.html?intcmp=headernav

http://www.ca.com/us/products/application-performance-monitoring.html


Additional Resources

Next Generation Performance Testing Executive Brief – http://bit.ly/2alTsrrForrester SV TEI Study – http://bit.ly/CASV-TEI Service Virtualization Today – www.servicevirtualization.comGartner Market Guide for API Testing and Virtualization – http://bit.ly/Gartner_SV-GuideKey Capabilities of CA Service Virtualization Whitepaper - http://bit.ly/CASV_WPReal Customers. Real Results – Service Virtualization customer success stories www.ca.com/CDResultsIT Central Station – Real User Reviews and Comparison Guide https://www.itcentralstation.com/products/ca-service-virtualizationCA Service Virtualization Home Page - http://www.ca.com/us/products/ca-service-virtualization.html?intcmp=headernav

http://bit.ly/2alTsrr

http://bit.ly/2alTsrr

http://bit.ly/CASV-TEI

http://www.servicevirtualization.com/

http://bit.ly/Gartner_SV-Guide



http://bit.ly/CASV_WP



http://www.ca.com/CDResults

https://www.itcentralstation.com/products/ca-service-virtualization

https://www.itcentralstation.com/products/ca-service-virtualization


reinventing application performance testing with service virtualization

Technology