pre-con education: shift-left performance testing for shift-forward quality

World®’16

Shift-LeftPerformanceTestingforShift-ForwardQualityKarlMiller- TechnicalAccountManager- CATechnologies

DO5X10E

DEVOPS

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

Traditionalperformancetestingtypicallyrequiresthatallcomponentsoftheapplicationare“completed,”integratedanddeployedintoanappropriateenvironment.Thisresultsintestingnotbeingdoneuntillateinthedeliverycycleorsometimesskippedentirely.Whichcanthenleadtoalessthenoptimaluserexperience,expensivereworkandpotentiallossofbusiness.Manyorganizationshaveadoptedservicevirtualizationtoovercomethekeychallengesassociatedwithperformancetesting.Duringthissessionwe’llshowyouwhyandspecificallyhowservicevirtualizationenablesyoutodotestingearlyinthedevcyclebysimulatingunavailableproductionsystemsandmissingcomponents,helpsyoucontroltheinputs(likeresponsetimesand3rdpartysystemresponses)soyoucandomorenegativeandexploratorytesting,andenablestestinganddevelopmenttoworkinparallel.

KarlMiller

CATechnologiesTechnicalAccountManager,ContinuousDelivery


Agenda

PERSPECTIVE

CHALLENGESWITHTRADITIONALAPPROACHES

QANDA

REINVENTINGPERFORMANCETESTINGWITHCATECHNOL0GIES

ADVANTAGESANDREALWORLDEXAMPLES

DEMO

1

2

3

4

5

6


DevTestOps

ideaAGILEDEVELOPMENT AGILEOPERATIONS

Establishaflawlesscustomerexperiencewithappsoptimizedforperformance

Agilealignedtestingsupportedwithadvancedtechnology

EliminateconstraintsanddependenciesacrosstheSDLC

CUSTOMEREXPERIENCEAGILETESTING

Agilityacrossall3increasesspeed,reliabilityandefficiency


WhyDoAppsPerformPoorlyinProduction?

NotallComponentsareTested1

Dev/TestEnvironmentsDon’tMirrorProduction2

SystemsandDataarenotAvailableforTesting3


ReinventingPerformanceTesting

VirtualizeDependenciesofSystemUnderDevelopment/Test1

LeverageServiceVirtualizationandDriveTestingattheAPILayer3

MoveFunctionalVirtualServicestoPerformanceServer4

ConvertFunctionalTeststoPerformance/ConcurrencyTests5

CreateAutomatedFunctionalTestsattheComponentLevel2

PerformanceTestSystemUnderDevelopmentbackedbyVirtualServicesbeyondPeakLoads6


ServiceVirtualizationforPerformanceTestingofMultipleLayers

APPLICATIONSERVER

SystemUnderTest

LoadTestingTools

SystemUnderTest

LoadTestingTools

AllOperationsVirtualized ESB

LEGACYINTERFAC

E

ORACLE

DB2I-SERIES

MAINFRAME

JDBCOracle

JDBCDB2

TCP/IPwithXMLResponse

TCP/IPFixedLength

HTTPSCustomDataProtocol


BenefitstoPerformanceTestingVirtualServices+currenttestingtools

CHALLENGECreatingalabcapableofhandlingproductioncapacityloadsmaynotberealisticallypossible.

APPLICATION“X”

CURRENTTESTINGTOOLS

Thousandsoftransactions

BillMeLater

PayPal

TNTExpress

3rd PartyService

LoadTestattheComponentLevel

TestforVolumeandCapacityIndependently

LocateBottlenecksearly

Virtualize3rdPartySystems,MainframeData:AlwaysReady,HighlyScalable

Immune toAvailabilityandConstraintIssues

Varytheresponsetimesfromthevirtualservices!

RunthousandsoftransactionsthroughtheactualapplicationontheactualhardwarewithNOIMPACTonyourexistingsystems!


DecomposePerformanceExpectations“IntotheLifecycle”

Verify

0.3s

Lookup

0.5s

Response

1.2sApplication

SLA:2.1Seconds


DecomposinganSLAintoPerformanceExpectationsataComponentLevel

Test1

ComponentPerformanceTestingAgainstVSE

Response

1.2s

Verify

Lookup

Application

0.9s

VirtualServiceEnvironment(VSE)

VirtualService


MakeTestEnvironmentsas“Life-Like”asPossible

What’stheDifferenceBetweenStubs,Mocks,andServiceVirtualization?


Don’tSkimpontheData.SurroundandCapture!Whyis90%oftheITDevelopment&Testcostweighingdowndelivery?

In-ScopeSystem:Environmentsunderyourteam’scontrol

Out-ScopeSystem:Data,servicesandappsyoudependon

=1 =10-1000x

Dev&TestTeams

SystemunderDev

In-ScopeData

App1

App2

App3

App4

App5

Saas

Out-of-ScopeDependencies


VirtualDataisGeneratedontheBasisofaMessageDefinitionandInjectedintoaDeployedVirtualService


+LinkingPerformanceMonitoringandTesting


VirtualServicesSupportPerformanceModelVirtualServiceModelprovidescompanionsthatcanbeconfiguredforVirtualServiceThinkScale,BatchResponseandRecurringThinktimescaletovarytheperformanceoftheVirtualServiceoveraperiod.

CompanionsforPerformancemodelinVirtualServiceModel


RegulateVirtualServiceResponseTimeCAServiceVirtualizationcanregulaterequesttimingandsimulateslownetwork.

Thinktimespec,specifiesthelatencyinms

Thinktimespecspecifiestheresponselatency.ThiscanbescaledusingThinktimescaleduringdeploymentorusingCompanions.HighThinktimespecorThinktimescalecansimulateslownetworkbehavior.


Demo

Stayconnectedatcommunities.ca.com


SaveMillionsininfrastructurecostsand

3rdpartyfees

DetectDefectsearlierintheSDLC

Eliminatecapacityconstraintsforperformancetesting

Reducedelayswithhigh-performance

maskingandsubsetting,andautomateddatadiscovery

Application“X”

TestCases

Thousandsoftransactions

Agent

PerformanceTestinginaBoxInnovateFasterwhileDeliveringaHighQualityExperience

ü Usethesametestcasescreatedandoptimizedforfunctionaltesting

ü SimulatebackendloadbyrunningtestthruanAPI

ü Simulatelifeliketestenvironmentsü Generatepurposebuilttestdataü Controloutlierconditionsü Adjustthinktimes


DeliverInnovations…Faster!TIMEWeek4Week2Week1

xWeek6Week3

DEV1 DEV2 DEV3DEV1 DEV2 DEV3

DEV1 DEV2 DEV3

DEV1 DEV2 DEV3DEV1 DEV2 DEV3

DEV1 DEV2 DEV3

App3App2App1

Week7Week5

Sequential

ParallelApp3App2App1

AccelerateApplicationDelivery

Simulateneededsystems…end-to-end

Createvirtualservices…easily

Increasedeveloperproductivity

1 2 3

CutofSetuptimefordev environments(from42daysto3)

ReducedTime-to-Marketbyforkeyapplications.

http://bcove.me/4rx4wt6x30Days

15%http://bit.ly/SV-TelefonicaChile


99% reduction in testing times-------------------

25% reduction in software defects

BusinessAutotrader sells advertising that allows private and trade buyers and sellers to exchange vehicles. More than 18 million people use Auto-Trader.com each month.

ChallengeTo meet customer demand for new online features and accessible across multiple devices, Autotrader needed to release software updates on a weekly basis. Testing, however, had become complex, costly and time-consuming.SolutionCA Service Virtualization enables Autotrader to virtualize and test new services and apps on a multitude of devices, while simulating both apps working normally and while experiencing performance issues. CA Services completed implementation and training in just six weeks.

Autotrader Avoids $300,000 in Testing Costs with CA Service Virtualization

Autotrader is improving the customer experience with faster time-to-market.

“The CA Technologies team blew the competition away with their

knowledge and passion.”


CAContinuousDeliveryPortfolioDevelopment OperationsTestPlanning Deploy

CAAgileRequirementsDesigner

CATestDataManager

CAContinuousApplicationInsight

CAApplicationTest

CAReleaseAutomation

AgileOps

AgileCentral

CAServiceVirtualization

CAReleaseAutomationStandard+ContinuousDeliveryEdition

APIM APM


Questions?


Resources

§ NextGenerationPerformanceTestingExecutiveBrief– http://bit.ly/2alTsrr§ ForresterSVTEIStudy– http://bit.ly/CASV-TEI§ ServiceVirtualizationToday– www.servicevirtualization.com§ GartnerMarketGuideforAPITestingandVirtualization –

http://bit.ly/Gartner_SV-Guide§ KeyCapabilitiesofCAServiceVirtualizationWhitepaper- http://bit.ly/CASV_WP§ RealCustomers.RealResults– ServiceVirtualizationcustomersuccessstories

www.ca.com/CDResults§ ITCentralStation– RealUserReviewsandComparisonGuide

https://www.itcentralstation.com/products/ca-service-virtualization§ CAServiceVirtualizationHomePage - http://www.ca.com/us/products/ca-

service-virtualization.html?intcmp=headernav


RecommendedSessions

SESSION# TITLE DATE/TIME

DO5T01SKeynote:End-to-EndIntegratedContinuousDeliveryEcosystem:HowtoDriveInnovationandDifferentiation

11/16/2016at10:30am

DO5T02PLeadershipPanel:DevOpsLeadersDiscusstheJourneytoContinuousDelivery

11/16/2016at01:45pm

DO5T03PLeadershipPanel:ContinuousDeliveryintheFinancialServicesIndustry

11/16/2016at04:30pm


MustSeeDemos

ModernizeAppDeliveryIntegratedCDTheater5- DOV501P

DeliverTestDataFasterTestDataManagerTheater5- DOV511P

DeliverBetterAppsServiceVirtualizationTheater5- DOV507P

OrchestrateYourReleaseReleaseAutomationTheater5- DOV513P


Thankyou.

Stayconnectedatcommunities.ca.com

pre-con education: shift-left performance testing for shift-forward quality

Technology