T r u s t t h e E x p e r t s

WHITE PAPER

Performance Testing of Card ApplicationsKey Considerations

“If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it”

David Cheriton

Performance is critical to cards-applications from both the acquiring and issuing perspectives. Organizations, especially financial institutions, recognize repercussions of poor performance on customers and end users. Amazon found that every 100 milliseconds of latency cost them 1% in sales.’1

The cards landscape spans a highly complex architecture as it involves a multitude of channels, applications and interfaces across diverse platforms that are integrated with core processing systems. As acquirers and issuers add new layers of security along with advanced fraud detection and risk analysis components, the burden on performance is ever increasing.

Although the average response time between card schemes and banks is between 140 and 500 milliseconds, card-schemes mandate further reduction in response times2,3,4. To improve cardholder experience at the point of sale, the issuer and acquirer response times are limited to 10 seconds after which transactions are timed out.

The overall objective of performance-improving measures is to reduce response times by 25% to 45%. As a consequence, the typical response times within each system should not exceed a second if one were to consider the following factors:

• The network response• Processing of authorization requests in the core systems • Processing of decisions in risk or fraud interface systems

This white paper examines the key considerations, approaches, activities and triggers that drive for performance testing and the challenges and risks involved in improving the performance of card systems and applications.

Introduction

T r u s t t h e E x p e r t s

2

WHITE PAPER

White Paper – Performance testing of card applications

Figure 1: Components in Performance Testing of a Cards Application

The figure depicts the typical components involved in performance testing in cards implementation. The figure shows the transaction and data flows from the Front-End and Back-End. • Front-End constitutes online authorizations and service requests through both self service and operator assisted channels.

• Back-End consists of clearing, settlement and enterprise systems for GL accounting, Data warehousing and interfaces from and to third parties.

T r u s t t h e E x p e r t s

White Paper – Performance testing of card applications 3

WHITE PAPER

Figure 2: Key Considerations for Performance Testing of a Cards Application

Phased and Modular Approach: The performance testing of cards systems involves multiple modules, applications and interfaces or channels. Before testing the performance across the implementation architecture, it is important to

assess the performance of each component. Any risks due to bottlenecks in a particular component which leads to degradation of the performance across the entire implementation architecture can adversely affect the testing schedule.

The key considerations for performance testing are summarized in Figure 2.

Approach to Planning of Performance testing

Acquiring / Issuing Considerations

● Identify the nature of business - acquiring and/or issuing

● Ascertain the nature of On Us/ not On Us transactions for performance

Credit / Debit Considerations

● Identify the nature of portfolio - credit, debit or both

● Ascertain how debit authorizations are handled - by cards application or banking system or frameworks like Base 24?

Online/ Batch Performance Considerations

● Review authorizations & Web access and service requests from online perspective

● Analyze overall batch process vs critical path, all modules vs key modules

Card Technology Considerations

● Assess usage of Magnetic Stripe vs Chip cards for authorizations involving crytogram functions

● Assess PIN based transactions involving PIN translations vs signature based transactions5

Interfaces / Surround systems● Consider inclusion / non -

inclusion of interfaces and surround systems - fraud, rewards, behavioral risk etc

● Considerations for specific fraud rules & triggers/reward programs/ strategies and case managements

Message Formats● Consider specific local/

domestic / native message formats when no off the shelf tools are available

● Consider three leg transactions - e.g. authorizations, approval and confirmation (200 210 202)

It is advisable to follow a phased and modular approach in testing the performance of cards application implementation as depicted in Figure 3.

4White Paper – Performance testing of card applications

WHITE PAPER

I. Discovery sessions help determine the performance testing objectives and should consider: -

• Results of prior performance testing (if any) on any of the components

• List of performance tools used or available• Acceptability on the usage of open source tools in the

environment• Performance issues or bottlenecks experienced till then.

II. Tool Compatibility Study: Performance testing for a card management system includes multiple protocols and applications. Thus, the usage of suite of tools (open source or commercial) would be effective for this domain. It is advisable to ascertain the compatibility of selected tools for the planned performance testing.

III. Volumetric Analysis and Data Profiling helps in establishing the expected volume of transactions, transaction mix and business scenarios for performance

tests. It helps in determining the composition of data required to simulate a production like load for performance testing. This should be done to arrive at the Work Load Matrix (WLM).

WLM is critical to ensuring that performance tests conducted are as close as possible to the production scenarios and anticipated growth. A structured questionnaire to collect the details will help in preparing the WLM for the performance testing.

IV. Strategize: Strategizing performance testing is the next

key activity. The performance test strategy would need to define the performance objectives, scope, approach, requisite environment and tools (established through a tool compatibility study exercise conducted prior to strategy), identify the risks and agree upon mitigation plans. The roles and responsibilities for all stakeholders will need to be clearly defined and agreed during strategy phase.

V. Baseline/Benchmark Test must be carried out before the data volumes are created. This will ensure that bottlenecks (if any) are rectified before the start of the actual Load test runs. The stability of the system and the response times under minimal load is established during this bench marking phase.

VI. Database volume creation is the next key activity after the baseline or benchmark testing. The approach for creating the database volume (creating new data or migrating scrambled production data in to the performance test environment) is typically finalized during the strategy phase. It is important to ensure that data volumes are created as per the portfolio and transaction history requirements specified in the strategy.

VII. Load/Volume/Stress/Endurance Tests are then performed to assess the stability of system under varying loads. The objectives of the each of these are shown in the table.

Component-level performance testing should precede the End-to-End and Joint Performance Test phases. However, component-level testing need not be repeated for cases where the results of a prior test prove that the component already meets the performance objectives. In such circumstances, the components can be considered only during End-to-End and Joint Performance Test phases.

Volumetric Analysis and Data Profiling is the cornerstone for conducting performance testing on cards applications and help in arriving at the throughput requirements for the testing.

• Volumetric analysis helps in establishing the expected volume of transactions, transaction mix and business scenarios for performance testing.

• Data profiling helps in determining the composition of data required to simulate a production like load for performance testing.

For testing Online performance, the considerations should include expected volume of transactions by Product-Scheme-Transaction, Source-Transaction Type, transaction arrival pattern and customer service requests by channel, product, service request type, and frequency of usage in a day, month.

For testing Batch performance, the considerations should include portfolio mix, percentage of accounts cycling in a batch, number of transactions or input files processed and output files generated along with business day processing patterns.

Figure 3: A Phased Approach for Performance Testing of a Cards Application

Key Activities in Performance Testing of Cards Applications

Com

pone

nt E.g., Specific Module for authorizations

All modules part of implementation architecture E.g., Card management system, Fraud interface, Base 24 etc

All modules part of implementation architecture & client systems: E.g., Card management system, Fraud interface, Base 24 & Bank host

Helps measure performance & bottlenecks of each individual components/system

Helps measure degradation of performance when other components are also active

Helps measure degradation of performance when client components are also active

End

to E

nd

Join

t

I. Discovery sessions help determine the performance testing objectives and should consider: -

• Results of prior performance testing (if any) on any of the components

• List of performance tools used or available• Acceptability on the usage of open source tools in the

environment• Performance issues or bottlenecks experienced till then.

II. Tool Compatibility Study: Performance testing for a card management system includes multiple protocols and applications. Thus, the usage of suite of tools (open source or commercial) would be effective for this domain. It is advisable to ascertain the compatibility of selected tools for the planned performance testing.

III. Volumetric Analysis and Data Profiling helps in establishing the expected volume of transactions, transaction mix and business scenarios for performance

5White Paper – Performance testing of card applications

WHITE PAPER tests. It helps in determining the composition of data required to simulate a production like load for performance testing. This should be done to arrive at the Work Load Matrix (WLM).

WLM is critical to ensuring that performance tests conducted are as close as possible to the production scenarios and anticipated growth. A structured questionnaire to collect the details will help in preparing the WLM for the performance testing.

IV. Strategize: Strategizing performance testing is the next

key activity. The performance test strategy would need to define the performance objectives, scope, approach, requisite environment and tools (established through a tool compatibility study exercise conducted prior to strategy), identify the risks and agree upon mitigation plans. The roles and responsibilities for all stakeholders will need to be clearly defined and agreed during strategy phase.

V. Baseline/Benchmark Test must be carried out before the data volumes are created. This will ensure that bottlenecks (if any) are rectified before the start of the actual Load test runs. The stability of the system and the response times under minimal load is established during this bench marking phase.

VI. Database volume creation is the next key activity after the baseline or benchmark testing. The approach for creating the database volume (creating new data or migrating scrambled production data in to the performance test environment) is typically finalized during the strategy phase. It is important to ensure that data volumes are created as per the portfolio and transaction history requirements specified in the strategy.

VII. Load/Volume/Stress/Endurance Tests are then performed to assess the stability of system under varying loads. The objectives of the each of these are shown in the table.

Test Type Definition

Load Test Assessment of system performance and stability for various workload levels

Volume Test Assessment of system performance and stability for various account/ card/ transaction database volume levels, vis-à-vis current database volume and future database volumes based on growth expectations

Stress Test To find out the system break point by increasing the workload levels beyond the peak load

Endurance Test Assessment of reliability and stability of the SUT under various volumes & loads sustained for a given duration of time

VIII. Reporting: Metrics would need to be provided at multiple levels (Web layer, Application layer, Database layer). Recommendations to fine tune the system can be provided based on the system behavior during performance testing.

Triggers for Performance Testing

A performance test may be required when:

• A new application module or component is introduced• A new institution or product is introduced, leading to a

significant change in the existing portfolio, architecture, or environment

• A change is introduced in the flow of information in the existing systems

• An increase in Load/ Volume occurs where performance testing was not carried out earlier.

• A significant change occurs in the code base

• A tuning change is made to address performance issues or bottleneck identified previously

Challenges Most of the challenges that arise in the performance testing of cards applications originate from the complexity of the implementations and the involvement of multiple stakeholders. Critical challenges include:

• Availability of production-like test environment for performance testing

• Simulation of real time business scenarios• Simulation of high database volumes• Pre-requisite activities for each of the parent and child

batch threads / programs• Coordination with multiple stakeholders - Application Development Team - IT Infrastructure - Project Teams - Client Team(s) - Third Party Team(s)

Triggers for Performance Testing

A performance test may be required when:

• A new application module or component is introduced• A new institution or product is introduced, leading to a

significant change in the existing portfolio, architecture, or environment

• A change is introduced in the flow of information in the existing systems

• An increase in Load/ Volume occurs where performance testing was not carried out earlier.

• A significant change occurs in the code base

• A tuning change is made to address performance issues or bottleneck identified previously

Challenges Most of the challenges that arise in the performance testing of cards applications originate from the complexity of the implementations and the involvement of multiple stakeholders. Critical challenges include:

• Availability of production-like test environment for performance testing

• Simulation of real time business scenarios• Simulation of high database volumes• Pre-requisite activities for each of the parent and child

batch threads / programs• Coordination with multiple stakeholders - Application Development Team - IT Infrastructure - Project Teams - Client Team(s) - Third Party Team(s)

5White Paper – Performance testing card applications

WHITE PAPER• Clear PIN/PIN block creation for transaction

authorization requests• Parameter configuration for performance testing

Risks

Risks involved in performance testing are:

• Incompatibility of the tools with the respective components

• Tool limitations• Non availability of test data as per the data

pre-requisites and volumes agreed for each run• Data mismatch between different systems and host for

Joint performance testing• Quality of migrated data• Non availability of production like environment for

performance testing• Data security

Disclaimer: All the documentation and other material contained herein is the property of Thinksoft Global Services and all intellectual property rights in and to the same are owned by Thinksoft Global Services. You shall not, unless previously authorized by Thinksoft Global Services in writing, copy, reproduce, market, license, lease or in any other way, dispose of, or utilize for profit, or exercise any ownership rights over the same. In no event, unless required by applicable law or agreed to in writing, shall Thinksoft Global Services, or any person be liable for any loss, expense or damage, of any type or nature arising out of the use of, or inability to use any material contained herein. Any such material is provided “as is”, without warranty of any type or nature, either express or implied. All names, logos are used for identification purposes only and are trademarks or registered trademarks of their respective companies.

For more details visit, www.thinksoftglobal.com

T r u s t t h e E x p e r t s

• Downtime during test execution due to problems in the Test environment

• Non availability of interfaces • Critical defects in the functional area, observed during

performance testing• Conducting a joint performance testing without / before

completion of E2E performance test

These risks should be addressed across the project.

Conclusion

Performance testing of cards applications needs comprehensive planning. It is also equally important that the performance testing plan identifies the key challenges and risks and includes strategies to mitigate them.

Bibliography1http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it2http://www.mastercard.com/us/company/en/newsroom/pr_mastercard_telefonica.html3http://www.cartes-bancaires.com/spip.php?article584http://www.mastercard.com/us/company/en/newsroom/masterCard_upgrades_global_processing.html5http://www.andyorrock.com/2009/09/analysis-by-transaction-class-at-an-acquirer.html#tp