john burgess redpaper - ibm
TRANSCRIPT
ibm.com/redbooks Redpaper
Front cover
IBM CICS Performance Series: FiTeq Authenticator Benchmark
John BurgessChris Hui
Simon MaJohn Weber
Performance and scalability benchmark methodology explained
Multiple configurations tested and the results compared
Benchmark driven to 8,000 CICS transactions per second
International Technical Support Organization
IBM CICS Performance Series: FiTeq Authenticator Benchmark
August 2014
REDP-5114-00
© Copyright International Business Machines Corporation 2014. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP ScheduleContract with IBM Corp.
First Edition (August 2014)
This edition applies to z/OS V1R13, CICS V5R1, and FiTeq Authenticator.
Note: Before using this information and the product it supports, read the information in “Notices” on page v.
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiAuthors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiNow you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiStay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 2. Benchmark objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Chapter 3. Benchmark application topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.1 The Listener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2 The Concurrent Child Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.3 Calling the Thales Hardware Security Module Simulator . . . . . . . . . . . . . . . . . . . . . . . . 83.4 The client simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.5 FiTeq Authenticator: VSAM file access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Chapter 4. Scaling CICS applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.1 TCP/IP port sharing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 Sharing VSAM files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.3 CICS and RLS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.3.1 Components of RLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.4 CICS Threadsafe applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 5. Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.1 Server hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.3 Client simulation hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.4 Hardware security module simulation hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Chapter 6. Measurement methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216.1 RMF MON I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.2 CICS performance monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.3 CICS interval statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.4 System load points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Chapter 7. Terms used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Chapter 8. Single CICS region configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Chapter 9. Single region results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279.1 Results for a single region without the FiTeq Validation Log . . . . . . . . . . . . . . . . . . . . 289.2 Results for a single region with the FiTeq Validation Log . . . . . . . . . . . . . . . . . . . . . . . 289.3 Example RMF report extract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289.4 CICS PA Wait Analysis for a single region without the Validation Log . . . . . . . . . . . . . 299.5 CICS PA Wait Analysis for a single region with the Validation Log. . . . . . . . . . . . . . . . 30
Chapter 10. Multiple CICS region configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
© Copyright IBM Corp. 2014. All rights reserved. iii
Chapter 11. Multiple region results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3511.1 Results for multiple regions without the FiTeq Validation Log. . . . . . . . . . . . . . . . . . . 3611.2 Results for multiple regions with the FiTeq Validation Log . . . . . . . . . . . . . . . . . . . . . 3611.3 CICS PA Wait Analysis for multiple regions without the Validation Log . . . . . . . . . . . 3611.4 CICS PA Wait Analysis for multiple regions with the Validation Log. . . . . . . . . . . . . . 37
Chapter 12. Tuning and configuration considerations . . . . . . . . . . . . . . . . . . . . . . . . . 39
Chapter 13. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Appendix A. Single and multiple region scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43A.1 Single region scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44A.2 Multiple region scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
iv IBM CICS Performance Series: FiTeq Authenticator Benchmark
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.
Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.
© Copyright IBM Corp. 2014. All rights reserved. v
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
CICS®DB2®IBM®
Redbooks®Redpaper™Redbooks (logo) ®
Resource Measurement Facility™RMF™z/OS®
The following terms are trademarks of other companies:
Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
The following terms are trademarks of FiTeq in the United States, other countries, or both:
Smarter Card™ FITEQ TRANSACTION SPECIFIC CODE™ (FTSC)
vi IBM CICS Performance Series: FiTeq Authenticator Benchmark
Preface
FiTeq is an IBM® Business Partner that specializes in fraud prevention technologies for the payments industry. This IBM Redpaper™ publication records the methodologies and results of a performance benchmark using the FiTeq Authenticator, which is a component of FiTeq’s family of Secure Transaction Solutions.
The FiTeq Authenticator is an IBM CICS® enabled application that was run under CICS Transaction Server for z/OS® V5.1 in this benchmark. The performance benchmark was conducted as a joint venture between IBM and FiTeq in January 2014.
In summary, the following FiTeq Authenticator application performance characteristics were demonstrated:
� A scalable solution: CPU usage scales linearly as the number of transactions per second increases.
� Cost-effective: Approximately only 500 microseconds of CPU per transaction were used for the single configuration.
� Efficient: Average response times below 20 milliseconds per transaction were maintained at a transaction rate exceeding 8,000 per second.
These benchmark test results have confirmed and validated that the FiTeq Authenticator is, in conjunction with the performance, reliability, and scalability provided by IBM z/OS and CICS architectures and associated hardware, fully capable of satisfying the requirements of all top financial institutes.
As a by-product of the FiTeq Authenticator performance test, the IBM World-Wide Solutions-Cross ISV Sizing team developed a FiTeq Authenticator Sizing Tool to forecast system requirements based on the transactions per second (TPS) and other system requirements of any future FiTeq client. As a result, the IBM pre-sale team and the FiTeq marketing team will be able to recommend the best fit and most cost-effective IBM software and hardware solution for a particular FiTeq client.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations, such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Authors
This paper was produced by a team of specialists from around the world working at the International Technical Support Organization, Raleigh Center.
John Burgess is a senior software engineer working in the IBM Hursley Laboratory. He is a member of the CICS performance team specializing in the performance of large systems.
© Copyright IBM Corp. 2014. All rights reserved. vii
Chris Hui is a software programmer at FiTeq in California. He has four years of experience in developing software for the credit card industry, and before that he graduated from USC with a Master’s degree in Computer Science. His areas of expertise include smart card personalization, databases, hardware security modules, and smart chip programming. He is proficient in many programming languages, especially C# and C++.
Simon Ma is a Director of Development at FiTeq in California, US. He has 30 years of experience in the financial and networking fields. He has worked at FiTeq for four years. His areas of expertise include system architecture, networking, online banking, and card personalization.
John Weber is a development director and software developer at FiTeq in California. He focuses on the back-end authentication of the payment card process, which includes the areas of CICS, databases, hardware security modules, and sockets.
Now you can become a published author, too!
Here’s an opportunity to spotlight your skills, grow your career, and become a published author—all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our papers to be as helpful as possible. Send us your comments about this paper or other IBM Redbooks® publications in one of the following ways:
� Use the online Contact us review Redbooks form found at:
ibm.com/redbooks
� Send your comments in an email to:
� Mail your comments to:
IBM Corporation, International Technical Support OrganizationDept. HYTD Mail Station P0992455 South RoadPoughkeepsie, NY 12601-5400
viii IBM CICS Performance Series: FiTeq Authenticator Benchmark
Stay connected to IBM Redbooks
� Find us on Facebook:
http://www.facebook.com/IBMRedbooks
� Follow us on Twitter:
http://twitter.com/ibmredbooks
� Look for us on LinkedIn:
http://www.linkedin.com/groups?home=&gid=2130806
� Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter:
https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
� Stay current on recent Redbooks publications with RSS Feeds:
http://www.redbooks.ibm.com/rss.html
Preface ix
x IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 1. Introduction
The FiTeq Authenticator is the back-end software piece of the FiTeq Solution that authenticates FiTeq’s Smarter Card. FiTeq’s Smarter Card generates the patented FiTeq Transaction Specific Code (FTSC) by a Europay, MasterCard, and Visa (EMV)-approved processor, enabling a dynamic authentication for every transaction. The FiTeq Authenticator is an IBM Customer Information Control System (CICS) Transaction Server application that is integrated with the issuer’s authorization platform and is accessible by a COBOL application programming interfaces (API) CALL to authenticate the FTSC embedded in the authorization request message. The FiTeq Authenticator supports two APIs: the card present API and the card not present (CNP) API. Because each FTSC is dynamically generated and good for only one transaction, and then expires, if the FTSC can be authenticated by the FiTeq Authenticator, it ensures that a genuine FiTeq Smarter Card is being used to make the transaction.
For a card present (CP) transaction, when a FiTeq Smarter Card is used at a conventional magstripe terminal, the one-time FTSC is embedded within the Track 2 data and submitted as part of the International Organization for Standardization (ISO) 8583 message sent from the merchant, through the Interchange Network, and to the issuer’s authorization system. The authorization system detects the Bank Identification Number (BIN) and subBIN as a FiTeq Smarter Card and calls the card present API with the Track 2 data for the FiTeq Authenticator to authenticate the FTSC. If the API returns a zero (0), the FTSC is authenticated and the issuer authorization system will then make the decision to approve or deny based on available funds.
1
© Copyright IBM Corp. 2014. All rights reserved. 1
For a card not present transaction (Ecommerce or phone order), when a FiTeq Display Smarter Card is used, a three-digit or four-digit FTSC will be shown on the on-card display to be used in lieu of the static card security code (CID), card verification code (CVC2), or card verification value (CVV2). Consider Internet shopping as an example. The one-time FTSC will be passed using the Card Security Code field submitted as part of the ISO 8583 message sent from the merchant, through the Interchange Network, and to the issuer’s authorization system. The authorization system detects the Bank Identification Number (BIN), which is the first six digits of an account number), and subBIN (the next two digits after the BIN of an account number) as a FiTeq Smarter Card and calls the card not present API with the Card Security Code (which is the card not present (CNP) FTSC), account number, expiration date, and so on) for the FiTeq Authenticator to authenticate. If the API returns a zero (0), the CNP FTSC is authenticated and the issuer authorization system will then make the decision to approve or deny based on available funds.
Figure 1-1 shows the data flow of the FiTeq Solution with a card present transaction and a card not present transaction.
Figure 1-1 Data flow of FiTeq Solution with Card Present (CP) and Card Not Present (CNP) transactions
Internet
MerchantAcquirer
Track2 (w FTSC)
Track2
Home PCs
Online Merchant
ISO 8583 (w Track2)
Interchange Network
Issuer’s Card Authorization
SystemFiTeq Authenticator
HSM
1a) API Call
Issuer’s Authorizer(Detects FiTeq Card based on BIN & sub-BIN)
AuthorizationDecision System
ISO 8583
1 8 8 9
Dynamic CID/CVC2/CVV2 is displayed (up to 4 digits) for
CNP transactions
ISO 8583 (PAN, Expiration Date,dynamic CID/CVC2/CVV2)
Where: API – Application Programming Interface; CNP – Card Not Present; CP – Card Present; CV: Customization Variable (0-3); FTSC: FiTeq Transaction Specific CodeTM; HSM: Hardware Security Module; PAN: Primary Account Number; SIB: Secret Information Block (0, 3-4); TCB: Transaction Counter Block (0-4); TIB: Transaction Information Block(0-2) (a.k.a. device ID, PAN Sequence Number or Sub-account ID);
1b) Final Decision
Track2
POS Exp. Service Date Code TIB CV TCB SIB
FiTeq Card generates a FiTeq Transaction Specific Code (FTSC) (consisting of TIB, CV, TCB, & SIB) populated in track2 discretionary data
; Account # = 1 5 0 3 1 0 1 0 1 5 8 64 73 8 ? x x x x
FIREWALL FIREWALL
FIR
EW
ALL
FIREWALL FIREWALL
FIR
EW
ALL
AuthenticatorDatabase
2 IBM CICS Performance Series: FiTeq Authenticator Benchmark
An FTSC is a dynamically generated, encrypted authentication code. A new FTSC is generated for each new transaction, and it is only valid for a specific transaction and for a specific card (and cardholder if a PIN is given). The FTSC consists of up to four sub-fields: TIB, CV, TCB, and SIB. Each sub-field is field position and field length configurable by the issuer. (A field length of zero means that the sub-field is omitted.) The FTSC is stored in the discretionary data field as the card swipe occurs and is passed from the FiTeq Smarter Card to the FiTeq Authenticator software for authentication during transaction authorization. If the FTSC is authenticated by the FiTeq Authenticator, the transaction is assumed to be conducted by a genuine FiTeq Smarter Card. The dynamic factors in the FTSC increase the level of confidence of the authentication.
Chapter 1. Introduction 3
4 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 2. Benchmark objectives
This benchmark has the following objectives:
� Ensure that the FiTeq Authenticator scales within a single region in a linear fashion.
� Ensure that the FiTeq Authenticator scales in a linear fashion when the transactions are distributed across multiple CICS regions.
� Demonstrate that the FiTeq Authenticator is fully capable of satisfying the requirements of all top financial institutions in terms of transactions per second (TPS).
� Provide capacity planning information for potential clients so that they can adequately size their hardware and software requirements.
Specifically, the FiTeq Authenticator performance evaluation has the following objectives:
� Collect the FiTeq Authenticator’s baseline response time in milliseconds per transaction for a single CICS region and multiple CICS regions under various system loads ranging from 500 TPS, 1,000 TPS, 2,000 TPS, 4,000 TPS, and up to 8,000 TPS.
� Calculate the average CPU used per transaction across the various loads.
This FiTeq Authenticator benchmark focused on the “Card Present” type of transactions. The “Card Not Present” transaction was not evaluated during this time and did not form any part of this report.
IBM Benchmark Centers provide proof of technology, proof of concept, and benchmark capabilities to help clients understand the ways that IBM systems and storage solutions can help them. This benchmark was conducted at the IBM Poughkeepsie Benchmark Center. For more information about IBM Benchmark centers, go to this website:
http://www.ibm.com/systems/services/benchmarkcenter/
2
© Copyright IBM Corp. 2014. All rights reserved. 5
6 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 3. Benchmark application topology
The FiTeq Authenticator code can be executed by issuing either a Program Call or an EXEC CICS LINK. It authenticates the client transaction based on the track data (account number, expiration date, sequence number, FiTeq Transaction Specific Code, transaction counter, and so on). The track data is passed by the Caller and shared secrets by calling the hardware security module (HSM) to encrypt and validate the FiTeq Transaction Specific Code. The FiTeq Authenticator code then returns the result code.
Communications Server IP CICS Sockets was the method chosen for this benchmark as the way to receive the data into CICS for FiTeq authentication.
3
© Copyright IBM Corp. 2014. All rights reserved. 7
3.1 The Listener
Communications Server IP CICS Sockets has two main programming models: the Iterative Server and the Concurrent Child Server. The programming model used here was the Concurrent Child Server.
With this model, a long-running, CICS transaction issues the appropriate TCP/IP calls to listen on a known port specified in the configuration file and waits for incoming connection requests. When an incoming connection request arrives, the Listener accepts it and obtains a new socket to pass to a CICS Child Server application program. The Listener program then issues an EXEC CICS START for the CICS Child transaction passing data in a temporary storage queue and using the GIVESOCKET to initiate the passing of the socket. The Child transaction issues a TAKESOCKET request to take ownership of the socket and issues an EXEC CICS RETRIEVE to access the data being passed from the client. The Listener waits for the Child Server transaction to take the new socket and then closes the socket. When this occurs, the receiving application assumes ownership of the socket and the Listener does not listen any longer.
3.2 The Concurrent Child Server
Each new Child transaction performs the following steps:
1. Issues EXEC CICS RETRIEVE to accept data passed by the EXEC CICS START. This data includes the socket descriptor and the concurrent server client ID, as well as optional additional data from the client.
2. Calls TAKESOCKET to take the socket from the Listener program.
3. Calls READ socket to continue the conversation with the client.
4. Calls the FiTeq Authenticator passing the track data. The FiTeq Authenticator communicates with the hardware security module (HSM) over TCP/IP. In this benchmark, the HSM is the Thales HSM Simulator.
5. Calls SEND to send the ISO8583 response message back to the requester client.
6. Calls CLOSE to close the socket.
3.3 Calling the Thales Hardware Security Module Simulator
The Child transaction that has been started by the Listener makes a call to the FiTeq Authenticator API, which calls the HSM to verify authentication. This call to the HSM is done via a configurable number of long-running CICS transactions that have established persistent TCP/IP connections with a number of HSMs. These long-running transactions are referred to as Channels. The application selects an available Channel and issues an EXEC ENQ to reserve its usage for this transaction until the response from the HSM has been received. It then uses EXEC CICS POST to wake up the Channel and uses shared storage to pass the data to it. The transaction then issues an EXEC CICS WAIT, which will be posted by the Channel when the response from the HSM has been received.
Note: All response times documented in this paper include the time spent communicating with the Thales HSM Simulator. Clients must evaluate and adjust their expectations based on the performance characteristics of the HSM model that is actually used.
8 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Figure 3-1 shows a sample FiTeq Authenticator Resource Manager managing six TCP logical Channels and two HSMs.
Figure 3-1 Authenticator resource manager manages multiple TCP logical channels and HSMs
The following numbers refer to Figure 3-1:
0. Initialization for each TCP Daemon (TCPD):
0a. Each TCP Daemon controls one logical channel (channel#) and connects to one HSM (each HSM supports one IP and up to 64 connections).
0b. Set up socket ID and status.
0c. Call WAIT EVENT (SendPtr [channel#]) to wait for an HSM request at 2b (Go to step 3).
1. Queue incoming HSM request:
1a. Get the next channel number (for example, iCH) in a round-robin fashion.
1b. Call ENQ to queue the HSM request on the assigned channel, for example, iCH.
2. Set up HSM command:
2a. Store HSM command and parameters.
2b. Call POST to wake up the TCPD waiting on this channel (at 0c).
2c. Call WAIT EVENT (RecvPtr [iCH]) to wait for the HSM command to complete (at 3e).
(When resumed) Read the HSM response and return code, call DEQ of this channel, and then return.
3. TCP Daemon forwards HSM command (continue from 0c):
3a. Send HSM command to the HSM via existing TCP/IP connection.
3b. Receive HSM response.
3c. Save HSM response.
3d. Set return code.
3e. Call POST to wake up the HSM requester at 2c.
3f. Go to 0c to wait for the next HSM request.
HSM#1(3a)
Global CICS Shared MemorygSockD &
Socket StatusiCH
(0b)0
1
2
3
4
Queued Transactions(ENQ/DEQ)
TCPD0
TCPD4
TCPD3
TCPD2
TCPD1
SendPtr
(0c), (2b)
(1a)
HSM Cmd
(2a)
RecvPtr
(2c), (3e)
Ret Code
(3d)
HSM Resp
(3c)
HSM#25 TCPD5
NextChannel#
(1b)
TCP Daemons (0a)
(3b)
Chapter 3. Benchmark application topology 9
3.4 The client simulation
Up to 10,000 clients making FiTeq authentication requests were simulated using a FiTeq developed Load Test Client, which is a multithreaded Microsoft Windows GUI application running on 64-bit Windows servers.
IBM Workload Simulator for z/OS can be used to simulate the client devices. However, because FiTeq already invested in their own network simulation, it was thought more efficient to proceed into the benchmark with software they were familiar with rather than switching to the unfamiliar.
Three Windows servers were in the IBM Benchmark Center and locally connected to the IBM logical partition (LPAR) under test through a 1 GB LAN. Running on this system, the Load Test Client software was capable of generating a load of 4,000 transactions per second (TPS). Each server is a 64-bit Windows server, at least Quad-core (2GHz+ each CPU), 64-GB memory, and 3-GB disk.
The Load Test Client tool allows the user to specify the following for each thread:
1. Target IP address and port.
2. Message rate (m number of transactions per n seconds).
3. Total number of messages.
4. Random back-off (when checked, the initial start time (sending the first message) for each thread will be spaced out via the random number between 0 and the average wait time). For example, if sending two messages per second, each thread will send one message every 0.5 seconds (which is the average wait time), then the random wait will be between 0 sec (no back-off) and 0.5 seconds.
When the Load Test Client is started, the main parent thread will perform the following tasks:
1. For each track 2 file in its working folder, start one thread.
2. Each thread emulates one Point of Sale (POS) terminal by performing the following tasks:
a. Read one line at a time of the track 2 file to construct one ISO8583 authorization request message.
b. Perform random back-off, if needed.
c. Connect to the target IP and port.
d. Call Block SEND to send the ISO8583 Authorization request message (137 bytes).
e. Call Block RECV to wait for the ISO8583 response message (106 bytes).
f. Parse the response message and log the internal FiTeq Authenticator API response time in milliseconds.
g. Close the TCP connection.
h. Based on the user-configured message rate and counting the network delay, wait the correct number of milliseconds and go to step a to send the next message.
i. Step a through h will be repeated until the pre-configured total number of messages is sent.
j. At the conclusion of the execution, the parent thread will report the overall response time and the FiTeq Authenticator API’s internal response time in milliseconds.
10 IBM CICS Performance Series: FiTeq Authenticator Benchmark
3.5 FiTeq Authenticator: VSAM file access
All benchmarks were conducted with 500,000 customer accounts preloaded into the Authenticator Customer Account file. This file is Read with Update intent and a Rewrite is issued to update the record.
The FiTeq Authenticator is configurable with or without an Authenticator Validation Log. With this file configured, an audit record will be written for each transaction.
The Validation Log is designed to serve two functions. First, it serves as an audit trail. Secondly, it enables the Customer Service Representative (CSR) via CICS online panels to conduct a real-time search of the transaction history of a given account number. Because all transactions, regardless of whether each transaction is approved or denied, will be returned to the issuer’s authorization system calling the FiTeq Authenticator API, the audit trail is already maintained by the issuer’s authorization system. The online search against the given account number is a nice feature, but it is not mandatory.
The Authentication Validation Log, after it is turned on, will have one new record added per transaction for an audit trail. Its data includes a unique primary key (Account Number + Time Stamp), authentication result code, and error code, if applicable.
Both the Customer Account file and the Validation Log file are defined as CICS recoverable files with LOG(UNDO).
Based on the above rationale, the FiTeq Authenticator supports a dynamic configuration to turn on and off the logging feature per an issuer’s requirements and preference.
The application also consists of three CICS data tables, which are relatively small in size and have the following attributes:
� Key: Contains each card portfolio’s master key cryptogram, which is used to derive the card-unique key within the HSM to be used in part for authentication.
� BIN: Contains locations and lengths of the vital track 2 fields, such as transaction counter, to be used in the authentication.
� Configuration: Contains key-encrypting key (KEK) information to be used in part for the batch import of cardholder accounts. It also supports dynamically turning on and off the Validation Log.
Chapter 3. Benchmark application topology 11
12 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 4. Scaling CICS applications
Well-written CICS applications scale linearly, both vertically and horizontally.
Well-written CICS applications scale vertically by taking advantage of advances in the coverage of Threadsafe CICS API commands and increasing throughput within a single CICS region. Recent releases of CICS have also enabled clients to run more applications in fewer regions by reducing virtual storage constraints, moving data from 24-bit and 31-bit addressing to 64-bit addressing. Consolidation of CICS regions also reduces the real storage footprint, which can reduce translation lookaside buffer misses and therefore reduce cycles. Back in CICS V4.2, the concurrency “Required” option on program definitions made it easier to have applications start on Open task control blocks (TCBs).
Horizontal scaling refers to spreading a workload across multiple CICS regions. Horizontal scaling can be useful if an application is becoming constrained by a single resource within a region and multiple regions can relieve this constraint. Or, it might be for reasons of availability and resilience.
There are many ways to scale work across multiple regions, depending on the configuration and resources being used by the applications. The incoming requests need to be distributed across regions and those regions need to be able to share data with integrity. CICS and TCP/IP both have interfaces with Workload Manager (WLM) that will distribute work based on chosen algorithms.
As part of the FiTeq application benchmark, the scalability of both vertical and horizontal configurations was evaluated.
The starting point for the benchmark was a single CICS region using VSAM files defined as using local shared resource (LSR) buffers. Requests arrived from the TCP/IP network over a single port via a single Listener, which, in turn, started the Child Server transaction. The Child Server transaction running the FiTeq Authenticator code accessed VSAM files via LSR pool buffers. LSR pool buffers can only be shared among transactions running within the same CICS region.
4
© Copyright IBM Corp. 2014. All rights reserved. 13
4.1 TCP/IP port sharing
For simplicity, TCP/IP port sharing was used with this benchmark to distribute work across cloned CICS regions. Because the clients were not using persistent connections, each new request required a new connection. This gave a constant even spread of work across all regions because each new request connection was redistributed. If persistent connection was used instead, it tied a client to a CICS region for the life of the benchmark and might not have given such a good distribution. In this benchmark, up to four CICS regions listened on the same TCP/IP port. As a request came in, it was a first come first served basis where the CICS region in the best position in terms of waiting for work gets to receive the connection request.
TCP/IP for port sharing can be configured by adding an entry to the Communications Server IP PROFILE.TCPIP configuration file to associate the shared port number with the corresponding CICS job names.
4.2 Sharing VSAM files
When the same application has been spread across multiple CICS regions, those regions need a mechanism for sharing access to the VSAM files. There are two main ways to achieve this. One is to use File Control Function Shipping where all requests are shipped to a single file owning region. The other option, which was the one chosen for this benchmark, is VSAM record-level sharing (RLS), which enables files to be shared across many regions within a sysplex.
4.3 CICS and RLS
Before RLS, CICS users were able to share VSAM data sets with integrity by using function shipping to a File Owning Region (FOR). With function shipping, one CICS region accesses the VSAM data set on behalf of other CICS regions. Requests to access the data set are shipped from the region where the transaction is running to the region that owns the file. Function shipping provides a solution for the CICS user, but it has limitations. Function shipping does not address the problems of sharing data sets between CICS regions and batch jobs. Also, note that with multiregion operation (MRO), the FOR is constrained to the single QR TCB and therefore to the speed of a single central processor (CP). RLS is not subject to this constraint because all access to files is done across the multiple Application Owning Regions (AORs) where the application resides. These RLS file requests are capable of running in Threadsafe mode, in which case they run on L8/9 TCBs. In non-Threadsafe mode, the RLS requests run on service request blocks (SRBs) within the CICS address spaces. LSR file requests can also run in Threadsafe mode when run in an AOR, but when MRO function shipped to an FOR, they run on the QR TCB.
4.3.1 Components of RLS
To coordinate and control access to VSAM files at a record sharing level, DFSMS, through the SMSVSAM address space, uses a number of components and facilities.
14 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Each LPAR where RLS access is required will have an SMSVSAM address space active. This address space manages all access to data sets that are opened in RLS mode. To achieve this, SMSVSAM communicates through the use of cross-system coupling facility (XCF) groups and coupling facility (CF) cache and lock structures with SMSVSAM address spaces running on other LPARs within the sysplex.
For a VSAM data set to be opened for RLS access, it must be storage management subsystem (SMS)-managed. Part of the process of defining the data set as SMS-managed requires, among other things, a Storage Class to be assigned. The Storage Class definition will contain a cache set name with a number of Coupling Facility Cache structures being assigned to the named cache set.
In addition to the cache structures used by SMSVSAM as intermediate storage between local memory and DASD, a lock structure, called IGWLOCK00, is used to provide sysplex-wide locking at the record level. (In z/OS v1R10, up to ten additional lock structures, called secondary lock structures, can now be defined). These lock structures, which only contain record locks, will provide better separation of workloads, and better CF balancing and availability.
Correct sizing of the CF cache and lock structures used by SMSVSAM is key to achieving optimum performance. You are strongly urged to consult the z/OS DFSMStvs Planning and Operating Guide, SC26-7348, for recommendations about the correct sizing of these structures.
4.4 CICS Threadsafe applications
CICS applications defined as Threadsafe can run concurrently on OPEN TCBs (L8 or L9 TCBs). Defining an application to be Threadsafe is achieved by specifying REQUIRED or THREADSAFE on the CONCURRENCY parameter of the program definition. Specifying REQUIRED enables the application to start on an Open TCB. Using THREADSAFE needs an IBM DB2® or MQ call to move the application to an Open TCB. Using Threadsafe applications reduces any contention for the CICS QR TCB and enables single CICS regions to achieve more throughput than non-Threadsafe applications. Also, applications that are Threadsafe have a reduction in TCB switching when using DB2 or MQ and therefore can deliver CPU savings.
The FiTeq Authenticator can be run in Threadsafe mode. For this benchmark, it was defined with Concurrency REQUIRED.
Communication Server IP CICS Sockets can also be configured to run on Open TCBs by specifying YES for the value of the OTE parameter. NO is the default. A value of YES causes the IP CICS sockets task-related user exit to execute on Open TCBs. This was set to YES for this benchmark.
Two other CICS parameters to consider when using Threadsafe applications are FORCEQR and FCQRONLY. FORCEQR forces programs defined as Threadsafe to run on the QR TCB, and FCQRONLY forces all CICS file control requests to run on the CICS QR TCB. So, unless these are set correctly, your applications might not be running where you want them to run.
Chapter 4. Scaling CICS applications 15
16 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 5. Environment
This chapter describes the hardware and software that was used when this benchmark was conducted. Any CPU timing that is documented in this Redpaper publication can be scaled to other IBM hardware by using the Large Systems Performance Report (LSPR).
5
© Copyright IBM Corp. 2014. All rights reserved. 17
5.1 Server hardware
The base hardware was EC12 2877-7A1:
� LPAR with 10 dedicated central processors (CPs)
� DASD DS8800
� Internal Coupling Facility with two dedicated CPs and interval control program (ICP) links
� OSA Express 4S 1 GB Ethernet
5.2 Software
The required software is listed:
� z/OS V1R13
� CICS Transaction Server V5.R1
� Communication Server IP Sockets for CICS
� IBM z/OS Resource Measurement Facility™ (RMF™)
� CICS Performance Analyzer
5.3 Client simulation hardware
The client simulation hardware is identified per three load clients as shown in Table 5-1, Table 5-2, and Table 5-3 on page 19.
Table 5-1 Client simulation hardware 1
Table 5-2 Client simulation hardware 2
System name: LoadTestClient1
Machine type and memory: x3950 16 GB RAM
Internal storage: 256 GB
Operating system: Windows 2012 STD x64
Role: Workload Driver (Load Test Client #1)
System name: LoadTestClient2
Machine type and memory: x3950 64 GB RAM
Internal storage: 100 GB
Operating system: Windows 2008 STD x64
Role: Workload Driver (Load Test Client #2)
18 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Table 5-3 Client simulation hardware 3
5.4 Hardware security module simulation hardware
The hardware security module (HSM) simulation hardware is identified in Table 5-4.
Table 5-4 HSM simulation hardware
System name: LoadTestClient3
Machine type and memory: x3950 64 GB RAM
Internal storage: 100 GB
Operating system: Windows 2008 STD x64
Role: Workload Driver (Load Test Client #3)
System name: HSMsimulator
Machine type and memory: x3950 16 GB RAM
Internal storage: 256 GB
Operating system: Windows 2012 STD x64
Role: HSM Simulator
Chapter 5. Environment 19
20 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 6. Measurement methodology
During the benchmark, the following performance data was collected while the workload was running at a steady state:
� Resource Measurement Facility (RMF) Monitor I � CICS performance monitoring � CICS interval statistics
6
© Copyright IBM Corp. 2014. All rights reserved. 21
6.1 RMF MON I
RMF Monitor I records system performance data at user-defined time intervals in System Management Facility (SMF) Type 70 - 78 records. It gives a performance view from a z/OS perspective. Among other resources, it records the performance of Workload Manager (WLM) Service classes. This, along with the use of WLM Reporting groups, can be used to record the CPU usage, transaction rate, and response times for CICS address spaces. RMF SMF records were post-processed using the ERBRMFPP utility.
6.2 CICS performance monitoring
CICS performance monitoring is activated by having MNPER=ON in the system initialization table (SIT) or by using the CEMT transaction to set it on dynamically after CICS startup. We also suggest setting RMI=YES in the MCT so CICS collects more granular information about the various, different resource managers.
CICS TS V5.R1 has its default set to RMI=YES.
CICS performance monitoring generates a record for every transaction, which contains details of its performance characteristics and resources that have been waited on. It includes CPU time, response time, suspend time, and a breakdown of the time waiting for each type of resource. These records are written as SMF 110 type 1 records.
These SMF records were post-processed using the CICS Performance Analyzer to produce detail reports, summary reports, wait time analysis, and more. CICS Performance Analyzer is an extremely useful tool for getting an insight into CICS regions. It is used as an aid to improving the performance of CICS workloads.
6.3 CICS interval statistics
CICS interval statistics are user-defined interval-based records written to SMF as SMF 110 type 2 records. These statistics records were post-processed by the CICS Performance Analyzer, which gives an overall view of the resources being used by the CICS address space. All counters are reset at the start of an interval, which makes it easy to see the rate at which resources are used.
For this benchmark, these statistics were used especially to see some of the performance characteristics of the VSAM files, specifically looking for File String waits and VSAM Control Interval conflicts.
6.4 System load points
Performance data was collected for various transaction rates ranging from high to low. The actual transaction rates in the system were increased by increasing the number of simulated clients. The aim was to achieve rates of 400 per second - 8000 per second.
22 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 7. Terms used
The following a list shows the terms that are used in the results tables and their explanations:
� ETR: External Transaction Rate is the number of transactions executed per second.
� Response time: Internal transaction response time measured in milliseconds by Resource Measurement Facility (RMF) or CICS monitoring.
� CICS CPU%: The percentage of one general-purpose central processor (CP) used by the CICS address spaces. Note that this value can exceed 100% when applications are running in Threadsafe mode because applications can be running concurrently on more than one CP.
� LPAR CPU%: This is how busy the logical partition (LPAR) is in terms of its usage of the 10 CPs that are allocated to it. This figure is taken from the RMF I CPU Activity report.
� CF CPU%: This is how busy the coupling facility (CF) is in terms of its usage of the two CPs defined to it. This data is taken from RMF III.
� SMSVSAM CPU%: The percentage of one general-purpose CP used by the SMSVSAM address space used in the VSAM record-level sharing (RLS) configuration.
� CICS CPU/Transaction: Microseconds of CPU per transaction. This is calculated as the average CPU percentage that is used by CICS address spaces divided by the transaction rate per second, for example, a CICS region is using 23.3% of a CP when running 3494 transactions per second. So, one second used 0.233 seconds of one CP. One transaction therefore uses 0.233/3494 = 0.000666 seconds of CPU (666 microseconds of CPU).
� LPAR CPU/Transaction: Similar to the CICS CPU/Transaction but based on the LPAR busy percentage, which includes all the other address spaces, such as TCP/IP, included in the workload.
7
© Copyright IBM Corp. 2014. All rights reserved. 23
24 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 8. Single CICS region configurations
The first set of measurements was taken using a single region to handle the incoming authentication requests. The aim of the benchmark was to ensure that as the transaction rate increased within a single region, the CPU and response times scaled linearly.
The VSAM files for this environment were configured to use local shared resource (LSR) pools. Two sets of measurements were taken: one set without the Validation Log configured and one set with the Validation Log. For a further description of the use of the Validation Log, see 3.5, “FiTeq Authenticator: VSAM file access” on page 11.
The diagram in Figure 8-1 on page 26 shows the single region configuration.
8
© Copyright IBM Corp. 2014. All rights reserved. 25
Figure 8-1 Authenticator performance test configuration with a single CICS region
FIREWALL (SonicWALL TZ210)
FIREWALL
FiTeq Data Center
4
VPN
FIREWALLWAN IP: w.w.w.w
LAN IP: y.y.y.y SonicWALL TZ210WAN IP: w.w.w.wLAN IP: n.n.n.n
x.x.x.x:623
HSM cmd Req 2
IBM Benchmark Center IBM LAN1 (1G)
Router
FiTeq IBM Benchmark Test Mainframe LPAR#1(z/OS 1.13) (10 CPs, 1 CICS region)
HSM Simulator ServerRuns HSM Simulator(s)
y.y.y.y:9998
3
2
Req 1
6 : Perf. Port: x.x.x.x: 1511
: Development: Authenticator Performance Test: Authenticator connects to HSM Simulator
IBM’s Router:x.x.x.x and
y.y.y.y
FiTeq LAN1 (1 G)
Load Test Client ServersRun Loadtest Client &
send up to 10,000 TPS
FiTeq Authenticator
(COBOL)AuthorizerEmulator
CICS 5.1 (Region 1)
6Port: 1511
Remote Desktop Server
WinSvr1(HSMSim)
WinSvr4(LoadTest
Client)WinSvr3
(Load Test
Client)WinSvr2
(LoadTest
Client1)
Authenticator DB
FIREWALL
FIREWALL (SonicWALL TZ210)
26 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 9. Single region results
This chapter documents the results from the two sets of single region benchmark measurements, one without the FiTeq Validation Log and one with the FiTeq Validation Log. The following tables show the relevant performance data that was extracted from Resource Measurement Facility (RMF) for the various external transaction rates (ETRs). For an explanation of each column, see Chapter 7, “Terms used” on page 23.
9
© Copyright IBM Corp. 2014. All rights reserved. 27
9.1 Results for a single region without the FiTeq Validation Log
Table 9-1 shows the results extracted from RMF for four transaction rates ranging from low to high without the FiTeq Validation Log configured.
Table 9-1 Results for single region without FiTeq Validation Log
9.2 Results for a single region with the FiTeq Validation Log
Table 9-2 shows the results extracted from RMF for four transaction rates ranging from low to high with the FiTeq Validation Log configured.
Table 9-2 Results for single region with FiTeq Validation Log
9.3 Example RMF report extract
The following data in Figure 9-1 on page 29 shows where the data in the result tables was extracted from in the RMF Monitor I reports. For example, this data refers to the third measurement point in the Single region without the FiTeq Validation Log.
For this measurement point, there was only one CICS region running in the Report Class CICSF1A0 as shown by the AVE field.
The CICS CPU% that was taken from the - - - APPL %- - - field is the average percentage of one central processor (CP) that has been used over the interval. It is the (CPU + SRB Service time)/interval elapsed time. So, when multiple regions or multiple task control blocks (TCBs) are involved, this value can exceed 100%. In this report, it used an average of 95.75% of a CP.
The Response Time Milliseconds was extracted from TRANS-TIME HHH.MM.SS.TTT, and on the ACTUAL row, 4 milliseconds is reported.
ETR Response timemilliseconds
CICS CPU% LPAR CPU% CICS CPU/Tranmicroseconds
LPARCPU/TRANmicroseconds
497.60 3 23.70 3.44 476 691
969.38 5 47.72 6.41 492 661
1941.80 4 95.75 12.22 493 629
3494.82 4 186.41 23.30 533 666
ETR Response timemilliseconds
CICS CPU% LPAR CPU% CICS CPU/Tranmicroseconds
LPARCPU/TRANmicroseconds
494.34 5 27.12 3.84 548 776
956.19 8 53.13 6.94 555 725
1862.21 8 106.13 13.50 569 724
3114.24 47 185.90 23.17 596 744
28 IBM CICS Performance Series: FiTeq Authenticator Benchmark
To create a report that contains both the CPU time and the transaction rate for CICS regions, specify the same report class for both the JES/STC subsystem type classification and the CICS subsystem classification.
Figure 9-1 Sample RMF Monitor I Report for a single CICS region without the FiTeq Validation Log
Figure 9-2 is an example where the LPAR CPU% was extracted from. It shows an RMF Activity report for the same third measurement point. It shows the total of the ten CPs to be on average 12.22% busy.
Figure 9-2 Sample RMF Activity Report for a single CICS region without the FiTeq Validation Log
9.4 CICS PA Wait Analysis for a single region without the Validation Log
During the benchmark period, CICS Performance Analyzer (PA) was used to understand any bottlenecks causing response times to increase. CICS PA Wait Analysis is a useful report for understanding the performance characteristics of CICS transactions. Figure 9-3 on page 30 shows an example of this report for the highest transaction rate in the single region case without the Validation Log. The System Management Facilities (SMF) data has been extracted for a 3-minute period and represents the Child transaction (P511) started by the long running Listener for each incoming authentication request.
The average response time shown in Figure 9-3 on page 30 is 5.5 milliseconds. Theoretically, this needs to match the response recorded by RMF as shown in row four of Table 9-1 on page 28. However, this data was collected for a shorter time interval and focuses on only one transaction type (P511).
Chapter 9. Single region results 29
A CICS PA Wait Analysis report gives a breakdown of all the resources the transactions waited on for this transaction. Thirty-seven percent of its response time was accumulated by WTCEWAIT. For this application, this equates to time waiting for the response from the hardware security module (HSM). The transaction issues an EXEC CICS WAIT after passing its request to the Channel. When the Channel has the response from the HSM, it issues an EXEC CICS POST to wake up the waiting transaction.
There are other resource waits, such as Journal I/O wait and File I/O wait time, but these are insignificant. Any time spent in ENQDELAY is time spent waiting to get exclusive use of one of the Channels to the HSM. The application uses an EXEC ENQUEUE to reserve a Channel.
The transaction rate is 628591/180secs = 3492 transactions per second and, as expected, is similar to that recorded by RMF.
Figure 9-3 Sample CICS PA Wait Analysis for a single CICS region without the Validation Log
9.5 CICS PA Wait Analysis for a single region with the Validation Log
Figure 9-4 on page 31 shows the CICS Wait Analysis for the single region with the FiTeq Validation Log configured. The response times have increased compared to the response times without the log. This is expected because every transaction needs to write a record to the VSAM key-sequenced data set (KSDS) file. Continually writing records from many transactions to a single file can have a performance impact in two ways. Contention at the Control Interval can occur if the access pattern involves writing records with similar keys. Control Interval and Control Area splits might happen if enough free space is not allocated at data set define time. If, at some point, records are also deleted, you need to use CA Reclaim, a function of VSAM, to avoid any data set fragmentation overhead and any processing of empty control intervals (CIs). CA Reclaim also eliminates the need to take files offline to perform VSAM data set reorganizations.
The data for the Validation Log being present shows the increase in response time to be mainly attributed to File I/O Wait time with the additional Writes and some additional Journal I/O Wait time due to the before images of the Validation Log also being written to the CICS system log.
30 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Figure 9-4 Sample CICS PA Wait Analysis for a single CICS region with the Validation Log
Chapter 9. Single region results 31
32 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 10. Multiple CICS region configurations
This set of measurements was taken using multiple, cloned CICS regions to handle the incoming authentication requests. The aim of the benchmark was to ensure that the FitTeq Authenticator transactions were capable of being easily scaled across multiple regions, and as the transaction rate increased, the CPU and response times scaled linearly. This benchmark also identified any bottlenecks not exposed in the single region environments.
Up to four CICS regions were cloned, all listening on the same TCP/IP port for incoming requests.
VSAM files for this environment were configured to use VSAM record-level sharing (RLS) so that the VSAM data was capable of being shared among multiple CICS regions. Two sets of measurements were taken: one set without the Validation Log configured and one set with the Validation Log. For a further description of the use of the Validation Log, see 3.5, “FiTeq Authenticator: VSAM file access” on page 11.
The diagram in Figure 10-1 on page 34 shows the configuration of the multiple regions.
10
© Copyright IBM Corp. 2014. All rights reserved. 33
Figure 10-1 Authenticator performance test configuration with four CICS regions and Coupling Facility
FIREWALL (SonicWALL TZ210)
FIREWALL
FiTeq Data Center
4
VPNFIREWALL
WAN IP: w.w.w.wLAN IP: y.y.y.y SonicWALL TZ210
WAN IP: w.w.w.wLAN IP: n.n.n.n
x.x.x.x:623
HSM cmd Req 2
IBM Benchmark Center IBM LAN1 (1G)
Router
FiTeq IBM Benchmark Test Mainframe LPAR#1(z/OS 1.13) (10 CPs, 4 CICS regions, 1 CF)
HSM Simulator ServerRuns HSM Simulator(s)
y.y.y.y:9998
3
2
Req 1
6 : Perf. Port: x.x.x.x: 1511
: Development: Authenticator Performance Test: Authenticator connects to HSM Simulator
IBM’s Router:
FiTeq LAN1 (1 G)
Load Test Client ServersRun Loadtest Client &
send up to 10,000 TPS
FiTeqAuthenticator
(COBOL)AuthorizerEmulator
CICS 5.1 (Region 4)
6Port: 1511
FiTeq Authenticator
(COBOL)AuthorizerEmulator
CICS 5.1 (Region 3)
6Port: 1511
FiTeq Authenticator
(COBOL)AuthorizerEmulator
CICS 5.1 (Region 2)
6Port: 1511
Remote Desktop Server
WinSvr1(HSMSim)
WinSvr4(Load Test
Client)WinSvr3
(LoadTest
Client)WinSvr2
(LoadTest
Client1)
FiTeqAuthenticator
(COBOL)AuthorizerEmulator
CICS 5.1 (Region 1)
6Port: 1511
Authenticator DB
CouplingFacility
(CF, 2 CPs
FIREWALL
FIREWALL (SonicWALL TZ210)
34 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 11. Multiple region results
This chapter documents the results from the two sets of benchmark measurements, one without the FiTeq Validation Log and one with the FiTeq Validation Log. The results shown here demonstrate the ability to easily scale the FiTeq application across multiple CICS regions. For an explanation of each column, see Chapter 7, “Terms used” on page 23.
11
© Copyright IBM Corp. 2014. All rights reserved. 35
11.1 Results for multiple regions without the FiTeq Validation Log
Table 11-1 shows the results extracted from Resource Measurement Facility (RMF) for four transaction rates ranging from low to high without the FiTeq Validation Log configured.
Table 11-1 Results for multiple regions without the FiTeq Validation Log
11.2 Results for multiple regions with the FiTeq Validation Log
Table 11-2 shows the results extracted from RMF for five transaction rates ranging from low to high with the FiTeq Validation Log configured.
Table 11-2 Results for multiple regions with FiTeq Validation Log
11.3 CICS PA Wait Analysis for multiple regions without the Validation Log
Figure 11-1 on page 37 shows a CICS Performance Analyzer (PA) Wait Analysis report for the high-end transaction rate for the measurements taken without the Validation Audit Log.
The report covers a 3-minute interval so the transaction rate equates to 1561135/180 seconds = 8672 transactions per second, which is similar to the transaction rate reported by RMF and recorded in Table 11-1. The average response time is 14.6 milliseconds with Local Enqueue wait time being the largest part of the suspend time. This indicates that the transactions are starting to have to wait for a free channel to access the hardware security module (HSM). This might have been improved by increasing the number of channels or increasing the number of HSMs so that there was less contention accessing an HSM.
ETR Response timemilliseconds
CICS CPU%
SMSVSAM CPU%
LPAR CPU%
CF CPU%
CICS CPU/Tranmicroseconds
LPARCPU/TRANmicroseconds
499.34 4 34.26 0.26 4.59 0.20 686 919
983.61 9 68.62 0.49 8.56 0.40 697 870
1966.19 9 137.38 0.93 16.66 0.90 698 847
8680.75 14 633.96 4.04 75.99 3.80 730 875
ETR Response timemilliseconds
CICS CPU%
SMSVSAM CPU%
LPAR CPU%
CF CPU%
CICS CPU/Tranmicroseconds
LPARCPU/TRANmicroseconds
498.89 8 41.97 0.62 5.68 0.50 841 1138
962.09 14 82.24 1.24 10.09 1.00 854 1048
1923.00 15 163.80 2.42 19.68 2.00 851 1023
3742.32 20 321.09 4.72 39.47 3.90 857 1054
5091.03 35 443.92 6.59 52.69 5.50 871 1034
36 IBM CICS Performance Series: FiTeq Authenticator Benchmark
To share VSAM files across regions, the configuration was changed to use VSAM record-level sharing (RLS) in place of VSAM local shared resource (LSR). Any waits for file access will now show up in RLSWAIT time and not FCIOWTT.
Note that in pure Threadsafe applications, any File Wait, either RLS or LSR, will not show up in CICS monitoring data because the application does not need to go through the CICS Dispatcher to issue any wait while it is running on its own Open task control block (TCB). For tuning purposes during the benchmark, FCQRONLY was set to YES so that these file requests were switched to the QR TCB. Therefore, a CICS wait was needed and the RLSWAIT or the FCIOWTT was accounted for in the monitoring records. For this application, it had an insignificant effect on performance because there are a limited number of file accesses. Setting FCQRONLY=NO will avoid the TCB switching, but some analysis data is lost.
Figure 11-1 Sample CICS PA Wait Analysis for multiple CICS regions without the Validation Log
11.4 CICS PA Wait Analysis for multiple regions with the Validation Log
Figure 11-2 on page 38 is a CICS PA Wait Analysis report showing the effect of introducing the Validation Log in the multiregion environment. The throughput has dropped to 915425/180 = 5085 transactions per second from 8680. This is due to a combination of the overall internal response time increasing because of the extra file I/O and the way in which the simulated client is operating. The number of clients at the higher transaction rate was fixed at 10,000 with a fixed delay between receiving a response and sending the next request. As the CICS response time increases so does the time between each client submitting a new request, therefore, with a fixed number of clients, the throughput drops off.
However, taking this into consideration, the report shows an average file access time for a total of two requests to be 29.3 milliseconds at this high transaction rate. To improve this file access time, it will require a study of more performance data, including RMF Mon III Coupling Facility, and RLS data and a study of the RLS System Management Facilities (SMF) 42 records. For the purposes of this benchmark and the allotted time, this was not feasible.
Chapter 11. Multiple region results 37
Figure 11-2 Sample CICS PA Wait Analysis for multiple CICS regions with the Validation Log
38 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 12. Tuning and configuration considerations
Getting the best out of applications running in CICS starts with a review of the application design and looking for any obvious potential constraints and contention issues. A review of how the application might scale beyond current requirements to cater for increased throughput in the future also needs to be considered.
With regards to the FiTeq Authenticator, considerations were made for both a single CICS region’s growth and multiple regions’ growth.
Single region growth can be limited by the QR task control block (TCB) being constrained to the speed of a single central processor (CP). The FiTeq application was set as being Threadsafe to reduce usage of the QR TCB and to enable increased concurrency. The program definition was defined as CONCURRENCY REQUIRED. This will start the application on an Open TCB as opposed to only moving to an Open TCB when a call to a Resource Manager Interface (RMI) that uses an Open TCB is made. The Communications Server IP CICS Sockets option OTE=YES was also set so that all the socket API calls used Open TCBs and not the QR TCB. For full Thread safety, ensure that FCQRONLY=NO and FORCEQR=NO are set.
If virtual storage becomes a single region constraint, a review of storage usage is suggested. In the case of the FiTeq application, all transactions had their TASKDATALOC set to ANY and the program definitions had their DATALOCATION set to ANY. The default is currently BELOW so this needs overriding to reduce 24-bit storage constraints. The last resort for virtual storage relief is to split the CICS regions and use multiple regions to support the application.
Applications that are distributed across multiple CICS regions need to share the same VSAM data and maintain integrity. For this benchmark, VSAM RLS was chosen. Multiregion operation (MRO) Function Shipping might also have been used. Although MRO Function Shipping is less expensive in terms of CPU usage, the CICS-supplied mirror transactions, (CSMI), for example, CSM1 and CSM2, are restricted to the QR TCB. These File Owning Regions are therefore constrained by the speed of a single CP.
12
© Copyright IBM Corp. 2014. All rights reserved. 39
Accessing VSAM files in either of these ways can require some thought in file design and tuning. If files have records added by transactions, enough Free Space needs to be allocated at file define time to reduce control interval (CI) and control area (CA) splits. Consideration must also be given to the use of VSAM CA Reclaim. If possible, define keys so that they have random access when writing to the VSAM key-sequenced data sets (KSDSs) rather than something like a time stamp sequence where records keep getting added to the same CI concurrently.
When updating records in LSR mode, CI contention can occur when multiple transactions try to Read for Update records within the same CI. In the RLS case, although CI locking does not happen, VSAM will be re-merging these updates to the same CI, which can cause overhead. If files that are mostly Read for Update have fewer records per CI, this can reduce CI contention. Also, note that browsing of VSAM LSR files using STARTBR and READNEXT results in a lock on the CI, so that other transactions will not be able to complete any Read for Update in that CI until the browse is ended. Limiting the time between STARTBR and ENDBR reduces potential contention.
The CICS Performance Analyzer (PA) Wait Analysis report also showed another resource, the Journal I/O wait time. This is time spent by the transaction waiting for its before image log records to be hardened before it can complete an update of a VSAM record. Although this was not significant, this configuration used DASDONLY log streams. Journal I/O wait times might have been improved by using Coupling Facility structures for Logstreams.
40 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Chapter 13. Conclusions
The CICS configuration chosen for this benchmark demonstrates the FiTeq Authenticator scales linearly both in terms of CPU time per transaction and response times.
The best performing configurations are the configurations without the FiTeq Validation Log.
The single region configuration achieves around 3494 transactions per second while maintaining internal response times in the region of 4 milliseconds. On average across the four measurement points, a transaction uses 0.000498 seconds of CPU time.
The multiple regions configuration achieves 8680 transactions per second while maintaining internal response times in the region of 14 milliseconds. On average across the five measurement points, a transaction uses 0.000703 seconds of CPU time.
An increase in CPU per transaction moving from single region to multiple regions is expected due to the following factors:
� Use of VSAM record-level sharing (RLS) instead of local shared resource (LSR)
� Increased concurrent task control block (TCB) access to central processors (CPs)
� Logical partition (LPAR) busy % increasing
Regarding the use of the Validation Log, although at the higher transaction rates it does as expected and increases the response times, a detailed study about how to improve it by tuning was not performed due to time constraints. Although it was thought that it might have been possible to improve the throughput, the use of the Validation Log is a configuration option.
Overall, the FiTeq Authentication Solution demonstrates to be efficient in terms of CPU usage and scales well both vertically and horizontally as the throughput rate increases. Its design is such that it can be used in various configurations depending on a client’s needs.
13
© Copyright IBM Corp. 2014. All rights reserved. 41
42 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Appendix A. Single and multiple region scaling
The single and multiple region scaling charts are in this appendix.
A
© Copyright IBM Corp. 2014. All rights reserved. 43
A.1 Single region scaling
Figure A-1 shows linear scaling of CPU usage as the transaction rate increases with two configurations: without the Validation Log (red line) and with the Validation Log (blue line).
Figure A-1 CPU usage versus transaction rate for single CICS region without log and with log
44 IBM CICS Performance Series: FiTeq Authenticator Benchmark
A.2 Multiple region scaling
Figure A-2 shows linear scaling of CPU usage as the transaction rate increases with two configurations: without the Validation Log (red line) and with the Validation Log (blue line).
Figure A-2 CPU usage versus transaction rate for multiple CICS regions without log and with log
Appendix A. Single and multiple region scaling 45
46 IBM CICS Performance Series: FiTeq Authenticator Benchmark
Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this paper.
IBM Redbooks
The following IBM Redbooks publications provide additional information about the topic in this document. Note that some publications referenced in this list might be available in softcopy only.
� IBM CICS Performance Series: A Processor Usage Study of Ways into CICS, REDP-4906
� IBM CICS Performance Series: CICS and VSAM RLS, REDP-4905
� IBM CICS Performance Series: CICS, DB2, and Thread Safety, REDP-4860
You can search for, view, download or order these documents and other Redbooks, Redpapers, Web Docs, draft and additional materials, at the following website:
ibm.com/redbooks
Online resources
These websites are also relevant as further information sources:
� IBM
http://www.ibm.com/
� FiTeq
http://www.fiteq.com/
Help from IBM
IBM Support and downloads
ibm.com/support
IBM Global Services
ibm.com/services
© Copyright IBM Corp. 2014. All rights reserved. 47
48 IBM CICS Performance Series: FiTeq Authenticator Benchmark
®
REDP-5114-00
INTERNATIONAL TECHNICALSUPPORTORGANIZATION
BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE
IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.
For more information:ibm.com/redbooks
Redpaper™
IBM CICS Performance Series: FiTeq Authenticator Benchmark
Performance and scalability benchmark methodology explained
Multiple configurations tested and the results compared
Benchmark driven to 8,000 CICS transactions per second
FiTeq is an IBM Business Partner that specializes in fraud prevention technologies for the payments industry. This IBM Redpaper publication records the methodologies and results of a performance benchmark using the FiTeq Authenticator, which is a component of FiTeq’s family of Secure Transaction Solutions.
The FiTeq Authenticator is an IBM CICS enabled application that was run under CICS Transaction Server for z/OS V5.1 in this benchmark. The performance benchmark was conducted as a joint venture between IBM and FiTeq in January 2014.
In summary, the following FiTeq Authenticator application performance characteristics were demonstrated:
� A scalable solution: CPU usage scales linearly as the number of transactions per second increases.
� Cost-effective: Approximately only 500 microseconds of CPU per transaction were used for the single configuration.
� Efficient: Average response times below 20 milliseconds per transaction were maintained at a transaction rate exceeding 8,000 per second.
These benchmark test results confirmed and validated that the FiTeq Authenticator is, in conjunction with the performance, reliability, and scalability provided by IBM z/OS and CICS architectures and associated hardware, fully capable of satisfying the requirements of all top financial institutes.
As a by-product of the FiTeq Authenticator performance test, the IBM World-Wide Solutions-Cross ISV Sizing team developed a FiTeq Authenticator Sizing Tool to forecast system requirements based on the transactions per second (TPS) and other system requirements of any future FiTeq client. As a result, the IBM pre-sale team and the FiTeq marketing team will be able to recommend the best fit and most cost-effective IBM software and hardware solution for a particular FiTeq client. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations, such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Back cover