os/390 parallel sysplex configuration volume 1: overvie · 2000. 9. 12. · os/390 parallel sysplex...

OS/390Parallel SysplexConfigurationVolume 1: Overview

Frank Kyne,Luiz Fadel, Michael Ferguson,RafaelGarcia,Keith George,MasayukiKojima, Alan Murphy,Henrik Thorsen

International Technical Support Organization

www.redbooks.ibm.com

SG24-5637-00

International Technical Support Organization

OS/390 Parallel Sysplex ConfigurationVolume 1: Overview

August 2000

SG24-5637-00

Take Note!

Before using this information and the product it supports, be sure to read the general information inAppendix A, “Special Notices” on page 233.

First Edition (August 2000)

This edition applies to

Program Name, Program Number Version, Release NumberCICS/ESA, 5655-018 4.1CICS TS for OS/390, 5655-147 1.3DB2 for OS/390, 5655-DB2 5.1DB2 UDB for OS/390, 5645-DB2 6.1DB2 UDB for OS/390, 5675-DB2 7.1DFSMS/MVS, 5695-DF1 1.5IMS/ESA, 5695-176 5.1IMS/ESA, 5655-158 6.1IMS/ESA, 5655-B01 7.1System Automation for OS/390, 5645-045 1.3

for use with:

Program Name Version, Release NumberOS/390, 5645-001 1.3OS/390, 5647-A01 2.4OS/390, 5647-A01 2.5OS/390, 5647-A01 2.6OS/390, 5647-A01 2.7OS/390, 5647-A01 2.8OS/390, 5647-A01 2.9OS/390, 5647-A01 2.10

Comments may be addressed to:IBM Corporation, International Technical Support OrganizationDept. HYJ Mail Station P0992455 South RoadPoughkeepsie, New York 12601-5400

When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in anyway it believes appropriate without incurring any obligation to you.

Copyright International Business Machines Corporation 2000. All rights reserved.Note to U.S. Government Users — Documentation related to restricted rights — Use, duplication or disclosure issubject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.

Contents

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiOS/390 Parallel Sysplex Configuration Volume 1: Overview . . . . . . . . . . xiiiOS/390 Parallel Sysplex Configuration Volume 2: Cookbook . . . . . . . . . . xivOS/390 Parallel Sysplex Configuration Volume 3: Connectivity . . . . . . . . xiv

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiOS/390 Parallel Sysplex Configuration Volume 1: Overview . . . . . . . . . . xviiOS/390 Parallel Sysplex Configuration Volume 2: Cookbook . . . . . . . . . . xviiOS/390 Parallel Sysplex Configuration Volume 3: Connectivity . . . . . . . xviii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xixThe Team That Wrote These Redbooks . . . . . . . . . . . . . . . . . . . . . . xixComments Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxx

OS/390 Parallel Sysplex Configuration Volume 1: Overview1.0 Introduction to the Configuration of a Parallel Sysplex . . . . . . . . . . . . 11.1 How to Use These Redbooks - Read This First . . . . . . . . . . . . . . . . 11.2 The Purpose of These Books . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 Base Hardware Levels for These Books . . . . . . . . . . . . . . . . . . . . 61.4 Main Reasons for Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . 6

1.4.1 Continuous Application Availability . . . . . . . . . . . . . . . . . . . . 71.4.2 Workload Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.4.3 Nondisruptive Addition of Scalable CPC Capacity . . . . . . . . . . . . 81.4.4 Reduced Total Cost of Computing . . . . . . . . . . . . . . . . . . . . . 101.4.5 Single System Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.5 Value of Parallel Sysplex - A Summary . . . . . . . . . . . . . . . . . . . . . 131.6 The Distinction between Basic and Parallel Sysplex . . . . . . . . . . . . . 13

1.6.1 Basic Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.6.2 Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.7 How Do I Quickly Get to Parallel Sysplex? . . . . . . . . . . . . . . . . . . . 141.8 Function Support Table for Pre-G5 9672 CPCs . . . . . . . . . . . . . . . . 15

2.0 High-Level Design Concepts for Parallel Sysplex . . . . . . . . . . . . . . . 172.1.1 Important Rules of Thumb Information . . . . . . . . . . . . . . . . . . 18

2.2 Deciding If Parallel Sysplex Is Right for You . . . . . . . . . . . . . . . . . . 182.2.1 S/390 Partners in Development Software . . . . . . . . . . . . . . . . . 23

2.3 Parallel Sysplex High-Level Design . . . . . . . . . . . . . . . . . . . . . . . 262.3.1 How Many Parallel Sysplexes Do I Need? . . . . . . . . . . . . . . . . 262.3.2 System Symmetry When Configuring a Parallel Sysplex . . . . . . . . 382.3.3 What Different 'Plexes Are There? . . . . . . . . . . . . . . . . . . . . . 392.3.4 How Do 'Plexes Relate to Each Other? . . . . . . . . . . . . . . . . . . 422.3.5 'Plex Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.4 CF Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712.4.1 XES Providing Software Access to the CF . . . . . . . . . . . . . . . . 73

2.5 Data Integrity and Buffer Pool Consistency Considerations . . . . . . . . . 782.5.1 Data Integrity before Parallel Sysplex . . . . . . . . . . . . . . . . . . . 782.5.2 Data Integrity in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . 792.5.3 Locking in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . 82

2.6 Dynamic Workload Balancing in Parallel Sysplex . . . . . . . . . . . . . . 832.6.1 Dynamic Workload Balancing Exploiters . . . . . . . . . . . . . . . . . 84

Copyright IBM Corp. 2000 iii

3.0 Continuous Availability in Parallel Sysplex . . . . . . . . . . . . . . . . . . . 893.1 Why Availability Is Important . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3.1.1 Parallel Sysplex Is Designed to Allow Management of Redundancy . 913.1.2 Planned Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 933.1.3 Unplanned Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 953.1.4 Scope of an Outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.2 Software Considerations for Availability . . . . . . . . . . . . . . . . . . . . 973.2.1 Data Set Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973.2.2 Couple Data Set Considerations . . . . . . . . . . . . . . . . . . . . . . 983.2.3 JES3 and Continuous Availability . . . . . . . . . . . . . . . . . . . . . . 983.2.4 Subsystem Considerations . . . . . . . . . . . . . . . . . . . . . . . . 101

3.3 Network Considerations for Availability . . . . . . . . . . . . . . . . . . . 1043.3.1 VTAM Generic Resources Function . . . . . . . . . . . . . . . . . . . 1043.3.2 Persistent Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1043.3.3 Multi-Node Persistent Sessions . . . . . . . . . . . . . . . . . . . . . . 1053.3.4 High Performance Routing . . . . . . . . . . . . . . . . . . . . . . . . . 1073.3.5 VTAM Systems Management . . . . . . . . . . . . . . . . . . . . . . . 1073.3.6 TCP/IP Virtual IP Addressing . . . . . . . . . . . . . . . . . . . . . . . 1083.3.7 Dynamic VIPA Takeover . . . . . . . . . . . . . . . . . . . . . . . . . . 1103.3.8 Sysplex Distributor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.4 Hardware Considerations for Availability . . . . . . . . . . . . . . . . . . . 1113.4.1 Number of CPCs in Parallel Sysplex . . . . . . . . . . . . . . . . . . . 1113.4.2 Redundant Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1113.4.3 Isolate the CF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1123.4.4 Additional CF Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1153.4.5 I/O Configuration Redundancy . . . . . . . . . . . . . . . . . . . . . . 1153.4.6 Sysplex Timer Redundancy . . . . . . . . . . . . . . . . . . . . . . . . 1163.4.7 RAS Features on IBM CPCs . . . . . . . . . . . . . . . . . . . . . . . . 116

3.5 Limitations to Continuous Availability . . . . . . . . . . . . . . . . . . . . . 1163.6 Recovery Considerations for Availability . . . . . . . . . . . . . . . . . . . 117

3.6.1 Sysplex Failure Management (SFM) . . . . . . . . . . . . . . . . . . . 1173.6.2 Automatic Restart Management (ARM) . . . . . . . . . . . . . . . . . 123

3.7 Disaster Recovery Considerations in Parallel Sysplex . . . . . . . . . . . 1263.7.1 Multi-Site Sysplexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1273.7.2 Disaster Recovery Data . . . . . . . . . . . . . . . . . . . . . . . . . . 1303.7.3 CICS Disaster Recovery Considerations . . . . . . . . . . . . . . . . 1333.7.4 DB2 Disaster Recovery Considerations . . . . . . . . . . . . . . . . . 1333.7.5 IMS Disaster Recovery Considerations . . . . . . . . . . . . . . . . . 1343.7.6 Geographically Dispersed Parallel Sysplex Offering . . . . . . . . . 1353.7.7 More Information on GDPS . . . . . . . . . . . . . . . . . . . . . . . . 1383.7.8 Recommended Sources of Disaster Recovery Information . . . . . 139

4.0 Workloads in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . 1414.1 e-business and Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . 143

4.1.1 Web Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1444.1.2 HTML Serving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1474.1.3 3270 Emulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1474.1.4 CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1484.1.5 DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1564.1.6 IMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

4.2 Transaction Management in Parallel Sysplex . . . . . . . . . . . . . . . . 1694.2.1 Dynamic Transaction Routing . . . . . . . . . . . . . . . . . . . . . . . 1694.2.2 CICS Transactions in a Parallel Sysplex . . . . . . . . . . . . . . . . 1694.2.3 CICSPlex Systems Manager (CICSPlex SM) . . . . . . . . . . . . . . 1854.2.4 CICS Transaction Affinity Utility . . . . . . . . . . . . . . . . . . . . . . 187

iv Parallel Sysplex Configuration, Volume 1: Overview

4.2.5 IMS Transaction Manager in Parallel Sysplex . . . . . . . . . . . . . 1874.3 Database Management in Parallel Sysplex . . . . . . . . . . . . . . . . . 194

4.3.1 DB2 Data Sharing Considerations . . . . . . . . . . . . . . . . . . . . 1944.3.2 IMS DB Data Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2034.3.3 CICS/VSAM Record Level Sharing Considerations . . . . . . . . . . 2074.3.4 S/390 Partners in Development Databases . . . . . . . . . . . . . . . 213

4.4 Batch Workload Considerations . . . . . . . . . . . . . . . . . . . . . . . . 2134.4.1 JES2 Considerations in Parallel Sysplex . . . . . . . . . . . . . . . . 2134.4.2 JES3 Considerations in Parallel Sysplex . . . . . . . . . . . . . . . . 2144.4.3 Can I Have JES2 and JES3 in the Same Sysplex . . . . . . . . . . . 2154.4.4 Batch Workload Balancing and Parallel Sysplex . . . . . . . . . . . 2154.4.5 IBM BatchPipes for OS/390 . . . . . . . . . . . . . . . . . . . . . . . . 215

4.5 Network Workload Balancing Capabilities . . . . . . . . . . . . . . . . . . 2194.5.1 VTAM Generic Resources Function . . . . . . . . . . . . . . . . . . . 2194.5.2 TCP Workload Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . 225

4.6 APPC/MVS and Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . 2264.7 TSO/E and Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

4.7.1 MAS Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2274.7.2 Query Management Facility Workload Considerations . . . . . . . . 228

4.8 Test Considerations in Parallel Sysplex . . . . . . . . . . . . . . . . . . . 2284.8.1 Testing Implications in Parallel Sysplex . . . . . . . . . . . . . . . . . 228

4.9 How to Select Applications to Exploit Parallel Sysplex . . . . . . . . . . 231

Appendix A. Special Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Appendix B. Related Publications . . . . . . . . . . . . . . . . . . . . . . . . . 237B.1 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237B.2 IBM Redbooks collections . . . . . . . . . . . . . . . . . . . . . . . . . . . 239B.3 Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239B.4 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241B.5 Other References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

How to Get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247IBM Redbook FAX Order Form . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

IBM Redbooks evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

OS/390 Parallel Sysplex Configuration Volume 2: Cookbook1.0 CPC and CF Configuration in Parallel Sysplex . . . . . . . . . . . . . . . . . 11.1 An Overview of CPC and CF Configuration Tasks . . . . . . . . . . . . . . 31.2 CPC and CF Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2.1 CPC/CF Configuration Options . . . . . . . . . . . . . . . . . . . . . . . 61.2.2 Deciding between the CPC Options . . . . . . . . . . . . . . . . . . . . 61.2.3 Processor Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.4 CF Capabilities That Affect Configuration Options . . . . . . . . . . . . 91.2.5 CF Link Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3 CPC Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.3.1 Sizing and Modelling Options for CPC . . . . . . . . . . . . . . . . . . . 15

Contents v

1.3.2 Calculate Additional Capacity for Data Sharing . . . . . . . . . . . . . 161.3.3 Logical Partitioning (LPAR) Impact on Sizing . . . . . . . . . . . . . . 201.3.4 Sizing the CPC Storage for Parallel Sysplex . . . . . . . . . . . . . . . 211.3.5 Impact of Parallel Sysplex on the Number of I/Os . . . . . . . . . . . . 25

1.4 CF CP Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.4.1 CF Topology Considerations . . . . . . . . . . . . . . . . . . . . . . . . 261.4.2 Deriving Required CF CP Capacity . . . . . . . . . . . . . . . . . . . . . 301.4.3 Hot Standby CFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411.4.4 CPC/CF Parallel Sysplex Sizing Tools . . . . . . . . . . . . . . . . . . . 42

1.5 CF Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431.5.1 CF Sender Channels (TYPE=CBS, TYPE=CFS or TYPE=ICS) . . . . 441.5.2 CF Receiver Channels (TYPE=CBR, TYPE=CFR or TYPE=ICR) . . . 441.5.3 CF Link Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441.5.4 How Many CF Links Do I Need? . . . . . . . . . . . . . . . . . . . . . . 451.5.5 Effect of Distance on Fiber Links . . . . . . . . . . . . . . . . . . . . . . 46

1.6 CF Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471.6.1 Estimating CF Storage Requirements . . . . . . . . . . . . . . . . . . . 491.6.2 Sample CF Storage Map . . . . . . . . . . . . . . . . . . . . . . . . . . . 511.6.3 CF Structure Size Summary . . . . . . . . . . . . . . . . . . . . . . . . . 521.6.4 Summary Table for CF Storage Sizing . . . . . . . . . . . . . . . . . . . 54

1.7 Review Final Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.0 A Structured View of CF Structures . . . . . . . . . . . . . . . . . . . . . . . 572.1 CF Structures Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.1.1 CF Structure Exploiters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622.1.2 Structure and Connection Disposition . . . . . . . . . . . . . . . . . . . 662.1.3 CF Volatility/Nonvolatility . . . . . . . . . . . . . . . . . . . . . . . . . . 67

2.2 CF Storage for Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682.2.1 Initial Structure Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 692.2.2 Using Central and Expanded Storage for the CF . . . . . . . . . . . . . 762.2.3 Structure Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772.2.4 CF Exploiter Specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.3 XCF Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792.3.1 XCF Structure Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792.3.2 XCF Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792.3.3 XCF Structure Placement . . . . . . . . . . . . . . . . . . . . . . . . . . 812.3.4 XCF Structure Rebuild Considerations . . . . . . . . . . . . . . . . . . 812.3.5 XCF Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . . 82

2.4 VTAM Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832.4.1 VTAM Structures Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 832.4.2 VTAM Structures Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . 842.4.3 VTAM Structures Placement . . . . . . . . . . . . . . . . . . . . . . . . 862.4.4 VTAM Structures Rebuild Considerations . . . . . . . . . . . . . . . . . 872.4.5 VTAM Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . 88

2.5 RACF Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892.5.1 RACF Structure Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892.5.2 RACF Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892.5.3 RACF Structure Placement . . . . . . . . . . . . . . . . . . . . . . . . . 912.5.4 RACF Structure Rebuild Considerations . . . . . . . . . . . . . . . . . 912.5.5 RACF Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . 92

2.6 JES2 Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932.6.1 JES2 Structure Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932.6.2 JES2 Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932.6.3 JES2 Structure Placement . . . . . . . . . . . . . . . . . . . . . . . . . . 942.6.4 JES2 Structure Rebuild Considerations . . . . . . . . . . . . . . . . . . 95

vi Parallel Sysplex Configuration, Volume 1: Overview

2.6.5 JES2 Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . . 952.7 IMS Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

2.7.1 IMS Structure Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 962.7.2 IMS Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 972.7.3 IMS Structure Placement . . . . . . . . . . . . . . . . . . . . . . . . . 1012.7.4 IMS Structure Rebuild Considerations . . . . . . . . . . . . . . . . . . 1012.7.5 IMS Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . 103

2.8 DB2 Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1042.8.1 DB2 Structures Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 1042.8.2 DB2 Structures Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . 1052.8.3 DB2 Structures Placement . . . . . . . . . . . . . . . . . . . . . . . . . 1152.8.4 DB2 Structures Rebuild Considerations . . . . . . . . . . . . . . . . . 1162.8.5 DB2 Structures and Volatility . . . . . . . . . . . . . . . . . . . . . . . 118

2.9 Tape Allocation Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . 1202.9.1 Tape Allocation Structure Usage . . . . . . . . . . . . . . . . . . . . . 1202.9.2 Tape Allocation Structure Sizing . . . . . . . . . . . . . . . . . . . . . 1202.9.3 Tape Allocation Structure Placement . . . . . . . . . . . . . . . . . . 1212.9.4 Tape Allocation Structure Rebuild Considerations . . . . . . . . . . 1212.9.5 Tape Allocation Structures and Volatility . . . . . . . . . . . . . . . . 122

2.10 System Logger Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . 1232.10.1 System Logger Structure Usage . . . . . . . . . . . . . . . . . . . . 1232.10.2 System Logger Structure Sizing . . . . . . . . . . . . . . . . . . . . . 1242.10.3 System Logger Structure Placement . . . . . . . . . . . . . . . . . . 1282.10.4 System Logger Structure Rebuild Considerations . . . . . . . . . . 1292.10.5 System Logger Structures and Volatility . . . . . . . . . . . . . . . . 131

2.11 DFSMS Enhanced Catalog Sharing (ECS) Structures . . . . . . . . . . . 1322.11.1 DFSMS ECS Structure Usage . . . . . . . . . . . . . . . . . . . . . . 1322.11.2 DFSMS ECS Structure Sizing . . . . . . . . . . . . . . . . . . . . . . 1332.11.3 DFSMS ECS Structure Placement . . . . . . . . . . . . . . . . . . . . 1332.11.4 DFSMS ECS Structure Rebuild Considerations . . . . . . . . . . . . 1332.11.5 DFSMS ECS Structures and Volatility . . . . . . . . . . . . . . . . . . 134

2.12 CICS/VSAM RLS Structures . . . . . . . . . . . . . . . . . . . . . . . . . . 1352.12.1 CICS/VSAM RLS Structure Usage . . . . . . . . . . . . . . . . . . . . 1352.12.2 CICS/VSAM RLS Structure Sizing . . . . . . . . . . . . . . . . . . . . 1362.12.3 CICS/VSAM RLS Structure Placement . . . . . . . . . . . . . . . . . 1392.12.4 CICS/VSAM RLS Structure Rebuild Considerations . . . . . . . . . 1402.12.5 CICS/VSAM RLS Structures and Volatility . . . . . . . . . . . . . . . 140

2.13 CICS Shared Temporary Storage Structures . . . . . . . . . . . . . . . . 1412.13.1 CICS Shared Temporary Storage Structure Usage . . . . . . . . . . 1412.13.2 CICS Shared Temporary Storage Structure Sizing . . . . . . . . . . 1412.13.3 CICS Shared Temporary Storage Structure Placement . . . . . . . 1422.13.4 CICS Shared Temporary Storage Structure Rebuild Considerations 1422.13.5 CICS Shared Temporary Storage Structures and Volatility . . . . . 143

2.14 CICS Coupling Facility Data Tables Structures . . . . . . . . . . . . . . . 1442.14.1 CICS CFDT Structure Usage . . . . . . . . . . . . . . . . . . . . . . . 1442.14.2 CICS CFDT Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . 1442.14.3 CICS CFDT Structure Placement . . . . . . . . . . . . . . . . . . . . 1452.14.4 CICS CFDT Structure Rebuild Considerations . . . . . . . . . . . . 1452.14.5 CICS CFDT Structures and Volatility . . . . . . . . . . . . . . . . . . 145

2.15 CICS Named Counter Server Structures . . . . . . . . . . . . . . . . . . 1462.15.1 CICS Named Counter Server Structure Usage . . . . . . . . . . . . 1462.15.2 CICS Named Counter Server Structure Sizing . . . . . . . . . . . . 1462.15.3 CICS Named Counter Server Structure Placement . . . . . . . . . 1462.15.4 CICS Named Counter Server Structure Rebuild Considerations . . 1472.15.5 CICS Named Counter Server Structures and Volatility . . . . . . . 147

Contents vii

2.16 GRS Star Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1482.16.1 GRS Star Structure Usage . . . . . . . . . . . . . . . . . . . . . . . . 1482.16.2 GRS Star Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . . 1482.16.3 GRS Star Structure Placement . . . . . . . . . . . . . . . . . . . . . 1502.16.4 GRS Star Structure Rebuild Considerations . . . . . . . . . . . . . . 1502.16.5 GRS Star Structures and Volatility . . . . . . . . . . . . . . . . . . . 151

2.17 WLM Multisystem Enclaves Structure . . . . . . . . . . . . . . . . . . . . 1522.17.1 WLM Enclaves Structure Usage . . . . . . . . . . . . . . . . . . . . . 1522.17.2 WLM Enclaves Structure Sizing . . . . . . . . . . . . . . . . . . . . . 1522.17.3 WLM Enclaves Structure Placement . . . . . . . . . . . . . . . . . . 1522.17.4 WLM Enclaves Rebuild Considerations . . . . . . . . . . . . . . . . 1532.17.5 WLM Enclaves Structures and Volatility . . . . . . . . . . . . . . . . 153

2.18 MQSeries Shared Queue Structures . . . . . . . . . . . . . . . . . . . . . 1542.18.1 Setup for MQ Shared Queues . . . . . . . . . . . . . . . . . . . . . . 154

2.19 BatchPipes Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1562.19.1 BatchPipes Structure Usage . . . . . . . . . . . . . . . . . . . . . . . 1562.19.2 BatchPipes Structure Sizing . . . . . . . . . . . . . . . . . . . . . . . 1562.19.3 BatchPipes Structure Placement . . . . . . . . . . . . . . . . . . . . 1572.19.4 BatchPipes Structure Rebuild Considerations . . . . . . . . . . . . 1572.19.5 BatchPipes Structures and Volatility . . . . . . . . . . . . . . . . . . 158

Appendix A. Tools and Services Catalog . . . . . . . . . . . . . . . . . . . . . 159A.1.1 ARMWRAP (ARM Wrapper) . . . . . . . . . . . . . . . . . . . . . . . . 164A.1.2 BatchPipes Integrated Service Offering . . . . . . . . . . . . . . . . . 164A.1.3 BWATOOL (Batch Workload Analysis Tool) . . . . . . . . . . . . . . 165A.1.4 CATMERGE (Catalog Merge/Rebuild Tool) . . . . . . . . . . . . . . . 165A.1.5 CAU (CICS Affinity Tool) . . . . . . . . . . . . . . . . . . . . . . . . . . 165A.1.6 CD14 (CICS/ESA Log Stream and CF Sizing Migration Utility) . . . 166A.1.7 CF Sizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167A.1.8 CH16 (CICS/ESA Temporary Storage Overview) . . . . . . . . . . . 167A.1.9 CH17 (CICS/ESA Journaling and Recovery Options) . . . . . . . . . 168A.1.10 CP2000 (Capacity Planning 2000) . . . . . . . . . . . . . . . . . . . . 168A.1.11 CP2000 PCR (Processor Capacity Reference) . . . . . . . . . . . . 169A.1.12 CP2000 Quick-Sizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169A.1.13 CS17 (CICS/ESA Interdependencies Utility) . . . . . . . . . . . . . . 170A.1.14 CS1D (CICS TS for OS/390 Offline Log Stream Processor) . . . . . 170A.1.15 CS1E (CICS Sample Global User Exit Program for XLGSTRM) . . 171A.1.16 CVRCAP (CICS/VSAM RLS Capacity Planning) . . . . . . . . . . . 171A.1.17 DB2SPLIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171A.1.18 Education Offerings for Parallel Sysplex . . . . . . . . . . . . . . . 172A.1.19 EPSO (Enhanced Parallel Sysplex Offering) . . . . . . . . . . . . . 172A.1.20 GMMA (Goal Mode Migration Aid) . . . . . . . . . . . . . . . . . . . 174A.1.21 GRSRNL (GRS RNL Tool) . . . . . . . . . . . . . . . . . . . . . . . . 174A.1.22 HSA Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175A.1.23 IMSSMQCP (IMS Shared Message Queue Capacity Planner) . . . 175A.1.24 IMSSPLIT (IMS Split) . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A.1.25 LOGREC Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A.1.26 LPAR/CE (LPAR Capacity Estimator) . . . . . . . . . . . . . . . . . . 177A.1.27 LSPR/PC (Large Systems Performance Reference/PC) . . . . . . . 177A.1.28 OPERLOG Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178A.1.29 OS/390 Integration Test: Downloads . . . . . . . . . . . . . . . . . . 178A.1.30 Parallel Sysplex Configuration Assistant . . . . . . . . . . . . . . . 179A.1.31 Parallel Sysplex Operator Certification . . . . . . . . . . . . . . . . 181A.1.32 PLEX.EXE (Sysplex/Parallel Sysplex Information Utility) . . . . . . 181A.1.33 PMOS (Performance Management Offerings and Services) . . . . 182

viii Parallel Sysplex Configuration, Volume 1: Overview

A.1.34 RACCARDS (RACF Database Merge/Rebuild Tool) . . . . . . . . . 182A.1.35 RLSLKSZ (RLS Lock Sizer) . . . . . . . . . . . . . . . . . . . . . . . 183A.1.36 SMF42RPT (Print SMF Type 42 Records) . . . . . . . . . . . . . . . 183A.1.37 SNAP/SHOT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184A.1.38 SOFTCAP (Capacity Effects of Software Migration) . . . . . . . . . 184A.1.39 SWPRICER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184A.1.40 SYSPARM (Symbolic Parmlib Parser) . . . . . . . . . . . . . . . . . 184

Appendix B. RMF Reporting in Parallel Sysplex . . . . . . . . . . . . . . . . . 187B.1.1 Parallel Sysplex Reports . . . . . . . . . . . . . . . . . . . . . . . . . 187B.1.2 Parallel Sysplex RMF Postprocessor Reports . . . . . . . . . . . . . 187B.1.3 CF Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188B.1.4 RMF Report Showing Different CF Channel Types . . . . . . . . . . 195B.1.5 RMF XCF Activity Reports . . . . . . . . . . . . . . . . . . . . . . . . . 197B.1.6 Parallel Sysplex RMF Monitor III Reports . . . . . . . . . . . . . . . . 201B.1.7 IRLM Long Lock Detection Report . . . . . . . . . . . . . . . . . . . . 209B.1.8 Local System Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . 210B.1.9 RMF Performance Data Related to Parallel Sysplex . . . . . . . . . 210B.1.10 RMF Spreadsheet Functions . . . . . . . . . . . . . . . . . . . . . . . 211B.1.11 Data Sharing Charge-Back Considerations . . . . . . . . . . . . . . 211B.1.12 Recent RMF Enhancements . . . . . . . . . . . . . . . . . . . . . . . 211B.1.13 General Hints and Tips . . . . . . . . . . . . . . . . . . . . . . . . . . 212

Appendix C. System Logger Considerations . . . . . . . . . . . . . . . . . . . 213C.1 System Logger Stream Concept . . . . . . . . . . . . . . . . . . . . . . . . 213C.2 Remote Site Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213C.3 Stream Offload from the CF . . . . . . . . . . . . . . . . . . . . . . . . . . 214C.4 Log Stream Subsystem Data Set Interface . . . . . . . . . . . . . . . . . 214C.5 System Logger and the Volatility State of the CF . . . . . . . . . . . . . . 215C.6 DASD Log Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

C.6.1 Managing Log Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216C.7 Staging Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

C.7.1 Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . 217C.8 System Logger Performance Recommendations . . . . . . . . . . . . . . 217C.9 System Logger Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . 219

C.9.1 List Enhancements - OW21383 . . . . . . . . . . . . . . . . . . . . . . 219C.9.2 DISPLAY LOGGER Command . . . . . . . . . . . . . . . . . . . . . . . 221

Appendix D. Tuning DB2 Structures . . . . . . . . . . . . . . . . . . . . . . . . 223D.1 Shared Communications Area (SCA) . . . . . . . . . . . . . . . . . . . . . 223D.2 GBPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223D.3 IRLM Lock Structure Used for DB2 Data Sharing . . . . . . . . . . . . . . 227

D.3.1 Tuning the Lock Usage . . . . . . . . . . . . . . . . . . . . . . . . . . 227D.3.2 Changing the Size of the Lock Structure . . . . . . . . . . . . . . . . 229

Appendix E. Functional Differences between IBM 9672-Based CPCs . . . . . 231

Appendix F. MSU Values for Selected IBM CPCs and CFs . . . . . . . . . . . 237F.1 MSU Values for Selected IBM CPCs . . . . . . . . . . . . . . . . . . . . . 237

F.1.1 IBM 9672 Generation 6 CPCs . . . . . . . . . . . . . . . . . . . . . . . 237F.1.2 IBM 9672 Generation 5 CPCs and CFs . . . . . . . . . . . . . . . . . . 238F.1.3 IBM 9672 Generation 4 CPCs and CFs . . . . . . . . . . . . . . . . . . 239F.1.4 IBM 9672 Generation 3 CPCs and CFs . . . . . . . . . . . . . . . . . . 240

Appendix G. Special Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Contents ix

Appendix H. Related Publications . . . . . . . . . . . . . . . . . . . . . . . . . 245H.1 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245H.2 IBM Redbooks collections . . . . . . . . . . . . . . . . . . . . . . . . . . . 247H.3 Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247H.4 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249H.5 Other References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251


Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257


Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283


OS/390 Parallel Sysplex Configuration Volume 3: Connectivity1.0 I/O Connectivity in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . 11.1 Fibre Connection (FICON) in FCV Mode . . . . . . . . . . . . . . . . . . . . 3

1.1.1 FICON Exploitation of Existing ESCON Infrastructure . . . . . . . . . . 51.1.2 System Automation for OS/390 . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Enterprise Systems Connection (ESCON) . . . . . . . . . . . . . . . . . . . 61.2.1 Enhanced Multiple Image Facility (EMIF) . . . . . . . . . . . . . . . . . 71.2.2 ESCON Extended Distance Feature (XDF) . . . . . . . . . . . . . . . . . 71.2.3 ESCON Director (ESCD) Configuration Guidelines . . . . . . . . . . . . 81.2.4 IBM 9036 Remote Channel Extender Configuration Guidelines . . . . 101.2.5 IBM 9729 Optical Wavelength Division Multiplexer . . . . . . . . . . . 111.2.6 IBM 2029 Dense Wavelength Division Multiplexer . . . . . . . . . . . . 131.2.7 FTS Direct Attach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.3 Channel Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.3.1 ESCON Logical Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.3.2 DASD Control Unit Considerations . . . . . . . . . . . . . . . . . . . . . 151.3.3 ESCON Channel-to-Channel (CTC) Considerations . . . . . . . . . . . 161.3.4 Tape Control Unit Considerations . . . . . . . . . . . . . . . . . . . . . 161.3.5 OEM Hardware Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.0 Network Connectivity for Parallel Sysplex . . . . . . . . . . . . . . . . . . . 192.1 Network Access to Parallel Sysplex Using VTAM . . . . . . . . . . . . . . . 21

2.1.1 APPN Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.1.2 Use of APPN in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . 242.1.3 High Performance Routing (HPR) . . . . . . . . . . . . . . . . . . . . . . 272.1.4 High Performance Data Transfer (HPDT) . . . . . . . . . . . . . . . . . 302.1.5 Session Manager Implications in Parallel Sysplex . . . . . . . . . . . 312.1.6 Communications Management Configuration . . . . . . . . . . . . . . 33

2.2 NCPs in a CMC Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.2.2 SNI Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.2.3 Enterprise Extender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.3 Network Access to Parallel Sysplex Using TCP/IP . . . . . . . . . . . . . . 422.3.1 Availability Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . 422.3.2 Workload Balancing Enhancements . . . . . . . . . . . . . . . . . . . . 462.3.3 Summary of Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4 Connecting to the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

x Parallel Sysplex Configuration, Volume 1: Overview

2.4.1 IBM 3745/6 Communications Controller Family . . . . . . . . . . . . . 572.4.2 Open Systems Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.4.3 Channel-Attached Routers . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.0 Sysplex Timer Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . 693.1 Overview of Time Management . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.1.1 TOD Clock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713.1.2 ESA/390 Time-Coordination Requirements . . . . . . . . . . . . . . . . 723.1.3 External Time Reference (ETR) Time . . . . . . . . . . . . . . . . . . . 733.1.4 ETR Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733.1.5 OS/390 Support for the Sysplex Timer . . . . . . . . . . . . . . . . . . . 743.1.6 PR/SM LPAR and Sysplex Timer . . . . . . . . . . . . . . . . . . . . . . 743.1.7 Multiple Clocks and Their Uses . . . . . . . . . . . . . . . . . . . . . . . 75

3.2 Configuration Considerations for an ETR Network . . . . . . . . . . . . . . 753.2.1 Planning for Availability of Sysplex Timer . . . . . . . . . . . . . . . . 753.2.2 CPC Support for Sysplex Timer . . . . . . . . . . . . . . . . . . . . . . . 813.2.3 S/390 CPC Sysplex Timer Attachment Features . . . . . . . . . . . . . 813.2.4 9037 Port Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . 843.2.5 9037-2 Console Selection and Considerations . . . . . . . . . . . . . . 853.2.6 Planning for Time Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 873.2.7 9037 Distance Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . 883.2.8 Planning for Cabling Requirements . . . . . . . . . . . . . . . . . . . . 90

3.3 9037 RPQs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913.4 Migration from a 9037-1 to a 9037-2 Sysplex Timer Configuration . . . . . 92

3.4.1 Requirements during Sysplex Timer Migration . . . . . . . . . . . . . 923.4.2 Sysplex Timer Migration Procedures Not Supported . . . . . . . . . . 923.4.3 Supported Sysplex Timer Migration Procedures . . . . . . . . . . . . . 93

3.5 Recommendations for Continuous Sysplex Timer Availability . . . . . . . 943.5.1 Sysplex Timer Recovery Overview . . . . . . . . . . . . . . . . . . . . . 943.5.2 Primary Timer Assignment of a 9037-2 . . . . . . . . . . . . . . . . . . 963.5.3 Power Sources for Timers and Extenders . . . . . . . . . . . . . . . . . 963.5.4 Fiber Optic Cable Routing . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.6 Concurrent Maintenance and Upgrade Considerations . . . . . . . . . . . 973.6.1 Concurrent Maintenance of Sysplex Timers . . . . . . . . . . . . . . . 973.6.2 Upgrade Options for Sysplex Timers . . . . . . . . . . . . . . . . . . . 98

3.7 Planning for System Availability . . . . . . . . . . . . . . . . . . . . . . . . . 983.7.1 SFM Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.0 Consoles and Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . 1014.1 Multisystem Consoles in Parallel Sysplex . . . . . . . . . . . . . . . . . . 102

4.1.1 OS/390 Consoles in Parallel Sysplex . . . . . . . . . . . . . . . . . . 1034.1.2 How Many Consoles Do You Need . . . . . . . . . . . . . . . . . . . . 1084.1.3 Availability and Performance Considerations . . . . . . . . . . . . . 1094.1.4 Automation Implications . . . . . . . . . . . . . . . . . . . . . . . . . . 110

4.2 Hardware Management Consoles in a Parallel Sysplex . . . . . . . . . . 1114.2.1 Hardware Management Console (HMC) . . . . . . . . . . . . . . . . . 1114.2.2 HMC Configuration Considerations . . . . . . . . . . . . . . . . . . . 1124.2.3 Configuring the LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1134.2.4 Operating the HMC Remotely . . . . . . . . . . . . . . . . . . . . . . . 1134.2.5 Automating Operation of the IBM 9672 . . . . . . . . . . . . . . . . . 1144.2.6 Can I Use Integrated Consoles Instead of OS/390 Consoles . . . . . 1164.2.7 How Many HMCs Should I Order . . . . . . . . . . . . . . . . . . . . . 1174.2.8 Installing a New CPC on an Existing HMC LAN . . . . . . . . . . . . 1184.2.9 Battery Backup of the HMC . . . . . . . . . . . . . . . . . . . . . . . . 1184.2.10 Updating LIC on the HMC/SE . . . . . . . . . . . . . . . . . . . . . . 118

Contents xi

4.2.11 Security Considerations . . . . . . . . . . . . . . . . . . . . . . . . . 118

Appendix A. Systems Management Products for Parallel Sysplex . . . . . . 121A.1 Systems Management Software in Parallel Sysplex . . . . . . . . . . . . 124A.2 OS/390 Security Server (RACF) Sysplex Exploitation . . . . . . . . . . . 124

A.2.1 RACF Data Sharing Modes . . . . . . . . . . . . . . . . . . . . . . . . 124A.2.2 RACF Remote Sharing Facility . . . . . . . . . . . . . . . . . . . . . . 128

A.3 Performance Management Products and Parallel Sysplex . . . . . . . . 128A.3.1 Workload Manager (WLM) in the Parallel Sysplex Environment . . 129A.3.2 OS/390 Resource Measurement Facility (RMF) in Parallel Sysplex 133A.3.3 Performance Reporter for OS/390 and Parallel Sysplex . . . . . . . 137A.3.4 System Measurement Facility (SMF) . . . . . . . . . . . . . . . . . . 138

A.4 Operations Management Products and Parallel Sysplex . . . . . . . . . 139A.4.1 Data and Storage Backup and Recovery . . . . . . . . . . . . . . . . 139A.4.2 Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140A.4.3 System Automation for OS/390 . . . . . . . . . . . . . . . . . . . . . . 142A.4.4 Tivoli Business Systems Manager . . . . . . . . . . . . . . . . . . . . 145A.4.5 Job Scheduling in Parallel Sysplex . . . . . . . . . . . . . . . . . . . 146

A.5 Change Control and Configuration Management . . . . . . . . . . . . . . 148A.5.1 SMP/E Configuration Guidelines . . . . . . . . . . . . . . . . . . . . . 148A.5.2 Hardware Configuration Definition (HCD) Considerations . . . . . . 151A.5.3 Hardware Configuration Manager (HCM) . . . . . . . . . . . . . . . . 151A.5.4 Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152A.5.5 Shared Master Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . 155A.5.6 Shared System Residence (SYSRES) Volume . . . . . . . . . . . . . 156A.5.7 Considerations for Couple Data Sets . . . . . . . . . . . . . . . . . . 158A.5.8 Rebuild for CF Maintenance (Rebuild Enhancement) . . . . . . . . . 163

A.6 Parallel Sysplex Education . . . . . . . . . . . . . . . . . . . . . . . . . . . 165A.6.1 Parallel Sysplex Certification Program . . . . . . . . . . . . . . . . . 166

Appendix B. Special Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Appendix C. Related Publications . . . . . . . . . . . . . . . . . . . . . . . . . 171C.1 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171C.2 IBM Redbooks collections . . . . . . . . . . . . . . . . . . . . . . . . . . . 173C.3 Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173C.4 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175C.5 Other References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177


Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183


Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209


xii Parallel Sysplex Configuration, Volume 1: Overview

Figures

OS/390 Parallel Sysplex Configuration Volume 1: Overview1. Scalability in Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . . 122. Combined Environments for Data Sharing . . . . . . . . . . . . . . . . . . 333. Combined Environments for Data Sharing (GBP Duplexing) . . . . . . . 334. Combined Environments for Resource Sharing . . . . . . . . . . . . . . . 345. Examples of a Test Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . 366. Sharing DASD between Multiple Sysplexes/GRS Complexes . . . . . . . 397. Parallel Sysplex Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 438. JESplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479. Pipeplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

10. GRSplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5111. RACFplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5312. SMSplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5513. HSMplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5714. BCSplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5915. OAMplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6116. WLMplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6317. RMFplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6518. OPCplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6719. CICSplex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6920. Systems, Groups, and Members in an XCF Sysplex . . . . . . . . . . . . 7721. Multisystem Data Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8122. Multisystem Data Sharing in Parallel Sysplex . . . . . . . . . . . . . . . . 8223. Failure Domain Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9824. TCP/IP VIPA Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11125. Isolating a Failing OS/390 . . . . . . . . . . . . . . . . . . . . . . . . . . . 12026. Sample GDPS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 13127. Peer-to-Peer Remote Copy Configuration . . . . . . . . . . . . . . . . . 13328. Extended Remote Copy Configuration . . . . . . . . . . . . . . . . . . . . 13429. Example Configurations for DB2 Data Sharing Groups . . . . . . . . . . 13630. Example of Geographically Dispersed Parallel Sysplex (GDPS) . . . . 13931. Sample Scalable Web Server Configuration . . . . . . . . . . . . . . . . 14732. Separation of Application and Presentation Logic in CICS . . . . . . . 15133. CICS e-business Connectivity Options . . . . . . . . . . . . . . . . . . . 15234. CTG with Java Applet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15535. CTG with Java Servlet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15636. CICS Web Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15737. DB2 Access Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15938. IMS Communication Options . . . . . . . . . . . . . . . . . . . . . . . . . 16339. Role of IMS TOC/IMS Connect . . . . . . . . . . . . . . . . . . . . . . . . 16540. IMS Workload Balancing with VTAM GR . . . . . . . . . . . . . . . . . . 16741. Load Balancing across Telnet Servers . . . . . . . . . . . . . . . . . . . 16842. IMS Load Balancing with IND . . . . . . . . . . . . . . . . . . . . . . . . . 16943. IMS Connect Workload Balancing Capability . . . . . . . . . . . . . . . . 17044. Evolution of Multi-Region Operation in CICS . . . . . . . . . . . . . . . . 17345. Sample CICS Data Sharing Subsystem Configuration . . . . . . . . . . 17646. Sample Generic Resource Group in an IMS Environment . . . . . . . . 19047. Sample IMS Systems Sharing Message Queues . . . . . . . . . . . . . 19248. IMS Workload Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19449. DB2 Data Sharing Group in a Parallel Sysplex . . . . . . . . . . . . . . 198

Copyright IBM Corp. 2000 xiii

50. DB2 Data Sharing in a Parallel Sysplex . . . . . . . . . . . . . . . . . . . 20051. Sample Data Sharing Group in an IMS Environment . . . . . . . . . . . 20652. VSAM Record Level Sharing in a Parallel Sysplex . . . . . . . . . . . . 21053. Pipeplex in a Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . 21954. Using VTAM Generic Resources in a Parallel Sysplex . . . . . . . . . . 223

OS/390 Parallel Sysplex Configuration Volume 2: Cookbook1. Activity Sequence When Configuring a CPC and CF . . . . . . . . . . . . 62. Bipolar and CMOS Technology Trends . . . . . . . . . . . . . . . . . . . . 103. Sample Lock Response Times . . . . . . . . . . . . . . . . . . . . . . . . . 144. Impact of Dynamic CF Dispatching and Shared CF CPs . . . . . . . . . . 365. Sample CF Storage Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546. Effect of Number of DB2 Databases on DB2 SCA Structure Size . . . . 1077. Part 1 of 2 - RMF CF Activity Report - Usage Summary . . . . . . . . . 1918. Part 2 of 2 - RMF CF Activity Report - Usage Summary Continued . . 1929. RMF CF Activity Report - Structure Activity . . . . . . . . . . . . . . . . 194

10. RMF CF Activity Report - Subchannel Activity . . . . . . . . . . . . . . . 19711. RMF Channel Path Activity Report with ICB Channels . . . . . . . . . . 19812. RMF Channel Path Activity Report with IC Channels . . . . . . . . . . . 19813. RMF XCF Activity Report - Usage by System . . . . . . . . . . . . . . . 20014. RMF XCF Activity Report - Usage by Member . . . . . . . . . . . . . . . 20115. Part 1 of 2 - RMF XCF Activity Report - Path Statistics . . . . . . . . . . 20216. Part 2 of 2 - RMF XCF Activity Report - Path Statistics Continued . . . 20317. SYSSUM Report with Sysplex Summary Data . . . . . . . . . . . . . . . 20518. SYSRTD Report with Response Time Data . . . . . . . . . . . . . . . . . 20619. SYSWKM Report for Subsystem CICS . . . . . . . . . . . . . . . . . . . . 20720. SYSENQ Report with Sysplex-Wide Enqueue Delays . . . . . . . . . . . 20821. CFOVER Report with CF Overview Data . . . . . . . . . . . . . . . . . . 20822. CFSYS Report with CFs and Systems Data . . . . . . . . . . . . . . . . . 20923. CFACT Report with CF Activity Data . . . . . . . . . . . . . . . . . . . . . 21024. CACHSUM Report with Cache Subsystem Data . . . . . . . . . . . . . . 21125. ILOCK Report with IRLM Long Lock Detection Data . . . . . . . . . . . 21226. Extract from LIST LOGSTREAM Report . . . . . . . . . . . . . . . . . . . 22227. Display Logger Command Output Examples . . . . . . . . . . . . . . . . 22428. Sample Output of DB2 Group Detail Statistics . . . . . . . . . . . . . . . 228

OS/390 Parallel Sysplex Configuration Volume 3: Connectivity1. FICON vs. ESCON Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62. FICON Use of Fiber Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . 73. Relationship between Logical Paths, ESCDs, and Links . . . . . . . . . . 114. ESCON Logical Paths Configuration . . . . . . . . . . . . . . . . . . . . . . 175. HPR Connections in a Parallel Sysplex . . . . . . . . . . . . . . . . . . . . 306. Session Managers in a Parallel Sysplex . . . . . . . . . . . . . . . . . . . 347. CMC Is Part of Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . 368. CMC Is Outside Parallel Sysplex with Local Connectivity . . . . . . . . . 379. CMC Is Outside Parallel Sysplex and Remote . . . . . . . . . . . . . . . . 38

10. Use of Enterprise Extender in a Mixed SNA/IP Environment . . . . . . . 4311. VIPA Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4512. Dynamic VIPA Takeover Support . . . . . . . . . . . . . . . . . . . . . . . . 4713. Non-Disruptive VIPA Takeover Support . . . . . . . . . . . . . . . . . . . . 4814. Using Telnet to Access 3270 Applications in a Parallel Sysplex . . . . . 5115. Sysplex Distributor Configuration . . . . . . . . . . . . . . . . . . . . . . . 5516. Using 3745 or 3746 As a Network Gateway to a Parallel Sysplex . . . . 6217. Using the OSA As Network Gateway to a Parallel Sysplex . . . . . . . . 66

xiv Parallel Sysplex Configuration, Volume 1: Overview

18. OSA Failure Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6819. OSA Failure Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6920. 9037-2 Expanded Availability Configuration . . . . . . . . . . . . . . . . . . 7921. 9037-1 Expanded Availability Configuration . . . . . . . . . . . . . . . . . . 7922. 9037-2 Basic (One 9037) Configuration . . . . . . . . . . . . . . . . . . . . 8023. 9037-1 Basic (One 9037) Configuration . . . . . . . . . . . . . . . . . . . . 8124. 9037-2 Expanded Basic (One 9037) Configuration . . . . . . . . . . . . . . 8225. 9037-1 Expanded Basic (One 9037) Configuration . . . . . . . . . . . . . . 8226. Dual Port Attachment to 9037 Sysplex Timers . . . . . . . . . . . . . . . . 8427. Example of Two 9672 CPCs with Master Cards and External Cables . . 8528. Example of Four 9672 CPCs with Master Cards and External Cables . . 8629. Sysplex Timer Network in a Bridged LAN Environment . . . . . . . . . . 8830. Non-Configurable Expanded Availability Configuration . . . . . . . . . . . 9431. Non-Supported Migration: All CPCs Continuously Available . . . . . . . 9532. Example of Multisystem Sysplex with 3 CPCs . . . . . . . . . . . . . . . . 9733. Console Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10634. Example Console Configuration . . . . . . . . . . . . . . . . . . . . . . . 11135. The HMC and the Support Elements . . . . . . . . . . . . . . . . . . . . . 11436. RACF Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12737. System Automation for OS/390: Operations Interface . . . . . . . . . . 14538. NetView and System Automation for OS/390 Association in a Sysplex 14639. Example of a Shared SYSRES Parallel Sysplex Environment . . . . . . 15140. Introducing a New Software Level into Parallel Sysplex . . . . . . . . . 15241. Sysplex Couple Data Set Size - Maximum Configuration . . . . . . . . 16442. Sysplex Couple Data Set Size - Likely Configuration . . . . . . . . . . . 16543. REBUILD POPULATECF Command Example . . . . . . . . . . . . . . . . 167

Figures xv

xvi Parallel Sysplex Configuration, Volume 1: Overview

Tables

OS/390 Parallel Sysplex Configuration Volume 1: Overview1. Parallel Sysplex Configuration Roadmap . . . . . . . . . . . . . . . . . . . 62. Pre-9672 G5 CF Level Support . . . . . . . . . . . . . . . . . . . . . . . . . 173. Pre-9672 G5 Parallel Sysplex-related Features . . . . . . . . . . . . . . . 184. Parallel Sysplex Configuration Roadmap - High-Level Design . . . . . . 195. IBM CICS and DFSMS Support of Parallel Sysplex . . . . . . . . . . . . . 236. IBM IMS and DB2 Support of Parallel Sysplex . . . . . . . . . . . . . . . . 247. GRS Ring and Star Comparison (IBM Benchmark: SG24-4356) . . . . . . 378. 'Plex Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 729. Continuous Availability in Parallel Sysplex Roadmap . . . . . . . . . . . 92

10. Rebuild Characteristics in Connectivity Failure . . . . . . . . . . . . . . 12311. Workloads in Parallel Sysplex Roadmap . . . . . . . . . . . . . . . . . . 14512. OS/390 Web Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14613. CICS Web Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15214. Effects of BatchPipes on Charge-Back Variables . . . . . . . . . . . . . 220

OS/390 Parallel Sysplex Configuration Volume 2: Cookbook1. CPC and CF Sizing and Configuration in Parallel Sysplex Roadmap . . 42. CP Speed of IBM 9672 and 9674 Processors . . . . . . . . . . . . . . . . . 103. Coefficient E for IMS DB, DB2 and CICS/VSAM RLS Data Sharing . . . . 214. HSA Size Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275. Structure Failure-Independence Requirements . . . . . . . . . . . . . . . 316. CF Exploitation by Product . . . . . . . . . . . . . . . . . . . . . . . . . . . 337. CFCC Levels and Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 358. Sample CF Access Service Times for Lock Requests . . . . . . . . . . . 439. Reduction in Sender ITR/km CF Link (Synchronous CF Accesses) . . . . 48

10. CF Storage Allocation Estimate Summary . . . . . . . . . . . . . . . . . . 5611. CF Structures Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6112. CF Structure Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . 6413. Structure Rebuild Characteristics . . . . . . . . . . . . . . . . . . . . . . . 6714. Auto Alter Exploiters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7715. XCF Signalling Structures Sizes: MAXMSG(750) . . . . . . . . . . . . . . . 8216. VTAM GR Structure Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8717. RACF Structure Sizes: Database 4096 4 KB Elements . . . . . . . . . . . 9318. RACF Structure Sizes: Database 8192 4 KB Elements . . . . . . . . . . . 9319. JES2 Structure Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9620. JES2 Checkpoint Placement Recommendations . . . . . . . . . . . . . . . 9621. Storage Estimate for the DB2 SCA Structure . . . . . . . . . . . . . . . . 10722. DB2 Buffer Pool Sizing Factors (GBPCACHE CHANGED) . . . . . . . . 11223. DB2 Buffer Pool Sizing Factors (GBPCACHE ALL) . . . . . . . . . . . . 11224. Recommendations for Lock Structure Size . . . . . . . . . . . . . . . . . 11725. Tape Allocation Structure Size . . . . . . . . . . . . . . . . . . . . . . . . 12326. OPERLOG Structure Size . . . . . . . . . . . . . . . . . . . . . . . . . . . 12727. LOGREC Structure Size Specifications . . . . . . . . . . . . . . . . . . . 12828. ECS Recommended Structure Sizes . . . . . . . . . . . . . . . . . . . . . 13529. VSAM LSR Buffer Pool Sizing Example . . . . . . . . . . . . . . . . . . . 13830. Effect of MAXSYSTEM on Lock Table Entry Size . . . . . . . . . . . . . 13931. Sample Lock Structure Allocation Estimates . . . . . . . . . . . . . . . . 14032. Global Requests, Lock Entries, and GRS Lock Structure Size . . . . . 152

Copyright IBM Corp. 2000 xvii

33. BatchPipes Structure Sizing Guide (2- and 4-System Sysplex) . . . . . 15934. BatchPipes Structure Sizing Guide (8- and 16-System Sysplex) . . . . 15935. Tools Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16136. Functional Differences between IBM 9672-Based CPCs . . . . . . . . . 23337. IBM S/390 Parallel Enterprise Server G6 - MSU Values . . . . . . . . . 23938. IBM S/390 Parallel Enterprise Server G5 - MSU Values . . . . . . . . . 24039. IBM S/390 Parallel Enterprise Server G4 - MSU Values . . . . . . . . . 24140. IBM 9674-C05 - MSU Values . . . . . . . . . . . . . . . . . . . . . . . . . . 24141. IBM S/390 Parallel Enterprise Server G3 - MSU Values . . . . . . . . . 24242. IBM 9674-C04 - MSU Values . . . . . . . . . . . . . . . . . . . . . . . . . . 242

OS/390 Parallel Sysplex Configuration Volume 3: Connectivity1. Connectivity in the Parallel Sysplex Roadmap . . . . . . . . . . . . . . . . 52. Network Connectivity for the Parallel Sysplex Roadmap . . . . . . . . . . 233. HPR Support in IBM Products . . . . . . . . . . . . . . . . . . . . . . . . . 314. TCP/IP Workload Balancing Options . . . . . . . . . . . . . . . . . . . . . . 585. Sysplex Timer Considerations Roadmap . . . . . . . . . . . . . . . . . . . 726. Consoles and Parallel Sysplex Roadmap . . . . . . . . . . . . . . . . . . 1047. Systems Management Products in a Parallel Sysplex Roadmap . . . . 1258. IBM Systems Management Software Supporting the Parallel Sysplex 1269. References Containing Information on the Use of System Symbols . . 157

xviii Parallel Sysplex Configuration, Volume 1: Overview

Preface

The Parallel Sysplex Configuration redbooks consist of three volumes that helpyou design and configure a Parallel Sysplex. These redbooks assist you inmaking the key hardware and software choices necessary to meet yourperformance, capacity, and availability requirements.

In addition to discussing many sysplex-related topics not documented elsewhere,these books contain, for the IBM large systems customer, practical informationthat has been collected from many sources and brought together into a single,comprehensive reference.

The redbooks provide rules of thumb for configuring a Parallel Sysplex. Theydiscuss central processing complex sizing, CF sizing, the Sysplex Timer,connectivity guidelines, the implications of Parallel Sysplex for IBM softwaresuch as OS/390 V2R8 and related subsystem configurations, and useful tools andservices. Network, systems management and availability considerations arealso discussed.

These redbooks are a starting point for those involved in designing andconfiguring a Parallel Sysplex. They also may be used by those already havinga Parallel Sysplex when further exploitation is being considered. The booksrefer to other relevant publications that cover specific areas in more depth.

These books are an update to an existing set of books:

OS/390 MVS Parallel Sysplex Configuration Volume 1: Overview, SG24-2075OS/390 MVS Parallel Sysplex Configuration Volume 2: Cookbook, SG24-2076OS/390 MVS Parallel Sysplex Configuration Volume 3: Connectivity,SG24-2077

New information since the last books were published (January 1998) has beenadded, and information pertaining to back levels of hardware and software hasbeen removed. The old books continue to be orderable for customers usingthose hardware or software levels.

The Team That Wrote These RedbooksThese projects were designed, managed and partly documented by:Henrik Thorsen International Technical Support Organization, PoughkeepsieFrank Kyne International Technical Support Organization, Poughkeepsie

The authors of these documents are:Erik Bach IBM DenmarkChris Barwise IBM UKFrank Byrne IBM UKDave Clitherow IBM UKRichard Cwiakala IBM USANoshir Dhondy IBM USALuiz Fadel IBM BrazilLinda Ferdinand IBM USAMichael Ferguson IBM AustraliaPer Fremstad IBM NorwayRafael Garcia IBM USAKeith George IBM UKDick Jorna IBM NetherlandsKazumasa Kawaguchi IBM Japan

Kuno Kern IBM GermanyMasayuki Kojima IBM JapanFrank Kyne IBM IrelandGeoff Leong IBM CanadaJock Marcks IBM AustraliaDennis Moore IBM USAAlan Murphy IBM IrelandRachel Pickering IBM UKAlvaro Salla IBM BrazilAlain Servais IBM FranceKeld Teglgaard IBM DenmarkHenrik Thorsen IBM DenmarkIan Waite IBM UK

Copyright IBM Corp. 2000 xix

These publications are the result of a series of residencies conducted at the InternationalTechnical Support Organization, Poughkeepsie Center. However, much valuableinformation and assistance was received from various organizations across IBM. Theproduction of these redbooks is a team effort. The organizations that participated include:

Washington System Center, US M&SS/390 Parallel Center, S/390 DivisionS/390 Development, S/390 Division

Also, thanks to the following people for reviewing these publications, providing material and offeringinvaluable advice and guidance. Their contributions have made these books eminently more accurateand more readable than they otherwise might have been.

Bil l Abbot ComputerAssociates

Robert M. Abrams IBM USARiaz Ahmad IBM USAMichelle Ahearn Computer

AssociatesJay Aiken IBM USAMaria Sueli Almeida ITSO AlmadenSteve Anania IBM USAJaime Anaya IBM USAStuart Arnell IBM UKJohn Arwe IBM USAAlan Ashcroft IBM UKJudi Bank IBM USAPaola Bari IBM USAHarley Beier IBM USAEd Berkel IBM USAEvanne Bernardo IBM USAConnie Beuselinck IBM USAPage Borchetta IBM USAMark Brooks IBM USAJane Brockbank IBM UKPaolo Bruni ITSO AlmadenFred Bucci IBM USACharlie Burger IBM USAChuck Calio IBM USAJohn Campbell IBM USAWalt Caprice IBM USABernice Casey IBM USAJose Castano IBM USAAlfred Christensen IBM USARich Conway IBM USARoy Cornford IBM UKCurt Cotner IBM USAMike Cox IBM USAMary Crisman IBM USAVijay Dalal IBM USAJerry Dearing IBM USAMac Devine IBM USAGeorge Dillard IBM USAKristin Donceel IBM BelgiumTerry Draper IBM UKJames Doyle IBM USAJoann Duguid IBM USAGregory Dunlap IBM USAMary Duval IBM USAGreg Dyck IBM USAScott Fagen IBM USABob Fantom IBM USAErika Fenn Software AGMartin Ferrier IBM UKJohn Fitch IBM USAJonathan Follows IBM ITSO RaleighJim Fyffe IBM USADavid Fox IBM USA

Peter Garvin IBM USARob Geiner IBM USATony Giaccone IBM USARusty Godwin IBM USADionne Graff IBM USASteve Grabartis IBM USAJennifer Green IBM USASusan Greenlee IBM USABob Haimowitz IBM USAJim Hall IBM USADick Hannan IBM USARicardo Haragutchi IBM BrazilJohnathan Harter IBM USAEvan Haruta IBM USASteve Heinbuch IBM CanadaAudrey Helffrich IBM USAGary Henricks IBM USASteffen Herzog IBM GermanyKen Hewitt IBM AustraliaJuergen Holtz IBM GermanyJanice Hui IBM AustraliaPatricia Jakubik IBM USADave Jeffries IBM FranceCurt Jews IBM USAKim Johnson IBM USAPaul Johnson IBM UKMichael Jordan IBM USAJeff Josten IBM USAKen Kapulka IBM USAGary King IBM USACarl Klitscher IBM New ZealandRaimo Korhonen CompMeas, FinlandJeff Kubala IBM USARavi Kumar IBM USAPaul Lekkas IBM GermanyDave Levish IBM USARich Z. Lewis IBM USAPatrick LiVecchi IBM USAAlan Little IBM USASherman Lockard IBM USANick Matsakis IBM USAJohn Matush IBM USAGeoff Meacock IBM UKKershaw Mehta IBM USAGeoff Mi l ler IBM USARoy Moebus IBM USAThomas Murphy IBM USAStephen Nichols IBM USAArthur Neil IBM UKGeoff Nicholls IBM AustraliaJeff Nick IBM USAMadeline Nick IBM USARon Northrup IBM USAMarianne Orman IBM UKRon Parrish IBM USA

Betty Patterson IBM USAAnthony Pearson IBM USAKris Perry IBM USAJames Perlik IBM USADavid B. Petersen IBM USASilvio Podcameni IBM BrazilPaul Powell IBM USARichard Prewitt IBM USADavid Raften IBM USATom Rankin IBM USAPatrick Rausch IBM USAFrank Ricchio IBM USAPaul Rogers IBM USATom Russell IBM CanadaEugene Sale IBM USAHany Salem IBM USAChris Saul IBM USAJonathan Scott IBM UKJoachim Schmalzried IBM GermanyMartin Schulman IBM USATom Shaw IBM USAJoseph Shelak IBM USASim Schindel IBM FranceCarol Shanesy IBM USABeth Somers IBM USAOrrin Stevens Computer

AssociatesNat Stevenson IBM USABil l Stil lwell IBM USAJeff Stokes IBM USADavid Surman IBM USAJohn Swift IBM UKLisa Taylor IBM USAClarisse Taaffe-Hedglin IBM USAMichael Teuffel IBM GermanyDouglas Trimboli IBM USAKen Trowell IBM USAJohn Tuccio IBM USASøren Understrup IBM DenmarkRobert Vaupel IBM GermanyAmbrose Verdibello IBM USAPhil Wakelin ITSO AlmadenKathy Walsh IBM USASharon Wang IBM USAJames Warnes IBM USAVern Watts IBM USAKlaus Wegele IBM GermanyMaryela Weihrauch IBM USASuzie Wendler IBM USAGail Whistance IBM USADavid Whitney IBM USAJoseph Wi nkelbauer IBM USALynn Wi nkelbauer IBM USADavid Yackel IBM USADouglas Zobre IBM USA

Further thanks to the staff and the editorial team of the International Technical Support Organization,Poughkeepsie.

xx Parallel Sysplex Configuration, Volume 1: Overview

To everyone else who contributed, our sincere thanks.

Comments welcome

Your comments are important to us!

We want our redbooks to be as helpful as possible. Please send us your comments about this or otherredbooks in one of the following ways:

Fax the evaluation form found in “IBM Redbooks evaluation” on page 277 to the fax number shownon the form.

Use the online evaluation form found at http://www.redbooks.ibm.com/

Send your comments in an Internet note to [email protected]

Preface xxi

xxii Parallel Sysplex Configuration, Volume 1: Overview

Chapter 1. Introduction to the Configuration of a Parallel Sysplex

These redbooks are designed as cookbooks. They contain a series of “recipes”you may use to design your new Parallel Sysplex or enhance your existingParallel Sysplex. Which “recipes” you use and in what order you use themdepends on what you want to achieve: the books are not intended to be readsequentially. These books should not be your only source of information. Theyare intended to consolidate other sources of information at a suitable level ofdetail.

For a list of the sources used, refer to:

“Recommended Sources of Further Information” at the beginning of eachchapter.

The packages and books listed in Appendix B, “Related Publications” onpage 237.

Recommended Sources of Further Information

The following sources provide support for the information in this chapter:

OS/390 MVS Setting Up a Sysplex, GC28-1779

OS/390 Parallel Sysplex Overview: Introducing Data Sharing andParallelism in a Sysplex, GC28-1860

The Parallel Sysplex task roadmap, available on the Web site:http://www.s390.ibm.com/pso/task.html#roadmap

1.1 How to Use These Redbooks - Read This FirstThis section contains a series of roadmaps to guide you through the books in afast and efficient way. The overall roadmap is like a long-distance route plannerto determine your overall path through the books. Each chapter then contains amore detailed “local roadmap” to guide you through the chapter efficiently.

Each chapter also contains a list of sources of relevant informationrecommended for further reading.

Copyright IBM Corp. 2000 1

Welcome to the S/390 Parallel Sysplex Home Page!

In line with IBM's direction to provide more information via the Internet, thereis now a Web site dedicated to the topic of Parallel Sysplex. The site isintended to provide up-to-date Parallel Sysplex-related information, and to actas a hub from which you can be directed to other relevant sites. To makethe use of this site easier, you should add the address of the following Website to your bookmarks in your Web browser:http://www.s390.ibm.com/pso/

At the time of writing, this Web site was organized into the following sections:

News and events related to Parallel Sysplex.

Library and education pages. These contain information about ParallelSysplex education offerings, information about the announced CF Levels,and a list of the available Parallel Sysplex-related white papers.

Availability characteristics. These are short documents that discuss howOS/390 features can be used to improve the availability of your ParallelSysplex.

Announcement information for latest IBM S/390 processors and majorsoftware products.

Parallel Sysplex customer success stories.

IBM Parallel Sysplex-related service offerings. This includes informationon the IBM certification program and the Geographically DispersedParallel Sysplex offering.

Downloads. This provides a link to the OS/390 Integration Test Web sitewhere sample Parmlib members and other system definitions can bedownloaded.

Support. This provides links to the OS/390 Enhanced HOLDDATA site, theWashington Systems Center FLASHes, and information about howVM/ESA can be used to create a test Parallel Sysplex environment.

2 Parallel Sysplex Configuration, Volume 1: Overview

A Few Words on the Terminology Used in These Books

To avoid confusion, you should read the following list of terms as usedthroughout these books. The terms that we use may be different than youhave seen elsewhere, but they are used consistently within these books:

CF rather than Coupling Facility partition.

CF link rather than coupling link, ISC link, or Coupling Facility channel.

CFCC rather than Coupling Facility Control Code.

CP rather than CPU, engine, or processor.

CPC rather than CEC, processor, model, computer, server, or machine.

DASD rather than disk.

External link identifies any CF link that connects between different CPCs.This includes ISC (50 MB and 100 MB) links and ICB (copper) links.

Failure-Independent CF identifies any CF that resides in a CPC that doesnot contain any other images (CF or OS/390) from the same sysplex.

Fiber link rather than ISC or real link. Contrast fiber links to ICs (internal)and ICBs (copper) CF links.

GBP rather than group buffer pool.

IC rather than Internal Coupling Link.

ICB rather than Integrated Cluster Bus or Cluster Bus Link.

ICF rather than Internal Coupling Facility.

Image rather than OS/390 image.

LP rather than PR/SM LPAR partition, partition, logical partition, or LPAR.

MVS and OS/390 are used in the same context.

Parallel Sysplex, rather than Parallel Sysplex cluster.

RACF and OS/390 Security Server are used in the same context.

Standalone CF rather than external CF or 9674.

S/390 Partners in Development, rather than solution developers, businesspartners, or independent software vendors (ISVs).

Storage rather than memory or RAM.

Sysplex rather than complex or systems complex.

VTAM, OS/390 Communications Server, e-network CommunicationsServer, SecureWay Communications Server, and Communications Serverfor OS/390 are all used in the same context.

For more information on the terms and acronyms used in this book, refer tothe sections “Glossary” on page 249 and “List of Abbreviations” onpage 269.

Chapter 1. Introduction to the Configuration of a Parallel Sysplex 3

Table 1 is the “roadmap” that will help to guide your route through the book.The roadmap can help you read the right information in the right order.

Table 1 (Page 1 of 2). Parallel Sysplex Configuration Roadmap

You wanttoconfigure:

If you are especially interested in: Then refer to:

Parallel Sysplex

An introduction to these books and tothe task of configuring a ParallelSysplex:

How to use these booksWhat is a Parallel Sysplex?What are the advantages of aParallel Sysplex?

Chapter 1, “Introduction to the Configuration of aParallel Sysplex” on page 1

High level design concepts for ParallelSysplex

What are other “plexes”?

Chapter 2, “High-Level Design Concepts forParallel Sysplex” on page 17

Availability in Parallel SysplexSoftware considerationsHardware considerationsNetwork considerationsDisaster recovery

Chapter 3, “Continuous Availability in ParallelSysplex” on page 89

Workloads and Parallel Sysplexe-business workloadsTransaction managementDatabase managementBatch workloadsTSO/E in a Parallel Sysplex

Chapter 4, “Workloads in Parallel Sysplex” onpage 141

CPC and CF configurationTechnology choicesCF optionsCPC capacitiesCF exploiters

1.0, “CPC and CF Configuration in Parallel Sysplex”on page 3 in Volume 2: Cookbook, SG24-5638

Details of CF structures:What size should I define?Where should I place my structures?What about CF volatility?What about structure rebuild?

2.0, “A Structured View of CF Structures” onpage 59 in Volume 2: Cookbook, SG24-5638

Parallel Sysplex connectivity 1.0, “I/O Connectivity in Parallel Sysplex” onpage 3 in Volume 3: Connectivity, SG24-5639

Network connectivityGeneral issuesGateway options

2.0, “Network Connectivity for Parallel Sysplex” onpage 21 in Volume 3: Connectivity, SG24-5639

Sysplex Timer considerations 3.0, “Sysplex Timer Considerations” on page 71 inVolume 3: Connectivity, SG24-5639

How do I configure consoles in aParallel Sysplex?HMC considerations

4.0, “Consoles and Parallel Sysplex” on page 103in Volume 3: Connectivity, SG24-5639


Table 1 (Page 2 of 2). Parallel Sysplex Configuration Roadmap

You wanttoconfigure:

If you are especially interested in: Then refer to:

Systems management in ParallelSysplex

RACFPerformance productsOperations managementSMP/EHCD

Appendix A, “Systems Management Products forParallel Sysplex” on page 123 in Volume 3:Connectivity, SG24-5639

Tools to assist in the process ofdesigning Parallel Sysplexconfigurations

Appendix A, “Tools and Services Catalog” onpage 161 in Volume 2: Cookbook, SG24-5638

RMF reports related to Parallel Sysplex Appendix B, “RMF Reporting in Parallel Sysplex”on page 189 in Volume 2: Cookbook, SG24-5638

Tuning DB2 GBP, SCA and IRLMstructures

Appendix D, “Tuning DB2 Structures” on page 225in Volume 2: Cookbook, SG24-5638

What hardware features are availableon which IBM CPCs (for 9672 G4 andprior models, 9674, 9021-711, and 9121,see Table 2 on page 15 and Table 3on page 16 in this chapter). Also seeSG24-2075, SG24-2076, SG24-2077.

Appendix E, “Functional Differences between IBM9672-Based CPCs” on page 233 in Volume 2:Cookbook, SG24-5638

1.2 The Purpose of These BooksThese books have been written to help you configure a Parallel Sysplex. Theemphasis in these books is on high level configuration design. By this we meanthat you:

Order the right hardware and software.Decide how it will work. For example:− Will all subsystems run on every image?− How will my subsystems work in normal operation?− How will I operate and manage the subsystems/sysplex?− What happens if a component fails?− How will systems management functions work across the sysplex?

The books are not designed to help you justify the use of a Parallel Sysplex, norto implement it (install it and make it work). They are designed with twopurposes in mind:

1. For new users of Parallel Sysplex.

To help you make the initial design decisions so that implementation doesnot uncover additional hardware or software that is needed, or fundamentalmisconceptions about how it should work.

2. For existing users of Parallel Sysplex.

To provide information regarding enhancing your Parallel Sysplex by utilizingthe latest functions and facilities of new hardware and software.


For information to help you estimate the value of Parallel Sysplex to yourbusiness, ask your IBM representative for a copy of the BV390 Package fromMKTTOOLS.

Initially you will probably only be interested in approximate sizings. Later youwill need to be sure that the hardware and software ordered is both correct andcomplete. If you have not thoroughly considered the operational issues listed inthese books, you may find that you have to make alterations at a later date.

You can expect to make several iterations through the books, at different levelsof detail, as your Parallel Sysplex evolves from an idea to a firm decision.

These books bring together new information and information that is alreadyavailable but scattered among many different sources. They contain the latestinformation based on the experience at the time of writing. As Parallel Sysplexcontinues to evolve, you should always check the latest information.

These books contain information about the environments that are capable ofbeing configured now. These are primarily the DB2, IMS TM, IMS DB, CICS, andCICS/VSAM RLS transaction processing and data sharing environments. Thesebooks also contain information about other exploiters including OS/390 functionssuch as JES, RACF, VTAM, BatchPipes, system logger, shared tape, EnhancedCatalog Sharing, and GRS.

Recommendation to Check Background Information

The content of these books is based on many sources. These sourcesinclude information from relevant Web sites, FORUMS, whitepapers, otherredbooks, product documentation and so forth.

For a deeper understanding of the background information, always check thedetailed information.

1.3 Base Hardware Levels for These BooksThe minimum hardware level required for all the facilities described in this bookis the IBM 9672 G5 CPC. Many, but not all, of the facilities will work on previouslevels of CPCs. Mixed levels of hardware may cause some functions of aParallel Sysplex to work differently.

Information about the older levels of hardware—9021s, 9121s, 9674s, and pre-G59672s—was removed to make the document more readable. Also, at the time ofwriting many customers have moved to the G5 and later generations of 9672CPCs, so it was felt that the old information was no longer required.

More information regarding facilities available on previous levels of CPCs can befound in 1.8, “Function Support Table for Pre-G5 9672 CPCs” on page 15. Also,for those customers that are still using these older levels, the previous level ofthese books is still orderable. The names and order numbers are:

OS/390 MVS Parallel Sysplex Configuration Volume 1: Overview, SG24-2075OS/390 MVS Parallel Sysplex Configuration Volume 2: Cookbook, SG24-2076OS/390 MVS Parallel Sysplex Configuration Volume 3: Connectivity,SG24-2077


1.4 Main Reasons for Parallel SysplexThe main reasons for moving to a Parallel Sysplex are briefly discussed here.They are covered more thoroughly in OS/390 Parallel Sysplex Overview:Introducing Data Sharing and Parallelism in a Sysplex, GC28-1860.

For a discussion of the key design points for Parallel Sysplex, refer to “S/390Cluster Technology: Parallel Sysplex,” in IBM Systems Journal, Volume 36 No. 2,G321-0127.

1.4.1 Continuous Application AvailabilityWhen you have a single copy of any system component, hardware, software, ordata, you are inevitably exposed to system outages because of either failure ofthe component or because of planned changes to the component that require itto be taken offline.

One of the goals of Parallel Sysplex is to eliminate the impact that scheduledoutages have on application availability, and minimize the effects of anunscheduled outage by allowing work to continue executing on the remainingsystems in the sysplex. This requires, among other things, that the system bedesigned for redundancy within the Parallel Sysplex. Applications must becapable of running across multiple systems, with access to the data beingpossible from at least two systems. If at least two instances of a resource exist,your applications can continue to run even if one of the resources fails.

It is important to remember that Parallel Sysplex only provides the facilities forcontinuous availability. Parallel Sysplex on its own will not eliminate scheduledor unscheduled application outages; the application itself must also be designedfor continuous availability. Sharing data is only one part of this design.

A classic example is where IMS databases are made unavailable to online usersto allow for batch updates. These batch update programs could be re-written asBatch Message Processing (BMP) programs which can perform updates whilestill having the databases available for online users.

Another example is that data in a DB2 database may be unavailable while it isbeing reorganized (and this will occur no matter how many data sharingmembers you have). DB2 for OS/390 reduces the impact of database reorgs bythe provision of the online reorg utility. This utility allows full access to the datafor most of the time that the reorg is taking place.

Similar considerations apply to other IBM subsystems.

IBM also provides a facility called FlashCopy which allows data on DASDvolumes to be copied while it is still being accessed by online users. FlashCopyis available in the 2105 Enterprise Storage Server. FlashCopy is completelyself-contained within the 2105 and does not require any additional host software.

On the earlier Ramac Virtual Array (RVA) DASD subsystem, a similar functioncalled SnapShot was available. SnapShot is a combination of hardware featuresin the RVA and host software features provided by IBM Extended FacilitiesProduct (IXP). Both products interact with DFSMSdss to provide high speeddumps, drastically reducing the impact of data backups.


More discussion on continuous application availability is found in Chapter 3,“Continuous Availability in Parallel Sysplex” on page 89, and in the ParallelSysplex Availability Checklist white paper, available via the Parallel Sysplexhome page, or directly from:http://www.s390.ibm.com/ftp/marketing/position/availchk_parsys.pdf

1.4.2 Workload BalancingWithout workload balancing, installations with multiple CPCs have often had toupgrade one CPC to provide more capacity, while another CPC may have hadspare capacity. The alternative was to redistribute work manually, which istime-consuming and can only handle short-term imbalances. Manualintervention also has the potential to introduce errors and could not be used todynamically and constantly balance workload across the installed CPCs.

With a Parallel Sysplex, the basic framework exists for workload balancing.There is a small cost for doing workload balancing in a Parallel Sysplex, but thecost by and large is not related to the number of systems, and the workloadbalancing advantages are significant. Many subsystems exploit this frameworkto redistribute work across systems, thus allowing the systems to be run athigher utilizations, and allowing spare capacity anywhere in the Parallel Sysplexto be used to satisfy the demands of the workload.

All the major IBM subsystems—CICS, DB2, IMS, MQSeries, TSO, VTAM, TCP/IP,and JES—contain support for Parallel Sysplex workload balancing. This isdiscussed further in 2.6, “Dynamic Workload Balancing in Parallel Sysplex” onpage 83. More discussion on subsystem exploitation of workload balancing isfound in Chapter 4, “Workloads in Parallel Sysplex” on page 141.

1.4.3 Nondisruptive Addition of Scalable CPC CapacityIf a part of your workload cannot be split between multiple images or has grownlarger than the largest available single-image system, Parallel Sysplex may offera solution to allow you to add additional capacity (both MIPS and CPC storage)without rewriting the applications or re-partitioning the databases. For example,it may allow multiple instances of the application to share the data acrossmultiple images.

More discussion on how to share data in a Parallel Sysplex is found in 4.3,“Database Management in Parallel Sysplex” on page 194.

1.4.3.1 GranularityOnce a Parallel Sysplex has been set up, the increments in which capacity canbe added can vary from small to large quite easily. Capacity is available ondemand. You can match your investment in hardware more closely to thebusiness needs, rather than being forced to take large capacity (and cost)increments at arbitrary points in your workload growth. This is particularly trueif you have a number of different CPC types installed, and the flexibility to runyour work on any of those CPCs. So if you require an additional 200 MIPS, youupgrade the CPC with 200 MIPS engines. If you just require an additional 100MIPS, then you upgrade the CPC with 100 MIPS engines.

Another option is to add an additional CPC. The ability with Parallel Sysplex torun work on any image, coupled with the cloning facilities in OS/390, make it farsimpler to add a new image than was the case in the past. The feasibility of thisoption is an important consideration when designing your Parallel Sysplex. A


new CPC can be added quickly, providing ESCON director ports are available foraccess to the I/O subsystems, and extensive use of symbolic parameters permitsthe cloning of software.

For more information on CPCs participating in a Parallel Sysplex, refer to 1.0,“CPC and CF Configuration in Parallel Sysplex” on page 3 in Volume 2:Cookbook, SG24-5638.

For more information on connectivity in a Parallel Sysplex, refer to 1.0, “I/OConnectivity in Parallel Sysplex” on page 3 in Volume 3: Connectivity,SG24-5639.

1.4.3.2 Just-In-Time Nondisruptive GrowthWith a Parallel Sysplex, more options exist for providing more capacity. Theprevious options were to upgrade or replace existing CPCs (vertical growth).The Parallel Sysplex allows new CPCs to be added alongside existing CPCs(horizontal growth). There is also the hybrid option of both vertical andhorizontal growth.

Which option you choose will depend on several factors, including:

Availability requirementsWhether there is an upgrade pathWhether there is a larger CPC availableCP speed considerationsRestrictions imposed by existing configurationCost

A Parallel Sysplex may allow CPCs to be added or removed nondisruptively.Upgrades might be accommodated by taking a CPC out of the Parallel Sysplexand upgrading it, while continuing to run the workload on the remaining CPCs inthe Parallel Sysplex. The upgraded CPC can then be reintroduced to the Par

os/390 parallel sysplex configuration volume 1: overvie · 2000. 9. 12. · os/390 parallel sysplex...

Documents