utilities

ibm.com/redbooks

DB2 for z/OS and OS/390 Version 7Using the Utilities Suite

Paolo BruniMarcelo Antonelli

Tom CrockerDavy GoethalsRama Naidoo

Bart Steegmans

Description of the new utilities and the recently enhanced functions

Recommendations for best performance and availability

Advice for operational scenarios

Front cover

http://www.redbooks.ibm.com/


International Technical Support Organization

DB2 for z/OS and OS/390 Version 7 Using the Utilities Suite

August 2001

SG24-6289-00

© Copyright International Business Machines Corporation 2001. All rights reserved.Note to U.S Government Users - Documentation related to restricted rights - Use, duplication or disclosure is subject to restrictions setforth in GSA ADP Schedule Contract with IBM Corp.

First Edition (August 2001)

This edition applies to Version 7 of IBM DATABASE 2 Universal Database Server for z/OS and OS/390 (DB2 for z/OS and OS/390 Version 7), Program Number 5675-DB2, and the DB2 Version 7 Utilities Suite, Program Number 5697-E98.

Comments may be addressed to:IBM Corporation, International Technical Support OrganizationDept. QXXE Building 80-E2650 Harry RoadSan Jose, California 95120-6099

When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

Take Note! Before using this information and the product it supports, be sure to read the general information in “Special notices” on page 299.

Contents

Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiThe team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiSpecial notice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xixIBM trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xixComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Part 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 Contents of this redbook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Brief overview of all DB2 Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 Online utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.2 Stand-alone utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Utilities at work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.4 Major changes with DB2 V7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.4.1 Functional enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.4.2 New packaging of utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Part 2. Planning for DB2 Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Chapter 2. Wildcarding and Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.1 Prior to Version 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2 Wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.1 What is Wildcarding? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.2 Benefits of using Wildcards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.3 How to use Wildcarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.4 Using multiple Wildcards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.2.5 LISTDEF expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.2.6 Recommendations for the use of Wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2.7 Current restrictions in the use of Wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.3.1 Benefits of using Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.3.2 How to use Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.3.3 Naming standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.3.4 Intelligent data set sizing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.3.5 Recommendations for the use of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.3.6 Current restrictions in the use of Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4 Combining Wildcards and Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.4.1 Compatibility with utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.4.2 Mixing the old and the new . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.4.3 Object in restricted status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.4.4 Utility restartability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

© Copyright IBM Corp. 2001 iii

2.4.5 Use with partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.5 OPTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.5.1 OPTIONS PREVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.5.2 Other OPTIONS functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Chapter 3. Inline executions and parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.1 Inline executions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.1.1 Inline COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.1.2 Inline RUNSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.1.3 Enhanced DISPLAY UTILITY for Inline COPY and statistics . . . . . . . . . . . . . . . . 483.1.4 Converting to inline executions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.2 COPY and RECOVER of a list of DB2 objects in parallel . . . . . . . . . . . . . . . . . . . . . . . 483.2.1 COPY in parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.2.2 RECOVER in parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.2.3 Restrictions for parallel COPY and RECOVER. . . . . . . . . . . . . . . . . . . . . . . . . . . 503.2.4 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.2.5 Converting COPY/RECOVER jobs to use parallelism . . . . . . . . . . . . . . . . . . . . . 51

3.3 LOAD and REORG Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.3.1 SORTKEYS with DB2 V5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.3.2 SORTKEYS with DB2 V6 and DB2 V7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.3.3 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.3.4 Dropping NPIs before REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.3.5 Dropping NPIs before LOAD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.3.6 Converting LOAD/REORG jobs to use SORTKEYS. . . . . . . . . . . . . . . . . . . . . . . 573.3.7 REORG INDEX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.4 REBUILD INDEX Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.4.1 Rebuilding the partitioning index of a partitioned table space using PIB . . . . . . . 583.4.2 Rebuilding a non-partitioning index using PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.4.3 Rebuilding all indexes of a partitioned table space. . . . . . . . . . . . . . . . . . . . . . . . 593.4.4 Rebuilding all indexes of a non-partitioned table space . . . . . . . . . . . . . . . . . . . . 603.4.5 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.4.6 Enabling PIB with REBUILD INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.5 Partition parallelism with the LOAD Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.5.1 LOAD partition parallelism without PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633.5.2 LOAD partition parallelism with PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.5.3 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.6 Partition parallelism with the UNLOAD Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663.6.1 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.7 Considerations for running parallel subtasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Chapter 4. Triggering and controlling executions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694.1 Triggering limits prior to DB2 V7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

4.1.1 REORG INDEX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.1.2 REORG TABLESPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.1.3 COPY utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.1.4 Trigger limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

4.2 New values with DB2 V7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.2.1 REORG index with LEAFNEAR and LEAFFAR . . . . . . . . . . . . . . . . . . . . . . . . . . 774.2.2 When to run RUNSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.2.3 VSAM data set extents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.2.4 Reclaiming dead space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 814.2.5 LOB space management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

4.3 Trends from historical statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

iv DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

4.3.1 Monitoring space growth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844.3.2 Compression dictionary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

4.4 Real-Time Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854.4.1 Real-Time Statistics tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.4.2 Description of statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.4.3 Impact of utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874.4.4 Maintaining in-memory statistics in data sharing . . . . . . . . . . . . . . . . . . . . . . . . . 884.4.5 Statistics accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.4.6 DSNACCOR stored procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

4.5 DSNUTILS and Control Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.6 Data administration tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.6.1 DB2 Administration Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944.6.2 DB2 Automation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Chapter 5. Managing partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015.1 Why partition? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1025.2 Altering partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035.3 SQL and partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1045.4 Utilities and partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.4.1 Partition independence and parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055.4.2 Non-partitioning indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Part 3. Executing DB2 Utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Chapter 6. Loading data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1096.1 Inline COPY and Inline RUNSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

6.1.1 Inline COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1106.1.2 Inline RUNSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1106.1.3 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

6.2 Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1116.3 Partition parallelism within one LOAD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

6.3.1 Partition parallelism without Parallel Index Build. . . . . . . . . . . . . . . . . . . . . . . . . 1196.3.2 Partition parallelism with Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

6.4 Preformat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1276.5 Reuse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1286.6 Cross Loader. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

6.6.1 Using SQL statements in the utility input stream . . . . . . . . . . . . . . . . . . . . . . . . 1306.6.2 Using the Cross Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

6.7 Online LOAD Resume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1486.7.1 Invoking Online LOAD Resume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1486.7.2 Behavior of Online LOAD Resume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1486.7.3 Online LOAD Resume performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1516.7.4 Online LOAD Resume examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

6.8 Conclusions and recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Chapter 7. Reorganizing data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1617.1 Why REORG ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1627.2 When to REORG ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1627.3 Types of REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

7.3.1 SHRLEVEL NONE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1637.3.2 SHRLEVEL REFERENCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1647.3.3 SHRLEVEL CHANGE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1647.3.4 Availability summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1657.3.5 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Contents v

7.4 Planning for Online REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1667.5 Shadow data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1667.6 Phases of Online REORG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

7.6.1 UNLOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1677.6.2 RELOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1677.6.3 SORT and BUILD as compared to SORTBLD . . . . . . . . . . . . . . . . . . . . . . . . . . 1707.6.4 LOG phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1717.6.5 SWITCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1717.6.6 BUILD2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

7.7 Controlling Online REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767.7.1 DRAIN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767.7.2 DRAIN_WAIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1777.7.3 RETRY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1787.7.4 RETRY_DELAY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1787.7.5 MAXRO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1797.7.6 LONGLOG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807.7.7 DELAY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807.7.8 TIMEOUT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807.7.9 DEADLINE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1817.7.10 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7.8 Further options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1817.8.1 LISTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1817.8.2 SORTKEYS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.8.3 SORTDATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.8.4 NOSYSREC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.8.5 DFSORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837.8.6 KEEPDICTIONARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837.8.7 REUSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1847.8.8 PREFORMAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

7.9 DISCARD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1857.10 UNLOAD EXTERNAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1877.11 Index REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1887.12 Catalog reorganization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1887.13 Inline Utilities with REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

7.13.1 Inline COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1897.13.2 Inline RUNSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Chapter 8. Unloading data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1918.1 Output data sets from UNLOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1928.2 UNLOADing data prior to V7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1938.3 UNLOAD from image copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

8.3.1 UNLOAD using FROMCOPY and FROM TABLE. . . . . . . . . . . . . . . . . . . . . . . . 1968.3.2 Image copy from compressed table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1978.3.3 Advantages of UNLOAD using image copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

8.4 Unload data from table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1988.4.1 Unloading from a list of table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1998.4.2 Unload by partition and parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2008.4.3 UNLOAD with SHRLEVEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2028.4.4 Unload from table space using the FROM TABLE option. . . . . . . . . . . . . . . . . . 2028.4.5 Converting the character output data to other internal code. . . . . . . . . . . . . . . . 204

8.5 Terminating or restarting UNLOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2078.6 Creating a shadow catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

vi DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Chapter 9. Recovering data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2099.1 RECOVER using the LISTDEF command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2109.2 Parallel RECOVER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

9.2.1 Restriction for parallel RECOVER. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2129.3 LOGONLY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

9.3.1 Copy taken outside DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2139.3.2 DB2 object names with REORG and FASTSWITCH . . . . . . . . . . . . . . . . . . . . . 2159.3.3 Recovering a DB2 object from Concurrent COPY . . . . . . . . . . . . . . . . . . . . . . . 2159.3.4 Recovering without an image copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2159.3.5 Recovering a single piece of a multi-linear page set. . . . . . . . . . . . . . . . . . . . . . 2169.3.6 LOGONLY recovery improvements in V7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

9.4 Log Apply considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2169.5 Fast Log Apply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

9.5.1 Fast Log Apply buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2189.5.2 Sort the log records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2189.5.3 When is Fast Log Apply bypassed? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

9.6 Index recovery from COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2199.7 Modify enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2219.8 REPORT RECOVERY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2229.9 DB2 Change Accumulation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2249.10 QUIESCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2259.11 CHECK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

9.11.1 CHECK INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2269.11.2 CHECK DATA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2289.11.3 CHECK LOB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Chapter 10. Copying data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23110.1 COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

10.1.1 Inline COPY with LOAD and REORG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23210.1.2 Copying indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23210.1.3 Lists. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23510.1.4 COPY parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23510.1.5 The CHANGELIMIT option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23810.1.6 The CHECKPAGE option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24010.1.7 Full or incremental copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24110.1.8 SHRLEVEL REFERENCE or SHRLEVEL CHANGE . . . . . . . . . . . . . . . . . . . . 24210.1.9 Tape or disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

10.2 COPYTOCOPY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24310.2.1 COPYTOCOPY options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24410.2.2 Performance measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

10.3 Concurrent COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24610.3.1 Concurrent COPY SHRLEVEL REFERENCE using FILTERDDN . . . . . . . . . . 249

Chapter 11. Gathering statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25311.1 Why collect statistics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25411.2 When to run RUNSTATS? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25411.3 RUNSTATS options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

11.3.1 LISTDEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25411.3.2 UPDATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25511.3.3 HISTORY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25611.3.4 KEYCARD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25711.3.5 FREQVAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25811.3.6 SAMPLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25911.3.7 FORCEROLLUP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Contents vii

11.3.8 REPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26211.4 RUNSTATS and LOB table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26211.5 New statistics collected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26511.6 Removing HISTORY statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26711.7 RUNSTATS performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

Part 4. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

Appendix A. Tables for Real-Time Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275Sample DDL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275TABLESPACESTATS column definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277INDEXSPACESTATS column definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279Impact of utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Appendix B. Sample JCL for copying Catalog and Directory objects . . . . . . . . . . . . 287

Appendix C. Creating a shadow catalog. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289DDL to define objects in shadow data base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289Unload job to unload DSNDB06 with UNLOAD utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290Load data into shadow catalog using LOAD utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292RUNSTATS on shadow data base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

Appendix D. UNICODE support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Generate the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Activate the UNICODE table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

Appendix E. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

System requirements for downloading the Web material . . . . . . . . . . . . . . . . . . . . . . . 297How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

Special notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

viii DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Figures

1-1 Packaging of DB2 Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132-1 LISTDEF scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212-2 LISTDEF expansion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222-3 Example disposition with Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272-4 Automatic data set sizing for the REORG utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292-5 TEMPLATE and LISTDEF combined (V6 shown for comparison). . . . . . . . . . . . . . . 312-6 Utility/LISTDEF/Template cross reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312-7 OPTIONS functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352-8 Utility support for PREVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363-1 Recovery scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443-2 Copy a list of DB2 objects in parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493-3 Recovering a list of DB2 objects in parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503-4 SORTKEYS with DB2 V5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523-5 SORTKEYS with DB2 V6, V7, and Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . 533-6 Rebuilding a partitioned index with PIB using SORTKEYS . . . . . . . . . . . . . . . . . . . . 583-7 Rebuilding a non-partitioned index with PIB using SORTKEYS . . . . . . . . . . . . . . . . 593-8 Rebuilding all indexes of a partitioned table space with PIB using SORTKEYS . . . . 603-9 Rebuilding all indexes of a non-partitioned table space with PIB using SORTKEYS 613-10 LOAD syntax for activating parallel partition loading . . . . . . . . . . . . . . . . . . . . . . . . . 633-11 Partition parallel LOAD without PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643-12 Partition parallel LOAD with PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653-13 Unloading a partitioned table space in parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674-1 LEAFDIST distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704-2 Optimizing data clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714-3 Remove indirect row reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724-4 Fields updated with space information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764-5 RUNSTATS to collect space statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774-6 Logical and physical view before reorganization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784-7 Logical and physical view after reorganization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794-8 Fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834-9 Non-fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834-10 RTS overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854-11 Collecting RTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874-12 DSNACCOR overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894-13 DB2 UDB Control Center - select table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904-14 REORG basic options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 914-15 REORG SHRLEVEL CHANGE options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924-16 REORG data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 934-17 DB2 administration menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944-18 DB2 performance queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954-19 Indexes with large leaf page distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954-20 DB2 Automation Tool main panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964-21 Utility profile display panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974-22 New utilities profile data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974-23 Utilities profile options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 984-24 Image copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 984-25 RUNSTATS options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994-26 REORG table space options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

© Copyright IBM Corp. 2001 ix

4-27 REORG index options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1005-1 Altering partitioning index key values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036-1 LOAD with Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1116-2 LOAD partition parallelism without Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . 1196-3 LOAD partition parallelism with Parallel Index Build . . . . . . . . . . . . . . . . . . . . . . . . 1236-4 DB2 family Cross Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296-5 Online LOAD Resume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1497-1 Phases of Online REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1677-2 Common REORG phases using SYSREC data set . . . . . . . . . . . . . . . . . . . . . . . . 1697-3 LOG phase of Online REORG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1707-4 FASTSWITCH processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1727-5 FASTSWITCH termination and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1747-6 BUILD2 phase parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767-7 When to rebuild a dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1838-1 Enhanced functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1948-2 UNLOAD — Syntax diagram, main part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1958-3 Summary of Unloading from copy data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1988-4 Unload from table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1999-1 RECOVER with PARALLEL keyword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2129-2 DB2 installation panel DSNTIPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21710-1 Parallel COPY with 3 subtasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23610-2 CHECKPAGE option of COPY utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24010-3 COPYTOCOPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24311-1 Monitoring space growth with RUNSTATS HISTORY. . . . . . . . . . . . . . . . . . . . . . . 25611-2 Using PAGESAVE to determine dictionary effectiveness . . . . . . . . . . . . . . . . . . . . 25711-3 LOB management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26411-4 SPACE Statistics tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26611-5 MODIFY STATISTICS Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

x DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Tables

1-1 Major changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102-1 Object results table by keyword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192-2 Pattern-matching character comparison table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202-3 TEMPLATE variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282-4 Data dispositions for dynamically allocated data sets . . . . . . . . . . . . . . . . . . . . . . . . 332-5 Data dispositions for dynamically allocated data sets on RESTART. . . . . . . . . . . . . 343-1 Inline COPY in LOAD and REORG TABLESPACE with two indexes . . . . . . . . . . . . 423-2 LOAD utility with inline statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473-3 REORG utility with inline statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483-4 COPY TABLESPACE without and with PARALLEL . . . . . . . . . . . . . . . . . . . . . . . . . 513-5 RECOVER TABLESPACE without and with PARALLEL. . . . . . . . . . . . . . . . . . . . . . 513-6 LOAD utility with 2 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543-7 LOAD utility with 3 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553-8 LOAD utility with 6 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553-9 REORG SHRLEVEL NONE utility with 2 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 553-10 REORG SHRLEVEL NONE utility with 3 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 553-11 REORG SHRLEVEL NONE utility with 6 indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 553-12 REORG 3 partitions with 1 NPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563-13 REORG 3 partitions with 5 NPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563-14 REBUILD of a partitioned index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613-15 REBUILD of a non-partitioned index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623-16 REBUILD INDEX(ALL). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623-17 LOAD partition parallelism without PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653-18 LOAD partition parallelism with PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664-1 COPY CHANGELIMIT with single value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734-2 COPY CHANGELIMIT with double values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734-3 Trigger limits and utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744-4 New tables in SYSHIST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764-5 Sample output from the SQL query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805-1 Number of partition and partitioned table space sizes . . . . . . . . . . . . . . . . . . . . . . . 1026-1 Online LOAD Resume compared with SQL insert application. . . . . . . . . . . . . . . . . 1517-1 DB2 enhancements to the REORG utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1617-2 Online REORG phase unlimited READ / WRITE. . . . . . . . . . . . . . . . . . . . . . . . . . . 1658-1 UNLOAD utility phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1928-2 Performance measurements and comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2079-1 Total number of processes used for RECOVER in parallel . . . . . . . . . . . . . . . . . . . 2119-2 DB2 V7 catalog table entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2159-3 Fast apply buffer sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2189-4 Modify recovery enhancement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2229-5 Phases of CHECK DATA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2289-6 Phases of CHECK LOB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22910-1 Performance of COPYTOCOPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24611-1 Allowable HISTORY/UPDATE combinations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25611-2 Sampling performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26111-3 Statistic tables updated by HISTORY option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26611-4 Deleting statistics gathered by HISTORY SPACE. . . . . . . . . . . . . . . . . . . . . . . . . . 26911-5 Deleting statistics gathered by HISTORY ACCESSPATH. . . . . . . . . . . . . . . . . . . . 27011-6 Deleting statistics gathered by HISTORY ALL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

© Copyright IBM Corp. 2001 xi

11-7 RUNSTATS performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271A-1 Columns definition for TABLESPACESTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277A-2 Columns definition for INDEXSPACESTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

xii DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Examples

2-1 Version 6 and before . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192-2 Version 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192-3 Excluding objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192-4 Sample LISTDEF - Recovery job. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212-5 Sample output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242-6 Template not invoking data set sizing . . . . . . . . . . . . . . . . . . . . . . . . . . 262-7 Template invoking data set sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262-8 Template using tape with STACK option . . . . . . . . . . . . . . . . . . . . . . . . 262-9 Using multiple Templates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262-10 Using LISTDEF and Templates together . . . . . . . . . . . . . . . . . . . . . . . . 323-1 Initiating multiple Inline COPY within LOAD utility . . . . . . . . . . . . . . . . . 403-2 Initiating Inline COPY when loading at partition level . . . . . . . . . . . . . . 413-3 SYSCOPY entry for Inline COPY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423-4 LOAD REPLACE LOG NO with Inline COPY and discards. . . . . . . . . . 443-5 REPORT RECOVERY showing logging during discard processing . . . 453-6 Using inline copies with DSN1COPY. . . . . . . . . . . . . . . . . . . . . . . . . . . 463-7 Inline COPY with DSN1PRNT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463-8 DISPLAY UTILITY using Inline COPY and statistics . . . . . . . . . . . . . . . 484-1 Stored procedure DSNUTILS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 946-1 LOAD of a non-partitioned table space with PIB . . . . . . . . . . . . . . . . . 1126-2 LOAD of non-partitioned table space with PIB; discard processing. . . 1136-3 Job output LOAD of a non-partitioned table space with PIB . . . . . . . . 1156-4 LOAD of a partitioned table space with PIB. . . . . . . . . . . . . . . . . . . . . 1166-5 Job output LOAD of partitioned table space with PIB; no parallelism . 1176-6 Partition parallelism without PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1196-7 Partition parallelism without PIB using a template for input data sets . 1206-8 Job output partition parallelism without PIB . . . . . . . . . . . . . . . . . . . . . 1216-9 Partition parallelism with PIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246-10 Job output partition parallelism with PIB . . . . . . . . . . . . . . . . . . . . . . . 1256-11 Partition parallelism with inline copies on the partition level . . . . . . . . 1266-12 Preformatting a table space and its index spaces during LOAD . . . . . 1286-13 Specifying the REUSE option during LOAD REPLACE. . . . . . . . . . . . 1286-14 Simple Cross Loader example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1306-15 List of dynamic SQL statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1316-16 Create a new table with the same layout as SYSIBM.SYSTABLES . . 1316-17 Comment and grant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1316-18 Declare a cursor for the Cross Loader. . . . . . . . . . . . . . . . . . . . . . . . . 1326-19 Usage of a cursor in the LOAD statement . . . . . . . . . . . . . . . . . . . . . . 132

© Copyright IBM Corp. 2001 xiii

6-20 Restartability of the EXEC SQL statement . . . . . . . . . . . . . . . . . . . . . 1336-21 JCL for testing the thread behavior of EXEC SQL. . . . . . . . . . . . . . . . 1336-22 Testing the thread behavior of EXEC SQL . . . . . . . . . . . . . . . . . . . . . 1346-23 JCL for verifying the commit scope of EXEC SQL . . . . . . . . . . . . . . . 1356-24 Verifying the commit scope of EXEC SQL. . . . . . . . . . . . . . . . . . . . . . 1356-25 Cross Loader example with matching columns . . . . . . . . . . . . . . . . . . 1396-26 Cross Loader example with AS clause and default columns. . . . . . . . 1406-27 Cross Loader example with IGNOREFIELDS and WORKDDN. . . . . . 1406-28 Populating a table containing a UDT column with the Cross Loader. . 1416-29 Cross loading from a table containing UDTs . . . . . . . . . . . . . . . . . . . . 1426-30 Cross loading example with a LOB column . . . . . . . . . . . . . . . . . . . . . 1436-31 Cross Loader example using DRDA . . . . . . . . . . . . . . . . . . . . . . . . . . 1446-32 Cross Loader example with aggregation . . . . . . . . . . . . . . . . . . . . . . . 1456-33 Loading a partitioned table space with the Cross Loader . . . . . . . . . . 1466-34 Using partition parallelism with the Cross Loader . . . . . . . . . . . . . . . . 1476-35 Creating an input load data set with the UNLOAD utility . . . . . . . . . . . 1526-36 Online LOAD Resume of a non-partitioned table space . . . . . . . . . . . 1526-37 Online LOAD Resume job output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1536-38 Online LOAD Resume duplicating the input data set. . . . . . . . . . . . . . 1546-39 Online LOAD Resume job output duplicate records in the input . . . . . 1556-40 LOAD SHRLEVEL NONE job output duplicate records in the input . . 1556-41 Using a DISCARD data set with Online LOAD Resume . . . . . . . . . . . 1566-42 Unloading a partitioned table space using parallelism. . . . . . . . . . . . . 1576-43 Job output from the parallel unload . . . . . . . . . . . . . . . . . . . . . . . . . . . 1576-44 Online LOAD Resume of a partitioned table space with parallelism . . 1586-45 Online LOAD Resume job output with parallelism. . . . . . . . . . . . . . . . 1597-1 DRAIN_WAIT parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1777-2 REORG output using DRAIN_WAIT parameter. . . . . . . . . . . . . . . . . . 1777-3 Online REORG with DRAIN_WAIT, RETRY and RETRY_DELAY . . . 1787-4 Online REORG DRAIN_WAIT, RETRY and RETRY_DELAY output . 1797-5 Changing DFSORT defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837-6 Specifying the REUSE option during REORG. . . . . . . . . . . . . . . . . . . 1847-7 Preformatting a table space and its index spaces during REORG . . . 1857-8 REORG using DISCARD processing. . . . . . . . . . . . . . . . . . . . . . . . . . 1867-9 REORG using DISCARD job output . . . . . . . . . . . . . . . . . . . . . . . . . . 1867-10 REORG UNLOAD EXTERNAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1877-11 REORG UNLOAD EXTERNAL job output . . . . . . . . . . . . . . . . . . . . . . 1887-12 Collecting inline statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1897-13 -DISPLAY UTILITY with active utility . . . . . . . . . . . . . . . . . . . . . . . . . . 1898-1 UNLOAD from an image copy data set . . . . . . . . . . . . . . . . . . . . . . . . 1968-2 Contents of SYSPUNCH data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1968-3 UNLOAD using FROMCOPY and FROM TABLE options . . . . . . . . . . 1978-4 Unload list of table spaces with LISTDEF . . . . . . . . . . . . . . . . . . . . . . 2008-5 Sample UNLOAD job for partition table space and parallelism . . . . . . 2008-6 Sample UNLOAD output by partition and parallelism . . . . . . . . . . . . . 2018-7 Unload selective tables from SYSDBASE using FROM TABLE . . . . . 2038-8 Sample database, table space, table, and subset of DSNHDECP . . . 2048-9 UNLOAD with character conversion to ASCII . . . . . . . . . . . . . . . . . . . 2058-10 UNLOAD with character conversion to UNICODE. . . . . . . . . . . . . . . . 2059-1 RECOVER using LISTDEF command with Wildcard. . . . . . . . . . . . . . 2109-2 LISTDEF OPTIONS PREVIEW job output. . . . . . . . . . . . . . . . . . . . . . 2109-3 Partial job output with seven objects in parallel . . . . . . . . . . . . . . . . . . 2119-4 RECOVER with object-list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

xiv DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

9-5 COPY table space and all indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 2199-6 RECOVER of the table space and indexes in parallel . . . . . . . . . . . . . 2209-7 RECOVER table space and indexes output . . . . . . . . . . . . . . . . . . . . 2209-8 REPORT RECOVERY sample job . . . . . . . . . . . . . . . . . . . . . . . . . . . 2229-9 REPORT RECOVERY sample output . . . . . . . . . . . . . . . . . . . . . . . . . 2239-10 Sample JCL to create new full image copy . . . . . . . . . . . . . . . . . . . . . 2249-11 QUIESCE with a list of table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 2259-12 QUIESCE using the LISTDEF command. . . . . . . . . . . . . . . . . . . . . . . 2259-13 QUIESCE of table space and all indexes . . . . . . . . . . . . . . . . . . . . . . 2269-14 Sample CHECK INDEX JCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2279-15 CHECK INDEX job output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2279-16 Sample CHECK DATA JCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2299-17 Sample CHECK LOB JCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2309-18 CHECK LOB job output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23010-1 Specifying lists in the COPY utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23510-2 Parallel COPY using LISTDEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23610-3 Output from Parallel COPY using LISTDEF . . . . . . . . . . . . . . . . . . . . 23710-4 -DISPLAY UTILITY for Parallel COPY . . . . . . . . . . . . . . . . . . . . . . . . 23810-5 Specifying CHANGELIMIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23910-6 Report output from COPY utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23910-7 COPYTOCOPY and LISTDEFs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24410-8 COPYTOCOPY using LASTFULLCOPY. . . . . . . . . . . . . . . . . . . . . . . 24510-9 Output job for Concurrent COPY with error . . . . . . . . . . . . . . . . . . . . . 24710-10 Output from incremental copy after a Concurrent COPY. . . . . . . . . . . 24810-11 Example of RECOVER using a Concurrent COPY . . . . . . . . . . . . . . . 24810-12 Copy statement without FILTERDDN . . . . . . . . . . . . . . . . . . . . . . . . . 25010-13 Copy statement using FILTERDDN. . . . . . . . . . . . . . . . . . . . . . . . . . . 25010-14 Complete JCL using FILTERDDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25010-15 Job output using FILTERDDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25111-1 Example of LISTDEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25511-2 RUNSTATS with LISTDEF job output . . . . . . . . . . . . . . . . . . . . . . . . . 25511-3 RUNSTATS using KEYCARD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25711-4 RUNSTATS KEYCARD job output . . . . . . . . . . . . . . . . . . . . . . . . . . . 25711-5 RUNSTATS using FREQVAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25811-6 RUNSTATS using FREQVAL sample output. . . . . . . . . . . . . . . . . . . . 25811-7 Multiple FREQVAL statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25811-8 RUNSTATS SAMPLE 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25911-9 RUNSTATS SAMPLE 25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26011-10 RUNSTATS SAMPLE 100. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26011-11 RUNSTATS REPORT sample output . . . . . . . . . . . . . . . . . . . . . . . . . 26211-12 RUNSTATS with LISTDEF for LOB table spaces . . . . . . . . . . . . . . . . 26311-13 RUNSTATS for LOB table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26311-14 RUNSTATS output for LOB table spaces . . . . . . . . . . . . . . . . . . . . . . 26311-15 New statistics SPACE and EXTENTS . . . . . . . . . . . . . . . . . . . . . . . . . 26711-16 Data set listing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26711-17 MODIFY STATISTICS by date using SPACE . . . . . . . . . . . . . . . . . . . 26911-18 Output job for table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269B-1 Copying the Catalog and Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . 287C-1 DDL to define a shadow catalog in data base DSHADOW . . . . . . . . . 289C-2 Unload DSNDB06 using the UNLOAD utility . . . . . . . . . . . . . . . . . . . . 290C-3 Sample output of SYSPUNCH data set for SYSVIEWS . . . . . . . . . . . 291C-4 Load data from DSNDB06 using the LOAD utility . . . . . . . . . . . . . . . . 292C-5 RUNSTATS on shadow catalog. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

Examples xv

D-1 JCL to create the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . 293D-2 Output from the generation job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294D-3 Contents of CUNUNI01 in SYS1.PARMLIB. . . . . . . . . . . . . . . . . . . . . 295D-4 Activate the new UNICODE table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295D-5 UNI=ALL output in SYSLOG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

xvi DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Preface

IBM has continuously enhanced the functionality, performance, availability, and ease of use of DB2 Utilities. Starting with DB2 for MVS/ESA Version 4, the progression of changes has increased. DB2 for z/OS and OS/390 Version 7 introduces new functions and a new way of packaging the utilities.

This IBM Redbook is the result of a project dedicated to the new DB2 Version 7 Utilities Suite product. It provides information on introducing Wildcarding in operational scenarios, optimizing concurrent execution of utilities for a shorter night window, information for triggering utilities execution, and considerations on partitioning.

It also describes the new functions of LOAD (Cross Loader and Online RESUME) and REORG, as well as the new UNLOAD and COPTYTOCOPY utilities, and it evaluates the impact of several options when using LOAD, REORG, RECOVER, COPY, and RUNSTATS.

This redbook concentrates on the recent enhancements. It implicitly assumes a basic level of familiarity with the utilities as provided by DB2 for OS/390 Version 6.

The team that wrote this redbookThis redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center.

Paolo Bruni is a Certified Consultant IT Architect on assignment as a Data Management Specialist at the International Technical Support Organization, San Jose Center, where he conducts projects on all areas of DB2 for z/OS and OS/390. Before joining the ITSO in 1998, Paolo worked in IBM Italy. During his 32 years with IBM, in development and in the field, Paolo’s work has been mostly related to database systems.

Marcelo Antonelli is a DB2 Specialist in Brazil. He has 16 years of experience working with DB2 in OS/390. Marcelo holds a graduate degree in System Analysis from PUCC in Campinas, Sao Paulo state. His areas of expertise include database design, performance, and DRDA connectivity. Marcelo currently supports an outsourcing commercial account, as well as assisting IBM technical professionals on DB2.

Tom Crocker is an IT Specialist in the UK. He has 14 years of experience in data management, and has worked with DB2 since Version 2. He has been employed with IBM for 2 years, working with SAP on OS/390. Before joining IBM, he worked as an independent consultant. Tom currently supports IBM internal SAP systems and has experience in insurance, banking, and retail.

Davy Goethals is a systems engineer in Belgium. He works for Sidmar, a steel producing company. He has experience as a DB2 system administrator since DB2 Version1 in 1985. He participated with Sidmar in multiple DB2 ESP and QPP programs, including the ESP for DB2 V7. His areas of expertise include OS/390 systems and Business Intelligence.

© Copyright IBM Corp. 2001 xvii

Rama Naidoo is a DB2 Specialist in Australia. He has 28 years of experience in data management and has worked on several non-IBM database management systems before joining IBM in 1989. Rama holds a postgraduate degree in information technology from Monash University in Melbourne, Australia. His areas of expertise include data modeling, scientific numeric modeling, and project management. Rama currently supports DB2 at a commercial account with one of the largest known databases.

Bart Steegmans is a DB2 Product Support Specialist in IBM Belgium who has recently joined the ITSO. He has over 12 years of experience in DB2. Before joining IBM in 1997, Bart worked as a DB2 system administrator at a banking and insurance group. His areas of expertise include DB2 performance and backup and recovery.

Thanks to the following people for their contributions to this project:

Corinne BaragoinYolanda Cascajo JimenezYvonne LyonInternational Technical Support Organization, San Jose Center

Roy CornfordDan CourterCraig FriskeJohn GarthKoshy JohnLaura Kunioka-WeisFung LeeRoger MillerMai NguyenMary PetrasJim RuddyAkira ShibamiyaYoichi TsujiGrace TzengTom Ulveman JensenMary Beth WhiteIBM Silicon Valley Laboratory

Rich ConwayVasilis KarrasInternational Technical Support Organization, Poughkeepsie Center

Michael ParbsIBM Australia

xviii DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Special noticeThis publication is intended to help managers and professionals understand and evaluate the performance and applicability to their environment of the new functions introduced by IBM DATABASE2 Universal Database Server for z/OS and OS/390 Version 7 and DB2 Version 7 Utilities Suite. The information in this publication is not intended as the specification of any programming interfaces that are provided by IBM DB2 UDB Server for z/OS and OS/390 Version 7 and the DB2 Version 7 Utilities Suite. See the PUBLICATIONS section of the IBM Programming Announcement for IBM DB2 UDB Server for z/OS and OS/390 Version 7 and DB2 Version 7 Utilities Suite for more information about what publications are considered to be product documentation.

IBM trademarksThe following terms are trademarks of the International Business Machines Corporation in the United States and/or other countries:

Comments welcomeYour comments are important to us!

We want our IBM Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways:

� Use the online Contact us review redbook form found at:

ibm.com/redbooks

� Send your comments in an Internet note to:

[email protected]

� Mail your comments to the address on page ii.

e (logo)® IBM ®S/390DRDAMVS/ESA

RedbooksRedbooks Logo OS/390DB2

Preface xix



http://www.redbooks.ibm.com/contacts.html

xx DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Part 1 Introduction

In this part we introduce the DB2 Utilities, summarize their evolution, and describe the contents of this redbook.

Part 1

© Copyright IBM Corp. 2001 1

2 DB2 for z/OS and OS/390 Version 7: Using the Utilities Suite

Chapter 1. Introduction

In this chapter we provide:

� A description of the contents of this redbook� A brief overview of all DB2 Utilities, both online and stand-alone� A summary of the DB2 Utilities enhancements with V4, V5, V6, and V7� The main enhancements available with V7� The changes in their packaging that occurred with DB2 V7

1


1.1 Contents of this redbookIn this part we briefly introduce all the DB2 Utilities, summarize their evolution, and highlight the recent changes occurred with DB2 V7.

Part 2, “Planning for DB2 Utilities” on page 15 includes overall considerations on functions related to how to plan for, and when to execute the DB2 Utilities. These considerations are distributed across four chapters describing the topics:

� Wildcarding and Templates � Inline executions and parallelism � Triggering and controlling executions � Managing partitions

Part 3, “Executing DB2 Utilities” on page 107 goes into the execution of the new DB2 Utilities and the new functions by describing the functionality and providing examples of usage for:

� Loading data� Reorganizing data� Unloading data � Recovering data� Copying data � Gathering statistics

Part 4, “Appendixes” on page 273 includes sample documentation, as follows:

� Details on the tables for the Real Time Statistics� Sample job stream for copying DB2 catalog and directory using LISTDEF and Template� Sample jobs that can be used to define a shadow catalog� Procedures to install and activate the 1140 (Euro English) to UNICODE 367 conversion for

usage with the UNLOAD utility� Description of how to download a job containing DDL to create and populate a shadow

DB2 catalog

1.2 Brief overview of all DB2 UtilitiesThere are two types of DB2 Utilities: online utilities and stand-alone utilities. In the next sections we briefly describe the functions of each utility in both groups.

1.2.1 Online utilitiesDB2 online utilities run as standard MVS batch jobs or stored procedures, and they require DB2 to be running. They invoke DB2 control facility services directly.

� CATMAINT

The CATMAINT online utility updates the catalog; it should be run during migration to a new release of DB2, or when instructed to do so by the IBM service.

� CHECK DATA

The CHECK DATA online utility checks table spaces for violations of referential and table check constraints, and reports information about violations that are detected. You run CHECK DATA after a conditional restart or a point-in-time recovery on all table spaces where parent and dependent tables might not be synchronized. CHECK DATA can be run against a base table space only, not a LOB table space.


� CHECK INDEX

The CHECK INDEX online utility tests whether indexes are consistent with the data they index, and issues warning messages when an inconsistency is found. You execute CHECK INDEX after a conditional restart or a point-in-time recovery on all table spaces whose indexes may not be consistent with the data. It should also be used before CHECK DATA to ensure that the indexes used by CHECK DATA are valid.

� CHECK LOB

The CHECK LOB online utility can be run against a LOB table space to identify any structural defects in the LOB table space and any invalid LOB values. You run the CHECK LOB online utility against a LOB table space that is marked CHECK pending (CHKP) to identify structural defects. If none are found, the CHECK LOB utility turns the CHKP status off.

� COPY

The COPY online utility creates up to four image copies of any table space, table space partition, data set of a linear table space, index space, index space partition.

� COPYTOCOPY

The COPYTOCOPY online utility, new with DB2 V7, makes image copies from an image copy that was taken by the COPY utility. This includes Inline Copies made by the REORG or LOAD utilities. Starting with either the local primary or recovery site primary copy, COPYTOCOPY can make up to three copies of one or more of the local primary, local backup, recovery site primary, and recovery site backup copies.

The copies are used by the RECOVER utility when recovering a table space or index space to the most recent time or to a previous time. These copies can also be used by MERGECOPY, UNLOAD, and possibly a subsequent COPYTOCOPY execution.

� DIAGNOSE

The DIAGNOSE online utility generates information useful in diagnosing problems. It is intended to be used only under the direction of your IBM Support Center.

� LOAD

The LOAD online utility is used to load one or more tables of a table space. The LOAD utility loads records into the tables and builds or extends any indexes defined on them. If the table space already contains data, you can choose whether you want to add the new data to the existing data or replace the existing data. The loaded data is processed by any edit or validation routine associated with the table, and any field procedure associated with any column of the table.

� MERGECOPY

The MERGECOPY online utility merges image copies produced by the COPY utility or Inline Copies produced by the LOAD or REORG utilities. It can merge several incremental copies of a table space to make one incremental copy. It can also merge incremental copies with a full image copy to make a new full image copy.

� MODIFY RECOVERY

The MODIFY online utility with the RECOVERY option deletes records from the SYSIBM.SYSCOPY catalog table, related log records from the SYSIBM.SYSLGRNX directory table, and entries from the DBD. You can remove records that were written before a specific date, or you can remove records of a specific age. You can delete records for an entire table space, partition, or data set.

Chapter 1. Introduction 5

� MODIFY STATISTICS

The MODIFY STATISTICS online utility, new with DB2 V7, deletes unwanted statistics history records from the corresponding catalog tables. You can remove statistics history records that were written before a specific date, or you can remove records of a specific age. You can delete records for an entire table space, index space, or index.

� QUIESCE

The QUIESCE online utility establishes a quiesce point (the current log RBA or log record sequence number (LRSN)) for a table space, partition, table space set, or list of table spaces and table space sets, and records it in the SYSIBM.SYSCOPY catalog table. An established QUIESCE point improves the probability of a successful RECOVER or COPY. You should run QUIESCE frequently between regular executions of COPY to establish regular recovery points for future point-in-time recovery.

� REBUILD INDEX

The REBUILD INDEX online utility reconstructs indexes from the table that they reference.

� RECOVER

The RECOVER online utility recovers data to the current state, or to its state at a previous point-in-time, by restoring a copy, then applying log records.

� REORG INDEX

The REORG INDEX online utility reorganizes an index space to improve access performance and reclaim fragmented space. You can specify the degree of access to your data during reorganization, and collect inline statistics using the STATISTICS keyword.

� REORG TABLESPACE

The REORG TABLESPACE online utility reorganizes a table space to improve access performance and reclaim fragmented space. In addition, the utility can reorganize a single partition or range of partitions of a partitioned table space. You can specify the degree of access to your data during reorganization, and collect inline statistics using the STATISTICS keyword. If you specify REORG TABLESPACE UNLOAD EXTERNAL, the data is unloaded in a format that is acceptable to the LOAD utility of any DB2 subsystem. You can also delete rows during the REORG by specifying the DISCARD option.

� REPAIR

The REPAIR online utility repairs data. The data can be your own data, or data you would not normally access, such as space map pages and index entries.

� REPORT

The REPORT online utility provides information about table spaces. Use REPORT TABLESPACESET to find the names of all the table spaces and tables in a referential structure, including LOB table spaces. Use REPORT RECOVERY to find information necessary for recovering a table space, index, or a table space and all of its indexes. The REPORT utility also provides the LOB table spaces associated with a base table space.

� RUNSTATS

The RUNSTATS online utility gathers summary information about the characteristics of data in table spaces, indexes, and partitions. DB2 records this information in the DB2 catalog and uses it to select access paths to data during the bind process. It is available to the database administrator for evaluating database design and to aid in determining when table spaces or indexes must be reorganized.

� STOSPACE

The STOSPACE online utility updates DB2 catalog columns that indicate how much space is allocated for storage groups and related table spaces and indexes.


� UNLOAD

The UNLOAD online utility unloads data from one or more source objects to one or more BSAM sequential data sets in external formats. The source can be DB2 table spaces or DB2 image copy data sets.

� The following new utility statements are to be used in conjunction with other utilities:

– TEMPLATE

The TEMPLATE utility control statement lets you allocate data sets, without using JCL DD cards, during the processing of LISTDEF list. In its simplest form, the TEMPLATE control statement defines the data set naming convention. The control statement can optionally contain allocation parameters that define data set size, location, and attributes.

– LISTDEF

The LISTDEF utility control statement defines a list of objects and assigns a name to the list. A complete LISTDEF control statement must include:

• The name of the list• An INCLUDE clause, optionally followed by additional INCLUDE or EXCLUDE

clauses to either include or exclude objects from the list• The type of objects the list contains, either TABLESPACES or INDEXSPACES• The object to be used in the initial catalog lookup

The search for objects can begin with databases, table spaces, index spaces, tables, indexes or other lists. The resulting list will only contain TABLESPACES, INDEXSPACES, or both.

– OPTIONS

The OPTIONS utility control statement specifies processing options that are applicable across many utility executions in a job step. By specifying various options, you can:

• Preview utility control statements• Override library names for LISTDEF lists or TEMPLATEs• Specify how to handle errors during list processing• Alter the return code for warning messages• Restore all default options

– EXEC SQL

The EXEC SQL utility control statement declares cursors or executes dynamic SQL statements.

1.2.2 Stand-alone utilitiesThe stand-alone utilities execute as batch jobs independent of DB2. They can be executed only by means of MVS JCL.

� DSNJLOGF

When writing to an active log data set for the first time, DB2 must preformat a VSAM control area before writing the log records. The DSNJLOGF utility avoids this delay by preformatting the active log data sets before bringing them online to DB2.

� DSNJU003

The DSNJU003 stand-alone utility changes the bootstrap data sets (BSDSs). You can use the utility to:

– Add or delete active or archive log data sets– Add or delete checkpoint records


– Create conditional restart control record to control the next start of the DB2 subsystem– Change the VSAM catalog name entry in the BSDS– Modify the communication record in the BSDS– Modify the value for the highest-written log RBA value (relative byte address within the

log) or the highest-offloaded RBA value

� DSNJU004

The Print Log Map (DSNJU004)utility lists the following information:

– Log data set name, log RBA association, and log LRSN for both primary and secondary copy of all active and archive log data sets

– Active log data sets that are available for new log data– Status of all conditional restart control records in the bootstrap data set– Contents of the queue of checkpoint records in the bootstrap data set– The communication record of the BSDS, if one exists– Contents of the quiesce history record– System and utility timestamps– Contents of the checkpoint queue

� DSN1CHKR

The DSN1CHKR utility verifies the integrity of DB2 directory and catalog table spaces. DSN1CHKR scans the specified table space for broken links, broken hash chains, and records that are not part of any link or chain. Use DSN1CHKR on a regular basis to promptly detect any damage to the catalog and directory.

� DSN1COMP

The DSN1COMP utility estimates space savings to be achieved by DB2 data compression in table spaces. This utility can be run on the following types of data sets containing uncompressed data:

– DB2 full image copy data sets– VSAM data sets that contain DB2 table spaces– Sequential data sets that contain DB2 table spaces (for example,DSN1COPY output)

DSN1COMP does not estimate savings for data sets that contain LOB table spaces or index spaces.

� DSN1COPY

With the DSN1COPY stand-alone utility, you can copy:

– DB2 VSAM data sets to sequential data sets– DSN1COPY sequential data sets to DB2 VSAM data sets– DB2 image copy data sets to DB2 VSAM data sets– DB2 VSAM data sets to other DB2 VSAM data sets

You can copy DSN1COPY sequential data sets to other sequential data sets. Using DSN1COPY, you can also print hexadecimal dumps of DB2 data sets and databases, check the validity of data or index pages (including dictionary pages for compressed data), translate database object identifiers (OBIDs) to enable moving data sets between different systems, and reset to 0 the log RBA that is recorded in each index page or data page.

DSN1COPY is compatible with LOB table spaces, when you specify the LOB keyword, and omit the SEGMENT and INLCOPY keywords.

� DSN1LOGP

The DSN1LOGP utility formats the contents of the recovery log for display.The two recovery log report formats are:


– A detail report of individual log records. This information helps IBM Support Center personnel analyze the log in detail. (This book does not include a full description of the detail report.)

– A summary report helps you:

• Perform a conditional restart• Resolve indoubt threads with a remote site• Detect problems with data propagation

You can specify the range of the log to process and select criteria within the range to limit the records in the detail report. For example, you can specify:

– One or more units of recovery identified by URID– A single database

By specifying a URID and a database, you can display recovery log records that correspond to the use of one database by a single unit of recovery.

� DSN1PRNT

With the DSN1PRNT stand-alone utility, you can print:

– DB2 VSAM data sets that contain table spaces or index spaces (including dictionary pages for compressed data)

– Image copy data sets– Sequential data sets that contain DB2 table spaces or index spaces

Using DSN1PRNT, you can print hexadecimal dumps of DB2 data sets and databases. If you specify the FORMAT option, DSN1PRNT formats the data and indexes for any page that does not contain an error that would prevent formatting. If DSN1PRNT detects such an error, it prints an error message just before the page and dumps the page without formatting. Formatting resumes with the next page. Compressed records are printed in compressed format. DSN1PRNT is especially useful when you want to identify the contents of a table space or index. You can run DSN1PRNT on image copy data sets as well as table spaces and indexes. DSN1PRNT accepts an index image copy as input when you specify the FULLCOPY option.

DSN1PRNT is compatible with LOB table spaces, when you specify the LOB keyword, and omit the INLCOPY keyword.

� DSN1SDMP

Under the direction of the IBM Support Center, you can use the IFC Selective Dump (DSN1SDMP) utility to:

– Force dumps when selected DB2 trace events occur.– Write DB2 trace records to a user-defined MVS data set.

1.3 Utilities at workIBM has always enhanced the functionality, performance, availability, and ease of use of the initial set of utilities. Starting with DB2 for MVS/ESA Version 4, the progression of changes has increased. Table 1-1 summarizes the changes that occurred in V4, V5, V6, and V7 of DB2. Details on functions and related performance by DB2 versions are reported in the redbooks DB2 for MVS/ESA Version 4 Non-Data-Sharing Performance Topics, SG24-4562, DB2 for OS/390 Version 5 Performance Topics, SG24-2213, DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351, and DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-6129.


Table 1-1 Major changes

Utility DB2 V4 DB2 V5 DB2 V6 DB2 V7

LOAD PI improvements.Path length reduction.

LOAD and Inline COPY (COPYDDN).Optional removal of work data sets for indexes (SORTKEYS). RELOAD phase performance.PREFORMAT option.

Collect inline statistics. Building indexes in parallel.SORTKEYS also used for parallel index build.

Family Cross Loader, Online Load Resume, Partition Parallelism within a job.

REORG NPI improvements.Catalog REORG.Path length reduction.SORTDATA

REORG and Inline COPY (COPYDDN).New SHRLEVEL NONE, REFERENCE, CHANGE option (Online REORG).Optional removal of work data sets for indexes (SORTKEYS). NOSYSRECPREFORMAT option.

Collect inline statistics.Building indexes in parallel.Discard and faster UNLOAD.Faster Online REORG.Threshold for execution.SORTKEYS also used for parallel index build.

Fast switch, BUILD2 phase parallelism, Drain and Retry.

RECOVER Usage of DFSMS CONCURRENT copies. RECOVER INDEX restartability. RECOVER INDEX SYSUT1 optional for performance choice.

Use inline copies from LOAD and REORG.RECOVER INDEX UNLOAD phase performance (like SORTDATA).

Fast Log Apply.RECOVER INDEX from copies (vs. REBUILD). Recovery of table space and indexes with single log scan.PARALLEL RECOVER.

COPY Design change to improve performance up to 10 times.CONCURRENT option for DFSMS.

Full copies inline with LOAD and REORG.CHANGELIM option to decide execution.

Copies of indexes.Parallelism.Check page.PARALLEL RECOVER.

RUNSTATS Modified hashing technique for CPU reduction.

New SAMPLE option to specify % of rows to use for non-indexed column statistics.New KEYCARD option for correlated key columns.New FREQVAL option for frequent value statistics with non-uniform distribution.

RUNSTATS executed inline with REORG, LOAD, and RECOVER or Rebuild index.Parallel table space and index.

Statistics History.

REBUILD N/A N/A (APAR) Inline RUNSTATS.SORTKEYS for indexes in parallel.

QUIESCE TABLESPACESET support.

TEMPLATE Dynamic allocation of data sets.


Other functions not included in the table, but worth mentioning, are the support for very large tables (DSSIZE) and the support for pieces for non-partitioning indexes.

1.4 Major changes with DB2 V7DB2 Version 7 introduces several new functions, as well as a different way of packaging the DB2 Utilities.

1.4.1 Functional enhancementsIn this section we briefly describe the enhancements introduced with DB2 Version 7. For a detailed description of the functions, please refer to the standard DB2 documentation or the redbook DB2 UDB Server for OS/390 and z/OS Version 7 Presentation Guide, SG24-6121. For performance information, you can refer to the recent redbook DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-5129.

Dynamic utility jobsDB2 V7 introduces two new utility control statements: TEMPLATE and LISTDEF. They provide for the dynamic allocation of data sets and for the dynamic processing of lists of DB2 objects with one utility invocation. These new statements can now be used by most DB2 Utilities. Using these new statements will dramatically change your utility jobs and reduce the cost of maintaining them. They are also a prerequisite for partition parallelism within the same job.

LOAD partition parallelismLOAD now provides enhanced availability when loading data and indexes on different partitions in parallel within the same job from multiple inputs.

Online LOAD RESUMEPrior to V7, DB2 restricts access to data during LOAD processing. V7 offers the choice of allowing user read and write access to the data during LOAD RESUME processing, so that loading data is concurrent with user transactions. This new feature of the LOAD utility is almost like a new utility of its own: it works internally more like a mass insert rather than a LOAD. The major advantage is availability with integrity.

LISTDEF Dynamic definition of lists.

UNLOAD To unload data from table spaces or copies.

COPYTOCOPY Additional recognized copies.

ALL Partition independence (NPI) from type 2 indexes.BSAM I/O buffers.Type 2 index performance.

BSAM striping for work data sets

Avoid delete and redefine of data sets (except for COPY).

New packaging, dynamic utility jobs.

Utility DB2 V4 DB2 V5 DB2 V6 DB2 V7


Cross LoaderA new extension to the point-in-time utility which allows you to load output from a SELECT statement. The input to the SELECT can be anywhere within the scope of DRDA connectivity.

Online REORG enhancementsOnline REORG no longer renames data sets, greatly reducing the time that data is unavailable during the SWITCH phase. You specify a new keyword, FASTSWITCH, which keeps the data set name unchanged and updates the catalog to reference the newly reorganized data set. Additional parallel processing improves the elapsed time of the NPIs BUILD2 phase of REORG SHRLEVEL(CHANGE) or SHRLEVEL(REFERENCE). New parameters allow more granularity in draining and waiting for resources to be available.

New utility — UNLOADThe UNLOAD utility unloads data from one or more source objects, table spaces, or image copies, to one or more BSAM sequential data sets in external formats. It is easier to use than the previously available REORG UNLOAD EXTERNAL, and it is faster than DSNTIAUL. UNLOAD also offers new capabilities when compared to the still-available REORG UNLOAD EXTERNAL.

New utility — COPYTOCOPYThe COPYTO COPY online utility provides the function of making additional copies, local and/or remote, and recording them in the DB2 Catalog.

RUNSTATS statistics historyChanges to RUNSTATS provide the possibility to keep multiple versions of the statistics across time and allow tools and users more flexibility in monitoring the trends in data changes.

New utility — MODIFY STATISTICSThe MODIFY STATISTICS online utility deletes unwanted RUNSTATS statistics history records from the corresponding catalog tables which are now created by a new RUNSTATS function (see “RUNSTATS statistics history” on page 12). You can remove statistics history records that were written before a specific date, or you can remove records of a specific age.

1.4.2 New packaging of utilities A basic set of core utilities are included as part of DB2 since Version 1 was first delivered.

These utilities initially provided a basic level of services to allow customers to manage data. Some customers have preferred to obtain such functions, however, from independent software vendors that have developed utility and tools offerings that offered additional performance, function, and features beyond that contained in the DB2 core utilities. With recent releases of DB2 for OS/390, in response to clear customer demand, IBM has invested in the improvement of the performance and functional characteristics of these utilities, as we have seen in the previous section.

With DB2 V7, most of the online utilities have been separated from the base product and they are now offered as separate products licensed under the IBM Program License Agreement (IPLA), and the optional associated agreements for Acquisition of Support. This combination of agreements provides to the users equivalent benefits to the previous traditional ICA license (see Figure 1-1).


Figure 1-1 Packaging of DB2 Utilities

The DB2 Utilities are grouped in these products:

� DB2 Operational Utilities

This product, program number 5655-E63, includes COPY, LOAD (including the new Cross Loader and Online LOAD RESUME), REBUILD INDEX, RECOVER, REORG TABLESPACE, REORG INDEX, RUNSTATS (enhanced with history), STOSPACE, EXEC SQL (new) and UNLOAD (new).

� DB2 Diagnostic and Recovery Utilities

This product, program number 5655-E62, includes CHECK DATA, CHECK INDEX, CHECK LOB, COPY, COPYTOCOPY (new), MERGECOPY, MODIFY RECOVERY, MODIFY STATISTICS (new), REBUILD INDEX, and RECOVER.

� DB2 Utilities Suite

This product, program number 5697-E98, combines the functions of both DB2 Operational Utilities and DB2 Diagnostic and Recovery Utilities in the most cost-effective option, and it is the subject of this redbook.

These products are installed separately from DB2 V7 when accessing user data; however, all of them are already available within DB2 when accessing the DB2 catalog, directory, or the sample database. You can install one or both of them, and give them a test run during the money-back guaranteed 30-day period. Verify the benefits they bring to your database operations.

The following utilities, and all stand-alone utilities (DSN1...), are considered core utilities, and are included and activated with DB2 V7: CATMAINT, DIAGNOSE, LISTDEF, OPTIONS, QUIESCE, REPAIR, REPORT, TEMPLATE, and DSNUTILS.

© 2000 IBM Corporation

Redbooks

Click here for optional figure #Click here for optional figure # YRDDPPPPUUU

Packaging of Utilities

Diagnostic and Recovery Utilities (5655-E62)

COPY, LOAD, REBUILD INDEX, RECOVER, REORG TABLESPACE, REORG INDEX, RUNSTATS, EXEC SQL, STOSPACE, UNLOAD

Operational Utilities (5655-E63)

Utilities Suite (5655-E98)

CHECK DATA, CHECK INDEX, CHECK LOB, COPY, COPYTOCOPY, MERGECOPY, MODIFY RECOVERY, MODIFY STATISTICS, REBUILD INDEX, RECOVER


Part 2 Planning for DB2 Utilities

In this part we include considerations on functions related on how to plan for, and when to execute, the DB2 Utilities. These considerations are discussed under the following topics:

� Chapter 2, “Wildcarding and Templates” on page 17� Chapter 3, “Inline executions and parallelism” on page 39� Chapter 4, “Triggering and controlling executions” on page 69� Chapter 5, “Managing partitions” on page 101

Part 2


Chapter 2. Wildcarding and Templates

DB2 Version 7 provides support for the use of pattern matching and dynamic allocation of data sets required by utilities. This alleviates the need for multiple invocations of utilities for similarly named objects and their removal from the JCL of utility data sets. This combination should improve DBA productivity by largely reducing tasks such as JCL writing.

In this chapter we introduce the concepts and discuss in detail the use of Wildcards and Templates, while specific implementation details are provided in the appropriate utility chapters.

2


2.1 Prior to Version 7 With DB2 versions prior to Version 7, if multiple objects are to be processed by a utility, then multiple utility statements have to be coded in the SYSIN ddname, and the appropriate data sets have to be coded in the JCL. This typically involves multiple jobs, or job steps, for each invocation of a utility, and the associated maintenance of the jobs when new objects are introduced to the DB2 subsystem. With DB2 Version 7, these issues are addressed with the introduction of Wildcarding and the use of Templates.

2.2 WildcardsThis section shows how to use the LISTDEF statements to provide Wildcarding, and the benefits of their use.

2.2.1 What is Wildcarding?Wildcarding provides the ability to define a list of DB2 objects, based upon a pattern rather than explicit names, and to assign a name to that list, via the LISTDEF statement. The list name makes the list available for subsequent execution as the object of a utility control statement or as an element of another LISTDEF.

DB2 will automatically generate a list of objects that matches the supplied pattern, and pass the list to one or more specific utilities for processing.

2.2.2 Benefits of using WildcardsThese are the benefits that the LISTDEF control statement provides:

� The LISTDEF statement can contain a list of specific objects, a generic expression, or both.

� Objects in the list can be either included or excluded to match requirements.

� Housekeeping is isolated from the effects of the introduction of new objects to the DB2 instance.

� Consistency points across multiple objects can be ensured by including all logically related objects in a single LISTDEF.

� Referential integrity (RI) sets can be processed in a single statement.

2.2.3 How to use WildcardingWildcarding is instigated by the inclusion of the LISTDEF statement in the utility job stream, prior to the utility control cards. The LISTDEF can be specified either in the SYSIN ddname or in a library allocated to the SYSLISTD ddname. The default ddname can be overridden via the OPTIONS statement.

A list-name is allocated to the LISTDEF as part of the control statement. If both the SYSIN and SYSLISTD are included, and the same list-name is found in both, then the instream LISTDEF will be used.

Objects can be specified within the LISTDEF at database through to index level. Which objects are returned depends upon whether table spaces or indexes have been requested. The various results are shown in Table 2-1.


Table 2-1 Object results table by keyword

Objects are added to the list via the keyword INCLUDE and removed from the built list via EXCLUDE. If an EXCLUDE attempts to remove an object that has not been included, then the EXCLUDE is ignored.

Lists generated by the LISTDEF statement are ordered, but are not sorted, and do not contain duplicates. If restart is required, then a checkpoint list is used at restart.

Following are examples comparing the use of LISTDEF and the method used prior to DB2 Version 7. The Version 6 JCL (see Example 2-1) will have to be altered for every new object that is required to be quiesced in database DBA.

Example 2-1 Version 6 and before

.....//SYSIN DD *QUIESCE TABLESPACE DBA.X

TABLESPACE DBA.Y TABLESPACE DBA.Z

No changes are necessary in Version 7 (see Example 2-2).

Example 2-2 Version 7

//SYSIN DD *LISTDEF DBA INCLUDE TABLESPACE DBA.*QUIESCE LIST DBA

As shown in Example 2-3, there is the ability to EXCLUDE objects from the list of objects passed on to the utility.

Example 2-3 Excluding objects

//SYSIN DD *LISTDEF DBA INCLUDE TABLESPACE DBA.*

EXCLUDE TABLESPACE DBA.ZQUIESCE LIST DBA

Object Type TABLESPACE keyword results INDEXSPACE keyword results

Database All table spaces within database All index spaces within database

Table space Ignored All index spaces related to tables in that table space

Table Table space containing the table All index spaces related to the table

Index space Table space containing related table Ignored

Index Table space containing related table Index space containing the index

List Table spaces from the expanded referenced list

Index spaces from the expanded referenced list

Important: A secondary INCLUDE statement may return a previously EXCLUDED object to the list.

Chapter 2. Wildcarding and Templates 19

The LIST keyword is supported by the utilities listed in Table 2-2, and where possible, the utility will optimize the list processing.

Table 2-2 Pattern-matching character comparison table

Further examples are displayed under the utilities that are able to accept LISTDEF statements. For a full list of utilities that can use LISTDEF, refer to Figure 2-6 on page 31.

2.2.4 Using multiple WildcardsAnother feature of LISTDEF is the ability to specify more than one set of Wildcards in an input stream, whether instream or via SYSLISTD. Each LISTDEF is processed and a list is generated for each one, but any list generated is only processed when it is referenced by a utility statement.

Be aware that each LISTDEF containing a wildcard will result in calls to the DB2 Catalog to satisfy the list. Thus, jobs may run longer than necessary when a large number of LISTDEFs are present in one job and are not referenced. Therefore, while including all LISTDEFs in every utility job would appear simpler, care should be taken with this approach.

A LISTDEF statement will only be expanded once; therefore, it can be referenced by many utilities in the job step without incurring the cost of expansion multiple times.

2.2.5 LISTDEF expansionWhen DB2 expands your list definition, that is, when it generates a list of objects according to your LISTDEF statement, the sequence of steps DB2 uses influences the result. You must know this sequence in order to code more complicated list definitions correctly.

Utility Method of processing

CHECK INDEX Grouped by related table space

COPY Processed in order specified on a single call to COPY

COPY PARALLEL Keyword is supported

MERGECOPY Processed in order specified

MODIFY RECOVERY Processed in order specified

MODIFY STATISTICS Processed in order specified

QUIESCE Processed in order specified on a single call to QUIESCE

REBUILD Grouped by related table space

RECOVER Processed in order specified on a single call to RECOVER

REORG Processed in order specified

REPORT Processed in order specified

RUNSTATS INDEX Grouped by related table space

RUNSTATS TABLESPACE Processed in order specified


Here is a simple example of this: Five table spaces and their unique indexes reside in the two databases DBX and DBY. The partitioned table space PTS1 has an additional non-partitioned index IXP1NP. After the previous recovery of a table space set to a prior point-in-time, since the indexes have not been recovered to the same point-in-time, a rebuild of all indexes on all tables in all table spaces of this table space set must be completed. Normally, if no COPY enabled indexes are involved, the first INCLUDE clause of the LISTDEF statement shown in Figure 2-1 contains the statements needed to fulfill this task.

Figure 2-1 LISTDEF scenario

An example of the complete recovery job with LISTDEF is listed in Example 2-4.

Example 2-4 Sample LISTDEF - Recovery job

LISTDEF RECLIST INCLUDE TABLESPACE DBY.TS% RILISTDEF RBDLIST INCLUDE INDEXSPACES TABLESPACE DBY.T% RIRECOVER LIST RECLIST TOLOGPOINT X'xxxxxxxxxxxxxxxx'REBUILD INDEX LIST RBDLIST

A further example is one where we assume that all partitioned indexes are to be rebuilt, per partition, and that they reside in database DBY. If all partitioned index space names in DBX start with ‘IXP’, then Figure 2-1 shows an alternative way, possible with V7. Again, the advantage is that you do not have to change the DB2 V7 job, if table spaces or indexes are dropped or added — as long as established naming conventions are followed.

The LISTDEF is expanded in four steps; see Figure 2-2.

LISTDEF expansion

DBX DBY

TS4RI RI RI

TS0 PTS1 TS2 TS3

UIXP1

NUIXP1NP

//SYSIN DD *LISTDEF RBDLIST INCLUDE INDEXSPACES TABLESPACE DBY.T* RI EXCLUDE INDEXSPACE DBX.IXP* INCLUDE INDEXSPACE DBX.IXP* PARTLEVELREBUILD INDEX LIST RBDLIST/*

//SYSIN DD *REBUILD INDEX (ALL) TABLESPACE DBY.TS3REBUILD INDEX (ALL) TABLESPACE DBY.TS4REBUILD INDEX authid.IXP1 PART 1REBUILD INDEX authid.IXP1 PART 2REBUILD INDEX authid.IXP1 PART 3REBUILD INDEX authid.IXP1NPREBUILD INDEX (ALL) TABLESPACE DBX.TS2/*

UIX2

UIX3

UIX4

V7

V6


Figure 2-2 LISTDEF expansion

1. An initial catalog lookup is performed to find and list the explicitly specified object or objects which match the specified pattern.In our example, in the first INCLUDE clause, table spaces are searched which match the pattern DBY.T*. The two matching table spaces, DBY.TS3 and DBY.TS4, are listed in the example at the right side of step 1.

2. Related objects are added or removed depending on the presence of the RI, BASE, LOB, or ALL keywords. Two types of relationships are supported, referential and auxiliary relationships. More specifically:

– If you specify RI, then the TABLESPACESET process is invoked and referentially related objects are added to the list. As a result, all referentially connected table spaces — by their tables — are included in the list. Figure 2-2 shows these four table spaces.

– If you specify LOB or ALL, then auxiliary related LOB objects are added to the list.

– If you specify BASE or ALL, then auxiliary related base objects are added to the list.

– If you specify LOB, then base objects are excluded from the list.

– If you specify BASE, then LOB objects are excluded from the list.

3. This step consists of two sub-steps:

a. All related objects of the requested type are searched and added to the list.This step can be skipped if the list built so far already consists of objects of the requested type, either table spaces or index spaces. Otherwise, the two cases must distinguished:

• If a list of table spaces is required, that is, the type specification is TABLESPACES, but the initial catalog lookup has been done with another type of object, for example, with index spaces or indexes, then related table spaces are added to the list. Obviously, those table spaces are related, in which the base tables reside of the indexes or index spaces already in the list.

LISTDEF expansion - the steps

Merge withprevious list result(here:EXCLUDE)

List all index spacesmatching DBX.IXP*

DBX.IXP1DBX.IXP1NP

DBY.IX3DBY.IX4DBX.IX2

B

Merge withprevious list result(here:INCLUDE)

List all index spacesmatching DBX.IXP* PARTLEVEL

DBX.IXP1 PART 1DBX.IXP1 PART 2DBX.IXP1 PART 3DBX.IXP1NP

DBY.IX3DBY.IX4DBX.IX2DBX.IXP1 PART 1DBX.IXP1 PART 2DBX.IXP1 PART 3DBX.IXP1NP

C

1

2

3

4

List alltable spacesmatching DBY.T*

DBY.TS3DBY.TS4

DBY.TS3DBY.TS4DBX.PTS1DBX.TS2

List all related objects of requested type (here: IS) & filter

DBY.IX3DBY.IX4DBX.IXP1DBX.IXP1NPDBX.IX2

Merge withprevious list result(here: n/a)

DBY.IX3DBY.IX4DBX.IXP1DBX.IXP1NPDBX.IX2

Add / removerelated objects(here: RI)

A

INCLUDE INDEXSPACES TABLESPACE DBY.T* RI EXCLUDE INDEXSPACE DBX.IXP* INCLUDE INDEXSPACE DBX.IXP* PARTLEVEL

3 1 2ABC

4LISTDEF RBDLIST


• If a list of index spaces is required, that is, the type specification is INDEXSPACES, but the initial catalog lookup has been done with another type of object, for example, with table spaces or tables, then the related index spaces are added to the list.

• In Figure 2-2, in the first INCLUDE clause, the four table spaces have five index spaces all together. These index spaces are added to the list.

b. A filtering process takes place:

• If INCLUDE TABLESPACES is specified, then all objects which are not table spaces are removed from the list. Analog rules apply for INCLUDE INDEXSPACES, EXCLUDE TABLESPACES, and EXCLUDE INDEXSPACES.

• The specification of COPY YES or NO is also taken into account in this step.

• In Figure 2-2, in the first INCLUDE clause, INCLUDE INDEXSPACES has been specified. Therefore, all table spaces are removed from the list. The list now contains five index spaces only, as you can see in the diagram.

4. If the keyword INCLUDE is specified, the objects of the resulting list are added to the list derived from the previous INCLUDE or EXCLUDE clauses, if not in the list already. In other words, an INCLUDE of an object already in the list is ignored; the list contains no duplicates. If the keyword EXCLUDE is specified, the objects of the resulting list are removed from the list derived from the previous INCLUDE or EXCLUDE clause. An EXCLUDE of an object not in the list is ignored. In Figure 2-2, for the first INCLUDE clause (A), there is no previous list to merge with, therefore, this step is skipped.

All four steps are explained for the first sample INCLUDE clause (A). Next, the EXCLUDE clause (B) and then the second INCLUDE clause (C) are processed. This is processed in a similar way as described above:

� Step 1 is similar to step 1 (already explained) of the first INCLUDE clause (A).

� Step 2 is skipped, as no relationship type was specified, neither RI, ALL, BASE, nor LOB.

� Step 3 is skipped, as spec-type INDEXSPACES is assumed if obj-type is INDEXSPACE. Therefore, there is no need to look for related objects, nor to filter out objects of another type.

� In step 4 for the EXCLUDE clause (B), two objects are removed from the list generated by the first INCLUDE clause (A).

� In step 4 for the second INCLUDE clause (C), four objects are added to the list generated after the EXCLUDE clause (B).

A sample of the utility output is shown in Example 2-5, to show the expansion of the LISTDEF.

Note: The purpose of the last example is to conceptually present the resolution of a LISTDEF statement in order to help with defining LISTDEF statements properly. The purpose is not to precisely document DB2’s implementation of the sequence of the expansion steps (which is in fact different) in order to be generally applicable, not only for the example. DB2 may perform step 2 after step 3, for example, if you specify INCLUDE TABLESPACES INDEX authid.ix RI. Furthermore the consideration of the PARTLEVEL keyword may be postponed to the end, that is, after step 3.


Example 2-5 Sample output

- OUTPUT START FOR UTILITY, UTILID = TEMP - OPTIONS PREVIEW - PROCESSING CONTROL STATEMENTS IN PREVIEW MODE - OPTIONS STATEMENT PROCESSED SUCCESSFULLY - LISTDEF RBDLIST INCLUDE INDEXSPACES TABLESPACE DBY.T* RI EXCLUDE INDEXSPACE DBX.IXP* INCLUDE INDEXSPACE DBX.IXP* PARTLEVEL - LISTDEF STATEMENT PROCESSED SUCCESSFULLY - EXPANDING LISTDEF RBDLIST - PROCESSING INCLUDE CLAUSE TABLESPACE DBY.T* <- A: INCLUDE - CLAUSE IDENTIFIES 5 OBJECTS <- after step 3 - PROCESSING EXCLUDE CLAUSE INDEXSPACE DBX.IXP* <- B: EXCLUDE - CLAUSE IDENTIFIES 2 OBJECTS <- after step 3 - PROCESSING INCLUDE CLAUSE INDEXSPACE DBX.IXP* <- C: INCLUDE - CLAUSE IDENTIFIES 4 OBJECTS <- after step 3 - LISTDEF RBDLIST CONTAINS 7 OBJECTS <- final list- LISTDEF RBDLIST EXPANDS TO THE FOLLOWING OBJECTS: LISTDEF RBDLIST -- 00000007 OBJECTS INCLUDE INDEXSPACE DBY.IX3 INCLUDE INDEXSPACE DBY.IX4 INCLUDE INDEXSPACE DBX.IXP1NP INCLUDE INDEXSPACE DBX.IX2 INCLUDE INDEXSPACE DBX.IXP1 PARTLEVEL(00001) INCLUDE INDEXSPACE DBX.IXP1 PARTLEVEL(00002) INCLUDE INDEXSPACE DBX.IXP1 PARTLEVEL(00003) DSNUGUTC - REBUILD INDEX LIST RBDLIST DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBY.IX3 DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBY.IX4 DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBX.IXP1NP DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBX.IXP1 DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBX.IXP1 DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBX.IXP1 DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE DBX.IX2 DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

2.2.6 Recommendations for the use of WildcardsWildcarding should be used wherever possible to isolate the JCL from changes to DB2 objects. This will directly result in less effort being required to ensure the integrity of the DB2 instance.

Group logically related objects into the same LISTDEF to ensure consistency.

Use partitioned data set (PDS) members as input to reduce maintenance across JCL and to ensure consistency.

Ensure that all databases are referenced by at least one LISTDEF, or are explicitly coded.

Revisit LISTDEFs periodically to ensure the integrity and completeness of the definitions. Naming standards for user defined objects would assist in this process and allow for even less maintenance of LISTDEF.

Use OPTION PREVIEW mode to review the output from LISTDEF.

Use in conjunction with Templates. See 2.3, “Templates” on page 25 for the full benefits.


2.2.7 Current restrictions in the use of WildcardsThe use of Wildcarding cannot be used against the catalog or directory objects. If either of these are required in a LISTDEF statement, then these have to be explicitly coded.

The options RI and PARTLEVEL are mutually exclusive.

Unless the OPTION statement is used, if one object fails due to restriction, then all objects in the list are not processed. By using OPTION EVENT(ITEMERROR, SKIP) this can be avoided.

LISTDEF is not supported by the following utilities:

� DIAGNOSE� CATMAINT� STOSPACE� REPAIR� CHECK DATA

2.3 TemplatesTemplates are a method of defining data set masks that are to be used by utilities to dynamically allocate data sets required for successful execution of the utility. They can be used for work data sets as well as utility data sets, for instance, image copy data sets. When used in conjunction with LISTDEF, this provides a powerful mechanism for executing utilities, allows faster development of job streams, and requires fewer modifications when the underlying list of database objects changes.

Templates allow the writing of data sets to both disk and tape units and incorporates all the features normally coded within the DD statement. This also allows for the stacking of data sets on tapes automatically. The Template statement executes entirely in the UTILINIT phase in preparation for the subsequent utility. Output from the Template control statement is a dynamic allocation Template with an assigned name for later reference.

The use of Templates also introduces the concept of automatic intelligent data set sizing. For this to be efficient, RUNSTATS have to be current.

2.3.1 Benefits of using TemplatesThe benefits of using Templates include:

� Less reliance on specialist JCL knowledge� Free DBA resource for more important tasks� Enforcement of naming standards� Standardization of data set allocations� Multiple Templates per job� Used for workfiles� Changes to naming standards quickly and easily implemented� Automatic data set sizing� Used alongside LISTDEF for full flexibility� Automatically defines GDG bases where required


2.3.2 How to use TemplatesTemplates can be used to eliminate the requirement for DD statements within the JCL allowing for standardized JCL to be used for all the utilities that support the use of Templates.

Templates are initiated through the use of the TEMPLATE keyword in the utility job stream before the utility commands. Like LISTDEF, the Templates can either be coded within the SYSIN ddname, or in a data set allocated to the job stream via the default SYSTEMPL ddname. This is the default and can be overridden by using the OPTIONS keyword.

The minimum requirement for a Template statement is a name, for reference by the utility command, and the data set naming mask. If no sizing information is supplied, then DB2 calculates the space required for the data set. If the Template supplies any other parameters, then these parameters take precedence; see Example 2-6 and Example 2-7.

Example 2-6 Template not invoking data set sizing

TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.L) UNIT(SYSDA) SPACE(15,15) TRK DISP(NEW,CATLG,CATLG) VOLUMES(SBOX57)

Example 2-7 Template invoking data set sizing

TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.L)

Templates can be used to write to either disk or tape. Support is included for stacking multiple files onto a single tape via the STACK option; see Example 2-8. Support for DF/SMS is also included.

Example 2-8 Template using tape with STACK option

TEMPLATE REMOTDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.R) UNIT ATL2 VOLUMES(TST002) STACK YES

There can be more than one Template per job stream (see Example 2-9). Each one must have a unique name, and a utility can reference more than more than one Template, if that utility supports multiple output ddnames, for example, COPY.

Example 2-9 Using multiple Templates

TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.L) UNIT(SYSDA) SPACE(15,15) TRK DISP(NEW,CATLG,CATLG) VOLUMES(SBOX57) TEMPLATE REMOTDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.R) UNIT ATL2 VOLUMES(TST002) STACK YES LISTDEF DB01A INCLUDE TABLESPACE DBX.TSY COPY LIST DB01A FULL YES SHRLEVEL CHANGE COPYDDN(LOCALDDN) RECOVERYDDN(REMOTDDN)


If the DISPOSITION option is not specified in the Template definition, DB2 will use the disposition defaults for the utility concerned. DB2 will also take into account any restart implications. For an example using REORG, see Figure 2-3.

Figure 2-3 Example disposition with Templates

Further examples of Templates are given under the relevant utility sections.

2.3.3 Naming standardsVariables are supported within DB2 that allow data sets to be built dynamically. DB2 does not check to ensure that they are unique until execution of the utility. Therefore, care must be exercised when constructing data set names to ensure uniqueness. Normal OS/390 data set naming conventions apply, for example, all qualifiers must begin with an alphabetical character and the length cannot exceed 44 characters, and so on.

The use of Templates allows for the standardization of data set names across the DB2 instance, and allows easy identification of the data set type created, if the relevant variables are used in construction of the name, for example, &IC for image copy.

A full list of the currently allowable variables that can be used within the Template statement is provided in Table 2-3.

Note: The DISP field in TEMPLATES does NOT support the typical default MVS syntax, such as DISP=(,,KEEP). In Templates, ALL three parameters must be specified.

D IS P O S IT IO N w ith the TE M P LA TE

E xam ple : R E O R G

status norm al te rm ina tion abnorm al te rm ination

N ew u tility S YS R EC N E W C A TLG C A TLGexecution S YS U T1 N E W D E LE T E C A TLG

S O R T O U T N E W D E LE T E C A TLG

R estarted S YS R EC M O D C A TLG C A TLGutility S YS U T1 M O D D E LE T E C A TLG

S O R T O U T M O D D E LE T E C A TLG


Table 2-3 TEMPLATE variables

2.3.4 Intelligent data set sizingTo use DB2 automatic data set sizing, Templates must be used. As stated earlier, if the space parameters are missing from the Template statement, then DB2 estimates the sizes of the of the data sets based on a utility-specific formula. For this feature to work successfully, the object statistics have to be kept up-to-date.

The input values used in these formulas mostly come from the DB2 catalog tables. The high-used RBA is read at open time and it is maintained by the buffer manager. It is the most current value, updated before it is even written to the ICF catalog.

All these formulas are documented in the standard DB2 manuals. Figure 2-4 is an example of the specific formulas for the data sets for the REORG utility, in cases where you use:

� REORG UNLOAD CONTINUE SORTDATA KEEPDICTIONARY SHRLEVEL NONE� SHRLEVEL REFERENCE

JOB variables

UTILITY variables

OBJECTvariables

DATE and TIMEvariables

JOBNAME UTIL (utility name truncated)

DB (dbname)

DATE (yyyyddd)

STEPNAME ICTYPE (“F” or “I”)

TS(table space name)

YEAR (yyyy)

USERID or US LOCREM or LR(“L” or “R”)

IS(index space name)

MONTH(mm)

SSID(subsystem ID or DS group attach name)

PRIBAC or PB(“P” or “B”)

SN(space name)

DAY(dd)

PART(five digits)

JDATE or JU(Julian date yyyyddd)

LIST(name of the list)

JDAY(ddd)

SEQ(sequence number of the object in the list)

TIME(hhmmss)

HOUR

MINUTE

SECOND or SC


Figure 2-4 Automatic data set sizing for the REORG utility

2.3.5 Recommendations for the use of TemplatesTemplates should be used whenever possible, and used in conjunction with LISTDEF, which provides a powerful mechanism for controlling the housekeeping of a DB2 instance. Templates can be used without LISTDEFs where LISTDEF is not supported, for example, copying of catalog and directory.

Use variables to make data set names meaningful; for example, use &LR and &IC to identify image copy data sets.

Use PDS members as input to reduce maintenance.

Define Templates twice, once for tape (where applicable) and once for disk, to allow for quicker interchanging between devices.

Use the STACK option for tape data sets.

Use PREVIEW mode to review the output from LISTDEF and Templates.

Use the GDG option to ensure that GDGs are utilized consistently.

Space allocation with the TEMPLATE

If the SPACE option is not specified in the TEMPLATE definition, DB2 will calculate the space needed.

Sample: REORG UNLOAD CONTINUE SORTDATA SHRLEVEL NONE or REFERENCE

SYSUT1, SORTOUT

DB2 catalogSYSDISC

SYSPUNCH

COPYDDN, RECOVERYDDN

Data setinformation

secondary:

always 10 %

SYSREC

primary = (#keys) * (key length) bytes, where: - #keys = #table rows * #indexes - key length = (largest key length) + 20

primary = 10 % of (max row length) * (#rows) bytes

primary = ((#tables * 10) + (#cols * 2)) * 80 bytes

primary = hi-used RBA bytes

primary = ( hi-used RBA ) + ( #records * 12 + length of longest clustering key)) bytes

Note: Ensure that the PTF for APAR PQ45835 has been applied to fully enable the STACK option feature.


2.3.6 Current restrictions in the use of TemplatesThe only variables that can be used with Templates are those listed in Table 2-3 on page 28. Further variables may be added in future releases.

Column functions are not supported in Templates, for example, SUBSTR.

Data sets whose size is calculated as too large for the DYNALLOC interface must be allocated using DD cards. Message DSNU1034I is issued if this condition is met.

2.4 Combining Wildcards and TemplatesThe use of LISTDEF often requires the use of Templates. If the requirement is for a utility to process a dynamic list of DB2 objects, that is, a list which may increase or decrease over time, you can use LISTDEF. As a result, many online utilities require data set allocations for each object processed. Therefore, the number (and parameters) of these allocations must be dynamic too. This can be implemented with Templates, thus supporting a dynamic number of data set allocations, fitting to the object or to the list of objects being processed by a utility.

For example, primary and backup copies are required per partition for all partitioned table spaces in the database DBX whose names start with PT. This leads to the LISTDEF statement shown in Figure 2-5. Not knowing how many table spaces exist which match that list definition, and not knowing how many partitions these table spaces have, the Template is used for dynamically allocating the needed copy data sets. As the Template includes the variable &PRIBAC, this Template can be used in place of both DD names, the one for the primary copy and the one for the backup copy.

At job execution, DB2 first processes the LISTDEF statement, then the Template definition is read and then stored. When DB2 has read the COPY statement, the list COPYLIST is generated and DB2 copies each item on the list, assuming that there are no restrictions. At this time, DB2 knows all the details for the variables and can substitute them into the Template variables stored previously.

With DB2 V6, data set allocations are done at the beginning of the job step. With Templates and lists in V7, the dynamic allocations are executed when the utility is processing the objects for which the allocation is needed.

If several executions of CHECK INDEX, REBUILD INDEX, RUNSTATS INDEX utilities are invoked for multiple indexes for a list of objects, performance can be improved. For instance, in the case of two table spaces with three indexes each, consider this coding for the LISTDEF:

LISTDEF myindexes INCLUDE INDEXSPACES TABLESPACE db1.ts1INCLUDE INDEXSPACES db2.ts2

DB2 generates CHECK INDEX statements for all indexes of the same table space as follows:

CHECK INDEX db1.ix1,db1.ix2,db1ix3 CHECK INDEX db21.ix1,db2.ix2,db2ix3

Thus the table space scans are reduced to two instead of a possible six.

Notes:

� Apply the PTF for PQ45835 for problems with COPY when using LISTDEF and TEMPLATE.

� Apply the PTF for PQ50223 for problems when loading partitions using TEMPLATE.


Figure 2-5 TEMPLATE and LISTDEF combined (V6 shown for comparison)

2.4.1 Compatibility with utilitiesTemplates and LISTDEF can be used according to the table in Figure 2-6. Templates can obviously only be used where an output data set is required by the utility, such as COPY, but not MODIFY.

Figure 2-6 Utility/LISTDEF/Template cross reference

TE M P LA TE and L IS TD E F com bined//C O P Y P R I1 D D D S N =D B X .P T S 1.P 00001.P .D 2000199,// D S IP =...,U N IT= ...,S P AC E =...//C O P Y P R I2 D D D S N =D B X .P T S 1.P 00002.P .D 2000199,// D S IP =...,U N IT= ...,S P AC E =...//C O P Y P R I3 D D D S N =D B X .P T S 1.P 00003.P .D 2000199,// D S IP =...,U N IT= ...,S P AC E =...//C O P Y S E C 1 D D D S N =D B X .P TS 1.P 00001.B .D 2000199,// D S IP = ...,U N IT=...,S P AC E =...//C O P Y S E C 2 D D D S N =D B X .P TS 1.P 00002.B .D 2000199,// D S IP = ...,U N IT=...,S P AC E =...//C O P Y S E C 3 D D D S N =D B X .P TS 1.P 00003.B .D 2000199,// D S IP = ...,U N IT=...,S P AC E =...//S Y S IN D D *C O P Y TAB LE S P AC E D B X .P TS 1 D S N U M 1 C O P Y D D N (C O P Y P R I1,C O P Y S E C 1)C O P Y TAB LE S P AC E D B X .P TS 1 D S N U M 2 C O P Y D D N (C O P Y P R I2,C O P Y S E C 2)C O P Y TAB LE S P AC E D B X .P TS 1 D S N U M 3 C O P Y D D N (C O P Y P R I3,C O P Y S E C 3)/*

V 7

V 6

//* none o f the D D statem ents above//S Y S IN D D *L IS T D E F C O P Y LIS T IN C LU D E TAB LE S P AC E D B X .P T* P AR T LE V E LTE M P LATE C O P Y TM P L D S N ( & D B ..& T S ..P & P AR T..& P R IB AC ..D & JD AT E . )C O P Y L IS T C O P Y LIS T C O P Y D D N ( C O P Y T M P L,C O P Y TM P L )/*

Utility LISTDEF TEMPLATECHECK DATA No YesCHECK INDEX Yes YesCHECK LOB No YesCOPY Yes YesCOPYTOCOPY Yes YesLOAD No YesMERGECOPY Yes YesMODIFY RECOVERY Yes n/aMODIFY STATISTICS Yes n/aQUIESCE Yes n/aREBUILD INDEX Yes YesRECOVER Yes n/aREORG INDEX Yes YesREORG TABLESPACE Yes YesREPORT Yes n/aRUNSTATS Yes n/aUNLOAD Yes Yes

Object Wildcard and Dynamic Allocation by Utility


2.4.2 Mixing the old and the newIn some cases you have to use a mixture of the old and the new method of processing. For example, the Catalog and Directory cannot be processed via the LISTDEF command using Wildcards, they have to be coded explicitly. See Appendix B, “Sample JCL for copying Catalog and Directory objects” on page 287.

The use of LISTDEF and Templates go hand-in-hand. The ability to generate lists gives rise to the need for dynamic allocation, therefore, the use of both together is recommended, where their use is supported by the utility (see Figure 2-6). However, this does not mean that you cannot use them independently. For example, if naming standards do not lend themselves to the use of LISTDEF, then Templates can still be used to provide dynamic allocations. Example 2-10 provides an example using the new UNLOAD utility.

Example 2-10 Using LISTDEF and Templates together

TEMPLATE ULDDDN DSN(DB2V710G.&DB..&TS..UNLOAD) UNIT(SYSDA) SPACE(45,45) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(DB2V710G.&DB..&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE U7G01T11.TSCUST FROMCOPY DB2V710G.U7G01T11.TSCUST.D2001145.T022853L PUNCHDDN PNHDDN UNLDDN ULDDDN FROM TABLE PAOLOR1.CUSTOMER (C_CUSTKEY,C_NAME,C_ADDRESS) WHEN ( C_CUSTKEY > 1000 )

2.4.3 Object in restricted statusWhen a DB2 object is in a restricted state, for example, STOP, that prevents a utility from executing successfully, this is NOT recognized during LISTDEF processing. The LISTDEF and Template processing is executed normally, for example, lists are built and Templates are constructed. When the utility processes the restricted object, it will produce an Unavailable Resource message. The actions of the utility then depend upon the OPTIONS control statement. If OPTIONS(ITEMERROR,SKIP) is specified, processing continues with the next object and will set a return code of 8. If this OPTION is not specified, then processing stops at the failing object, again with a return code of 8. If the utility running would normally terminate with an abend in this situation, ITEMERROR will not convert the abend to a RC8.

The benefits of using LISTDEF and Templates in this situation is that data sets are only allocated at the time that they are required, therefore in the above scenario, there are no extraneous data sets to delete. If Templates are not being used, then it is the users responsibility to ensure that the data sets are managed correctly.

2.4.4 Utility restartabilityWhen restarting a utility job stream that is utilizing LISTDEF and Templates, restartability within the job is controlled via a checkpoint list built from the LISTDEF, and is stored in SYSUTILX. The LISTDEF is not reevaluated again. DB2 will automatically alter the disposition of the required data set to that required for a restart, for example, using REORG (see Figure 2-3 on page 27).


Default values for disposition vary depending upon the utility and whether the utility is restarting. Default values are shown in Table 2-4 and Table 2-5.

Table 2-4 Data dispositions for dynamically allocated data sets

ddname CHECKDATA

CHECK INDEX OR CHECK LOB

COPY LOAD MERGE COPY

REBUILD INDEX

REORG INDEX

REORG TABLE SPACE

UNLOAD

SYSREC Ignored Ignored Ignored OLD,KEEP,KEEP

Ignored Ignored Ignored NEW,CATLG,CATLG

NEW,CATLG,CATLG

SYSDISC Ignored Ignored Ignored NEW,CATLG,CATLG

Ignored Ignored Ignored NEW,CATLG,CATLG

Ignored

SYSPUNCH Ignored Ignored Ignored Ignored Ignored Ignored Ignored NEW, CATLG, CATLG

NEW, CATLG, CATLG

SYSCOPY Ignored Ignored NEW, CATLG, CATLG

NEW, CATLG, CATLG

NEW, CATLG, CATLG

Ignored Ignored NEW, CATLG, CATLG

Ignored

SYSCOPY2 Ignored Ignored NEW, CATLG, CATLG

NEW, CATLG, CATLG

NEW, CATLG, CATLG


Ignored

SYSRCPY1 Ignored Ignored NEW, CATLG, CATLG

NEW, CATLG, CATLG

NEW, CATLG, CATLG


Ignored

SYSRCPY2 Ignored Ignored NEW, CATLG, CATLG

NEW, CATLG, CATLG

NEW, CATLG, CATLG


Ignored

SYSUT1 NEW, DELETE, CATLG

NEW, DELETE, CATLG

Ignored NEW, DELETE, CATLG

Ignored NEW, DELETE, CATLG

NEW, DELETE, CATLG

NEW, DELETE, CATLG

Ignored

SORTOUT NEW, DELETE, CATLG

Ignored Ignored NEW, DELETE, CATLG

Ignored Ignored NEW, DELETE, CATLG

NEW, DELETE, CATLG

Ignored

SYSMAP Ignored Ignored Ignored NEW, CATLG, CATLG

Ignored Ignored Ignored Ignored Ignored

SYSERR NEW, CATLG, CATLG




Table 2-5 Data dispositions for dynamically allocated data sets on RESTART

2.4.5 Use with partitionsPartitions must be specified slightly differently from non-partitioned table spaces. If the utility is to process at the partition level, then first the table space (DSNUM 0) is EXCLUDED and then the partitions are INCLUDED using PARTLEVEL.

ddname CHECKDATA

CHECK INDEX OR CHECK LOB

COPY LOAD MERGE COPY

REBUILD INDEX

REORG INDEX

REORG TABLE SPACE

UNLOAD

SYSREC Ignored Ignored Ignored OLD,KEEP,KEEP

Ignored Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

SYSDISC Ignored Ignored Ignored MOD,CATLG,CATLG

Ignored Ignored Ignored MOD,CATLG,CATLG

Ignored

SYSPUNCH Ignored Ignored Ignored Ignored Ignored Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

SYSCOPY Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

MOD,CATLG,CATLG

Ignored Ignored MOD,CATLG,CATLG

Ignored

SYSCOPY2 Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

MOD,CATLG,CATLG


Ignored

SYSRCPY1 Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

MOD,CATLG,CATLG


Ignored

SYSRCPY2 Ignored Ignored MOD,CATLG,CATLG

MOD,CATLG,CATLG

MOD,CATLG,CATLG


Ignored

SYSUT1 MOD,DELETE,CATLG

MOD,DELETE,CATLG

Ignored MOD,DELETE,CATLG

Ignored MOD,DELETE,CATLG

MOD,CATLG,CATLG

MOD,DELETE,CATLG

Ignored

SORTOUT MOD,DELETE,CATLG

Ignored Ignored MOD,DELETE,CATLG

Ignored Ignored MOD,DELETE,CATLG

MOD,DELETE,CATLG

Ignored

SYSMAP Ignored Ignored Ignored MOD,CATLG,CATLG


SYSERR MOD,CATLG,CATLG




2.5 OPTIONS The utility control statement OPTIONS is provided to complement the functionality offered by LISTDEF and Templates. OPTIONS provide a general mechanism to enter options that affect utility processing across multiple utility executions in a job step. Figure 2-7 illustrates the functions that can be utilized by specifying the respective keywords in the OPTIONS statement. OPTIONS should be coded within the SYSIN in front of the utility control statements to which the options apply. Any subsequent OPTIONS statements entirely supersede any prior statements.

Figure 2-7 OPTIONS functions

2.5.1 OPTIONS PREVIEWSpecifying OPTIONS(PREVIEW) will expand the list definitions and the Template definitions and report their contents whenever a utility statement, for example COPY, follows. All utility control statements are parsed for syntax errors, but normal utility execution will not take place. If the syntax is valid, all LISTDEF lists and TEMPLATE data set names which appear in SYSIN will be expanded and the results printed to the SYSPRINT data set. A useful feature of this process is that the expanded list from the SYSPRINT data set can be captured and used inside a utility control statement to avoid the repetition of the expansion process. Lists from the SYSLISTD DD data set and Template data set names from the SYSTEMPL DD data set which are referenced by a utility invocation will also be expanded.

OPTIONS(PREVIEW) does not check the validity of the TEMPLATE data set names, their syntax, or whether the data sets names are unique.

For a preview example, see the previous job output in Example 2-5 on page 24 showing the LISTDEF expansion.

Function OPTIONS keyword [parameter]Test feature PREVIEW

Indentify / override default DD names:

- for list definitions LISTDEFDD dd-name

- for templates TEMPLATEDD dd-name

Error handling EVENT ( ITEMERROR, HALT ! SKIP )

Handling of warning messages EVENT ( WARNING, RC0 ! RC4 ! RC8 )

Restore default options OFF

and:

Utility activation key KEY

Several other functions: OPTIONS

Example: OPTIONS LISTDEFDD MYLISTS EVENT ( ITEMERROR, SKIP, WARNING, RC8 ) COPY LIST COPYLIST COPYDDN ( COPYTMPL, COPYTMPL )Reset rule: An OPTIONS statement replaces any prior OPTIONS statement


The option OPTIONS(PREVIEW) is identical to the PREVIEW EXEC parameter (see Figure 2-8), which turns on the preview mode for the whole job step. With the new OPTIONS(PREVIEW) utility control statement, you have a more detailed granularity: Preview mode, via the utility control statement, can be switched on and off multiple times within a single job step, but when preview mode has been initiated by the JCL parameter PREVIEW, the OPTIONS control statement cannot switch it off.

PREVIEW can also be initiated through DSNUTILS, and the use of “ANY” suppresses all dynamic allocations done by DSNUTILS. Using PREVIEW, this list can be saved before the actual utility is run, and if there is an error, the job output will show the list item where it failed. Since DSNUTILS does not provide for storing the reduced list, this saved list can be edited by removing the already processed items and starting again with this reduced list.

Figure 2-8 Utility support for PREVIEW

2.5.2 Other OPTIONS functionalityOPTIONS can also be used in the following circumstances:

� It can override default ddnames for LISTDEF and Templates:

OPTIONS LISTDEFDD(listindd) TEMPLATEDD(tempindd)

� DB2 handling of errors can be controlled; this can only be used when processing a list.

– To halt on such errors during list processing (default):

OPTIONS EVENT(ITEMERROR, HALT)

– To skip any error and keep going:

OPTIONS EVENT(ITEMERROR, SKIP)

� Handling Return code 4 from a utility is available whether using a list or not:

OPTIONS EVENT(WARNING,RC0)OPTIONS EVENT(WARNING,RC4)OPTIONS EVENT(WARNING,RC8)

Utility support for PREVIEW

DSNUPROC

DSNUTILB

DSNUTILS

PREVIEW mode is invoked like RESTART(CURRENT):

EXEC DSNUPROC,SYSTEM=ssnn,UID=utility-id,UTPROC=restart-parm

EXEC PGM=DSNUTILB,PARM='ssnn,utility-id,restart-parm'

CALL DSNUTILS (utility-id,restart-parm,utstmt,retcode,utility-name,....)

'PREVIEW' 'ANY'

'PREVIEW'


� Restoring defaults can be accomplished as follows:

OPTIONS OFF

Note: No other keywords may be specified with the OPTIONS OFF setting.

OPTIONS OFF is equivalent to:

OPTIONS LISTDEFDD SYSLISTD TEMPLATEDD SYSTEMPL EVENT (ITEMERROR, HALT, WARNING, RC4)

As usual for utility control statements, parentheses are required for a list, in contrast to a single item, to be specified.


Chapter 3. Inline executions and parallelism

In each new version of DB2, IBM has improved the performance of the utilities. This is to address the needs of customers who are challenged with increasing amounts of data and a shrinking batch window.

In this chapter we will focus on parallel executions, which is one of the techniques that DB2 uses to shorten the elapsed times of utilities.

During the three last versions of DB2, different forms of parallelism have been introduced. Consider the following:

� Inline executions, which is the execution of different types of utilities in parallel subtasks instead of executing them serially. Examples are COPY and RUNSTATS running embedded and in parallel with LOAD and REORG utilities.

� Execution of one type of utility on different DB2 objects in parallel subtasks. Examples are COPY/RECOVER of a list of table spaces in parallel or rebuilding the different indexes of a table space in parallel during LOAD or REORG TABLESPACE.

� Execution of a utility on different partitions of a partitioned table space, or parts of a partitioned index in parallel subtasks. Examples are rebuilding the parts of a partitioned index in parallel, unloading the partitions of a partitioned table space in parallel, and loading the partitions of a partitioned table space in parallel.

Before, some of this parallelism could be achieved by the users by submitting different jobs in parallel, sometimes ending up with considerable contentions. Today, all these parallel executions are done in one job, using parallel subtasks.

In this chapter, we discuss the following capabilities:

� Inline COPY with LOAD REPLACE and REORG TABLESPACE � Inline RUNSTATS with LOAD,REORG TABLESPACE,REORG INDEX and REBUILD

INDEX� COPY/RECOVER a list of table spaces and/or indexes in parallel� Building the indexes of a table space in parallel during LOAD or REORG TABLESPACE� Rebuilding a partitioned index in parallel during REBUILD INDEX� Rebuilding multiple indexes in parallel with REBUILD INDEX� Loading partitions of a partitioned table space in parallel� Unloading partitions of a partitioned table space in parallel

3


Also in this chapter, we also discuss the use of SORTKEYS. Originally, SORTKEYS eliminated the need for intermediate workfiles (SYSUT1or SORTOUT) when building the indexes by passing the extracted keys in memory to the sort process. Today the same keyword SORTKEYS is also used for enabling Parallel Index Build (PIB).

Please note that we cover only those aspects which are common for the different utilities. For more details and examples, we refer to the chapters on the individual utilities. Some performance figures are shown to give an idea what reduction in elapsed times can be expected by using parallelism in DB2 utilities.

3.1 Inline executionsThe term inline executions refers to the execution of different types of utilities in parallel subtasks instead of executing them serially. Examples are COPY and RUNSTATS running embedded and in parallel with LOAD and REORG utilities.

Before DB2 V5, when executing utilities, such as REORG or LOAD of a table space, without logging, the DB2 object is left in a restrictive state, COPYPEND, and an image copy is required to make the object fully available to the applications. To ensure that the correct access paths are chosen, and the other statistics are current, the RUNSTATS utility is also recommended to be run to update the statistics of the table. These utilities added to the time and resource (CPU, I/O) required to carry out the original utility. DB2 addressed these issues with the introduction of the Inline COPY and RUNSTATS functionality. These enhancements ensured, when the original utility completed, that the object was fully available to the application, and that the statistics were current.

Starting with Version 5, DB2 introduced the ability to take image copies of a table space in parallel with some utilities. With Version 6 this ability was extended to include the gathering of statistics. These features are known as “inline” utilities, and are run as subphases of the originating utility.

3.1.1 Inline COPY Version 5 of DB2 introduced the ability of LOAD REPLACE and REORG TABLESPACE to take an image copy while loading or reloading data into a table; see Example 3-1. This leads to the table space NOT being put into COPYPEND status even if LOG NO is specified. Up to four copies are allowed (two primary, two recovery) and the process is initiated by the use of the COPYDDN/RECOVERYDDN control statements, as detailed in Example 3-1. All image copies taken are taken as a full SHRLEVEL REFERENCE copy.

Example 3-1 Initiating multiple Inline COPY within LOAD utility

LOAD DATA REPLACE COPYDDN ddname1,ddname2 RECOVERYDDN dname3,ddname4 INTO TABLE tbname

REORG TABLESPACE dbname.tsname COPYDDN ddname1,ddname2 RECOVERYDDN dname3,ddname4


Example 3-2 shows the Inline COPY at partition level.

Example 3-2 Initiating Inline COPY when loading at partition level

LOAD DATA INTO TABLE tbname PART 2 REPLACE COPYDDN ddname1

REORG TABLESPACE dbname.tsname PART 2 COPYDDN ddname1

Although DB2 V6 introduced image copies of indexes (see 10.1.2, “Copying indexes” on page 232), DB2 does not support the concept of Inline COPY of an index. Not having an image copy of an index defined with the COPY YES attribute will put the index in the ICOPY state, but in contrast with the COPYP status, this is not a restrictive state.

The inline image copy data sets are not exactly the same as the data sets produced by the COPY utility:

� Logically they are the same, except that:

– Data pages may be out of sequence and/or repeated. If repeated, the last occurrence is the correct page.

– Space map pages may be out of sequence and repeated.

– Compression dictionary, if built, occurs twice. The second one is the correct one.

– Additional space may be required due to the duplication of pages that occurs because the image copy data set contains the full copy plus all the incremental copies.

� SYSCOPY is updated with ICTYPE = F and SHRLEVEL = R. The originating utility can be found in STYPE (R for LOAD REPLACE LOG YES, S for LOAD REPLACE LOG NO, W for REORG LOG NO, X for REORG LOG YES). If STYPE is blank or contains another value, the image copy is not an Inline COPY.

� DSN1COPY and DSN1PRNT require extra options to process inline copies. See “Using inline copies with DSN1COPY” on page 45 and “Using inline copies with DSN1PRNT” on page 46.

� Inline COPY data sets from the LOAD utility are NOT recommended for use with RECOVER TOCOPY, because they are taken during the LOAD phase before any DISCARD phases; therefore copies may contain unique index and/or RI violations. They can be used in point-in-time recovery. If a RECOVERY TOCOPY is executed, then all indexes are placed in recovery pending status, and dependent table spaces, if any, are placed in check pending status. RECOVER INDEX and CHECK DATA must be run.

The benefits of taking an inline image copy is that the elapsed time is reduced when compared with running the utility followed by a copy, and that the time unavailable to the applications is also shorter. Table 3-1 shows the performance measurements of LOAD and REORG with Inline COPY on a DB2 table with two indexes, one partitioning index and one non-partitioning index. The figures are taken from the introduction of Inline COPY as at Version 5 DB2.

Note: If COPYDDN is specified without the REPLACE option, the LOAD is terminated.

Chapter 3. Inline executions and parallelism 41

Table 3-1 Inline COPY in LOAD and REORG TABLESPACE with two indexes

These figures show that considerable savings can be made in elapsed time, while consuming virtually the same amount of CPU time, by the use of Inline COPY.

Inline copies in SYSCOPYDB2 records inline copies in SYSIBM.SYSCOPY as full image copies with ICTYPE = ‘F’ and SHRLEVEL REFERENCE (see Example 3-3.) In addition, DB2 Version 7 records these details about the copy:

� COPYPAGESF: The number of pages written to the copy data set� NPAGESF: The number of pages in the table space or index at the time of the Inline COPY� CPAGESF: Total number of changed pages

Example 3-3 SYSCOPY entry for Inline COPY

DBNAME TSNAME DSNUM ICTYPE START_RBA COPYPAGESF NPAGESF CPAGESF U7G01T11 TSLINEI 0 R 0000620116C9 -0.1000000000000000E+01 -0.1000000000000000E+01 -0.1000000000000000E+01 U7G01T11 TSLINEI 0 F 00006A50A357 +0.3393800000000000E+05 +0.3378200000000000E+05 +0.3378200000000000E+05

Two other new columns in SYSCOPY are JOBNAME and AUTHID, which together will enable the originating job of a utility to be traced. In Example 3-3, you can see the SYSCOPY entry for the originating LOAD REPLACE utility (ICTYPE = ‘R’) and the corresponding Inline COPY (ICTYPE=’F’). The Inline COPY will have STYPE=’R’.

Using inline copiesAlthough inline copies are recorded in SYSCOPY as full image copies with SHRLEVEL REFERENCE, they are different from normal image copies because they can contain more pages, as well as duplicate pages.

UTILITY ALONE + COPY With Inline COPY

DELTA ALONE (%)

DELTA + COPY (%)

CPUTime

ElapTime

CPUTime

ElapTime

CPUTime

ElapTime

CPUTime

ElapTime

CPUTime

ElapTime

LOAD 926 1067 955 1302 954 1086 +3 +1.8 -0.1 -16.6

REORG 829 1262 858 1497 856 1257 +3.3 -0.4 -0.2 -16.0

All times are in seconds.

The table illustrates CPU and elapsed time for:

� The utility itself (alone)� The utility and copy run separately (+ COPY)� The utility with Inline COPY (with Inline COPY)

Details are given in percent, comparing utility with Inline COPY figures to:

� The utility alone (Delta alone)� The utility and COPY run as two separate jobs (Delta +Copy)

Full Image copy on the same table space took 29 seconds CPU time and 235 seconds elapsed.

In the + COPY column, the full image copy numbers are added to the utility’s figures to show the entire duration for the two jobs


Inline copies produced during the LOAD phase of the LOAD utility can contain data that was removed afterwards during the INDEXVAL and ENFORCE phases (duplicates and RI violating rows).

During the LOG phase of REORG, incremental image copies are taken. These incremental copies are appended to the image copy data set created during the RELOAD phase. The full image copy, which results from a successful Online REORG, will probably contain duplicate pages, and therefore it will be bigger than other full image copies of this table space. The later pages will contain the most recent changes.

The DB2 RECOVER utility will take this into account and apply the most recent pages, but there are some restrictions regarding the stand-alone utilities DSN1COPY and DSN1PRNT.

Using inline copies with the RECOVER utilityAs previously stated, inline copies should not be used in a RECOVER TOCOPY scenario. The RECOVER utility handles the inline copies differently from “normal” copies.

In an Inline COPY taken by the REORG utility, there is a full image copy and multiple incremental copies in the data set. RECOVER process the data pages in order, and any duplicate pages are replaced as the utility progresses. As the incremental image copies contain any change pages, these are used to replace the corresponding pages in the table spaces. Thus, at the end the table, spaces are correctly recovered.

For an Inline COPY taken with the LOAD REPLACE LOG NO option, the image copy is taken during the LOAD phase of the utility. This means that any rows discarded during the INDEXVAL or ENFORCE stage are included in the copy. Recovery to a point-in-time applies the image copy and then has to process the discarded rows. See Figure 3-1 for an example timeline.

If the Inline COPY is taken at time T1, and then discard processing removes a row at time T2, RECOVER ensures that if a recovery to time T3 is required, then the row discarded is not present when the recovery is complete. The method of ensuring data integrity is that the discard processing is logged even when LOG NO is specified. This ensures that data integrity is not compromised. See Example 3-4 for an example of a LOAD. After this LOAD was completed, a REPORT RECOVERY was run to show that logging had taken place between the image copy and the QUIESCE point. This logging is due to the discard processing being logged; see Example 3-5. This report shows that there was logging between the copy being taken and the QUIESCE point.


Figure 3-1 Recovery scenario

Example 3-4 LOAD REPLACE LOG NO with Inline COPY and discards

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = TEMP0DSNU050I DSNUGUTC - LOAD REPLACE COPYDDN(SYSCOPY) SORTKEYS SORTDEVT SYSDA SORTNUM 3 LOG NO DSNU650I ( DSNURWI - INTO TABLE SYSADM.TABEMP DSNU350I ( DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE DB1.TABEMP NUMBER OF PAGES=16 AVERAGE PERCENT FREE SPACE PER PAGE = 25.87 PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:00:16 DSNU304I ( DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=9 FOR TABLE SYSADM.TABEMP DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=9 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:18 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=9 ELAPSED TIME=00:00:00 DSNU345I DSNURBXD - UNIQUE INDEX KEY DUPLICATES KEY FROM INPUT RECORD '1' LOADED AT RID X'0000000201', INDEX = SYSADM.TABI, TABLE = SYSADM.TABEMP, RECNO = 9, RID = X'0000000203' DSNU348I ( DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=1 FOR INDEX SYSADM.TABI PART 1 DSNU348I ( DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=4 FOR INDEX SYSADM.TABI PART 2 DSNU348I ( DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=1 FOR INDEX SYSADM.TABI PART 3 DSNU348I ( DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=1 FOR INDEX SYSADM.TABI PART 4 DSNU258I DSNURBXD - BUILD PHASE STATISTICS - NUMBER OF INDEXES=1 DSNU343I DSNURBXD - BUILD PHASE STATISTICS - 2 DUPLICATE KEY ERRORS ENCOUNTERED DSNU259I DSNURBXD - BUILD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU428I DSNURELD - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE DB1.TABEMP DSNU355I DSNURVIX - INDEXVAL PHASE STATISTICS - 2 DUPLICATE KEY ERRORS CORRECTED BY DELETING 2 DATA ROWS DSNU356I DSNURVIX - INDEXVAL PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU399I DSNURRIS - LOAD UTILITY ERROR SUMMARY REPORT ERROR INPUT ERROR TABLE FIELD/FANSET RELATED SEVERITY RECORD TYPE NAME NAME ERROR 1 1 DUP. KEY SYSADM.TABEMP SYSADM.TABI 0 1 9 DUP. KEY SYSADM.TABEMP SYSADM.TABI 1 DSNU396I DSNURREP - REPORT PHASE COMPLETE, ELAPSED TIME=00:00:010DSNU050I DSNUGUTC - QUIESCE TABLESPACE DB1.TABEMP DSNU477I ( DSNUQUIA - QUIESCE SUCCESSFUL FOR TABLESPACE DB1.TABEMP DSNU474I ( DSNUQUIA - QUIESCE AT RBA 000001F648C8 AND AT LRSN 000001F648C8 DSNU475I DSNUQUIB - QUIESCE UTILITY COMPLETE, ELAPSED TIME= 00:00:01 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

Recovery using an Inline COPY from LOAD REPLACE LOG NO

T0LOAD REPLACE LOG NOwith inline copy starts

T1LOAD phase endsimage copy taken

T2INDEXVAL/ENFORCEdiscard 1 row

T3Recovery to this pointrequired


Example 3-5 REPORT RECOVERY showing logging during discard processing

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = TEMP0DSNU050I DSNUGUTC - REPORT RECOVERY TABLESPACE DB1.TABEMP DSNUM ALL DSNU581I ( DSNUPREC - REPORT RECOVERY TABLESPACE DB1.TABEMP DSNU593I ( DSNUPREC - REPORT RECOVERY ENVIRONMENT RECORD: ' MINIMUM RBA: 000000000000 ' MAXIMUM RBA: FFFFFFFFFFFF ' MIGRATING RBA: 000000000000 DSNU582I ( DSNUPPCP - REPORT RECOVERY TABLESPACE DB1.TABEMP SYSCOPY ROWS TIMESTAMP = 2001-06-06-18.35.39.288433, IC TYPE = *S*, SHR LVL = , DSNUM = 0000, START LRSN =000001F433B3 DEV TYPE = , IC BACK = , STYPE = , FILE SEQ = 0000, PIT LRSN = 000000000000 LOW DSNUM = 0000, HIGH DSNUM = 0000 JOBNAME = SYSADM1 , AUTHID = SYSADM , COPYPAGESF = -1.0E+00 NPAGESF = -1.0E+00 , CPAGESF = -1.0E+00 DSNAME = ROY.ROYEMP , MEMBER NAME = TIMESTAMP = 2001-06-06-18.35.42.362598, IC TYPE = F , SHR LVL = R, DSNUM = 0000, START LRSN =000001F53BA6 DEV TYPE = 3380 , IC BACK = , STYPE = S, FILE SEQ = 0000, PIT LRSN = 000000000000 LOW DSNUM = 0000, HIGH DSNUM = 0000 JOBNAME = SYSADM1 , AUTHID = SYSADM , COPYPAGESF = 1.6E+01 NPAGESF = 4.8E+01 , CPAGESF = 4.8E+01 DSNAME = SYSADM.COPY.G0017V00 , MEMBER NAME = TIMESTAMP = 2001-06-06-18.35.45.133665, IC TYPE = *Q*, SHR LVL = , DSNUM = 0000, START LRSN =000001F648C8 DEV TYPE = , IC BACK = , STYPE = W, FILE SEQ = 0000, PIT LRSN = 000000000000 LOW DSNUM = 0000, HIGH DSNUM = 0000 JOBNAME = SYSADM1 , AUTHID = SYSADM , COPYPAGESF = -1.0E+00 NPAGESF = -1.0E+00 , CPAGESF = -1.0E+00 DSNAME = ROY.ROYEMP , MEMBER NAME = DSNU586I ( DSNUPSUM - REPORT RECOVERY TABLESPACE DB1.TABEMP SUMMARY DSNU588I ( DSNUPSUM - NO DATA TO BE REPORTED DSNU583I ( DSNUPPLR - SYSLGRNX ROWS FROM REPORT RECOVERY FOR TABLESPACE DB1.TABEMP UCDATE UCTIME START RBA STOP RBA START LRSN STOP LRSN PARTITION MEMBER ID 060601 18351187 000001F27B30 000001F2CCF9 000001F27B30 000001F2CCF9 0001 0000 060601 18351192 000001F29E35 000001F2CCF9 000001F29E35 000001F2CCF9 0002 0000 060601 18351195 000001F2B374 000001F2CCF9 000001F2B374 000001F2CCF9 0003 0000 060601 18351198 000001F2C850 000001F2CCF9 000001F2C850 000001F2CCF9 0004 0000 060601 18351259 000001F319AF 000001F35000 000001F319AF 000001F35000 0001 0000 060601 18351263 000001F31D23 000001F35000 000001F31D23 000001F35000 0002 0000 060601 18351264 000001F32094 000001F35000 000001F32094 000001F35000 0003 0000 060601 18351266 000001F323CC 000001F35000 000001F323CC 000001F35000 0004 0000 060601 18353047 000001F4866C 000001F4866C 000001F4866C 000001F4866C 0001 0000 060601 18353048 000001F4866C 000001F4866C 000001F4866C 000001F4866C 0002 0000 060601 18353049 000001F4866C 000001F4866C 000001F4866C 000001F4866C 0003 0000 060601 18353052 000001F4866C 000001F4866C 000001F4866C 000001F4866C 0004 0000 060601 18353937 000001F4E7D3 000001F4E7D3 000001F4E7D3 000001F4E7D3 0001 0000 060601 18353942 000001F4EA89 000001F4EA89 000001F4EA89 000001F4EA89 0002 0000 060601 18353946 000001F4ECFD 000001F4ECFD 000001F4ECFD 000001F4ECFD 0003 0000 060601 18353956 000001F4EF71 000001F4EF71 000001F4EF71 000001F4EF71 0004 0000 060601 18354201 000001F5320A 000001F5320A 000001F5320A 000001F5320A 0001 0000 060601 18354212 000001F5347E 000001F5347E 000001F5347E 000001F5347E 0002 0000 060601 18354221 000001F536F2 000001F536F2 000001F536F2 000001F536F2 0003 0000 060601 18354235 000001F53966 000001F53966 000001F53966 000001F53966 0004 0000 060601 18354281 000001F56736 000001F5AEE8 000001F56736 000001F5AEE8 0001 0000 DSNU584I ( DSNUPPBS - REPORT RECOVERY TABLESPACE DB1.TABEMP ARCHLOG1 BSDS VOLUMES DSNU588I ( DSNUPPBS - NO DATA TO BE REPORTED DSNU586I ( DSNUPSUM - REPORT RECOVERY TABLESPACE DB1.TABEMP SUMMARY DSNU588I ( DSNUPSUM - NO DATA TO BE REPORTED DSNU589I ( DSNUPREC - REPORT RECOVERY TABLESPACE DB1.TABEMP COMPLETE DSNU580I DSNUPORT - REPORT UTILITY COMPLETE - ELAPSED TIME=00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

Using inline copies with DSN1COPYAs described above, the Inline COPY produced during REORG is different from a normal full image copy. The generated Inline COPY is a collection of one full image copy and two or more incremental copies. Since the incremental copies are not merged with the full image copy created during the RELOAD phase, but appended at the end, there may well be a number of duplicate pages in this data set.


Should an Inline COPY be used with DSN1COPY, the error message shown in Example 3-6 would be generated.

Example 3-6 Using inline copies with DSN1COPY

DSN1999I START OF DSN1COPY FOR JOB DB2RES9D DSN1COPYDSN1989I DSN1COPY IS PROCESSED WITH THE FOLLOWING OPTIONS:NO CHECK/NO PRINT/ 4K/NO IMAGECOPY/NON-SEGMENT/NUMPARTS = 0/NO OBIDXLDSSIZE=DSN1998I INPUT DSNAME = DBSG.SC390DB1.TSE.UPD1 , SEQDSN1997I OUTPUT DSNAME = DB2V710S.DSNDBC.SC390DB1.SCDB1TSE.I0001.A001, VSAMDSN1984I UNEXPECTED PAGE NUMBER, EXPECTING: 0009CC0000 10000000 00000000 00000110 00002A08 000009CC 00000000 0000F000 00000000.... LINES ARE ALL ZERO.0FE0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000D5DSN1993I DSN1COPY TERMINATED, 00002508 PAGES PROCESSED

To avoid this problem, a new parameter has been added to DSN1COPY, INLCOPY, which specifies that the input is an Inline COPY.

Using inline copies with DSN1PRNTWhen you run DSN1PRNT on an Inline COPY, it will also have some difficulties because the pages are not stored in the correct sequence. If you use DSN1PRNT with the FORMAT option, the utility will dump all pages which are not in the correct sequence, see Example 3-7.

Example 3-7 Inline COPY with DSN1PRNT

00199701 20FF0000 00001980 010100F0 0000F000 00000000 0000F000 F000F000 ......0000F000 0000F000 0000F000 0000F000 00F000 ..0...DSN1984I UNEXPECTED PAGE NUMBER, EXPECTING: 000009CCPAGE: # 00000001 ---------------------------------------------------------------*** PAGE IN ERROR - NOT FORMATTABLE ***0000 10000000 00000000 00000110 00002A08 000009CC 00000000 0000F000 00000000.... LINES ARE ALL ZERO.0FE0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000D5DSN1984I UNEXPECTED PAGE NUMBER, EXPECTING: 000009CDPAGE: # 00000000 ---------------------------------------------------------------*** PAGE IN ERROR - NOT FORMATTABLE ***0000 10000000 00000000 00000018 0116000A 00000001 00C90000 00000000 000000010020 01010000 00000000 C4C2E2F1 00091000 00000000 00000000 0001001D 00000000

As with DSN1COPY, a new parameter has been added to DSN1PRNT, INLCOPY, which specifies that the input is an Inline COPY.

3.1.2 Inline RUNSTATSInline statistic gathering was introduced to the LOAD, REORG TABLESPACE, REORG INDEX, and REBUILD INDEX utilities with DB2 Version 6. This feature allows for the collection of statistics to be taken during the LOAD and BUILD phases of the utility, thereby eliminating the need for the execution of a separate RUNSTATS utility after the finishing of the utility.

The main objective of inline statistics is to reduce the total elapsed time of a REORG, LOAD or RECOVER/REBUILD index followed by statistics gathering. This is achieved by the elimination of additional scans of the data for gathering the statistics. By having the statistics gathering function within the utility job, administrative and scheduling procedures can be simplified. The collection is initiated by the inclusion of the STATISTICS keyword, and all RUNSTATS options are supported, including the HISTORY functionality. See Chapter 11, “Gathering statistics” on page 253, introduced in DB2 Version 7.


These are some restrictions for inline statistics gathering:

� LOAD with Inline statistics is only valid for REPLACE or RESUME NO options.

� A utility working on a part or part range of a table space cannot collect statistics on NPIs.

� Table space statistics collected during REORG TABLESPACE SHRLEVEL CHANGE do not reflect the changes to the originating table space after the UNLOAD phase. If a large number of updates are expected during the Log Apply phase, and accurate statistics are required, then it is better to run a RUNSTATS afterwards.

� Index statistics gathered during REORG INDEX SHRLEVEL CHANGE do not reflect the changes after the BUILD phase.

� Inline statistics cannot be used during a REORG of the directory or catalog table spaces containing links.

� Utilities restarted using RESTART(CURRENT) may not have inline statistics gathered. Using RESTART(PHASE) allows statistic gathering.

� Rows discarded during the LOAD process may or may not be reflected in the statistics. If a large number of discards are expected and accurate statistics are required, then the use of inline statistics is not recommended. A RUNSTATS utility should be run afterwards instead.

Performance measurementsThe LOAD and REORG utilities were run on a V6 system against a 5-million-row table with 10 partitions, each partition containing approximately 500,000 rows with 26 columns and a row length of 118 bytes. The table has two indexes, the partitioning index and a non-partitioning index. See DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351-00, for more details.

The LOAD and the REORG are run followed by the RUNSTATS utility and compared against the same utility with inline statistics specified.

LOAD with inline statisticsThe Table 3-2 summarizes the comparison of running the LOAD utility followed by the RUNSTATS against the LOAD utility with inline statistics.

Table 3-2 LOAD utility with inline statistics

As shown, there is a 34% reduction in the elapsed time using inline statistics. Notice also that the elapsed time of the LOADs are very similar, 603 seconds versus 597 seconds; this is due to the elimination of an additional scan of the data for the statistics collection. Also worthwhile noting is that this is not at the expense of a significant increase in CPU time, which is almost stable.

REORG with inline statisticsThe table Table 3-3 summarizes the comparison of running the REORG utility followed by RUNSTATS against the REORG utilizing inline statistics.

LOAD followed by RUNSTATS V6

LOAD with inline STATISTICS V6

Delta (%)

CPUTime

Elapsed Time

CPUTime

Elapsed Time

CPUTime

Elapsed Time

554 914 (597) 560 603 0% -34%

Note: All times are in seconds; bracketed figures are elapsed times for the LOAD utility


Table 3-3 REORG utility with inline statistics

As shown, a 23% reduction in elapsed time is achieved. Again, also worth noting is the similarity of the elapsed times for the REORGs, 993 seconds versus 966 seconds; this is again due to the elimination of additional scans of the data. As with LOAD, there is no significant difference in the amount of CPU that is consumed.

3.1.3 Enhanced DISPLAY UTILITY for Inline COPY and statisticsDISPLAY UTILITY has been enhanced to include details of the subphases representing the inline utilities. Example 3-8 shows the output from a DISPLAY UTILITY command while a LOAD utility is executing with both Inline COPY and inline statistics.

Example 3-8 DISPLAY UTILITY using Inline COPY and statistics

DSNU105I -DB2G DSNUGDIS - USERID = PAOLOR2 MEMBER = UTILID = LDCPSTAT PROCESSING UTILITY STATEMENT 1 UTILITY = LOAD PHASE = RELOAD COUNT = 1810654 NUMBER OF OBJECTS IN LIST = 1 LAST OBJECT STARTED = 1 STATUS = ACTIVE DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPY COUNT = 31486 DSNU111I -DB2G DSNUGDIS - SUBPHASE = RUNSTATS COUNT = 28672DSN9022I -DB2G DSNUGCCC '-DIS UTIL' NORMAL COMPLETION

3.1.4 Converting to inline executionsWhen converting existing utility jobs to start using inline executions, attention should be paid to the operational implications of introducing additional parallel subtasks. See 3.7, “Considerations for running parallel subtasks” on page 67 for more details.

3.2 COPY and RECOVER of a list of DB2 objects in parallelAnother type of parallelism in DB2 utilities is the execution of one type of utility on different DB2 objects in parallel subtasks. In this chapter we will discuss the COPY and RECOVER of a list of table spaces in parallel.

As previously possible with RECOVER, DB2 V6 introduced the ability to also COPY a group of objects in one COPY statement. This is done by specifying a LIST of table spaces, index spaces, or indexes after the COPY/RECOVER statement. With DB2 V7 you can also specify a previously defined listdef-name. One of the benefits of the list processing is to create a consistent backup for the objects in the list:

REORG followed by RUNSTATS V6

REORG with inline STATISTICS V6

Delta(%)

CPUTime

Elapsed Time

CPUTime

Elapsed Time

CPUTime

Elapsed Time

549 1283 (966) 541 993 0% -23%

Note: All times are in seconds; bracketed figures are elapsed times for the REORG utility


� With COPY SHRLEVEL REFERENCE, DB2 will first drain all writers on all objects of the list before starting the actual copying. All image copies will have the same RBA/LRSN value in SYSIBM.SYSCOPY, and the copy set results in a good recovery point if you need to PIT RECOVER these objects back to this RBA/LRSN.

� With COPY SHRLEVEL CHANGE, DB2 will drain the writers object by object, and the RBA/LRSN values will be different for each object. The set will not be consistent for PIT recovery.

Furthermore you can reduce the elapsed time for COPY and RECOVER when you specify a list of objects, by processing the objects in the list in parallel:

� You do this by specifying the number of objects to process in parallel with the new option PARALLEL (num-objects).

3.2.1 COPY in parallelAn example using COPY PARALLEL(3) with a list of five objects is illustrated in Figure 3-2.

Figure 3-2 Copy a list of DB2 objects in parallel

In this example, COPY creates 3 subtask sets. Each subtask set consists of two subtasks, one for reading the table/index and one for writing to the image copies. The objects 1, 2, and 3 are assigned to subtask set 1, 2, and 3, respectively. Object 3 finishes first, so object 4 is assigned to subtask set 3. Object 1 finishes next, and object 5 is assigned to subtask set 1. While the read subtask is putting pages into a pipe, the write subtask is pulling them out and writing them to the output data set.

See also 10.1.4, “COPY parallelism” on page 235.

3.2.2 RECOVER in parallelAn example using RECOVER PARALLEL(3) with a list of 5 objects is illustrated in Figure 3-3.

CopyDatasets

CopyDatasets

Object 1Object 2Object 3Object 4Object 5

Read Subtask 1Object

Spaces

Read Subtask 2

Read Subtask 3

W rite Subtask 1

W rite Subtask 2

W rite Subtask 3

Pipe

ObjectSpaces

ObjectSpaces

PipePipe

Copy Parallel (3)

CopyDatasets


Figure 3-3 Recovering a list of DB2 objects in parallel

In this example, RECOVER creates 3 subtask sets. A subtask set consists of two subtasks, one for reading the image copies and one for writing to the object space. Objects 1, 2, and 3 are assigned to subtask set 1, 2, and 3, respectively. Object 3 finishes first, so object 4 is assigned to subtask set 3. Then, object 1 finishes next and object 5 is assigned to subtask set 1. While the read subtask is putting pages into the pipe, the write subtask is pulling them out and writing them to the object space(s).

Parallelism is only achieved in the RESTORE phase. The Log Apply processing does not begin until all objects have been restored. During the Log Apply processing, the log will only be scanned once, and recovery will use the fast Log Apply feature, as explained in 9.5, “Fast Log Apply” on page 217.

See also 9.2, “Parallel RECOVER” on page 211.

3.2.3 Restrictions for parallel COPY and RECOVERYou should be aware of the following considerations:

� Parallelism is only enabled by specifying the PARALLEL keyword. If you do not specify the PARALLEL keyword, parallelism is not enabled. If you do not specify num-objects or you specify 0, DB2 will determine the optimal number of objects to process in parallel. If you specify a number bigger than 1, this is a maximum value. DB2 may reduce the number of objects processed in parallel if there is a virtual storage constraint in the utility batch address space.

� Parallelism support is only provided for copying to, and restoring from, DASD devices.

� COPY to tape is single-threaded. If a tape volume is encountered in the list, processing for the remainder of the list waits until the object using a tape has been completely copied or restored. For example, assume there are six objects in a COPY list, PARALLEL(4) is requested, the fourth object COPYDDN is a tape and the rest use DASD. Objects 1, 2, 3, and 4 begin to be processed. Suppose object 2 finishes first. Since object 4 is copied to tape, DB2 cannot begin processing object 5 until the copy of object 4 is complete. Once object 4 finishes, objects 5 and 6 can be processed in parallel.

R ecover P aralle l (3 )

O bject 1O bject 2O bject 3O bject 4O bject 5

R ead S ubtask 1

O bjectS p aces

R ead S ubtask 2

R ead S ubtask 3

W rite S ubtask 1

W rite S ubtask 2

W rite Subtask 3

P ipe

O b jectS paces

O b jectS p aces

P ipeP ipe

C o p yD atasets

C o pyD atasets

C o pyD atasets


3.2.4 Performance measurementsThe COPY and RECOVER utilities were run on a V6 system against a 5-million-row table with 10 partitions, each partition containing approximately 500,000 rows with 26 columns and a row length of 118 bytes. See DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351-00 for more details.

COPY with and without keyword PARALLELA comparison of using COPY with and without PARALLEL in a list is summarized in Table 3-4.

Table 3-4 COPY TABLESPACE without and with PARALLEL

There is a 75% elapsed time reduction when using the keyword PARALLEL. The number of objects that are copied in parallel was 10. Due to I/O contention on the underlying table spaces and image copy data sets, a 10 times improvement of elapsed times was not achieved. There was not a significant difference in CPU times.

RECOVER with and without keyword PARALLELA comparison running the RECOVER utility specifying a list of objects, with and without the PARALLEL keyword, is summarized in. There are no update log records to apply in both cases.

Table 3-5 RECOVER TABLESPACE without and with PARALLEL

A 71% elapsed time reduction is achieved by using the PARALLEL keyword. All 10 partitions were restored in parallel. Due to I/O contention on the underlying table spaces and image copy data sets, a 10 times improvement of elapsed times was not achieved. There was not a significant difference in CPU times.

3.2.5 Converting COPY/RECOVER jobs to use parallelismWhen converting existing COPY/RECOVER utility jobs to start using lists and parallel executions, attention should be paid to the operational implications of introducing additional parallel subtasks. See 3.7, “Considerations for running parallel subtasks” on page 67 for more details.

COPY without PARALLEL V6

COPY with PARALLEL V6 Delta(%)

CPUTime

Elapsed Time

CPUTime

Elapsed Time

CPUTime

Elapsed Time

11 234 12 58 0% -75%

Note: All times are in seconds.

RECOVER without PARALLEL V6

RECOVER WITH PARALLEL V6

Delta(%)

CPUTime

Elapsed Time

CPUTime

Elapsed Time

CPUTime

Elapsed Time

14 249 14 71 0% -71%



3.3 LOAD and REORG Parallel Index BuildAnother example of parallelism of one type of utility on different DB2 objects in parallel subtasks is rebuilding the different indexes of a table space in parallel during LOAD or REORG TABLESPACE. This is called Parallel Index Build (PIB.)

Note: In this chapter, REORG means REORG TABLESPACE unless otherwise specified.

PIB was introduced in DB2 V6 with the use of the SORTKEYS option. Because SORTKEYS already existed in DB2 V5, we will first try to explain the difference between SORTKEYS in DB2 V5 and DB2 V6.

3.3.1 SORTKEYS with DB2 V5DB2 Version 5 introduced the concept of SORTKEYS to eliminate the need for multiple I/Os to access the keys that are needed to build indexes. The index keys are passed in storage to a single sort and build subtask, which resulted in the indexes being built serially. This is illustrated in Figure 3-4 for REORG. The SORT and BUILD are two subsequent phases.

Figure 3-4 SORTKEYS with DB2 V5

The table space can be partitioned or non-partitioned. In case of a partitioned table space, the building of the partitioned index during LOAD or REORG is not different from the building of the other indexes. It is done serially with the other indexes.

SORTKEYS eliminates the need for intermediate workfiles (SYSUT1or SORTOUT) when building the indexes, by passing the extracted keys in memory to the sort process. However, SYSUT1 and SORTOUT are still required for the LOAD utility, which uses them for processing errors detected during the RELOAD, VALIDATE, or ENFORCE phase, and in passing foreign keys from the SORT phase to the ENFORCE phase.

Tab le Space

Index 2

Index 1

Index 3

Bu ildS ort

keys

K eys passed through S ort ExitsM ost I/O o f keys e lim inated

Indexes bu iltseria lly

U nload

(R EO R G )

(R e)load


3.3.2 SORTKEYS with DB2 V6 and DB2 V7DB2 Version 6 introduced multiple pairs of sort and build subtasks so that indexes are built in parallel, thereby further improving the elapsed time of the LOAD and REORG utilities. This is illustrated in Figure 3-5 for REORG.

All this is done in the new SORTBLD phase, which replaces the SORT and BUILD phases. The sort subtask sorts the extracted keys, while the build subtask builds the index. This is also done partially in parallel, because the index build starts as soon as the sort subtask passes the first sorted keys. Moreover the SYSUT1 and SORTOUT work data sets are not used during the SORTBLD because the key/rid pairs are now directly piped in memory between the subtasks, eliminating considerable I/O processing.

Figure 3-5 SORTKEYS with DB2 V6, V7, and Parallel Index Build

The table space can be partitioned or non-partitioned. In case of a partitioned table space, the building of the partitioned index during LOAD or REORG is not different from the building of the other indexes. Building of the partitioned index is always done on the entire index level, even if the table space has no NPIs. This in contrast with the REBUILD INDEX reported in 3.4.1, “Rebuilding the partitioning index of a partitioned table space using PIB” on page 58.

Each sort subtask uses its own sort work data set and its own message data set (not shown). If inline statistics are also collected, there is a separate RUNSTATS subtask for each build task that collects the sorted keys and updates the catalog in parallel. When collecting statistics on the table space, an additional subtask runs in parallel with the RELOAD phase.

LOAD and REORG Parallel Index Build is automatically selected if the following conditions are true:

� For LOAD, SORTKEYS n must be specified, n being an estimation of the total number of keys to be sorted. If you specify n equal to zero, PIB will not be activated.

� For REORG, SORTKEYS must be specified.

There are different ways to influence the number of parallel subtasks. See 3.7, “Considerations for running parallel subtasks” on page 67 for more details.

Table Space

Index 2

Index 1

Index 3

BuildSort

keys

M ultip le Sort/Build task pairs provide para lle lism

Indexes builtin para lle l

Unload

(REO RG )

(Re)load

SO RTBLD


If fewer subtasks are available than there are indexes to be built, then the indexes are assigned to the subtask in the order that they were created. The order of creation can be determined from the SYSIBM.SYSINDEXES catalog table, column CREATEDS, after Version 5. Prior to Version 5, the OBIDs can be used to determine the order.

Parallelism during BUILD2 phase (DB2 V7 only) When doing an Online REORG of a single partition or a partition range of a partitioned table space, the actual updating of the NPI is done in a separate phase called the BUILD2 phase. See also 7.6.6, “BUILD2” on page 175.

DB2 V7 improves the performance of the BUILD2 phase by updating the NPIs in parallel. Optimally, the number of subtasks will be equivalent to the number of NPIs, with one subtask handling all updates for all logical partitions on the same NPI.

This parallelism is only activated when specifying SORTKEYS for REORG SHRLEVEL REFERENCE. For SHRLEVEL CHANGE, it is always activated.

Even with one NPI, a performance improvement can be noticed in V7, because of internal design changes in the handling of the log writes.

3.3.3 Performance measurementsThe following measurements show performance improvements for:

� The SORTKEYS option, between DB2 V5 and DB2 V6� The BUILD2 phase, between DB2 V6 and DB2 V7

Comparing SORTKEYS between DB2 V5 and DB2 V6 The following LOAD and REORG utilities were run on a DB2 system against a 5-million-row table with 10 partitions, each partition containing approximately 500,000 rows with 26 columns and a row length of 118 bytes. See DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351-00 for more details.

The measurements taken compare LOAD and REORG SHRLEVEL NONE with Parallel Index Build using 2, 3, and 6 indexes in DB2 Version 6 against DB2 Version 5 (level SUP9904).

LOAD with sortkeysThe measured results for this case are shown in Table 3-6, Table 3-7, and Table 3-8.

Table 3-6 LOAD utility with 2 indexes

Important: Although you cannot specify SORTKEYS for REORG SHRLEVEL CHANGE, it always operates as if the SORTKEYS parameter was specified. See also 7.6.3, “SORT and BUILD as compared to SORTBLD” on page 170.

DB2 Version 5 with SORTKEYS


Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

248 354 261 345 +5% -3%





In the example with two indexes (see Table 3-6), there is a reduction of 3%. This increases to 18% with three indexes (see Table 3-7), and 36% with six indexes (see Table 3-8).

REORG SHRLEVEL NONE with sortkeysThe measured improvements with the REORG utility are shown in Table 3-9, Table 3-10, and Table 3-11.

Table 3-9 REORG SHRLEVEL NONE utility with 2 indexes





Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

323 415 344 339 +7% -18%




Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

559 571 615 363 +10% -36%




Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

243 773 242 773 0% 0%




Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

316 842 324 779 +3% -8%




Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

547 993 601 809 +10% -19%



As with LOAD, it can be seen that the more indexes are defined, the greater the savings, although with two indexes (see Table 3-9), there are no savings in elapsed time. With three indexes (see Table 3-10), a saving of 8% can be achieved. This rises to 19% when six indexes are created (see Table 3-11).

From the figures from both the LOAD and REORG utilities, it can be deduced that the more indexes are created, the greater the savings in elapsed time. This is at the expense of greater CPU, due to the increase in subtasks processing the indexes. In the cases above, the number of parallel tasks was equivalent to twice the number of indexes (one for sort and one for build).

Comparing BUILD2 phase between DB2 V6 and DB2 V7The following REORG SHRLEVEL REFERENCE utilities were run on a DB2 system against a 20-million-row table with 20 partitions, each partition containing approximately 1 million rows with 26 columns and a row length of 119 bytes. The table space contains one NPI. See “DB2 for z/OS and OS/390 Version 7 Performance Topics SG24-6129-00” for more details.

Table 3-12 summarizes the comparison of the BUILD2 phase between DB2 Version 6 and DB2 Version 7 for the REORG utility using SHRLEVEL REFERENCE, SORTKEYS, NOSYSREC and Inline COPY, when reorganizing 3 partitions.

Table 3-12 REORG 3 partitions with 1 NPI

Although no parallelism was used, running REORG against a range of partitions, with only one NPI in V7, shows improvements of 33% CPU time and 36% elapsed time, because of the different handling of log writes.

The number of NPIs was increased to five and the same test was run. The results are shown in Table 3-13.

Table 3-13 REORG 3 partitions with 5 NPI

The BUILD2 phase now returns an 83% saving in elapsed time when reorganizing with 5 NPIs. The relative increase in Delta CPU time, from -33% to -29%, is due to the managing of the five parallel subtasks used to build the NPIs in parallel. From this, it can be concluded that the greater the number of NPIs, the greater the savings in elapsed time, although this is at the cost of CPU time.

DB2 V6 DB2 V7 Delta (%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

BUILD2 166 176 111 112 -33% -36%


DB2 V6 DB2 V7 Delta (%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

BUILD2 825 874 587 152 -29% -83%



3.3.4 Dropping NPIs before REORGBecause REBUILD INDEX can unload all the keys in parallel when rebuilding a single NPI (see 3.4, “REBUILD INDEX Parallel Index Build” on page 57), dropping the NPIs before the REORG and rebuilding them afterwards with REBUILD INDEX can still be quicker than using SORTKEYS in DB2 Version 6 and DB2 Version 7, when reorganizing large partitioned table spaces. However, you have to consider that:

� The elapsed time of a REORG can become less important by using Online REORG (SHRLEVEL REFERENCE or CHANGE), allowing concurrent access by applications.

� This is a far more complex process than submitting a single REORG job with the SORTKEYS option.

� Dropping and recreating indexes may require extra rebinds and further unavailability.

3.3.5 Dropping NPIs before LOADDB2 V7 enables loading the partitions of a partitioned table in parallel (see 3.5, “Partition parallelism with the LOAD Utility” on page 62). Because of the subsequent performance improvements provided by the new parallelism, dropping the NPIs before the LOAD of the entire table space and rebuilding them afterwards with REBUILD INDEX is not recommended in DB2 V7.

3.3.6 Converting LOAD/REORG jobs to use SORTKEYSWhen adding SORTKEYS to existing LOAD/REORG utility jobs to start using Parallel Index Build (PIB), attention should be paid to the operational implications of introducing additional parallel subtasks. See 3.7, “Considerations for running parallel subtasks” on page 67 for more details.

For examples of LOAD jobs using PIB, see 6.2, “Parallel Index Build” on page 111.

3.3.7 REORG INDEXSORTKEYS can only be used with REORG TABLESPACE. It cannot be specified with REORG INDEX. Reorganizing an index will always unload and rebuild the index serially. Parallelism will not be invoked.

3.4 REBUILD INDEX Parallel Index BuildSince DB2 V6, Parallel Index Build (PIB) can also be used to reduce the elapsed time taken by the REBUILD INDEX utility. REBUILD INDEX is the new name for the utility that was known as RECOVER INDEX in previous versions of DB2. PIB is also activated by using the SORTKEYS option.

The method used by PIB is similar to the methods used by the LOAD and REORG utilities that are described in 3.3, “LOAD and REORG Parallel Index Build” on page 52.

However, the main difference is that REBUILD INDEX can also execute the UNLOAD phase in parallel for partitioned table spaces. The utility starts multiple subtasks, ideally one per partition, to unload the data; these are then passed to individual SORT subtasks and BUILD subtasks.


We also need to distinguish between the individual rebuild of a partitioned index, the individual rebuild of a non-partitioned index (NPI), and the rebuild of all indexes of a partitioned and non-partitioned table space.

3.4.1 Rebuilding the partitioning index of a partitioned table space using PIBThis case applies if we only rebuild the partitioning index of a partitioned table space, or if we do a REBUILD INDEX(ALL) and the partitioned table space contains no NPIs. In this case, if we specify SORTKEYS, the partitions will be unloaded in parallel, and the individual parts of the partitioned index will be built in parallel during the SORTBLD phase.

Figure 3-6 shows three unload subtasks piping the index keys to three sort subtasks and then onto three build subtasks. In this example, inline statistics were not gathered (otherwise there would be three more subtasks).

Figure 3-6 Rebuilding a partitioned index with PIB using SORTKEYS

3.4.2 Rebuilding a non-partitioning index using PIBThis case applies if we rebuild a non-partitioning index of a partitioned table space. In this case, if we specify SORTKEYS, the partitions will be unloaded in parallel, and the NPI will be built by a subsequent MERGE and BUILD task.

When building a non-partitioning index, the unload and sort run in parallel, and the sort subtasks pipe the data to a single merge subtask which in turn pipes the data to a single build task. If inline statistics are collected, then there will be a single statistics subtask. This is shown in Figure 3-7.

Table Space

keys

Partition 1

Partition 2

Partition 3

Indexes builtin paralle l

Index

Index

Index

partition 1

partition 2

partition 3

UNLO AD

BuildSort

SO RTBLD


Figure 3-7 Rebuilding a non-partitioned index with PIB using SORTKEYS

With this enhancement, it is no longer required to submit separate REBUILD INDEX PART n jobs to unload the keys, then merge the SYSUT1 outputs in order to create a large NPI. DB2 can now easily create a large NPI using the REBUILD INDEX.

3.4.3 Rebuilding all indexes of a partitioned table spaceThis case applies if we do a REBUILD INDEX(ALL) on a partitioned table space that contains at least one NPI. In this case, if we specify SORTKEYS, the table space will be unloaded in parallel by as many subtasks as indexes to be built, and the indexes will be built in parallel during the SORTBLD phase.

Unlike the case where the partitioned table space has no NPIs, the building of the partitioned index is done on the entire index level. This because building the parts of the partitioned index in parallel would not decrease the total elapsed time of the utility job.

Figure 3-8 shows three unload subtasks piping the index keys to three sort subtasks and then onto three build subtasks. In this example, inline statistics were not gathered (otherwise there would be three more subtasks).

Tab le S pace

keys

P artition 1

P artition 2

P artition 3

S o rt

U N LO A D

Index

M erge B u ild


Figure 3-8 Rebuilding all indexes of a partitioned table space with PIB using SORTKEYS

You can force DB2 to build the parts of the partitioned index in parallel as well, by specifying in the REBUILD INDEX command a list of all the individual index parts with the PART keyword, or by using a list defined in a LISTDEF with the PARTLEVEL keyword. However, this approach is not recommended, as it will increase the CPU time and not reduce the elapsed time of the whole utility job.

3.4.4 Rebuilding all indexes of a non-partitioned table spaceThis case applies if we do a REBUILD INDEX(ALL) on a non-partitioned table space that contains at least two indexes. In this case, if we specify SORTKEYS, the indexes will be built in parallel during the SORTBLD phase.

Figure 3-9 shows three sort subtasks and three build subtasks for building three indexes in parallel. In this example, inline statistics were not gathered (otherwise there would be three more subtasks).

Tab le S pace

keys

P artition 1

P artition 2

P artition 3

Indexes bu iltin para lle l

Index 2

Index 1

Index 3

P I

N P I 1

N P I 2

U N LO A D

B uildS ort

S O R TB LD


Figure 3-9 Rebuilding all indexes of a non-partitioned table space with PIB using SORTKEYS

3.4.5 Performance measurementsThe following REBUILD INDEX utilities were run on a DB2 V6 system against a 5-million-row table with 10 partitions, each partition containing approximately 500,000 rows with 26 columns and a row length of 118 bytes. The table space has a partitioned index and one NPI. See DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351-00, for more details.

Rebuilding a partitioned indexTable 3-14 shows the measured results.

Table 3-14 REBUILD of a partitioned index

Using SORTKEYS shows an 81% reduction in elapsed time. There are 30 parallel tasks with SORTKEYS, 10 for UNLOAD, 10 for SORT and 10 for BUILD, used to process each index partition in parallel.

Table Space

Index 2

Index 1

Index 3

BuildSort

keys

M ultip le Sort/Build task pairs provide paralle lism

Indexes builtin paralle l

Unload

SO RTBLD

DB2 Version 6 without SORTKEYS


Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

73 365 80 71 +10% -81%



Rebuilding a non-partitioned indexTable 3-15 shows the measured results.

Table 3-15 REBUILD of a non-partitioned index

Using SORTKEYS shows a 69% reduction in elapsed time. There are 22 parallel tasks with SORTKEYS, 10 for UNLOAD, 10 for SORT, 1 for MERGE, and 1 for BUILD.

Rebuilding two indexesTable 3-16 shows the measured results for the building of the partitioned index and the NPI using REBUILD INDEX(ALL).

Table 3-16 REBUILD INDEX(ALL)

Using SORTKEYS shows a 66% reduction in elapsed time. There were only 6 parallel tasks used with SORTKEYS, 2 for UNLOAD, 2 for SORT, and 2 for BUILD.

3.4.6 Enabling PIB with REBUILD INDEXThe parallelism of the REBUILD INDEX will automatically be used if SORTKEYS is specified.


3.5 Partition parallelism with the LOAD UtilityAs already explained in 3.3, “LOAD and REORG Parallel Index Build” on page 52, since DB2 V6, LOAD benefits from the use of PIB for building the indexes in parallel during LOAD. But with DB2 V7, a new form of parallelism has been added when loading partitioned table spaces.

Partition parallelism with the LOAD Utility enables the LOAD utility to load partitions in parallel. This can be used with or without PIB (SORTKEYS n). Both of these features reduce the elapsed time of the LOAD utility.



Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

76 300 102 92 +34% -69%




Delta(%)

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

156 504 163 173 +4% -66%



To activate parallel partition loading, the following rules apply:

� Separate input data sets have to be supplied per partition.

� The syntax of the LOAD has to specify each partition to be loaded via the INTO TABLE option; see Figure 3-10.

The need to supply separate input files is made easier with the TEMPLATE functionality described in Chapter 2, “Wildcarding and Templates” on page 17, and it is recommended that this feature be used for loading partitions in parallel.

Figure 3-10 LOAD syntax for activating parallel partition loading

3.5.1 LOAD partition parallelism without PIBEven without the SORTKEYS n option being specified, LOAD will launch multiple RELOAD subtasks, optimally one for each partition, and in correspondence of each input data set. Each of these RELOAD subtasks reads the records from its input data set and loads into its partition. The keys for all indexes are extracted and, paired together with the RIDs of the just loaded records, they are written to the common SYSUT1 data set. After sorting, the normal BUILD phase is performed serially loading the indexes. This in shown in Figure 3-11.

The number of RELOAD tasks is determined by:

� Number of CPUs� Available virtual storage� Available number of DB2 threads

The existing message DSNU397I explains the constraints on number of tasks. Furthermore, there is a new message:

DSNU364I PARTITIONS WILL BE LOADED IN PARALLEL, NUMBER OF TASKS = nnn

See 3.7, “Considerations for running parallel subtasks” on page 67 for more considerations when introducing multiple parallel subtasks.

LOAD - syntax for parallel partition loading

LOADINTO TABLE tb-name PART 1 INDDN inddn1 DISCARDDN

discddn1INTO TABLE tb-name PART 2 INDDN inddn2 DISCARDDN

discddn2...INTO TABLE tb-name PART n INDDN inddnn DISCARDDN

discddnn

DSNU,DB2I, DSNUTILS support via templates


Figure 3-11 Partition parallel LOAD without PIB

For examples of LOAD jobs using partition parallelism without PIB, see 6.3.1, “Partition parallelism without Parallel Index Build” on page 119.

3.5.2 LOAD partition parallelism with PIBAs already explained, PIB is activated by specifying the SORTKEYS n option, n being an estimation of the number of keys to be sorted. If you specify n equals zero, PIB will not be activated.

The accelerating effects of specifying SORTKEYS n are as follows:

� The SORT, the BUILD, and optionally the inline STATISTICS phases are performed partially in parallel. This way, the elapsed time of a LOAD job can be considerably reduced.

� Considerable I/O processing is eliminated by not using SYSUT1 and SORTOUT.

By using parallel partition LOAD and PIB together, DB2 will initiate subtasks for the LOAD phase and the SORTBLD phase. For loading partitions in parallel, the optimum is to initiate one subtask per partition to be loaded and two for each index build (sort and build), although this is not always possible due to constraints, such as virtual memory, DB2 threads available, and processors available. See 3.7, “Considerations for running parallel subtasks” on page 67 for more details.

SYSREC1 Part 1

Part 2

Part 3

Part 4

SYSREC2

SYSREC3

PI

Error/Map

NPI1

NPI2

SYSUT1

RELOAD

RELOAD

RELOAD

RELOAD

SORT BUILD

BUILD

SORTWKnn

BUILD

key/RID pairs

SYSREC4

SORTOUT


Figure 3-12 Partition parallel LOAD with PIB

For examples of LOAD jobs using partition parallelism with PIB, see 6.3.2, “Partition parallelism with Parallel Index Build” on page 123.

3.5.3 Performance measurementsThe following LOAD utilities were run on a DB2 V7 system against a 50-million-row table with 20 partitions, each partition containing approximately 2,5 million rows with 26 columns and a row length of 119 bytes. See DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-6129-00, for more details.

Table 3-17 shows the results of loading the table utilizing parallel partition loading only, but not using PIB, because the SORTKEYS n clause was omitted.

Table 3-17 LOAD partition parallelism without PIB

SYSREC1 Part 1

Part 2

Part 3

Part 4

SYSREC2

SYSREC3

SYSREC4

PISORTBUILD

SORTBUILD

SORTBUILD

RELOAD

RELOAD

RELOAD

RELOAD

SW01WKxx

SW02WKxx

SW03WKxx

SW01WKnn

SW02WKnn

SW03WKnn

Error/Map

NPI1

NPI2

SORTBLD

SORTBLD

SORTBLD

LOAD TABLE PARALLEL PARTITION LOAD

Delta (%)

No. ofIndexes

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

1 599 1087 735 734 +22.7% -32.5%

2 1052 1785 1106 1247 +5.1% -30.1%

3 1411 2187 1472 1730 +4.3% -20.9%

6 2459 3650 2573 3480 +4.6% -4.7%



When loading partitions in parallel without building NPIs in parallel (without specifying SORTKEYS n), if there is only one index, the CPU time degrades up to 22.7% and the elapsed time improves up to 32.5%. When there are more then one index on a partition table, the CPU overhead is lower, even though managing the parallel subtasks for loading partitions in parallel contributes to an increase in CPU time.

Table 3-18 shows the comparison of running LOAD utility for the whole table with the LOAD utility running in parallel for all partitions and building NPIs in parallel by specifying SORTKEYS n with an estimated value greater than 0.

For tables without NPIs, the parallel partition LOAD with SORTKEYS n improves up to 48.2% in elapsed time compared to loading the whole table with SORTKEYS n, and with only 9.1% CPU time degradation. Managing the parallel subtasks to load partitions in parallel contributes to an increase in CPU time.

Table 3-18 LOAD partition parallelism with PIB

The use of SORTKEYS n and parallel partitions leads to great improvements in the performance of the LOAD utility, and it is strongly recommended that these feature are used where there is more than one NPI index on the table. The number of NPIs is very important in the use of SORTKEYS n and it is this factor that enables the LOAD utility to drive down the cost of the utility

See 3.7, “Considerations for running parallel subtasks” on page 67 for other considerations.

3.6 Partition parallelism with the UNLOAD UtilityAs explained in Chapter 8, “Unloading data” on page 191, DB2 V7 introduced the new UNLOAD utility with a better performance and functionality then REORG UNLOAD EXTERNAL.

When unloading a partitioned table space into individual data sets (one per partition), the UNLOAD utility automatically activates multiple parallel subtasks. Optimally, the number of subtasks will be equal to the number of partitions to be unloaded, but most of the times it will be limited to the number of CPUs available.


The parallelism of the UNLOAD utility is enabled by using the PART option in the UNLOAD statement or using the PARTLEVEL option in the LISTDEF command. See 8.4.2, “Unload by partition and parallelism” on page 200 for more details.

LOAD TABLE PARALLEL PARTITION LOAD

Delta (%)

No. ofIndexes

CPUTime

ElapsedTime

CPUTime

ElapsedTime

CPUTime

ElapsedTime

1 656 1393 716 722 +9.1% -48.2%

2 1036 1404 1175 982 +13.4% -30.1%

3 1398 1737 1573 998 +12.5% -42.5%

6 2536 2417 2835 1385 +11.8% -42.7%



3.6.1 Performance measurementsThe following UNLOAD utilities were run on a DB2 V7 system against a 16 columns table with approximately 6 million rows per partition. The processor is a IBM 7-way G6 with 7 CPUs available. See “DB2 for z/OS and OS/390 Version 7 Performance Topics SG24-6129-00” for more details.

Figure 3-13 Unloading a partitioned table space in parallel

The elapsed times for unloading 1 to 4 partitions are the same. There is no extra cost in elapsed time for unloading the 3 extra partitions.

The elapsed time only increase a little bit, when unloading 7 partitions in parallel. The increase in elapsed time is due to handling more subtasks and partitions. The elapsed times for unloading 1 or 7 partitions in parallel are almost the same.

The elapsed time for unloading 14 partitions in parallel shows that the elapsed time is more than double, because the UNLOAD utility could only start 7 subtasks, which have to unload more than one partition each, and the overhead to handle more partitions than subtasks increased too.

3.7 Considerations for running parallel subtasks The actual number of parallel subtasks scheduled by DB2 in a utility job can be less than the maximum expected number. DB2 determines the degree of parallelism, that is, the number of parallel subtasks based on the following factors:

� Maximum number of subtasks to be expected� Available virtual storage in the batch region� Number of CPUs available� Available number of DB2 threads and connections� Number of user-allocated sort work data sets, if not using dynamic allocation� Number of user-allocated sort message data sets, if UTPRINT not allocated to SYSOUT

86 86 89

191

1 4 7 14Number of partitions

0

50

100

150

200

250

Elapsed time (sec)


Each sort subtask uses its own sort work data sets and its own message data set. DB2 cannot use more subtasks than the number of data sets allocated. Manually allocating the sort work data sets and/or the sort message data sets is a way to control the number of subtasks:

� Allocate your own sort work data sets for each sort subtask by specifying them in the JCL with ddnames SWnnWKmm, where nn is the number of subtasks and mm is the number allocated to each subtask.

� Allocate your own message data sets for each sort subtask by specifying them in the JCL with ddnames UTPRINnn, where nn is the number of subtasks.

If you specify both sort workfile DD statements and UTPRINT data sets, the number of sort subtasks equals the smallest number of data sets specified.

With many utilities now running parallel subtasks, the following items are worth noting:

� Ensure that IDBACK is large enough to accommodate all the parallel threads that are required to run concurrently. The number of threads required per utility varies from utility to utility, also between phases within the same utility, and is dependent upon the options specified. It is recommended to increase IDBACK to a number that is unlikely to be reached to avoid any failures. If this is problematic, then a scheduling product, such as Tivoli Operation Planning and Control (OPC), could also be used to control the number of jobs running concurrently, and thus the number of threads within DB2. CTHREAD may also need increasing in line with IDBACK.

� Virtual storage requirements will rise dependent upon the number of subtasks being run by the utility. Ensure that the REGION parameter is large enough.

� The amount of real storage available will have to be considered. DB2 will limit the number of subtasks dependent upon available virtual storage. If REGION=0M is coded on the JOBCARD, then the subtasks can have as much virtual storage as is required. With tens of subtasks not uncommon, this could cause problems with real storage exhaustion (excessive paging).

� Increasing the number of parallel subtasks will also increase the CPU consumption of the utility jobs.

� Sufficient DASD will have to be available to support the multiple sort and work data sets.

� Using templates for the sort and work data sets is the best way to enable maximum parallelism.


Chapter 4. Triggering and controlling executions

Triggering utilities on DB2 object is a two-step process:

� The first step consists of identifying one or more statistical values in the DB2 catalog related to the DB2 object.

� The second step consists of utilizing these statistical values, directly or indirectly, to decide on the activation of the relevant utility function on the DB2 object.

Every DB2 site has one method or another to trigger their utilities on DB2 objects during their “housekeeping” window. Some of these trigger limits are within the utility control statements, while others are external to the utilities. In most cases, either ISV products or in-house written programs are used to query the catalog and generate the utility statements.

In this chapter we cover these features:

� New REORG limits with V6� New values with V7� Trends from historical statistics� Real-Time Statistics� New thresholds from new stored procedures� DSNUTILS and control center� DB2Admin and automation tools

We describe the statistical fields within the catalog in detail and show how these fields can be used to determine the limits for utilities. We also cover the new history catalog tables and their use in calculating trends in growth in DB2 objects, such as determining the compression and decompression dictionary build.

Two sections deal with stored procedures and DB2 tools. In these sections, we show examples of how the stored procedure and tools can be used to control the utility functions on DB2 objects.

4


4.1 Triggering limits prior to DB2 V7REORG INDEX and REORG TABLESPACE introduced utility triggering options in DB2 V6 that can be used to trigger the REORG utility when the limits are exceeded. In DB2 V5, the CHANGELIMIT option of COPY was introduced, which can be used to trigger full or incremental image copies. We will discuss these triggering limits in detail.

4.1.1 REORG INDEXA new option LEAFDISTLIMIT was introduced in DB2 V6 and continued to V7. LEAFDIST is a column in SYSINDEXPART and the new V7 catalog table SYSINDEXPART_HIST. LEAFDIST indicates the average number of pages that are between successive leaf pages in the index. Leaf pages can have page gaps whenever index keys are deleted, or when there are index leaf page splits caused by an insert that cannot fit onto a full page. If the key cannot fit on the page, DB2 moves half of the index entries onto a new page, which might be far away from the "home" page. Figure 4-1 provides a simple illustration of the computation of LEAFDIST, assuming the number of pages to be 10000.

Figure 4-1 LEAFDIST distortion

� There is a navigation from root page to leaf page 8 which extends to leaf page 9998. This gives a physical page gap of 9990.

� Similarly, the page gaps are obtained for other leaf page navigations, giving page gaps of 9989, 9988, and 9989.

� LEAFDIST is calculated as indicated in Figure 4-1.

If the partition index space is reorganized without the PART keyword, then the entire index is reorganized. The LEAFDISTLIMIT is compared to the LEAFDIST value of each individual partition index, and REORG INDEX is triggered if any one of the partition LEAFDIST values exceeds LEAFDISTLIMIT.

The optimum value for LEAFDIST is 0 when the index space is fully organized and FREEPAGE is set to 0. DB2 for OS/390 V5 Administration Guide, SC26-8957-03, recommends a threshold value of 200. The same value is set as a default for the REORG INDEX utility. Thus, we recommend using LEAFDISTLIMIT 200 in REORG INDEX to trigger REORG.

Leaf Page 10 Leaf Page 11

HOT JAKLeaf Page 8 Leaf Page 9

FRI GRU

0-32


FUS IST

LEAFDIST = (sum(physical page gaps)*100)/#nleafs = ((9990+9989+9989+9988)*100)/10000 = 399

9990

9989

9989

9988

LEAFDIST Limit


DB2 V7 introduced two new statistical fields, LEAFNEAR and LEAFFAR, which provide better measurements of index disorganization. These two statistical fields are discussed in detail in 4.2.1, “REORG index with LEAFNEAR and LEAFFAR” on page 77.

4.1.2 REORG TABLESPACESimilarly, two new options were introduced in V6 of REORG TABLESPACE, namely OFFPOSLIMIT and INDREFLIMIT, which can be used to trigger the REORG utility. The same two options are also included in V7 of REORG.

OFFPOSLIMIT keywordThis basically indicates the degradation of the clustering index of a table within the table space. When rows are accessed by cluster index, the performance will be affected due to high synchronous I/O associated data pages not within the prefetch range. When the CLUSTERATIO is 100% it indicates that data in the table space is in cluster order. In Figure 4-2 we have a table space on the left that is unclustered. The table space on the right is in cluster order after the REORG.

Figure 4-2 Optimizing data clustering

Three values NEAROFFPOS, FAROFFPOS, and CARDF in SYSINDEXPART provide more details on the disorganized status of the table space. These values are used in the computation of OFFPOSLIMIT to trigger REORG TABLESPACE:

OFFPOSLIMIT = ((NEAROFFPOS + FAROFFPOS)/CARDF) * 100

� CARDF: This is the number of rows in the table space.

� NEAROFFPOS: For non-segmented table spaces, a page is considered near-off the present page if the difference between the two page numbers is greater than or equal to 2, and less than 16. For segmented table spaces, a page is considered near-off the present page if the difference between the two page numbers is greater than or equal to 2, and less than SEGSIZE * 2.

� FAROFFPOS: This is the number of times it would be necessary to access a different, "far-off" page when accessing all the data records in index order. For non-segmented table spaces, a page is considered far-off the present page if the two page numbers differ by 16 or more. For segmented table spaces, a page is considered far-off the present page if the two page numbers differ by SEGSIZE * 2 or more.

TSR1R2R3R4

TSR3

R4

R1

R2

deleted

deleted

deleted

Clustering IX

K1K2K3K4

Organize the data in clustering orderSYSINDEXPART: CLUSTERATIO, NEAROFFPOSF, FAROFFPOSF

Chapter 4. Triggering and controlling executions 71

The FAROFFPOS value is more significant, because a synchronous I/O is issued when the next page is not within the prefetch range. REORG TABLESPACE is triggered if the OFFPOSLIMIT is exceeded. The default value is 10.

INDREFLIMIT keywordWhen an update is done to a row, the resulting length of the row may be longer than the original record. In such cases, if the row cannot be fit into the original page, DB2 places the record in another page. However, the original RID in the index still points to the former page. In such a case, an entry is made in the original data page which is a pointer to the new page where the updated records are stored. This is termed as indirect row reference. The diagram in Figure 4-3 is an example of such an indirect reference. There could be an extra I/O involved to read the extra page into buffer pool.

Figure 4-3 Remove indirect row reference

Two values in SYSTABLEPART provide quantitative values for the measurement of indirect addressing — NEARINDREF and FARINDREF:

� NEARINDREF is the number of indirect references to overflow rows on another page within 64 pages.

� FARINDREF is the number of indirect references to overflow rows outside the 64 page limit.

When considering the statistics, both values indicate disorganization of the data. When rows are placed in another page than their home page, lock avoidances cannot be used, so DB2 has to take a lock. That can lead to performance degradation, especially in a data sharing environment.

When the INDREFLIMIT is specified in REORG TABLESPACE, the REORG utility will calculate the value as:

INDREFLIMIT = ((NEARINDREF + FARINDREF)/CARDF)*100

If this value exceeds the specified value in REORG, then REORG is performed. The default value is 10.

Remove Indirect Row References

update col10 = 'long character value....'; rid="0000000201" (record identifier indirect reference) overflow rid="0000000301" (new location of Row 1)

Page Header

ID Map

ptr-row1

Row 2

Row 3

Row 4

Page Header

ID Map

Row 1

page 2 page 3

page id


We recommend that you run REORG TABLESPACE, if the percent of indirect row references is greater than 10% for a non-data sharing environment, and for a data sharing environment if the percent of indirect row references is greater than 5%. You may also want to consider changing the PCTFREE value to allow more expansion without overflowing.

4.1.3 COPY utilityThere are no statistical field values in SYSTABLEPART or SYSINDEXPART that can be used to trigger the COPY utility. The only option is CHANGELIMIT, which can be used to trigger image copy of a table space, and which has been available since DB2 V5. The syntax is:

CHANGELIMIT (percentage-value1,percentage-value 2)

COPY scans the header page of the table space for space map bits and the image copy bit LRSN.

Single value for CHANGELIMITIf only a single percentage-value1 (pv1) is specified for the CHANGELIMIT, then image copy for the table space is triggered based on the criteria as listed in Table 4-1.

Table 4-1 COPY CHANGELIMIT with single value

Double values specified for CHANGELIMITIf both the lower bound and upper bound percentage values are specified (pv1,pv2), then image copy for the table space is triggered based on the criteria as listed in Table 4-2.

Table 4-2 COPY CHANGELIMIT with double values

Note: If the partitioned table space is reorganized without the PART keyword, then the limits are calculated for each partition. REORG is triggered when the limits exceed any one of the partitions.

Note: When the table space is reorganized, so are the associated index spaces. If you have a job stream that generates REORG statements, first you generate reorganization of table spaces, followed by index spaces. Your generator job should be able to remove reorganization of index spaces when the associated table space is reorganized.

%Changed pages COPY TABLESPACE

pv1 > 0 and %changed pages < pv1 Yes, incremental image copy

%changed pages > or = pv1 Yes, full image copy

%changed pages = 0 No image copy

CHANGELIMIT(0) Yes, full image copy

%Changed pages COPY TABLESPACE

pv1 < %changed pages < pv2 Yes, incremental image copy

%changed pages > or = pv2 Yes, full image copy

pv1 = pv2 AND %changed pages > or = pv1 Yes, full image copy

%changed pages < or = pv1 NO image copy

CHANGELIMIT(0) Yes, full image copy


If CHANGELIMIT(0), then a full image copy is always taken. Obviously, the additional option of REPORTONLY can be used with CHANGELIMIT. Then only image copy information is displayed, image copies are not taken, only recommended.

The default value for CHANGELIMIT, when specified, is (1,10).

COPY scans the spacemap pages of the table space for modified page indicators. This will recall table spaces that are migrated even with the REPORTONLY option.

4.1.4 Trigger limitsIn summary, there are REORG options that can be used trigger the execution of REORG utility. These limiting values are in SYSTABLEPART and SYSINDEXPART. RUNSTATS with UPDATE SPACE must be run prior to the REORG utility or the space statistics must be current. COPY table space can be triggered with CHANGELIMIT. Table 4-3 lists a summary of these KEYWORDS and the recommended limits.

Table 4-3 Trigger limits and utilities

4.2 New values with DB2 V7DB2 V7 did not introduce any new keywords or options in utilities that could be used to trigger utility execution. Nevertheless, V7 has introduced many new statistical fields in DSNDB06 which can be used to generate the appropriate utility control statements for application DB2 objects. Here we list all the columns in their corresponding tables with a brief description of each of the column fields.

SYSCOPY:� COPYPAGESF is the number of pages written to the copy data set. This value is needed

by the RECOVER utility to know the number of pages in the image data set (full/incremental).

� NPAGESF is the number of pages in the table space or index at the time of Inline COPY.

Note: CHANGELIMIT option is not available for COPY INDEXSPACE. Furthermore, COPY does not support incremental image copy for index space.

Trigger KEYWORD Limits Remark

LEAFDISTLIMIT > 200 REORG INDEXSPACE

OFFPOSLIMIT >10% REORG TABLESPACE

INDREFLIMIT >10% REORG TABLESPACE

CHANGELIMIT pv1 0 < %changed pages < pv1 COPY TS incremental copy

%changed pages > or = to pv1 COPY TS full image copy

pv1=0

CHANGELIMIT (pv1,pv2) pv1 < %changed pages < pv2 COPY TS incremental copy

%changed pages > or = to pv2 COPY TS full image copy

pv1 = pv2 AND %changed pages > or = to pv1

pv1=pv2=0


� CPAGESF is the number of changed pages.

� JOBNAME is a column whose value indicates the job name of the utility.

� AUTHID is a column whose value indicates the authorization ID of the utility.

SYSINDEXES:� SPACEF is the number of kilobytes of DASD allocated to the index. The value is -1 if

statistics have not been gathered.

SYSINDEXPART:� SPACEF is the number of kilobytes of DASD allocated to the index partition. The value is

-1 if statistics have not been gathered.

� DSNUM is the number of data sets. The value is -1 if statistics have not been gathered.

� EXTENTS is the number of data set extents. The value is -1 if statistics have not been gathered.

� LEAFNEAR is the number of leaf pages physically near previous leaf page for successive active leaf pages. The value is -1 if statistics have not been gathered.

� LEAFFAR is the number of leaf pages located physically far away from previous leaf pages for successive (active leaf) pages accessed in an index scan. The value is -1 if statistics have not been gathered.

� PSEUDO_DEL_ENTRIES is the number of pseudo deleted entries in the index. These entries are marked as “logically” deleted but still physically remain in the index. For a non-unique index, the value is the number of RIDs that are pseudo deleted. For an unique index the value is the number of both keys and RIDs that are pseudo deleted.The value is -1 if statistics have not been gathered.

SYSTABLEPART:� SPACEF is the number of kilobyte DASD allocated to the table space partition. The value

is -1 if statistics have not been gathered.

� DSNUM is the number of data sets. The value is -1 if statistics have not been gathered.

� EXTENTS is the number of data set extents. The value is -1 if statistics have not been gathered.

SYSTABLES:� NPAGESF is the number of pages used by the table.The value is -1 if statistics have not

been gathered.

� SPACEF is the number of kilobyte DASD allocated to the table. The value is -1 if statistics have not been gathered.

� AVGROWLEN is the average length of rows for the tables in the table space. If the table space is compressed, the value is the compressed row length. If the table space is not compressed, the value is the uncompressed row length. The value is -1 if statistics have not been gathered.

The space statistics are kept in the catalog database DSNDB06 in the two table spaces SYSDBASE and SYSHIST, as shown in Figure 4-5.

The SYSTABLEPART, SYSINDEXPART, and SYSLOBSTATS tables contain space statistics and no optimizer statistics. SYSTABLES contains both space and optimizer statistics.


Space management fieldsFigure 4-4 contains a subset of these columns that are used for space management. The new space management fields introduced in DB2 V7 are shown in italic bold text.

Figure 4-4 Fields updated with space information

In addition to the new columns, DB2 V7 also created a new table space, SYSHIST, with historical statistical information, which contains tables that are near duplicates to the actual tables in SYSDBASE and SYSSTATS. Table 4-4 provides a list these tables and the corresponding tables in SYSHIST.

Table 4-4 New tables in SYSHIST

Next we discuss the impact of the new columns and the new HIST tables and their usage in triggering the RUNSTATS, REORG, and COPY utilities.

SYSTABLEPART:

� CARD, CARDF� FARINDREF, NEARINDREF� PAGESAVE� PERCACTIVE, PERCDROP� SPACE, SPACEF, PQTY, SQTY, SECQTYI, DSNUM, EXTENTS

SYSINDEXPART:

� CARDF� FAROFFPOSF, NEAROFFPOSF, CLUSTERATIOF� LEAFFAR, LEAFNEAR, LEAFDIST� PSEUDO_DEL_ENTRIES� SPACE, SPACEF, PQTY, SQTY, SECQTYI, DSNUM, EXTENTS

SYSLOBSTATS:

� FREESPACE, AVGSIZE� ORGRATIO

SYSTABLES:

� AVGROWLEN

Table space Table name HIST table name in SYSHIST

SYSDBASE SYSTABLES SYSTABLES_HIST

SYSTABLEPART SYSTABLEPART_HIST

SYSINDEXES SYSINDEXES_HIST

SYSINDEXPART SYSINDEXPART_HIST

SYSCOLUMNS SYSCOLUMNS_HIST

SYSSTATS SYSCOLDIST SYSCOLDIST_HIST

SYSTABSTATS SYSTABSTATS_HIST

SYSLOBSTATS SYSLOBSTATS_HIST

SYSINDEXSTATS SYSINDEXSTATS_HIST


RUNSTATSRUNSTATS must be run regularly to update the tables with the relevant statistics. Statistics collected by RUNSTATS can be divided into two major categories.

First, there are access path or optimizer statistics, which are used by the optimizer to select the best access path to the data.

Second, there are space statistics, which are used to:

� Determine the optimum primary and secondary allocations used for table spaces and indexes, and help tune the setting of PCTFREE and FREEPAGE.

� Determine when to reorganize. This is a major area using different statistics to decide for different types of objects.

� Determine when to gather statistics again.

We will limit our discussion to the space statistics. When RUNSTATS is run with UPDATE SPACE, then only those fields listed in Figure 4-4 on page 76 are updated with space statistics. Additionally, if the HISTORY SPACE is specified, then the corresponding history tables in SYSHIST are also updated with the relevant space statistics. Figure 4-5 provides a pictorial view of all the tables that are updated with the RUNSTATS command. Refer to Chapter 11, “Gathering statistics” on page 253 for details.

Figure 4-5 RUNSTATS to collect space statistics

4.2.1 REORG index with LEAFNEAR and LEAFFARLEAFDIST measures index leaf disorganization (refer to 4.1.1, “REORG INDEX” on page 70), but it is not as good a measurement as using LEAFNEAR and LEAFFAR.

LEAFDIST measures the average gap between leaf pages. For many cases, this is a good measure. However, in a very large index, where the index is growing and requires page splits, LEAFDIST exaggerates the disorganization.

Space Statistics Tables

SYSLOBSTATS_HIST

Database DSNDB06 (Catalog)

SYSTABLEPART

SYSINDEXPART

SYSTABLES

SYSTABLEPART_HIST

SYSINDEXPART_HIST

SYSTABLES_HIST

SYSDBASE SYSHIST

SYSLOBSTATS

RUNSTATS db.ts INDEX(ALL) UPDATE SPACE HISTORY SPACE


Referring back to Figure 4-1 on page 70, the index has almost 10,000 pages, and only 4 of the pages are out of position (LEAFFAR=4) with jumps. However, calculating the average gap gives 399, or an average jump of 3.99 pages. The value is almost twice the recommended 200 threshold for determining the index needs to be reorganized! However, with only 4 leaves being out of position, reorganization is really not needed.

In V7, LEAFFAR and LEAFNEAR were introduced to measure physical leaf disorganization, or the number of physical leaf pages not in an optimal position. As shown in Figure 4-6, there is a logical and physical view of an index. The logical view is the tree shown at the top (2 levels in our example). If we were accessing the data by scanning for keys "FRISKE" through "JACKSON", the 4 leaf pages would be scanned as shown.

This same scanning of pages looking at physical page access is shown at the bottom of Figure 4-6. The first page is physical page 78, and the other leaf pages are located at physical locations 79, 13, and 14. A jump backward or ahead more than 1 page represents non-optimal physical ordering. As in other statistics, LEAFNEAR represents this jump within 64 pages and LEAFFAR represents the jump outside the same interval.

In our example, there are 2 big jumps: page 1 to page 78, and page 79 to page 13. These 2 jumps are recorded in LEAFFAR.

When considering the statistics, the LEAFFAR value is more significant and likely to affect performance than the LEAFNEAR value, but both these values indicate disorganization of the leaf pages, which can lead to performance degradation.

Figure 4-6 Logical and physical view before reorganization

After reorganizing the index, the leaf pages are in the optimal physical position as in Figure 4-7. For small indexes, LEAFNEAR and LEAFFAR will be 0 after reorganization. For large indexes, LEAFNEAR may not be 0 after reorganization. This is because space map pages are periodically placed throughout the page set, and the jump over space map pages shows up as a count in LEAFNEAR (space pages in a table space are also the reason why NEAROFFPOS may not get to zero after reorganization of a table space).

Root Page 2

Index Scan (before Reorg)

logical view

DAR-JAK


FRI GRULeaf Page 13 Leaf Page 14

HOT JAK



HOT JAK

physical view (LEAFFAR=2)

0-32 65-96


The percentage of leaf pages in disorganization can be calculated based on the values of LEAFFAR in SYSINDEXPART and NLEAF in SYSINDEXES as follows:

Physical leaf disorganization = (LEAFFAR/NLEAF)*100

You should consider REORG INDEX if the percentage of leaf disorganization is greater than 10%. Review changing PCTFREE or FREEPAGE to allow keys to be inserted without splitting as often.

When using DB2 V7 we recommend that you use the above formula to determine when to run the REORG INDEX utility, instead of using the LEAFDISTLIMIT keyword for REORG INDEX utility. The usage of LEAFDIST should be discontinued after V6.

Figure 4-7 Logical and physical view after reorganization

4.2.2 When to run RUNSTATSRUNSTATS are regularly run at many sites to gather statistics for access path optimizer and space usage. In regard to the question of when to run RUNSTATS on an object, the following are our recommendations:

� Run RUNSTATS whenever REORG or LOAD REPLACE or LOAD RESUME YES is performed on the object. You may use the inline statistic keyword STATISTICS and UPDATE option to update the statistics.

� Estimate pages changed since last RUNSTATS. This can be achieved by using the new SYSCOPY columns PAGESF, CPAGESF, and TIMESTAMP. Consider the following SQL query on a sample table:

SELECT A.DBNAME, A.TSNAME, A.TIMESTAMP, A.NPAGESF, A.CPAGESF FROM SYSIBM.SYSCOPY A, SYSIBM.SYSTABLESPACE B WHERE A.DBNAME = B.DBNAME AND (match database name) A.TSNAME = B.NAME AND (match table space name) A.ICTYPE = 'F' AND (SYSCOPY full image copy) A.TIMESTAMP > B.STATSTIME; (copies newer than RUNSTATS update)

Root Page 2

Index Scan (after Reorg)

logical view

DAR-JAK



HOT JAK


HOT JAKLeaf Page 8 Leaf Page 9

FRI GRU

physical view (LEAFFAR, LEAFNEAR = 0)

0-32 65-96


The query shown returns all SYSCOPY rows for an object which have been inserted since statistics were last collected on a table space. In this particular example, there have been three full image copies since statistics were last gathered.

Table 4-5 shows the sample output of the SQL query.

Table 4-5 Sample output from the SQL query

The estimated percentage of changed pages can be calculated as follows:

Estimate % changed = (10/100) 10% 10%

+ ((15/100) - ((10%)*(15/100)) 15% - 1.5% 23.5%

+ ((5/100) - ((23.5%)*(5/100)) 5% - 1.2% 27.3%

To estimate the percent of pages changed, an iterative process is followed:

1. The first image copy contained 100 pages, 10 of which were changed, or 10%.

2. The next image copy has 15 changed pages, or 15%. The combination of copies 1 and 2 would be a combined 25%, but if we assume both runs had random changes, the same page might be counted twice. Removing these counts gives 25% - (10%*15%), or 23.5%.

3. The same iterative process gives 23.5% + 5% - (23.5%*5%), or 27.3%.

A threshold limit can be set on the estimated percentage of changed pages to trigger RUNSTATS on the table space or index space.

Some DB2 database administrators are reluctant to update the access path statistics too frequently. In such cases, the DBA may consider running RUNSTATS with the UPDATE SPACE UPDATE HISTORY option only. The performance of RUNSTATS can be enhanced with the SAMPLE keyword on large objects; refer to Chapter 11, “Gathering statistics” on page 253 for details.

4.2.3 VSAM data set extentsThe maximum number of extents has increased from 119 to 251 with DFP 1.4. Although this was a concern in older devices, it is less of a performance consideration with new devices like RVA and ESS where data is physically stored in small pages all over the I/O devices. Nevertheless it is still a performance consideration with DB2 V6 during the open/close and the SWITCH phase of online REORG.

There are upper bounds to the number of extents. The size of each extent may influence the limit of maximum volumes per data set (59 per data set and 133 volumes per STOGROUP).

Continuous monitoring of the extents in SYSTABLEPART and SYSINDEXPART is necessary to track objects that exceed the number of extents past the threshold value of 50. When this value is exceeded, REORG should be scheduled for the object (TS or IX). You will need to review the values of PRIQTY and SECQTY. If one or both these values have changed, then avoid the REUSE parameter, so that the data set can be deleted and recreated with the new extents. PCTFREE and FREEPAGE may also be reduced to squeeze more data into less space.

DBNAME TSNAME TIMESTAMP NPAGESF CPAGESF

DB TS 2000-08-17-22.51.38.426693 1.00E+02 1.00E+01

DB TS 2000-08-18-09.51.25.298632 1.00E+02 1.50E+01

DB TS 2000-08-18-09.52.51.765528 1.00E+02 5.00E+01


With DB2 V7, with improvements to parallel data set open, FASTSWITCH for online REORG, and the disk capabilities provided by PAV, the requirement to reduce the number of extents is certainly not so critical in terms of performance. However, it is good practice to keep the environment under control in terms of DSMAX and VTOC size.

4.2.4 Reclaiming dead spaceHere we have two fields that can used to trigger REORG of table space or index space:

� PERCDROP for simple table spaces in SYSTABLEPART, available since V6.� PSEUDO_DEL_ENTRIES for Indexes SYSINDEXPART, introduced in V7.

Dropping tables in a simple table spaceSimple table spaces can accumulate dead space that is reclaimed during reorganization. If this dead space is not regularly reclaimed, it may result in acquiring more extents than needed to add additional data rows.

For simple table spaces, dropping a table results in that tables data rows remaining. The amount of dead space for a simple table space can be tracked directly with PERCDROP in SYSTABLEPART.

We recommend that you run REORG TABLESPACE, if the value of PERCDROP in SYSTABLEPART is greater than 10%.

Removing pseudo deleted entriesIndexes can accumulate dead space that is reclaimed during reorganization. If this dead space is not regularly reclaimed, it may result in acquiring more extents than needed to add additional keys.

For an index, deleted keys are marked as pseudo deleted. Actual cleaning up will not occur except during certain processes like before a page split.

You can calculate the percentage of RIDs that are pseudo deleted based on the values of PSEUDO_DEL_ENTRIES and CARDF in SYSINDEXPART:

(PSEUDO_DEL_ENTRIES/CARDF)*100

You can then determine if you should run REORG INDEX to physically remove the pseudo deleted entries from the index.

To minimize the CPU cost of an index scan and reclaim space, it is important to remove pseudo deleted entries. Every time a SQL statement makes a scan of an index, it has to scan all entries in the index, including pseudo deleted entries, that have not yet been removed.

We recommend that you run REORG INDEX, if the percentage of pseudo deleted entries is greater than 10%.

4.2.5 LOB space managementThree fields from SYSLOBSTATS can be used to manage LOB table spaces

� FREESPACE, kilobytes of free space in extents with respect to HURBA� AVGSIZE, average number of bytes per LOB� ORGRATIO, optimal physical location of active LOBs


Reclaim space for LOB table spacesThe FREESPACE gives an indication of how many more LOBs can be added into the existing extents already allocated.

LOB table spaces can accumulate dead space that is reclaimed during reorganization. If this dead space is not regularly reclaimed, it may result in acquiring more extents than needed to add additional LOBs.

For LOB table spaces, an updated LOB will be written without reclaiming the old version of the LOB immediately.

When FREESPACE approaches zero for a LOB table space, it is a good time to reclaim the space, but there are no direct statistics indicating how much will be reclaimed. You can calculate the percentage of free space for LOB table spaces based on the values of FREESPACE in SYSLOBSTATS and SPACEF in SYSTABLEPART:

(FREESPACE/SPACEF)*100

We recommend that you reorganize LOB table spaces, if the percent of free space is less than 10%.

LOBs AVGSIZEAVGSIZE is the average size of all LOBs in the table space, or the total number of LOB bytes divided by the number of LOBs. For example, if you had an employee column for a picture of the employee saved as a LOB, the format used (GIF, JPEG, and so on) will affect the size. The typical value of size within the column is indicated by AVGSIZE.

AVGSIZE does not contribute to triggering of any utility. Instead it can be used to estimate the data set sizes of work and sort data sets in LOB data set management.

LOBs ORGRATIOORGRATIO is a measure of fragmentation or non-optimal organization of the LOBs stored in the table space. A value of 1.0 is optimal.

A LOB column always has an auxiliary index which locates the LOB within the LOB table space. Access path is not an issue, because LOB access is always via a probe using the auxiliary index. However, performance can be affected if LOBs are scattered into more physical pieces than necessary, thus involving more I/O to materialize; see Figure 4-8.

Our example shows 4 LOBs as accessed from the auxiliary index. These LOBs are stored in chunks of pages. A chunk is 16 contiguous pages of a LOB. If the size of a LOB is smaller than a chunk, then it is expected to fit in 1 chunk. If the size is greater than a chunk, then it is optimized to fit into the minimum number of chunks (LOB size/chunk size). If, because of fragmentation within chunks, LOBs are split up to store in more chunks than optimally, the ORGRATIO will increase.

In our example, Figure 4-8, the first LOB (from rowid 1) has an extra chunk because the LOB is small enough to store in one chunk instead of two. Likewise, the fourth LOB can be stored in two chunks instead of three. The formula is shown for ORGRATIO.

Note: Apply the PTF for PQ42864 to allow RUNSTATS on a LOB table space to produce correct statistics values for FREESPACE, ORGRATIO, PERCACTIVE, and PCTROWCOMP.


Figure 4-8 Fragmented LOB table space

After reorganization of the LOB space, Figure 4-9, the LOBs are placed in the optimal number of chunks. Since there are no extra chunks allocated, the ORGRATIO is 1.0 according to the formula shown.

Figure 4-9 Non-fragmented LOB table space

LOBs Orgratio (fragmented)

No accesspath stats for LOBsOrgratio = (#extra chunk LOBs+#LOBs)/#LOBs) = (2+4)/4, or 1.5

rowid 1

rowid 4rowid 3rowid 2

chunk 1 chunk 2 chunk 3 chunk 4 chunk 5

Auxiliary Index

Auxiliary Table (LOB Tablespace)

LOBs Orgratio (After REORG)

Improved performance after REORGOrgratio = (#extra chunk LOBs+#LOBs)/#LOBs) = (0+4)/4, or 1.0

rowid 1

rowid 4rowid 3rowid 2

chunk 1 chunk 2 chunk 3 chunk 4 chunk 5

Auxiliary Index

Auxiliary Table (LOB Tablespace)


REORG does apply PCTFREE and FREEPAGE parameters or will delete and redefine the data set during reorganization of a LOB table space. If the number of extents are a problem, they can be changed (DB2 managed) by altering the PQTY and SQTY values, and then running RECOVER TOCOPY to delete and redefine the data sets with these new space parameters.

4.3 Trends from historical statisticsAs shown in Table 4-4 on page 76, DB2 V7 has introduced a number of historical statistical tables in catalog DSNDB06 (table space SYSHIST). These tables contain all statistical fields of the “parent” catalog tables with date and timestamp (see Figure 4-5 on page 77). The historical statistical data can be used to set thresholds and trigger utilities. In this section we will discuss the following:

� Monitor space growth of table space and index space� KEEPDICTIONARY keyword of REORG and LOAD utility

4.3.1 Monitoring space growthTable SYSTABLEPART_HIST has two fields CARDF and SPACEF which contain the values for total number of rows and total amount of DASD space allocated to the table space or partition space respectively. Similar fields also exist for index space in SYSINDEXPART_HIST.

CARDF and SPACEF can be tracked regularly for space growth of an object. Assuming constant growth over time, these numbers can be used to derive growth trends to plan for future needs. Consider the following sample SQL:

SELECT MAX(CARDF), MIN(CARDF), ((MAX(CARDF)-MIN(CARDF))*100)/MIN(CARDF),(DAYS(MAX(STATSTIME))-DAYS(MIN(STATSTIME)))FROM SYSIBM.SYSTABLEPART_HIST WHERE DBNAME='DB' AND TSNAME='TS';

Assuming that the number of rows is constantly increasing, so that the highest number is the latest, the query shows the percentage of rows added over a specific time period. This could be extrapolated to scale on a monthly or yearly basis.

COPY utilityThe space growth and SYSCOPY entries of a DB2 object can be used in determining the requirement for an image copy. If the space growth is greater than a pre-determined percentage growth value since the last full image copy, then the COPY utility can be triggered to take a full image copy of the object.

4.3.2 Compression dictionary When COMPRESS YES is set for a table space, the compression dictionary is initially built either via LOAD or REORG utility. The PAGESAVE field in SYSTABLEPART and SYSTABLEPART_HIST gives the percentage of pages saved due to the compression. The performance of the compression and decompression dictionary depends on the update activity on the table space. Over a time period, the PAGESAVE can be reduced below a threshold_limit value, and the DBA needs to rebuild the dictionary.


Regularly building the dictionary is expensive, and it should be avoided. The PAGESAVE value in SYSTABLEPART_HIST can be used to monitor the effectiveness of the dictionary. If the following situation occurs:

(max(pagesave)-min(pagesave))*100 /max(pagesave) > threshold_limit

Then you should remove the keyword KEEPDICTIONARY from the REORG or LOAD utility. The utility will build a new dictionary, or else code the keyword in the utility command. We recommend a threshold_limit of 10%.

4.4 Real-Time StatisticsThe objective of this new function, made available through maintenance on DB2 V7, is to help in the database administration of DB2. DB2 now provides Real-Time Statistics to be used by database administrators or automated task scheduler to determine which objects require REORG, RUNSTATS or COPY. A new stored procedure, DSNACCOR, provides a general interface to query the new tables where the Real-Time Statistics (RTS) are gathered. See Figure 4-10.

Figure 4-10 RTS overview

The first planned users of this function are the DB2 Control Center for OS/390, the SAP’s Control Center Management System, and DB2 tools such as the DB2 Automation Tool.

User programs can also take advantage of RTS rather than using the more static catalog statistics.

Note: RTS are introduced with the PTFs for APAR PQ48447 and PQ48448, DSNACCOR is introduced with the PTF for APAR PQ46859.

DB2 RTS M anager inserts and updates Real-Tim e Statistics (tim er interval) DSNACCO R (stored procedure) queries DB2 catalog and Real-Tim e Statistics

Real-Tim e Statistics

DB2Catalog

DB2

Real-Tim eStatistics

tables

DSNACCOR

CC/390or

m onitorprogram


4.4.1 Real-Time Statistics tablesThe statistics are collected in real-time, kept in memory, and periodically written to user defined DB2 tables from which applications can query the statistics.

Table definitionThe statistics are contained within a user created database called DSNRTSDB, and a segmented table space called DSNRTSTS. The RTS tables are:

� SYSIBM.SYSTABLESPACESTATS

It contains table space statistics, one row per table space or partition

� SYSIBM.SYSINDEXSPACESTATS

It contains index space statistics, one row per index space or index partition

The tables must be defined with row level locking and CCSID EBCDIC. A dedicated buffer pool will improve the efficiency while updating statistics.

Appendix A, “Tables for Real-Time Statistics” on page 275 shows the sample DDL to create these objects and gives a description of the columns in these tables. These definitions are contained in a new member of the DB2 sample programs library DSN710.SDSNSAMP(DSNTESS).

4.4.2 Description of statisticsThese are the statistics collected:

� Number of rows, LOB values, or index entries modified since the last RUNSTATS, REORG, or COPY

� Physical space information such as number of preformatted pages, allocated space and the number of extents

� Number of distinct pages updated since the last COPY or the time of the first update since the last COPY

DB2 will always generate in-memory statistics for each table space and index space in DB2.

Statistics are maintained for each partition if the page set is partitioned. DB2 will only externalize these statistics when:

� The required database, table space, tables, and indexes are created.

� The objects are created with the specified object names, schema name, and the specified attributes.

� The RTS database is started in RW mode; immediately after creation, this database is stopped.

DB2 externalizes the statistics at the following events:

� A user specified time interval is reached. This interval is set in the DSNZPARM parameter STATSINT. The default is 30 minutes.

� STOP of the RTS database externalizes in-memory statistics for all objects in DB2.

� STOP of a table space externalizes in-memory statistics for the specified object(s).

� STOP DB2 MODE(QUIESCE) externalizes in-memory statistics for all objects in DB2; a STOP DB2 MODE(FORCE) does not externalize in-memory statistics, and any in-memory statistics are lost.


Figure 4-11 Collecting RTS

When externalizing in-memory statistics, DB2 inserts a row for each partition or non-partitioned page set. If a row already exists it will be updated. The absolute statistical values are replaced with the new values, and incremental values are summed with the in-memory statistics. Active statistics blocks are ordered in memory in clustered order, and then inserted using the clustering index for optimal performance.

When the Statistics Manager attempts to update the RTS, objects might be in lock contention or other resource-not-available conditions that prohibit the updates from succeeding. The strategy for the Statistics Manager is to ensure that it does NOT fail and cause DB2 to crash, or cause the update requester to fail. The Statistics Manager will note the failure with a console message and attempt the externalizing the next update cycle.

DROP of any table space or index will cause their statistics rows to be deleted. If the statistics table rows cannot be deleted for any reason, the DROP is NOT failed. Therefore, after DROP, rows may exist in the statistics tables that do not belong to a table space or index (orphaned rows). The orphaned rows can be removed by using SQL, or the orphaned rows can remain in the statistics tables. If, subsequently, a table space or index is created with the same DBID and PSID, DB2 will re-initialize the orphaned row before updating any values in that row.

4.4.3 Impact of utilitiesGenerally, only SQL inserts, deletes, and so on, are accumulated in-memory, but certain utility operations (LOAD, REPLACE, REORG, or RECOVER) can significantly affect the statistics stored in the statistics tables. Each utility and its impact on the statistics is described in Appendix A, “Tables for Real-Time Statistics” on page 275.

By default, the DSNACCOR stored procedure:Queries both RTS tables Applies algorithms, using ROT thresholds and limits to the objects in the tables Reports back any objects that require REORG, RUNSTATS or COPY

DSNACCOR

SYSIBM.INDEXSPACESTATS

call DSNACCOR

DB2 Temporary Table (DSNACCOR_RS_LOC)

(Apply ROT thresholds and limits )

(defaults)

SYSIBM.TABLESPACESTATS


Non-DB2 utilities do not have the ability to affect the in-memory statistics. Therefore, an object that is the target of a non-DB2 COPY, LOAD, REBUILD, REORG, or RUNSTATS can cause incorrect statistics to be inserted in the RTS tables unless the following process is observed:

� STOP the target table space(s) or index(es). This will write the in-memory statistics to the RTS tables and initialize the in-memory counters.

� At the end of the utility; update the statistics tables with new totals, timestamps, and zero incremental counter values.

� Subsequent SQL operations updating in-memory statistics generate correct values with respect to the utility end point.

4.4.4 Maintaining in-memory statistics in data sharingIn data sharing, each DB2 member updates their statistics in a serial process. Each member reads the target row from the statistics table, and while holding a lock, aggregates their in-memory statistics, and updates the statistics tables with the new totals. The RTS start interval is specified for each DB2 member.

DB2 performs locking based upon the LOCKSIZE of the DSNRTSDB.DSNRTSTS table space. Reading of the statistics tables uses isolation level CS and “current data yes” semantics.

Utilities that reset page sets to empty can invalidate other DB2 members in-memory statistics. The member that resets a page set will notify the other DB2 members that a reset has occurred and the in-memory statistics are invalidated. If the notify process fails, the utility running on the "resetting" member will not fail. The appropriate timestamp (ReorgLastTime, StatsLastTime or CopyLastTime) is set to NULL to indicate that the table statistics are unknown. The timestamp is set to NULL for each utility, as stated in the above utility sections.

4.4.5 Statistics accuracyIn general the RTS are accurate values. But several factors can cause the table values to become inaccurate:

� Certain utility restart scenarios� DB2 subsystem failure� Notify failure in a data sharing environment

In data sharing, in a coexistence environment the statistics can be inaccurate until all DB2 members are updated to Version 7.

If the statistic values are in question, accurate values can be reestablished by a REORG, RUNSTATS, and COPY on the suspect objects.

4.4.6 DSNACCOR stored procedureThis stored procedure will query the new DB2 RTS tables to determine which DB2 objects should be reorganized, should have their statistics updated, should be image copied, have exceeded number of extents, or are in a restricted state. The default scope for this stored procedure is to scan "all" data in the RTS tables and provide recommendations for "any" condition mentioned above. A summary of these functions is provided in Figure 4-12.


Figure 4-12 DSNACCOR overview

To override either the scope of the data queried or the type of recommendation evaluated, the caller will pass along the appropriate parameter with the call to the stored procedure. Parameters can also be passed that override the default threshold settings. Calls to the stored procedure that do not provide parameters will employ default parameters (or thresholds) where appropriate.

4.5 DSNUTILS and Control CenterThe DB2 UDB Control Center and reduced “functions”, DB2 Connect for Windows NT, 2000, and ME, are packaged with DB2 V7. These products can be installed on a Windows based work station and configured to connect to the DB2 subsystem on OS/390. Please refer to the redbook DB2 UDB for OS/390 Version 6 Management Tools Package, SG24-5759, for details of installation and customization of DB2 Connect and Control Center (CC).

Here we show a simple example to reorganize a table space from the CC. Once in the CC, connect to the DB2 subsystem by clicking on the database alias. CC will invoke a logon window for userid and password, which are passed to the host for validation. You will be presented with a window of all DB2 objects, buffer pool, alias, databases, and so on.

Click on the databases, which will expand to a list of all databases in the DB2 subsystem. Select the DB2 objects in a database by clicking on the database that you wish to run the utility. Then click on the table spaces. You will be presented with table spaces within the database.

In our example, we clicked on database U7G01T11 and obtained a list of table spaces, as shown in Figure 4-13. Then we right-clicked on table space TSLINEI and got a drop-down menu which consisted of all utilities that can be run on the table space.

Allocate RTS blocks for each updated objectsAt first update since the pageset/partition is openedIn DBM1 address space

Free RTS blocks whenPagesets/partitions are closed andAfter statistics are written to RTS tables

In a data sharing system, statistics are collected by each member In-memory statistics are always collected even if RTS is not enabled

Real-Time Statistics in memory collection

DB2AReal-TimeStatistics

tablesDB2B


Figure 4-13 DB2 UDB Control Center - select table space

We wanted to reorganize the table space TSLINEI, so we clicked on Reorganize. The next window (see Figure 4-14) shows the REORG options for TSLINEI. Among all the options, please note the option Specify indicators for INDREFLIMIT and OFFPOSLIMIT, which can be used to trigger the REORG of the table space TSLINEI.


Figure 4-14 REORG basic options


The next window (see Figure 4-15) is obtained when REORG with SHRLEVEL CHANGE is selected and the radio button for the option is clicked. This window contains values for options associated with SHRLEVEL CHANGE.

Figure 4-15 REORG SHRLEVEL CHANGE options


The window shown in Figure 4-16 contains the data set allocation definitions when TEMPLATE are used for dynamic allocation of work and sort data sets. Finally, the stored procedure DSNUTILS is generated with the Show SQL button. This stored procedure is run on the host when the OK button is clicked.

Figure 4-16 REORG data sets

Similarly, all other utilities can be generated on the Control Center and run on the host as utility stored procedures. The complete procedure to set up a stored procedure environment is described in the redbook DB2 UDB for OS/390 Version 6 Management Tools Package, SG25-5759. Briefly, you need to:

� Define the workload environment in OS/390 WLM; the default is WLMENV.

� Start the DB2 stored procedure that is managed by the workload manager (for example, DB2GWLM).

� Perform a RACF access setup in the DSNR class.


Example 4-1 shows the stored procedure, DSNUTILS.

Example 4-1 Stored procedure DSNUTILS

CALL SYSPROC.DSNUTILS(UTILITY ID, NO, TEMPLATE CCCOPY DSN DB2.&SSID..&DB..&SN..D&JDAY.&HOUR.&MINUTE..&LOCREM.&PRIBAC..COPY TEMPLATE CCUNLOAD DSN DB2.&SSID..&DB..&SN..D&JDAY.&HOUR.&MINUTE..UNLOAD TEMPLATE CCSORTIN DSN DB2.&SSID..&DB..&SN..D&JDAY.&HOUR.&MINUTE..SORTIN TEMPLATE CCSRTOUT DSN DB2.&SSID..&DB..&SN..D&JDAY.&HOUR.&MINUTE..SORTOUT REORG TABLESPACE U7G01T11.TSLINEI LOG NO RECOVERYDDN(CCCOPY) SHRLEVEL CHANGE DEADLINE NONE MAPPINGTABLE PAOLOR1.MAP_TABLE MAXRO 300 DRAIN WRITERS LONGLOG CONTINUE TIMEOUT TERM OFFPOSLIMIT 10 INDREFLIMIT 10 UNLOAD CONTINUE UNLDDN CCUNLOAD WORKDDN(CCSORTIN,CCSRTOUT) SORTDEVT SYSDA SORTNUM 8, INTEGER, NONE, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0, , , 0)

4.6 Data administration toolsThe DB2 Administration Tool and the DB2 Automation Tool are two tools that belong to the family of IBM’s DB2 Data Management area. They provide functionalities to identify objects that require RUNSTATS, REORG, and so on. We will discuss each tool separately. For more details on these tools or all others, refer to the Web site:

http://ibm.com/software/data/db2imstools/

4.6.1 DB2 Administration ToolDB2 Administration Tool (DB2ADM) is an online product that enables database administrators to navigate the DB2 catalog easily. It has various functions, but we will concentrate only on functions that can be used to trigger utility execution on objects. Please refer to DB2 Administration Tools for OS/390 User’s Guide Version 2 Release 1, SC27-0974-01 for details of the product.

Figure 4-17 shows the administration menu. Choose Option 3 to get the next panel.

Figure 4-17 DB2 administration menu


Figure 4-18 shows how you can query your databases.

Figure 4-18 DB2 performance queries

Choosing Option 7 and database DSNDB06, we get the list of indexes with large leaf page distances reported in Figure 4-19.

Figure 4-19 Indexes with large leaf page distance

Select the index SYSIBM.DSHHKX01 which has the LEAFDISTLIMIT = 318. This index is a candidate for REORG. DB2ADM will generate the required JCL to run the REORG utility. Review the JCL, modify LEAFDISTLIMIT as appropriate, and submit the job.

Other options in Figure 4-18 can be selected to trigger the RUNSTATS, REORG, and STOSPACE utilities on DB2 objects.


4.6.2 DB2 Automation ToolDB2 Automation Tool for z/OS helps you to realize the full potential of your DB2 subsystem by continuously and automatically coordinating the execution of specific DB2 utilities on a 24x7 basis. Here are the primary capabilities provided:

� Automatic execution of the following utilities against specific objects:

– Image copy– REORG– RUNSTATS

� Manual, periodic, or rule-based execution with a combination of parameters and options

� Development of job and profiles through intuitive ISPF panels and special syntax without needing JCL skills

� Support that allows multiple database administrations to securely maintain their own sets of profiles

Now, without relying on repeated manual interventions, every DB2 administrator is able to add maximum value to the enterprise by extracting full performance from even the most heavily used database environment.

In this section we will concentrate only on the utility part of the tool and show some ISPF panels that are used to define the profiles and trigger values for the utilities. For further details on this tool please see http://www.ibm.com/software/data/db2imstools/details/#admin on the Web, or refer to DB2 Automation Tool for z/OS, V1R1, User's Guide, SC27-1189-00.

Option 2 from the main panel, Figure 4-20, requires a stored utility profile name.

Figure 4-20 DB2 Automation Tool main panel

If the profile is not available, then the profile create panel is displayed, as shown in Figure 4-21.


Figure 4-21 Utility profile display panel

Enter a profile name, which is stored in the tools control tables (defined as part of the installation), as shown in Figure 4-22.

Figure 4-22 New utilities profile data

The utilities profile options panel where you select the utilities is displayed, as shown in Figure 4-23.


Figure 4-23 Utilities profile options

Figure 4-24 is for the COPY utility. The CHANGELIMIT values can be set to trigger image copy.

Figure 4-24 Image copy options


Figure 4-25 is for RUNSTATS. Although there are no trigger limits, the options for the UPDATE keyword can still be set.

Figure 4-25 RUNSTATS options

Figure 4-26 is for the REORG table space. The INDREFLIMIT and OFFPOSLIMIT can be set to trigger the reorganization of the table space.

Figure 4-26 REORG table space options


Figure 4-27 is for REORG index space. The LEAFDISTLIMIT can be set to trigger reorganization of index space.

Figure 4-27 REORG index options


Chapter 5. Managing partitions

Partitioning is the ability to physically “split” DB2 table spaces into multiple “parts” to allow the utilities and SQL to operate at a subset level of the table. This process greatly enhances the availability of DB2 objects. Partitioning also enables a total storage of data up to 16 TB; this size has steadily increased since V4, up to 16,256 GB in V6.

In this chapter we examine the reasons for partitioning, and the best ways to manage them.

5


5.1 Why partition?With DB2 V3, partitioning offered a solution to make more manageable large table spaces that otherwise would have required a very long down time when DB2 Utilities executions were needed. Partitions could be handled one at the time, but not independently, dividing the down time into several jobs of more reasonable duration.

The following releases introduced more and more partition independence, for DB2 Utilities execution, and for SQL, providing for a better distribution of the data and therefore less contention, and for parallel access and better performance. Another reason for partitioning is to increase the amount of data that can be stored in a single (partitioned) table space. As shown in Table 5-1, the limit has increased from 64 GB to 16 TB between Version 4 and Version 6.

Table 5-1 Number of partition and partitioned table space sizes

The partitioning of table spaces results in smaller “parts” of data, that is, many smaller data sets combining up to one larger table space. SQL can exploit this feature automatically by only scanning the relevant partitions for data that is required to match the WHERE clause, and by running the SQL in parallel against separate partitions. If possible, this is done automatically and does not require any extra coding in the SQL statement. This can increase performance of SQL by only reading from individual partitions rather than the whole table space, thus reducing I/O times.

The placement of individual partitions, and their partitioning index partitions, is possible to reduce contention across devices and channels. This can be achieved by either using user managed data sets or by coding ACS routines to handle individual partition data sets. Be aware that in DB2 Version 7 the data set names can switch naming standards following a REORG using FASTSWITCH. See 7.6.5, “SWITCH” on page 171.

Utilities also exploit the features of partitioning. Utilities can act upon one, all, or a range of partitions, which can greatly reduce the elapsed time of a utility. With partitioning, many utilities can start parallel subtasks to further increase the savings in elapsed time. In previous releases of DB2, any non-partitioning indexes on a partitioned table space caused contention when running parallel invocations of the same utility against different partitions. This contention is vastly reduced by utility parallelism and subtasks, thus eliminating the need to drop and recreate non-partitioning indexes (NPIs) before utilities such as REORG.

Partitioning helps enable true 24x7 high availability and should be used for all large table spaces where continuous availability is an issue.

DB2 Version Number of partitions Maximum size Total maximum size

V4* 1 to 16 4 GB 64 GB

17 to 32 2 GB 64 GB

33 to 64 1 GB 64 GB

V5** 254 4 GB 1,016 GB

V6*** 254 64 GB 16,256 GB

Note: * For a maximum total of 64 GB

Note: ** Requires LARGE parameter in CREATE TABLESPACE

Note: *** Requires DSSIZE parameter, DFSMS/MVS 1.5 and SMS managed table space


5.2 Altering partitionsOver time, the data in a partitioned table space can become skewed, which results in some partitions being larger than other partitions. This can lead to degradation of performance in the larger partitions. To change this, the key values in the partitioning index have to be altered to reflect the new distribution of data.

Prior to DB2 V6, the sequence for altering the key values in a partitioning index was as follows:

1. Stop the table space and start it in read-only mode.

2. Unload the entire table space.

3. Drop the table space.

4. Redefine the table space with new limitkey definitions.

5. Redefine any indexes and non-partitioning indexes (NPIs).

6. Reload the unloaded data.

7. Run RUNSTATS.

8. Rebind affected plans and packages.

9. Reissue necessary grants for authorization to the original objects.

10.Redefine any synonyms, views, and referential constraints.

11.Make new image copies for recoverability.

This is a complicated process and leads to the whole table space being unavailable during the process, which could be lengthy, depending upon the size of the table.

Version 6 offers the ability to ALTER the key range of a partitioning index. When ALTERING the key values, DB2 sets a restrictive status of REORGP (REORG pending) on the affected partitions. Access to the data is only restricted to the partitions with this status; no other partitions are affected and are accessible. To remove the REORGP status, a REORG has to be run against all the affected partitions (it is recommended that this be a REORG SHRLEVEL NONE).

Figure 5-1 Altering partitioning index key values

PART 1 PART 2 PART 3 PART 4 PART 5

REORGP

1949 1959 1969 1979 >=1989

ALTER INDEX BIRTH_YEAR PART 3 VALUES '1966'

Chapter 5. Managing partitions 103

In Figure 5-1, the values of the LIMITKEY has been altered to 1966. This puts both partition 3 and 4 into REORGP status. Both are affected because data from partition 3 will now be placed in partition 4 (rows with BIRTH_YEAR > 1966 and < 1969). Both partitions will now need to be reorganized to redistribute the data. Be aware that if you alter all the partitions, or alter enough keys to place to affect all partitions, then the whole table space will become unavailable due to REORGP being set. A full REORG or REORGs for all partitions will have to be undertaken.

This enhancement could be used to allow extra partitions for future growth and then altering key values to move rows into the new partition. This will assist database administrators to expand the range of values in the last partition without affecting the availability of the other partitions.

Partitioning can also be used to prepare for future growth of data. For example, a table space may be partitioned on YEAR, as per Figure 5-1, and may contain partitions for future years. These partitions can be created small (48/48) and increased when required, that is, when the data for that year starts being produced. Using this method enables the physical design of a database to recognize future growth, while deferring disk usage until it is required.

5.3 SQL and partitioningWe have briefly explained some of the benefits to SQL processing from partitioning: that is, less data in a partition, therefore less rows to read, therefore less I/O to perform which leads to faster performance.

Partitioning is generally transparent to SQL, but partitioning has also some indirect impact on SQL due to the restriction that was placed on UPDATE statements. The restriction was that the columns of a partitioning index could NOT be updated. This option to remove this restriction was introduced in DB2 Version 6 with the ZPARM field PARTKEYU in panel DSNTIPB.

PARTKEYU can have the following values:

� NO: This eliminates the ability to update partitioning key columns.� SAME: This allows the updating of key columns, but only within the same partition.� YES: This allows full update activity against the key columns without restrictions, and it is

the default.

Any attempt to update a key that violates the PARTKEYU ZPARM results in an SQLCODE -904 error message.

Be aware that using YES or SAME will allow users to update primary keys that could invalidate logical design rules.

When planning to partition, it is essential that the application code be fully understood. If there is updating of the “soon-to-be” partitioning key, then these programs will have to be changed, or the PARTKEYU ZPARM field set to YES. If the change is made without application checking, applications that worked before will stop working.

If the application cannot be changed, and the value of PARTKEYU is to remain NONE, then an artificial key must be chosen to act as the partitioning key and be built into the design of the database.


5.4 Utilities and partitioningMost utilities have been able to operate at the partition level for some time. Recent enhancements to utilities have allowed single utilities to act in parallel against multiple partitions. The problematic area of non-partitioning indexes (NPIs) has also been addressed by the use of parallelism and subtasks. In this section, we look at how to use partitions with utilities.

In early releases of DB2, it was advised to drop NPIs before running utilities, run multiple occurrences of a utility in parallel, and then recreate the NPIs. In very large table spaces, it was even advised to set multiple RECOVER INDEX (now renamed REBUILD INDEX) jobs running in parallel, stop them after the UNLOAD phase, sort and merge the SYSUT1 files, and then feed the data set back into a restarted REBUILD (RECOVER) INDEX utility. This method reduced the SORT time, which for very large sorts, tends to increase exponentially.

With partitioning and parallelism, this advice is no longer current. By the use of SORTKEYS, the REBUILD INDEX will start multiple subtasks, optimally three subtasks (one unload, one SORT, and one to build the index), for each partition of a partitioning index. If an NPI exists, there will be a MERGE step added, and a single subtask, per NPI, to build the index. This reduces contention on the NPI and reduces total elapsed time. The total number of subtasks is determined by the amount of available resources. See 3.4, “REBUILD INDEX Parallel Index Build” on page 57, for more details on the REBUILD INDEX function.

The SORTKEYS keyword enables the parallel index build of indexes. For example, each load task takes input from a sequential data set and loads the data into a corresponding partition. The utility then extracts index keys and passes them in parallel to the sort task that is responsible for sorting the keys for that index. If there is too much data to perform the sort in memory, the sort product writes the keys to the sort work data sets. The sort tasks pass the sorted keys to their corresponding build task, each of which builds one index. If the utility encounters errors during the load, DB2 writes error and error mapping information to the error and map data sets.

5.4.1 Partition independence and parallelismThe key to getting utilities to work in parallel is partitioning, from which the full benefit of parallelism can be gained.

Each partition can be operated upon independently from the others by most utilities, thus removing the need to run utilities against all the data when only a subset requires acting upon. For example, a partitioned table space may be partitioned on YEAR, once the year has passed this data becomes stable. Therefore, it is not necessary to run REORGs on it. If the table space was not partitioned, all data would be REORGed on every run of REORG. With partitioning, only the partition in use needs to be REORGed.

Independence of partitions is a critical factor that is further enhanced by DB2 Version 7 with the ability to run parallel utilities (both automatically or manually). Contention has also been reduced on the non-partitioned indexes by using subtasks to build the indexes. Parallel index build allows for a partitioning index to have its partitions build in parallel by multiple subtask. It can unload multiple partitions in parallel, something that had to be manually organized prior to Version 6.

The LOAD utility can now operate on partitions in parallel as well as individual partitions or the whole table space.

Chapter 5. Managing partitions 105

Benefits like Parallel Index Build (see 3.4, “REBUILD INDEX Parallel Index Build” on page 57) and LOAD parallelism (see 6.3, “Partition parallelism within one LOAD” on page 118) can only be gained with partitioned table spaces.

5.4.2 Non-partitioning indexesNon-partitioning indexes (NPIs) have long been considered a bottleneck on utility performance. Parallelism is one method aimed at reducing the outage caused by building NPIs (more subtasks running in parallel therefore less down time). Another method of improving performance was the introduction of the PIECESIZE parameter.

PIECESIZE was introduced to allow DBAs to decide how big an NPI data set could become before creating an additional data set for an index. The default is 2 GB, which means that a data set over 2 GB would be in two or more data sets. By setting the PIECESIZE smaller, the NPI will occupy more data sets. These, like partitions, can then be intelligently placed to ensure that there is little contention across devices and channels.


Part 3 Executing DB2 Utilities

In this part, we describe the functionality of the DB2 Utilities and provide examples of their use:

� Chapter 6, “Loading data” on page 109� Chapter 7, “Reorganizing data” on page 161� Chapter 8, “Unloading data” on page 191� Chapter 9, “Recovering data” on page 209� Chapter 10, “Copying data” on page 231� Chapter 11, “Gathering statistics” on page 253.

Part 3


Chapter 6. Loading data

The primary method for loading data is the LOAD utility. Its specific purpose is to populate DB2 tables with data. Before DB2 V7, the input to LOAD was only a sequential data set. With DB2 V7, the Cross Loader was introduced: This new DB2 Utility statement allows the input of the LOAD utility to be the result set of an SQL query launched against tables on a local or remote server.

With the LOAD utility you can:

� Load rows into an empty table� Add new rows to a table that is not empty� Load rows into multiple tables within one table space� Empty a table space before reloading the table(s)� Filter the input data using criteria supplied by the user� Check the input data on unique, referential, and other constraint violations, and discard

failing records to a sequential data set

In this chapter, we will mainly focus on new LOAD options introduced since DB2 V5, such as the following:

� Inline COPY and Inline RUNSTATS� Sortkeys � Partition parallelism � Preformat� Reuse� Cross Loader� Online Resume


6.1 Inline COPY and Inline RUNSTATSWe first discuss the ability of producing an image copy and current statistics during the loading of a table space.

6.1.1 Inline COPYBefore DB2 V5, when you did a LOAD of a table space with LOG NO, the table space was always put in a restrictive COPYP status. The COPYP status prevents applications from writing to the table space. A full image COPY with SHRLEVEL REFERENCE or SHRLEVEL CHANGE is needed to remove the restricted state. DB2 forces the existence of this full image copy to make the table space recoverable after the LOAD operation. The RECOVER utility will use this full image copy and subsequent log records to RECOVER the table space if needed.

DB2 V5 introduced the ability to create a full image copy while loading the table space with the LOAD utility. This inline image copy is created during the RELOAD phase. The table space is fully accessible to applications after the LOAD and an additional full image is no longer necessary to guarantee the recoverability of the table space. So the unavailability time of the table space during LOAD is reduced.

The inline image copy is triggered by the presence of the COPYDDN (and optionally RECOVERYDDN) keyword of the LOAD utility. It can only be used with LOAD REPLACE. After a LOAD RESUME YES with LOG NO or LOAD RESUME NO with LOG NO the table space is still put into the restrictive COPYP status, and a full image copy with SHRLEVEL REFERENCE is still needed to make the table space available to writing applications and to make the table space recoverable.

You can neither create an inline image copy of an index space, nor can you create an inline incremental image copy. Also, you cannot take an inline image copy of a LOB table space. An inline image copy is always recorded in SYSCOPY as a full image copy with SHRLEVEL REFERENCE, although it is not recommended for use with RECOVERY TOCOPY.

See 3.1.1, “Inline COPY” on page 40 for more details.

6.1.2 Inline RUNSTATSIt is always recommended to keep the statistics of a table space and related objects current after the LOAD of a table space. This to ensure that the correct access paths are chosen by the optimizer and that triggering of other utilities, based on these statistics, is correct. Before DB2 V6, you could achieve this by running a RUNSTATS utility immediately after the LOAD utility.

DB2 V6 introduced the ability to update the statistics while loading the table space, thus eliminating the need for an extra RUNSTATS utility and reducing the total elapsed time.

The inline statistics is triggered by the STATISTICS option of the LOAD utility. It can be used with LOAD REPLACE and LOAD RESUME NO, except on LOB table spaces. All RUNSTATS options are supported, including the new HISTORY functionality introduced in DB2 V7. With LOAD RESUME YES, you still have to run a RUNSTATS utility after the LOAD if you want to make the statistics current. The same applies for LOB table spaces.

If any parallelism is used during the LOAD (see 6.2, “Parallel Index Build” on page 111 and 6.3, “Partition parallelism within one LOAD” on page 118), the statistics will be gathered in parallel as well, both for table space and index statistics. This will occur as additional subtasks associated with the reload and/or the build subtasks.


See 3.1.2, “Inline RUNSTATS” on page 46 for more details.

6.1.3 RecommendationsUse the aforementioned inline utilities as much as possible for reducing the elapsed time and unavailability of your table spaces during LOAD:

� Use LOAD REPLACE with LOG NO and COPYDDN (/RECOVERYDDN): This is the most effective way to LOAD or RELOAD your table space. After this, the table space will be fully available to all applications and fully recoverable.

� Use LOAD REPLACE and LOAD RESUME NO with STATISTICS: This is the most effective way to keep your statistics current after LOAD or RELOAD.

6.2 Parallel Index Build Parallel Index Build (PIB) was introduced in DB2 V6. It allows to reduce the elapsed time of a LOAD job by sorting the index keys and rebuilding the indexes in parallel, rather than sequentially. It can be used for both non-partitioned and partitioned table spaces, provided that there is more than one index.

In contrast with REBUILD INDEX, if a partitioned table space only contains the partitioned index, the parts of the partitioned index will never be built in parallel during LOAD. However, the partitioned index and the non-partitioned indexes will be built in parallel.

PIB is illustrated in Figure 6-1 for a table space containing 3 indexes.

Figure 6-1 LOAD with Parallel Index Build

PIB is activated by the SORTKEYS n option, n being an estimation of the total number of keys to be sorted. If you specify n = 0, PIB will not be activated. DB2 will pass a value based on n to DFSORT (or other SORT product used) to allow the sort subtasks to allocate their proper work data sets. If the estimation is far from the real number of keys to be sorted, the SORT might fail. See DB2 Universal Database for OS/390 and z/OS Utility Guide and Reference, SC26-9945-00, regarding how to calculate an appropriate value of n.

B u ildIndex 2

Index 1

Index 3

Indexes bu iltin

pa ra lle l

Tab le S pace

S ort

keys

M u ltip le S o rt/B u ild task pa irs p rov ide pa ra lle lism

LO A D

S O R T B LD

Chapter 6. Loading data 111

DB2 V5 introduced the SORTKEYS option to optimize the sort by eliminating multiple I/O accesses to workfiles when building the indexes. DB2 V6 used the same option to activate the PIB as well.

The indexes are built in parallel by multiple pairs of sort and build subtasks (optimally one pair per index). All this is done in the new SORTBLD phase, which replaces the SORT and BUILD phases. The sort subtask sorts the extracted keys, while the build subtask builds the index. This is also done partially in parallel, because the index build starts as soon as the sort subtask passes the first sorted key. Moreover the SYSUT1 and SORTOUT work data sets are no longer used because the key/RIDs pairs are now directly piped in memory between the subtasks, eliminating considerable I/O processing.

Optimally with PIB, the number of subtasks pairs is equal to the number of indexes to be built. DB2 determines the degree of parallelism, that is, the number of subtask pairs based on the following factors:

� Number of indexes to be built� Available virtual storage in the batch region� Number of CPUs available

See 3.7, “Considerations for running parallel subtasks” on page 67 for more information.

You have some manual control over the degree of parallelism:

� You can specify sort workfile DD statements (SWnnWKnn) in the JCL for the number of parallel subtasks you want, instead of using dynamic allocation.

� You can also route the UTPRINT output to a number of UTPRINnn data sets, one for each parallel subtasks you want, instead of using SYSOUT=*.

If you specify both sort workfile DD statements and UTPRINT data sets, the number of subtask pairs equals the smallest number of data sets specified.

See also 3.3, “LOAD and REORG Parallel Index Build” on page 52.

In Example 6-1 we show the loading of a non-partitioned table space using PIB. The input data set is allocated in the JCL with a DD card SYSREC, and contains about 50000 records. We use templates to specify the Inline COPY data sets and work data sets (with SPACE parameters because DB2 is unable to estimate them properly). We also specify Inline RUNSTATS. The table space contains 3 indexes. We activate the PIB by specifying SORTKEYS 150000 (150000 = 3 indexes * 50000 keys/index).

Example 6-1 LOAD of a non-partitioned table space with PIB

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSREC DD DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.UNLOAD,DISP=SHR //SYSIN DD * TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(10,10) CYL TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS 150000 COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER)


WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPN WHEN(1:2 = X'000E') (CASE_KEY POSITION(003:006) INTEGER ,SEVERITY_KEY POSITION(007:010) INTEGER ,RECIEVED_VIA_KEY POSITION(011:014) INTEGER ,TYPE_KEY POSITION(015:018) INTEGER ,ASSIGNED_TO_KEY POSITION(019:022) INTEGER ,TOD_KEY POSITION(023:026) INTEGER ,PRODUCT_KEY POSITION(027:030) INTEGER ,CUSTOMER_KEY POSITION(031:034) INTEGER ,STATUS_KEY POSITION(035:038) INTEGER ,RESOLUTION_KEY POSITION(039:042) INTEGER ,REASON_CLOSED_KEY POSITION(043:046) INTEGER ,CLOSED_BY_KEY POSITION(047:050) INTEGER ,TIME_CREATED_KEY POSITION(051:054) INTEGER ,TIME_CLOSED_KEY POSITION(055:058) INTEGER ,NEW_CASE_VOLUME POSITION(060:063) INTEGER NULLIF(059)=X'FF' ,OPEN_CASE_VOLUME POSITION(065:068) INTEGER NULLIF(064)=X'FF' ,CLOSED_CASE_VOLUME POSITION(070:073) INTEGER NULLIF(069)=X'FF' ,CLOSED_ONFIRSTCONT POSITION(075:078) INTEGER NULLIF(074)=X'FF' ,DAYS_TO_RESOLUTION POSITION(080:083) INTEGER NULLIF(079)=X'FF' ,AVERAGE_DAYS_TO_RE POSITION(085:088) INTEGER NULLIF(084)=X'FF' ,CLOSED_BY_OTHERS POSITION(090:093) INTEGER NULLIF(089)=X'FF' ,CASE_SUBJECT POSITION(095:196) VARCHAR NULLIF(094)=X'FF' ,CASE_NOTE POSITION(198:0454) VARCHAR NULLIF(197)=X'FF' ,LAST_COMMENT POSITION(456:712) VARCHAR NULLIF(455)=X'FF' )

In Example 6-2 we LOAD the same table space by specifying the input data set with a template and adding discard processing. For this, we have to add a template for the input, discard, error, and mapping data sets.

Example 6-2 LOAD of non-partitioned table space with PIB; discard processing

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..UNLOAD) TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..DISCARD) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P)


SPACE(10,10) CYL TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS 150000 INDDN(TSYSREC) DISCARDDN(TSYSDISC) ERRDDN(TSYSERR) MAPDDN(TSYSMAP) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPN WHEN(1:2 = X'000E') (CASE_KEY POSITION(003:006) INTEGER ,SEVERITY_KEY POSITION(007:010) INTEGER ,RECEIVED_VIA_KEY POSITION(011:014) INTEGER ,TYPE_KEY POSITION(015:018) INTEGER ,ASSIGNED_TO_KEY POSITION(019:022) INTEGER ,TOD_KEY POSITION(023:026) INTEGER ,PRODUCT_KEY POSITION(027:030) INTEGER ,CUSTOMER_KEY POSITION(031:034) INTEGER ,STATUS_KEY POSITION(035:038) INTEGER ,RESOLUTION_KEY POSITION(039:042) INTEGER ,REASON_CLOSED_KEY POSITION(043:046) INTEGER ,CLOSED_BY_KEY POSITION(047:050) INTEGER ,TIME_CREATED_KEY POSITION(051:054) INTEGER ,TIME_CLOSED_KEY POSITION(055:058) INTEGER ,NEW_CASE_VOLUME POSITION(060:063) INTEGER NULLIF(059)=X'FF' ,OPEN_CASE_VOLUME POSITION(065:068) INTEGER NULLIF(064)=X'FF' ,CLOSED_CASE_VOLUME POSITION(070:073) INTEGER NULLIF(069)=X'FF' ,CLOSED_ONFIRSTCONT POSITION(075:078) INTEGER NULLIF(074)=X'FF' ,DAYS_TO_RESOLUTION POSITION(080:083) INTEGER NULLIF(079)=X'FF' ,AVERAGE_DAYS_TO_RE POSITION(085:088) INTEGER NULLIF(084)=X'FF' ,CLOSED_BY_OTHERS POSITION(090:093) INTEGER NULLIF(089)=X'FF' ,CASE_SUBJECT POSITION(095:196) VARCHAR NULLIF(094)=X'FF' ,CASE_NOTE POSITION(198:0454) VARCHAR NULLIF(197)=X'FF' ,LAST_COMMENT POSITION(456:712) VARCHAR NULLIF(455)=X'FF' )

An extract of the job output is shown in Example 6-3. Here you can see that DB2 used 9 subtasks for building the 3 indexes in parallel (3 pairs of sort, build and 3 statistics) during the SORTBLD phase. Inline COPY and inline statistics are produced.


Example 6-3 Job output LOAD of a non-partitioned table space with PIB

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..UNLOAD) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..DISCARD) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA REPLACE LOG NO SORTKEYS 150000 INDDN(TSYSREC) DISCARDDN(TSYSDISC) ERRDDN(TSYSERR) MAPDDN(TSYSMAP) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPN WHEN(1:2=X'000E') DSNU650I -DB2G DSNURWI - (CASE_KEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - SEVERITY_KEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - RECIEVED_VIA_KEY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - TYPE_KEY POSITION(15:18) INTEGER, DSNU650I -DB2G DSNURWI - ASSIGNED_TO_KEY POSITION(19:22) INTEGER, DSNU650I -DB2G DSNURWI - TOD_KEY POSITION(23:26) INTEGER, DSNU650I -DB2G DSNURWI - PRODUCT_KEY POSITION(27:30) INTEGER, DSNU650I -DB2G DSNURWI - CUSTOMER_KEY POSITION(31:34) INTEGER, DSNU650I -DB2G DSNURWI - STATUS_KEY POSITION(35:38) INTEGER, DSNU650I -DB2G DSNURWI - RESOLUTION_KEY POSITION(39:42) INTEGER, DSNU650I -DB2G DSNURWI - REASON_CLOSED_KEY POSITION(43:46) INTEGER, DSNU650I -DB2G DSNURWI - CLOSED_BY_KEY POSITION(47:50) INTEGER, DSNU650I -DB2G DSNURWI - TIME_CREATED_KEY POSITION(51:54) INTEGER, DSNU650I -DB2G DSNURWI - TIME_CLOSED_KEY POSITION(55:58) INTEGER, DSNU650I -DB2G DSNURWI - NEW_CASE_VOLUME POSITION(60:63) INTEGER NULLIF(59)=X'FF', DSNU650I -DB2G DSNURWI - OPEN_CASE_VOLUME POSITION(65:68) INTEGER NULLIF(64)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_CASE_VOLUME POSITION(70:73) INTEGER NULLIF(69)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_ONFIRSTCONT POSITION(75:78) INTEGER NULLIF(74)=X'FF', DSNU650I -DB2G DSNURWI - DAYS_TO_RESOLUTION POSITION(80:83) INTEGER NULLIF(79)=X'FF', DSNU650I -DB2G DSNURWI - AVERAGE_DAYS_TO_RE POSITION(85:88) INTEGER NULLIF(84)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_BY_OTHERS POSITION(90:93) INTEGER NULLIF(89)=X'FF', DSNU650I -DB2G DSNURWI - CASE_SUBJECT POSITION(95:196) VARCHAR NULLIF(94)=X'FF', DSNU650I -DB2G DSNURWI - CASE_NOTE POSITION(198:454) VARCHAR NULLIF(197)=X'FF', DSNU650I -DB2G DSNURWI - LAST_COMMENT POSITION(456:712) VARCHAR NULLIF(455)=X'FF') DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.UNLOAD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.SORTOUT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.DISCARD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSERR DDNAME=SYS00005 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.SYSERR DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSMAP DDNAME=SYS00006 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.SYSMAP DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSCOPY DDNAME=SYS00007 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.D2001166.T213106.P DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TRECOVER DDNAME=SYS00008 DSN=PAOLOR3.PAOLOR3N.PAOLOR3N.D2001166.T213106.R DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU395I DSNURPIB - INDEXES WILL BE BUILT IN PARALLEL, NUMBER OF TASKS = 9 DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE PAOLOR3N.PAOLOR3N NUMBER OF PAGES=2693 AVERAGE PERCENT FREE SPACE PER PAGE = 7.99 PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:00:13 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3N.PAOLOR3N SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR3.EXAMPN SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPN SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR PAOLOR3N.PAOLOR3N SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-17.31.11.762304


DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=48420 FOR TABLE PAOLOR3.EXAMPN DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=48420 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:14 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=48420 FOR INDEX PAOLOR3.I_PAOLOR3N_2 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=48420 FOR INDEX PAOLOR3.I_PAOLOR3N_3 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_3 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_3 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPN SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_3 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-17.31.11.823132 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_2 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_2 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPN SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_2 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-17.31.11.824661 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=48420 FOR INDEX PAOLOR3.I_PAOLOR3N_1 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_1 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_1 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPN SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_PAOLOR3N_1 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-17.31.11.815587 DSNU391I DSNURPTB - SORTBLD PHASE STATISTICS. NUMBER OF INDEXES = 3 DSNU392I DSNURPTB - SORTBLD PHASE COMPLETE, ELAPSED TIME = 00:00:02 DSNU428I DSNURELD - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLOR3N.PAOLOR3N DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

In Example 6-4 we show the loading of a partitioned table space with 3 partitions using PIB. Here we are not using the partition parallelism introduced in DB2 V7 as explained in 6.3, “Partition parallelism within one LOAD” on page 118. The 3 input data sets are concatenated in the JCL to form one single sequential data set allocated to the DD card SYSREC containing about 600000 records. We use templates to specify the Inline COPY data sets and work data sets. We also use Inline RUNSTATS. The table space contains 2 indexes (the partitioned index and one NPI). We activate the PIB by specifying SORTKEYS 1200000 (1200000 = 2 indexes * 600000 keys/index). We verified that DB2 will use 6 subtasks for building the 2 indexes in parallel (2 pairs of sort, build and 2 statistics) during the SORTBLD phase. Inline COPY and inline statistics are produced.

Example 6-4 LOAD of a partitioned table space with PIB

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSREC DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00001,DISP=SHR // DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00002,DISP=SHR // DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00003,DISP=SHR //SYSIN DD * TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..DISCARD) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS 1200000 DISCARDDN(TSYSDISC) ERRDDN(TSYSERR) MAPDDN(TSYSMAP)


COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPP (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )

An extract of the job output is shown in Example 6-5. Here you can see that DB2 used 6 subtasks for building the 2 indexes in parallel (2 pairs of sort, build and 2 statistics) during the SORTBLD phase. Inline COPY and inline statistics are produced.

Example 6-5 Job output LOAD of partitioned table space with PIB; no parallelism

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..DISCARD) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA REPLACE LOG NO SORTKEYS 1200000 DISCARDDN(TSYSDISC) ERRDDN(TSYSERR) MAPDDN( TSYSMAP) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SORTOUT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.DISCARD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSERR DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSERR DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSMAP DDNAME=SYS00005 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSMAP DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSCOPY DDNAME=SYS00006 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T005804.P DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TRECOVER DDNAME=SYS00007 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T005804.R DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU395I DSNURPIB - INDEXES WILL BE BUILT IN PARALLEL, NUMBER OF TASKS = 6 DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE PAOLOR3P.PAOLOR3P NUMBER OF PAGES=26152 AVERAGE PERCENT FREE SPACE PER PAGE = 6.47 PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:00:36 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU610I -DB2G DSNUSUPT - SYSTABSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-20.58.11.634612


DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=662397 FOR TABLE PAOLOR3.EXAMPP DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=662397 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:37 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=223976 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 1 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=224000 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 2 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=214421 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 3 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPI - SYSINDEXSTATS CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-20.58.11.673853 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=662397 FOR INDEX PAOLOR3.I_EXAMPP_2 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-20.58.11.675953 DSNU391I DSNURPTB - SORTBLD PHASE STATISTICS. NUMBER OF INDEXES = 2 DSNU392I DSNURPTB - SORTBLD PHASE COMPLETE, ELAPSED TIME = 00:00:25 DSNU428I DSNURELD - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLOR3P.PAOLOR3P DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

In 6.3, “Partition parallelism within one LOAD” on page 118, we show that in DB2 V7, parallelism can be increased by also loading the partitions in parallel.

6.3 Partition parallelism within one LOADMore and more customers have to load enormous amounts of data, such as for data warehouse applications. In the meantime, the availability requirements have reduced the batch windows. For these customers, loading the data takes too long.

Prior to V7, one solution was to make the table spaces partitioned and try to load the partitions in parallel. The only way of doing this was by submitting different jobs in parallel, each job loading a different partition or set of partitions. This worked fine if there were no non-partitioned indexes (NPIs). If there were NPIs, you ended up with considerable contention problems. For that reason, many customers first dropped the NPIs and then recreated them afterwards.

In DB2 V7 an improvement has been made allowing partitions to be loaded in parallel in the same job and reducing the NPI contention. This single job will now read one input data set per partition and launch multiple subtasks, optimally one for each partition.

In order to allow each partition to be loaded from a separate data set, with discards (optionally) written to discard data sets for that partition, the LOAD syntax allows the specification of the INDDN and DISCARDDN keywords as part of the INTO TABLE PART specification.

Make sure that the individual input data sets only contain records for the corresponding partition, as all other records will be discarded.

Optimally the number of load subtasks will be equal to the number of partitions to be loaded. DB2 determines the degree of parallelism, that is, the number of load subtasks, based on the following factors:

� Number of CPUs� Available virtual storage in the batch region� Available number of DB2 threads

See also 3.7, “Considerations for running parallel subtasks” on page 67.


It is still recommended to submit multiple parallel jobs per partition if there are no NPIs. This to have the parts of the partitioned index built in parallel, which is not the case when using the new partition parallel LOAD function.

There are two ways you can deploy this partition parallel LOAD: with or without Parallel Index Build (PIB).

6.3.1 Partition parallelism without Parallel Index BuildIn this case the subtasks will read the records from the input data sets and load the partitions in parallel. They will extract the keys for the indexes and write them to the common SYSUT1 data set. After sorting, the normal BUILD phase is performed, serially loading the indexes (partitioned and non-partitioned). This sequence allows the NPI contention to be avoided. This is illustrated in Figure 6-2.

Figure 6-2 LOAD partition parallelism without Parallel Index Build

See also 3.5.1, “LOAD partition parallelism without PIB” on page 63.

In Example 6-6 we show the loading of the same partitioned table space as in Example 6-4 on page 116, but now using partition parallelism without PIB. The 3 input data sets are now allocated to a separate DD card in the JCL, one per partition. We use templates to specify the Inline COPY data sets and work data sets. We also ask for Inline RUNSTATS. The 3 partitions will be loaded in parallel and the indexes will be built serially, avoiding NPI contention. We do not specify the SORTKEYS option.

Example 6-6 Partition parallelism without PIB

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSREC1 DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00001,DISP=SHR

SYSREC1 Part 1

Part 2

Part 3

Part 4

SYSREC2

SYSREC3

PI

Error/Map

NPI1

NPI2

SYSUT1

RELOAD

RELOAD

RELOAD

RELOAD

SORT BUILD

BUILD

SORTWKnn

BUILD

key/RID pairs

SYSREC4

SORTOUT


//SYSREC2 DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00002,DISP=SHR //SYSREC3 DD DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00003,DISP=SHR /SYSIN DD * TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPP PART 1 INDDN SYSREC1 (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 2 INDDN SYSREC2 (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 3 INDDN SYSREC3 (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )

In Example 6-7 we LOAD the same table space by specifying the input data sets with a template (using the &PART variable) and adding discard processing. For this, we have to add a template for the input, discard, error, and mapping data sets.

Example 6-7 Partition parallelism without PIB using a template for input data sets

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL


TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) ERRDDN(TSYSERR) MAPDDN(TSYSMAP) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPP PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )

An extract of the job output is shown in Example 6-8. Here you can see that DB2 used 3 subtasks for loading the 3 partitions in parallel. The 2 indexes are build serially. Inline COPY and inline statistics are produced.

Example 6-8 Job output partition parallelism without PIB

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA REPLACE LOG NO COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) ERRDDN(TSYSERR) MAPDDN( TSYSMAP) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER,


DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00005 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00003 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00006 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00003 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00007 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00008 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SORTOUT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSERR DDNAME=SYS00009 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSERR DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSMAP DDNAME=SYS00010 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSMAP DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSCOPY DDNAME=SYS00011 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T012408.P DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TRECOVER DDNAME=SYS00012 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T012408.R DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU364I DSNURPPL - PARTITIONS WILL BE LOADED IN PARALLEL, NUMBER OF TASKS = 3 DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE PAOLOR3P.PAOLOR3P NUMBER OF PAGES=26148 AVERAGE PERCENT FREE SPACE PER PAGE = 6.46 PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:00:34 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU610I -DB2G DSNUSUPT - SYSTABSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.24.16.038794 DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.24.16.086631 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU610I -DB2G DSNUSUPT - SYSTABSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.24.16.256441 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=223976 FOR TABLE PAOLOR3.EXAMPP PART=1 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=224000 FOR TABLE PAOLOR3.EXAMPP PART=2 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=214421 FOR TABLE PAOLOR3.EXAMPP PART=3 DSNU428I DSNURILD - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLOR3P.PAOLOR3P DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=662397 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:34 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=1324794 ELAPSED TIME=00:00:09 DSNU348I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=223976 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 1 DSNU348I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=224000 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 2 DSNU348I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=214421 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 3 DSNU349I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=662397 FOR INDEX PAOLOR3.I_EXAMPP_2 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPI - SYSINDEXSTATS CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL


DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.24.53.477255 DSNU258I DSNURBXD - BUILD PHASE STATISTICS - NUMBER OF INDEXES=2 DSNU259I DSNURBXD - BUILD PHASE COMPLETE, ELAPSED TIME=00:00:31 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

6.3.2 Partition parallelism with Parallel Index Build In this case, having specified SORTKEYS, the subtasks will also read the records from the input data sets and load the partitions in parallel. But the indexes will also be built in parallel by multiple pairs of sort and build subtasks using PIB (optimally one pair per index). This is illustrated in Figure 6-3.

Figure 6-3 LOAD partition parallelism with Parallel Index Build

As already explained in 6.2, “Parallel Index Build” on page 111, this is done in the new SORTBLD phase, which replaces the SORT and BUILD phases. Moreover, the SYSUT1 and SORTOUT work data sets are no longer used, because the key/RIDs pairs are now directly piped in memory between subtasks, thus eliminating considerable I/O processing. Because each NPI is built by its own build task, NPI contention is eliminated.

Partition parallelism with PIB is activated by the SORTKEYS n option, n being an estimation of the number of keys to be sorted. If you specify n equals to zero, PIB will not be activated.

DB2 V5 introduced the SORTKEYS option to use Sort and eliminate multiple I/O accesses to workfiles when building the indexes. DB2 V6 used the same option to activate PIB as well. In V7 this feature can also be used in combination with partition parallel LOAD.

See also 3.5.2, “LOAD partition parallelism with PIB” on page 64.

S Y SR E C 1 P a rt 1

P a r t 2

P a rt 3

P a r t 4

S Y SR E C 2

S Y SR E C 3

S Y SR E C 4

P IS O R TB U IL D

S O R TB U IL D

S O R TB U IL D

R E L O A D

R E L O A D

R E L O A D

R E L O A D

S W 0 1 W K x x

S W 0 2 W K x x

S W 0 3 W K x x

S W 0 1 W K n n

S W 0 2 W K n n

S W 0 3 W K n n

E rro r/M a p

N P I1

N P I2

S O R T B L D

S O R T B L D

S O R T B L D


In Example 6-9 we show the loading of the same partitioned table space, as we did in Example 6-4 on page 116, but now using partition parallelism with PIB. The 3 input data sets are each dedicated to a partition using a template with the &PART variable. We also use templates to specify the Inline COPY data sets and work data sets. We also ask for Inline RUNSTATS. The 3 partitions will be loaded in parallel and the indexes will be built in parallel as well. There will be no NPI contention. We activated the PIB by specifying SORTKEYS 1200000.

Example 6-9 Partition parallelism with PIB

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS 1200000 COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) ERRDDN(TSYSERR) MAPDDN(TSYSMAP) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPP PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )


An extract of the job output is shown in Example 6-10. Here you can see that DB2 used 3 subtasks for loading the 3 partitions in parallel. DB2 used also 6 subtasks for building the 2 indexes in parallel (2 pairs of sort, build and 2 statistics) during the SORTBLD phase. Inline COPY and inline statistics are produced.

Example 6-10 Job output partition parallelism with PIB

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..P) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME..R) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA REPLACE LOG NO SORTKEYS 1200000 COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) ERRDDN( TSYSERR) MAPDDN(TSYSMAP) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMPP PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00005 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.U00003 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00006 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D00003 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00007 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00008 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SORTOUT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSERR DDNAME=SYS00009 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSERR DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSMAP DDNAME=SYS00010 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.SYSMAP DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSCOPY DDNAME=SYS00011 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T014603.P DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TRECOVER DDNAME=SYS00012 DSN=PAOLOR3.PAOLOR3P.PAOLOR3P.D2001167.T014603.R


DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU395I DSNURPIB - INDEXES WILL BE BUILT IN PARALLEL, NUMBER OF TASKS = 6 DSNU364I DSNURPPL - PARTITIONS WILL BE LOADED IN PARALLEL, NUMBER OF TASKS = 3 DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE PAOLOR3P.PAOLOR3P NUMBER OF PAGES=26148 AVERAGE PERCENT FREE SPACE PER PAGE = 6.46 PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:00:41 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU610I -DB2G DSNUSUPT - SYSTABSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.46.11.417150 DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.46.11.554041 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU610I -DB2G DSNUSUPT - SYSTABSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR PAOLOR3P.PAOLOR3P SUCCESSFUL DSNU620I -DB2G DSNURDRT - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.46.11.803353 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=223976 FOR TABLE PAOLOR3.EXAMPP PART=1 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=224000 FOR TABLE PAOLOR3.EXAMPP PART=2 DSNU303I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=214421 FOR TABLE PAOLOR3.EXAMPP PART=3 DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=662397 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:41 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=223976 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 1 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=224000 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 2 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=214421 FOR INDEX PAOLOR3.I_EXAMPP_1 PART 3 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPI - SYSINDEXSTATS CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUPC - SYSCOLSTATS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_1 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.46.11.380239 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=662397 FOR INDEX PAOLOR3.I_EXAMPP_2 DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR PAOLOR3.EXAMPP SUCCESSFUL DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR3.I_EXAMPP_2 SUCCESSFUL DSNU620I -DB2G DSNURDRI - RUNSTATS CATALOG TIMESTAMP = 2001-06-15-21.46.11.380753 DSNU391I DSNURPTB - SORTBLD PHASE STATISTICS. NUMBER OF INDEXES = 2 DSNU392I DSNURPTB - SORTBLD PHASE COMPLETE, ELAPSED TIME = 00:00:25 DSNU428I DSNURELD - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLOR3P.PAOLOR3P DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

In Example 6-11 we show how to use templates to create inline copies on the partition level instead of having one single Inline COPY data set on the table space level. This is done to avoid contention on the single Inline COPY data set from the different load subtasks.

Example 6-11 Partition parallelism with inline copies on the partition level

TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSMAP DSN(PAOLOR3.&DB..&TS..SYSMAP) SPACE(10,10) CYL TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..T&TIME..P&PART.) SPACE(100,10) CYL

Note: All examples shown in 6.2, “Parallel Index Build” on page 111 and 6.3, “Partition parallelism within one LOAD” on page 118, were run on a system that was not tuned for optimal performance. They are shown for example only. Actions for tuning I/O and buffer pools are needed to avoid bottlenecks and achieving maximum parallelism if you start using Parallel Index Build and Partition Parallelism.


TEMPLATE TRECOVER DSN(PAOLOR3.&DB..&TS..T&TIME..R&PART.) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA LOG NO SORTKEYS 1200000 ERRDDN(TSYSERR) MAPDDN(TSYSMAP) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS TABLE(ALL) INDEX(ALL KEYCARD) INTO TABLE PAOLOR3.EXAMPP PART 1 REPLACE INDDN(TSYSREC) DISCARDDN(TSYSDISC) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 2 REPLACE INDDN(TSYSREC) DISCARDDN(TSYSDISC) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMPP PART 3 REPLACE INDDN(TSYSREC) DISCARDDN(TSYSDISC) COPYDDN(TSYSCOPY) RECOVERYDDN(TRECOVER) (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )

6.4 PreformatThe PREFORMAT option of the LOAD utility was introduced in DB2 V5. It will preformat a table space and its associated index spaces during LOAD time and prepare it for INSERT processing. It reduces execution time delays due to formatting and eliminates the need to preformat new pages in a table space during INSERT processing. However, when the preformatted space is utilized and DB2 has to extend the table space, normal data set extending and preformatting will occur.

Preformatting causes all free pages in the VSAM cluster (between the high used RBA and high allocated RBA) to be preformatted. This includes secondary extents that may have been allocated. Preformatting occurs after the data has been loaded and the indexes are built.

Preformatting can operate on the entire table space and its index spaces, or on a partition of a partitioned table space and its corresponding partition index space. Note that preformatting an entire partitioned table space at the table space level can inhibit concurrent processing of separate partitions. In this case you should specify PART integer PREFORMAT to serialize on the partition level, as illustrated in Example 6-12.


Example 6-12 Preformatting a table space and its index spaces during LOAD

Preformatting an entire table space and associated index spaces: LOAD DATA PREFORMAT INTO TABLE table name

Preformatting a table space partition and associated index partition:LOAD DATA INTO TABLE table name PART 1 PREFORMAT

Preformatting is only recommended for table spaces and index spaces that are part of high INSERT applications where normal formatting can cause unacceptable delays or inconsistent elapsed times. It is not recommended for:

� Tables that are only populated by the LOAD REPLACE utility (for example, table spaces that are refreshed every day with LOAD REPLACE). In that case, PREFORMAT will only cause an additional elapsed time of the LOAD processing.

� Tables that are mainly used for query processing. The additional preformatted pages can cause table space scans to read additional empty pages, extending the elapsed time for these queries.

Use the PREFORMAT option with care. General use of this option in all your LOAD jobs is not recommended. The best results may be seen when the final size of the table space is known, and when there is a high ratio of inserts to read operations.

Preformatting can only be used with LOAD SHRLEVEL NONE.

6.5 ReuseREUSE is a feature of the LOAD utility introduced in DB2 V5. It can only be used with the LOAD REPLACE option and with STOGROUP defined data sets, also called DB2-managed data sets. Normally, if you do not specify REUSE, LOAD REPLACE of a DB2-managed data set will delete and redefine the underlying VSAM data sets. With the REUSE option, the underlying VSAM data sets will not be deleted and redefined, but only logically reset; that means the high used RBA is reset back to zero.

The logical reset time of a VSAM data set is much shorter than the physical delete and redefine time. Savings of 66% in elapsed time and 30% in CPU time have been measured on a LOAD REPLACE of an empty table space with 128 partitions. But bear in mind that the logical reset of a VSAM data set will not reclaim any secondary extents.

The REUSE option can operate on the entire table space and its index spaces, or on a partition of a partitioned table space and its corresponding partition index space. In the last case you should specify PART integer REPLACE REUSE as illustrated in Example 6-13.

Example 6-13 Specifying the REUSE option during LOAD REPLACE

Logically reset and reuse of an entire table space and associated index spaces: LOAD DATA REPLACE REUSE INTO TABLE table name

Logically reset and reuse of a table space partition and associated index partition:LOAD DATA INTO TABLE table name PART 1 REPLACE REUSE

Note: DB2 V7 introduced asynchronous INSERT preformatting. This new function will asynchronously preformat the next range of new pages during INSERT processing if the current page is within a predefined range from the end of the formatted pages.


We recommend the use of REUSE option:

� If you want to preserve the allocated space on the disk volumes between consecutive LOAD REPLACE operations (for I/O tuning reasons or to avoid space problems in case of reallocation).

� To reduce CPU and elapsed times, if you have partitioned table spaces with a lot of underlying VSAM data sets.

� To increase the throughput by avoiding service task contention, if you have a lot of concurrent LOAD REPLACE utilities run in parallel. Since DB2 V6, DB2 can have up to 20 service tasks run in parallel.

Do not use the REUSE option if you want to do any of the following:

� Reduce the number of secondary extents.

� Activate altered PRIQTY and SECQTY values.

� Move the data sets to another STOGROUP.

6.6 Cross LoaderThe Cross Loader is a new capability in DB2 V7 that allows you to transfer data from one location to another location within a single utility job. Refer to Figure 6-4.

Figure 6-4 DB2 family Cross Loader

The data is read from the source location with a dynamic SQL statement and loaded in a table at the target location by the LOAD utility. It is a single job process that replaces the typical sequence of jobs of unloading, file transfer, and loading the data. The source data can be on the local system, on a remote DRDA server, or on any system accessible via Data Joiner.

The Cross Loader was introduced in DB2 V7 after GA with PTF UQ55541 for APAR PQ45268 and PTF UQ55542 for APAR PQ46759. Therefore, you should check if these PTFs, and all their prerequisites, are applied to your DB2 system before trying to use the Cross Loader.

DB2 Family Cross Loader

SELECTLOADLOADLOCAL DB2,LOCAL DB2,

DRDA, orDRDA, orDataJoinerDataJoinerDataData

ConversionConversion

DB2 forDB2 forOS/390OS/390and z/OSand z/OS

DB2 familyDB2 familyOracleOracleSybaseSybaseInformixInformixIMSIMSVSAMVSAMSQL ServerSQL ServerNCR TeradataNCR Teradata


The Cross Loader is a new function of the LOAD utility that will be made available to all members of the DB2 family. Currently it is only available in DB2 V7 for OS390 and z/OS, but the source data can reside on any remote system that can act as a DRDA server.

A typical Cross Loader example consists of the definition of the dynamic SQL statement via the EXEC SQL DECLARE CURSOR utility statement, followed by a LOAD utility statement referring to this cursor. This is illustrated in Example 6-14. In this example we LOAD an existing summary table called EMPSUMMARY with data coming from the local sample table DSN8710.EMP. The aggregation is done in the SQL statement of the CURSOR definition.

Example 6-14 Simple Cross Loader example

EXEC SQLDECLARE C1 CURSOR FOR SELECT JOB,MAX(SALARY) AS MAX_SAL,MIN(SALARY) AS MIN_SAL FROM DSN8710.EMP GROUP BY JOB ENDEXEC

LOAD DATA REPLACEINCURSOR C1INTO TABLE EMPSUMMARY

EXEC SQL is a new DB2 V7 utility statement explained in more detail in 6.6.1, “Using SQL statements in the utility input stream” on page 130

INCURSOR is a new DB2 V7 LOAD option explained in more detail in 6.6.2, “Using the Cross Loader” on page 136

6.6.1 Using SQL statements in the utility input streamDB2 V7 gives you the ability to execute dynamic SQL statements within the utility input stream. This is done via the new EXEC SQL utility control statement.

EXEC SQL utility control statementEXEC SQL is a new utility statement placed anywhere in the utility input stream. It can be used for two purposes:

� Executing a non-select dynamic SQL statement before, between or after the actual utility statements.

� Declaring a cursor with a SQL select statement for use with the LOAD utility (Cross Loader). The declare cursor produces a result table.

The EXEC SQL statement requires no extra privileges other than the ones required by the SQL statements itself.

Executing a non-select dynamic SQL statementExecuting a non-select dynamic SQL statement is done by putting it between the EXEC SQL and ENDEXEC keywords in the utility input stream:

EXEC SQL non-select dynamic SQL statement ENDEXEC


You can only put one SQL statement between the EXEC SQL and ENDEXEC keywords. The SQL statement can be any dynamic SQL statement that can be used as input for the EXECUTE IMMEDIATE statement, as listed in Example 6-15.

Example 6-15 List of dynamic SQL statements

CREATE,ALTER,DROP a DB2 objectRENAME a DB2 tableCOMMENT ON,LABEL ON a DB2 table, view, or columnGRANT,REVOKE a DB2 authority

DELETE,INSERT,UPDATE SQL operationLOCK TABLE operation

EXPLAIN a SQL statement

SET CURRENT register

COMMIT,ROLLBACK operation

(See also “Thread behavior and commit scope of the EXEC SQL utility statement” on page 133).

In Example 6-16 we create a new table in the default database DSNDB04 with the same layout as SYSIBM.SYSTABLES.

Example 6-16 Create a new table with the same layout as SYSIBM.SYSTABLES

EXEC SQL CREATE TABLE PAOLOR3.SYSTABLES LIKE SYSIBM.SYSTABLESENDEXEC

In Example 6-17 we give this table a comment and allow everybody read access. Note the two separate steps.

Example 6-17 Comment and grant

EXEC SQL COMMENT ON TABLE PAOLOR3.SYSTABLES IS 'SAMPLE CATALOG EXTRACT'ENDEXEC EXEC SQL GRANT SELECT ON PAOLOR3.SYSTABLES TO PUBLIC ENDEXEC

In the same way, we are able to create indexes on this table, create views on it, and so on. All this is done in the utility input stream.

Declaring a cursor for use with the Cross LoaderThe cursor definition has to be put between the EXEC SQL and ENDEXEC keywords in the utility input stream:

EXEC SQL DECLARE cursor-name CURSOR FOR select statement ENDEXEC


In Example 6-18 we declare a cursor for extracting rows from SYSIBM.SYSTABLES corresponding with the 100 most recently created tables. In V7 we can do this using the new SQL fetch-first-clause.

Example 6-18 Declare a cursor for the Cross Loader

EXEC SQL DECLARE C1 CURSOR FOR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ORDER BY CREATEDTS DESC FETCH FIRST 100 ROWS ONLY ENDEXEC

In Example 6-19 we show how this cursor can now be used in a LOAD statement (Cross Loader).

Example 6-19 Usage of a cursor in the LOAD statement

LOAD DATA INCURSOR C1 INTO TABLE PAOLOR3.SYSTABLES

The SQL statement in the declare cursor definition can be any valid SQL statement including joins, unions, data conversions, aggregations, special registers, and UDFs. The source data can be on a local server or remote server using DRDA access or Data Joiner. See 6.6.2, “Using the Cross Loader” on page 136 for more details.

Error behavior and restartability of the EXEC SQL utility statementThe EXEC SQL utility statements are executed in a new utility phase called the EXEC phase.

The SQL statement placed after the EXEC SQL keyword is parsed and checked for errors during its execution. It is not checked during the UTILINIT phase of the utility. If an invalid SQL statement is found during the execution of the EXEC SQL, the utility job immediately ends with return code 8. If the SQL statement is valid, but fails during execution (with a negative SQL code), the utility also immediately ends with return code 8 as well. So be aware that if you have syntax errors in an EXEC SQL statement and the utility job gets started, the previous EXEC SQL statements and utility statements are already executed before the utility ends. You might have to remove these from the input stream before rerunning the job.

Normally, a utility that encounters an SQL error during the EXEC SQL statement execution always ends with return code 8 and never abends with ABEND04E. So the utility is not in a stopped state, the utility is not restartable with the RESTART option and a TERMINATE UTILITY command is NOT necessary. But be aware that all previous EXEC SQL and utility statements are executed successfully and might have to be removed first before rerunning the job.

Currently it is impossible to influence the behavior of the utility job after a failing EXEC SQL statement. An OPTION to allow to discard the failing EXEC SQL, and to continue the utility step when the EXEC SQL failed, is currently not available in DB2 V7.


If the utility input stream contains both EXEC SQL statements and other utility statements, and a utility statement failed so that DB2 put the utility in the stopped state, the utility step is restartable with the RESTART keyword. During restart, all the non-select dynamic SQL statements from EXEC SQL statements already executed, are skipped, except the ones with SQL SET statements. All the declare cursor definitions within EXEC SQL statements already executed, are reactivated so that they can be used in the following LOAD statements.

This can be illustrated with Example 6-20. This job contains one non-select dynamic SQL statement (CREATE TABLE), one cursor definition, and one utility LOAD statement. If the CREATE TABLE fails with a negative SQL code, the utility will immediately end with return code 8 and the utility will not be restartable with the RESTART keyword. If the CREATE TABLE executes successfully, but the DECLARE CURSOR fails, the utility will also end with return code 8, but the table will have been created. If both CREATE TABLE and DECLARE CURSOR execute successfully, but the LOAD statement fails so that DB2 puts the utility in the stopped state (for example because of a resource unavailable condition) the utility will be restartable. During restart the CREATE TABLE statement will be skipped but the DECLARE CURSOR statement will be re-executed, so that it can be used in the LOAD statement.

Example 6-20 Restartability of the EXEC SQL statement

EXEC SQL CREATE TABLE MYTABLE LIKE SYSIBM.SYSTABLES ENDEXEC EXEC SQL DECLARE C1 CURSOR FOR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ORDER BY CREATEDTS DESC FETCH FIRST 100 ROWS ONLY ENDEXEC LOAD DATA INCURSOR C1 INTO TABLE MYTABLE

Thread behavior and commit scope of the EXEC SQL utility statementThe EXEC SQL statement runs in a thread that is separate from the utility thread. This implies that the EXEC SQL SET CURRENT register operations do influence all following EXEC SQL statements in the utility stream input, but they do not influence the real utility statements like LOAD, REORG, and so on.

We can illustrate this with the utility job in Example 6-21. This job runs with USER=PAOLOR3. So the DB2 primary AUTHID is PAOLOR3. We change the current SQLID with EXEC SQL to PAOLOR4 and try to see what table creator is used if we do not prefix our tables in the following EXEC SQL statements and other utility statements.

Example 6-21 JCL for testing the thread behavior of EXEC SQL

//PAOLOR3A JOB (ACCOUNT),'PAOLOR3',NOTIFY=PAOLOR3,USER=PAOLOR3 //* // EXEC PGM=DSNUTILB,PARM='DB2G,MEPL' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSIN DD * EXEC SQL SET CURRENT SQLID = 'PAOLOR4' ENDEXEC EXEC SQL CREATE TABLE MYTABLE LIKE SYSIBM.SYSTABLES


ENDEXEC EXEC SQL DECLARE C1 CURSOR FOR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ORDER BY CREATEDTS DESC FETCH FIRST 100 ROWS ONLY ENDEXEC LOAD DATA INCURSOR C1 INTO TABLE PAOLOR4.MYTABLE LOAD DATA RESUME(YES) INCURSOR C1 INTO TABLE MYTABLE

The resulting job output is shown in Example 6-22.

In the first EXEC SQL, the current SQLID is set to PAOLOR4. In the following EXEC SQL, the table MYTABLE is created without specifying a table creator. We verified in the DB2 catalog that this table is created with CREATOR = PAOLOR4, which proves that the previous current SQLID was used in this EXEC SQL. If we try to LOAD this table with the Cross Loader we have to specify PAOLOR4.MYTABLE otherwise the LOAD fails, because the LOAD utility thread does not use the current SQLID set in the previous EXEC SQL. Instead, its current SQLID is still equal to the primary AUTHID, PAOLOR3, which it inherited from the USER keyword in the JCL.

Example 6-22 Testing the thread behavior of EXEC SQL

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = MEPL DSNU050I DSNUGUTC - EXEC SQL SET CURRENT SQLID='PAOLOR4' ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU050I DSNUGUTC - EXEC SQL CREATE TABLE MYTABLE LIKE SYSIBM.SYSTABLES ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU050I DSNUGUTC - EXEC SQL DECLARE C1 CURSOR FOR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE='T' ORDER BYCREATEDTS DESC FETCH FIRST 100 ROWS ONLY ENDEXEC DSNU050I DSNUGUTC - LOAD DATA INCURSOR C1 DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR4.MYTABLE DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=100 FOR TABLE PAOLOR4.MYTABLE DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=100 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU050I DSNUGUTC - LOAD DATA RESUME(YES) INCURSOR C1 DSNU650I -DB2G DSNURWI - INTO TABLE MYTABLE DSNU056I -DB2G DSNUGMAP - TABLE 'PAOLOR3.MYTABLE' NOT FOUND DSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

Each block of EXEC SQL and ENDEXEC is a separate unit-of-work. Each block can only contain a single SQL statement. The unit-of-work is always committed when the SQL statement is executed successfully. Although the COMMIT and ROLLBACK statements are accepted, these statements do not perform any function.

So it is impossible to create your own unit-of-work consisting of multiple EXEC SQL statements and end that unit-of-work with EXEC SQL COMMIT ENDEXEC. It is also impossible to have an EXEC SQL statement executed and undo its work by a following EXEC SQL ROLLBACK ENDEXEC command.


We verified the above behavior with the utility job shown in Example 6-23. In the first EXEC SQL we create a new table. This is followed by an EXEC SQL ROLLBACK to try to undo the CREATE statement. In the third EXEC SQL we populate this table with an INSERT statement and try to undo the INSERT in the fourth EXEC SQL. If the ROLLBACK statement in the second EXEC SQL would undo the CREATE, we expect the third EXEC SQL to fail.

Example 6-23 JCL for verifying the commit scope of EXEC SQL

//PAOLOR3A JOB (ACCOUNT),'PAOLOR3',NOTIFY=PAOLOR3,USER=PAOLOR3 // EXEC PGM=DSNUTILB,PARM='DB2G,MEPL' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSIN DD * EXEC SQL CREATE TABLE PAOLOR3.SYSTABLESR LIKE SYSIBM.SYSTABLES ENDEXEC EXEC SQL ROLLBACK ENDEXEC EXEC SQL INSERT INTO PAOLOR3.SYSTABLESR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' AND CREATOR LIKE 'PAOLOR%' ENDEXEC EXEC SQL ROLLBACK ENDEXEC

The result of this job is shown in Example 6-24. As you can see, all four EXEC SQL statements executed successfully. We verified with SPUFI that the table PAOLOR3.SYSTABLESR was successfully created and populated with 35 rows. So the EXEC SQL ROLLBACK statements did not influence the previous EXEC SQL statements.

Example 6-24 Verifying the commit scope of EXEC SQL

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = MEPL DSNU050I DSNUGUTC - EXEC SQL CREATE TABLE PAOLOR3.SYSTABLESR LIKE SYSIBM.SYSTABLES ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU050I DSNUGUTC - EXEC SQL ROLLBACK ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU050I DSNUGUTC - EXEC SQL INSERT INTO PAOLOR3.SYSTABLESR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE='T' AND CREATOR LIKE 'PAOLOR%' ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU1196I DSNUGSQL - SQLERRD = 0 0 35 1127790002 0 0 SQL DIAGNOSTIC INFORMATION DSNU1196I DSNUGSQL - SQLERRD = X'00000000' X'00000000' X'00000023' X'4338B5B2' X'00000000' X'00000000' SQL DIAGNOSTIC INFORMATION DSNU050I DSNUGUTC - EXEC SQL ROLLBACK ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

So, although you can code COMMIT and ROLLBACK statements after the EXEC SQL keyword, they do not influence the behavior of previous EXEC SQL commands, and their use seems to be meaningless in this context.


Possible usage of the EXEC SQL utility statementThe primary use of the EXEC SQL utility statement is meant for declaring cursors for use with the LOAD utility (Cross Loader). But it can also be used to execute any non-select dynamic SQL statement before, between or after regular utility statements. Examples are:

� DDL creation of the target table for the LOAD utility (and its related objects like database, table space, indexes, views)

� DDL creation of the mapping table and index before a REORG table space SHRLEVEL CHANGE

� Dropping of the mapping table after a successful REORG table space SHRLEVEL CHANGE

� DDL alter of space related values like PRIQTY, SECQTY, FREEPAGE and PCTFREE values before a REORG or LOAD utility

� DDL alter of INDEX partitions before REORG (for partitioning key changes)

� GRANT statements (example: grant select authorities after successful LOAD)

� SQL delete of “old” records before LOAD RESUME YES

� SQL update of an application related control table or SQL insert into an application related control table after successful LOAD (with the current timestamp)

EXEC SQL eliminates additional job steps in a job to execute dynamic SQL statements before, between or after the regular utility steps. It can simplify the JCL coding by eliminating this dynamic SQL applications like DSNTIAD or DSNTEP2 from the JCL stream and it enables to merge different utility steps, separated by dynamic SQL applications, into one single utility step. But be aware of the restrictions imposed by the EXEC SQL statement.

6.6.2 Using the Cross LoaderAs already explained, two steps are necessary to use the Cross Loader:

1. Declare a cursor with the EXEC SQL statement.2. LOAD the data into the target table using above cursor

Benefits of EXEC SQL:

� You can execute any non-select dynamically preparable SQL statement within the utility input stream.

� You can declare cursors for use with the LOAD utility, including joins, unions, conversions, aggregations, and remote DRDA access.

� Successfully executed SQL statements are skipped during restart of the utility.

� In many cases, the need for extra dynamic SQL programs in the utility job stream is eliminated.

� Considerable simplification of JCL is possible.

Restrictions of EXEC SQL:

� There are no select statements.� There is no control after error: the whole utility step stops after the first SQL error.� There is no concept of unit-of-work consisting of multiple SQL statements.� There are no comments possible between SQL statements.


Declaring a cursor for use with the Cross LoaderThe following rules apply to the cursor you DECLARE in the EXEC SQL statement:

� You must always declare the cursor before the LOAD statement. Otherwise you get a message DSNU1190I CURSOR NOT DECLARED.

� The cursor name can only be declared once within the whole utility input stream. It can be referred in multiple LOAD statements for LOADING different target tables with the same data. If you declare an existing cursor, you get the error message DSNU1189I CURSOR ALREADY DECLARED.

� The table being loaded cannot be part of the select statement. So you cannot LOAD into the same table where you defined the cursor.

� The column names in the result set of the select statement must be identical to the column names in the table being loaded. This can be achieved by using the AS clause in the SQL statement. If you have column names in the result set which do not match any column name in the target table you get the error message DSNU053I FIELD 'colname' NOT FOUND or the error message DSNU329I FIELD 'colnumber' IS NOT DEFAULTABLE. Pay special attention to derived column names which are the result of column functions such as SUM or AVG.

� You are able to skip unwanted columns in the result set with the LOAD IGNOREFIELDS YES option, which skips any columns in the cursor result set that are not present in the target table being load. However use this IGNOREFIELDS option with care, as it also skips misspelled columns you wanted to LOAD.

� The sequence of the columns in the result set may differ from the sequence of the columns in the table being loaded. DB2 matches the columns by their names and not by their sequence.

� The number of columns in the cursor can be less than the number of columns in the target table. All missing columns are loaded with their default values. If the missing columns are defined in the target table as NOT NULL without default, the LOAD fails with message DSNU323I COLUMN 'colname' IS OMITTED.

� If the data types in the target table do not match with the data types in the cursor, DB2 tries to convert as much as possible between compatible data types. Examples are from CHAR to VARCHAR or from INTEGER to FLOAT. If the conversion fails, you get messages like DSNU390I INVALID CONVERSION FOR FIELD ‘columnname’ (conversion error detected before the actual LOAD during the matching process) or DSNU334I INPUT FIELD 'columnname' INVALID FOR 'tablename', ERROR CODE cc (conversion error detected during the LOAD). You might use DB2 supplied built-in functions or own developed UDFs in the SQL statement to force more sophisticated conversions. An example is the CHAR function which allows you to convert from INTEGER to CHARACTER.

� If the encoding scheme of the source data in the cursor and the target table differ, DB2 automatically converts the encoding schemes. An example may be conversion from EBCDIC to UNICODE or from ASCII to EBCDIC. Remember that referencing tables with more than one encoding scheme (ASCII,EBCDIC or UNICODE) in a single SQL statement is not supported and finishes with SQLCODE -873.

� If the target table contains UDTs, you do not have to specify the appropriate casting functions in the cursor SQL statement to LOAD the table. If the source data in the cursor contains UDTs, you also do not have to specify casting functions in the select list of the cursor SQL statement. But additional WHERE clauses in the cursor SQL statement might require casting functions, or you could end up with SQLCODE -401.

� If your table contains LOB columns, the maximum length is 32 KB. You cannot use the Cross Loader if you want to transfer LOB columns larger than 32 KB. Instead you should use a program with embedded SQL. If a table contains LOB columns, there is at least one


ROWID column. If this ROWID column is created with the GENERATED ALWAYS clause (the recommended default) you cannot specify this column in the select list of the cursor. Instead DB2 generates the target ROWID value during loading. If you do specify the ROWID column in the select list you get message DSNU269I FIELD columnname IS NOT ALLOWED. If the ROWID column is created with the GENERATED BY DEFAULT clause, you may specify this column in the select list. In this case the source ROWID value is copied. Do not forget to create a unique index on the ROWID column if you specified GENERATED BY DEFAULT or you fail with error message DSNU309I NOT ALL REQUIRED UNIQUE INDEXES HAVE BEEN DEFINED FOR TABLE tablename.

� Apart from the aforementioned rules, the SQL statement in the declare cursor definition can be any valid SQL statement including joins, unions, conversions, aggregations, special registers, UDFs. The source data can be local or remote using DRDA access or Data Joiner. Remote tables are always referred by their three-part-name LOCATION.CREATOR.NAME or by an ALIAS name CREATOR.NAME. You cannot use an explicit CONNECT statement to connect to the remote location.

Binding the Cross Loader packageThe Cross Loader uses package DSNUGSQL in collection DSNUTIL that must be bound local and at the remote DRDA servers you want to transfer data from. Locally it must be bound with the option DBPROTOCOL(DRDA) to allow the DRDA protocol to use three-part-names. It uses the default system utility plan DSNUTIL. The bind command on the local system is done in installation job DSNTIJSG. You can bind the package on a remote DRDA system with a bind command like:

BIND PACKAGE (remote-location-name.DSNUTIL) - COPY(DSNUTIL.DSNUGSQL) COPYVER(V7R1) -CURRENTDATA(NO) ISOLATION(CS) -ACTION(ADD) OPTIONS(COMMAND)

LOAD the data into the target table using a cursorCross loading is done using the new option INCURSOR cursor. With this option you tell the LOAD to get the data from the result set of the cursor instead of getting it from a sequential data set referred by the DD statement or template INDDN (with default SYSREC).

You can use any option of the LOAD utility except the following:

� SHRLEVEL CHANGE: There is no support for online LOAD in the Cross Loader. If you specify SHRLEVEL CHANGE, you get the message DSNU070I KEYWORD OR OPERAND 'SHRLEVEL CHANGE' INVALID WITH INCURSOR.

� FORMAT: You cannot specify the UNLOAD or SQL/DS formats.

� FLOAT(IEEE): The cursor always returns FLOAT(S390).

� EBCDIC,ASCII,UNICODE: The Cross Loader always automatically converts the encoding schemes. The target table must be created with the correct CCSID.

� NOSUBS: You must accept substitution characters in a string.

� CONTINUEIF: You cannot treat each input record as a part of a larger record.

� WHEN: You cannot use the WHEN option to filter the result set of the cursor. Instead, filter the result set by using the appropriate WHERE clauses in the select statement.

� Field specifications: You cannot specify field specifications in the LOAD Statement. If you specify field specifications, you get the message DSNU331I FIELD LISTS ARE NOT ALLOWED WITH INCURSOR KEYWORD.


The same cursor can be reused multiple times in the utility job step. It can be referenced in different LOAD statements to LOAD the same data in different target tables. It can also be reused in one LOAD statement containing multiple INTO TABLE clauses to LOAD the same data in different target tables (in the same table space).

If a cursor is used more than once in the utility job step, the data is transferred more than once as well. Each time you refer to a cursor name in a LOAD statement, the SQL statement is re-executed. There is no buffering nor reuse of the previous transferred data. So the result sets can differ if the source data is being updated or if you use time dependent functions like CURRENT TIMESTAMP.

It is recommended that you LOAD your data in clustering sequence. If loading from a sequential data set this can be done by first sorting the sequential data set in clustering sequence. With the Cross Loader this can be achieved by sorting the result set of the cursor by using an ORDER BY statement on the clustering columns.

You can LOAD partitions in parallel by specifying a different cursor for each partition.

Cross Loader examplesExample 6-25 is a simple example of using the Cross Loader, where the columns in the cursor match exactly the columns in the target table (same column names, same column types, same order). The target table PAOLOR3.EXAMP1 is created in the default database DSNDB04 using the EXEC SQL clause. The table space is implicitly created. We use the Cross Loader to create a subset of the catalog table SYSIBM.SYSTABLES. We want to LOAD without logging and an inline image copy to be taken to prevent the table space to be put in the restrictive COPYP status. We use a template for the image copy data set. We have to provide a SPACE parameter because DB2 cannot estimate the space parameters of a new table space.

Example 6-25 Cross Loader example with matching columns

EXEC SQL CREATE TABLE PAOLOR3.EXAMP1 (CREATOR CHAR(8), NAME VARCHAR(18), DBNAME CHAR(8), TSNAME CHAR(8)) ENDEXEC EXEC SQL DECLARE C1 CURSOR FOR SELECT CREATOR, NAME, DBNAME, TSNAME FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL LOAD DATA REPLACE LOG NO INCURSOR C1 COPYDDN(TSYSCOPY) INTO TABLE PAOLOR3.EXAMP1


In Example 6-26 we show how the SQL AS clause can be used to match the column names in the cursor with the column names in the target table. Note that the order of the columns in the cursor select and in the table are different. Conversions from CHAR to VARCHAR are done automatically for the CHAR fields in the cursor. We must use the CHAR function to convert the OBID field from INTEGER to CHAR. The first column of the table is a TIMESTAMP column that is not present in the select statement and gets populated with its default value being CURRENT TIMESTAMP. Apart from an inline image copy we also want to collect STATISTICS. You should use the SQL AS clause for every derived column name that is the result of columns functions (SUM,MAX,...) or UDFs.

Example 6-26 Cross Loader example with AS clause and default columns

EXEC SQL CREATE TABLE PAOLOR3.EXAMP2 (COL1 TIMESTAMP NOT NULL WITH DEFAULT, COL2 VARCHAR(18), COL3 VARCHAR(18), COL4 VARCHAR(18), COL5 VARCHAR(18), COL6 VARCHAR(18)) ENDEXEC EXEC SQL DECLARE C2 CURSOR FOR SELECT DBNAME AS COL4, TSNAME AS COL5, CREATOR AS COL2, NAME AS COL3, CHAR(OBID) AS COL6 FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL LOAD DATA REPLACE REUSE LOG NO INCURSOR C2 COPYDDN(TSYSCOPY) STATISTICS INTO TABLE PAOLOR3.EXAMP2

In Example 6-27 we show the use of the IGNOREFIELDS YES LOAD option to ignore columns in the cursor that are not present in the target table. We also created a clustering unique index on the target table PAOLOR3.EXAMP3. We added the ORDER BY clause in the cursor definition to LOAD the table in clustering sequence. We have to add templates TSYSUT1 and TSORTOUT for the WORKDDN work data sets because of the index.

Example 6-27 Cross Loader example with IGNOREFIELDS and WORKDDN

EXEC SQL CREATE TABLE PAOLOR3.EXAMP3 (CREATOR CHAR(8), NAME VARCHAR(18), DBNAME CHAR(8), TSNAME CHAR(8)) ENDEXEC EXEC SQL


CREATE UNIQUE INDEX PAOLOR3.XEXAMP3 ON PAOLOR3.EXAMP3 (CREATOR,NAME) USING STOGROUP SYSDEFLT CLUSTER ENDEXEC EXEC SQL DECLARE C3 CURSOR FOR SELECT * FROM SYSIBM.SYSTABLES WHERE TYPE = 'T' ORDER BY CREATOR,NAME ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..&UTILID..SYSUT1) SPACE(100,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..&UTILID..SORTOUT) SPACE(100,10) CYL LOAD DATA REPLACE LOG NO INCURSOR C3 COPYDDN(TSYSCOPY) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS INTO TABLE PAOLOR3.EXAMP3 IGNOREFIELDS YES

We will now LOAD a table containing UDT columns. In Example 6-28 we first create and populate a table containing a UDT column. This UDT is defined as CHAR(8) WITH COMPARISONS. We do not have to specify the appropriate casting function in the select list of the cursor SQL statement to LOAD the table.

Example 6-28 Populating a table containing a UDT column with the Cross Loader

EXEC SQL CREATE DISTINCT TYPE PAOLOR3.TABLE_OWNER AS CHAR(8) WITH COMPARISONS ENDEXEC EXEC SQL CREATE TABLE PAOLOR3.EXAMP4 (COL1 PAOLOR3.TABLE_OWNER, COL2 VARCHAR(18), COL3 CHAR(8), COL4 CHAR(8)) ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XEXAMP4 ON PAOLOR3.EXAMP4 (COL1,COL2) USING STOGROUP SYSDEFLT ENDEXEC EXEC SQL DECLARE C4 CURSOR FOR SELECT CREATOR AS COL1, NAME AS COL2, DBNAME AS COL3, TSNAME AS COL4 FROM SYSIBM.SYSTABLES WHERE TYPE = 'T'


ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..&UTILID..SYSUT1) SPACE(100,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..&UTILID..SORTOUT) SPACE(100,10) CYL LOAD DATA REPLACE LOG NO INCURSOR C4 COPYDDN(TSYSCOPY) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS INTO TABLE PAOLOR3.EXAMP4

In Example 6-29 we copy a subset of this table back to PAOLOR3.EXAMP3 with the Cross Loader, converting the UDT back to the standard CHAR data type. We do not have to specify any casting function in the select list of the cursor but we have to specify it in the WHERE clause to create the subset. We only want to copy the rows corresponding with COL1 equals to ‘SYSIBM’ and rename the CREATOR value to ‘SYSIBMX’ to prevent duplicate values in PAOLOR3.EXAMP3. As we cannot use an Inline COPY with LOAD RESUME(YES), we have to take the image copy with an additional COPY statement.

Example 6-29 Cross loading from a table containing UDTs

EXEC SQL DECLARE C5 CURSOR FOR SELECT 'SYSIBMX' AS CREATOR, COL2 AS NAME, COL3 AS DBNAME, COL4 AS TSNAME FROM PAOLOR3.EXAMP4 WHERE COL1 = PAOLOR3.TABLE_OWNER('SYSIBM') ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..&UTILID..SYSUT1) SPACE(100,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..&UTILID..SORTOUT) SPACE(100,10) CYL LOAD DATA RESUME(YES) LOG NO INCURSOR C5 WORKDDN(TSYSUT1,TSORTOUT) INTO TABLE PAOLOR3.EXAMP3 COPY TABLESPACE DSNDB04.EXAMP3 COPYDDN(TSYSCOPY) SHRLEVEL REFERENCE


In Example 6-30 we populate a table containing a LOB column with the Cross Loader. Remember that the maximum LOB size that can be transferred by the Cross Loader is 32 KB. We first create the target base table, auxiliary table, and corresponding table spaces and indexes with the EXEC SQL statement. We declare the ROWID as NOT NULL GENERATED ALWAYS and do not copy the ROWID value from the source table. After the LOAD we collect STATISTICS on all table spaces using a LISTDEF with the ALL LOB indicator keyword to include both base and LOB table spaces.

Example 6-30 Cross loading example with a LOB column

EXEC SQL CREATE DATABASE PAOLOR3L ENDEXEC EXEC SQL CREATE TABLESPACE DSN8S71B IN PAOLOR3L USING STOGROUP DSN8G710 ENDEXEC EXEC SQL CREATE TABLE PAOLOR3.EXAMP6 (EMPNO CHAR(06) NOT NULL, EMP_ROWID ROWID NOT NULL GENERATED ALWAYS, RESUME CLOB(5K), PRIMARY KEY (EMPNO)) IN PAOLOR3L.DSN8S71B ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XEXAMP6 ON PAOLOR3.EXAMP6 (EMPNO ASC) USING STOGROUP DSN8G710 ENDEXEC EXEC SQL CREATE LOB TABLESPACE DSN8S71N IN PAOLOR3L LOG NO ENDEXEC EXEC SQL CREATE AUX TABLE PAOLOR3.AUX_EMP_RESUME IN PAOLOR3L.DSN8S71N STORES PAOLOR3.EXAMP6 COLUMN RESUME ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XAUX_EMP_RESUME ON PAOLOR3.AUX_EMP_RESUME ENDEXEC EXEC SQL DECLARE C6 CURSOR FOR SELECT EMPNO, RESUME FROM DSN8710.EMP_PHOTO_RESUME ENDEXEC LISTDEF LISTLOB INCLUDE TABLESPACE PAOLOR3L.* ALL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA INCURSOR C6 INTO TABLE PAOLOR3.EXAMP6 WORKDDN(TSYSUT1,TSORTOUT) RUNSTATS TABLESPACE LIST LISTLOB INDEX(ALL)


In Example 6-31 we use the Cross Loader to transfer data from a DB2 UDB database on a Windows NT server to DB2 for z/OS. The table in the UDB NT database is reached through DRDA using its three part name. We first had to setup TCP/IP connectivity between our z/OS DB2 system and the NT server by populating the CDB tables SYSIBM.LOCATION, SYSIBM.IPNAMES and SYSIBM.USERNAMES. In this case the remote location name is CIC_DMPC. We then bound the DSNUGSQL package to the NT server using a command similar to the one in “Binding the Cross Loader package” on page 138. In this example we copy the whole table to the host. This table is a fact table belonging to a datamart which is organized as a star schema.

Example 6-31 Cross Loader example using DRDA

EXEC SQL CREATE TABLESPACE TSEXAMP7 IN PAOLOR3L USING STOGROUP SGLP0002 PRIQTY 720 SECQTY 720 ; ENDEXEC EXEC SQL CREATE TABLE PAOLOR3.EXAMP7 (CASE_KEY INTEGER NOT NULL , SEVERITY_KEY INTEGER NOT NULL , RECIEVED_VIA_KEY INTEGER NOT NULL , TYPE_KEY INTEGER NOT NULL , ASSIGNED_TO_KEY INTEGER NOT NULL , TOD_KEY INTEGER NOT NULL , PRODUCT_KEY INTEGER NOT NULL , CUSTOMER_KEY INTEGER NOT NULL , STATUS_KEY INTEGER NOT NULL , RESOLUTION_KEY INTEGER NOT NULL , REASON_CLOSED_KEY INTEGER NOT NULL , CLOSED_BY_KEY INTEGER NOT NULL , TIME_CREATED_KEY INTEGER NOT NULL , TIME_CLOSED_KEY INTEGER NOT NULL , NEW_CASE_VOLUME INTEGER , OPEN_CASE_VOLUME INTEGER , CLOSED_CASE_VOLUME INTEGER , CLOSED_ONFIRSTCONT INTEGER , DAYS_TO_RESOLUTION INTEGER , AVERAGE_DAYS_TO_RE INTEGER , CLOSED_BY_OTHERS INTEGER , CASE_SUBJECT VARCHAR(100) , CASE_NOTE VARCHAR(255) , LAST_COMMENT VARCHAR(255) ) IN PAOLOR3L.TSEXAMP7 ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XEXAMP7 ON PAOLOR3.EXAMP7 (CASE_KEY , SEVERITY_KEY , RECIEVED_VIA_KEY , TYPE_KEY , ASSIGNED_TO_KEY , TOD_KEY , PRODUCT_KEY , CUSTOMER_KEY , STATUS_KEY , RESOLUTION_KEY , REASON_CLOSED_KEY,


CLOSED_BY_KEY , TIME_CREATED_KEY , TIME_CLOSED_KEY ) USING STOGROUP SGLP0002 PRIQTY 720 SECQTY 720 ; ENDEXEC EXEC SQL DECLARE C7 CURSOR FOR SELECT * FROM CIC_DMPC.DB2INST2.CIC_FACT_TABLE ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE SORTKEYS 40000 INCURSOR C7 WORKDDN(TSYSUT1,TSORTOUT) COPYDDN(TSYSCOPY) STATISTICS INTO TABLE PAOLOR3.EXAMP7

In Example 6-32 we copy aggregated information from the same datamart to the host. The result set is a star join between the fact table and some dimension tables of the datamart.

Example 6-32 Cross Loader example with aggregation

EXEC SQL CREATE TABLESPACE TSEXAMP8 IN PAOLOR3L USING STOGROUP SGLP0002 PRIQTY 720 SECQTY 720 ; ENDEXEC EXEC SQL CREATE TABLE PAOLOR3.EXAMP8 (TYPE_ALIAS VARCHAR(40) , SEVERITY_ALIAS VARCHAR(40) , PRODUCT_ALIAS VARCHAR(40) , DAYS_TO_RESOLUTION INTEGER ) IN PAOLOR3L.TSEXAMP8 ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XEXAMP8 ON PAOLOR3.EXAMP8 (TYPE_ALIAS , SEVERITY_ALIAS , PRODUCT_ALIAS ) USING STOGROUP SGLP0002 PRIQTY 720 SECQTY 720 ; ENDEXEC EXEC SQL DECLARE C8 CURSOR FOR SELECT A.TYPE_ALIAS, B.SEVERITY_ALIAS, C.PRODUCT_ALIAS, SUM(D.DAYS_TO_RESOLUTION) AS DAYS_TO_RESOLUTION FROM CIC_DMPC.DB2INST2.TYPE A, CIC_DMPC.DB2INST2.SEVERITY B, CIC_DMPC.DB2INST2.PRODUCT C, CIC_DMPC.DB2INST2.CIC_FACT_TABLE D


WHERE A.TYPE_KEY = D.TYPE_KEY AND B.SEVERITY_KEY = D.SEVERITY_KEY AND C.PRODUCT_KEY = D.PRODUCT_KEY GROUP BY A.TYPE_ALIAS,B.SEVERITY_ALIAS,C.PRODUCT_ALIAS ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA REPLACE SORTKEYS 1000 INCURSOR C8 WORKDDN(TSYSUT1,TSORTOUT) COPYDDN(TSYSCOPY) STATISTICS INTO TABLE PAOLOR3.EXAMP8

In Example 6-33 we load a partitioned table space with the Cross Loader. In this case we only use one cursor. The source and target tables are identical.

Example 6-33 Loading a partitioned table space with the Cross Loader

EXEC SQL CREATE TABLESPACE TSEXAMP9 IN PAOLOR3L USING STOGROUP SGLP0002 PRIQTY 50000 SECQTY 5000 BUFFERPOOL BP0 NUMPARTS 3 ENDEXEC EXEC SQL CREATE TABLE PAOLOR3.EXAMP9 (PS_PARTKEY INTEGER NOT NULL , PS_SUPPKEY INTEGER NOT NULL , PS_AVAILQTY INTEGER NOT NULL , PS_SUPPLYCOST REAL NOT NULL , PS_COMMENT VARCHAR(199) FOR SBCS DATA NOT NULL ) IN PAOLOR3L.TSEXAMP9 ENDEXEC EXEC SQL CREATE UNIQUE INDEX PAOLOR3.XEXAMP9 ON PAOLOR3.EXAMP9 (PS_PARTKEY ASC , PS_SUPPKEY ASC , PS_SUPPLYCOST ASC ) USING STOGROUP SGLP0002 PRIQTY 1200 SECQTY 400 CLUSTER (PART 1 VALUES(27000) , PART 2 VALUES(54000) , PART 3 VALUES(80000) ) BUFFERPOOL BP2 ENDEXEC EXEC SQL DECLARE C9 CURSOR FOR SELECT * FROM PAOLOR4.PARTSUPP2 ORDER BY 1 ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.)


SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..&UTILID..SYSUT1) SPACE(100,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..&UTILID..SORTOUT) SPACE(100,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS INCURSOR C9 COPYDDN(TSYSCOPY) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS INTO TABLE PAOLOR3.EXAMP9

In Example 6-34 we load the same partitioned table space using partition parallelism. Therefore we declare one different cursor per partition, transferring the data belonging to that particular partition.

Example 6-34 Using partition parallelism with the Cross Loader

EXEC SQL DECLARE C91 CURSOR FOR SELECT * FROM PAOLOR4.PARTSUPP2 WHERE PS_PARTKEY <= 27000 ORDER BY 1 ENDEXEC EXEC SQL DECLARE C92 CURSOR FOR SELECT * FROM PAOLOR4.PARTSUPP2 WHERE PS_PARTKEY > 27000 AND PS_PARTKEY <= 54000 ORDER BY 1 ENDEXEC EXEC SQL DECLARE C93 CURSOR FOR SELECT * FROM PAOLOR4.PARTSUPP2 WHERE PS_PARTKEY > 54000 ORDER BY 1 ENDEXEC TEMPLATE TSYSCOPY DSN(PAOLOR3.&DB..&TS..D&DATE..T&TIME.) SPACE(100,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..&UTILID..SYSUT1) SPACE(100,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..&UTILID..SORTOUT) SPACE(100,10) CYL LOAD DATA REPLACE LOG NO SORTKEYS COPYDDN(TSYSCOPY) WORKDDN(TSYSUT1,TSORTOUT) STATISTICS INTO TABLE PAOLOR3.EXAMP9 PART 1 INCURSOR C91 INTO TABLE PAOLOR3.EXAMP9 PART 2 INCURSOR C92 INTO TABLE PAOLOR3.EXAMP9 PART 3 INCURSOR C93


6.7 Online LOAD ResumeThe Online LOAD Resume is a new capability in DB2 V7 that allows you to load data with the LOAD utility, while having concurrent SQL access on the same table space.

The classic LOAD drains all access to the table space which prevents the data to be available for SQL applications. Therefore many DB2 sites wrote their own INSERT programs for loading data, thus avoiding the drain and allowing concurrent SQL access. Another reason for writing INSERT programs is to use triggers which are fired by the INSERT statements and can be used for additional controls on the correctness of the input data.

With the new Online LOAD Resume, you can LOAD data with minimal impact on SQL applications and without writing and maintaining INSERT programs. It behaves externally as a LOAD, but it works internally like a mass INSERT. Therefore, triggers will be fired as well. The index building, duplicate key and referential constraint checking will be handled by SQL insert processing. The records which fail the insert will be written to the discard data set.

Because the Online LOAD Resume internally works like a SQL INSERT, this kind of LOAD is slower than the classic LOAD. However you have to trade off performance for availability and data integrity forced by triggers.

6.7.1 Invoking Online LOAD ResumeThe LOAD statement offers a new option, SHRLEVEL, with two possible values:

� SHRLEVEL NONE invokes the classic LOAD. It specifies that applications have no concurrent access to the table space or partition. This is the default.

� SHRLEVEL CHANGE invokes the new Online LOAD Resume capability. It specifies that applications can concurrently read and write the table space or partition being loaded.

SHRLEVEL CHANGE always requires RESUME YES and LOG YES and invokes a completely different processing. Although the syntax is like the classical LOAD (and the input is provided as before), internally each input record is placed in the table space via a mechanism comparable to an individual INSERT.

However, this INSERT is handled by the Data Manager and not by the RDS. Therefore, most of the SQL overhead is omitted.

6.7.2 Behavior of Online LOAD ResumeAs shown in Figure 6-5, Online LOAD Resume really behaves as a mixture of the classic LOAD and SQL INSERT processing.


Figure 6-5 Online LOAD Resume

INSERT behaviorFollowing are some of the consequences of INSERT processing.

SerializationEven though full INSERT statements are not generated, LOAD RESUME YES SHRLEVEL CHANGE will functionally operate like SQL INSERT. Whereas the classic LOAD drains the table space, thus inhibiting any SQL access, these INSERTs act like normal INSERTs by using claims when accessing an object. That is, they behave like any other SQL application and can run concurrently with other, even updating, SQL applications. The records are stored one after each other in the same sequence as they occur in the input sequential data set.

LoggingOnly LOG YES is allowed. All inserts are logged. After loading, the table space or partition is not put in the COPYP restrictive state and no COPY is required afterwards.

Because all inserts are logged, they can be captured by DpropR for propagation.

TriggersTriggers are fired like with normal SQL INSERT processing

Referential Integrity and check constraints Only ENFORCE CONSTRAINTS is allowed. The foreign key values in each input record must already exist as primary key in the parent table, otherwise the record will not be accepted. Similarly, if an input record violates a check constraint, it will not be accepted.

When you load a self-referencing table the foreign key value in each input record:

� Must already exist as primary key value in the table� Must be provided as primary key value within this record

Mixture of LOAD and INSERT

On-line LOAD RESUMESyntaxLOAD DATA

RESUME YESSHRLEVEL CHANGEINTO TABLE TEST.CARS ( CARID POSITION ( 1: 3) CHAR , PRODUCER POSITION ( 5:17) CHAR , MODEL POSITION (20:29) CHAR ...

001 MOTOR CORP. STAR002 ENGINE MASTER SPEEDY003 MOBILES, LTD. CONFOR...

INSERT INTO TEST.CARS VALUES ('001' ,'MOTOR CORP.' ,'STAR' ); INSERT INTO TEST.CARS VALUES ('002' ,'ENGINE MASTER' ,'SPEEDY' );INSERT INTO TEST.CARS VALUES ('003' ,'MOBILES, LTD.' ,'CONFOR' );...

Claims (not drains)LOG YES onlyFireParent key must existFirst acceptedPreservedUsed (not provided)

Serialization:Logging:Triggers:RI:Duplicate keys:Clustering:Free space:

Security:LOAD (not INSERT) privilege

Timeout of the LOAD:Like utility (not like SQL appl.)

Processing


This is different from the classical LOAD, and it forces you to sort the input in such a way that these requirements are met, rather than sorting the input in clustering index sequence, which you used to do with the LOAD.

After loading, the table space will never be put in the CHECKP restrictive state and no CHECK DATA is required afterwards.

Duplicate keysWhen uniqueness of a column is required, INSERTs are accepted as long as they provide different values for this column. Following INSERTs with the same values are not accepted. This is different from the classic LOAD procedure, which discards all records having the same value for such a column.

You may be forced to change your handling of the discarded records accordingly, when you change existing LOAD jobs to SHRLEVEL CHANGE.

ClusteringWhereas the classic LOAD utility with RESUME YES stores the new records (in the sequence of the input) at the end of the already existing records; the new Online LOAD Resume tries to insert the records in available free pages as close to clustering order as possible; additional free pages are not created. As you probably insert a lot of rows, these are likely to be stored out of the clustering order (OFFPOS records).

So, a REORG may be needed after the classic LOAD, as the clustering may not be preserved, but also after the new Online LOAD Resume, as OFFPOS records may exist. A RUNSTATS with SHRLEVEL CHANGE UPDATE SPACE followed by a conditional REORG is recommended.

Free spaceFurthermore the free space, obtained either by PCTFREE or by FREEPAGE, is used by these INSERTs of the Online LOAD Resume, in contrast to the classical LOAD, which loads the pages thereby providing these types of free space.

As a consequence, a REORG may be needed after an Online LOAD Resume.

UTILITY behaviorOnline LOAD Resume also behaves as a utility:

PhasesThe phases of Online LOAD Resume are:

UTILINIT, RELOAD, DISCARD, REPORT, UTILTERM

The following phases do not occur:

SORT, BUILD, INDEXVAL, ENFORCE

Error handling and messagesThe messages that can be issued during the utility execution are the same as for the LOAD utility, even though the data is accessed differently.

The DISCARD and the REPORT phase are performed like in the classic LOAD. Input records which fail the insert will be written to the discard data set. Error information is stored in the error data set.


Incompatible LOAD optionsSHRLEVEL CHANGE is incompatible with these options:

LOG NO, ENFORCE NO, KEEPDICTIONARY, SORTKEYS, STATISTICS, COPYDDN, RECOVERYDDN, PREFORMAT, REUSE, and PART REPLACE

LockingLock contention with SQL applications is avoided due to a intelligent management of the commit scope. DB2 manages the commit scope dynamically monitoring the current locking situation.

If the utility gets a timeout, it will be handled like a utility timeout and not like an SQL application timeout. So you will get messages like DSNT500I RESOURCE UNAVAILABLE REASON 00c9008e and the utility will be put in a stopped state. The job step will be restartable.

Concurrency with other utilitiesLOAD SHRLEVEL CHANGE cannot run concurrently on the same target object with online REORG SHRLEVEL CHANGE and COPYTOCOPY. But it can run concurrently on the same target object with the following utilities:

� COPY TABLESPACE or INDEXSPACE SHRLEVEL CHANGE� MERGECOPY� RECOVER ERROR RANGE� CONCURRENT COPY SHRLEVEL REFERENCE after copy is logically completed� RUNSTATS SHRLEVEL CHANGE� UNLOAD SHRLEVEL CHANGE

Online LOAD RESUME YES cannot run against a table space that is in COPY pending, CHECK pending or RECOVER pending status. Similarly, it cannot run against an index space that is in CHECK pending or RECOVER pending status.

RestartDuring RELOAD, internal commit points are set, therefore, RESTART(CURRENT) is possible as with the classic LOAD.

Partition parallelismPartition parallelism is supported.

SecurityOnline LOAD Resume requires the LOAD privilege and not the INSERT privilege.

6.7.3 Online LOAD Resume performanceThe measurements in Table 6-1 were performed using an IBM G6 3-way processor and ESS disks on a table with 10 partitions, 1 partitioned index and 1 non-partitioned index (NPI). It summarizes the comparison of an user application program inserting rows in random with Online LOAD Resume.

Table 6-1 Online LOAD Resume compared with SQL insert application

Inserting 2,000,000 rows via program

Online LOAD Resume 2,000,000 rows

Delta %

CPU time 241 171 -29.0%

Elapsed time 326 256 -21.5%

Note: All times are in seconds


Using Online LOAD Resume for inserting rows in a table can be up to 21.5% faster in elapsed time than using a user application and even the CPU time is improved up to 29%. When using the Online LOAD Resume, instead of a user application, you avoid the cost of developing and maintaining a user application. Another advantage is that Online LOAD Resume automatically monitors the lock situation and changes the commit interval dynamically.

6.7.4 Online LOAD Resume examplesIn this section we will provide some examples of Online LOAD Resume (SHRLEVEL CHANGE) for non-partitioned and partitioned table spaces. For the partitioned table spaces we will show the use of partition parallelism. We will also show the different handling of duplicate input rows by LOAD SHRLEVEL CHANGE compared to the classic LOAD SHRLEVEL NONE.

Online LOAD Resume of a non-partitioned table spaceWe will now show how to load a non-partitioned table space with Online LOAD Resume. This is the same table space as in Example 6-31 on page 144. We first created the input data set by unloading the existing table with the new V7 UNLOAD utility. The JCL and syntax of the unload utility is shown in Example 6-35.

Example 6-35 Creating an input load data set with the UNLOAD utility

// EXEC PGM=DSNUTILB,PARM='DB2G,UNLOAD' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * TEMPLATE TUNLDDN DSN(PAOLOR3.&DB..&TS..UNLOAD) TEMPLATE TPUNCH DSN(PAOLOR3.&DB..&TS..PUNCH) UNLOAD TABLESPACE PAOLOR3L.TSEXAMP7 UNLDDN TUNLDDN PUNCHDDN TPUNCH

In Example 6-36 we first empty the table with a mass DELETE statement and than repopulate it using the Online LOAD Resume utility. The input data set is specified by a template TSYSREC. Although there are no SORT, BUILD, and INDEXVAL phases with Online LOAD Resume, the WORKDDN data sets are mandatory if the table has indexes.

Example 6-36 Online LOAD Resume of a non-partitioned table space

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * EXEC SQL DELETE FROM PAOLOR3.EXAMP7 ENDEXEC TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..UNLOAD) TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA SHRLEVEL CHANGE RESUME(YES)


INDDN(TSYSREC) WORKDDN(TSYSUT1,TSORTOUT) INTO TABLE PAOLOR3.EXAMP7 WHEN(1:2 = X'000E') (CASE_KEY POSITION(003:006) INTEGER ,SEVERITY_KEY POSITION(007:010) INTEGER ,RECIEVED_VIA_KEY POSITION(011:014) INTEGER ,TYPE_KEY POSITION(015:018) INTEGER ,ASSIGNED_TO_KEY POSITION(019:022) INTEGER ,TOD_KEY POSITION(023:026) INTEGER ,PRODUCT_KEY POSITION(027:030) INTEGER ,CUSTOMER_KEY POSITION(031:034) INTEGER ,STATUS_KEY POSITION(035:038) INTEGER ,RESOLUTION_KEY POSITION(039:042) INTEGER ,REASON_CLOSED_KEY POSITION(043:046) INTEGER ,CLOSED_BY_KEY POSITION(047:050) INTEGER ,TIME_CREATED_KEY POSITION(051:054) INTEGER ,TIME_CLOSED_KEY POSITION(055:058) INTEGER ,NEW_CASE_VOLUME POSITION(060:063) INTEGER NULLIF(059)=X'FF' ,OPEN_CASE_VOLUME POSITION(065:068) INTEGER NULLIF(064)=X'FF' ,CLOSED_CASE_VOLUME POSITION(070:073) INTEGER NULLIF(069)=X'FF' ,CLOSED_ONFIRSTCONT POSITION(075:078) INTEGER NULLIF(074)=X'FF' ,DAYS_TO_RESOLUTION POSITION(080:083) INTEGER NULLIF(079)=X'FF' ,AVERAGE_DAYS_TO_RE POSITION(085:088) INTEGER NULLIF(084)=X'FF' ,CLOSED_BY_OTHERS POSITION(090:093) INTEGER NULLIF(089)=X'FF' ,CASE_SUBJECT POSITION(095:196) VARCHAR NULLIF(094)=X'FF' ,CASE_NOTE POSITION(198:0454) VARCHAR NULLIF(197)=X'FF' ,LAST_COMMENT POSITION(456:712) VARCHAR NULLIF(455)=X'FF' )

As a result, 48420 records will be inserted into the empty table. See Example 6-37 for an extract of the job output.

Example 6-37 Online LOAD Resume job output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - EXEC SQL DELETE FROM PAOLOR3.EXAMP7 ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION DSNU1198I DSNUGSQL - SQLSTATE = 01504 SQLSTATE RETURN CODE DSNU1196I DSNUGSQL - SQLERRD = 0 0 48420 1142301703 0 0 SQL DIAGNOSTIC INFORMATION DSNU1196I DSNUGSQL - SQLERRD = X'00000000' X'00000000' X'0000BD24' X'44162407' X'00000000' X'00000000' SQL DIAGNOSTIC INFORMATION DSNU1197I DSNUGSQL - SQLWARN0-5 = W,,,,W, SQL WARNINGS DSNU1197I DSNUGSQL - SQLWARN6-A = ,,,, SQL WARNINGS DSNU050I DSNUGUTC - TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..UNLOAD) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA SHRLEVEL CHANGE RESUME(YES) INDDN(TSYSREC) WORKDDN(TSYSUT1,TSORTOUT) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP7 WHEN(1:2=X'000E') DSNU650I -DB2G DSNURWI - (CASE_KEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - SEVERITY_KEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - RECIEVED_VIA_KEY POSITION(11:14) INTEGER,


DSNU650I -DB2G DSNURWI - TYPE_KEY POSITION(15:18) INTEGER, DSNU650I -DB2G DSNURWI - ASSIGNED_TO_KEY POSITION(19:22) INTEGER, DSNU650I -DB2G DSNURWI - TOD_KEY POSITION(23:26) INTEGER, DSNU650I -DB2G DSNURWI - PRODUCT_KEY POSITION(27:30) INTEGER, DSNU650I -DB2G DSNURWI - CUSTOMER_KEY POSITION(31:34) INTEGER, DSNU650I -DB2G DSNURWI - STATUS_KEY POSITION(35:38) INTEGER, DSNU650I -DB2G DSNURWI - RESOLUTION_KEY POSITION(39:42) INTEGER, DSNU650I -DB2G DSNURWI - REASON_CLOSED_KEY POSITION(43:46) INTEGER, DSNU650I -DB2G DSNURWI - CLOSED_BY_KEY POSITION(47:50) INTEGER, DSNU650I -DB2G DSNURWI - TIME_CREATED_KEY POSITION(51:54) INTEGER, DSNU650I -DB2G DSNURWI - TIME_CLOSED_KEY POSITION(55:58) INTEGER, DSNU650I -DB2G DSNURWI - NEW_CASE_VOLUME POSITION(60:63) INTEGER NULLIF(59)=X'FF', DSNU650I -DB2G DSNURWI - OPEN_CASE_VOLUME POSITION(65:68) INTEGER NULLIF(64)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_CASE_VOLUME POSITION(70:73) INTEGER NULLIF(69)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_ONFIRSTCONT POSITION(75:78) INTEGER NULLIF(74)=X'FF', DSNU650I -DB2G DSNURWI - DAYS_TO_RESOLUTION POSITION(80:83) INTEGER NULLIF(79)=X'FF', DSNU650I -DB2G DSNURWI - AVERAGE_DAYS_TO_RE POSITION(85:88) INTEGER NULLIF(84)=X'FF', DSNU650I -DB2G DSNURWI - CLOSED_BY_OTHERS POSITION(90:93) INTEGER NULLIF(89)=X'FF', DSNU650I -DB2G DSNURWI - CASE_SUBJECT POSITION(95:196) VARCHAR NULLIF(94)=X'FF', DSNU650I -DB2G DSNURWI - CASE_NOTE POSITION(198:454) VARCHAR NULLIF(197)=X'FF', DSNU650I -DB2G DSNURWI - LAST_COMMENT POSITION(456:712) VARCHAR NULLIF(455)=X'FF') DSNU320I -DB2G DSNURWI - RESUME(YES) WAS SPECIFIED FOR EMPTY TABLESPACE DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.UNLOAD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.UNLOAD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SORTOUT DSNU1114I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS LOADED =48420 FOR TABLE PAOLOR3.EXAMP7 DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=48420 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:35 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

In Example 6-38 we empty the table space and duplicate the input data to force duplicate records in the input. In this case the Online LOAD Resume will insert the first 48420 records and reject another 48420 records which will be reported. We had to add a DFSPARM data set to increase the SORT storage workarea from its default value of 4096K to a value of 24000K. Otherwise the sort failed with message ICE046A 0 SORT CAPACITY EXCEEDED - RECORD COUNT 34767 during the REPORT phase.

Example 6-38 Online LOAD Resume duplicating the input data set

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //DFSPARM DD * OPTION MAINSIZE=24000K //SYSREC DD DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.UNLOAD,DISP=SHR // DD DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.UNLOAD,DISP=SHR //SYSIN DD * EXEC SQL DELETE FROM PAOLOR3.EXAMP7 ENDEXEC TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA SHRLEVEL CHANGE RESUME(YES) WORKDDN(TSYSUT1,TSORTOUT) INTO TABLE PAOLOR3.EXAMP7

WHEN(1:2 = .................


See Example 6-39 for an extract of the job output.

Example 6-39 Online LOAD Resume job output duplicate records in the input

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - EXEC SQL DELETE FROM PAOLOR3.EXAMP7 ENDEXEC..........................................................DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA SHRLEVEL CHANGE RESUME(YES) WORKDDN(TSYSUT1,TSORTOUT) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP7 WHEN(1:2=X'000E') DSNU650I -DB2G DSNURWI - (CASE_KEY POSITION(3:6) INTEGER,............................................................DSNU320I -DB2G DSNURWI - RESUME(YES) WAS SPECIFIED FOR EMPTY TABLESPACE DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SORTOUT DSNU1117I -DB2G DSNURBWF - UNIQUE INDEX KEY DUPLICATES KEY OF INDEXED RECORD AT RID X'0000000201', INDEX = PAOLOR3.XEXAMP7, TABLE = PAOLOR3.EXAMP7, RECNO = 48421 DSNU1117I -DB2G DSNURBWF - UNIQUE INDEX KEY DUPLICATES KEY OF INDEXED RECORD AT RID X'0000000202', INDEX = PAOLOR3.XEXAMP7, TABLE = PAOLOR3.EXAMP7, RECNO = 48422 ........................................................................repeating this message for every duplicate input record........................................................................DSNU1118I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - 48420 DUPLICATE KEY ERRORS ENCOUNTERED DSNU1114I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS LOADED =48420 FOR TABLE PAOLOR3.EXAMP7 DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=96840 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:01:31 DSNU399I DSNURRIS - LOAD UTILITY ERROR SUMMARY REPORT ERROR INPUT ERROR TABLE FIELD/FANSET RELATED SEVERITY RECORD TYPE NAME NAME ERROR 1 48421 DUP. KEY PAOLOR3.EXAMP7 PAOLOR3.XEXAMP7 0 1 48422 DUP. KEY PAOLOR3.EXAMP7 PAOLOR3.XEXAMP7 0 ........................................................................repeating this message for every duplicate input record........................................................................DSNU396I DSNURREP - REPORT PHASE COMPLETE, ELAPSED TIME=00:00:04 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

This behavior is different from the LOAD SHRLEVEL NONE where all 96840 records would have been loaded during the RELOAD phase and all 96840 records deleted again during the INDEXVAL phase, as illustrated in the job output in Example 6-40.

Example 6-40 LOAD SHRLEVEL NONE job output duplicate records in the input

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - EXEC SQL DELETE FROM PAOLOR3.EXAMP7 ENDEXEC DSNU1180I DSNUGSQL - SQLCODE = 000, SUCCESSFUL EXECUTION ................................................................DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA SHRLEVEL NONE RESUME(YES) WORKDDN(TSYSUT1,TSORTOUT) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP7 WHEN(1:2=X'000E') DSNU650I -DB2G DSNURWI - (CASE_KEY POSITION(3:6) INTEGER,..................................................................DSNU320I -DB2G DSNURWI - RESUME(YES) WAS SPECIFIED FOR EMPTY TABLESPACE DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT


DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP7.SORTOUT DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=96840 FOR TABLE PAOLOR3.EXAMP7 DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=96840 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:16 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=96840 ELAPSED TIME=00:00:02 DSNU345I DSNURBXD - UNIQUE INDEX KEY DUPLICATES KEY FROM INPUT RECORD '44654' LOADED AT RID X'00000F861D', INDEX = PAOLOR3.XEXAMP7, TABLE = PAOLOR3.EXAMP7, RECNO = 93074, RID = X'000014EE09' ........................................................................repeating this message for every duplicate input record........................................................................DSNU258I DSNURBXD - BUILD PHASE STATISTICS - NUMBER OF INDEXES=1 DSNU343I DSNURBXD - BUILD PHASE STATISTICS - 96840 DUPLICATE KEY ERRORS ENCOUNTERED DSNU259I DSNURBXD - BUILD PHASE COMPLETE, ELAPSED TIME=00:00:12 DSNU355I DSNURVIX - INDEXVAL PHASE STATISTICS - 96840 DUPLICATE KEY ERRORS CORRECTED BY DELETING 96840 DATA ROWS DSNU356I DSNURVIX - INDEXVAL PHASE COMPLETE, ELAPSED TIME=00:00:47 DSNU399I DSNURRIS - LOAD UTILITY ERROR SUMMARY REPORT ERROR INPUT ERROR TABLE FIELD/FANSET RELATED SEVERITY RECORD TYPE NAME NAME ERROR 1 1 DUP. KEY PAOLOR3.EXAMP7 PAOLOR3.XEXAMP7 0 1 2 DUP. KEY PAOLOR3.EXAMP7 PAOLOR3.XEXAMP7 0 ........................................................................repeating this message for every duplicate input record........................................................................DSNU396I DSNURREP - REPORT PHASE COMPLETE, ELAPSED TIME=00:00:08 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

Online LOAD Resume and LOAD SHRLEVEL NONE will both reject duplicate records in the input data set that already existed in the table before the LOAD. In this case there is no difference between both LOAD methods.

If you want to process the rejected records you can save them in a DISCARD file. This is done using the DISCARDDN option and the ERRDDN option as shown in Example 6-41. Here we use templates for both options. Remember the possible different number of discarded records between LOAD SHRLEVEL CHANGE and LOAD SHRLEVEL NONE. You may be forced to change your handling of the discarded records accordingly, when you change existing LOAD jobs to SHRLEVEL CHANGE.

Example 6-41 Using a DISCARD data set with Online LOAD Resume

TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..UNLOAD) TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..DISCARD) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA SHRLEVEL CHANGE RESUME(YES) INDDN(TSYSREC) DISCARDDN(TSYSDISC) WORKDDN(TSYSUT1,TSORTOUT) ERRDDN(TSYSERR) INTO TABLE PAOLOR3.EXAMP7

WHEN(1:2 = .................


Online LOAD Resume of a partitioned table space using parallelismWe will now show how you can use partition parallelism with Online LOAD Resume. This is the same table space as in Example 6-32 on page 145. We first created the input data set by unloading the existing table with the new V7 UNLOAD utility. The JCL and syntax of the unload utility is shown in Example 6-42. Each partition is unloaded in parallel to a separate unload data set. This is achieved by using the &PART. variable in the template for the unload data sets.

Example 6-42 Unloading a partitioned table space using parallelism

// EXEC PGM=DSNUTILB,PARM='DB2G,UNLOAD' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * TEMPLATE TUNLDDN DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(100,10) CYL TEMPLATE TPUNCH DSN(PAOLOR3.&DB..&TS..PUNCH) DISP(OLD,KEEP,KEEP) UNLOAD TABLESPACE PAOLOR3L.TSEXAMP9 UNLDDN TUNLDDN PUNCHDDN TPUNCH

As shown in Example 6-43, the 3 partitions are unloaded in 3 unload data sets using 2 parallel tasks.

Example 6-43 Job output from the parallel unload

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = UNLOAD DSNU050I DSNUGUTC - TEMPLATE TUNLDDN DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(100,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TPUNCH DSN(PAOLOR3.&DB..&TS..PUNCH) DISP(OLD,KEEP,KEEP) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - UNLOAD TABLESPACE PAOLOR3L.TSEXAMP9 UNLDDN TUNLDDN PUNCHDDN TPUNCH DSNU1201I DSNUUNLD - PARTITIONS WILL BE UNLOADED IN PARALLEL, NUMBER OF TASKS = 2 DSNU397I DSNUUNLD - NUMBER OF TASKS CONSTRAINED BY CPUS DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TPUNCH DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.PUNCH DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TUNLDDN DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.U00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TUNLDDN DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.U00002 DSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=223976 FOR TABLESPACE PAOLOR3L.TSEXAMP9 PART 1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TUNLDDN DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.U00003 DSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=224000 FOR TABLESPACE PAOLOR3L.TSEXAMP9 PART 2 DSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=214421 FOR TABLESPACE PAOLOR3L.TSEXAMP9 PART 3 DSNU253I DSNUUNLD - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=662397 FOR TABLE PAOLOR3.EXAMP9 DSNU252I DSNUUNLD - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=662397 FOR TABLESPACE PAOLOR3L.TSEXAMP9 DSNU250I DSNUUNLD - UNLOAD PHASE COMPLETE, ELAPSED TIME=00:00:20 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0


In Example 6-44 you can see the utility statements to LOAD the partitioned table space in parallel with Online LOAD Resume. We also included DISCARD data sets for discard processing. As with the non-partitioned table space the WORKDDN data sets are mandatory, although there are no SORT,BUILD and INDEXVAL phases with Online LOAD Resume.

Example 6-44 Online LOAD Resume of a partitioned table space with parallelism

// EXEC PGM=DSNUTILB,PARM='DB2G,PAOLOR3' //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * EXEC SQL DELETE FROM PAOLOR3.EXAMP9 ENDEXEC TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL LOAD DATA SHRLEVEL CHANGE RESUME(YES) WORKDDN(TSYSUT1,TSORTOUT) ERRDDN(TSYSERR) INTO TABLE PAOLOR3.EXAMP9 PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2 = X'0018') (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMP9 PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2 = X'0018') (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR ) INTO TABLE PAOLOR3.EXAMP9 PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2 = X'0018') (PS_PARTKEY POSITION(03:006) INTEGER ,PS_SUPPKEY POSITION(07:010) INTEGER ,PS_AVAILQTY POSITION(11:014) INTEGER ,PS_SUPPLYCOST POSITION(15:018) FLOAT(21) ,PS_COMMENT POSITION(19:219) VARCHAR )


The resulting job output can be seen in Example 6-45. As you can see, 3 parallel tasks were used.

Example 6-45 Online LOAD Resume job output with parallelism

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNU050I DSNUGUTC - EXEC SQL DELETE FROM PAOLOR3.EXAMP9 ENDEXEC DSNU1184I DSNUGSQL - SQLCODE = 100, NOT FOUND: ROW NOT FOUND FOR FETCH, UPDATE, OR DELETE, OR THE RESULT OF A QUERY IS AN EMPTY TABLE DSNU1198I DSNUGSQL - SQLSTATE = 02000 SQLSTATE RETURN CODE DSNU1195I DSNUGSQL - SQLERRP = DSNXRSTD SQL PROCEDURE DETECTING ERROR DSNU1196I DSNUGSQL - SQLERRD = -160 0 0 1155167978 0 0 SQL DIAGNOSTIC INFORMATION DSNU1196I DSNUGSQL - SQLERRD = X'FFFFFF60' X'00000000' X'00000000' X'44DA76EA' X'00000000' X'00000000' SQL DIAGNOSTIC INFORMATION DSNU1197I DSNUGSQL - SQLWARN0-5 = W,,,,W, SQL WARNINGS DSNU1197I DSNUGSQL - SQLWARN6-A = ,,,, SQL WARNINGS DSNU050I DSNUGUTC - TEMPLATE TSYSREC DSN(PAOLOR3.&DB..&TS..U&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSDISC DSN(PAOLOR3.&DB..&TS..D&PART.) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSERR DSN(PAOLOR3.&DB..&TS..SYSERR) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSYSUT1 DSN(PAOLOR3.&DB..&TS..SYSUT1) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE TSORTOUT DSN(PAOLOR3.&DB..&TS..SORTOUT) SPACE(10,10) CYL DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LOAD DATA SHRLEVEL CHANGE RESUME(YES) WORKDDN(TSYSUT1,TSORTOUT) ERRDDN(TSYSERR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP9 PART 1 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2=X'0018') DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP9 PART 2 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2=X'0018') DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU650I -DB2G DSNURWI - INTO TABLE PAOLOR3.EXAMP9 PART 3 INDDN(TSYSREC) DISCARDDN(TSYSDISC) WHEN(1:2=X'0018') DSNU650I -DB2G DSNURWI - (PS_PARTKEY POSITION(3:6) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPKEY POSITION(7:10) INTEGER, DSNU650I -DB2G DSNURWI - PS_AVAILQTY POSITION(11:14) INTEGER, DSNU650I -DB2G DSNURWI - PS_SUPPLYCOST POSITION(15:18) FLOAT(21), DSNU650I -DB2G DSNURWI - PS_COMMENT POSITION(19:219) VARCHAR) DSNU320I -DB2G DSNURWI - RESUME(YES) WAS SPECIFIED FOR EMPTY TABLESPACE DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00001 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.U00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00002 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.D00001 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00003 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.U00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSDISC DDNAME=SYS00004 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.D00002 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSREC DDNAME=SYS00006 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.D00003 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSUT1 DDNAME=SYS00007 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.SYSUT1 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSORTOUT DDNAME=SYS00008 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.SORTOUT


DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=TSYSERR DDNAME=SYS00009 DSN=PAOLOR3.PAOLOR3L.TSEXAMP9.SYSERR DSNU364I DSNURPPL - PARTITIONS WILL BE LOADED IN PARALLEL, NUMBER OF TASKS = 3 DSNU1121I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS LOADED =223976 FOR TABLE PAOLOR3.EXAMP9 PART=1 DSNU1121I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS LOADED =224000 FOR TABLE PAOLOR3.EXAMP9 PART=2 DSNU1121I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS LOADED =214421 FOR TABLE PAOLOR3.EXAMP9 PART=3 DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=662397 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:04:01 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

6.8 Conclusions and recommendationsAs shown in this chapter, the functionality and usability of LOAD has improved a lot during the last three DB2 versions. Major new usability features include the use of TEMPLATES and EXEC SQL. Major new functionality features include the Cross Loader and Online LOAD Resume. Major performance improvements can be achieved by using Partition Parallelism, Inline COPY, Inline RUNSTATS and Parallel Index Build (PIB) by means of the SORTKEYS option.

Here are some recommendations:

� Always sort the input data in clustering sequence before loading. Do this with an ORDER BY clause when using the Cross Loader.

� Use templates whenever possible to allocate the LOAD work data sets dynamically.

� Use EXEC SQL to execute dynamic SQL statements before, between or after utility statements and to simplify the JCL of the utility jobs. Bear in mind that the use of EXEC SQL is not limited to the LOAD utility, but can be used with any utility.

� Use the Cross Loader to LOAD data from any DRDA source, eliminating unloading and file transfers.

� Use Online LOAD Resume to LOAD data if concurrent access from applications is needed.

� Use LOAD REPLACE with LOG NO and Inline COPY and inline statistics.

� Use LOAD RESUME NO with inline statistics.

� Use PIB by specifying SORTKEYS n to have your indexes built in parallel; n being an estimation of the number of keys to be sorted.

� Use multiple input data sets to LOAD partitions in parallel, reducing elapsed LOAD times and NPI contention.


Chapter 7. Reorganizing data

The DB2 Utility for reorganizing data is called REORG. Table 7-1 summarizes the major changes and enhancements that DB2 has added to the REORG utility.

Table 7-1 DB2 enhancements to the REORG utility

In this chapter we provide details on these new features:

� SORTDATA� Online REORG with COPY and RUNSTATS� SORTKEYS� Index parallelism/NPIs� Reuse� Preformat� Online REORG

– BUILD2– Wait/DRAIN– Fast Switch...– Statistics

7

DB2 V4 DB2 V5 DB2 V6 DB2 V7

NPI Improvements.

Catalog REORG.

Path length reduction.

SORTDATA.

REORG and Inline COPY(COPPYDDN).

New SHRLEVEL NONE,REFERENCE,CHANGEoption (Online REORG).

Optional removal of workdata sets for indexes(SORTKEYS).

NOSYSRECPREFORMAT option.

REUSE

Collect inline statisticsBuild indexes in parallel.

Discard and faster UNLOAD.

Faster Online REORG.

Threshold for execution.

SORTKEYS also used forparallel index build.

Avoid data set with FASTSWITCH.

Parallel process on BUILD2 phase.

More granularity on drain and waiting resource.

LISTDEF and TEMPLATE.

New options: LIST, DRAIN_WAIT, RETRY, RETRY_DELAY,HISTORY,FORCEROLLUP


7.1 Why REORG ?The need for REORG is driven by the volatility of the data, either via DML (SELECT, INSERT, DELETE, UPDATE) or via the LOAD utility (RESUME YES). The changes in the data can lead to disorganization of the data and cause longer I/O time to return data. It is also required as a method for implementing an increase in space allocated to DB2 objects.

The REORG utility unloads the contents of a table space, or index space, and rebuilds the object in optimum order, as specified by the clustering index. This will ensure that optimum performance can be achieved by SQL referencing the data via the clustering index. REORG and RUNSTATS allow the DB2 optimizer to choose better access paths.

Until Version 5 of DB2 reorganizing an object meant that the object was unavailable to all processes while the REORG was executing. As applications and data grew and the drive for 24x7 processing increased, this unavailability became more and more unacceptable. In DB2 Version 5 the concept of Online REORG was introduced. This utility has been improved further in Versions 6 and Version 7, specifically to drive down any outage caused by the REORG.

7.2 When to REORG ?There are many statistics in the DB2 catalog that can indicate when a REORG should be run; these include LEAFDIST, NEAROFFPOS, FAROFFPOS among others. Also degradation of performance indicates that a REORG may be necessary and the number of extents allocated to a DB2 object can also indicate a REORG is necessary.

In DB2 Version 6 there are new triggers added to the REORG syntax that allows REORGs to be conditionally executed dependent upon the values in the catalog. Also introduced is the ability to run a REPORTONLY parameter that will indicate whether a REORG will be run or not without actually running it.

The triggers that can be specified by the REORG utility are:

� TABLE SPACES

– OFFPOSLIMIT value (default 10)

When specified, the following SQL is executed by the REORG:

SELECT CARF,(NEAROFFPOSF + FAROFFPOSF) *100 / CARDFFROM SYSIBM.SYSINDEXPARTWHERE CARDF > 0AND (NEAROFFPOSF + FAROFFPOSF) * 100 / CARDF > :offposlimit value

This query shows the degradation of the cluster index of a table within the table space. Any rows returned from this query is selected for REORG. See “OFFPOSLIMIT keyword” on page 71 for further details.

– INDREFLIMIT value (default 10)


SELECT CARD, (NEARINDREF + FARINDREF) * 100 / CARDFFROM SYSIBM.SYSTABLEPARTWHERE CARDF > 0AND (NEARINDREF + FARINDREF) * 100 / CARDF > :indreflimit value

This query selects how many indirect row references exist for a table. Any rows returned from this query are selected for REORG. See “INDREFLIMIT keyword” on page 72 for further details.


� INDEX SPACES

– LEAFDISTLIMIT value (default 200)


SELECT LEAFDIST FROM SYSIBM.SYSINDEXPARTWHERE LEAFDIST > :leafdistlimit value

This query selects any indexes whose average number of pages between successive leaf pages is greater than the specified value. Any rows returned are selected for REORG. See 4.1.1, “REORG INDEX” on page 70 for further details.

The return code set by the REORG indicates whether a limit has been met:

� RC1: No limits have been met.� RC2 or greater: Limits have been met and action has been taken.

If REPORTONLY is specified, the return codes are set as above. If the REORG is executed, then the return code may be higher, dependent upon the REORG options specified — for example, LOG NO without an Inline COPY gives RC4.

To ensure that correct information is given by the queries, RUNSTATS must be kept current.

7.3 Types of REORGThere are now three types of REORG, and these are discussed below along with recommendations for their use.

7.3.1 SHRLEVEL NONEThis is the “classic” REORG — it is the most restrictive of all REORGs. During the UNLOAD phase, read-only access is allowed to the data. Once the RELOAD phase starts all readers are drained and no access is allowed until the REORG has finished, assuming Inline COPY is taken during the REORG.

This type of REORG is suitable when there is a batch window available for dedicated housekeeping. It is also suitable if there is insufficient disk available to hold “shadow” data sets during the REORG, and if the REUSE parameter is coded, then the “old” data sets are not deleted, but are reused. When using REUSE, ensure that the data sets are big enough to hold the data plus any free space that may be reinstated by the REORG. See 7.8.7, “REUSE” on page 184 for more information on the REUSE parameter.

SHRLEVEL NONE must be used for the catalog and directory objects and for LOB table spaces.

Intervention is required if the REORG is unsuccessful. This may involve restarting the utility or recovering of the objects.

Important: RETURN CODE 0 is no longer given by REORG when using triggering. A successful REORG is indicated by 2 or 4. RC4 is returned if a pending flag has been set.

Ensure that all JCL condition code checking is correct.

Chapter 7. Reorganizing data 163

7.3.2 SHRLEVEL REFERENCEThis is the first type of ONLINE REORG. SHRLEVEL REFERENCE means that only readers are allowed access to the data, by placing the object in UTRO status, until the SWITCH phase begins. At this point the object is placed into UTUT status and readers removed via a drain on the objects. This means that the table is unavailable to readers until the SWITCH is complete.

By using the DRAIN_WAIT parameter carefully, in conjunction with RETRY, applications need not be given a resource unavailable message and should only notice a slight degradation of response time during this phase.

If the DRAIN WAIT parameter is set to less than IRLMRWT, the application timeout parameter, REORG will give up its drains and return the objects to UTRO before any application processes have timed out, if draining a reader in the SWITCH phase. If REORG cannot drain the writers at the start of the UNLOAD phase, access is set to UTRW. If this happens, REORG will wait the time specified by RETRY_DELAY and will retry the drains, assuming that RETRY is specified. If unsuccessful again, the process is repeated until either the REORG is successful or the maximum number of retries is reached.

If the DRAIN WAIT parameter is set to more than the IRLMRWT, then the REORG has a better chance of completing, but applications are more likely to suffer unavailable resources or timeout.

In the event of the REORG being unsuccessful, the utility will return the original table/index space to its original state and give up all drains. The utility will then act on the TIMEOUT parameter. If it is TERM, the utility will delete all extra data sets, including shadow data sets, and terminate the utility, and no further intervention is required as the original object is still fully available.

To use SHRLEVEL REFERENCE more space is required to hold the “shadow” data sets. These will be the same size as defined in the catalog, assuming DB2 managed data sets.

SHRLEVEL REFERENCE is ideal for objects that are read-only for long periods of time but have time available for the SWITCH phase to complete — for example, QMF users table spaces or tables with LOAD RESUME YES.

7.3.3 SHRLEVEL CHANGEThe third flavour of REORG is SHRLEVEL CHANGE. As the name implies this gives almost 100% access to the data during the entire REORG process.

SHRLEVEL CHANGE works by allowing both readers and writers access to the data until the last log iteration and the SWITCH phase. During the RELOAD phase, applications can access the data in UTRW mode, with any updates being written to the log as normal. Once the RELOAD is finished, REORG starts processing the log records and applying them to the new “shadow” data sets. The log is processed in cycles, called iterations, until DB2 decides that there is little enough log to apply that it can meet the time scales specified in MAXRO parameter.

When this occurs, DB2 will switch the objects into one of two statuses:

� UTRO status, if DRAIN WRITERS is specified� UTUT status, if DRAIN ALL is specified

Then it will issue drains against the objects.


If successful, the last iteration of the log will be processed, the object status will be set to UTUT (if not already done), the readers will be drained (if not already done), and the SWITCH phase will commence (see 7.6.5, “SWITCH” on page 171). When complete, the drains are released, status is set to RW, and REORG deletes unwanted data sets, including the original object data sets. REORG is recorded in SYSCOPY along with any inline copies.

If the drains are unsuccessful, REORG releases any drains it has acquired, returns the status to UTRW, and returns to processing the log. It will wait for the time specified by the RETRY_DELAY parameter (see 7.7.4, “RETRY_DELAY” on page 178) and then restarts the process of changing status and getting drains again. This process is repeated until either the REORG is successful or the maximum number of retries has been reached. Once this has been reached, the utility will then act on the TIMEOUT parameter. If it is TERM, the utility will delete all extra data sets, remove the utility, and place all objects back in RW status.

As per SHRLEVEL REFERENCE, if the DRAIN WAIT is specified as less than the IRLMRWT value, then applications should not be timed out by the REORG.

This method of REORG allows for continuous availability. It should be used in a situation where there is constant access to the DB2 object by updating transactions, and where a high availability is demanded by users.

7.3.4 Availability summaryTable 7-2 shows the availability of data during the different phases of the three types of REORG:

Table 7-2 Online REORG phase unlimited READ / WRITE

7.3.5 RecommendationsFollowing are some recommendations for the use of the various types of REORGs:

� SHRLEVEL NONE must be used for the catalog and directory and for LOBs.� SHRLEVEL NONE is for use when disk space is not available for shadow data sets.� SHRLEVEL NONE should be used to reset REORGP status.� SHRLEVEL REFERENCE is for use where a dedicated batch window is available.

This option is preferred above SHRLEVEL NONE for ease of operations.� SHRLEVEL REFERENCE is for use on (mostly) read-only table spaces.� SHRLEVEL CHANGE is for use when high availability is demanded.

UNLOAD RELOAD SORT BUILD LOG SWITCH BUILD2

NONE R UN UN UN NA NA NA

REFERENCE R R R R NA UN NA

CHANGE R/W R/W R/W R/W RW UN UN-NPI *

Note: R = READ, UN == UNAVAILABLE, NA = NOT APPLICABLE, R/W = READ and WRITE, UN-NPI = UNAVAILABLE for NPI

The last log iteration is not shown in SHRLEVEL CHANGE where the availability is either R or UN, depending upon the DRAIN option.

* For REORGS of individual partitions or ranges of partitions.


With improvements in DB2 Version 7, REORG SHRLEVEL CHANGE should become the standard method of reorganizing table spaces and indexes. The greatest improvement in both usability and performance is the introduction of FASTSWITCH, which eliminates the need for renaming of data sets.

7.4 Planning for Online REORGTo run REORG SHRLEVEL CHANGE on a table space, a mapping table needs to be created prior to running the REORG. This table is used to map any changes in the RIDS during the LOG phases after the data has been RELOADed. This is not required for a reorganization of an index.

A separate mapping table is needed for each REORG running concurrently, even if the parallel jobs are operating on different objects or partition(s). If the REORGs are running serially, then the same mapping table can be used. When sizing the mapping table and index, it is the size of the index that is important, as this is used by the Online REORG. The table space that the mapping table resides in can be sized as small as one track, irrespective of the size of the table space being reorganized.

Ensure that there is sufficient disk available for the shadow data sets, as these are created first by the REORG. If a different pool of disk is to be used for the shadow, then issue an ALTER STOGROUP prior to running the REORG.

If DB2 data sets are user managed, then the target data sets have to be created before starting the Online REORG. Be aware that these data sets should not be deleted afterwards, as DB2 will now be using these as the “live” data sets, assuming that FASTSWITCH is being used (see “FASTSWITCH” on page 172). Also, ensure that the correct naming standards are used. If reorganizing individual partitions, no NPIs will be renamed, but a shadow data set should still be created.

7.5 Shadow data setsPrior to DB2 Version 7, the term “shadow data sets” referred to the ‘S’ instance of the table space that was renamed back to the ‘I’ instance. This method is no longer used with Version 7 and the term “shadow data set” now refers to the new name for the instance, for example, ‘I’ or ‘J’, depending upon the existing data set. See “FASTSWITCH” on page 172 for more details.

Tip: When running multiple Online REORGs in parallel, and creating the mapping table in the job(s), create the mapping tables in separate databases. This reduces contention on the DBD when the mapping table is dropped after completion of the REORG.

Note: Better performance of the mapping table can be achieved by removing the NULLS attribute from the mapping table. An example of CREATE TABLE is included in job DSNTEJ1,.contained in the DB2 sample libraries. This is a change from when DB2 V5 initially shipped this function.


7.6 Phases of Online REORGOnline REORG is, by definition, a REORG that is run with SHRLEVEL REFERENCE or SHRLEVEL CHANGE. In this section the different phases of the REORG are described, see Figure 7-1. For the different types of REORG, different phases are executed. In this section, all phases are described/ See DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945, regarding which phases are executed by which type of REORG.

Figure 7-1 Phases of Online REORG

7.6.1 UNLOADDuring the UNLOAD phase of Online REORG the data is UNLOADed in preparation for RELOADing back into the table space, or partition of a table space. If SHRLEVEL CHANGE is specified, the table space is fully available for all access. If SHRLEVEL REFERENCE is specified, then the objects are placed in UTRO status.

SORTDATA, NOSYSREC, and SORTKEYS are implicitly set for an Online REORG SHARLEVEL CHANGE, assuming that all conditions are met — for example, a clustering index is available. If the table space being reorganized does not have a clustering index available, then NOSYSREC and SORTKEYS will not be specified, and an UNLOAD data set must be provided.

7.6.2 RELOADDuring the RELOAD phase, data is RELOADed into a shadow copy of that table space or partition, while applications can read and write to the original copy of the table space.

Online Reorg Phases

UNLOAD RELOAD SORT BUILD LOG SWITCH BUILD2

Read/Write access tolerated Data unavailability

Only if REORG .... PART + NPIs


During RELOAD, DB2 will create the shadow data sets in the same STOGROUP as the original object, unless the ALTER STOGROUP command has been issued, in which case it will honor the new STOGROUP. These shadow data sets will not be deleted and will become the operation data sets. Therefore ensure that they are specified on the correct volumes, if the STOGROUP has been altered.

It is recommended that if a large NPI exists on a partitioned table space, and individual partitions are being reorganized, then the data set for the logical partition should be manually created. If it is created by DB2, then DB2 will create a full size shadow of the NPI, when only a portion of it is required.

For user-managed data sets, the shadow data sets need to be created using IDCAMS DEFINE CLUSTER before starting the Online REORG. The number of data sets must match the number of objects or partitions being reorganized. Additionally, you need to create data sets for every logical partition of an NPI.

LOG YES is not allowed during the RELOAD phase of an Online REORG, although this does not have the same implications as it does with a SHRLEVEL NONE REORG.

Online REORG forces an image copy to be taken during the process, if COPYDDN is not specified COPYDDN(SYSCOPY) is assumed and the appropriate DD card or Template must exist. The first part of the Inline COPY is taken during the RELOAD process (see Figure 7-2), and during the LOG phase, incremental image copies are added to the data set.

Note: Ensure that shadow data sets are correctly named with either the ‘I’ or ‘J’. The current name can be retrieved from the catalog using the following SQL:

SELECT DBNAME,TSNAME,IPREFIXFROM SYSIBM.SYSTABLEPARTWHERE DBNAME ='dbname' AND TSNAME ='psname'|

and

SELECT DBNAME,IXNAME,IPREFIXFROM SYSIBM.SYSINDEXES X,SYSIBM.SYSINDEXPART YWHERE X.NAME =Y.IXNAME AND X.CREATOR =Y.IXCREATORAND X.DBNAME ='dbname'AND X.INDEXSPACE ='psname';

The original data sets will still remain, and have to be deleted manually using IDCAMS DELETE CLUSTER


Figure 7-2 Common REORG phases using SYSREC data set

The number of incremental copies added will depend upon the number of log iterations undertaken, and will be finished during the LOG Phase (see Figure 7-3). Two is the minimal number of incremental copies that will be added, one prior to the final log iteration and one after the final log iteration.

The SYSCOPY record produced by an Inline COPY during an Online REORG contains:

� ICTYPE = ‘F’� SHRLEVEL = ‘R’� STYPE = ‘W’

See 3.1.1, “Inline COPY” on page 40 for further details on inline copies.

Note: There is only one copy data set produced. There are no incremental image copy data sets produced.

Imagecopy

UNLOAD RELOAD

SORTBLD

SORT BUILD

Original Tablespace

SYSREC

Shadow Copy of orig. TS

SW01WKnn

SW02WKnn

Index1

Index2

DB2 Log

Insert Update Delete Delete Insert Insert Update Delete Delete InsertInsert Update Delete Delete Insert


Figure 7-3 LOG phase of Online REORG

After running an Online REORG, an informational pending state (ICOPY) will be placed on index spaces of related indexes that have the COPY YES attribute. Since ICOPY is not a restrictive state, inline image copies of indexes are not performed during REORG, although it is recommended to take an image copy after the REORG.

7.6.3 SORT and BUILD as compared to SORTBLDAs already mentioned above, the SORTKEYS parameter is always used in an Online REORG. If a suitable clustering index is defined, SORTKEYS means that the index keys will be sorted in memory, and the indexes will be built in parallel during the Sort and Build (SORTBLD) phase. If no suitable index is available, then the SORTBLD is replaced by a SORT and BUILD phase, and work data sets have to be supplied.

SORTKEYS is always used, because it will lead to performance improvements when more than one index needs to be created due to the parallelism. Since the mapping table is built at the same time, Online REORG almost always has to build more than one index. This is valid since Version 6, because in Version 5, the mapping index is built, for the most part, during RELOAD phase, but in insert mode, not load mode.

MAXRO 100DEADLINE NONEDRAIN ALLLONGLOG CONTINUETIMEOUT ABEND

Write access allowed in this iteration = YES NO

1st iteration 300 sec. Estimate for nextiteration: 250 sec.

2nd iteration200 sec. next: 150 sec.

3rd iteration160 sec. next:80 sec.

3. Last iteration120 sec.

Imagecopy taken during

RELOAD Phase

1. Incremental imagecopy is taken2. Drain is performed to stop new log generation

DB2 LOG

4. Final Incremental imagecopy is taken

IIC IIC+ +

LOG phase

(minimum)


7.6.4 LOG phaseThe LOG phases processes the log records for any updates that have occurred since the UNLOAD process began. Any changes that have occurred are now applied to the shadow data sets. The log is read and applied in an iterative manner. While doing this, DB2 estimates how long it will take to process the next iteration of the log and before starting on the next iteration will estimate the amount of time it will take to process this iteration. At incremental image copy is taken prior to the last iteration of the log.

When DB2 estimates that the amount of time required to process the next iteration of the log is less than the parameter specified by the MAXRO parameter (see 7.7.5, “MAXRO” on page 179), DB2 makes a final pass through the log, and the next action by REORG is dependent upon the value of DRAIN (see 7.7.1, “DRAIN” on page 176). Up to this point, the original table space is still available to applications.

If DRAIN ALL is specified, DB2 will place all the objects into UTUT status and try and get a drain on the objects, via the Buffer Manager. If successful, DB2 will apply the last of the log and enter the SWITCH phase. If unsuccessful, DB2 will return to the LOG phase, release all drains, and set the status back to its original state.

If DRAIN WRITERS is specified, DB2 will place all the objects into UTRO and try to drain all writers. If successful, DB2 will apply the last of the log and, before entering the SWITCH phase, it will change the statuses to UTUT and drain all readers. If DB2 is unsuccessful at any time, it will reset the status to the original, release all drains, and re-enter the LOG phase.

DB2 will now continue processing the log until the MAXRO value is met again and the RETRY_DELAY period has passed (see 7.7.4, “RETRY_DELAY” on page 178). Once these conditions are met DB2 re-enters the final log iteration process again. This process continues until the limit for retries has been met, see 7.7.3, “RETRY” on page 178. At this point REORG takes action dependent upon the TIMEOUT parameter, see 7.7.8, “TIMEOUT” on page 180.

The final incremental image copy is taken at the end of the final log iteration process. At this point the table space is in either UTRO or UTUT status, this means that the image copy is equivalent to a SHRLEVEL REFERENCE copy. This means that the whole image copy is a SHRLEVEL REFERENCE copy and is recorded in SYSCOPY as such.

7.6.5 SWITCHAfter applying the log records successfully, DB2 drains the remaining readers, if DRAIN WRITERS was specified, and places the objects in UTUT status. It is now ready to start the SWITCH phase and rename the data sets.

Until DB2 Version 7, DB2 would rename the ‘I’ data sets to a ‘T’ version and then the ‘S’ data sets to the original ‘I’ version after deleting the original data sets. DB2 calls AMS to undertake the renaming, and the time taken by this method depends upon the amount of data sets to be renamed and the access to the MVS catalog.

With DB2 Version 7, a new option is introduced that removes the need for the renaming of data sets and leads to vast improvements in the SWITCH process. This option, FASTSWITCH, is discussed in the next section.

Important: There is only ONE entry in SYSCOPY for the image copy. This is because the incremental image copies are appended to the FULL image copy so there is only one data sets. There is no need to run MERGECOPY to merge the incremental to the FULL copy.


FASTSWITCHPrior releases of DB2 maintained the fifth node of the linear data sets as a fixed value ‘I0001’. This fifth node of the data set name is referred to as the instance node.

With DB2 Version 7, the first character in the instance node can alternate between a value of ‘I’ or ‘J’. In the SWITCH phase, now called FASTSWITCH, all data sets involved in the Online REORG are no longer renamed, as this can be very time-consuming. Instead, the catalog and the directory are updated to point to the shadow data sets rather than to the original data sets, and the original data sets are deleted; see Figure 7-4.

Figure 7-4 FASTSWITCH processing

Some ERP applications, such as SAP and PeopleSoft, design DB2 table spaces with several hundred indexes. Others use partitioned table spaces with some hundred partitions. In both cases, several hundred data sets must be renamed in the SWITCH phase. For the renaming, DB2 invokes Access Method Services (AMS), AMS in turn invokes MVS supervisor calls (SVCs) which result in further cascading SVCs, for example, for checking whether the new name exists already, and whether the rename was successful.

Therefore, the elapsed time of the SWITCH phase can become too long, which can result in applications timing out as the objects are in UTUT status during this time. To avoid this scenario FASTSWITCH has been introduced.

FASTSWITCH is a revised alternative to the processing that speeds up the SWITCH phase by removing invocation of AMS to rename the data sets. The alternative process, invoked by FASTSWITCH, in the various phases, is as follows:

� In the UTILINIT phase, DB2 creates shadow data sets. The fifth qualifier of these data sets is now ‘J0001’, if it is currently ‘I0001’.

FASTSWITCH phaseBefore

REORGUTILINIT SWITCH

AfterREORG

V5, V6

CatalogUpdateI ==> J

UTILTERM

Renames

J

I

I

S

V7 Fast SWITCH

T

I

or viceversa:J ==> I

last LOG

iteration

BU

ILD2

SQLAccess

SQLAccess


� In the SWITCH phase, DB2 updates the catalog and the object descriptor (OBD) from ’I’ to ’J’ to indicate that the shadow object has become the active or valid database object. During that time, SQL applications cannot access the table space.

� After the SWITCH phase, the SQL applications can resume their processing, now on the new ’J0001’ data sets.

� In the UTILTERM phase, DB2 deletes the obsolete original data sets with the instance node ’I0001’, if the data sets are DB2 managed.

As a result of the removal of AMS calls and associated SVC calls, the SWITCH phase is much shorter, therefore applications are less likely to time out.

When reorganizing individual partitions, it is important to remember that not all partitions will have the same fifth node identifier. Some may be ‘I’ or ‘J’, while any NPIs on the table space will be the original node, that is, the same as before the REORG started.

Other points to note about FASTSWITCH are these:

� All new objects are created with ‘I’.

� Node value is recorded in SYSINDEXPART, SYSTABLEPART, column IPREFIX.

� ZPARM &SPRMURNM controls the default for FASTSWITCH (1 = FASTSWITCH). This is defaulted to 1 if not overridden.

� The FASTSWITCH value specified or defaulted in ZPARMS can be overridden by the FASTSWITCH option in the REORG statement.

� S0001 data sets are not tolerated in DB2 V7 even when not using normal SWITCH processing, they have to be J0001.

� DB2 does not handle user managed data sets. All creation and deletion are the responsibility of the user.

� Stand-alone DB2 utilities, such as DSN1COPY, DSN1PRNT, do not identify the correct instance. Instead, query the catalog first to identify the correct data sets.

The only restriction with FASTSWITCH is that it cannot be used with the catalog or directory objects, including their indexes. If specified, the option is ignored, and normal switch processing is used.

If TERM UTIL is issued for the REORG during the SWITCH phase, the objects being REORGed will be returned to their original states and original data sets.

When using concurrent copy, the RECOVER utility has extra work to do to ensure that the correct name is restored for the data sets. Consider the following example:

Note:

� If DB2 data sets are SMS-controlled, ensure that the ACS routines can accommodate the new DB2 naming standard.

� During the last log iteration and during the BUILD2 phase, the SQL access is limited, too; but we ignore this here.

� Ensure that all in-house written code recognizes the new naming standards.


If a concurrent copy is taken, DB2 will check the instance node and place this information in SYSCOPY column STYPE (‘C’ for ‘I0001 and ‘’J’ for ‘J0001). Assuming that the original node is ‘I’, following a FASTSWITCH REORG changes the node to ‘J’. If a QUIESCE point is established after the concurrent copy, then REORG with FASTSWITCH runs. If a recovery is initiated to that QUIESCE point, the following happens:

� Concurrent copy is restored. This has the old instance name, that is, ‘I’ before FASTSWITCH took place.

� RECOVER renames the data set to correct the mismatch, for example, ‘I’ to ‘J’.

� RECOVER proceeds with normal recovery processing, in this case, LOGAPPLY.

See Figure 7-5 for a pictorial representation.

Figure 7-5 FASTSWITCH termination and recovery

FASTSWITCH should be used for all REORGs, where allowed, because it reduces the elapsed time of the SWITCH phases, thereby improving the availability of the objects.

If RECOVER recovers the table space to a point before the FASTSWITCH occurred, the RECOVER utility will still rename the data set to the instance currently in the DB2 catalog. It will not update the catalog.

Fast SWITCH - termination and recovery

- TERM UTIL(reorg-util) during SWITCH phaseAll table space objects are returned to their statesprior to the start of the utility

PIT Recovery works in spite of changed data set namesEven when using concurrent copies:

Instancenode:

- in catalog

Fast SwitchReorg

Quiesce

- of data set

ConcurrentCopy

SYSCOPYICTYPE STYPEF C (J)QW

I (J) J (I)

Log ApplyRestore Rename

PIT Recovery to Quiesce RBA

I (J) J (I) I (J) J (I)

I (J)


7.6.6 BUILD2The BUILD2 phase is only executed for a REORG TABLESPACE PART SHRLEVEL CHANGE or REFERENCE utility when non-partitioning indexes (NPIs) exist for the table space. After completion of the SWITCH phase, all reorganized table space partitions and index space partitions are renamed, and the I or J data sets are now reorganized. Since reorganizing one or more partitions of a table space should not lead to any data unavailability for the unaffected partitions, the data sets of non-partitioning indexes cannot be renamed during the SWITCH phase.

In this case, the REORG utility creates shadow data sets for the NPI, one for each NPI, in which it creates index entries for the logical partitions corresponding to the partitions being reorganized. In Figure 7-6, only the entries of the partitions 2 and 3 are being reorganized. Accordingly, the shadow data sets only contain a subset of the index entries for the NPIs. Instead of renaming the data sets, DB2 initiates the BUILD2 phase, during which Online REORG locates the next key in the logical partition of the NPI which matches the REORG PART statement and updates the key RIDs with the new value from the shadow index (which has been built during the SORTBLD phase).

With DB2 V7, DB2 introduces BUILD2 Parallelism, that is, DB2 dispatches several subtasks, one for each NPI, to perform the updates of the entries in the original NPI data sets. Each subtask is assigned an original NPI and the shadow copy of the logical partition.

The parallelism during the BUILD2 phase introduces two benefits:

� Shorter elapsed time� Greater availability

The degree of parallelism is governed by:

� CPUs available� Available DB2 threads� Number of NPIs

This implies improvements for all cases with more than one NPI, because the elapsed time of the BUILD2 phase is equivalent to the time it takes to process the logical partition with the most RIDS. For further consideration on parallelism within utilities, see 3.7, “Considerations for running parallel subtasks” on page 67.


Figure 7-6 BUILD2 phase parallelism

Unlike COPY, there is no keyword to control the number of parallel subtasks for the BUILD2 phase of REORG. Information about the number of subtasks is provided by the -DIS UTIL command and by message DSNU114I in the job output.

7.7 Controlling Online REORGThis section describes the parameters that affect the operation of Online REORG:

7.7.1 DRAINDRAIN specifies the action that REORG will take when entering the final log iteration. The two options are:

� ALL

REORG will place the object in UTUT status and drain all writers and readers. This should be used when there is a lot of SQL update activity and a large number of deadlocks occur.

� WRITERS

REORG will set the status to UTRO and only drain the writers when entering the last log iteration. When entering the SWITCH phase, DB2 will set the status to UTUT, then drain the readers.

Tip: If reorganizing individual partitions of a partitioned table space and large NPIs exist, create the shadow data sets manually. If DB2 creates the data sets, a full-size NPI is created when only a portion is required.

BUILD2 parallelism

On-line REORG PART (2:3) ==>

Data NPI 1Part. Page Emp-No City

1 2 E0 NYE1 LA

3 E2 NYE3 LA

2 2 E5 LAE4 NY

3 E6 NY3 2 E7 LA

E9 NY3 E8 LA

Key (Part Page

Entry)

LA 1 2 21 3 22 2 13 2 13 3 1

NY 1 2 11 3 12 2 22 3 13 2 2

Instance node: S0nnn (V5,V6) or J0nnn/I0nnnn (V7),where nnn first partition in the range, here: 002

Shadow dataset for NPI

LA 2 2 23 2 13 2 2

NY 2 2 12 3 13 3 1

V5, V6:Sequential

V7:Parallelper NPI

Update entriesrather than replace data set

b

NPI1


The option selected will directly affect the availability of the object to the users. Specifying WRITERS, the default, will provide for the greatest availability, but will increase the REORG time, the number of deadlocks, and the possibility of failure at the SWITCH phase.

With Version 7, the new options RETRY, DRAIN_WAIT and RETRY_DELAY are available. Either DRAIN ALL or DRAIN WRITERS can be used with RETRY. The status is always returned to UTRW. Use the option most suitable to the expected amount of update processing.

7.7.2 DRAIN_WAITThis option specifies the number of seconds that REORG will wait for DML activity to finish when issuing a drain before timing out. The value specified can range between 0 and 1800. If omitted, then the IRLMRWT and UTIMOUT from ZPARMS values are used. DRAIN_WAIT provides a more consistent wait time than the ZPARM options, since it refers to the aggregate time for a set of objects. Example 7-1 shows an example of DRAIN_WAIT being set to 50 seconds, while Example 7-2 shows the job output for the utility. The utility waited 50 seconds in the SWITCH phase before abending.

Example 7-1 DRAIN_WAIT parameter

//DSNUPROC.SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 INCLUDE TABLESPACE DB246129.TS612902 TEMPLATE MARCELO UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..T&TIME.) DISP(MOD,CATLG,CATLG) REORG TABLESPACE LIST LISTA1 SHRLEVEL CHANGE MAPPINGTABLE DSN8710.MAP_TBL COPYDDN MARCELO DRAIN_WAIT 50

Example 7-2 REORG output using DRAIN_WAIT parameter

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = TEMP DSNU050I DSNUGUTC - LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 INCLUDE TABLESPACE DB246129.TS612902 DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE MARCELO UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..T&TIME.) DISP(MOD,CATLG,CATLG) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - REORG TABLESPACE LIST LISTA1 SHRLEVEL CHANGE MAPPINGTABLE DSN8710.MAP_TBL COPYDDN MARCELO DRAIN_WAIT 50 DNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DB246129.TS612901 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=MARCELO DDNAME=SYS00001 DSN=PAOLOR4.UTIL.TS612901.T205136 DSNU252I DSNUGSRT - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=160 FOR TABLESPACE DB246129.TS612901 DSNU250I DSNUGSRT - UNLOAD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=160 FOR TABLE PAOLOR7.CUST DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=160 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=320 ELAPSED TIME=00:00:00 DSNU349I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=160 FOR INDEX PAOLOR7.XCUSTNO DSNU349I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=160 FOR INDEX DSN8710.XMAP_TBL DSNU258I DSNURBXD - BUILD PHASE STATISTICS - NUMBER OF INDEXES=2 DSNU259I DSNURBXD - BUILD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU590I -DB2G DSNURDRN - RESOURCE NOT AVAILABLE, REASON=X'00C9008E', ON DB246129.TS612901 PROHIBITS PROCESSING DSNT500I DSNUGBAC - RESOURCE UNAVAILABLE REASON 00C200EA TYPE 00000200 NAME DB246129.TS612901 DSNU017I DSNUGBAC - UTILITY DATA BASE SERVICES MEMORY EXECUTION ABENDED, REASON=X'00E40009'


When calculating a value for DRAIN_WAIT, consider the following:

� If this value is greater than IRLMRWT, applications may experience timeouts and the availability of the object will be reduced.

� If this value is less than IRLMRWT, applications should not experience timeouts, because the utility will relinquish the drains before timeout occurs. Applications may experience a slight degradation in performance during this time.

Therefore, examine the requirement for the REORG to finish successfully and quickly. If it is less than the availability requirements, set this limit lower than IRLMRWT. Similarly, when used with RETRY parameters, it is recommended to set this value lower than IRLMRWT.

7.7.3 RETRYRETRY specifies the number of times that REORG will attempt to drain all objects being REORGed. When REORG fails to get a drain, the RETRY count is decremented by 1, when the count reaches 0, the REORG terminates in accordance with the TIMEOUT parameter. Before retrying the drains, REORG will make the objects fully available again and process more log, for the time specified in the RETRY_DELAY parameter. Before retrying the drain, the MAXRO condition must be met. If the RETRY parameter is not specified, there is no retry processing and the REORG ends unsuccessfully.

The amount of retries directly affects:

� The processing costs of the REORG

� The amount and duration of read-only access (DRAIN WRITERS) or no access (DRAIN ALL)

� The size of the image copy data sets, due to an increase in the incremental copies being taken.

It is recommended that RETRY should be specified to increase the chance of REORG finishing successfully and should be used in conjunction with RETRY_DELAY to allow time for threads to complete.

7.7.4 RETRY_DELAYThis parameter specifies the amount of time, in seconds, that REORG will wait before attempting to drain the objects in the case of RETRY being specified. The default is 300 seconds, although in most occasions this is probably too high and can be safely reduced.

Example 7-3 Online REORG with DRAIN_WAIT, RETRY and RETRY_DELAY

//DSNUPROC.SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 INCLUDE TABLESPACE DB246129.TS612902 TEMPLATE MARCELO UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..T&TIME.) DISP(MOD,CATLG,CATLG) REORG TABLESPACE LIST LISTA1 SHRLEVEL CHANGE MAPPINGTABLE DSN8710.MAP_TBL COPYDDN MARCELO DRAIN_WAIT 50 RETRY 2 RETRY_DELAY 10


Example 7-4 Online REORG DRAIN_WAIT, RETRY and RETRY_DELAY output

DSNU349I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=160 FOR INDEX PAOLOR7.XCUSTNO DSNU349I -DB2G DSNURBXA - BUILD PHASE STATISTICS - NUMBER OF KEYS=160 FOR INDEX DSN8710.XMAP_TBL DSNU258I DSNURBXD - BUILD PHASE STATISTICS - NUMBER OF INDEXES=2 DSNU259I DSNURBXD - BUILD PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU1122I -DB2G DSNURLOG - JOB PAOLOR4A PERFORMING REORG WITH UTILID TEMP UNABLE TO DRAIN DB246129.TS612901. RETRY 1 OF 2 WILL BE ATTEMPTED IN 10 SECONDS DSNU1122I -DB2G DSNURLOG - JOB PAOLOR4A PERFORMING REORG WITH UTILID TEMP UNABLE TO DRAIN DB246129.TS612901. RETRY 2 OF 2 WILL BE ATTEMPTED IN 10 SECONDS DSNU590I -DB2G DSNURDRN - RESOURCE NOT AVAILABLE, REASON=X'00C9008E', ON DB246129.TS612901 PROHIBITS PROCESSING DSNT500I DSNUGBAC - RESOURCE UNAVAILABLE REASON 00C200EA TYPE 00000200 NAME DB246129.TS612901 DSNU017I DSNUGBAC - UTILITY DATA BASE SERVICES MEMORY EXECUTION ABENDED, REASON=X'00E40009'

Example 7-3 and Example 7-4 have shown the JCL and the output from an Online REORG using the DRAIN_WAIT, RETRY and RETRY_WAIT parameters. In total, the job waited for 2 minutes and 50 seconds, which can be calculated using as follows:

TOTAL TIME = (DRAIN_WAIT + RETRY_DELAY) * RETRY + DRAIN_WAIT

7.7.5 MAXROMAXRO specifies goal for the maximum amount of time that REORG is allowed to keep the object in UTRO, for DRAIN WRITERS, or UTUT, for DRAIN ALL, while applying the last iteration of log records. During log processing, REORG calculates the amount of time that will be taken to apply the log records. If this value is less than the MAXRO value, then DB2 enters the last log iteration, sets the object status (UTUT or UTRO), and attempts to drain the objects.

The value for MAXRO can be an integer value or the word DEFER. The value DEFER indicates to REORG that it is not to enter the final log iteration until told. REORG will keep processing the log until the MAXRO value is changed to an integer value. The MAXRO is changed to an integer via the command -ALTER UTILITY(utilid) MAXRO(integer). Once changed, the REORG begins processing the log as normal, looking for a time to enter the last log iteration.

The benefits of MAXRO DEFER is that it enables the REORG to be finished at a time that is convenient to the application. An Online REORG could be left processing in the background and finished by the ALTER command when low activity is detected. Be aware that the image copy data sets could get large if there is a lot of update activity during the DEFER period.

The DEFER option is recommended for controlling the finishing time of a REORG when the time that is available for SWITCH phase is variable. This option should be used in conjunction with LONGLOG CONTINUE.

If DEFER is specified, and DB2 determines that the actual time for an iteration and the estimated time for the next iteration are both less than 5 seconds, DB2 adds a 5-second pause to the next iteration to reduce resource consumption. The first time this situation occurs for a given execution of REORG, DB2 sends the message DSNU362I to the console. The message states that the number of log records that must be processed is small and that the pause will occur.

When this message is issued, this is an appropriate time to execute ALTER UTILITY to change the MAXRO value, and thus cause REORG to finish. DB2 adds the pause whenever the situation occurs. However, DB2 sends the message only if 30 minutes have elapsed since the last message was sent for a given execution of REORG. DB2 will issue an operator message every 30 minutes.


7.7.6 LONGLOGWhen REORG is processing the log and the number of log records being processed is not dropping significantly, or actually increasing, DB2 will issue a message to the console to this effect. This signifies that DB2 is not catching up on the log processing, due to the amount of update activity against the object being reorganized. The LONGLOG parameter can be utilized in this situation. It has three values:

� TERM

This terminates the utility after the delay specified by the DELAY parameter.

� DRAIN

In effect, this forces the final iteration of the log process after the delay specified.

� CONTINUE

This continues processing until the time on the JOBCARD statement is reached.

A value of CONTINUE, the default, along with MAXRO DEFER, means that the REORG never switches to the shadow data sets and allows continuous read and write access to the original objects. Processing will continue when the MAXRO has been altered to an integer value.

This parameter affects the size of the first incremental image copy. If LONGLOG is CONTINUE and the update activity is significant, then the incremental size will be larger.

It is recommended that you use LONGLOG CONTINUE when specifying MAXRO DEFER.

7.7.7 DELAYThis specifies the time, in seconds, between the LONGLOG condition being met and the console message being issued, and is the time that REORG performs the action specified in the LONGLOG parameter.

7.7.8 TIMEOUTTIMEOUT specifies the action to be taken if the REORG is timed out during the LOG or the SWITCH phase.

The options are:

� TERM

This option issues a TERM UTIL command ending the utility with RC8 and leaves the objects in RW status

� ABEND

Does not terminate the utility and leaves the objects in UTRO or UTUT depending upon the phase and the DRAIN options

Online REORG is only restartable in the BUILD2 and SWITCH phase, and therefore, ABEND is the default.

The recommendation is that ABEND is the correct parameter for long running REORGs to allow for restartability and to utilize machine resources wisely. Using ABEND will have an affect on the availability of the object, because restart will be needed or a manual terminate.

For smaller table spaces and shorter running REORGs, TERM should be used where restartability is not an issue.


7.7.9 DEADLINEDEADLINE is either a timestamp value or a timestamp expression that indicates a time when the SWITCH phase must finish by. If it is not specified, then there is no deadline.

If REORG estimates that it cannot finish processing by the deadline, then it will issue the DISPLAY UTILITY command, terminate the REORG, and issue the message DSNU374I with reason code 2. All objects are left in a non-restrictive status.

The output from the DISPLAY UTILITY should be viewed and decisions made as to whether the deadline time is suitable for the object and the workload at the time the REORG was running.

7.7.10 RecommendationsThe chances of Online REORGs completing successfully is greatly increased if there are good application standards regarding commit frequency. Long units of work can inhibit the SWITCH phase because drains cannot be obtained. Therefore it is recommended that Online REORGs should not be run while long running units of work are executing.

When moving to DB2 Version 7, it is strongly recommended that the following options should be used for Online REORG:

� DRAIN� DRAIN_WAIT� RETRY� RETRY_DELAY

These options will increase the chances of REORG finishing successfully without impacting the applications.

If an Online REORG fails, the implications are not as severe as for a SHRLEVEL NONE REORG. The Online REORG can be terminated, dependent upon which phase, and all objects revert to their original state. This potentially can lead to fewer out-of-hours support being necessary.

It is also recommended that inline copies and statistics (at least for space) are gathered during the REORG to minimize impact on resources and to increase availability.

7.8 Further optionsThis section details some of the other options available with REORG, both online and offline varieties.

7.8.1 LISTSWith DB2 Version 7, the concept of LISTDEF and TEMPLATES was introduced (see Chapter 2, “Wildcarding and Templates” on page 17), which enables lists of objects to be built and acted upon by a utilities. All objects in a given list are processed, potentially in parallel, by the same utility with the same options. Templates allow for dynamic data set allocations and automatic intelligent data set sizing.

REORG can fully exploit this functionality, and the use of the LISTDEF and TEMPLATE functionality is highly recommended to reduce administration.


Caution must be exercised, because all options for the REORG are applied to all objects. Therefore, care must be taken to ensure that all objects included in the list are suitable for the REORG parameters — for example, that the DEADLINE expression specifies a deadline that is realistic for the objects.

It is recommended that the lists built should contain table spaces/index spaces of a similar nature to which “global” parameters can be applied.

For examples of LISTDEF, see Example 7-1 on page 177 and Example 7-3 on page 178.

7.8.2 SORTKEYSSORTKEYS eliminates the need for index keys to be written to data sets for sorting and then read from the data set for building of the index. All keys are instead passed in memory to a combined SORT and BUILD phase named SORTBLD. SORTKEYS can now also work in parallel using multiple subtasks to reduce the elapsed time of the REORG.

In Version 7, REORG can now exploit parallel index build features which allows non-partitioning indexes to be built in parallel with a separate subtask per build; see 7.6.6, “BUILD2” on page 175 for further discussions. This reduces the contention on the NPI build that was a problem in DB2 Version 6.

Further information about REORGs and use of SORTKEYS can be found in Chapter 3, “Inline executions and parallelism” on page 39.

7.8.3 SORTDATASORTDATA specifies that the data in the table space is to be UNLOADed via a table space scan. Once UNLOADed the data is then sorted into clustering sequence using DFSORT or similar program.

This option is recommended unless any of the following conditions are true:

� The data is in near-perfect clustered order and REORG is being done to reclaim space.

� There is insufficient SORTWORK space available.

� The record length for the composite record is greater than 32760.

� REORG is running against catalog or directory table spaces that are prohibited from using this option.

� An explicitly defined clustering index does not exist.

SORTDATA is strongly recommended unless the data set being reorganized is very large and the SORTWORK space is at a premium.

7.8.4 NOSYSRECNOSYSREC specifies that the output from the SORTDATA is to be the input for reloading without externalizing the data to the SYSREC data set. This option is only available if SORTKEYS is specified and the SHRLEVEL of the REORG is REFERENCE or NONE.

The advantage of using NOSYSREC is the reduction in elapsed time due to the removal of I/O to the SYSREC data set.


Using this option has restart implications in the event of a failure:

� If the utility fails during UNLOAD of a SHRLEVEL REFERENCE REORG, then the REORG has to be restarted from the start of this phase.

� If the utility fails during RELOAD of a SHRLEVEL NONE REORG, then the REORG has to be terminated and a RECOVER TABLESPACE utility. It is therefore recommended that image copies should be taken immediately prior to the execution of the REORG.

SHRLEVEL CHANGE does not use a SYSREC file, and therefore this option is automatically used along with SORTDATA and SORTKEYS.

7.8.5 DFSORTWhen specifying SORTDATA or SORTKEYS with REORG against a large table, it may be necessary to override the default DFSORT parameters. To achieve this, add a DFSPARM DD card in the JCL; see Example 7-5. In this example, the storage available to the sort process is set to 24M, with work data sets being spread across 8 volumes.

Example 7-5 Changing DFSORT defaults

//DFSPARM DD * OPTION MAINSIZE=24000K,NOWRKREL,DYNALLOC=(WRKDSK,8),DSPSIZE=60

This method can also be used to override the FILESZ parameter, which may be necessary when dealing with compressed rows.

7.8.6 KEEPDICTIONARYThe option KEEPDICTIONARY tells REORG TABLESPACE not to rebuild the compression dictionary during RELOAD, but to keep the dictionary currently in the table space. This option is only available for compressed table spaces and where a compression dictionary exists. If a dictionary does not exist, then one is built, and warning messages are issued.

The building of a dictionary is an overhead to the REORG utility and is an overhead that needs not be incurred each time a table space is REORGed. It does not guarantee a better functioning dictionary. It is recommended that the effectiveness of the existing dictionary is monitored by using the column PAGESAVE in the new Version 7 catalog table SYSTABLEPART_HIST.

By monitoring the PAGESAVE value, it can be determined when the degradation of the dictionary from the original compression is greater than 10%. At this point, a dictionary rebuild should be scheduled by omitting the KEEPDICTIONARY keyword. See Figure 7-7, for a “rule of thumb“ to help decide when to rebuild a dictionary.

Figure 7-7 When to rebuild a dictionary

pagesave 73% 71% 69% 67% 64%

if (max(pagesave)-min(pagesave))*100 /max(pagesave) > 10= (73-64)*100/73= 12


7.8.7 REUSEREUSE is a new feature of the REORG utility introduced in DB2 V5. It can only be used with STOGROUP defined data sets, also called DB2-managed data sets and REORG SHRLEVEL NONE. Normally, if you do not specify REUSE, the REORG of a DB2-managed data set will delete and redefine the underlying VSAM data sets. With the REUSE option, the underlying VSAM data sets will not be deleted and redefined, but only logically reset; that means the high used RBA is reset back to zero.

The logical reset time of a VSAM data set is much shorter than the physical delete and redefine time. REORG REUSE will not reclaim any secondary extents because it is only logically resetting the VSAM data set.

The REUSE option can operate on the entire table space and its index spaces, or on a partition of a partitioned table space and its corresponding partition index space. In the latter case, you should specify the PART integer as illustrated in Example 7-6.

Example 7-6 Specifying the REUSE option during REORG

Logically reset and reuse of an entire table space and associated index spaces: REORG TABLESPACE DB1.TS1 REUSE ........

Logically reset and reuse of a table space partition and associated index partition:REORG TABLESPACE DB1.TS1 REUSE PART 1

We recommend using the REUSE option in the following circumstances:

� If you want to preserve the allocated space on the disk volumes between consecutive REORG utilities (for I/O tuning reasons or to avoid space problems in case of reallocation).

� To reduce CPU and elapsed times, if you have partitioned table spaces with a lot of underlying VSAM data sets.

� To increase the throughput by avoiding service task contention, if you have a lot of concurrent REORG utilities run in parallel. Since DB2 V6, DB2 can have up to 20 service tasks run in parallel.

Do not use the REUSE option:

� If you want to reduce the number of secondary extents. � If you want to activate altered PRIQTY and SECQTY values.� If you want to move the data sets to another STOGROUP. � If running REORG SHRLEVEL REFERENCE/CHANGE.

7.8.8 PREFORMATThe PREFORMAT option of the REORG utility was introduced in DB2 V5. It will preformat a table space and its associated index spaces during the REORG and prepare it for INSERT processing. This eliminates the need to preformat new pages in a table space during INSERT processing and reduce execution time delays. However, when the preformatted space is utilized and DB2 has to extend the table space, normal data set extending and preformatting will occur.

Preformatting causes all free pages in the VSAM cluster (between the high used RBA and high allocated RBA) to be preformatted. This includes secondary extents that may have been allocated. Preformatting occurs after the data has been loaded and the indexes are built.


Preformatting can operate on the entire table space and its index spaces, or, it can operate on a partition of a partitioned table space and its corresponding partition index space. Preformatting an entire partitioned table space at the table space level can inhibit concurrent processing of separate partitions. In that case, you should specify PART integer PREFORMAT to serialize on the partition level, as illustrated in Example 7-7.

Example 7-7 Preformatting a table space and its index spaces during REORG

Preformatting an entire table space and associated index spaces: REORG TABLESPACE DB1.TS1 PREFORMAT

Preformatting a table space partition and associated index partition:REORG TABLESPACE DB1.TS1 PREFORMAT PART 1

Preformatting is only recommended for table spaces and index spaces that are part of high INSERT applications with high performance requirements where normal formatting can cause unacceptable delays or inconsistent elapsed times.

It is not recommended for tables that are mainly used for query processing. The additional preformatted pages can cause table space scans to read additional empty pages, extending the elapsed time for these queries.

So, you should use the PREFORMAT option with care. General use of this option in all your REORG jobs is not recommended. Best results may be seen when the final size of the table space is known and when there is a high ratio of inserts to read operations.

7.9 DISCARDPrior to DB2 Version 6, the only method of removing data from a table space was using one of the following methods:

� SQL DELETE from the table and then REORG afterwards

� DSNTIAUL to select the rows to be KEPT and then run LOAD REPLACE

� DSNTIAUL to UNLOAD all rows and then manipulate the UNLOAD file before running a LOAD REPLACE of the table space.

The last two options can only be used for single table spaces. For table spaces containing multiple tables, all tables would have to be UNLOADed and RELOADed.

DB2 Version 6 introduced the DISCARD parameter to the REORG. This parameter allows to delete selected records during a full REORG process (UNLOAD CONTINUE or UNLOAD PAUSE) and also allows data to be written to a discard file, specified by DISCARDDN keyword. Only rows that did not meet the selection criteria are RELOADed back into the table space. The selection process is specified by using the syntax FROM TABLE table WHEN condition. If DISCARD is specified without a WHEN clause, no rows are selected for discarding.

The WHEN clause conditions convert directly to stage 1 type predicates, and therefore the following restrictions apply:

� Column-to-column comparisons are not allowed.� Arithmetic operations are not allowed.� LIKE on columns with FIELDPROCs are not allowed.

Note: DB2 V7 introduced asynchronous INSERT preformatting. This new function will asynchronously preformat the next range of new pages during INSERT processing if the current page is within an internally predefined range from the end of the formatted pages.


� Literal values for ASCII tables must be in HEX.

As with DSNTIAUL, a LOAD statement is generated to allow the discarded or UNLOADed rows to be loaded into another table space. This feature can be used to remove “older” data and place it in history tables that reside on slower devices, or the data could be archived off to another media, for example, fiche.

An example of using the discard process is given in Example 7-8, which shows all rows in table POALOR2.LINEITEM with L_PARTKEY = 107038 being discarded into the file DISOUT. NOSYSREC, SORTDATA, and SORTKEYS have been specified to avoid allocating a SYSREC data set. The job output from the example is shown in Example 7-9.

Example 7-8 REORG using DISCARD processing

TEMPLATE DISOUT DSN(DB2V710G.&DB..&TS..D&DATE..DISOUT) VOLUMES(SBOX58,SBOX59, SBOX57, SBOX60) TEMPLATE PUNCH DSN(DB2V710G.&DB..&TS..D&DATE..SYSPUNCH) VOLUMES(SBOX58,SBOX59, SBOX57, SBOX60) TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..P&PA..L) VOLUMES(SBOX58,SBOX59, SBOX57, SBOX60) REORG TABLESPACE U7G01T11.TESTNPIS SORTKEYS NOSYSREC SORTDATA SORTNUM 4 SHRLEVEL REFERENCE COPYDDN(LOCALDDN) PUNCHDDN(PUNCH) DISCARDDN(DISOUT) DISCARD FROM TABLE POALOR2.LINEITEM WHEN (L_PARTKEY = 107038)

Example 7-9 REORG using DISCARD job output

1DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CPITEM0DSNU050I DSNUGUTC - TEMPLATE DISOUT DSN(DB2V710G.&DB..&TS..D&DATE..DISOUT) VOLUMES(SBOX58,SBOX59,SBOX57,SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - TEMPLATE PUNCH DSN(DB2V710G.&DB..&TS..D&DATE..SYSPUNCH) VOLUMES(SBOX58,SBOX59,SBOX57, SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..P&PA..L) VOLUMES(SBOX58,SBOX59,SBOX57, SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - REORG TABLESPACE U7G01T11.TESTNPIS SORTKEYS NOSYSREC SORTDATA SORTNUM 4 SHRLEVEL REFERENCE COPYDDN(LOCALDDN) PUNCHDDN(PUNCH) DISCARDDN(DISOUT) DISCARD DSNU650I -DB2G DSNUUGMS - FROM TABLE POALOR2.LINEITEM WHEN DSNU650I -DB2G DSNUUGMS - (L_PARTKEY=107038) DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=PUNCH DDNAME=SYS00001 DSN=DB2V710G.U7G01T11.TESTNPIS.D2001177.SYSPUNCH DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=DISOUT DDNAME=SYS00002 DSN=DB2V710G.U7G01T11.TESTNPIS.D2001177.DISOUT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LOCALDDN DDNAME=SYS00003 DSN=DB2V710G.U7G01T11.TESTNPIS.D2001177.P00000.L DSNU241I -DB2G DSNURBDC - DICTIONARY WITH 4096 ENTRIES HAS BEEN SUCCESSFULLY BUILT FROM 1360 ROWS FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 1 DSNU241I -DB2G DSNURBDC - DICTIONARY WITH 4096 ENTRIES HAS BEEN SUCCESSFULLY BUILT FROM 1108 ROWS FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 2 DSNU241I -DB2G DSNURBDC - DICTIONARY WITH 4096 ENTRIES HAS BEEN SUCCESSFULLY BUILT FROM 1114 ROWS FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 3 DSNU253I DSNUUMSG - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=1951540 FOR TABLE POALOR2.LINEITEM DSNU253I DSNUUMSG - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS DISCARDED=8 FOR TABLE POALOR2.LINEITEM DSNU252I DSNUGSRT - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=1951540 FOR TABLESPACE U7G01T11.TESTNPIS DSNU250I DSNUGSRT - UNLOAD PHASE COMPLETE, ELAPSED TIME=00:00:43 DSNU395I DSNURPIB - INDEXES WILL BE BUILT IN PARALLEL, NUMBER OF TASKS = 4 DSNU400I DSNURBID - COPY PROCESSED FOR TABLESPACE U7G01T11.TESTNPIS NUMBER OF PAGES=33413 AVERAGE PERCENT FREE SPACE PER PAGE = 3.38


PERCENT OF CHANGED PAGES =100.00 ELAPSED TIME=00:01:32 DSNU244I -DB2G DSNURWT - COMPRESSION REPORT FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 1 22598 KB WITHOUT COMPRESSION 12624 KB WITH COMPRESSION 44 PERCENT OF THE BYTES SAVED FROM COMPRESSED DATA ROWS 100 PERCENT OF THE LOADED ROWS WERE COMPRESSED 118 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH 67 BYTES FOR AVERAGE COMPRESSED ROW LENGTH 5985 PAGES REQUIRED WITHOUT COMPRESSION 3414 PAGES REQUIRED WITH COMPRESSION 42 PERCENT OF THE DB2 DATA PAGES SAVED USING COMPRESSED DATA DSNU244I -DB2G DSNURWT - COMPRESSION REPORT FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 2 22508 KB WITHOUT COMPRESSION 12694 KB WITH COMPRESSION 43 PERCENT OF THE BYTES SAVED FROM COMPRESSED DATA ROWS 100 PERCENT OF THE LOADED ROWS WERE COMPRESSED 118 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH 68 BYTES FOR AVERAGE COMPRESSED ROW LENGTH 5961 PAGES REQUIRED WITHOUT COMPRESSION 3451 PAGES REQUIRED WITH COMPRESSION 42 PERCENT OF THE DB2 DATA PAGES SAVED USING COMPRESSED DATA DSNU244I -DB2G DSNURWT - COMPRESSION REPORT FOR TABLE SPACE U7G01T11.TESTNPIS, PARTITION 3 174993 KB WITHOUT COMPRESSION 98307 KB WITH COMPRESSION 43 PERCENT OF THE BYTES SAVED FROM COMPRESSED DATA ROWS 100 PERCENT OF THE LOADED ROWS WERE COMPRESSED 118 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH 67 BYTES FOR AVERAGE COMPRESSED ROW LENGTH 46345 PAGES REQUIRED WITHOUT COMPRESSION 26329 PAGES REQUIRED WITH COMPRESSION 43 PERCENT OF THE DB2 DATA PAGES SAVED USING COMPRESSED DATA DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=1951540 FOR TABLE POALOR2.LINEITEM DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED=1951540 DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:01:33 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=200364 FOR INDEX POALOR2.PXL#OKSDRFSKEPDC PART 1 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=199549 FOR INDEX POALOR2.PXL#OKSDRFSKEPDC PART 2 DSNU393I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=1551627 FOR INDEX POALOR2.PXL#OKSDRFSKEPDC PART 3 DSNU394I -DB2G DSNURBXA - SORTBLD PHASE STATISTICS - NUMBER OF KEYS=1951540 FOR INDEX POALOR2.TESTNPI DSNU391I DSNURPTB - SORTBLD PHASE STATISTICS. NUMBER OF INDEXES = 2 DSNU392I DSNURPTB - SORTBLD PHASE COMPLETE, ELAPSED TIME = 00:00:07 DSNU387I DSNURSWT - SWITCH PHASE COMPLETE, ELAPSED TIME = 00:00:01 DSNU428I DSNURSWT - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TESTNPIS DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

7.10 UNLOAD EXTERNALREORG UNLOAD EXTERNAL is a faster method for unloading data than previous methods, for example DSNTIAUL. As with DSNTIAUL and the REORG DISCARDS option a punch data set is provided to enable the data to be loaded into another table. To specify the data to be UNLOADed the WHEN clause is used in the same way as described in 7.9, “DISCARD” on page 185. An example is shown in Example 7-10 and the output in Example 7-11.

Example 7-10 REORG UNLOAD EXTERNAL

TEMPLATE PUNCH DSN(DB2V710G.&DB..&TS..D&DATE..PUNCH) VOLUMES(SBOX58,SBOX59, SBOX57, SBOX60) TEMPLATE UNLOAD DSN(DB2V710G.&DB..&TS..D&DATE..UNLD) VOLUMES(SBOX58,SBOX59, SBOX57, SBOX60) REORG TABLESPACE U7G01T11.TESTNPIS SORTKEYS SORTNUM 4 SHRLEVEL NONE PUNCHDDN(PUNCH) UNLDDN(UNLOAD)

Note: The enhancement described in this section has also been made available for DB2 Version 5 with APAR PQ19897.


UNLOAD EXTERNAL FROM TABLE POALOR2.LINEITEM WHEN (L_PARTKEY = 107838)

Example 7-11 REORG UNLOAD EXTERNAL job output

1DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CPITEM0DSNU050I DSNUGUTC - TEMPLATE PUNCH DSN(DB2V710G.&DB..&TS..D&DATE..PUNCH) VOLUMES(SBOX58,SBOX59,SBOX57,SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - TEMPLATE UNLOAD DSN(DB2V710G.&DB..&TS..D&DATE..UNLD) VOLUMES(SBOX58,SBOX59,SBOX57,SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - REORG TABLESPACE U7G01T11.TESTNPIS SORTKEYS SORTNUM 4 SHRLEVEL NONE PUNCHDDN(PUNCH) UNLDDN( UNLOAD) UNLOAD EXTERNAL DSNU650I -DB2G DSNUUGMS - FROM TABLE POALOR2.LINEITEM WHEN DSNU650I -DB2G DSNUUGMS - (L_PARTKEY=107838) DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=UNLOAD DDNAME=SYS00001 DSN=DB2V710G.U7G01T11.TESTNPIS.D2001177.UNLD DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=PUNCH DDNAME=SYS00002 DSN=DB2V710G.U7G01T11.TESTNPIS.D2001177.PUNCH DSNU253I DSNUUMSG - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=26 FOR TABLE POALOR2.LINEITEM DSNU252I DSNURULD - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=26 FOR TABLESPACE U7G01T11.TESTNPIS DSNU250I DSNURULD - UNLOAD PHASE COMPLETE, ELAPSED TIME=00:00:21DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

UNLOAD EXTERNAL has largely been replaced by the new utility UNLOAD, which is covered in Chapter 8, “Unloading data” on page 191.

7.11 Index REORGIndex REORG utilizes many of the same options as specified for table space, and the descriptions and recommendations apply equally to both types of objects. The main difference is that Online REORG of an index does not require a mapping table to be used, and Inline COPY cannot be used, even if the COPY YES attribute is set.

7.12 Catalog reorganizationIt is recommended that catalog and directory indexes should be reorganized in the same way that applications indexes are reorganized. When catalog indexes become disorganized, this affects both the performance of queries against the catalog and DB2 performance when index access is used.

Catalog and directory table spaces need only be reorganized infrequently and mainly for the reclamation of space or for the performance of queries that use table space scans against the catalog.

7.13 Inline Utilities with REORGREORG can now run both the COPY and RUNSTATS during the RELOAD phase of the utility.

Note: PTF UQ54731 for APAR PQ49114 should be applied to disable FASTSWITCH on catalog index REORGs.


7.13.1 Inline COPYAs well as taking RUNSTATS inline, a full image copy data set can also be taken inline during the reorganization of a table space. Again, the advantages are a reduction in the total elapsed time when compared to the running of two separate utilities. Also, the table space is not left in copy pending status at the end of a REORG LOG NO, thus increasing data availability. The copy is recorded in SYSCOPY as a FULL SHRLEVEL REFERENCE copy.

The content of the image copy is different for a REORG SHRLEVEL CHANGE, as it can contain between 1 and ‘n’ incremental image copies in addition to the full copy. The number of incremental image copies will depend upon the number of log iterations activated by the REORG utility. They are still valid for the RECOVER utility.

� See 3.1.1, “Inline COPY” on page 40 for more information about taking inline copies.

� See 10.1.1, “Inline COPY with LOAD and REORG” on page 232 for more information on copies taken together with the REORG utility.

7.13.2 Inline RUNSTATSSince Version 6 of DB2, it is possible to gather the statistics during the execution of the REORG utility. Using this option reduces the elapsed time of a REORG followed by RUNSTATS. This is achieved by eliminating the need to for RUNSTATS to scan the data to collect the statistics. Instead, this is achieved by reading the data during the REORG.

See Example 7-12 for the use of Statistics command. Example 7-13 gives the output from a -DISPLAY UTIL command, which shows RUNSTATS running as a subphase of the RELOAD step.

Example 7-12 Collecting inline statistics

//DSNUPROC.SYSIN DD * REORG TABLESPACE UGG01T11.TSPSUPP1 STATISTICS TABLE(ALL) SAMPLE (100)

Example 7-13 -DISPLAY UTILITY with active utility

DSNU105I -DB2G DSNUGDIS - USERID = PAOLOR4 MEMBER = UTILID = TEMP PROCESSING UTILITY STATEMENT 1 UTILITY = REORG PHASE = RELOAD COUNT = 554876 NUMBER OF OBJECTS IN LIST = 1 LAST OBJECT STARTED = 1 STATUS = ACTIVE DSNU111I -DB2G DSNUGDIS - SUBPHASE = RUNSTATS COUNT = 20378 DSN9022I -DB2G DSNUGCCC '-DIS UTIL' NORMAL COMPLETION

Important: The size of the image copy data set taken by REORG SHRLEVEL CHANGE will be larger than a FULL image copy due to the incremental copies appended to the data set. The size is proportional to the number of log iterations and the amount of changed pages during the iterations.


Version 7 allows the collection of historical statistics, and this option can also be used during the REORG utility. The statistics collected during a REORG SHRLEVEL CHANGE reflect the statistics at the end of the RELOAD phase, they do no reflect any changes applied during the LOG phases.

� See 3.1.2, “Inline RUNSTATS” on page 46 for more information on running RUNSTATS within a utility.

� See Chapter 11., “Gathering statistics” on page 253 for more information on the RUNSTATS options.


Chapter 8. Unloading data

In prior versions of DB2, data has been unloaded by using the DB2 supplied assembler program DSNTIAUL, the REORG UNLOAD EXTERNAL utility, and the stand-alone utility DSN1COPY.

DB2 V7 has introduced the new UNLOAD utility. UNLOAD can be used to unload data from one or more source objects to one or more BSAM sequential data sets in external formats. The source can be:

� DB2 table spaces� DB2 Image Copy data sets

When comparing to previous methods, the UNLOAD utility provides:

� Enhanced functionality� Better performance� Improved concurrency� Higher availability� Low cost of operation

The online functions of UNLOAD allow the unload of data from DB2 tables with SHRLEVEL CHANGE, which enhances the continuous availability of DB2 subsystems.

The UNLOAD utility requires SELECT authority on the table or tables in the table space. This is similar to the DSNTIAUL unload programs, but differs from REORG UNLOAD EXTERNAL, which requires REORG utility authority.

The DISCARD option of REORG, which can be used to discard selected data with the WHEN option during UNLOAD PAUSE and UNLOAD CONTINUE, is not supported in the UNLOAD utility and the REORG UNLOAD EXTERNAL.

8


8.1 Output data sets from UNLOADUNLOAD outputs two data sets which can be used for reload with the LOAD utility:

� The SYSREC data set is associated with the UNLDDN option of UNLOAD. The data set contains the output data from the UNLOAD utility, which is input to a subsequent LOAD utility for loading data into another table space. This data set can be defined in the JCL, and the DDNAME is provided on the UNLDDN option. A TEMPLATE command can also be used to define the data set characteristics and the template name is input in UNLDDN. The data set is dynamically created by the UNLOAD utility with the TEMPLATE option. The TEMPLATE command is required when unloading multiple table spaces, either with multiple UNLOAD statements, or by using the LISTDEF command to supply a list of table spaces to a single UNLOAD utility statement.

� The SYSPUNCH data set is associated to the PUNCHDDN option of UNLOAD. The data set is used by the UNLOAD utility to store the table definition (see Example 8-2 on page 196), which is used by the LOAD utility to load data into another table. When the punch data set is required, its allocation is similar to SYSREC either via DDNAME in the JCL or via TEMPLATE.

AuthorizationTo execute the UNLOAD utility, the user needs one of the following privileges:

� Ownership of the tables� SELECT privilege on the tables� DBADM authority on the databases� SYSADM authority� SYSCNTL authority for catalog tables only

Execution phases of UNLOADTable 8-1 describes the three phases of the UNLOAD utility.

Table 8-1 UNLOAD utility phases

Phase Description

UTILINIT Initialization and setup

UNLOAD Unloading records to sequential data sets. One pass through the input data set is made. If UNLOAD is processing a table or partition, DB2 takes internal commits to provide commit points at which to restart in case of operation should halt in this phase.

UTILTERM Cleanup


8.2 UNLOADing data prior to V7Prior to DB2 V7, data was unloaded from DB2 tables using these methods:

� The DB2 supplied assembler program DSNTIAUL

DSNTIAUL evolved over various versions to finally accept the selection criterion with SQL statements in DB2 V4 onwards. Although this program was useful and widely used by DB2 end users, it was slow and lacked functions like data conversion to other formats. An SQL statement was required for each table that was unloaded which was advantages when selective data was unloaded with the WHERE clause. DSNTIAUL cannot unload data from image copy.

� REORG UNLOAD EXTERNAL

This has been the other popular choice of DBAs to unload data from the source and load it at the destination. REORG UNLOAD EXTERNAL can only unload data from a table space. It cannot unload data from the image copy.

� DSN1COPY of full image copies with XLAT translation

All tables in a table space were copied to the destination table space. The source and destination table DDL must be identical, although the OBID can be converted to the destination OBID via the XLAT translation. RI constraints are not imposed. DSN1COPY of image copy from SHRLEVEL CHANGE is not encouraged, although it can be done with care.

� Independent Software Vendor (ISV) products

These tools are commonly used at many DB2 sites to transfer data mainly from production to development for development or stress testing. The ISV products also provide the capabilities to unload data from the image copy and to load into the destination using either ISV LOAD utility or the DB2 LOAD utility.

� Programs written in-house.

These mainly use imbedded SQL in the host language to SELECT from source tables and INSERT into destination tables.

Enhanced functionality in V7Figure 8-1 summarizes all the functions of the UNLOAD utility. The diagram emphasizes that REORG UNLOAD EXTERNAL utility is a subset of the UNLOAD utility.

In addition to the functions provided by REORG UNLOAD external, the UNLOAD utility provides:

� Unload from image copy data sets created by COPY utility, both full and incremental� Unload from Inline COPY data sets created by REORG and LOAD utility� Unload from data sets created by stand-alone DSN1COPY utility� Encoding scheme, ASCII and UNICODE� SHRLEVEL REFERENCE and SHRLEVEL CHANGE� Sampling, limiting of rows� Partition parallelism

Chapter 8. Unloading data 193

Figure 8-1 Enhanced functions

In this chapter we discuss each function in detail and show examples of JCL and job outputs.

8.3 UNLOAD from image copyUnloading data from image copy is not supported by REORG UNLOAD EXTERNAL utility and the DSNTIAUL program. The UNLOAD utility can be used to unload from data sets that are created by the following utilities:

� COPY� LOAD inline image copy� MERGECOPY� REORG TABLESPACE inline image copy� DSN1COPY

Image copy created by concurrent copy is not supported by UNLOAD.

The syntax for the UNLOAD utility from image copy is shown in Figure 8-2. Here are some minor considerations when using the FROMCOPY option:

� Only a single copy data set name can be specified using the FROMCOPY option. Use FROMCOPYDDN if multiple image copies are concatenated to a DD statement.

� The table space must exist when UNLOAD is run.

� The table space should not have been dropped and recreated since the image copy was taken.

table space

FROM TABLE selectionRow selectionExternal format

numeric date / time

Formatting NOPAD for VARCHARlength, null field

created by:COPYMERGECOPYDSN1COPY

UNLOADREORG UNLOAD EXTERNAL

singledata set

copydata set

data setfor part2

data setfor part1

SHRLEVEL REFERENCE / CHANGESampling, limitation of rowsGeneral conversion options:

encoding scheme, format Field list:

selecting, ordering, positioning, formatting

Partition parallelism

Note: The table space of the image copy MUST exist in the host DB2 subsystem for unloading from copy data sets. Copies of dropped table spaces are not supported.


� If the table was altered with ALTER ADD COLUMN after the image copy was taken, UNLOAD will set system or user defined defaults for the added column when the data is unloaded from such an image copy.

� Image copy created by Inline COPY operation (LOAD or REORG TABLESPACE) can contain duplicate pages. If duplicate pages exist, the UNLOAD utility issues a warning message, and all the qualified rows in the duplicate pages will be unloaded into the output data set.

� If incremental copy is specified as the source, only the rows in the data set are unloaded.

FROMCOPYDDN can be used in place of FROMCOPY when multiple copies need to be specified to UNLOAD. The copies may consist of:

� Full image copy and incremental copies. Consider using the MERGECOPY utility to merge the image copies to a single copy and input to UNLOAD.

� Image copy created with the DSNUM option, which can be a copy of a partition or a piece of a non-partitioned table space. These pieces can be concatenated under a DDNAME to form a single input data set image.

� The order of data sets in DDNAME concatenation is important. UNLOAD might output unpredictable results if the most recent image copy data sets and older image copies are intermixed.

Figure 8-2 UNLOAD — Syntax diagram, main part

Tip: TEMPLATE can be used for the SYSPUNCH and SYSREC data set definitions identified as PUNCHDDN and UNLDDN options in the UNLOAD syntax respectively. LISTDEF cannot be used to pass a list to the LIST option to specify image copy data sets.

ERROR 1 SHRLEVEL CHANGE ISOLATION CSformat-spec ERROR integer ISOLATION CS SHRLEVEL CHANGE ISOLATION UR REFERENCE

source-spec:

unload-spec:

UNLOAD DATA from-table-spec unload-spec from-table-spec source-spec from-table-spec LIST listdef-name

TABLESPACE db-name.ts-name ts-name PART integer int1:int2

FROMCOPY data-set-name FROMVOLUME CATALOG vol-ser FROMCOPYDDN dd-name PUNCHDDN SYSPUNCH UNLDDN SYSREC PUNCHDDN dd-name UNLDDN dd-name template-name template-name

UNLOAD - Syntax diagram


Example 8-1 is an example of an UNLOAD from an image copy of CUSTOMER data. The image copy data set was created by the COPY utility with SHRLEVEL CHANGE. The job produces two data sets associated to SYSPUNCH and SYSREC respectively. The FROMCOPY points to an image copy data set.

Example 8-1 UNLOAD from an image copy data set

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD //SYSIN DD * TEMPLATE ULDDDN DSN(DB2V710G.&DB..&TS..UNLOAD) UNIT(SYSDA) SPACE(45,45) CYL DISP(NEW,CATLG,CATLG) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(DB2V710G.&DB..&TS..PUNCH) UNIT(SYSDA) SPACE(45,45) CYL DISP(NEW,CATLG,CATLG) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE U7G01T11.TSCUST FROMCOPY DB2V710G.U7G01T11.TSCUST.D2001145.T022853L PUNCHDDN PNHDDN UNLDDN ULDDDN

The contents of the data set associated with PUNCHDDN (SYSPUNCH) is displayed in Example 8-2. If the space allocation is not specified in the TEMPLATE statement (refer back to Example 8-1), then DB2 calculates the space requirements using the formulas:

SYSPUNCH = ((#tables * 10) + (#cols * 2)) * 80 bytesSYSREC = ((high used RBA) + (#records * (12 + length of longest clustering key)) bytes

Example 8-2 Contents of SYSPUNCH data set

TEMPLATE U4851714 DSN(DB2V710G.&DB..&TS..UNLOAD1) DISP(OLD,CATLG,CATLG) LOAD DATA INDDN U4851714 LOG NO RESUME YES EBCDIC CCSID(01140,00000,00000) INTO TABLE "PAOLOR1 "."CUSTOMER " WHEN(00001:00002 = X'0011') ( "C_CUSTKEY " POSITION( 00003:00006) INTEGER , "C_NAME " POSITION( 00007:00033) VARCHAR , "C_ADDRESS " POSITION( 00034:00075) VARCHAR , "C_NATIONKEY " POSITION( 00076:00079) INTEGER , "C_PHONE " POSITION( 00080:00094) CHAR(015) , "C_ACCTBAL " POSITION( 00095:00098) FLOAT(21) , "C_MKTSEGMENT " POSITION( 00099:00108) CHAR(010) , "C_COMMENT " POSITION( 00109:00227) VARCHAR )

8.3.1 UNLOAD using FROMCOPY and FROM TABLEThe FROMCOPY option UNLOADs all tables belonging to a table space. In cases where a table space contains more than one table, and the requirement is to UNLOAD only a single table, then the FROM TABLE can be used to UNLOAD only the selected table. Example 8-3 shows the syntax for unloading from image copy for only a single table. The example also highlights the options to unload only three columns out of the eight columns defined on the table (Example 8-2) with the WHEN clause, which can be used to reduce the number of rows to unload.


A further filter on the volume of unloaded data can be achieved with the SAMPLE option. SAMPLE indicates the percentage of data to be unloaded. If the WHEN clause is used to unload selective rows, then SAMPLE is applied only on rows qualified by the WHEN selection condition. .

Example 8-3 UNLOAD using FROMCOPY and FROM TABLE options

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD //SYSIN DD * TEMPLATE ULDDDN DSN(DB2V710G.&DB..&TS..UNLOAD) UNIT(SYSDA) SPACE(45,45) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(DB2V710G.&DB..&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE U7G01T11.TSCUST FROMCOPY DB2V710G.U7G01T11.TSCUST.D2001145.T022853L PUNCHDDN PNHDDN UNLDDN ULDDDN FROM TABLE PAOLOR1.CUSTOMER SAMPLE 50.0 (C_CUSTKEY,C_NAME,C_ADDRESS) WHEN ( C_CUSTKEY > 1000 )

8.3.2 Image copy from compressed table spaceCompressed rows from image copy data set can be unloaded only when the dictionary for decompression has been retrieved. UNLOAD ignores a compressed row with a warning message if the dictionary pages have not been read when the compressed row is encountered. An error counter is incremented, and UNLOAD terminates with an error message if the error count exceeds MAXERR.

If the image copy is an incremental copy, or a copy of a partition or partitions, then the compression dictionary pages must exist in the same data set, otherwise UNLOAD will fail and DB2 will issue an error message.

8.3.3 Advantages of UNLOAD using image copyFigure 8-3 summarizes the advantages of unloading from image copies:

� Unloading from image copy does not interfere with the host table space and table. No locks are taken on table space and index spaces, thus avoiding lock contentions. The only reference is to the DB2 catalog for table definitions.

� The status of the table space does not affect the UNLOAD utility when unloaded from an image copy. The table space may be in STOP status or other restrict status.

� Either all columns of a table or a subset of columns of table can be unloaded using the FROM TABLE option. Data selection can be further qualified by the WHEN option and the the SAMPLE option.

Note: The sampling is applied per individual table. If the rows from multiple tables are unloaded with sampling enabled, the referential integrity between the tables might be lost


Figure 8-3 Summary of Unloading from copy data sets

8.4 Unload data from table spaceIn this section we discuss the UNLOAD utility to unload data directly from a table space with SHRLEVEL CHANGE and SHRLEVEL REFERENCE. The SHRLEVEL option improves concurrency when compared to REORG UNLOAD EXTERNAL. Figure 8-4 summarizes all the functions of UNLOAD from tables space. We explain each function in detail with examples.

Unloading from copy data sets

Advantages:No interference with SQL accessesUnload data when table space is stoppedSelection of rows and columns as for table spaces -rather than processing entire copies with DSN1COPY

Supported copy types:Copies created by COPY, MERGECOPY, DSN1COPYConcatenated copy data setsFull and incremental copiesInline copies from LOAD or REORG

Prerequisite:Table space must exist

Not supported:Separate output data sets per partitionConcurrent copiesCopies of dropped tablesCopy data sets of multiple table spacesUnload of LOB columns


Figure 8-4 Unload from table space

8.4.1 Unloading from a list of table spacesUnloading a list of table spaces and the related tables can be done using the LISTDEF command as in Example 8-4. Here we are unloading all tables in table spaces beginning with TS* in database U7G01T11. The LISTDEF will expand the Wildcard U7G01T11.TS* to the appropriate table space names and group them under the label UNLIST. This list is then passed to UNLOAD utility as UNLOAD LIST list-name. All the SYSPUNCH and SYSREC data sets are dynamically allocated using the TEMPLATE command. At the end of the UNLOAD utility, two data sets of the form DB2V710G.RAMA.TS*.PUNCH and DB2V710G.RAMA.TS*.UNLOAD are created for each table space that is processed by the UNLOAD utility. A sample content of the SYSPUNCH data set is displayed in Example 8-2 on page 196.

When unloading multiple table spaces with a LISTDEF, you must also define a data set TEMPLATE that corresponds to all the table spaces and specify the template-name in the UNLDDN option.

RestrictionsIndex spaces, LOB table spaces and directory objects must not be included in the LISTDEF definition. The FROM TABLE-spec of UNLOAD is not supported with the LIST option.

Unloading from table spacesA list of table spaces

LISTDEF

An entire table space

Specific partitions PART keyword orLISTDEF

Certain tables onlyFROM TABLE option

Certain rows onlyWHEN clauseSAMPLE, LIMIT

Certain columns only

Not supported:FROM TABLE option for listsFollowing source objects:

Auxiliary (LOB) table spaces Table spaces in DSNDB01Index spacesDropped tablesViews Global temporary tables

Granularity

SHRLEVELCHANGE

ISOLATION CS (default)ISOLATION UR

REFERENCE

Note: Unloading from a list of table spaces is fully supported by REORG UNLOAD EXTERNAL


Example 8-4 Unload list of table spaces with LISTDEF

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD, // UTSTATS='' //SYSIN DD * TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) LISTDEF UNLIST INCLUDE TABLESPACE U7G01T11.TS* UNLOAD LIST UNLIST PUNCHDDN PNHDDN UNLDDN ULDDDN

8.4.2 Unload by partition and parallelismData can be unloaded by partition by specifying the PART option in the UNLOAD statement. When using the LISTDEF command, specify PARTLEVEL. An output data set must be allocated for each partition for the UNLOAD to use parallel tasks to unload the data by partition. The number of parallel tasks is limited by the number of CPUs in the LPAR and the number of partitions.

UNLOAD does not activate parallel unloads if only a single output data set is allocated to each table space even though the PART or PARTLEVEL option is coded in the UNLOAD utility statement or LISTDEF command respectively. TEMPLATE can be used to dynamically allocate an output data set per partition by using the &PA key word.

Example 8-5 Sample UNLOAD job for partition table space and parallelism

TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..P&PA..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) LISTDEF UNLIST INCLUDE TABLESPACE U7G01T11.TS* PARTLEVEL UNLOAD LIST UNLIST PUNCHDDN PNHDDN UNLDDN ULDDDN

In Example 8-5 we unloaded a list of table spaces using the LISTDEF Wildcard specification and PARTLEVEL. We also allocated the output data sets using TEMPLATE with &PA in the DSN definitions. It can be seen in the output of Example 8-6 that DB2 has used two parallel tasks to unload the data by partition. UNLOAD has also created three data sets via TEMPLATE definitions as output data sets, one for each partition. If the &PA key word was not allocated to the DSN of TEMPLATE ULDDDN, then DB2 would have allocated only a single output data set, and the unload of data from partitions would be done in sequence.


Example 8-6 Sample UNLOAD output by partition and parallelism

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = UNLOADDSNU050I DSNUGUTC - TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..P&PA..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLYDSNU050I DSNUGUTC - TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLYDSNU050I DSNUGUTC - LISTDEF UNLIST INCLUDE TABLESPACE U7G01T11.TS* PARTLEVELDSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLYDSNU050I DSNUGUTC - UNLOAD LIST UNLIST PUNCHDDN PNHDDN UNLDDN ULDDDNDSNU1039I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE U7G01T11.TSPSUPP1 PARTITION 1DSNU1039I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE U7G01T11.TSPSUPP1 PARTITION 2DSNU1039I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE U7G01T11.TSPSUPP1 PARTITION 3DSNU1201I DSNUUNLD - PARTITIONS WILL BE UNLOADED IN PARALLEL, NUMBER OF TASKS = 2DSNU397I DSNUUNLD - NUMBER OF TASKS CONSTRAINED BY CPUSDSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=PNHDDN DDNAME=SYS00003 DSN=DB2V710G.RAMA.TSPSUPP1.PUNCHDSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=ULDDDN DDNAME=SYS00004 DSN=DB2V710G.RAMA.TSPSUPP1.P00001.UNLOADDSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=ULDDDN DDNAME=SYS00005 DSN=DB2V710G.RAMA.TSPSUPP1.P00002.UNLOADDSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=108000 FOR TABLESPACE U7G01T11.TSPSUPP1PART 1DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=ULDDDN DDNAME=SYS00006 DSN=DB2V710G.RAMA.TSPSUPP1.P00003.UNLOADDSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=108000 FOR TABLESPACE U7G01T11.TSPSUPP1 PART 2 DSNU251I DSNUULQB - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=446421 FOR TABLESPACE U7G01T11.TSPSUPP1 PART 3 DSNU253I DSNUUNLD - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=662421 FOR TABLE PAOLOR4.PARTSUPP1 DSNU252I DSNUUNLD - UNLOAD PHASE STATISTICS - NUMBER OF RECORDS UNLOADED=662421 FOR TABLESPACE U7G01T11.TSPSUPP1 DSNU250I DSNUUNLD - UNLOAD PHASE COMPLETE, ELAPSED TIME=00:00:18

Unload using REORG UNLOAD EXTERNALData can be unloaded using REORG UNLOAD external from partition table spaces. The differences between the UNLOAD and REORG are:

� REORG UNLOAD EXTERNAL does not activate parallelism for a partitioned table space when unloading by PART or PARTLEVEL in LISTDEF command.

� REORG UNLOAD EXTERNAL requires a single SYSPUNCH data set for each partition when unloading by PART or PARTLEVEL in LISTDEF command.


8.4.3 UNLOAD with SHRLEVELOne of the interesting features of UNLOAD utility is the ability to unload data from table spaces with SHRLEVEL CHANGE or REFERENCE. This feature is not available with REORG UNLOAD EXTERNAL. Users require SELECT authority on the tables in the table space.

SHRLEVEL CHANGESHRLEVEL CHANGE allows users to update the tables in the table space while data is being unloaded. When data is fetched from the table space with ISOLATION CS, the UNLOAD utility assumes CURRENTDATA(NO). This ensures that uncommitted data is not unloaded and retains data currency. Data can also be unloaded with ISOLATION UR, where any uncommitted data will be unloaded. No locks are taken on the objects; this allows other DB2 operations to continue on the objects from which the data is being unloaded.

SHRLEVEL REFERENCEThis operation allows users to read the data during the unload. All writers are drained from the table space before commencement of the unload. When data is unloaded from multiple partitions, the drain lock will be obtained for all of the selected partitions in the UTILINIT phase.

8.4.4 Unload from table space using the FROM TABLE optionWhen data is unloaded from a single table space or partitioned table space, the FROM TABLE can be used to selectively unload data for a particular table from the table space. This is particularly useful when a table space contains multiple tables and the user is interested in one or more tables only. In Example 8-7 we unload three of the fourteen tables defined in SYSDBASE table space. Only a single set of SYSPUNCH and SYSREC data sets are created by the UNLOAD. The first two bytes of each record in the output data set identifies the OBID of the table. The LOAD utility uses the WHEN option to identify the output data related to each table. Please refer back to Example 8-2 on page 196 for sample contents of the SYSPUNCH data set and the WHEN option of the LOAD utility.

Using the WHEN optionThe WHEN option can be used in association with FROM TABLE to unload data from the table that satisfies the selection-condition that follows the WHEN option. The selection-condition specifies a condition that is true, false, or unknown about a given row. When the condition is true, the row qualifies for UNLOAD. When the condition is false or unknown, the row does not qualify. The WHEN condition can be used to UNLOAD both FROMCOPY of an image copy and from a table space. For a complete description of the selection condition, please refer to Chapter 29 of DB2 UDB for OS/390 and z/OS V7 Utility Guide and Reference, SC26-9945-00.

In Example 8-7 we also show the usage of the WHEN option with selection criteria. Each WHEN option applies to the FROM TABLE option only. Here we unload rows from SYSTABLEPART, SYSTABLES and SYSTABLESPACE where the WHEN option explicitly qualifies the data selection criteria for each table respectively.

Using the SAMPLE optionThe SAMPLE option may be used to unload a sample percentage of rows from the table specified by FROM TABLE option. When SAMPLE is used in association with WHEN option, the sampling is applied to rows that qualify the selection criteria of the WHEN option. The user may specify explicit an sample value for each table; the default is 100 percent.


Using the LIMIT optionThe LIMIT option can be used to limit the total number of records unloaded from a table. If the number of unloaded rows reaches the specified limit, message DSNU1202 is issued for the table and no more rows are unloaded from the table. The process continues to unload qualified rows from the other tables.

When partition parallelism is activated, the LIMIT option is applied to each partition instead of the entire table.

Example 8-7 Unload selective tables from SYSDBASE using FROM TABLE

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD, // UTSTATS='' //SYSIN DD * TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE DSNDB06.SYSDBASE PUNCHDDN PNHDDN UNLDDN ULDDDN SHRLEVEL CHANGE ISOLATION CS FROM TABLE SYSIBM.SYSTABLEPART SAMPLE 50.0 (PARTITION, TSNAME, DBNAME, IXNAME VARCHAR 10 STRIP BOTH TRUNCATE) WHEN (PARTITION=0) FROM TABLE SYSIBM.SYSTABLES LIMIT 100 WHEN (CREATOR='SYSIBM') FROM TABLE SYSIBM.SYSTABLESPACE WHEN (DBNAME='U7G01T11')

The job outputs two dataset that contain the SYSPUNCH and SYSREC information.DB2V710G.RAMA.SYSDBASE.PUNCH DB2V710G.RAMA.SYSDBASE.UNLOAD

Unload table with field selection listData can be unload from a table by specifying only selective fields. The default is to select all fields in the table. In Example 8-7 we select only 4 fields from SYSTABLEPART and all fields from SYSTABLES and SYSTABLESPACE.

Note: If the rows from multiple tables are unloaded with sampling enabled, the referential integrity between the tables may be lost.

Note: If multiple tables are unloaded from with the LIMIT option, the referential integrity between the tables may be lost.


Apply column functions to output fieldColumn functions such as STRIP, TRUNCATE, packed, ZONED and others can be applied to the output value of data fields. In Example 8-7 on page 203, we applied the STRIP function to remove all leading and trailing blanks from the VARCHAR field. We also specified the output length of the VARCHAR field as 10 bytes, which will ensure that the output data set is a fixed length.

If the length parameter is omitted, the default is the smaller of 255 and the maximum length defined on the source table column.

For a complete list and description of column functions on data types, please refer to Chapter 29 of DB2 UDB for OS/390 and z/OS V7 Utility Guide and Reference, SC26-9945-00.

8.4.5 Converting the character output data to other internal codeWhen a table space or table is created with default character set, the SCCSID value from DSNHDECP is used as the default character representation. Example 8-8 has sample CREATE statements to create database, table space, and table with CCSID EBCDIC as the default character set. Since SCCSID in DSNHDECP (macro DSNHDECM) is 37, the table DEPT will be created with the US English character set.

Tables can also be created with the European English character set of 1140 (Euro support) by changing the CCSID EBCDIC to CCSID 1140.

Example 8-8 Sample database, table space, table, and subset of DSNHDECP

CREATE DATABASE DSN8D71A STOGROUP DSN8G710 BUFFERPOOL BP0 CCSID EBCDIC; CREATE TABLESPACE DSN8S71E IN DSN8D71A USING STOGROUP DSN8G710 PRIQTY 20 SECQTY 20 ERASE NO NUMPARTS 4 (PART 1 USING STOGROUP DSN8G710 PRIQTY 12 SECQTY 12, PART 3 USING STOGROUP DSN8G710 PRIQTY 12 SECQTY 12) LOCKSIZE PAGE LOCKMAX SYSTEM BUFFERPOOL BP0 CLOSE NO COMPRESS YES CCSID EBCDIC; CREATE TABLE DSN8710.DEPT (DEPTNO CHAR(3) NOT NULL, DEPTNAME VARCHAR(36) NOT NULL, MGRNO CHAR(6) , ADMRDEPT CHAR(3) NOT NULL, LOCATION CHAR(16) , PRIMARY KEY(DEPTNO)) IN DSN8D71A.DSN8S71D CCSID EBCDIC; COMMIT;


DSNHDECP sample definitionDSNHDECM CHARSET=ALPHANUM, X ASCCSID=1252, X AMCCSID=65534, X AGCCSID=65534, X SCCSID=37, X MCCSID=65534, X GCCSID=65534, X USCCSID=367, X UMCCSID=1208, X UGCCSID=1200, X ENSCHEME=EBCDIC, X APPENSCH=EBCDIC, X

All character type data can be converted from the host internal code, predominantly from EBCDIC to other data types, such as ASCII and UNICODE on the S/390 DB2 databases. The UNLOAD statement in Example 8-9 converts all the character fields on table REGION into ASCII.

The alternate way to convert to ASCII is to use the CCSID option in the UNLOAD, as follows:

UNLOAD TABLESPACE U7G01T11.TSREGION PUNCHDDN PNHDDN UNLDDN ULDDDN CCSID(01252,00000,00000)

Example 8-9 UNLOAD with character conversion to ASCII

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD, // UTSTATS='' //SYSIN DD * TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE U7G01T11.TSREGION PUNCHDDN PNHDDN UNLDDN ULDDDN ASCII NOPAD

In Example 8-10, table PART in table space TSPART was created with CCSID 1140, Euro English code page. The table space TSPART was unloaded and CCSID was converted to UNICODE. UNLOAD converted all character data from CCSID 1140 to 367.

Example 8-10 UNLOAD with character conversion to UNICODE

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD, // UTSTATS='' //SYSIN DD * TEMPLATE PNHDDN DSN(DB2V710G.RAMA.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE ULDDDN DSN(DB2V710G.RAMA.&TS..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) UNLOAD TABLESPACE U7G01T11.TSPART PUNCHDDN PNHDDN UNLDDN ULDDDN UNICODE


Restrictions� UNLOAD is not supported for table spaces in DSNDB01.

� The FROM TABLE option cannot be used when the LIST option is already specified.

� The following table spaces and image copy source objects are not supported:

– Auxiliary (LOB) table spaces– Index spaces– Dropped tables– Views– Global temporary tables

Performance measurementsWe performed the measurements in an uncontrolled environment, using the LINEITEM table that has 1951548 rows and 33832 active pages. The table has 16 columns with a mixture of INTEGER, CHAR, VARCHAR, DATE and FLOAD data types. We made a full copy of the table space with SHRLEVEL REFERENCE. We obtained the measurements for the three jobs:

� UNLOAD TABLESPACE U7G01T11.TSLINEI.....� UNLOAD TABLESPACE U7G01T11.TSLINEI FROMCOPY image copy� REORG TABLSPACE U7G01T11.TSLINEI UNLOAD EXTERNAL.....

The top half of Table 8-2 has both the elapsed time and CPU times in seconds. The second half of the table contains the percentage differences when compared to the measurements taken for first job.

There is a 36.9% increase in elapsed time when the unload is done from an image copy. Most of this elapse time can be attributed to the I/O from the input sequential data where the I/O buffer size is limited by the number of buffers (DCB BUFNO) allocated to the data set. In this job there were 5659 read I/Os to the data set as compared to 533 prefetches for the other two jobs. The elapsed time of REORG UNLOAD EXTERNAL is similar in value to UNLOAD from table space.

Note: UNLOAD utilizes OS/390 services for CCSID conversion from 1140 to 367. There is no direct translation from 1140 to 367. Please refer to Appendix D, “UNICODE support” on page 293 for generation of the translation table and installation of OS/390 UNICODE support.


Table 8-2 Performance measurements and comparison

The CPU measurements show an increase of 23% when data is unloaded using the REORG utility. There is an insignificant amount of CPU time differences between unload from table space and unload from image copy using the UNLOAD utility. On the other hand, there is a 37% increase in elapsed time for unload from image copy.

UNLOAD from image copies only reads the catalog and does not access the table space. We recommend unload from image copy if elapsed time is not a concern and high availability is more important to your environment.

8.5 Terminating or restarting UNLOADIn this section we discuss terminating or restarting the UNLOAD utility.

Terminating UNLOADThe UNLOAD utility can be terminated with TERM UTILITY command during the UNLOAD phase. The output records in SYSREC are not erased, but the data set remains incomplete until the data set is deleted or the UNLOAD utility is restarted.

Restarting UNLOADWhen restarting a terminated UNLOAD utility, the following occurs:

� Processing starts with the table spaces or partitions that had not yet been completed.

� For a table space or partitions that were being processed at termination, UNLOAD resets the output data sets and processes those table spaces or partitions again.

� When the source is one or more image copy data sets (FROMCOPY or FROMCOPYDDN), UNLOAD always starts processing from the beginning.

UNLOAD TABLESPACE REORG UNLOAD EXTERNAL

Active TS (A) FROMCOPY image copy (B) TABLESPACE db.ts (C)

Elapsed (sec) CPU (sec) Elapsed (sec) CPU (sec) Elapsed (sec) CPU (sec)

37.68 20.42 51.60 20.70 37.74 25.11

The percentage changes (delta %) are calculated with respective to measurement made with UNLOAD Utility from active table space.

Delta %(B-A)*100/A

Delta %(C-A)*100/A

36.9% 1.3% 0.15% 22.96%

Note: Verify that the PTF for APAR PQ50223 is applied to avoid a problem identified by the message:

DSNU1036I DSNURELD - UNABLE TO ESTIMATE SPACE REQUIREMENTS FOR INDDN/UNLDDN

This message is generated when a partitioned table space with PARTLEVEL in LISTDEF is unloaded with the UNLOAD utility and loaded with the LOAD utility.


8.6 Creating a shadow catalogSome DB2 sites create a duplicate catalog database (shadow database) and populate it in order to minimize contention. In Appendix C, “Creating a shadow catalog” on page 289, we list four jobs that:

� Define the shadow database DSHADOW.� Unload data from DSNDB06 using the UNLOAD utility.� Load data using the LOAD utility.� Create statistics for database DSHADOW using the RUNSTATS utility.

The DDL for the shadow database DSHADOW and its objects was generated using the DB2 Administration Tool via the GEN command on database DSNDB06. The output was edited to define the table spaces with STOGROUP SYSDEFLT, and the tables with the LIKE SQL syntax. A copy of the DDL is displayed in Example C-1 on page 289.

� The UNLOAD utility was used to unload data from all table spaces in DSNDB06 except SYSJAUXA and SYSJAUXB, which are related to LOB auxiliary tables.

� The SYSPUNCH data sets were edited to change all SYSIBM to SHADOW, the new table owner in database DSHADOW.

� The LOAD statement was modified to change RESUME YES to RESUME NO REPLACE.

The edited SYSPUNCH data sets may be saved for future loads, since the catalog table definitions do not change within a single DB2 SM level unless by an APAR. Example C-3 on page 291 contains a sample of the SYSVIEWS load statements. Please note that the TEMPLATE command is used to define the input data set to LOAD utility. Thus a consistent naming standard which complies with the TEMPLATE DSN definition for the output data set from the UNLOAD utility will be beneficial.

The source for these jobs is available for download as described in Appendix E., “Additional material” on page 297.


Chapter 9. Recovering data

RECOVER is the primary online utility that can be used to recover DB2 objects, table spaces, index spaces and indexes when a DB2 object is in an unstable or unacceptable state. The restrictive state of the object may be caused by hardware or software failures. The utility recovers data to the current state or to a previous point-in-time by restoring a copy, then applying log records.

In this chapter, we discuss these aspects of the RECOVER utility:

� Using the LISTDEF command for input to LIST keyword� PARALLEL keyword and its restrictions and performance� The various image copy data sets and their input to RECOVER� Fast Log Apply

We also briefly cover the following utilities:

� MODIFY� DB2 Change Accumulation Tool� QUIESCE� CHECK INDEX� CHECK INDEX� CHECK DATA� CHECK LOB� REPORT RECOVERY

9


9.1 RECOVER using the LISTDEF commandLISTDEF is a new feature introduced in V7 that assists in defining the DB2 objects with Wildcards representation. Please refer to Chapter 2, “Wildcarding and Templates” on page 17 for details of LISTDEF and TEMPLATE statements.

The RECOVER utility supports the list generated by the previous LISTDEF command. Example 9-1 shows a sample job with the OPTIONS PREVIEW command that lists the table spaces which will participate in the RECOVER utility.

Example 9-1 RECOVER using LISTDEF command with Wildcard

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=PAOLOR1, // UTSTATS='' //SYSIN DD * OPTIONS PREVIEW LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.* INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL EXCLUDE TABLESPACE U7G01T11.TSLINEI EXCLUDE TABLESPACE U7G01T11.TESTNPIS EXCLUDE TABLESPACE U7G01T11.TSP* RECOVER LIST DB01A PARALLEL 10

Example 9-2 is the associated output from the JCL in Example 9-1. Please note that the LISTDEF expansion does not include the table spaces that are explicitly excluded in the LISTDEF command.

Example 9-2 LISTDEF OPTIONS PREVIEW job output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR1 DSNU050I DSNUGUTC - OPTIONS PREVIEW DSNU1000I DSNUGUTC - PROCESSING CONTROL STATEMENTS IN PREVIEW MODE DSNU1035I DSNUILDR - OPTIONS STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.* INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVELEXCLUDE TABLESPACE U7G01T11.TSLINEI EXCLUDE TABLESPACE U7G01T11.TESTNPIS EXCLUDE TABLESPACE U7G01T11.TSP* DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU1020I -DB2G DSNUILSA - EXPANDING LISTDEF DB01A DSNU1021I -DB2G DSNUILSA - PROCESSING INCLUDE CLAUSE TABLESPACE U7G01T11.* DSNU1022I -DB2G DSNUILSA - CLAUSE IDENTIFIES 11 OBJECTS DSNU1021I -DB2G DSNUILSA - PROCESSING INCLUDE CLAUSE TABLESPACE U7G01T11.TSLINEI DSNU1022I -DB2G DSNUILSA - CLAUSE IDENTIFIES 3 OBJECTS DSNU1021I -DB2G DSNUILSA - PROCESSING EXCLUDE CLAUSE TABLESPACE U7G01T11.TSLINEI DSNU1022I -DB2G DSNUILSA - CLAUSE IDENTIFIES 1 OBJECTS DSNU1021I -DB2G DSNUILSA - PROCESSING EXCLUDE CLAUSE TABLESPACE U7G01T11.TESTNPIS DSNU1022I -DB2G DSNUILSA - CLAUSE IDENTIFIES 1 OBJECTS DSNU1021I -DB2G DSNUILSA - PROCESSING EXCLUDE CLAUSE TABLESPACE U7G01T11.TSP* DSNU1022I -DB2G DSNUILSA - CLAUSE IDENTIFIES 4 OBJECTS DSNU1023I -DB2G DSNUILSA - LISTDEF DB01A CONTAINS 8 OBJECTS DSNU1010I DSNUGPVV - LISTDEF DB01A EXPANDS TO THE FOLLOWING OBJECTS: LISTDEF DB01A -- 00000008 OBJECTS INCLUDE TABLESPACE U7G01T11.TSORDER INCLUDE TABLESPACE U7G01T11.TSCUST INCLUDE TABLESPACE U7G01T11.TSSUPLY INCLUDE TABLESPACE U7G01T11.TSNATION INCLUDE TABLESPACE U7G01T11.TSREGION


INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL(00001) INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL(00002) INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL(00003) DSNU050I DSNUGUTC - RECOVER LIST DB01A PARALLEL 5 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

The utility list manager will invoke RECOVER once for the entire list generated by the LISTDEF. Therefore, LIST cannot be used with these commands:

� PAGE� ERROR RANGE� TOCOPY� DSNUM. Instead use PARTLEVEL as in Example 9-1

9.2 Parallel RECOVERDB2 objects can be recovered in parallel using the PARALLEL keyword. The number of parallel objects to recover is determined by the value following PARALLEL keyword. If the num-objects value is 0 or is not specified, RECOVER will determine the optimum number of objects to process in parallel based on available storage, CTHREAD, and IDBACK values in ZPARM. Assuming the number of optimum processes to be N, Table 9-1 shows the maximum number of parallel tasks for a specified num-object value.

Table 9-1 Total number of processes used for RECOVER in parallel

The job in Example 9-1 was rerun without the OPTIONS PREVIEW command and PARALLEL 10. The output in Example 9-3 indicates that RECOVER processed seven objects in parallel.

Example 9-3 Partial job output with seven objects in parallel

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR1 DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.* INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL EXCLUDE TABLESPACE U7G01T11.TSLINEI EXCLUDE TABLESPACE U7G01T11.TESTNPIS EXCLUDE TABLESPACE U7G01T11.TSP* DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - RECOVER LIST DB01A PARALLEL 10 DSNU397I DSNUCBMT - NUMBER OF TASKS CONSTRAINED BY CONNECTIONS DSNU427I DSNUCBMT - OBJECTS WILL BE PROCESSED IN PARALLEL, NUMBER OF OBJECTS = 7

PARALLEL (num-objects) Number of parallel tasks

0 orNot specified

N

> N N

< N num-objects

Note: If the number of parallel tasks are limited, the following message can be issued:

‘DSNU397I DSNUCBMT - NUMBER OF TASKS CONSTRAINED BY CONNECTIONS‘

Check the values of IDBACK and CTHREAD in ZPARM. Increase the value of IDBACK.

Chapter 9. Recovering data 211

Figure 9-1 shows the schematic diagram of the parallel processes involved in a recovery job with PARALLEL 3. Each object is serviced by two subtasks, one to READ the input data from an external data set, the second to WRITE the data, piped from the READ subtask, to the object space. When recovery of object 3 finishes, the subtask initiates recovery of object 4 while the other subtasks are still processing object 1 and 2 respectively.

Parallelism is only achieved in the RESTORE phase. The Log Apply processing does not begin until all objects have been restored.

Figure 9-1 RECOVER with PARALLEL keyword

9.2.1 Restriction for parallel RECOVERThe following restrictions apply to RECOVER utility with keyword PARALLEL.

� Parallelism support is only provided for copying to, and restoring from, disk devices.

� RECOVER from tape is single-threaded. If a tape volume is encountered in the list, processing for the remainder of the list waits until the object using a tape has been completely restored.

– For example, assume there are 6 objects in a RECOVER list, PARALLEL(4) is requested. The 4th object is a tape data set, and the rest use disk.

– Objects 1, 2, 3, and 4 begin to be processed. Suppose object 2 finishes first.

– Since object 4 is recovered from a tape data set, DB2 cannot begin processing object 5 until the recovery of object 4 is complete.

– Once object 4 finishes, objects 5 and 6 can be processed in parallel.

Restriction: This is only true when all the image copies reside on disk. If RECOVER encounters a tape volume in the list, processing of the remaining objects pauses until the tape object has completed, and then parallel processing resumes.

Recover Parallel (3)

Object 1Object 2Object 3Object 4Object 5

Read Subtask 1

ObjectSpaces

Read Subtask 2

Read Subtask 3

Write Subtask 1

Write Subtask 2

Write Subtask 3

Pipe

ObjectSpaces

ObjectSpaces

PipePipe

CopyDatasets

CopyDatasets Copy

Datasets


� DB2 may reduce the number of objects processed in parallel if there is a virtual storage constraint in the utility batch address space.

� DB2 disables parallelism if the PARALLEL keyword is omitted.

� The keywords, PARALLEL and LOGONLY, are mutually exclusive for the RECOVER utility.

In order to benefit from parallelism, the RECOVER utility must be run with the PARALLEL keyword and an object list must be provided. RECOVER supports two forms of object lists:

� The object-list explicitly defined to RECOVER utility as in Example 9-4.

� LISTDEF command to build the object-list and pass the list to RECOVER using the LIST keyword, see 9.1, “RECOVER using the LISTDEF command” on page 210.

In both cases, RECOVER will dispatch the optimum number of subtasks to recover the objects in parallel.

We recommend using the PARALLEL option with the value 0 or without a value, and let RECOVER determine the optimum number of parallel subtasks.

Example 9-4 RECOVER with object-list

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=PAOLOR1, // UTSTATS='' //SYSIN DD * RECOVER TABLESPACE U7G01T11.TSORDER TABLESPACE U7G01T11.TSCUST TABLESPACE U7G01T11.TSSUPLY TABLESPACE U7G01T11.TSNATION TABLESPACE U7G01T11.TSREGION PARALLEL

9.3 LOGONLYA DB2 object is recovered with LOGONLY option in the following instances:.

� A backup of the object is restored outside DB2.� A Concurrent COPY of the object is restored outside of DB2’s control.� An image copy was never taken and RECOVER will use LOGONLY.

9.3.1 Copy taken outside DB2Recovery with the LOGONLY keyword is only required if the DB2 object (table space or index space) is restored from a backup copy that was not created by DB2 utilities, such as COPY, LOAD, or REORG. Therefore there is no entry in SYSCOPY. One such scenario is when the DB2 objects are backed up using a DFSMShsm volume dump.

The external backup of the object must have been done when one or more of the following conditions is true:

� The DB2 subsystem is inactive.� The object is closed using the STOP DATABASE(db) SPACE(space-name).� The object was QUIESCEd just prior to the offline backup with WRITE YES.


This ensures that the restore of the object is to a point of consistency, DB2 will check that the data set identifiers (DBID, PSID, OBID) match those in the DB2 catalog. RECOVER will fail with message DSNU548I if the identifiers do not match. RECOVER applies only log records to the data set that were written after the point that is recorded in the data set page header. RECOVER will skip any image copies that were made prior to this point of consistency. The object will be recovered to the current point-in-time unless the TORBA or TOLOGPOINT keyword is specified in the RECOVER statement in combination with LOGONLY keyword.

Follow these steps to recover objects from backups created offline:

1. Stop the object using the command:

-STOP DB(db) SPACE(space)

2. Restore the object that needs to recovered. This will overwrite the contents of the object space with the backup data set contents.

3. Start the object for utility only with the command:

-START DB(db) SPACE(space) ACCESS(UT)

4. To recover the object to current point-in-time, run the RECOVER utility with the command:

RECOVER TABLESPACE db.ts LOGONLY

5. To recover to a point-in-time, run the RECOVER utility with the commands:

RECOVER TABLESPACE db.ts LOGONLY TORBA X’byte-string’ orRECOVER TABLESPACE db.ts LOGONLY TOLOGPOINT X’byte-string’

6. If recovering a table space to point-in-time, then the related indexes need to be rebuilt with the REBUILD utility:

REBUILD INDEX(ALL) TABLESPACE db.ts

7. Allow access to the recovered object with command:

-START DB(db) SPACE(space) ACCESS(RW)

8. Image copy of the object with COPY utility is recommended.

Note: The TORBA point can be obtained from:

� SYSCOPY � BSDS when archive log with mode quiesce� REPORT RECOVERY report� ssidMSTR address space DSNJ099I message� QUIESCE job output� Copy SHRLEVEL REFERENCE

Important: Please be aware of the instance node of the VSAM data set during the restore (refer to 9.3.2, “DB2 object names with REORG and FASTSWITCH” on page 215). If the DB2 object is reorganized with FASTSWITCH, the node may have changed, either I or J as indicated by the IPREFIX column (see Table 9-2). If the object is backed up with DFSMShsm, then the data set will be restored with the same name as the VSAM cluster at the time of the backup. The DFSMShsm restored data set needs to be manually renamed to the correct VSAM cluster name as indicated by the IPREFIX value.


9.3.2 DB2 object names with REORG and FASTSWITCHPrior to V7, all physical data sets had the naming convention hlq.DSNDBC.db.ts.I0001.Annn, where the I0001 indicates the node name. DB2 V7 introduced new columns in the catalog to record the node name of the physical VSAM data set for table spaces and index spaces, as listed in Table 9-2.

Table 9-2 DB2 V7 catalog table entries

If the DB2 object is reorganized using the REORG SHRLEVEL CHANGE or SHRLEVEL REFERENCE with FASTSWITCH option, the catalog records the node name of the DB2 object in SYSTABLEPART or SYSINDEXPART. When the DB2 object is image copied using the COPY utility and CONCURRENT keyword, the STYPE field in SYSCOPY is recorded with the value C (I0001) or J (J0001).

9.3.3 Recovering a DB2 object from Concurrent COPYThe COPY utility with CONCURRENT keyword invokes DFSMShsm to create image copies and update SYSCOPY with ICTYPE=F and STYPE=C or J. Please refer to Chapter 10, “Copying data” on page 231 for details of the COPY utility. The concurrent image copy must have been made with the table space in a consistent state. Concurrent COPY of DB2 object with an 8 KB, 16 KB, or 32 KB page size must be done with SHRLEVEL REFERENCE. COPY utility initially records ICTYPE=Q when concurrent image copy is taken with SHRLEVEL REFERENCE. When the concurrent image copy completes successfully, RECOVER changes ICTYPE=Q to ICTYPE=F.

The DB2 object can be recovered from Concurrent COPY in a similar manner to any recovery with ICTYPE=F. The object can be recovered TOCOPY, TOLOGRBA, or current point-in-time. DB2 will apply the logs after the object is restored from the copy and roll forward to the current point-in-time.

RECOVER handles the VSAM data set node names automatically (see Table 9-2 on page 215) if the object is reorganized with FASTSWITCH after the Concurrent COPY. During the recovery, the TOCOPY utility redefines the VSAM cluster to the correct IPREFIX node name.

9.3.4 Recovering without an image copyWhen a recovery of an object is attempted where there is no record of an image copy in SYSCOPY then one of the following will occur

� You will receive an error message and the recovery will fail:

DSNU510I - NO GOOD FULL IMAGE COPY DATA SET FOR RECOVERY OF TABLESPACE.....

DB2 table Column name Valid Values

SYSINDEXPART IPREFIX I or J

SYSTABLEPART IPREFIX I or J

SYSCOPY STYPE C or J for Concurrent COPY only

Note: Restrictions on using DFSMS Concurrent COPY

The RECOVER keywords PAGE or ERRORRANGE can not be used with DFSMS Concurrent COPY. If these keywords are specified, RECOVER will bypass any Concurrent COPY records when searching the SYSCOPY table for recoverable point.


� You will receive an error message and the recovery will fail:

DSNU528I - NO FULL IMAGE COPY WAS AVAILABLE AND THERE ARE NO UPDATES TO APPLY FROM THE DB2 LOG FOR TABLESPACE.....

� The object will be recovered from the HPGRBRBA if any updates are recorded in the log since the HPGRBRBA (see 9.3.6, “LOGONLY recovery improvements in V7” on page 216).

� If there is a quiesce point in SYSCOPY with ICTYPE=Q, then the object will be recovered from the quiesce point by applying logs. Obviously, if the start RBA is beyond the last archive log in BSDS, then the recovery will fail with an unavailable archive log.

9.3.5 Recovering a single piece of a multi-linear page setA single piece of an object (A0001, A0002 etc....) from a multi-piece linear page set can be recovered with the LOGONLY option. RECOVER opens the first piece of the page set to obtain the value of HPGRBRBA. If the data set is migrated by DFSMShsm, then the data set is recalled by DFSMShsm. Without LOGONLY, no data set recall is requested.

9.3.6 LOGONLY recovery improvements in V7Prior to DB2 V7, the field HPGRBRBA, the RECOVER base RBA field on the pageset header page, representing the starting log RBA value for a log-only recovery, was updated when:

� A pseudo close or close happened for an object.� A stop command was issued for the object.� A QUIESCE, LOAD utility was run against an object.

In DB2 V7, the HPGRBRBA value is updated every nth checkpoint, where n is controlled by the value of the DSNZPARM parameter DLDFREQ (down-level detection.)

9.4 Log Apply considerationsFor best performance of the Log Apply phase, the following items should be considered:

� Try to RECOVER a list of objects in one utility statement to take a single pass of the log.

� Keeping archive logs on disk provides the best possible performance.

� Consider the size and number of active log data sets. DB2 compares the recovery RBA to the STARTRBA and ENDRBA of the active logs. If the recovery RBA is held within the current active logs, recovery will be faster. This is due to active logs being always on disk. We recommend that you maintain at least half to a full day of log records in active logs.

� If the start log RBA is beyond the active logs, then DB2 will apply the log records from archive logs. Thus controlling archive log data sets by DFSMShsm is beneficial. DB2 optimizes recall of the data sets, after being recalled, the data sets are read from disk.

� If the archive log must be read from tape, DB2 optimizes access by means of ready-to-process and look-ahead-mount requests.

Note: Backing up a single piece of a multi-linear page set is not recommended. It can cause a data integrity problem if the backup is used to restore only a single data set at a later time.


9.5 Fast Log ApplyFast Log Apply was introduced in DB2 V6 to improve the restart of DB2 subsystems after a failure and the recovery of DB2 objects. It reduced both the elapsed time and the CPU time by utilizing an intermediate buffer storage area which reduces the I/O to the table spaces.

The Fast Log Apply (FLA) design consists of one log read task per recovery job and one or more Log Apply tasks employing the use of multi-tasking whenever possible. It also takes advantage of a list prefetch technique to nearly eliminate duplicate synchronous read operations for the same page.

It achieves these efficiencies by sorting groups of log records together that pertain to the same page set before applying those changes; it then applies those records in parallel using several Log Apply tasks.

DB2 uses FLA during the following processes:

– Forward phase of DB2 restart

– RECOVER utility

– START DATABASE for:• LPL recovery• GRECP recovery

Fast Log Apply is activated during the installation process by setting any non-zero value to Log Apply Storage in the DSNTIPL panel; see Figure 9-2. This sets a value to the ZPARM variable LOGAPSTG in the DSN6SYSP macro.

Figure 9-2 DB2 installation panel DSNTIPL


9.5.1 Fast Log Apply buffersDB2 obtains FLA buffers on behalf of recovery or restart tasks or during START DATABASE processing. The initial buffer size allocated depends on the particular process as summarized in Table 9-3.

Table 9-3 Fast apply buffer sizes

If the storage size specified is not available when the FLA feature attempts to allocate it, DB2 tries to allocate a smaller amount. It is possible that insufficient space is available for FLA to continue; in this case, DB2 reverts to normal log recovery. A minimum of 64 KB is necessary for the FLA feature.

FLA is a non-optional feature for DB2 restart even if you specify zero for the Fast Log Apply storage. A minimum buffer size of 10 MB is allocated during the Forward Log Recovery phase and is freed when this phase completes. The rationale behind this decision is that restart is the only process running and there should be no problem obtaining 10 MB of storage for the FLA feature at this time. Of course, if you choose a Log Apply storage value larger than 10 MB, DB2 uses that value during restart.

Each RECOVER utility job uses a size of 10 MB, if available, for the Fast Log Apply buffer. You need to consider the number of concurrent RECOVER utility jobs when determining the size of the FLA buffer. If you specify 30 MB for the total size of the Log Apply buffer, then only 3 concurrent jobs can use 10 MB each.

DB2 acquires FLA buffer storage at the start of the Log Apply phase and releases it at the end of the Log Apply phase of the RECOVER utility. If there are four concurrent RECOVER utility jobs running, the first three jobs get 10 MB each. If the fourth job arrives at the Log Apply phase while the other three jobs are also in the Log Apply phase, then the fourth job is not be able to use the FLA feature. However, recovery of the fourth job is not be delayed; its recovery simply occurs without FLA.

If the fourth job arrives, at the Log Apply phase, at the same time that one of these three jobs completes the Log Apply phase, then the fourth job may be able to get the 10 MB storage, and can then also take advantage of FLA.

9.5.2 Sort the log recordsThe log records are sorted in DBID, OBID, PART, PAGE, and LRSN or RBA sequence. The actual sorting is accomplished by a separate task for each RECOVER and START DATABASE command as well as during restart.

Each sorted FLA buffer is divided into several smaller groups organized by the object level (DBID.PSID.PART).

One task is dispatched to apply those log records associated with each group. Each task opens the data set, if necessary, builds and initiates the list prefetch. If the process requires the use of multiple tasks, they are run in parallel.

Process Allocated Buffer Size

RECOVER UtilitySTART DATABASE

10MB

DB2 Restart 10MB - 100MB


A maximum of 98 parallel tasks can be dispatched, one for each page set. If log records for more than 98 objects are found, then 98 tasks are scheduled. As a task completes, a new task is dispatched to apply the log records to the next object.

In addition, FLA takes advantage of the ability to perform up to 20 data set opens in parallel during restart.

9.5.3 When is Fast Log Apply bypassed?DB2 bypasses FLA automatically in the following situations:

� Single page recovery� Online page recovery� Minimum storage of 64 KB not available� Log apply storage value set to zero (for RECOVER utility only)

9.6 Index recovery from COPYIndexes of a table can be image copied using the COPY utility by altering the index space and setting COPY YES; for example:

ALTER INDEX creator.index-name COPY YES

This sets the COPY field in SYSIBM.SYSINDEXES to Y for YES. Another catalog field COPYLRSN in the same table records the RBA or the LRSN (in data sharing) of the index when it was altered to COPY YES or created with COPY YES. Please refer to section 10.1.2, “Copying indexes” on page 232.

One of the main advantages of the image copy index is the ability to recover the index independently from the table space or in parallel to the recovery of the table space. The latter saves an extra step to rebuild the index after the table space is recovered. In Example 9-5 we take an image copy of the table space U7G01T11.TSLINEI and all the indexes of this table space. The table space has three partitioned indexes and two NPIs. COPY utility dispatched six parallel tasks to copy the objects. The maximum number of tasks is constraint by virtual storage and by available threads specified by CTHREAD and IDBACK in ZPARM.

Example 9-5 COPY table space and all indexes

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=PAOLOR1, // UTSTATS='' //SYSIN DD * TEMPLATE LOCALDDN DSN(PAOLOR1.&DB..&TS..P&PA..T&TIME.L) UNIT(SYSDA) CYL DISP(MOD,CATLG,DELETE) VOLUMES(SBOX60) LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI PARTLEVEL COPY LIST DB01A FULL YES PARALLEL 10 SHRLEVEL CHANGE COPYDDN(LOCALDDN)


The recovery of the table space is done as in Example 9-6. We are attempting to recover the table space TSLINEI and all the indexes associated to the tables in the table space. Output in Example 9-7 shows that seven parallel tasks were dispatched to recover the table space, three partition indexes and two NPIs. All objects will be recovered to a point-in-time by applying the logs after the completion of the recovery from image copies.

An index can also be recovered individually either from image copy or by the REBUILD utility. The decision to image copy and recover an index will depend on the size of the index space. Refer to 10.1.2, “Copying indexes” on page 232.

Example 9-6 RECOVER of the table space and indexes in parallel

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=PAOLOR1, // UTSTATS='' //SYSIN DD * LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI PARTLEVEL RECOVER LIST DB01A PARALLEL 10

Example 9-7 RECOVER table space and indexes output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR1 DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL INCLUDE INDEXSPACES TABLESPACEU7G01T11.TSLINEI PARTLEVEL DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - RECOVER LIST DB01A PARALLEL 10 DSNU397I DSNUCBMT - NUMBER OF TASKS CONSTRAINED BY CONNECTIONS DSNU427I DSNUCBMT - OBJECTS WILL BE PROCESSED IN PARALLEL, NUMBER OF OBJECTS = 7 DSNU532I DSNUCBMT - RECOVER TABLESPACE U7G01T11.TSLINEI DSNUM 1 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.TSLINEI.P00001.T213329L WITH DATE=20010625 AND TIME=173408 IS PARTICIPATING IN RECOVERY OF TABLESPACE U7G01T11.TSLINEI DSNUM 1 DSNU532I DSNUCBMT - RECOVER TABLESPACE U7G01T11.TSLINEI DSNUM 2 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.TSLINEI.P00002.T213329L WITH DATE=20010625 AND TIME=173409 IS PARTICIPATING IN RECOVERY OF TABLESPACE U7G01T11.TSLINEI DSNUM 2 DSNU532I DSNUCBMT - RECOVER TABLESPACE U7G01T11.TSLINEI DSNUM 3 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.TSLINEI.P00003.T213329L WITH DATE=20010625 AND TIME=173507 IS PARTICIPATING IN RECOVERY OF TABLESPACE U7G01T11.TSLINEI DSNUM 3 DSNU532I DSNUCBMT - RECOVER INDEXSPACE U7G01T11.TESTNPI2 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.TESTNPI2.P00000.T213329L WITH DATE=20010625 AND TIME=173404 IS PARTICIPATING IN RECOVERY OF INDEXSPACE U7G01T11.TESTNPI2 DSNU532I DSNUCBMT - RECOVER INDEXSPACE U7G01T11.TESTNPI START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.TESTNPI.P00000.T213329L WITH DATE=20010625 AND TIME=173408 IS PARTICIPATING IN RECOVERY OF INDEXSPACE U7G01T11.TESTNPI

Note: Indexes with COPY NO will not have any image copies. These indexes will not participate in the recovery. Their status will be set to RBDP. These indexes must be rebuilt using the REBUILD utility. Use OPTIONS EVENT(ITEMERROR,SKIP) to bypass indexes with COPY NO when passing list from LISTDEF.


DSNU532I DSNUCBMT - RECOVER INDEXSPACE U7G01T11.PXL#OKSD DSNUM 1 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.PXL#OKSD.P00001.T213329L WITH DATE=20010625 AND TIME=173347 IS PARTICIPATING IN RECOVERY OF INDEXSPACE U7G01T11.PXL#OKSD DSNUM 1 DSNU532I DSNUCBMT - RECOVER INDEXSPACE U7G01T11.PXL#OKSD DSNUM 2 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.PXL#OKSD.P00002.T213329L WITH DATE=20010625 AND TIME=173400 IS PARTICIPATING IN RECOVERY OF INDEXSPACE U7G01T11.PXL#OKSD DSNUM 2 DSNU504I DSNUCBRT - MERGE STATISTICS FOR INDEXSPACE U7G01T11.PXL#OKSD DSNUM 1 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=2318 ELAPSED TIME=00:00:08 DSNU532I DSNUCBMT - RECOVER INDEXSPACE U7G01T11.PXL#OKSD DSNUM 3 START DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR1.U7G01T11.PXL#OKSD.P00003.T213329L WITH DATE=20010625 AND TIME=173425 IS PARTICIPATING IN RECOVERY OF INDEXSPACE U7G01T11.PXL#OKSD DSNUM 3 DSNU504I DSNUCBRT - MERGE STATISTICS FOR INDEXSPACE U7G01T11.PXL#OKSD DSNUM 2 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=2308 ELAPSED TIME=00:00:07 DSNU504I DSNUCBRT - MERGE STATISTICS FOR TABLESPACE U7G01T11.TSLINEI DSNUM 1 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=13595 ELAPSED TIME=00:00:24 DSNU504I DSNUCBRT - MERGE STATISTICS FOR INDEXSPACE U7G01T11.TESTNPI2 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=9751 ELAPSED TIME=00:00:27 DSNU504I DSNUCBRT - MERGE STATISTICS FOR INDEXSPACE U7G01T11.TESTNPI - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=10128 ELAPSED TIME=00:00:28 DSNU504I DSNUCBRT - MERGE STATISTICS FOR TABLESPACE U7G01T11.TSLINEI DSNUM 2 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=13831 ELAPSED TIME=00:00:35 DSNU504I DSNUCBRT - MERGE STATISTICS FOR INDEXSPACE U7G01T11.PXL#OKSD DSNUM 3 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=17912 ELAPSED TIME=00:00:29 DSNU504I DSNUCBRT - MERGE STATISTICS FOR TABLESPACE U7G01T11.TSLINEI DSNUM 3 - NUMBER OF COPIES=1 NUMBER OF PAGES MERGED=107667 ELAPSED TIME=00:01:25 DSNU500I DSNUCBDR - RECOVERY COMPLETE, ELAPSED TIME=00:01:28 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

9.7 Modify enhancementThe MODIFY RECOVERY utility provides support for cleaning up entries in the DB2 catalog (SYSCOPY) that are no longer useful for current data. DB2 V7 has introduced some maintenance that has massively improved the way the MODIFY RECOVERY utility performs when dealing with complex situations. The same enhancement has been made available to DB2 V5 and V6.

Note: Apply the PTF for APAR PQ45184 to benefit from this enhancement.


Some measurements were performed under DB2 V7 with and without this enhancement, for a customer SYSCOPY table with 975,341 rows, with 7,476 entry records for the specific tablespace. The results are listed in Table 9-4.

Table 9-4 Modify recovery enhancement

The improvements are over 90% in both CPU and elapsed time.

9.8 REPORT RECOVERYREPORT RECOVERY is an online utility that can be used to find information necessary for recovering a table space, index, or table space and all its indexes. The utility also provides the LOB table spaces associated with a base table space.

The output from REPORT RECOVERY consist of:

� Recovery history from the SYSIBM.SYSCOPY catalog table, including QUIESCE, COPY, LOAD, REORG, RECOVER TOCOPY and RECOVER TORBA history

� Log ranges from the SYSIBM.SYSLGRNX directory table

� Archive log data sets from BSDS

� Any indexes on the table space that are in informational COPY-pending status

In a data sharing environment, the output provides:

� The RBA when migrated to Version 7

� The high and low RBA values of the migrated member

� A list of any SYSLGRNX records from before data sharing was enabled that cannot be used to recover to any point-in-time after data sharing was enabled

� For SYSCOPY, the member that the image copy was deleted from

TABLESPACESETThe TABLESPACESET option can be used to find names of all table spaces and tables in a referential structure, including LOB table spaces.

Sample REPORT RECOVERY jobExample 9-8 is a sample JCL to report on table space DSN8S71D. The CURRENT option is specified to report only the SYSCOPY entries after the last recoverable point of the table space.

Example 9-8 REPORT RECOVERY sample job

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=REPORT //SYSIN DD * LISTDEF DB01A INCLUDE TABLESPACE DSN8D71A.DSN8S71D REPORT RECOVERY TABLESPACE LIST DB01A CURRENT ARCHLOG 1

MODIFY RECOVERY DELETE AGE(*) w/o fix for PQ45184 with fix for PQ45184

CPU time (sec) 119 1.7

Elapsed time (sec) 123 5.1


Example 9-9 is the associated output of the sample REPORT RECOVERY job. The output shows that a full image copy exist for the table space DSN8S71D at START LRSN 00010DEA81F9. The associated entry in SYSLGRNX shows that the table space DSN8S71D was open at UCDATE and UCTIME and at START LRSN 00010DF08B11. The 000000000000 value for STOP LRSN indicates that the table space is still open.

The SYSCOPY and SYSLGRNX information can be used to determine the recovery details of the table space. It can also be used to decide if an image copy is required. If the is an entry START LRSN in SYSLGRNX and its value is higher than the START LRSN of the last image copy entry in SYSCOPY then the table space is open for update. This table space is a candidate for image copy using the COPY utility.

One advantage of using REPORT RECOVERY to determine image copy status is that the base table space is not accessed. All the required information is obtained form SYSCOPY and SYSLGRNX. Therefore, if the base table space is migrated by DFSMShsm, it will not be recalled.

Example 9-9 REPORT RECOVERY sample output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CHECKIX DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE DSN8D71A.DSN8S71D DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - REPORT RECOVERY TABLESPACE LIST DB01A CURRENT ARCHLOG 1 DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DSN8D71A.DSN8S71D DSNU581I -DB2G DSNUPREC - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D DSNU585I -DB2G DSNUPREC - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D CURRENT DSNU593I -DB2G DSNUPREC - REPORT RECOVERY ENVIRONMENT RECORD: ' MINIMUM RBA: 000000000000 ' MAXIMUM RBA: FFFFFFFFFFFF ' MIGRATING RBA: 000000000000 DSNU582I -DB2G DSNUPPCP - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D SYSCOPY ROWS TIMESTAMP = 2001-06-28-20.04.50.300816, IC TYPE = F , SHR LVL = C, DSNUM = 0000, START LRSN =00010DEA81F9 DEV TYPE = 3390 , IC BACK = , STYPE = , FILE SEQ = 0000, PIT LRSN = 000000000000 LOW DSNUM = 0000, HIGH DSNUM = 0000 JOBNAME = PAOLOR1C, AUTHID = PAOLOR1 , COPYPAGESF = 3.0E+00 NPAGESF = 1.2E+01 , CPAGESF = 2.0E+00 DSNAME = PAOLOR1.DSN8D71A.DSN8S71D.P00000.T000446L , MEMBER NAME = DSNU586I -DB2G DSNUPSUM - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D SUMMARY DSNU588I -DB2G DSNUPSUM - NO DATA TO BE REPORTED DSNU583I -DB2G DSNUPPLR - SYSLGRNX ROWS FROM REPORT RECOVERY FOR TABLESPACE DSN8D71A.DSN8S71D UCDATE UCTIME START RBA STOP RBA START LRSN STOP LRSN PARTITION MEMBER ID 062801 20091248 00010DF08B11 000000000000 00010DF08B11 000000000000 0000 0000 DSNU584I -DB2G DSNUPPBS - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D ARCHLOG1 BSDS VOLUMES DSNU588I -DB2G DSNUPPBS - NO DATA TO BE REPORTED DSNU586I -DB2G DSNUPSUM - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D SUMMARY DSNU588I -DB2G DSNUPSUM - NO DATA TO BE REPORTED DSNU589I -DB2G DSNUPREC - REPORT RECOVERY TABLESPACE DSN8D71A.DSN8S71D COMPLETE DSNU580I DSNUPORT - REPORT UTILITY COMPLETE - ELAPSED TIME=00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0


Catalog and directory table spacesWhen running REPORT RECOVERY for:

� SYSIBM.SYSCOPY� SYSIBM.SYSLGRNX� SYSIBM.DBD01

You need to specify the CURRENT option to avoid unnecessary access of archive logs. If archive logs are on tape, then there will be unnecessary mounting of archive tapes. If the CURRENT option is specified and the last recoverable point does not exist on the active log, DB2 will access archive logs until the point is found.

9.9 DB2 Change Accumulation ToolThe DB2 Change Accumulation Tool creates full image copies of objects without the need to start or stop the database. By using this low-overhead tool, you can minimize database downtime and reduce recovery time.

The space you want to make a copy of must have a complete, valid full image copy registered in the SYSCOPY catalog table. The DB2 Change Accumulation Tool takes the full image copy, applies any incremental image copies, and then applies log data up to the specified recovery point. The resulting file is logged in SYSIBM.SYSCOPY as a primary local or backup full SHRLEVEL REFRENCE image copy that can be used to recover the object.

This process is similar to MERGECOPY, except that the DB2 Change Accumulation Tool allows you to select specific points up to which you want the image copy created. For instance, you can select the TOCURRENT recovery point, or a specific RBA from the log.

The DB2 Change Accumulation Tool also allows you to customize processing by specifying parameters via control controls in the JCL; see Example 9-10. For example, you can specify the recovery point or choose from two processing methods.

Example 9-10 Sample JCL to create new full image copy

//STEP1 EXEC PGM=GGC#MAIN,PARM='R61A' //STEPLIB DD DSN=DB2CHG.LOADLIB,DISP=SHR // DD DSN=DSN710.SDSNLOAD,DISP=SHR //TASKLIB DD DSN=DB2CHG.LOADLIB,DISP=SHR //SYSUDUMP DD SYSOUT=* //SYSOUT DD SYSOUT=* //SORTMSGS DD SYSOUT=* //DB2PARMS DD DSN=DB2CHG.DB2.CONTROL,DISP=SHR //FULLICPY DD DSN=NEW.FULLICPY, // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DISP=(NEW,CATLG,DELETE) //SYSINGGC DD * CHANGE_ACCUM( - DATA_BASE XXXXXXXX - "XXXXXXXX" = 8 CHAR DATA BASE NA SPACE_NAME YYYYYYYY - "YYYYYYYY" = 8 CHAR TABLE SPACE PARTITION 0 - USE ONLY ON PARTITIONED SPACES TO_CURRENT - TWO_PASS - NO_SYSCOPY_ROW - LOCAL_SITE) /*


The DB2 Change Accumulation Tool provides DB2 administrators with powerful tool for backing up data base objects in the most precise and lease destructive manner by:

� Making precise ‘point-in-time’ recovery of data base objects simple and reliable

� Allowing recovery routines to focus on single objects and previous states

� Producing SHRLEVEL REFERENCE image copies without the associated overhead and data locking

� Controlling the scope and specificity of image copy creation precisely through control cards

� Maintaining data integrity without recovery to RBA

� Reducing recovery session times significantly in many cases

� Providing low overhead and minimizing downtimes for high-volume, complex data bases with large number of tables and dependencies

For more details on the DB2 Change Accumulation Tool, please refer to DB2 Change Accumulation Tool for z/OS V1R1 User's Guide, SC27-1191-00, or see the Web site:

http://www.ibm.com/software/data/db2imstools/details/#rr

9.10 QUIESCEThe QUIESCE is an online utility that establishes consistent recovery point (the current log RBA or LRSN) for a table space, partition, table space set and index with COPY YES set. QUESCE supports a list of table spaces as in Example 9-11 or building of a list using the LISTDEF command as in Example 9-12.

Example 9-11 QUIESCE with a list of table spaces

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=QUIESCE //SYSIN DD * QUIESCE TABLESPACE DSNRLST.DSNRLS01 TABLESPACE DSNDB04.MYTABLE TABLESPACE DSNDB04.EXAMP2 TABLESPACE DSN8D71P.DSN8S71Q TABLESPACE DSNDB04.SYSTABLE WRITE YES

Example 9-12 QUIESCE using the LISTDEF command

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=QUIESCE //SYSIN DD * LISTDEF DB01A INCLUDE TABLESPACE DSN*.* EXCLUDE TABLESPACE DSN8D71A.DSN8S71D EXCLUDE TABLESPACE DSN8D71A.DSN8S71E EXCLUDE TABLESPACE DSNDB04.EXAMP1 QUIESCE LIST DB01A WRITE YES

A new TABLESPACESET option for QUIESCE was introduced in V6. This option specifies that all of the referentially related table spaces in the table space set are to be quiesced. A table space set is either:


� A group of table spaces that have a referential relationship� A base table space with all of its LOB table spaces� The associated indexes are quiesced only if WRITE YES is specified.

SYSCOPY is updated with ICTYPE=Q for each table space quiesced. Other enhancements and restrictions for the QUIESCE utility are:

� Remove the restriction of 1,165 maximum table spaces in the list.

� Duplicate table spaces or table space sets are quiesced only once.

� For a table space being quiesced with WRITE YES, a SYSCOPY record with ICTYPE=Q is also inserted for each of its indexes defined with COPY YES.

� PART and TABLESPACE are mutually exclusive keywords.

� An index or index space cannot be specified on a QUIESCE statement

The following Example 9-13 shows the quiesce of partition indexes of a partition table space TSLINEI when only the table space is specified in the QUIESCE list.

Example 9-13 QUIESCE of table space and all indexes

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CPITEM DSNU050I DSNUGUTC - LISTDEF DB02A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - QUIESCE LIST DB02A DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI PARTITION 1 DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.PXL#OKSD PARTITION 1 DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI2 DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI PARTITION 2 DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.PXL#OKSD PARTITION 2 DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI2 DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI PARTITION 3 DSNU481I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.PXL#OKSD PARTITION 3 DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI DSNU477I -DB2G DSNUQUIA - QUIESCE SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI2 DSNU474I -DB2G DSNUQUIA - QUIESCE AT RBA 00010DBEDA76 AND AT LRSN 00010DBEDA76 DSNU475I DSNUQUIB - QUIESCE UTILITY COMPLETE, ELAPSED TIME= 00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

9.11 CHECKThere are three online utilities, CHECK DATA, CHECK INDEX, CHECK LOB which are used to check the consistency of table spaces, index spaces and LOB table spaces respectively.

9.11.1 CHECK INDEXCHECK INDEX tests whether indexes are consistent with the underlying data, and issue warning messages when inconsistency is found. Check index should be executed

� After a condition restart of DB2 subsystem

Note: TABLESPACESET is not supported by LISTDEF. The LIST option cannot be used in conjunction with TABLESPACE or TABLESPACESET.


� Point-in-time recovery on all table spaces whose indexes may not be consistent with the data.

� Before CHECK DATA on table space to ensure indexes used by CHECK DATA are valid.

� Before CHECK DATA with DELETE YES

� To verify the auxiliary table index of LOB table space

Example 9-14 is a sample job to check the indexes of table space TSLINEI in data base U7G01T11. We used the TEMPLATE for work data set and LISTDEF to generate the list of objects for CHECK INDEX. Example 9-15 shows the output of the CHECK INDEX job. The LISTDEF command created a list of five indexes, three partition indexes and two NPI.

Example 9-14 Sample CHECK INDEX JCL

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=CHECKIX //SYSIN DD * TEMPLATE WORKDDN DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES(SBOX60,SBOX59) LISTDEF DB01A INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI PARTLEVEL CHECK INDEX LIST DB01A WORKDDN WORKDDN

Example 9-15 CHECK INDEX job output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CHECKIX DSNU050I DSNUGUTC - TEMPLATE WORKDDN DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES(SBOX60,SBOX59) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI PARTLEVEL DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - CHECK INDEX LIST DB01A WORKDDN WORKDDN DSNU1033I DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE U7G01T11.TESTNPI2 DSNU1033I DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE U7G01T11.TESTNPI DSNU1039I DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE U7G01T11.PXL#OKSD PARTITION 1 DSNU1039I DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE U7G01T11.PXL#OKSD PARTITION 2 DSNU1039I DSNUGULM - PROCESSING LIST ITEM: INDEXSPACE U7G01T11.PXL#OKSD PARTITION 3 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=WORKDDN DDNAME=SYS00001 DSN=DB2V710G.RAMA.CHECK.INDEX DSNU701I -DB2G DSNUKGET - 7806192 INDEX ENTRIES UNLOADED FROM 'PAOLOR4.TESTNPI2' DSNU701I -DB2G DSNUKGET - 7806192 INDEX ENTRIES UNLOADED FROM 'PAOLOR4.TESTNPI' DSNU700I -DB2G DSNUKGET - 801456 INDEX ENTRIES UNLOADED FROM INDEX='PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 1 DSNU700I -DB2G DSNUKGET - 798204 INDEX ENTRIES UNLOADED FROM INDEX='PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 2 DSNU700I -DB2G DSNUKGET - 6206532 INDEX ENTRIES UNLOADED FROM INDEX='PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 3 DSNU705I DSNUK001 - UNLOAD PHASE COMPLETE - ELAPSED TIME=00:01:21 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=23418576 ELAPSED TIME=00:02:43 DSNU719I -DB2G DSNUKTER - 7806192 ENTRIES CHECKED FOR INDEX 'PAOLOR4.TESTNPI2' DSNU719I -DB2G DSNUKTER - 7806192 ENTRIES CHECKED FOR INDEX 'PAOLOR4.TESTNPI' DSNU717I -DB2G DSNUKTER - 801456 ENTRIES CHECKED FOR INDEX 'PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 1 DSNU717I -DB2G DSNUKTER - 798204 ENTRIES CHECKED FOR INDEX 'PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 2 DSNU717I -DB2G DSNUKTER - 6206532 ENTRIES CHECKED FOR INDEX 'PAOLOR4.PXL#OKSDRFSKEPDC' PARTITION= 3 DSNU720I DSNUK001 - CHECKIDX PHASE COMPLETE, ELAPSED TIME=00:02:07 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0


CHECK INDEX does not correct any inconsistencies — it only reports whether or not a table space and its indexes are inconsistent. If the job returns a return code greater than zero, then you need to do the following:

� Examine the error messages from CHECK INDEX.

� REBUILD INDEX if the table space is correct.

� If index is correct, determine a consistent point-in-time for the table space, and run the RECOVER utility on the table space. Run CHECK INDEX again to verify consistency.

� If neither the table space nor its indexes are correct, determine a consistent point-in-time, then run the RECOVER job again, including the table space and its indexes all in the same list.

� Verify the point-in-time (TOLOGPOINT, TORBA, TOCOPY) for each object recovered. Use output from REPORT RECOVERY to determine a consistent point for both the table space and its indexes.

9.11.2 CHECK DATACHECK DATA checks the table spaces for violations of referential and table check constraints. It reports information about violation that are detected.

� If any violation is found, CHECK DATA puts the table space in CHECK-pending status.

� On successful execution, CHECK DATA resets the CHECK-pending status.

� CHECK DATA optionally deletes rows that violates referential or table check constraints if FOR EXCEPTION DELETE is specified:

– Requires DELETE privilege on table being checked– Requires INSERT privilege on exception table

� The AUXERROR INVALIDATE option requires UPDATE privilege on the base tables containing LOB columns.

Phases of CHECK DATAThe execution phases of CHECK DATA are summarized in Table 9-5.

Table 9-5 Phases of CHECK DATA

Phases Description

UTILINIT Initialization

SCANTAB Extract foreign keys; use foreign key index if it exist, else scan table

SORT Sort foreign keys if not extracted from foreign key index

CHECKDAT Look in primary indexes for foreign key parents, and issue messages to report errors detected

REPORTCK Copy error rows into exception tables, and delete them from source table if DELETE YES is specified

UTILTERM Cleanup


Sample CHECK DATA jobsExample 9-16 is a sample CHECK DATA job that checks two table spaces. The CHECK DATA utility supports list of table spaces, but it does not support the LIST option. Therefore, lists generated from the LISTDEF command cannot be input to the CHECK DATA utility. The utility supports TEMPLATE commands for the definition of work and error data sets.

Example 9-16 Sample CHECK DATA JCL

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=CHECKDA //SYSIN DD * TEMPLATE WORKDDN DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES(SBOX60,SBOX59) CHECK DATA TABLESPACE U7G01T11.TSLINEI TABLESPACE U7G01T11.TSPSUPP WORKDDN WORKDDN

9.11.3 CHECK LOBThe CHECK LOB online utility can be run against a LOB table space to identify any structural defects in the LOB table space and any invalid values. Run CHECK LOB when any one of the following situations occurs:

� LOB table space is marked CHKP.

� After a conditional restart or a point-in-time recovery on all table spaces where LOB table spaces might not be synchronized.

If no errors are found, CHECK LOB utility resets the CHKP and auxiliary warning (AUXW) statuses.

Phases of CHECK LOBTable 9-6 summarizes the phases of CHECK LOB.

Table 9-6 Phases of CHECK LOB

Note: The table space to be checked needs to be in CHECK pending (CHKP), else the utility will complete with RC=4 and message:

DSNU727I -DB2G DSNUKINP - TABLESPACE 'U7G01T11.TSLINEI' IS NOT CHECK PENDING

Phase Description

UTILINIT Initialization

CHECKLOB Scans all active pages of the LOB table space

SORT Sorts four types of records from CHECKLOB phase; reports four times the number of rows sorted

REPRTLOB Examines records that are produced by the CHECKLOB phase and sorted by the SORT phase, and issues error messages

UTILTERM Cleanup


Sample CHECK LOB jobExample 9-17 shows a sample CHECK LOB job that checks a list of LOB table spaces. The TEMPLATE command is used to define work data sets. Example 9-18 is the output of the CHECK LOB job. The LIST keyword is not supported by CHECK LOB, therefore lists generated by the LISTDEF command cannot be passed to the utility.

Example 9-17 Sample CHECK LOB JCL

//STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=CHECKLB //SYSIN DD * TEMPLATE DDN1 DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES(SBOX60,SBOX59) TEMPLATE DDN2 DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES(SBOX60,SBOX59) CHECK LOB EXCEPTIONS 3 TABLESPACE DSN8D71L.DSN8S71B TABLESPACE DSN8D71L.DSN8S71L TABLESPACE DSN8D71L.DSN8S71M TABLESPACE DSN8D71L.DSN8S71N WORKDDN DDN1,DDN2

Example 9-18 CHECK LOB job output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CHECKLB DSNU050I DSNUGUTC - TEMPLATE DDN1 DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES( SBOX60,SBOX59) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE DDN2 DSN(DB2V710G.RAMA.CHECK.INDEX) UNIT(SYSDA) DISP(NEW,DELETE,CATLG) VOLUMES( SBOX60,SBOX59) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - CHECK LOB EXCEPTIONS 3 TABLESPACE DSN8D71L.DSN8S71B TABLESPACE DSN8D71L.DSN8S71L TABLESPACE DSN8D71L.DSN8S71M TABLESPACE DSN8D71L.DSN8S71N WORKDDN DDN1,DDN2 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=DDN1 DDNAME=SYS00001 DSN=DB2V710G.RAMA.CHECK.INDEX DSNU795I DSNUK001 - CHECKLOB PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU042I DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=16 ELAPSED TIME=00:00:00 DSNU796I DSNUK001 - REPRTLOB PHASE COMPLETE, ELAPSED TIME=00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0


Chapter 10. Copying data

In this chapter we show and discuss the options available when copying DB2 data. We look first at the COPY utility and the methods used by the COPY utility. These include:

� Inline Copies with LOAD and REORG� Lists� COPY parallelism� The CHANGELIMIT option� CHECKPAGE option� FULL or INCREMENTAL option� REFERENCE or CHANGE option� COPY parameter of indexes� Tape or disk

We then discuss the new COPYTOCOPY utility, which allows you to make asynchronous extra image copies, and we briefly examine the Concurrent COPY option.

10


10.1 COPY The COPY utility is used to take secure images of table spaces and index spaces to allow for the recovery of data in the event of data loss or corruption.

10.1.1 Inline COPY with LOAD and REORGDB2 Version 5 introduced the ability to take image copies during the LOAD (REPLACE or RESUME NO) utility and during the REORG utility. This function was introduced to reduce the time the object was unavailable to the applications due to the COPYPEND flag being set.

Previously, after the execution of one of these utilities, assuming LOG NO is specified for speed, an image copy had to be taken to ensure the integrity of the data. The time taken to run the COPY added to the outage for the object. This new functionality addressed this issue.

When a LOAD REPLACE/LOAD RESUME NO is executed with an Inline COPY, the copy is taken during the LOAD phase. This means that if there are any rows on the table that are later discarded, these will still remain on the image copy. For this reason it is recommended NOT to use inline image copies in a RECOVERY TOCOPY operation. For more information; see “Using inline copies with the RECOVER utility” on page 43.

When an Inline COPY is taken by the REORG SHRLEVEL CHANGE utility, the contents of the image copy data set is different from the copy taken by LOAD. Online REORG takes an initial image copy during the RELOAD phase, then while it is applying changes from the log, it takes an incremental image copy and appends it onto the full image copy. This process is repeated during each log iteration as many times as needed to complete the online REORG. The end result is an image copy that has incremental copies added to the end of it, and therefore will contain duplicate pages.

This is not the same as an image copy produced using the MERGECOPY utility, which does not contain duplicates. The recover utility has been modified to handle these image copies, as it recognizes that the last version of the page is the valid one. The UNLOAD utility does not recognize these extra pages as duplicates and unloads all pages. This may lead to duplicate rows in the output data set; for more information; see 8.3, “UNLOAD from image copy” on page 194. DSN1COPY and DSN1PRNT will produce error messages when inline copies are used with them if the new keyword INLCOPY is not specified.

The use of Inline COPY is also discussed in 3.1.1, “Inline COPY” on page 40.

10.1.2 Copying indexesPrior to DB2 Version 6, the method of recovery of a DB2 object was to recover the table spaces and then to recover the indexes on the table(s). The RECOVER INDEX utility would read the table(s) and then a separate internal step was required to recreate the index(es) associated with the table(s). During the time of both recoveries, the tables were unavailable to the applications. The time taken by the recovery is addressed by DB2 Version 6 and the ability to copy indexes.

The Version 5 RECOVER INDEX utility was renamed to REBUILD INDEX to more accurate reflect the purpose of the utility, that is, to rebuild the index from the data in the table. The REBUILD INDEX utility is dependent upon the contents of the table space and therefore indexes cannot be built in parallel while the table space is being recovered.


DB2 Version 6 addressed this issue by introducing the ability to take full image copy indexes. These copies can then be used by the RECOVER INDEX utility. The RECOVER utility recovers an index in the same manner as a table space, that is, restoring an image copy and then applying the log. This method allows for faster recovering of an index, assuming timely image copies are taken, when compared to rebuilding the index, and also it allows for the recovery of table spaces and indexes in parallel, thus reducing the outage required.

To allow indexes to be copied, a new option is available on the CREATE/ALTER index DDL statements. COPY YES can be specified to indicate that the index can be copied by the COPY utility.

The COPY YES attribute:

� Allows DB2 full image copies and concurrent copies to be taken.� Allows the use of the RECOVER utility on the index.� Enables SYSCOPY recording for utility activity on the index.� Enables SYSLGRNX recording for the index.� Enables the setting of ICOPY and CHKP pending states for the index.� Enables down-level detection on the index.

Altering the index to COPY NO prohibits and disables the above and also removes the recovery information from SYSCOPY and SYSLGRNX for the index.

The following restrictions apply to copying an index:

� CHANGELIMIT cannot be used.� The copy data set is a sequential data set with a 4 KB LRECL.� Inline image copies cannot be used.� Incremental image copies cannot be used.� DSNUM ALL must be used for NPI.

The first new index state, ICOPY (informational copy pending) indicates that a copy of the index should be taken. This is set when an unrecoverable from the log event has occurred on the index or the underlying table space, this means that one of the following has happened:

� REBUILD INDEX� REORG INDEX� REORG TABLESPACE (LOG YES or LOG NO)� LOAD TABLE (LOG YES or LOG NO)

After the ICOPY state has been set, the index must be copied if the RECOVER utility is to be used against the index. If the copy is not taken, then in the event of a recovery a REBUILD INDEX must be used to build the index from the underlying table. The ICOPY state does NOT prevent any processing against the table or the index. This state is also set on initial creation of the index with COPY YES and if the index is altered to be COPY YES.

The second new index state is CHKP (check pending). It indicates that the index might be out of step with the underlying table and that the CHECK INDEX utility should be run. This state can be set by an index point-in-time recovery. During the running of the check the index is unavailable for any read or write activity.

The catalog SYSCOPY has a new column added, OTYPE, which has the values for ‘T’ for table space and ‘I’ for index. The index name is recorded under the column TSNAME and there is a new value of ‘B’ for ICTYPE which indicates REBUILD INDEX

Chapter 10. Copying data 233

Other utilities have been enhanced to work with indexes that have COPY YES set:

� RECOVER

– Support is included to recover from an image copy. This will end RC8 if COPY YES is not set, if an unrecoverable log event has occurred, or if there are no image copies for the index.

– The index is handled in the same way as table space recoveries.

� REPORT

– Indexes can now be specified.– Unusable image copies are marked, with (), to signify unsuitability.– The TABLESPACESET option reports all associated indexes.

� QUIESCE

– SYSCOPY row, type ‘Q’, is inserted for table spaces and associated indexes.– Indexes quiesced are identified in the output job.

� MODIFY

– MODIFY RECOVERY removes table space and associated index recovery information to keep the table space and index recovery information in sync.

� REBUILD INDEX

– This sets ICOPY status.– This updates SYSCOPY with ICTYPE = ‘B’.

� REORG INDEX

– This updates SYSCOPY with ICTYPE = ‘W’.– This sets ICOPY status.

� REPAIR

– SET INDEX support is added to reset CHKP and ICOPY.– LEVELID support for indexes is provided.

Some other changes are these:

� DISPLAY DATABASE

– New ADVISORY option is provided to display indexes in ICOPY status.– RESTRICT now displays indexes in CHKP status.

� START DATABASE

– ACCESS(FORCE) is expanded to include indexes.

� DSN1COPY/DSN1PRNT

– Option FULLCOPY now supports index image copies.

RecommendationsThe following are recommendations for the image copying of indexes:

� Use image copy on large indexes with low update activity that require high availability.� Copy indexes in conjunction with the underlying table space.� Recover table spaces and their indexes to the same point-in-time to avoid CHKP being set


10.1.3 ListsDB2 Version 6 introduced the ability to define lists to a COPY utility, that enabled a group of objects to be copied by the one COPY utility. This is different from the LISTDEF command, described in 2.2, “Wildcards” on page 18, in that the objects have to be explicitly coded in the COPY statement.

The utility processes the objects in the order that they are specified in the list. An example is shown in Example 10-1

Example 10-1 Specifying lists in the COPY utility

COPY TABLESPACE DB1.TS1 COPYDDN (IC1,IC2) RECOVERYDDN(IC3,IC4)INDEX SYSADM1.IND1 COPYDDN(IC5,IC6) RECOVERYDDN(IC7,IC8)

The phases of COPY can now switch between REPORT and COPY because each table space in the list with a CHANGELIMIT specification; see 10.1.5, “The CHANGELIMIT option” on page 238, will have a REPORT phase.

If SHRLEVEL REFERENCE is specified, then DB2 drains the write claim class on ALL table/index spaces during the UTILINIT phase, and holds the claim until the last object in the list has been copied. The SYSCOPY row for all the objects in the list are written at the same time when the last object has completed, and the START_RBA is the same for all rows, that being the RBA/LRSN after the last object has completed.

With SHRLEVEL CHANGE, DB2 establishes the read class claim on each object individually and releases the claim when it completes the object; it does not wait until every object has been completed. The SYSCOPY row for each object is inserted when the individual copy has completed and contains the START_RBA pertinent to the RBA/LRSN at the start of the processing for that object.

When processing a list, if COPY encounters an error with one of the objects it continues processing the next object in the list after issuing an error message. The return code at the end of the COPY utility will reflect the highest error set by the utility. To determine which objects caused the return code the utility output will have to be used.

With DB2 Version 7, the new functions of LISTDEF provide a dynamic method of building the LISTs and removes much of the LIST maintenance when new objects are created. See Chapter 2, “Wildcarding and Templates” on page 17 for details about LISTDEF.

10.1.4 COPY parallelismDB2 Version 6 introduced the concept of parallelism in the COPY and RECOVER utility. In this section we deal with the COPY aspect of parallelism (RECOVER is similar.)

The objective of the introduction of parallelism is to reduce the elapsed time of the COPY utility when run against a list of objects. To use parallelism with the COPY, and RECOVER, utility the new keyword PARALLEL n has to be specified; this tells DB2 the optimum number of parallel threads that are to be processed. If the keyword is omitted, then parallelism is not used. DB2 will decide the optimum number of parallel threads if the PARALLEL keyword is specified without a value. DB2 will also reduce the specified number of parallel tasks if there is a shortage of resources. If this is done, then message DSNU397I is issued, and the message DSNU427I will also be issued to identify the number of parallel tasks being used.


When using parallelism, COPY creates pairs of subtasks equivalent to the number of parallel task requested (or chosen by DB2). There are two subtasks per object, one to read the data and one to write the output. See Figure 10-1 for an example of COPY PARALLEL(3). If the list contains more than 3 objects, in this case, then the subtasks are reused for subsequent objects in the list. To calculate the number of the threads needed when PARALLEL is specified, use the formula:

– Threads = (n*2+1)

The value n is the number of objects that should be processed in parallel, regardless of the total number of objects in the list.

Parallelism is only supported for copies written to disk. If tape is used, then the parallelism is stopped until the copy to tape is finished. For example, if there are 6 objects in a list to be processed with PARALLEL(4) and the fourth object is written to tape, then objects 5 and 6 will not start until object 4 is finished, even if the other 3 threads have completed.

Restart is only supported from the last commit point of each object processed in parallel.

Figure 10-1 Parallel COPY with 3 subtasks

Example 10-2 show the statements required to build a COPY list using the LISTDEF functionality.

Example 10-2 Parallel COPY using LISTDEF

OPTIONS EVENT(ITEMERROR,SKIP) TEMPLATE LCOPY DSN(DB2V710G.&DB..&TS..T&TIME..P&PA.L) VOLUMES(SBOX59,SBOX60,SBOX58,SBOX57) LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI COPY LIST DB01A FULL YES PARALLEL(4) SHRLEVEL REFERENCE COPYDDN(LCOPY)

C o p yD atas ets

C o p yD atase ts

O b je c t 1O b je c t 2O b je c t 3O b je c t 4O b je c t 5

R e a d S u b ta s k 1O b jec t

S p aces

R e a d S u b ta s k 2

R e a d S u b ta s k 3

W rite S u b ta s k 1



P ip e

O b jec tS p aces

O b jec tS p a ces

P ip eP ip e

C o p y P a ra lle l (3 )

C o p yD a ta sets


It should be noted that the partitioning index does not have the COPY YES parameter. It is still included in the list and produces an error message stating that the copy parameter is not set. In this case the OPTIONS EVENT(ITEMERROR,SKIP) bypasses the error and COPY continues processing. The output from the job is shown in Example 10-3.

Example 10-3 Output from Parallel COPY using LISTDEF

1DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = CPITEM0DSNU050I DSNUGUTC - OPTIONS EVENT(ITEMERROR,SKIP) DSNU1035I DSNUILDR - OPTIONS STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - TEMPLATE LCOPY DSN(DB2V710G.&DB..&TS..T&TIME..P&PA.L) VOLUMES(SBOX59,SBOX60,SBOX58,SBOX57) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSLINEI PARTLEVEL INCLUDE INDEXSPACES TABLESPACE U7G01T11.TSLINEI DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY0DSNU050I DSNUGUTC - COPY LIST DB01A FULL YES PARALLEL(4) SHRLEVEL REFERENCE COPYDDN(LCOPY) DSNU427I DSNUBBID - OBJECTS WILL BE PROCESSED IN PARALLEL, NUMBER OF OBJECTS = 4

DSNU425I -DB2G DSNUBAII - INDEXSPACE U7G01T11.PXL#OKSD DOES NOT HAVE THE COPY YES ATTRIBUTE DSNU1027I -DB2G DSNUBAII - PROCESSING CONTINUES DUE TO OPTIONS ITEMERROR SKIP DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LCOPY DDNAME=SYS00001 DSN=DB2V710G.U7G01T11.TSLINEI.T002257.P00001L DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LCOPY DDNAME=SYS00002 DSN=DB2V710G.U7G01T11.TSLINEI.T002257.P00002L DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LCOPY DDNAME=SYS00003 DSN=DB2V710G.U7G01T11.TSLINEI.T002257.P00003L DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LCOPY DDNAME=SYS00004 DSN=DB2V710G.U7G01T11.TESTNPI2.T002257.P00000L DSNU400I DSNUBBID - COPY PROCESSED FOR INDEXSPACE U7G01T11.TESTNPI2 NUMBER OF PAGES=9751 AVERAGE PERCENT FREE SPACE PER PAGE = 0.00 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:00:22 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LCOPY DDNAME=SYS00005 DSN=DB2V710G.U7G01T11.TESTNPI.T002257.P00000L DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE U7G01T11.TSLINEI DSNUM 1 NUMBER OF PAGES=13595 AVERAGE PERCENT FREE SPACE PER PAGE = 3.34 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:00:27 DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE U7G01T11.TSLINEI DSNUM 2 NUMBER OF PAGES=13831 AVERAGE PERCENT FREE SPACE PER PAGE = 3.35 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:00:28 DSNU400I DSNUBBID - COPY PROCESSED FOR INDEXSPACE U7G01T11.TESTNPI NUMBER OF PAGES=10128 AVERAGE PERCENT FREE SPACE PER PAGE = 0.00 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:00:13 DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE U7G01T11.TSLINEI DSNUM 3 NUMBER OF PAGES=107667 AVERAGE PERCENT FREE SPACE PER PAGE = 3.36 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:01:22 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI DSNUM 1 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI DSNUM 2 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSLINEI DSNUM 3 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI2 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR INDEXSPACE U7G01T11.TESTNPI DSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

The number of parallel tasks is set to 4 and the number of objects available for copying is 5 (ignoring the rejected partitioning index). The output from a -DISPLAY UTIL command, in Example 10-4, shows that there were 4 pairs of subtasks running in parallel.


Example 10-4 -DISPLAY UTILITY for Parallel COPY

-db2g dis util(*) DSNU105I -DB2G DSNUGDIS - USERID = PAOLOR2 MEMBER = UTILID = CPITEM PROCESSING UTILITY STATEMENT 1 UTILITY = COPY PHASE = COPY COUNT = 0 NUMBER OF OBJECTS IN LIST = 6 LAST OBJECT STARTED = 0 STATUS = ACTIVE DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYW COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYR COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYW COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYR COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYW COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYR COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYR COUNT = 0 DSNU111I -DB2G DSNUGDIS - SUBPHASE = COPYW COUNT = 0 DSN9022I -DB2G DSNUGCCC '-DIS UTIL' NORMAL COMPLETION

Also worth noting is that the number of objects in the list is 6 (consisting of 3 table space partitions, 1 partitioning index, and 2 NPIs), and that the partitioning index is included and later rejected by the COPY utility.

10.1.5 The CHANGELIMIT optionCHANGELIMIT was introduced to automate the decision making process of whether to image copy, or not, and whether to take incremental or full copies. It also introduced a new phase to the copy process: the REPORT phase.

There are two modes to the CHANGELIMIT, a report-only mode and a run mode. If REPORTONLY is specified, DB2 will only execute the REPORT phase of the utility and will report on the following values:

� The total number of pages. This is the number of pages in the table space and may include preformatted pages. It is the number of pages that a full copy will copy.

� Empty pages in a segmented table space.

� Number of changed pages, that is, the number of pages an incremental copy would copy.

� Percentage of changed pages.

� Recommendation of the type of image copy required.

The report for table spaces will be displayed at the following levels:

� Partition level, if the table space is partitioned.� Data set level, if the table space is not partitioned.� The entire table space.

The recommendation given by the REPORT phase is based on values provided by the CHANGELIMIT parameter values. These values specify the range of changed pages expressed as a percentage of total pages in the table space.


One or two values can be supplied by the CHANGELIMIT parameter:

� One value:

– An incremental image copy is recommended if the percentage of changed pages is between zero and the supplied value.

– A full image copy is recommended if the percentage of changed pages is greater than the supplied value.

– No image copy is recommended if no pages have been changed.

� Two values (v1,v2):

– An incremental is recommended if the percentage of changed pages is greater than v1 and less than v2.

– A full copy is recommended if the percentage of changed pages is greater than v2.

– No image copy is recommended if the percentage of changed pages is less than v1.

The default values are 1,10. Example 10-5 illustrates the utility stream input to check for image copies using the range 0 to 25, while Example 10-6 shows the report from the COPY command.

Example 10-5 Specifying CHANGELIMIT

COPY LIST DB01A CHANGELIMIT(0,25) SHRLEVEL REFERENCE COPYDDN(LOCALDDN)

Example 10-6 Report output from COPY utility

4KB CHANGED PERCENT OF DBNAME TSNAME DSNUM PAGES PAGES CHANGED PAGES ICTYPE -------- -------- ----- ------------- ------------- ------------- ------ U7G01T11 TSLINEI 1 3,415 2 0.05 U7G01T11 TSLINEI 2 3,479 0 0.00 U7G01T11 TSLINEI 3 26,938 0 0.00 U7G01T11 TSLINEI ALL 33,832 2 < 0.01 I

The report shows that the COPY takes an incremental copy, at DSNUM ALL level.

COPY sets a return code dependent upon which option is taken:

� RC1 if the utility does not take an image copy� RC2 if an incremental copy is taken� RC3 if a full copy is taken� RC8 if a broken page has been encountered

This enables the job stream to automate the processing of image copies, that is, a REPORTONLY STEP is run and subsequent step process a full copy, or an incremental copy dependent upon the return code. If using this method ensure that the SYSCOPY DDNAME in the REPORTONLY step is set to DUMMY to ensure that extra data sets are not cataloged.

NOTE: Image copy data sets are always allocated by COPY, even if REPORTONLY is specified. Care should be exercised when using GDGs to ensure that valid image copies are not rolled out. This is the case whether specifying a DD statement or using Templates.


10.1.6 The CHECKPAGE optionThe COPY utility checks the integrity of data pages as it is copying the table space or index space. Previously, any additional checking had to be undertaken using DSN1COPY with the CHECK option specified.

Now additional checking can be undertaken using the CHECKPAGE option of the COPY utility, introduced in DB2 Version 6. This option is equivalent to the checking undertaken by DSN1COPY with CHECK. By specifying this option, DB2 will validate each page of the table space or index page as it is processed.

The benefit of using this option is that any errors in the pages are reported at the time that the backup is taken and not when the image copy is required by the RECOVER utility. The risk of there being any errors when taking the image is low, but the impact of finding out at recover time is high.

Figure 10-2 CHECKPAGE option of COPY utility

Figure 10-2 shows how CHECKPAGE can be used to validate images copies. Suppose a defect is introduced by an undetected hardware error (these are rare events) — it is then possible to create a defective image copy (time A). If this went undetected, the situation could arise where all image copies were defective and were not valid for recovery. A periodic COPY with the CHECKPAGE option would report the error (time B) which provides an opportunity to rectify the error, either using REPAIR or recovery back to a valid image copy. After repairing the error, a new valid image copy should then be taken.

Note: CHANGELIMIT opens each data set to determine whether an image copy is required or not. CHANGELIMIT will recall the data sets if migrated, if AUTORECALL is allowed. CHANGELIMIT will update the last referenced date on any data set it opens, therefore data sets will not get migrated if CHANGELIMIT is used frequently.

New CHECKPAG E option for CO PY

Table space and index space pagessupport for 4 KB, 8 KB, 16 KB, 32 KB page sizesno im pact on elapsed tim e, negligible increase in CPU tim e

Copy checks every index andtable space data and spacem ap page

CO PY TABLESPACE PAOLO R8.CHC KPAG E FULL YES CO PYD DN (CO PYL1) IND EXSPAC E PAOLOR 8.CH CK1V$I CO PYD DN (IXC PL1) CHECKPAGE PAR ALLEL(2) SHR LEVEL R EFEREN CE

im age copynot checkedreturn code=0

im age copycheckedreturn code=8

tim erecoverreturn code=0

contingency copy

A B C D

repair orrecover

Hardware error - undetected corruption

page 3 page 4page 1 page 2


If COPY identifies an invalid page, it will:

� Identify the page as broken.� Issue message DSNU518I if the page is a data page.� Issue message DSNU441I if the page is a space map page.� Complete RC8.� Set the copy pending flag on (COPY or ICOPY).

Remedial action should be initiated immediately by using either RECOVER PAGE (page numbers are given by DSNT518I and DSNU441I), RECOVER, or REPAIR.

The performance impact of using checking shows that the elapsed time is not impacted by this option and that the CPU time for the option shows a rise of 5% for indexes and 15% for table spaces, which is negligible for an I/O bound utility.

It is recommended that CHECKPAGE be incorporated into the normal image copy cycle to ensure the integrity of image copies. Be aware that CHECKPAGE will leave any object in copy pending, which may lead to a service outage, something that does not happen with the DSN1COPY.

10.1.7 Full or incremental copiesThe two types of COPY are the INCREMENTAL and FULL options. INCREMENTAL is used to only copy the pages that have been changed since the last full copy, while FULL, as the name implies, copies all the pages in the table space or index space. INCREMENTAL is not available for index copies.

In previous releases of DB2, the performance of COPY was improved significantly by changing the method of determining whether a page required copying. COPY now uses the space map bits and an image copy bit log record sequence number (LRSN) which provide an equivalent function. This means that the “dirty” page bit no longer needs resetting, which was a major overhead to the copy process. This change significantly reduced the elapsed time of the COPY utility.

This enhancement meant that the ability to take incremental copies was more realistic. Previously, the recommended limit for incremental was 10% of the pages being dirty; with this change, that limit is raised to 50%. To identify the percentage of changed pages that exist in a data set, use the CHANGELIMIT REPORTONLY option of COPY; see 10.1.5, “The CHANGELIMIT option” on page 238.

You can take incremental image copies if the following conditions apply:

� A full image copy of the table space exists� The COPY-pending status is not on for that table space� The last copy was taken without the CONCURRENT option� TRACKMOD YES is specified

When using incremental copy, one point to remember is that to improve recovery time, MERGECOPY should be run to incorporate the incremental copies and the full copy into a new full copy, this will improve the time taken by the RECOVER utility. When running MERGECOPY, DB2 will request that all the incremental copies and the full copy be available at the same time. This may cause a problem if the copies are on tape drives and there are more copies than drives available. Therefore, MERGECOPY should be run frequently, especially for copies taken to tape, but also for all media to speed up the recovery process.


10.1.8 SHRLEVEL REFERENCE or SHRLEVEL CHANGEThe option to take a copy with SHRLEVEL REFERENCE or SHRLEVEL CHANGE will depend upon the required availability of the table space to applications and users.

IF SHRLEVEL REFERENCE is specified, applications can only read the read the data and not update the data. This may be contrary to business requirements that demand higher availability. To help meet this need, the use of SHRLEVEL CHANGE should be used; this will allow full read and write access during the copy process.

Image copy data sets taken with SHRLEVEL CHANGE should NOT be used in a RECOVER TOCOPY utility, as it may contain data that is not committed. These copies should only be used in a point-in-time recovery to a previously defined point of consistency. Therefore, it is important to establish a point of consistency to be used a recovery point, this could be a QUIESCE RBA, an -ARCHIVE LOG MODE(QUIESCE) point, or the current time.

Some utilities, for example, REORG, demand that a SHRLEVEL REFERENCE copy be taken after the utility has completed, which led to the outage from REORGs being extended. REORG can now take inline image copies, which are SHRLEVEL REFERENCE copies, during execution which eliminates the delay in availability; see 10.1.1, “Inline COPY with LOAD and REORG” on page 232.

10.1.9 Tape or diskImage copies can be written to either tape or disk — the question is: which one?

The advantage of image copy to disk is that the elapsed time of the copy is less, due to the speed of the media and because DB2 can initiate parallelism when using lists with the copy; see 10.1.4, “COPY parallelism” on page 235. The number of tape units may also restrict the running of multiple copy jobs and parallelism. The preferred choice is therefore writing to disk.

The secondary, or recovery, image copies have traditionally been sent to tape, or any remote device, while sending the primary image copy to disk, by specifying the appropriate unit in the RECOVDDN option. This can drastically reduce the benefits of writing the primary copy to disk as the elapsed time of the utility is determined by the slowest device or narrowest bandwidth is case of transmission. With the new utility, COPYTOCOPY (refer to 10.2, “COPYTOCOPY” on page 243), this delay can be eliminated and the secondary or recovery copy initiated asynchronously, thus allowing higher availability of the application.

Other advantages of using disk over tape are:

� Faster elapsed time

� Faster recovery times, due to speed of the media and the ability to use parallelism

� Faster MERGECOPY, as well as the removal of problems with number of drives available for incremental copies and the same tape being used for full and incremental copies

� Availability of parallelism

With the cost of new disk coming down and the new disk technology, it is recommended that image copies be taken to disk and the new utility COPYTOCOPY be used to take recovery backups, or all should be taken to disk and then migrated with HSM type of function.

Note: Apply the PTF for PQ45835 for problems with COPY when using LISTDEF and TEMPLATE.


10.2 COPYTOCOPYThe COPY utility and the LOAD and REORG utilities with Inline COPY can produce local primary and backup copies and recovery site primary and backup copies. Taking two local and two recovery sites copies all at once can tie up four tape drives per utility, and if multiple copies are running concurrently, then there may be a need for more tape drives than are available to the site. In addition, some customers use remote attached tape drives for their recovery site copies, thus causing the copy to take longer and decreasing data availability because of bandwidth constraints. The requirement, therefore, is to be able to take a single copy and later make additional copies, asynchronously from the original, and record the copies in SYSCOPY; see Figure 10-3. This will increase the data availability of objects, due to reducing the elapsed time, and will also reduce the costs of running the COPY utility.

Figure 10-3 COPYTOCOPY

COPYTOCOPY is a new utility that addresses this issue, it can be used to make copies from an image copy that was taken by the COPY utility, including inline copies taken by REORG and LOAD utilities. COPYTOCOPY will leave the target object in read write access (UTRW), which allows for most other utilities and SQL statements to run concurrently on the same target objects. The utilities that are not allowed to run concurrently with COPYTOCOPY on the same target object are utilities that insert or delete records into SYSCOPY, these are:

� COPY� LOAD� MERGECOPY� MODIFY� RECOVER� QUIESCE� REORG

The only other utilities not allowed concurrently are utilities that operate with SYSCOPY as its target.

CopyToCopyCopyToCopy

image copies

Partitioned table spaceSimple table space

Segmented table spaces

Copy the copyRecord in catalog

COPYTOCOPY TABLESPACE DB1.TS1 FROMLASTFULLCOPY RECOVERYDDN(rempri)


Input to the utility is either the local primary or the recovery site primary copy from which COPYTOCOPY can make up to three copies of one or more of the following types:

� Local primary� Local backup� Recovery site primary� Recovery site backup

COPYTOCOPY does not support the following:

� DSNDB01.SYSUTILX and its indexes� DSNDB01.DBD01 and its indexes� DSNDB06.SYSCOPY and its indexes� Image copies taken with CONCURRENT option� Image copies taken by REORG part range

COPYTOCOPY does not check recoverability of an object.

The copies created by COPYTOCOPY can be used by RECOVER when recovering a table space and are also available to MERGECOPY, UNLOAD, and also another COPYTOCOPY.

Output from the utility is up to three “new” image copies and rows in SYSCOPY that describe the image copies. All entries in SYSCOPY are the same as the original copy, apart from ICBACKUP, which donates the type of image copy, for example, remote primary, and so on.

COPYTOCOPY is recommended for use when:

� Additional copies of objects that are high availability are required and the extra copies are taken to slower devices.

� The target object is required to remain in UTRW status, allowing SQL statements to run concurrently with the same target object.

10.2.1 COPYTOCOPY optionsCOPYTOCOPY can use the LISTDEF and Template functionality described in Chapter 2, “Wildcarding and Templates” on page 17. An example of this is shown in Example 10-7. This example also illustrates the statements required to take three copies from the local primary copy. It is not possible to use COPYTOCOPY to take a copy of the local primary and create another local primary copy.

Example 10-7 COPYTOCOPY and LISTDEFs

TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.&LOCREM.) VOLUMES(SBOX60) TEMPLATE RECOVDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.&LOCREM.) VOLUMES(SBOX60) TEMPLATE RECOVDD2 DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.Z) VOLUMES(SBOX60) LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSPSUPP1

Note: The TIMESTAMP field in SYSCOPY is the timestamp of the original image copy. If date and time are used in the data set name, as in Example 10-7, these values will contain the date and time that the COPYTOCOPY was run and not the date and time in the input copy data set.


COPYTOCOPY LIST DB01A FROMLASTCOPY COPYDDN(,LOCALDDN) RECOVERYDDN(RECOVDDN,RECOVDD2)

Options for identifying the copy to use as input are:

FROMLASTCOPY:

� Specifies the most recent image copy that was taken for the table space or index space to be the input to the COPYTOCOPY utility. This could be a full image copy or incremental copy retrieved from SYSIBM.SYSCOPY

FROMLASTFULLCOPY

� Specifies the most recent full image copy that was taken for the object to be the input to COPYTOCOPY job

FROMLASTINCRCOPY

� Specifies the most recent incremental image copy that was taken for the object to be the input to COPYTOCOPY job. If specified with an INDEX or INDEX SPACE this option will revert to FROMLASTFULLCOPY

FROMCOPY dsn

� Specifies a particular image copy data set (dsn) as the input to COPYTOCOPY job. This option is not valid for LIST. The GDG names must be specified fully.

Output from Example 10-7 is reproduced in the Example 10-8

Example 10-8 COPYTOCOPY using LASTFULLCOPY

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR2 DSNU050I DSNUGUTC - TEMPLATE LOCALDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.&LOCREM.) VOLUMES(SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE RECOVDDN DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.&LOCREM.) VOLUMES(SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE RECOVDD2 DSN(DB2V710G.&DB..&TS..D&DATE..T&TIME.Z) VOLUMES(SBOX60) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LISTDEF DB01A INCLUDE TABLESPACE U7G01T11.TSPSUPP1 DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - COPYTOCOPY LIST DB01A FROMLASTCOPY COPYDDN(,LOCALDDN) RECOVERYDDN(RECOVDDN,RECOVDD2)DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=LOCALDDN DDNAME=SYS00001 DSN=DB2V710G.U7G01T11.TSPSUPP1.D2001151.T233937L DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=RECOVDDN DDNAME=SYS00002 DSN=DB2V710G.U7G01T11.TSPSUPP1.D2001151.T233937R DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=RECOVDD2 DDNAME=SYS00003 DSN=DB2V710G.U7G01T11.TSPSUPP1.D2001151.T233937Z DSNU1403I DSNU2BCC - LOCAL SITE PRIMARY DATA SET DB2V710G.U7G01T11.TSPSUPP1.D2001151.T233722L WITH START_RBA 000058B794BC IS IN USE BY COPYTOCOPY FOR TABLESPACE U7G01T11.TSPSUPP1 DSNU1404I DSNU2BDR - COPYTOCOPY PROCESSING COMPLETED FOR TABLESPACE U7G01T11.TSPSUPP1 ELAPSED TIME = 00:00:38 NUMBER OF PAGES COPIED=24805 DSNU1406I DSNU2BDR - COPYTOCOPY COMPLETED. ELAPSED TIME = 00:00:38 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0


COPYTOCOPY will not allow the copying of an image copy, for example local primary, to another type, for example, remote primary, when an image copy of the same type with the same LRSN already exists; that is, a remote primary was taken at the same time as the local primary was taken. The utility does not support inline copies made by the REORG with part range. To copy these, use the individual DSNUM(n), because COPYTOCOPY copies only the specified partition into the output data sets.

10.2.2 Performance measurementsThe measurements, in Table 10-1, were taken using a ten partition table with 517,338 pages, and all are included in the measurement. The single COPY is of the format COPY TABLESPACE db.ts while the COPY LIST uses the LISTDEF command.

Table 10-1 Performance of COPYTOCOPY

The measurements shows that you can save up to 4.9% in elapsed time per copy by using COPYTOCOPY utility instead of using the COPY utility.

The main benefit from COPYTOCOPY is the ability to make additional image copies asynchronously and it mostly beneficial when making remote copies to slower devices.

10.3 Concurrent COPYThis option of the COPY utility enables you to make full image copies using DFSMS Concurrent COPY. The copy process is initiated by the DB2 COPY utility when you specify the CONCURRENT keyword on the COPY statement. The image copy is recorded in SYSCOPY with ICTYPE = F and STYPE = C for instance node “I0001“ and STYPE = J for instance node “ J0001”.

To use COPY to take DFSMS concurrent copies, you must have the following hardware and software installed:

� OS/390 Release 3.

� 3990 Model 3 or 3990 Model 6 controller at the extended platform attached to the disk. A COPY job fails if one or more of the table spaces are on a disk that does not have the right storage controller.

In Figure 10-9 you see the error messages you will get if you are using this option without the appropriated hardware and software support.

COPY utility COPYTOCOPY utility Delta(%)

CPU Time

Elapsed Time

CPU Time

Elapsed Time

CPU Time

Elapsed Time

Single COPY

11 182 10 173 -9.1 -4.9

COPY LIST

10 177 10 173 0 2.3



Example 10-9 Output job for Concurrent COPY with error

DSNU050I DSNUGUTC - COPY TABLESPACE U7G01T11.TSPSUPP2 COPYDDN(MARCELO) DSNUM 3 CONCURRENT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=MARCELO DDNAME=SYS00005 DSN=PAOLOR4.UTIL.TSPSUPP2.P00003.T233225 DSNU421I DSNUBBCM - START OF DFSMS MESSAGES PAGE 0001 5695-DF175 DFSMSDSS V2R10.0 DATA SET SERVICES 2001.177 19:32 PARALLEL ADR101I (R/I)-RI01 (01), TASKID 001 HAS BEEN ASSIGNED TO COMMAND 'PARALLEL' DUM OPT(3) DATAS(INCL(PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A003)) - CAN CONC SHA TOL(ENQF) WAIT(0,0) - OUTDD(SYS00005) ADR101I (R/I)-RI01 (01), TASKID 002 HAS BEEN ASSIGNED TO COMMAND 'DUM ' ADR109I (R/I)-RI01 (01), 2001.177 19:32:33 INITIAL SCAN OF USER CONTROL STATEMENTS COMPLETED. ADR014I (SCH)-DSSU (02), 2001.177 19:32:33 ALL PREVIOUSLY SCHEDULED TASKS COMPLETED. PARALLEL MODE NOW IN EFFECT ADR050I (002)-PRIME(01), DFSMSDSS INVOKED VIA APPLICATION INTERFACE ADR016I (002)-PRIME(01), RACF LOGGING OPTION IN EFFECT FOR THIS TASK ADR006I (002)-STEND(01), 2001.177 19:32:33 EXECUTION BEGINS ADR411W (002)-DTDSC(04), DATA SET PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A003 IN CATALOG UCAT.VSBOX01 ON VOLUME SBOX58 WAS NOT SERIALIZED ON REQUEST ADR356E (002)-UIMXT(01), TASK TERMINATED BY UIM EXIT (24) ADR735W (002)-T0MI (05), 2001.177 19:32:33 USE OF CONCURRENT COPY FAILED FOR DATA SET PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A003 ON VOLUME SBOX59, 002. SERIALIZATION WILL BE HELD AND CONCURRENT COPY WILL NOT BE USED. ADR356E (002)-UIMXT(01), TASK TERMINATED BY UIM EXIT (23) ADR801I (002)-DTDSC(01), DATA SET FILTERING IS COMPLETE. 1 OF 1 DATA SETS WERE SELECTED: 0 FAILED SERIALIZATION AND 0 FAILED FOR OTHER REASONS. ADR006I (002)-STEND(02), 2001.177 19:32:33 EXECUTION ENDS ADR013I (002)-CLTSK(01), 2001.177 19:32:33 TASK COMPLETED WITH RETURN CODE 0008 ADR012I (SCH)-DSSU (01), 2001.177 19:32:33 DFSMSDSS PROCESSING COMPLETE. HIGHEST RETURN CODE IS 0008 FROM: TASK 002

DSNU422I DSNUBBCM - END OF DFSMS MESSAGE DSNU409I DSNUBBUM - NO HARDWARE SUPPORT FOR TABLESPACE U7G01T11.TSPSUPP2 DSNUM 3 DSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

If you use the CONCURRENT option:

� You must supply either a COPYDDN ddname, a RECOVERYDDN ddname, or both.

� If the SYSPRINT DD card points to a data set, you must use a DSSPRINT DD card.

� You must use the SHRLEVEL REFERENCE option for table spaces with an 8 KB, 16 KB, or 32 KB page size.

Restrictions:

� You cannot use a copy made with DFSMS Concurrent COPY with the PAGE or ERRORRANGE options of the RECOVER utility.

� You cannot run the following DB2 stand-alone utilities on copies made by DFSMS Concurrent COPY:

– DSN1COMP, DSN1COPY, DSN1PRNT

– You can manually restore the DFSMS copy data set under a different name and run these utilities against the restored data set.


� If you try to take an INCREMENTAL image copy of a table space after a full image copy is taken using DFSMS Concurrent COPY, you will receive the DSNU402I message. Refer to Example 10-10. Please notice that the percent of changed pages is zero.

Example 10-10 Output from incremental copy after a Concurrent COPY

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = TEMP DSNU050I DSNUGUTC - TEMPLATE MARCELO UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..P&PA..T&TIME.) DISP(MOD,CATLG, CATLG) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - COPY TABLESPACE U7G01T11.TSPSUPP2 COPYDDN(MARCELO) DSNUM 1 FULL NO DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=MARCELO DDNAME=SYS00001 DSN=PAOLOR4.UTIL.TSPSUPP2.P00001.T003531 DSNU402I -DB2G DSNUBAIC - INCREMENTAL IMAGE COPY DISALLOWED FOR TABLESPACE U7G01T11.TSPSUPP2 DSNUM 1 FULL IMAGE COPY WILL BE TAKEN DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE U7G01T11.TSPSUPP2 DSNUM 1 NUMBER OF PAGES=3145 AVERAGE PERCENT FREE SPACE PER PAGE = 1.07 PERCENT OF CHANGED PAGES = 0.00 ELAPSED TIME=00:00:03 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSPSUPP2 DSNUM 1 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

When a recoverable point that indicates that a DFSMS Concurrent COPY is found in SYSCOPY, the RECOVER utility invokes a DFSMSdss RESTORE command to restore the copy. Refer to Example 10-11.

Example 10-11 Example of RECOVER using a Concurrent COPY

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = DSNTEX DSNU050I DSNUGUTC - RECOVER TABLESPACE U7G01T11.TSPSUPP2 DSNUM 1 REUSE TOCOPY PAOLOR4.UTIL.TSPSUPP2.T162257 DSNU515I DSNUCBAL - THE IMAGE COPY DATA SET PAOLOR4.UTIL.TSPSUPP2.T162257 WITH DATE=20010626 AND TIME=122304 IS PARTICIPATING IN RECOVERY OF TABLESPACE U7G01T11.TSPSUPP2 DSNUM 1 DSNU421I DSNUCBUI - START OF DFSMS MESSAGES PAGE 0001 5695-DF175 DFSMSDSS V2R10.0 DATA SET SERVICES 2001.177 16:34 ADR030I (SCH)-PRIME(01), DCB VALUES HAVE BEEN MODIFIED FOR SYSPRINT RESTORE DATASET(INCL(PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001)) - INDD(SYS00001) CANCE DYNALLOC REPLACE SHARE TOL(ENQF) WAIT(0,0) OUTDY(+ (SBOX58), (SBOX59), (SBOX60), (SBOX57)) ADR101I (R/I)-RI01 (01), TASKID 001 HAS BEEN ASSIGNED TO COMMAND 'RESTORE ' ADR109I (R/I)-RI01 (01), 2001.177 16:34:17 INITIAL SCAN OF USER CONTROL STATEMENTS COMPLETED. ADR050I (001)-PRIME(01), DFSMSDSS INVOKED VIA APPLICATION INTERFACE ADR016I (001)-PRIME(01), RACF LOGGING OPTION IN EFFECT FOR THIS TASK ADR006I (001)-STEND(01), 2001.177 16:34:17 EXECUTION BEGINS ADR780I (001)-TDDS (01), THE INPUT DUMP DATA SET BEING PROCESSED IS IN LOGICAL DATA SET FORMAT AND WAS CREATED BY DFSMSDSS VERSION 2 RELEASE 10 MODIFICATION LEVEL 0 ADR442I (001)-FRLBO(01), DATA SET PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001 PREALLOCATED, IN CATALOG UCAT.VSBOX01, ON VOLUME(S): SBOX58 ADR489I (001)-TDLOG(02), CLUSTER PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001 WAS RESTORED CATALOG UCAT.VSBOX01 COMPONENT PAOLOR1.DSNDBD.U7G01T11.TSPSUPP2.J0001.A001


ADR454I (001)-TDLOG(01), THE FOLLOWING DATA SETS WERE SUCCESSFULLY PROCESSED PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001 ADR006I (001)-STEND(02), 2001.177 16:34:21 EXECUTION ENDS ADR013I (001)-CLTSK(01), 2001.177 16:34:21 TASK COMPLETED WITH RETURN CODE 0000 ADR012I (SCH)-DSSU (01), 2001.177 16:34:21 DFSMSDSS PROCESSING COMPLETE. HIGHEST RETURN CODE IS 0000

With SHRLEVEL REFERENCE the objects are not available for update until the copy operation is logically completed:

� All writes are drained, and the objects being copied have a restrictive state of UTRO.

� The objects are quiesced to ensure that all updated buffers are written to disk before DFSMS Concurrent COPY is invoked. The SYSCOPY records with ICTYPE=Q are inserted to indicate that the objects are successfully quiesced.

� As soon as the DFSMS Concurrent COPY is logically complete, the objects are dedrained, and the restrictive state is changed from UTRO to UTRW.

When you use the DFSMS Concurrent COPY, you improve the availability of the application data. However, if your table space data sets have more empty space, the physical copy operation may take longer than the DB2 image copy. DFSMS Concurrent COPY does not process the space page map the way DB2 image copy processes it, so the output data sets of DFSMS Concurrent COPY may require more disk space than the output data sets created by the DB2 COPY utility.

10.3.1 Concurrent COPY SHRLEVEL REFERENCE using FILTERDDNConcurrent COPY was intended to work as follows when a list of table spaces is processed in one COPY statement with SHRLEVEL REFERENCE, with each output data set going to the same volume. As each table space is logically completed by DFSMS/DSS, it is put back into UTRW status by DB2, so that other activity can be performed on that table space. Normally, the Concurrent COPY is a two step process: logical completion, then physical completion.

Ideally, the logical completions of all the table spaces are done together, and all the actual copying can be done later. However, if you specify a list of table spaces to be copied by Concurrent COPY with SHRLEVEL REFERENCE, with each of the copies going to the same output device, all table spaces are put into UTRO and they each remain in UTRO until their respective copies are logically completed, one by one. This means that each table space on the list is not logically completed until the previous one is both logically and physically completed.

This is a big impact to customers who want to cut down the amount of time that their table spaces are unavailable. This is only a problem when each copy output data set is going to the same volume.

To get around this restriction with the existing COPY and DFSMS/DSS infrastructure, you can use the FILTERDDN keyword and copy a list of table spaces using Concurrent COPY to a single image copy output data set (or up to four identical data sets in the case of Local Backup, Recovery site Primary, and Recovery site Backup data sets), instead of the current method of specifying one image copy data set (or up to four in the case of LB, RP, and RB data sets) for each table space in the list. This way, all of the table spaces are processed with a single DFDSS DUMP statement.

Users who do not need relief from this availability problem can use the syntax without the FILTERDDN keyword.


To invoke this functionality, changes to the syntax of existing Concurrent COPY jobs are necessary.

Example 10-12 shows the syntax of a COPY statement without the FILTERDDN, whereas Example 10-13 is using the FILTERDDN option.

Example 10-12 Copy statement without FILTERDDN

COPY TABLESPACE DB1.TS1COPYDDN (SYSCOPY1)

TABLESPACE DB2.TS2COPYDDN (SYSCOPY2)

CONCURRENT

Example 10-13 Copy statement using FILTERDDN

COPY TABLESPACE DB1.TS1TABLESPACE DB2.TS2

FILTERDDN(filterdd)COPYDDN (SYSCOPY)

CONCURRENT

The usage of the FILTERDDN keyword allows for only one COPYDDN parameter for the entire list of table spaces. Additionally, the use of a new parameter, FILTERDDN, is required. FILTERDDN points to a DD card for a data set that is passed to DFSMS/DSS that contains a list of all the table spaces to be processed. The contents of the data set will be written by DB2, and will be transparent to the user. The user only has to provide a DD card for the FILTERDDN and make sure that the image copy data set is large enough to contain all the table spaces in the list.

When you use the FILTERDDN option, one row will be inserted in SYSIBM.SYSCOPY for each table space or partition (if DSNUM is specified) in the table space list. This is identical as when you are not using the FILTERDDN option. However, the DSNAME column in SYSCOPY will have the same data set name for each table space which is copied at one time using FILTERDDN.

The FILTERDDN keyword is supported when using the DSNUTILS stored procedure to start DB2 utilities.

A sample JCL using FILTERDDN and TEMPLATEs could look like Example 10-14, and the sample job output can be found in Example 10-15. As you can see in the job output, there is only one call to DFSMSdss for both partitions that are being copied.There is only one set of ‘ADR006I execution begins - execution ends’ messages, as well as message ADR801I, indicating that two data sets were selected when using FILTERDDN, instead of one for every call to DFSMSdss, if FILTERDDN is not used.

Example 10-14 Complete JCL using FILTERDDN

//BARTCC01 JOB ,CQM,CLASS=A,MSGCLASS=X, // NOTIFY=&SYSUID //JCLL JCLLIB ORDER=DB2V710G.PROCLIB //UTIL EXEC DSNUPROC,SYSTEM=DB2G,UID='COPY',UTPROC='' //* //SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE U7G01T11.TSPSUPP2 PARTLEVEL(1)


INCLUDE TABLESPACE U7G01T11.TSPSUPP PARTLEVEL(1) TEMPLATE CCDSN UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..P&PA..T&TIME.) DISP(MOD,CATLG,CATLG) TEMPLATE FILTDSN UNIT SYSDA DSN(PAOLOR4.&STEPNAME..FILTERDD) DISP(MOD,CATLG,DELETE) COPY LIST LISTA1 COPYDDN(CCDSN) FILTERDDN(FILTDSN) CONCURRENT

Example 10-15 Job output using FILTERDDN

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = COPY DSNU050I DSNUGUTC - OPTIONS KEY ACAZLHFZ DSNU1035I DSNUILDR - OPTIONS STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - LISTDEF LISTA1 INCLUDE TABLESPACE U7G01T11.TSPSUPP2 PARTLEVEL(1)

INCLUDE TABLESPACE U7G01T11.TSPSUPP PARTLEVEL(1) DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE CCDSN UNIT SYSDA DSN(PAOLOR4.&STEPNAME..&SN..P&PA..T&TIME.) DISP(MOD,CATLG,CATLG)DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - TEMPLATE FILTDSN UNIT SYSDA DSN(PAOLOR4.&STEPNAME..FILTERDD)

DISP(MOD,CATLG,DELETE) DSNU1035I DSNUJTDR - TEMPLATE STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - COPY LIST LISTA1 COPYDDN(CCDSN) FILTERDDN(FILTDSN) CONCURRENT DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=CCDSN DDNAME=SYS00001 DSN=PAOLOR4.UTIL.TSPSUPP.P00001.T183028 DSNU1038I DSNUGDYN - DATASET ALLOCATED. TEMPLATE=FILTDSN DDNAME=SYS00002 DSN=PAOLOR4.UTIL.FILTERDD DSNU421I DSNUBBCM - START OF DFSMS MESSAGES PAGE 0001 5695-DF175 DFSMSDSS V2R10.0 DATA SET SERVICES 2001.204 14:30 ADR030I (SCH)-PRIME(01), DCB VALUES HAVE BEEN MODIFIED FOR SYSPRINT PARALLEL ADR101I (R/I)-RI01 (01), TASKID 001 HAS BEEN ASSIGNED TO COMMAND 'PARALLEL' DUM OPT(3) DATAS(FILT(SYS00002)) - CAN CONC SHA TOL(ENQF) WAIT(0,0) - OUTDD(SYS00001) ADR101I (R/I)-RI01 (01), TASKID 002 HAS BEEN ASSIGNED TO COMMAND 'DUM ' ADR109I (R/I)-RI01 (01), 2001.204 14:30:33 INITIAL SCAN OF USER CONTROL STATEMENTS COMPLETED. ADR014I (SCH)-DSSU (02), 2001.204 14:30:33 ALL PREVIOUSLY SCHEDULED TASKS COMPLETED. PARALLEL MODE NOW IN EFFECT ADR050I (002)-PRIME(01), DFSMSDSS INVOKED VIA APPLICATION INTERFACE ADR016I (002)-PRIME(01), RACF LOGGING OPTION IN EFFECT FOR THIS TASK ADR006I (002)-STEND(01), 2001.204 14:30:33 EXECUTION BEGINS ADR411W (002)-DTDSC(04), DATA SET PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001 IN CATALOG UCAT.VSBOX01 ON VOLUME SBOX58 WAS NOT SERIALIZED ON REQUEST ADR411W (002)-DTDSC(04), DATA SET DB2V710G.DSNDBC.U7G01T11.TSPSUPP.I0001.A001 IN CATALOG UCAT.VSBOX09 ON VOLUME SBOX52 WAS NOT SERIALIZED ON REQUEST ADR801I (002)-DTDSC(01), DATA SET FILTERING IS COMPLETE. 2 OF 2 DATA SETS WERE SELECTED: 0 FAILED SERIALIZATION AND 0 FAILED FOR OTHER REASONS. ADR734I (002)-DTDSC(01), 2001.204 14:30:34 CONCURRENT COPY INITIALIZATION SUCCESSFUL FOR 2 OF 2 SELECTED DATA SETS.


SERIALIZATION FOR THIS DATA IS RELEASED IF DFSMSDSS HELD IT. THE INTERMEDIATE RETURN CODE IS 0004. ADR454I (002)-DTDSC(01), THE FOLLOWING DATA SETS WERE SUCCESSFULLY PROCESSED CLUSTER NAME PAOLOR1.DSNDBC.U7G01T11.TSPSUPP2.J0001.A001 CATALOG NAME UCAT.VSBOX01 COMPONENT NAME PAOLOR1.DSNDBD.U7G01T11.TSPSUPP2.J0001.A001 CLUSTER NAME DB2V710G.DSNDBC.U7G01T11.TSPSUPP.I0001.A001 CATALOG NAME UCAT.VSBOX09 COMPONENT NAME DB2V710G.DSNDBD.U7G01T11.TSPSUPP.I0001.A001 ADR006I (002)-STEND(02), 2001.204 14:30:41 EXECUTION ENDS ADR013I (002)-CLTSK(01), 2001.204 14:30:41 TASK COMPLETED WITH RETURN CODE 0004 ADR012I (SCH)-DSSU (01), 2001.204 14:30:41 DFSMSDSS PROCESSING COMPLETE. HIGHEST RETURN CODE IS 0004 FROM: TASK 002 DSNU422I DSNUBBCM - END OF DFSMS MESSAGE DSNU413I -DB2G DSNUBARI - CONCURRENT COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSPSUPP2

DSNUM 1 DSNU413I -DB2G DSNUBARI - CONCURRENT COPY SUCCESSFUL FOR TABLESPACE U7G01T11.TSPSUPP

DSNUM 1 DSNU401I DSNUBBCR - CONCURRENT COPY COMPLETE, ELAPSED TIME=00:00:08 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

When you use a data set that was produced with the FILTERDDN option (and contains the copy data of multiple data sets) in a RECOVER operation, DFDSS will still ask for the data set multiple times instead of restoring all table spaces in a single RESTORE operation.


Chapter 11. Gathering statistics

In this chapter we describe the use of RUNSTATS for gathering statistics on DB2 objects, and we also detail the ability to evaluate trend analysis by using the new History function.

We show:

� Why collect statistics?� When to use RUNSTATS� History tables� SAMPLE option� KEYCARD� FREQVAL� LOB table space� Performance� Inline execution with LOAD/REORG/REBUILD INDEX

11


11.1 Why collect statistics?Statistics about the data contained within DB2 objects are required for two main reasons:

� The first reason is to provide information about the space occupied by the data and the state of the underlying data sets. These figures can be used to:

– Determine the optimum sizing for the underlying data sets– Determine the correct settings for PCTFREE and FREEPAGE– Determine when to reorganize the data– Monitor growth for capacity planning– Monitor the effectiveness of the compression dictionary

� The second and main reason is to provide data that the optimizer can use to select the correct access path to the data for queries at either BIND or PREPARE time.

Both sets of statistics can also be used by DBAs to reevaluate physical database design

11.2 When to run RUNSTATS?RUNSTATS should be run whenever there has been a significant change in the makeup of the data within the table space. This enables the optimizer to accurately select the best access method for the query. We recommend collecting statistics after the following events:

� After a table is loaded.

� After an index is physically created.

� After a table space is reorganized.

� After you have run RECOVER TABLESPACE, REBUILD INDEX, or REORG INDEX.

� Before running REORG with the OFFPOSLIMIT, INDREFLIMIT, or LEAFDISTLIMIT options.

� After there have been extensive updates, deletions or insertions in a table space.

� To estimate pages changed since last RUNSTATS. You can see more details on 4.2.2, “When to run RUNSTATS” on page 79.

� After any other significant event affecting the data within the table.

Two utilities now have the ability to collect statistics inline as they are executing, for example, REORG and LOAD REPLACE. By collecting inline statistics, the total elapsed time of running the two utilities, for example REORG followed by RUNSTATS, is greatly reduced due to the elimination of the I/O required to perform RUNSTATS. This is done at the cost of a slight increase in CPU time. It is highly recommended that inline statistics be collected where a utility allows the option. For more details on inline statistics see 3.1.2, “Inline RUNSTATS” on page 46.

11.3 RUNSTATS optionsThis section deals with the options of the RUNSTATS utility.

11.3.1 LISTDEFLike most other utilities, RUNSTATS supports the use of the LISTDEF to pass a list of objects to a utility statement for further processing. The benefits of using LISTDEF are defined in Chapter 2, “Wildcarding and Templates” on page 17.


The list generated by the LISTDEF must only contain objects that can be used in the particular RUNSTATS command. Therefore the LISTDEF for RUNSTATS TABLESPACE must only contain table spaces and for RUNSTATS INDEX only indexes are allowed. If this is not the case the utility finishes with a return code of 8 and states that an object was invalid for the utility.

Example 11-1 illustrates the use of LISTDEF, while Example 11-2 shows the output.

Example 11-1 Example of LISTDEF

//SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 INCLUDE TABLESPACE DB246129.TS612902 RUNSTATS TABLESPACE LIST LISTA1

Example 11-2 RUNSTATS with LISTDEF job output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = STATS DSNU050I DSNUGUTC - LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 INCLUDE TABLESPACE DB246129.TS612902 DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - RUNSTATS TABLESPACE LIST LISTA1 DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DB246129.TS612901 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DB246129.TS612901 SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR7.CUST SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DB246129.TS612901 SUCCESSFUL DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-08-21.01.13.838652 DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DB246129.TS612902 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DB246129.TS612902 SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR PAOLOR7.ORD SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DB246129.TS612902 SUCCESSFUL DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-08-21.01.13.938889 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

11.3.2 UPDATEThe UPDATE option controls the scope of statistics that are collected by the RUNSTATS utility. The options are:

� ALL

– Indicates that all collected statistics are to be updated in the catalog

� ACCESSPATH

– Indicates that only statistics relevant to access paths are to be updated in the catalog

� SPACE

– Indicates that only statistics relevant to helping DBA’s monitor the status of objects are to be updated in the catalog

� NONE

– Indicates that no statistics are to be updated. This is valid only with REPORT.

It is important that statistics are collected regularly when using templates and dynamic data set sizing. If the statistics are not up-to-date, the wrong sizes will be allocated.

Chapter 11. Gathering statistics 255

11.3.3 HISTORYDB2 Version 7 introduces the ability to keep historical statistics relating to objects. By using the HISTORY option, DB2 will update the new history catalog tables; see 11.5, “New statistics collected” on page 265 for details of the new tables.

The options for HISTORY are the same as for UPDATE, and the scope of the UPDATE affects the options allowed for HISTORY. Table 11-1 illustrates the affect of UPDATE parameter on HISTORY.

Table 11-1 Allowable HISTORY/UPDATE combinations

By using the historical statistics, it is easy to use trend analysis to plan capacity, to identify if utilities are being run too often or not often enough, and to identify objects that may benefit from changes to its physical design. Figure 11-1 shows an example of monitoring space growth over time.

Figure 11-1 Monitoring space growth with RUNSTATS HISTORY

Assuming that the number of rows is constantly increasing so that the highest number is the latest, the query shows the percentage of rows added over specific time period. This could be extrapolated to provide an annual growth figure.

UPDATE HISTORY

ALL ALL, ACCESSPATH, SPACE, NONE

ACCESSPATH ACCESSPATH, NONE

SPACE SPACE, NONE

NONE NONE

Consider the following stats over time SYSTABLEPART_HIST

CARDF, SPACEFSYSINDEXPART_HIST

CARDF, SPACEFIf we can assume constant growth over time...

Monitoring space growth

SELECT MAX(CARDF), MIN(CARDF), Min and Max Rows ((MAX(CARDF)-MIN(CARDF))*100)/MIN(CARDF), % growth

(DAYS(MAX(STATSTIME))-DAYS(MIN(STATSTIME))) # days for growthFROM SYSIBM.SYSTABLEPART_HIST WHERE DBNAME='DB' AND TSNAME='TS';


Another example is the monitoring of the effectiveness of a compression dictionary. To determine how often a compression dictionary should be rebuilt, monitor the PAGESAVE value. Determine when the degradation of the dictionary from the original compression is greater than 10%. At this point, a dictionary rebuild should be scheduled by omitting the KEEPDICTIONARY keyword from the REORG; see Figure 11-2 for a rule of thumb to help decide when to rebuild a dictionary.

Figure 11-2 Using PAGESAVE to determine dictionary effectiveness

It is recommended that the effectiveness of the existing dictionary be monitored by using the column PAGESAVE in the new Version 7 catalog table SYSTABLEPART_HIST.

It is recommended that historical data be collected whenever RUNSTATS is run, as the overhead is insignificant.

11.3.4 KEYCARDKEYCARD indicates that RUNSTATS is to collect all of the distinct values in all of the 1 to n key column combinations for the specified indexes, where n is the number of columns in the index. For example, if KEYCARD is specified for a three column index (columns IXC1, IXC2, IXC3) then extra statistics are collected showing the combination of IXC1+IXC2. The values of IXC1+IXC2+IXC3 are collected in FULLKEYCARF and IXC1 statistics are held in FIRSTKEYCARF.

Example 11-3 shows KEYCARD being used with RUNSTATS, while Example 11-3 shows the output job.

Example 11-3 RUNSTATS using KEYCARD

//SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE U7G01T11.TSLINEI RUNSTATS TABLESPACE LIST LISTA1 TABLE(ALL) INDEX(ALL KEYCARD) REPORT YES

Example 11-4 RUNSTATS KEYCARD job output

DSNU616I DSNUSUCD SYSCOLDIST CATALOG STATISTICS FOR L_ORDERKEY,L_SHIPDATE CARDINALITY = 1.919843E+06 DSNU616I DSNUSUCD SYSCOLDIST CATALOG STATISTICS FOR L_ORDERKEY,L_SHIPDATE,L_RETURNFLAG CARDINALITY = 1.927767E+06 DSNU616I DSNUSUCD SYSCOLDIST CATALOG STATISTICS FOR L_ORDERKEY,L_SHIPDATE,L_RETURNFLAG,L_SUPPKEY CARDINALITY = 1.951546E+06 DSNU616I DSNUSUCD SYSCOLDIST CATALOG STATISTICS FOR L_ORDERKEY,L_SHIPDATE,L_RETURNFLAG,L_SUPPKEY,L_EXTENDEDPRICE CARDINALITY = 1.951548E+06

It is recommended that KEYCARD be used to collect index statistics when there is correlation in a multicolumn index keys and queries have less than fully matching equal predicates.

p a g e s a v e 7 3 % 7 1 % 6 9 % 6 7 % 6 4 %

i f ( m a x ( p a g e s a v e ) - m in ( p a g e s a v e ) ) * 1 0 0 /m a x ( p a g e s a v e ) > 1 0= (7 3 -6 4 ) * 1 0 0 /7 3= 1 2


11.3.5 FREQVALFREQVAL controls the collection of frequent value statistics. If you specify FREQVAL, the following options are mandatory:

� NUMCOLS

– Indicates the number of key columns to concatenate together when collecting frequent values from the specified index. Specifying “3“ means to collect frequent values on the concatenation of the first three key columns. The default is 1.

� COLS

– Indicates the number of frequent values to be collected. Specifying “15“ means collect 15 frequent values from the specified key columns. The default is 10.

Example 11-5 shows the FREQVAL syntax to collect the 5 most frequent values for the first two columns of the indexes on tables within table space U7G01T11.TSLINEI. A sample of the output is shown in Example 11-6.

Example 11-5 RUNSTATS using FREQVAL

//SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE U7G01T11.TSLINEI RUNSTATS TABLESPACE LIST LISTA1 TABLE(ALL) INDEX(ALL KEYCARD FREQVAL NUMCOLS 2 COUNT 5) REPORT YES

Example 11-6 RUNSTATS using FREQVAL sample output

DSNU616I DSNUSUCD SYSCOLDIST CATALOG STATISTICS FOR L_ORDERKEY,L_SHIPDATE FREQUENCY COLVALUE --------- -------- 2.0496549405907E-06 X'80176EA219950519' 1.537241205443E-06 X'8000006519960419' 1.537241205443E-06 X'800020C419920319' 1.537241205443E-06 X'8000298019950417' 1.537241205443E-06 X'8000612119920403' DSNU610I -DB2G DSNUSUCD - SYSCOLDIST CATALOG UPDATE FOR PAOLOR4.PXL#OKSDRFSKEPDC SUCCESSFUL

It is recommended to start with the default COUNT of 10 and adjust upward, as required. It is not recommended to use a high value for COUNT in an attempt to collect statistics for 100% of the data; this is not necessary and will increase CPU consumption.

If the frequent values are required for multiple columns, then the FREQVAL keyword should be repeated as shown in Figure 11-7.

Example 11-7 Multiple FREQVAL statements

//SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE U7G01T11.TSLINEI RUNSTATS TABLESPACE LIST LISTA1 TABLE(ALL) INDEX(ALL KEYCARD FREQVAL NUMCOLS 1 COUNT 5 FREQVAL NUMCOLS 2 COUNT 5 FREQVAL NUMCOLS 3 COUNT 5 FREQVAL NUMCOLS 4 COUNT 5 FREQVAL NUMCOLS 5 COUNT 5 ) REPORT YES


It is recommended that FREQVAL be used to collect frequent values statistics if there are literals within partially matching equal predicates. The same recommendation applies if there are input host variables and the query is reoptimized at run time.

11.3.6 SAMPLESAMPLE indicates the percentage of rows to sample when collecting column statistics. Any value from 1 through 100 can be specified. The default is 25, if SAMPLE is specified. If not specified, then sampling is not used, equivalent to SAMPLE 100.

When sampling, DB2 still reads all the pages because sampling is at the row level. This enhancement was introduced to minimize the vast amount of CPU required when hashing column values to estimate cardinality.

The default value of 25 provides a saving on CPU without compromising access path selection. In tests smaller sampling values were also used without noticing degradation of query performance.

Example 11-8, Example 11-9 and Example 11-10 shows the difference in cardinality when RUNSTATS are taken using SAMPLE 10, 25, and 100 respectively. It can be seen from the figures that the difference in the cardinality is marginal in all cases.

Example 11-8 RUNSTATS SAMPLE 10

Sel Name Owner T DB Name TS Name Cols Rows Checks * * * * * * * * ----- ------------------ -------- - -------- -------- ------ ----------- ------ * LINEITEM PAOLOR4 T U7G01T11 TSLINEI 16 1951548 0Columns of tableSelect Column Name Col No Col Type Length Scale Null Def FP Col Card * * * * * * * * * ------ ------------------ ------ -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 INTEGER 4 0 N N N 487775 L_PARTKEY 2 INTEGER 4 0 N N N 165747 L_SUPPKEY 3 INTEGER 4 0 N N N 10112 L_LINENUMBER 4 INTEGER 4 0 N N N 7 L_QUANTITY 5 INTEGER 4 0 N N N 50 L_EXTENDEDPRICE 6 FLOAT 4 0 N N N 934275 L_DISCOUNT 7 FLOAT 4 0 N N N 11 L_TAX 8 FLOAT 4 0 N N N 9 L_RETURNFLAG 9 CHAR 1 0 N N N 3 L_LINESTATUS 10 CHAR 1 0 N N N 2 L_SHIPDATE 11 DATE 4 0 N N N 2304 L_COMMITDATE 12 DATE 4 0 N N N 2240 L_RECEIPTDATE 13 DATE 4 0 N N N 2368 L_SHIPINSTRUCT 14 CHAR 25 0 N N N 4 L_SHIPMODE 15 CHAR 10 0 N N N 7 Columns of indexSel Column Name Seq No O Col Type Length Scale Null Def FP Col Card * * * * * * * * * *--- ------------------ ------ - -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 A INTEGER 4 0 N N N 487775 L_SHIPDATE 2 A DATE 4 0 N N N 2304 L_RETURNFLAG 3 A CHAR 1 0 N N N 3 L_SUPPKEY 4 A INTEGER 4 0 N N N 10112 L_EXTENDEDPRICE 5 A FLOAT 4 0 N N N 934275 L_DISCOUNT 6 A FLOAT 4 0 N N N 11



Select Column Name Col No Col Type Length Scale Null Def FP Col Card * * * * * * * * * ------ ------------------ ------ -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 INTEGER 4 0 N N N 487775 L_PARTKEY 2 INTEGER 4 0 N N N 208500 L_SUPPKEY 3 INTEGER 4 0 N N N 10112 L_LINENUMBER 4 INTEGER 4 0 N N N 7 L_QUANTITY 5 INTEGER 4 0 N N N 50 L_EXTENDEDPRICE 6 FLOAT 4 0 N N N 863540 L_DISCOUNT 7 FLOAT 4 0 N N N 11 L_TAX 8 FLOAT 4 0 N N N 9 L_RETURNFLAG 9 CHAR 1 0 N N N 3 L_LINESTATUS 10 CHAR 1 0 N N N 2 L_SHIPDATE 11 DATE 4 0 N N N 2304 L_COMMITDATE 12 DATE 4 0 N N N 2240 L_RECEIPTDATE 13 DATE 4 0 N N N 2400 L_SHIPINSTRUCT 14 CHAR 25 0 N N N 4 L_SHIPMODE 15 CHAR 10 0 N N N 7

Columns of index

Sel Column Name Seq No O Col Type Length Scale Null Def FP Col Card * * * * * * * * * * --- ------------------ ------ - -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 A INTEGER 4 0 N N N 487775 L_SHIPDATE 2 A DATE 4 0 N N N 2304 L_RETURNFLAG 3 A CHAR 1 0 N N N 3 L_SUPPKEY 4 A INTEGER 4 0 N N N 10112 L_EXTENDEDPRICE 5 A FLOAT 4 0 N N N 863540 L_DISCOUNT 6 A FLOAT 4 0 N N N 11


Select Column Name Col No Col Type Length Scale Null Def FP Col Card * * * * * * * * *------ ------------------ ------ -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 INTEGER 4 0 N N N 487775 L_PARTKEY 2 INTEGER 4 0 N N N 200704 L_SUPPKEY 3 INTEGER 4 0 N N N 10112 L_LINENUMBER 4 INTEGER 4 0 N N N 7 L_QUANTITY 5 INTEGER 4 0 N N N 50 L_EXTENDEDPRICE 6 FLOAT 4 0 N N N 835584 L_DISCOUNT 7 FLOAT 4 0 N N N 11 L_TAX 8 FLOAT 4 0 N N N 9 L_RETURNFLAG 9 CHAR 1 0 N N N 3 L_LINESTATUS 10 CHAR 1 0 N N N 2 L_SHIPDATE 11 DATE 4 0 N N N 2304 L_COMMITDATE 12 DATE 4 0 N N N 2240 L_RECEIPTDATE 13 DATE 4 0 N N N 2400 L_SHIPINSTRUCT 14 CHAR 25 0 N N N 4 L_SHIPMODE 15 CHAR 10 0 N N N 7


Columns of index

Sel Column Name Seq No O Col Type Length Scale Null Def FP Col Card * * * * * * * * * *--- ------------------ ------ - -------- ------ ------ ---- --- -- ----------- L_ORDERKEY 1 A INTEGER 4 0 N N N 487775 L_SHIPDATE 2 A DATE 4 0 N N N 2304 L_RETURNFLAG 3 A CHAR 1 0 N N N 3 L_SUPPKEY 4 A INTEGER 4 0 N N N 10112 L_EXTENDEDPRICE 5 A FLOAT 4 0 N N N 835584 L_DISCOUNT 6 A FLOAT 4 0 N N N 11

Table 11-2 shows a comparison of the performance of various levels of sampling.

Table 11-2 Sampling performance

It is recommended that SAMPLE 25 be used initially and testing undertaken to find the break even point that balances the savings in CPU against query performance..

11.3.7 FORCEROLLUPThis parameter controls the population of statistics for an empty partition of a partitioned table space. Prior to DB2 Version 7 if a partition was empty then the statistics reflected this fact. This could affect access path selection especially for packages bound prior to data being loaded into the partition. By using FORCEROLLUP, RUNSTATS will now populate the statistics for the partition with values reflecting the values in the other partitions. This resolves the access path selection issue and means that rebinding of packages need not be undertaken when empty partitions start being populated.

These are the options:

� YES indicates processing is to be done, even though some parts may not contain data.

� NO indicates processing will be done only if data is available.

It is recommended that FORCEROLLUP be used on table spaces with empty partitions.

RUNSTATS utility DELTA (%)

SAMPLE CPU Time

Elapsed Time

CPU Time

Elapsed Time

Total Getpages

10 16.38 22.59 50713

25 18.08 24.77 + 10.37 % + 9.65 % 50713

100 26.86 35.02 + 63.98 % + 55.06 % 50690

Note: All times are in secondsNote: The number of getpages are approximately equal

Note:

� If the sample keyword is not supplied, SAMPLE 100 is the RUNSTATS default.

� If the sample keyword is supplied, SAMPLE 25 is the default.

� DB2 extrapolates values, based on the percentage supplied by the SAMPLE parameter

� If 100% accuracy in required, do not use sampling.


11.3.8 REPORTThe REPORT option will write to SYSOUT the statistics that are to be placed in the catalog tables. If UPDATE NONE is specified, only the report is generated, and no catalog updates are executed. A sample of the report is shown in Example 11-11.

Example 11-11 RUNSTATS REPORT sample output

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = STATS DSNU050I DSNUGUTC - LISTDEF LISTA1 INCLUDE TABLESPACE DB246129.TS612901 DSNU1035I DSNUILDR - LISTDEF STATEMENT PROCESSED SUCCESSFULLY DSNU050I DSNUGUTC - RUNSTATS TABLESPACE LIST LISTA1 UPDATE NONE REPORT YES DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DB246129.TS612901 DSNU613I -DB2G DSNUSUTP - SYSTABLEPART CATALOG STATISTICS FOR DB246129.TS612901 PARTITION 0 CARD = 188 CARDF = 1.88E+02 NEARINDREF = 0 FARINDREF = 0 PERCACTIVE = 0 PERCDROP = 0 PAGESAVE = 0 SPACE = 12240 SPACEF = 1.224E+04 PQTY = 3000 SQTY = 300 DSNUM = 1 EXTENTS = 1 DSNU614I -DB2G DSNUSUTB - SYSTABLES CATALOG STATISTICS FOR PAOLOR7.CUST CARD = 188 CARDF = 1.88E+02 NPAGES = 94

NPAGESF = 9.4E+01 PCTPAGES = 26 PCTROWCOMP = 0 AVGROWLEN = 57 SPACEF = 1.1E+01 DSNU612I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG STATISTICS FOR DB246129.TS612901 NACTIVE = 360 NACTIVEF = 3.6E+02 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

11.4 RUNSTATS and LOB table spacesWhen using LISTDEFs to build lists for collecting statistics for LOB table spaces, the following options apply:

� ALL

– Specifies that both related BASE and LOB objects are to be included in the list. Auxiliary relationships will be followed from all objects resulting from the initial object lookup and both BASE and LOB objects will remain in the final enumerated list.

� BASE

– Specifies that only base table space (non-LOB) and index spaces are to be included in this element of the list. If the initial search for the object results in a base object, auxiliary relationship are not followed. If the initial search for the object results in a LOB object, the auxiliary relationship is applied to the base table space or index space, and only those objects become part of the resulting list.


� LOB

– Specifies that only LOB table spaces and related index spaces containing indexes on auxiliary tables are to be included in this element of the list. If the initial search for the object results in a LOB object, auxiliary relationships are not followed. If the initial search for the object results in a base object, the auxiliary relationship is applied to the LOB table space or index space, and only those objects become part of the resulting list.

If these options are not specified, then the statistics in the SYSLOBS and SYSLOBS_HIST will not updated. Example 11-12 shows the LISTDEF required to generate the correct lists for RUNSTATS to use when collecting statistics for a LOB table space. Specifying ALL automatically includes all related LOB objects, that is, BASE, AUX and LOB. Example 11-13 shows the statements required if not using LISTDEF, while Example 11-14 provides a sample output from the RUNSTATS utility.

Example 11-12 RUNSTATS with LISTDEF for LOB table spaces

//SYSIN DD * LISTDEF LISTA1 INCLUDE TABLESPACE DSN8D71L.DSN8S71B ALL RUNSTATS TABLESPACE LIST LISTA1 INDEX(ALL) REPORT YES UPDATE ALL HISTORY ALL

Example 11-13 RUNSTATS for LOB table spaces

//SYSIN DD * RUNSTATS TABLESPACE DSN8D71L.DSN8S71B INDEX (ALL) REPORT YES UPDATE SPACE HISTORY SPACE RUNSTATS TABLESPACE DSN8D71L.DSN8S71L INDEX (ALL) REPORT YES UPDATE ALL HISTORY ALL RUNSTATS TABLESPACE DSN8D71L.DSN8S71M INDEX (ALL) REPORT YES UPDATE ALL HISTORY ALL RUNSTATS TABLESPACE DSN8D71L.DSN8S71N INDEX (ALL) REPORT YES UPDATE ALL HISTORY ALL

Example 11-14 RUNSTATS output for LOB table spaces

DSNU050I DSNUGUTC - RUNSTATS TABLESPACE LIST LISTA1 INDEX(ALL) UPDATE ALL HISTORY ALL DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DSN8D71L.DSN8S71B DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DSN8D71L.DSN8S71B SUCCESSFUL DSNU610I -DB2G DSNUSUTB - SYSTABLES CATALOG UPDATE FOR DSN8710.EMP_PHOTO_RESUME SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DSN8D71L.DSN8S71B SUCCESSFUL DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR DSN8710.XEMP_PHOTO_RESUME SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR DSN8710.XEMP_PHOTO_RESUME SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR DSN8710.EMP_PHOTO_RESUME SUCCESSFUL DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-13-21.32.43.665184

DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DSN8D71L.DSN8S71L DSNU630I -DB2G DSNUSLOB - CATALOG STATISTICS FOR DSN8D71L.DSN8S71L AVGSIZE = 400517 FREESPACE = 248 ORGRATIO = 2.00 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DSN8D71L.DSN8S71L SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DSN8D71L.DSN8S71L SUCCESSFUL DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR DSN8710.XAUX_PSEG_PHOTO SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR DSN8710.XAUX_PSEG_PHOTO SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR DSN8710.AUX_PSEG_PHOTO SUCCESSFUL


DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-13-21.32.43.896458

DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DSN8D71L.DSN8S71M DSNU630I -DB2G DSNUSLOB - SYSLOBSTATS CATALOG STATISTICS FOR DSN8D71L.DSN8S71M AVGSIZE = 63117 FREESPACE = 140 ORGRATIO = 1.00 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DSN8D71L.DSN8S71M SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DSN8D71L.DSN8S71M SUCCESSFUL DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR DSN8710.XAUX_BMP_PHOTO SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR DSN8710.XAUX_BMP_PHOTO SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR DSN8710.AUX_BMP_PHOTO SUCCESSFUL DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-13-21.32.44.206266

DSNU1033I DSNUGULM - PROCESSING LIST ITEM: TABLESPACE DSN8D71L.DSN8S71N DSNU630I -DB2G DSNUSLOB - SYSLOBSTATS CATALOG STATISTICS FOR DSN8D71L.DSN8S71N AVGSIZE = 1210 FREESPACE = 140 ORGRATIO = 1.00 DSNU610I -DB2G DSNUSUTP - SYSTABLEPART CATALOG UPDATE FOR DSN8D71L.DSN8S71N SUCCESSFUL DSNU610I -DB2G DSNUSUTS - SYSTABLESPACE CATALOG UPDATE FOR DSN8D71L.DSN8S71N SUCCESSFUL DSNU610I -DB2G DSNUSUIP - SYSINDEXPART CATALOG UPDATE FOR DSN8710.XAUX_EMP_RESUME SUCCESSFUL DSNU610I -DB2G DSNUSUIX - SYSINDEXES CATALOG UPDATE FOR DSN8710.XAUX_EMP_RESUME SUCCESSFUL DSNU610I -DB2G DSNUSUCO - SYSCOLUMNS CATALOG UPDATE FOR DSN8710.AUX_EMP_RESUME SUCCESSFUL DSNU620I -DB2G DSNUSDRA - RUNSTATS CATALOG TIMESTAMP = 2001-06-13-21.32.44.376288

It is recommended that LISTDEF with option ALL be used for simplicity when building lists for LOB table spaces, because it automatically includes all objects related to the LOB.

Also, LOB columns statistics are not used for access path selection. For indexes on auxiliary tables, only the NLEVELS and FIRSTKEYCARDF columns in SYSIBM.SYSINDEXES have an effect on the access path.

Figure 11-3 shows details collected by RUNSTATS for LOB table spaces.

Figure 11-3 LOB management

LOB Management

SYSLOBSTATS CATALOG STATISTICS AVGSIZE = 4126 (average # bytes per lob) FREESPACE = 268 (kilobytes free space in extent) ORGRATIO = 1.50 (optimal physical location for active lobs) SYSTABLEPART Catalog StatisticsCARD = 32 (# lobs) CARDF = 3.2+01 (# lobs) SPACE = 288 (kilobytes of space allocated)SPACEF = 2.88E+02 (kb space; V7)PQTY = 24 (4K primary extent allocation; V5)SQTY = 48 (4K secondary extent allocations; V5)DSNUM = 1 (# datasets for tablespace; V7)EXTENTS = 2 (total primary & secondary extents; V7)


If using LOBs (Large Objects), the table space holding the actual LOB data is by managed statistics in SYSTABLEPART in the same way as shown for regular table spaces. However, there are also additional statistics saved in SYSLOBSTATS. These are:

� AVGSIZE

– This is the average size of all LOBs in the table space, that is, the total number of LOB bytes divided by the number of LOBs.

� FREESPACE

– This gives an indication of how many more LOBs can be added in the existing extents already allocated.

� ORGRATIO

– This is a measure of fragmentation or non-optimal organization of the LOBs stored in the table space. A value of 1.0 is optimal.

See 4.2.5, “LOB space management” on page 81 for more details.

11.5 New statistics collectedDB2 Version 7 has a new catalog table space called SYSHIST. This table space contains seven HISTORY tables:

� SYSCOLDIST_HIST � SYSCOLUMNS_HIST � SYSINDEXPART_HIST � SYSINDEXES_HIST � SYSLOBSTATS_HIST � SYSTABLEPART_HIST � SYSTABLES_HIST � SYSTABSTATS_HIST

History tables contain the statistics that were previously contained in their counterpart tables in SYSDBASE; see Figure 11-4. The HISTORY statistics tables are not identical, but do contain the same statistics columns. SYSDBASE contains only one row for each object/part which is updated when collecting statistics. For the history tables in SYSHIST, rows with statistics for each object/part are inserted when HISTORY is specified during RUNSTATS utility along with a STATSTIME value. The history statistics remain until the new MODIFY STATISTICS utility in Version 7 removes them; see 11.6, “Removing HISTORY statistics” on page 267.

Restriction:

� The TABLE option is invalid for LOB table space.

� SHRLEVEL REFERENCE or CHANGE, REPORT YES or NO, UPDATE ALL or NONE are the only options allowed or LOB table spaces.


Figure 11-4 SPACE Statistics tables

The following tables contain only SPACE statistics are:

� SYSTABLEPART_HIST� SYSINDEXPART_HIST� SYSLOBSTATS_HIST

These tables contain only ACCESSPATH statistics are:

� SYSCOLDIST_HIST� SYSCOLUMNS_HIST� SYSINDEXSTATS_HIST� SYSTABSTATS_HIST

And these tables contain both SPACE and ACCESSPATH related statistics:

� SYSTABLES_HIST� SYSINDEXES_HIST

These tables are populated by using the HISTORY option of the RUNSTATS utility; see 11.3.3, “HISTORY” on page 256.

Table 11-3 illustrates which tables are updated when HISTORY option is used.

Table 11-3 Statistic tables updated by HISTORY option

Space Statistics Tables

SYSLOBSTATS_HIST

Database DSNDB06 (Catalog)

SYSTABLEPART

SYSINDEXPART

SYSTABLES

SYSTABLEPART_HIST

SYSINDEXPART_HIST

SYSTABLES_HIST

SYSDBASE SYSHIST

SYSLOBSTATS

RUNSTATS db.ts INDEX(ALL) UPDATE SPACE HISTORY SPACE

SPACE ACCESSPATH ALL LOB table space

Table (ALL)

Index (ALL)

Table (ALL)

Index (ALL)

Table (ALL)

Index (ALL)

No param.

Index (ALL)

SYSCOLDIST_HIST X X

SYSCOLUMNS_HIST X X X X X

SYSINDEXES_HIST X X X X


There are also new fields added to the “current” statistics fields in DSNDB06 which can be used in determining when utilities are to be executed, these are shown in more detail in 4.2, “New values with DB2 V7” on page 74.

Two of the new fields, DSNUM and EXTENTS, are shown in the Example 11-15.

Example 11-15 New statistics SPACE and EXTENTS

TSNAME DBNAME SPACE SPACEF DSNUM EXTENTS * * * * * * -------- -------- ----------- ---------------------- ----------- ----------- TSPSUPP1 U7G01T11 72000 7.200000000000000E+04 1 7 TSPSUPP U7G01T11 72000 7.200000000000000E+04 1 7

A listing of the data sets underlying the table space shows the extents; see Example 11-16.

Example 11-16 Data set listing

Command - Enter "/" to select action Tracks %Used XT Device ------------------------------------------------------------------------------- DB2V710G.DSNDBD.U7G01T11.TSPSUPP.I0001.A003 1500 ? 7 3390 DB2V710G.DSNDBD.U7G01T11.TSPSUPP1.I0001.A003 1500 ? 7 3390

The SPACE field can also be shown to match the tracks listed. DB2 on a DASD MODEL 3390-3 allocates 12x4K pages per track, resulting in 48K per track. Therefore the relationship between SPACE(F) and tracks can be demonstrated as SPACE(F)/48 = TRACKS. In this example 72000/48 = 1500.

11.6 Removing HISTORY statisticsThe historical are never over written by the RUNSTATS utility, new records are always appended to the tables. To delete unwanted statistics history records from the History catalog tables the utility MODIFY STATISTICS has to be run.

SYSINDEXPART_HIST X X X

SYSINDEXSTATS_HIST X X

SYSTABLEPART_HIST X X X X X X

SYSTABSTATS_HIST X X X X

SYSTABLES_HIST X X X X X X X X

SYSLOBSTATS_HIST NP NP NP NP NP X X X

NP - Not Permitted

SPACE ACCESSPATH ALL LOB table space

Table (ALL)

Index (ALL)

Table (ALL)

Index (ALL)

Table (ALL)

Index (ALL)

No param.

Index (ALL)


Like MODIFY RECOVER, the statistics can be deleted using the following conditions:

� Statistics gathered before a specific date� Statistics of a specific age � All statistics history records related to the specified object from all catalog history tables.

The syntax for the MODIFY STATISTICS utility is given in Figure 11-5.

Figure 11-5 MODIFY STATISTICS Syntax

As shown in Figure 11-5, you can use LIST to specify the name of a previously-defined LISTDEF list name, or TABLESPACE, INDEXSPACE or INDEX can be explicitly coded.

There are three options to determine which statistics to delete. The delete options are:

� ALL

– Deletes all statistics history rows that are related to the specified object from all catalog history tables.

� ACCESSPATH

– Deletes all access-path statistics history rows that are related to the specified object from the following history tables:

SYSIBM.SYSCOLDIST_HISTSYSIBM.SYSCOLUMNS_HIST

� SPACE

– Deletes all space related statistics history rows related to the specified object from the following history tables:

SYSIBM.SYSINDEXPART_HISTSYSIBM.SYSTABLEPART_HISTSYSIBM.SYSLOBSTATS_HIST

Example 11-17 shows an example of deleting SPACE table space statistics based upon a date, while Example 11-18 shows the job output.


Example 11-17 MODIFY STATISTICS by date using SPACE

//DSNUPROC.SYSIN DD * MODIFY STATISTICS TABLESPACE U7G01T11.TSLINEI DELETE SPACE DATE(20010614)

Example 11-18 Output job for table space

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = TEMP DSNU050I DSNUGUTC - MODIFY STATISTICS TABLESPACE U7G01T11.TSLINEI DELETE SPACE DATE(20010614) DSNU1307I -DB2G DSNUMSTA - 6 SYSIBM.SYSTABLEPART_HIST ROWS WERE DELETED DSNU1300I -DB2G DSNUMSTA - MODIFY STATISTICS HAS COMPLETED SUCCESSFULLY DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

Table 11-4 shows the behavior of MODIFY STATISTICS and RUNSTATS SPACE

Table 11-4 Deleting statistics gathered by HISTORY SPACE

When running RUNSTATS TABLESPACE UPDATE SPACE HISTORY SPACE TABLE (ALL) (column 1), DB2 update tables SYSTABLEPART_HIST and SYSTABLES_HIST (column 1).

To delete those statistics, use MODIFY STATISTICS TABLESPACE dbname.tsname DELETE (ALL) (column 5), because if MODIFY STATISTICS TABLESPACE dbname.tsname DELETE SPACE (column 3) is used only SYSIBM.SYSTABLEPART is deleted, as shown in Example 11-18.

If running RUNSTATS TABLESPACE UPDATE SPACE HISTORY SPACE INDEX (ALL) (column 2), DB2 update tables SYSINDEXES_HIST, SYSIBM.SYSINDEXPART_HIST, SYSIBM.SYSTABLEPART_HIST and SYSTABLES_HIST (column 2).

Runstats

SPACE

MODIFYTable space

MODIFYIndex space

MODIFYTable space

MODIFYIndex space

Table (ALL)

Index (ALL)

Delete Space

DeleteSpace

Delete (ALL) Delete (ALL)

SYSCOLDIST_HIST

SYSCOLUMNS_HIST

SYSINDEXES_HIST X ND ERR

SYSINDEXPART_HIST X X X

SYSINDEXSTATS_HIST

SYSTABLEPART_HIST X X X ND X ND

SYSTABSTATS_HIST

SYSTABLES_HIST X X ND ND X ND

SYSLOBSTATS_HIST NP NP X NP X NP

ND - Not Deleted NP - Not Permitted

Note: Apply PTF UQ56653 for APAR PQ50666 to eliminate some inconsistencies in deleting RUNSTATS HISTORY entries related to indexes.


To delete those statistics, use MODIFY STATISTICS INDEXcreator.ixname DELETE (ALL) (column 5) or LISTDEF like Example 11-17, and MODIFY STATISTICS TABLESPACE dbname.tsname

In Table 11-5 you can see the behavior of MODIFY STATISTICS after RUNSTATS ACCESSPATH.

Table 11-5 Deleting statistics gathered by HISTORY ACCESSPATH

In Table 11-6 you can see the behavior of MODIFY STATISTICS after RUNSTATS ALL

Table 11-6 Deleting statistics gathered by HISTORY ALL

Runstats

ACCESSPATH

MODIFYTable space

MODIFYIndex space

MODIFYTable space

MODIFYIndex space

Table (ALL)

Index (ALL)

Delete Accesspath

DeleteAccesspath

Delete (ALL) Delete (ALL)

SYSCOLDIST_HIST X X X

SYSCOLUMNS_HIST X X X ND X X

SYSINDEXES_HIST X ND ERR

SYSINDEXPART_HIST

SYSINDEXSTATS_HIST X ND X

SYSTABLEPART_HIST

SYSTABSTATS_HIST X X ND ND X ND

SYSTABLES_HIST X X ND ND X ND

SYSLOBSTATS_HIST NP NP NP NP X NP


Runstats

ALL

MODIFYTable space

MODIFYIndex space

Table (ALL)

Index (ALL)

Delete ALL

DeleteALL

SYSCOLDIST_HIST X X

SYSCOLUMNS_HIST X X X X

SYSINDEXES_HIST X ERR

SYSINDEXPART_HIST X X

SYSINDEXSTATS_HIST X X

SYSTABLEPART_HIST X X X ND

SYSTABSTATS_HIST X X X ND

SYSTABLES_HIST X X X ND

SYSLOBSTATS_HIST X X X X



11.7 RUNSTATS performanceVarious RUNSTATS options were run to gather performance data on a partitioned table space with 7,900,000 rows. There were 3 partitions and no NPIs defined. Table 11-7 shows the comparison based using RUNSTATS SAMPLE 25 as a base figure.

Table 11-7 RUNSTATS performance

It is recommended to use the default sampling of 25 as a starting point. SAMPLE 25 can substantially reduced the CPU incurred while collecting sufficient statistics for the optimizer to select the best access path. This number can be reduced further if necessary, but access paths may experience degradation and should be monitored.

The HISTORY options in the RUNSTATS utility did not increase either CPU time or ELAPSED time significantly

The SAMPLE 100 is recommended when undertaking performance tuning and the cardinality of all non-index columns is required.

Runstats Utility Delta(%)

OPTIONS CPU Time

Elapsed Time CPU Time

Elapsed Time

Sample 25 72.20 87.69

Sample 25 + History Space

72.20 87.33 0 % -0.41 %

Sample 25 + History All

72.20 87.84 0 % +0.17 %

Sample 100 107.46 128.59 +48.83 % +46.64 %

Sample 100 + History Space

107.46 127.90 +48.83 % +45.85 %

Sample 100 + History All

107.50 129.47 +48.89 % +47.64 %

Sample 100 + History All + Keycard

109.23 131.23 +51.28 % +49.65 %

No Sample + No History

107.70 129.006 +49.16 % +47.11 %

Note: All times y are in seconds


Part 4 Appendixes

This part includes reference documentation distributed as follows:

� Appendix A, “Tables for Real-Time Statistics” on page 275:

Details on the tables that represent the data repository for the Real Time Statistics and the way that the execution of DB2 Utilities impacts their contents.

� Appendix B, “Sample JCL for copying Catalog and Directory objects” on page 287:

A sample job stream for use when securing the catalog and directory using the COPY utility. It illustrates a method of using LISTDEF and Template when processing the Catalog and Directory.

� Appendix C, “Creating a shadow catalog” on page 289:

Four jobs that can be used to define a shadow catalog, UNLOAD data from DSNDB06, LOAD into shadow catalog, and RUNSTATS using LISTDEF wildcard.

� Appendix D, “UNICODE support” on page 293:

Procedures to install and activate the 1140 (Euro English) to UNICODE 367 conversion for usage with the UNLOAD Utility.

� Appendix E, “Additional material” on page 297:

A description of how to download a job containing DDL to create and populate a shadow DB2 catalog.

Part 4


Appendix A. Tables for Real-Time Statistics

This appendix provides details on the tables that represent the data repository for Real-Time Statistics, and explains how the execution of DB2 Utilities impacts their contents.

Sample DDLUse the following SQL data definition statements to create the statistics database, table space, tables, and indexes. The names and declared attributes of the objects must not be changed. However, other attributes can be changed. For example, row locking can be specified. This sample DDL is also shipped in data set DSN710.SDSNSAMP, member name DSNTESS.

The amount of primary and secondary space to allocate can be calculated by using the formulas in the DB2 UDB for OS/390 and z/OS Version 7 Administration Guide, SC26-9931.

The approximate number of rows in the two statistics tables can be determined by the following SQL statement. The sample DDL used 20,000 objects as an estimate to determine the amount of space to request.

SELECT C1 + C2 FROM (SELECT COUNT(*) AS C1 FROM SYSIBM.SYSTABLEPART) AS T1,

(SELECT COUNT(*) AS C2 FROM SYSIBM.SYSINDEXPART) AS T2;CREATE DATABASE DSNRTSDB CCSID EBCDIC;CREATE TABLESPACE DSNRTSTS IN DSNRTSDBCCSID EBCDICCLOSE NOLOCKMAX 0SEGSIZE 32USING STOGROUP SYSDEFLTPRIQTY 1600SECQTY 160;CREATE TABLE SYSIBM.TABLESPACESTATS(DBNAME CHAR(8) NOT NULL,NAME CHAR(8) NOT NULL,PARTITION INTEGER NOT NULL,DBID SMALLINT NOT NULL,PSID SMALLINT NOT NULL,

A


UpdateStatsTime TIMESTAMP NOT NULL WITH DEFAULT,TotalRows FLOAT ,Nactive INTEGER ,Space INTEGER ,Extents SMALLINT ,LoadRLastTime TIMESTAMP ,ReorgLastTime TIMESTAMP ,ReorgInserts INTEGER ,ReorgDeletes INTEGER ,ReorgUpdates INTEGER ,ReorgUnClustIns INTEGER ,ReorgDisOrgLob INTEGER ,ReorgMassDelete INTEGER ,ReorgNearIndRef INTEGER ,ReorgFarIndRef INTEGER ,StatsLastTime TIMESTAMP ,StatsInserts INTEGER ,StatsDeletes INTEGER ,StatsUpdates INTEGER ,StatsMassDelete INTEGER ,CopyLastTime TIMESTAMP ,CopyUpdatedPages INTEGER ,CopyChanges INTEGER ,CopyUpdateLRSN CHAR(6) FOR BIT DATA,CopyUpdateTime TIMESTAMP )IN DSNRTSDB.DSNRTSTS CCSID EBCDIC;CREATE UNIQUE INDEX SYSIBM.TABLESPACESTATS_IXON SYSIBM.TABLESPACESTATS(DBID, PSID, PARTITION)CLUSTER CLOSE NO;CREATE TABLE SYSIBM.INDEXSPACESTATS(DBNAME CHAR(8) NOT NULL,INDEXSPACE CHAR(8) NOT NULL,PARTITION INTEGER NOT NULL,DBID SMALLINT NOT NULL,ISOBID SMALLINT NOT NULL,PSID SMALLINT NOT NULL,UpdateStatsTime TIMESTAMP NOT NULL WITH DEFAULT,TotalEntries FLOAT ,Nlevels SMALLINT ,Nactive INTEGER ,Space INTEGER ,Extents SMALLINT ,LoadRLastTime TIMESTAMP ,RebuildLastTime TIMESTAMP ,ReorgLastTime TIMESTAMP ,ReorgInserts INTEGER ,ReorgDeletes INTEGER ,ReorgAppendInsert INTEGER ,ReorgPseudoDeletes INTEGER ,ReorgMassDelete INTEGER ,ReorgLeafNear INTEGER ,ReorgLeafFar INTEGER ,ReorgNumLevels INTEGER ,StatsLastTime TIMESTAMP ,StatsInserts INTEGER ,StatsDeletes INTEGER ,StatsMassDelete INTEGER ,CopyLastTime TIMESTAMP ,CopyUpdatedPages INTEGER ,


CopyChanges INTEGER ,CopyUpdateLRSN CHAR(6) FOR BIT DATA,CopyUpdateTime TIMESTAMP )IN DSNRTSDB.DSNRTSTS CCSID EBCDIC;CREATE UNIQUE INDEX SYSIBM.INDEXSPACESTATS_IXON SYSIBM.INDEXSPACESTATS(DBID, ISOBID, PSID, PARTITION)CLUSTER CLOSE NO;

TABLESPACESTATS column definitionsTable A-1 Columns definition for TABLESPACESTATS

Column Content

DBNAME Name of the database.

NAME Name of the table space.

PARTITION Data set number within the table space. For partitioned table spaces, this value corresponds to the partition number for a single partition, or 0 for an entire non-partitioned table space.

DBID Internal identifier of the database.

PSID Internal identifier of the table space page set descriptor.

UPDATESTATSTIME The timestamp of when the row was inserted or last updated.

TOTALROWS Number of rows or LOBs in the table space or partition if the table space is partitioned. The total is the sum of rows in each table if the table space contains more than one table. NULL indicates that the value is unknown.

NACTIVE The number of active pages in the table space or partition. This value is equivalent to the number of preformatted pages. For multi-piece table spaces this is the sum of preformatted pages in all data sets. NULL indicates that the value is unknown.

SPACE The amount of space allocated to the table space or partition. The value is the total number of kilobytes allocated. For multi-piece linear page sets, this is the sum of space in all data sets. NULL indicates that the value is unknown.

EXTENTS The number of physical extents in the table space or partition. For multi-piece table spaces this is the number of extents of the last data set. NULL indicates that the value is unknown.

LOADRLASTTIME The timestamp of the last LOAD REPLACE on the table space or table space partition. NULL indicates that LOAD has never been run or the timestamp is unknown.

REORGLASTTIME The timestamp of the last REORG on the table space or table space partition. NULL indicates that REORG has never been run or the timestamp is unknown.

REORGINSERTS The number of records or LOBs inserted since the last REORG or LOAD REPLACE. NULL indicates that the value is unknown.

REORGDELETES The number of records or LOBs deleted since the last REORG or LOAD REPLACE. NULL indicates that the value is unknown.

REORGUPDATES The number of records updated since the last REORG or LOAD REPLACE. The number of LOB updates is accumulated in Reorganizers and ReorgDeletes since LOBs are deleted and inserted to accomplish a LOB update. NULL indicates that the value is unknown.

Appendix A. Tables for Real-Time Statistics 277

REORGDISORGLOB The number of LOBs inserted that were not considered perfectly chunked since the last REORG or LOAD REPLACE. A LOB is considered perfectly chunked if the pages allocated are contained within the minimum number of chunks possible. That is, pages allocated/chunk size = minimum chunks needed. NULL indicates that the value is unknown.

REORGUNCLUSTINS The number of records inserted that were not considered well clustered with respect to the clustering index since the last REORG or LOAD REPLACE. A record is considered well clustered if the page is inserted on a page within plus or minus 16 pages of the ideal candidate page. The ideal candidate page is determined by the clustering index. NULL indicates that the value is unknown.

REORGMASSDELETES The number of mass deletes from a segmented or LOB table space, or the number of dropped tables from a non-segmented table space, since the last REORG or LOAD REPLACE. NULL indicates that the value is unknown.

REORGNEARINDREF The number of overflow records created since the last REORG or LOAD REPLACE that were relocated “near” the pointer record. For nonsegregated table spaces, a page is considered "near" the present page if the two page numbers differ by 16 or less. For segmented table spaces, a page is considered "near" the present page if the two page numbers differ by (SEGSIZE * 2) or less. NULL indicates that the value is unknown.

REORGFARINDREF The number of overflow records created since the last REORG or LOAD REPLACE that were relocated “far” from the pointer record. For nonsegmented table spaces, a page is considered "far" from the present page if the two page numbers differ by 17 or more. For segmented table spaces, a page is considered "far" from the present page if the two page numbers differ by (SEGSIZE * 2) +1 or more. NULL indicates that the value is unknown.

STATSLASTTIME The timestamp of the last RUNSTATS on the table space or table space partition. NULL indicates that RUNSTATS has never been run or the timestamp is unknown.

STATSINSERTS The number of records or LOBs inserted since the last RUNSTATS. NULL indicates that the value is unknown.

STATSDELETES The number of records or LOBs deleted since the last RUNSTATS. NULL indicates that the value is unknown.

STATSUPDATES The number of records updated since the last RUNSTATS. The number of LOB updates is accumulated in ReorgInserts and ReorgDeletes since LOBs are deleted and inserted to accomplish a LOB update. NULL indicates that the value is unknown.

STATSMASSDELETES The number of mass deletes from a segmented or LOB table space, or the number of dropped tables from a non-segmented table space, since the last RUNSTATS. NULL indicates that the value is unknown.

COPYLASTTIME The timestamp of the last full image copy on the table space or table space partition. NULL indicates that COPY has never been run or the timestamp is unknown.

COPYUPDATEDPAGES The number of distinct pages updated since the last COPY. NULL indicates that the value is unknown.

COPYCHANGES The number of inserts, deletes and updates since the last COPY. This number approximates the number of log records to be processed to recover to currency. NULL indicates that the value is unknown.

COPYUPDATELRSN The LRSN or RBA of the first update after the last COPY. NULL indicates that the value is unknown.

Column Content


INDEXSPACESTATS column definitionsTable A-2 Columns definition for INDEXSPACESTATS

COPYUPDATETIME The timestamp of the first update after the last COPY. NULL indicates that the value is unknown.

Column Content

DBNAME Name of the database.

NAME Name of the index space.

PARTITION Data set number within the index space. For partitioned index spaces, this value corresponds to the partition number for a single partition, or 0 for an entire non-partitioned index space.

DBID Internal identifier of the database

ISOBID Internal identifier of the index space page set descriptor.

PSID Internal identifier of the table space page set descriptor in which the table, that the index is created on, exists.

UPDATESTATSTIME The timestamp of when the row was inserted or last updated.

TOTALENTRIES Number of index entries (including duplicates) in the index space or partition if the index is partitioned. NULL indicates that the value is unknown.

NLEVELS The number of levels in the index tree. NULL indicates that the value is unknown.

NACTIVE The number of active pages in the index space or partition. This value is equivalent to the number of preformatted pages. NULL indicates that the value is unknown.

SPACE The amount of space allocated to the index space or partition. The value is the total number of kilobytes allocated. NULL indicates that the value is unknown.

EXTENTS The number of physical extents in the index space or partition. For multi-piece index spaces this is the number of extents of the last data set. NULL indicates that the value is unknown.

LOADRLASTTIME The timestamp of the last LOAD REPLACE on the index space or partition. NULL indicates that LOAD has never been run or the timestamp is unknown.

REBUILDLASTTIME The timestamp of the last REBUILD INDEX on the index space or partition. NULL indicates that REBUILD INDEX has never been run or the timestamp is unknown.

REORGLASTTIME The timestamp of the last REORG on the index space or partition. NULL indicates that REORG has never been run or the timestamp is unknown.

REORGINSERTS The number of index entries inserted since the last REORG, REBUILD INDEX or LOAD REPLACE. NULL indicates that the value is unknown.

REORGDELETES The number of index entries deleted since the last REORG, REBUILD INDEX or LOAD REPLACE. NULL indicates that the value is unknown.

REORGAPPENDINSERT The number of index entries inserted since the last REORG, REBUILD INDEX or LOAD REPLACE that has a key value greater than a maximum key value in the index or partition of the index. NULL indicates that the value is unknown.

Column Content


Impact of utilitiesNext, we look at the values changed by the execution of DB2 utilities.

In the following sections, (1) and (2) have these meanings:

(1) This occurs if Real-Time Statistics are requested.(2) This occurs if an embedded image copy is requested.

REORGPSEUDODELETES The number of index entries that are currently pseudo deleted since the last REORG, REBUILD INDEX or LOAD REPLACE. NULL indicates that the value is unknown.

REORGMASSDELETES The number of times the index or index partition was mass deleted since the last REORG, REBUILD INDEX or LOAD REPLACE. NULL indicates that the value is unknown.

REORGLEAFNEAR Since the last REORG, REBUILD INDEX, or LOAD REPLACE, the number of index page splits that were near-off. A page is considered near-off the present page if the difference between the two page numbers is greater than 1 and less than 16. NULL indicates that the value is unknown.

REORGLEAFFAR Since the last REORG, REBUILD INDEX, or LOAD REPLACE, the number of index page splits that were far-off. A page is considered far-off the present page if the difference between the two page numbers is 16 or greater. NULL indicates that the value is unknown.

REORGNUMLEVELS The number of levels in the index tree added or removed since the last REORG, REBUILD INDEX, or LOAD REPLACE. NULL indicates that the value is unknown.

STATSLASTTIME The timestamp of the last RUNSTATS on the index space or index space partition. NULL indicates that RUNSTATS has never been run or the timestamp is unknown.

STATSINSERTS The number of index entries inserted since the last RUNSTATS. NULL indicates that the value is unknown.

STATSDELETES The number of index entries deleted since the last RUNSTATS. NULL indicates that the value is unknown.

STATSMASSDELETES The number of times the index or index space partition was mass deleted since the last RUNSTATS. NULL indicates that the value is unknown.

COPYLASTTIME The timestamp of the last full image copy on the index space or index space partition. NULL indicates that COPY has never been run or the timestamp is unknown. NULL indicates that the value is unknown.

COPYUPDATEDPAGES The number of distinct pages updated since the last COPY for COPY YES indexes. NULL indicates that the value is unknown.

COPYCHANGES The number of inserts and deletes since the last COPY for COPY YES indexes. This number approximates the number of log records to be processed to recover to currency. NULL indicates that the value is unknown.

COPYUPDATELRSN The LRSN or RBA of the first update after the last COPY for COPY YES indexes. NULL indicates that the value is unknown.

COPYUPDATETIME The timestamp of the first update after the last COPY for COPY YES indexes. NULL indicates that the value is unknown.

Column Content


Also, in the following sections, Reorg*, Stats*, and Copy* are a shorthand notation to describe the entire set of interval counters for the general category (Reorg, Stats, or Copy). The set does not include the timestamp column values.

LOAD REPLACE

TABLE SPACE OR TABLE SPACE PARTITION statistics After the object is reset to empty, and before the RELOAD phase, the following statistics are changed:

� LoadRLastTime is set to NULL.� TotalRows is set to 0.� Reorg* values are set to 0 (except ReorgUnClustIns) � ReorgUnClustIns is set to NULL.� Nactive, Space, and Extents are set to the actual values.� StatsLastTime is set to NULL (1).� Stats* values are set to 0 (1).� CopyLastTime is set to NULL (2).� Copy* values are set to 0 (2).

After the RELOAD phase, the following statistics are changed:

� LoadRLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace.

� TotalRows is set to the number of rows or LOBs loaded. Note that under some conditions the LOAD utility may not have an accurate count of loaded records (for example utility restart). In this case, and in cases where the total rows are unknown, the TotalRows value is set to NULL. TotalRows is the number of records loaded into the partition or table space and my be subsequently removed by the Index Validation phase or the Referential Integrity check. The removal of the records are counted as deletions.

� StatsLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace. (1)

� CopyLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace. (2)

INDEX OR INDEX PHYSICAL PARTITION statisticsAfter the object is reset to empty and before the BUILD phase, the following statistics are changed:

� LoadRLastTime is set to NULL.� TotalEntries is set to 0.� Reorg* values are set to 0.� Nlevels, Nactive, Space and Extents are set to the actual values.� StatsLastTime is set to NULL. (1)� Stats* values are set to 0. (1)� CopyLastTime is set to NULL. (2)� Copy* values are set to 0. (2)

After the BUILD phase, the following statistics are changed:

� LoadRLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace.


� TotalEntries is set to the number of index entries added. Note that, under some conditions, the LOAD utility may not have an accurate count of loaded entries (for example, utility restart). In this case, and in cases where the total entries are unknown, the TotalEntries value is set to NULL.

� StatsLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace. (1)

� CopyLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Load Replace. (2)

INDEX LOGICAL PARTITION statistics For Load Replace Part, the non-partitioning index (NPI) is not reset, therefore the NPIs statistics are not reset. The number of index inserts, deletes, and so on (REORG* statistics) are relevant after LOAD and are correct since the last reorg. The LoadRLastTime is not hanged until the entire NPI is specified and replaced.

LOAD RESUME YES, SHRLEVEL NONE AND SHRLEVEL CHANGE

TABLE SPACE OR TABLE SPACE PARTITION statisticsAfter the RELOAD phase, the following statistics are changed:

� TotalRows is set to TotalRows + (rows or LOBs) loaded during the RELOAD phase. Note that under some conditions the LOAD utility may not have an accurate count of loaded records (for example utility restart). In this case, and in cases where the total rows are unknown, the TotalRows value is set to NULL.

INDEX(ENTIRE) OR INDEX PHYSICAL PARTITION statistics After the BUILD phase, the following statistics are changed:

� TotalEntries is set to TotalEntries + index entries inserted during the BUILD phase.

INDEX LOGICAL PARTITION statisticsAfter the BUILD phase the following statistics are changed:

� TotalEntries is set to TotalEntries + index entries inserted during the BUILD phase.

REORG SHRLEVEL NONE

TABLE SPACE OR TABLE SPACE PARTITION statisticsAfter the object is reset to empty and before the RELOAD phase the following statistics are changed:

� ReorgLastTime is set to NULL.� TotalRows is set to 0.� Reorg* values are set to 0.� Nactive, Space and Extents are set to the actual values.� StatsLastTime is set to NULL. (1)� Stats* values are set to 0. (1)� CopyLastTime is set to NULL. (2)� Copy* values are set to 0. (2)

After the RELOAD phase, the following statistics are changed:

� ReorgLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the REORG.


� TotalRows is set to the number of rows or LOBs loaded. Note that under some conditions the REORG utility may not have an accurate count of loaded records (for example utility restart). In this case, and in cases where the total rows are unknown, the TotalRows value is set to NULL.

� StatsLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the REORG. (1)

� CopyLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the REORG. (2)

INDEX OR INDEX PHYSICAL PARTITION statistics After the object is reset to empty and before the BUILD phase the following statistics are changed:

� ReorgLastTime is set to NULL.� TotalEntries is set to 0.� Reorg* values are set to 0.� Nlevels, Nactive, Space and Extents are set to the actual values.� StatsLastTime is set to NULL. (1)� Stats* values are set to 0. (1)� CopyLastTime is set to NULL. (2)� Copy* values are set to 0. (2)



� TotalEntries is set to the number of index entries added. Note that under some conditions the REORG utility may not have an accurate count of loaded entries (for example utility restart). In this case, and in cases where the total entries are unknown, the TotalEntries value is set to NULL.

� StatsLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the REORG. (1)

� CopyLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the REORG. (2)

INDEX LOGICAL PARTITION statistics For Reorg Table Space Part, the non-partitioning index is not reset, therefore the NPIs statistics are not reset. The number of index inserts, deletes, and so on (REORG* statistics) are relevant after REORG and are correct since the last REORG. The ReorgLastTime is not changed until the entire NPI is specified and reorganized.

REORG SHRLEVEL REFERENCE OR CHANGE

TABLE SPACE OR TABLE SPACE PARTITION statistics After the SWITCH phase, the following statistics are changed:

� ReorgLastTime is set to the current timestamp.

� TotalRows is set to the number of rows loaded during the RELOAD phase, plus the net number of rows (inserted-deleted) added during LOG APPLY phase (SHRLEVEL CHANGE).

� Reorg* statistics accumulated during the LOG APPLY phase are externalized to the Real-Time Statistics tables.

� StatsLastTime is set to the current timestamp. (1)


� CopyLastTime is set to the current timestamp.

INDEX OR INDEX PHYSICAL PARTITION statistics After the SWITCH phase, the following statistics are changed:


� TotalEntries is set to the number of index entries added during the BUILD phase, plus the net number of index entries (inserted-deleted) added during LOG APPLY phase (SHRLEVEL CHANGE).

� Reorg* are accumulated during log apply and are updated in the Real-Time Statistics tables next time statistics are externalized for the object.

� StatsLastTime is set to the current timestamp. (1)

� CopyLastTime is set to the current timestamp.

INDEX LOGICAL PARTITION statistics For Reorg Table Space Part, the non-partitioning index is not reset, therefore the NPIs statistics are not reset. The number of index inserts, deletes, and so on (REORG* statistics) are relevant after REORG and are correct since the last reword. The ReorgLastTime is not changed until the entire NPI is specified and reorganized.

REBUILD INDEX

INDEX OR INDEX PHYSICAL PARTITION statistics After the object is reset to empty, and before the BUILD phase, the following statistics are changed:

� RebuildLastTime is set to NULL.� TotalEntries is set to 0.� Reorg* values are set to 0.� Nlevels, Nactive, Space, and Extents are set to the actual values.


� RebuildLastTime is set to the current timestamp. The timestamp may be different for every partition involved in the Rebuild.

� TotalEntries is set to the number of index entries added. Note that under some conditions the REBUILD utility may not have an accurate count of loaded entries (for example utility restart). In this case, and in cases where the total entries are unknown, the TotalEntries value is set to NULL.

INDEX LOGICAL PARTITION statisticsFor Rebuild of an index logical partition, the non-partitioning index is not reset, therefore the NPIs statistics are not reset. The number of index inserts, deletes, and so on (REORG* statistics) are relevant after Rebuild and are correct since the last REORG. The RebuildLastTime is not changed until the entire NPI is specified and rebuilt.


RUNSTATS UPDATE ALL

TABLE SPACE, INDEX AND PHYSICAL PARTITION statistics At the beginning of the RUNSTATS job, all DB2 data sharing members are notified that RUNSTATS is beginning. The in-memory statistics are externalized and reset. If notify fails to externalize all statistics then DB2 sets the StatsLastTime to NULL. If DB2 can't update StatsLastTime then a console message is produced. RUNSTATS will continue to run in either of the above cases. If notify is successful, the following statistics are changed:

� StatsLastTime is set to NULL.

� Stats* values are set to 0.

After the RUNSTATS phase, the following statistics are changed:

� StatsLastTime is set to the timestamp at the beginning of the RUNSTATS phase. The timestamp may be different for every partition involved in the RUNSTATS.

� Stats* are accumulated during RUNSTATS and are updated in the Real-Time Statistics tables the next time statistics are externalized for the object. Any changes that are accumulated are relative to the beginning of the RUNSTATS.

The StatsLastTime is only updated and the STATS* statistics are only reset when the utility was run with UPDATE ALL. If UPDATE NONE, SPACE or ACCESSPATH was specified, not all statistics are collected by RUNSTATS.

COPY AND MERGE

TABLE SPACE, INDEX AND PHYSICAL PARTITION statistics At the beginning of the Copy (full or incremental) or Merge all DB2 data sharing members are notified that the utility is beginning. The in- memory statistics are externalized and reset. If notify fails to externalize all statistics then DB2 sets the CopyLastTime to NULL. If DB2 can't update CopyLastTime then a console message is produced. Copy or Merge will continue to run in either of the above cases. If notify is successful, the following statistics are changed:

� CopyLastTime is set to NULL.� Copy* values are set to 0.

After the COPY or MERGE phase, the following statistics are changed:

� CopyLastTime is set to the timestamp at the beginning of the COPY or MERGE phase. The timestamp may be different for every partition involved in the Copy.

� Copy* are accumulated during Copy and are updated in the Real-Time Statistics tables the next time statistics are externalized for the object. Any changes that are accumulated are relative to the beginning of the Copy or Merge.

RECOVER TO CURRENCY

After recovery to currency, the in-memory counter fields are still valid. Nactive, Space and Extents column values are updated to reflect the new values.


RECOVER TO POINT-IN-TIME

After doing a recovery to a point-in-time, the statistics are potentially invalid. This is true for table space, index space, partition, or data set level recovery. For data set level recovery, the entire space row is affected. If the object is partitioned, only the corresponding partition row is affected. Therefore, the following statistics are set to NULL:

� Reorg*� Stats*� Copy*

Nlevels, Nactive, Space, and Extents column values are updated to reflect the new values.


Appendix B. Sample JCL for copying Catalog and Directory objects

This appendix provides a sample job stream for use when securing the Catalog and Directory using the COPY utility. It illustrates in Example B-1 a method of using LISTDEF and Template when processing the Catalog and Directory, where the use of Wildcards is not allowed. It has been updated to reflect PTF UQ80075 for APAR PQ74307, closed in October 16, 2003, which terminates the COPY utility job with CC=8 when DSNDB01.SYSUTILX is in the same SYSIN as other catalog objects.

Example: B-1 Copying the Catalog and Directory

//C713420X JOB (IVDB2001),'NAIDOO DB2',MSGCLASS=X,CLASS=Q, JOB01440// NOTIFY=C713420/*XEQ A02//*//TERM1 EXEC DB2CMD,SYSTEM=DB2H//SYSTSIN DD * DSN SYSTEM(DB2H) -TERM UTIL(RAMA*) END//*//COPY1 EXEC DSNUPROC,SYSTEM=DB2H,UID=RAMA1//SYSIN DD *OPTIONS EVENT ( ITEMERROR, SKIP )LISTDEF LIST1

INCLUDE TABLESPACE DSNDB01.DBD01 INCLUDE TABLESPACE DSNDB01.SPT01 INCLUDE TABLESPACE DSNDB01.SCT02 INCLUDE TABLESPACE DSNDB06.SYSDBASE INCLUDE TABLESPACE DSNDB06.SYSDBAUT INCLUDE TABLESPACE DSNDB06.SYSDDF INCLUDE TABLESPACE DSNDB06.SYSGPAUT INCLUDE TABLESPACE DSNDB06.SYSGROUP INCLUDE TABLESPACE DSNDB06.SYSGRTNS INCLUDE TABLESPACE DSNDB06.SYSHIST INCLUDE TABLESPACE DSNDB06.SYSJAUXA INCLUDE TABLESPACE DSNDB06.SYSJAUXB

B


INCLUDE TABLESPACE DSNDB06.SYSJAVA INCLUDE TABLESPACE DSNDB06.SYSOBJ INCLUDE TABLESPACE DSNDB06.SYSPKAGE INCLUDE TABLESPACE DSNDB06.SYSPLAN INCLUDE TABLESPACE DSNDB06.SYSSEQ INCLUDE TABLESPACE DSNDB06.SYSSEQ2 INCLUDE TABLESPACE DSNDB06.SYSSTATS INCLUDE TABLESPACE DSNDB06.SYSSTR INCLUDE TABLESPACE DSNDB06.SYSUSER INCLUDE TABLESPACE DSNDB06.SYSVIEWSTEMPLATE ICOPY

DSN(IVDB2GL.&DB..&TS..D&DT..T&TI.) DISP (NEW,CATLG,DELETE) UNIT SYSALLDA SPACE(30,30) CYLCOPY LIST LIST1 SHRLEVEL CHANGE PARALLEL 6

COPYDDN(ICOPY)//*//COPY2 EXEC DSNUPROC,SYSTEM=DB2H,UID=RAMA2//SYSIN DD *OPTIONS EVENT ( ITEMERROR, SKIP )TEMPLATE ICOPY

DSN(IVDB2GL.&DB..&TS..D&DT..T&TI.) DISP (NEW,CATLG,DELETE) UNIT SYSALLDA SPACE(15,15) CYLCOPY TABLESPACE DSNDB06.SYSCOPY

SHRLEVEL CHANGE COPYDDN(ICOPY)//*//COPY3 EXEC DSNUPROC,SYSTEM=DB2H,UID=RAMA3//SYSIN DD *OPTIONS EVENT ( ITEMERROR, SKIP )TEMPLATE ICOPY

DSN(IVDB2GL.&DB..&TS..D&DT..T&TI.) DISP (NEW,CATLG,DELETE) UNIT SYSALLDA SPACE(15,15) CYLCOPY TABLESPACE DSNDB01.SYSLGRNX

SHRLEVEL CHANGE COPYDDN(ICOPY)//*//COPY4 EXEC DSNUPROC,SYSTEM=DB2H,UID=RAMA4//SYSIN DD *OPTIONS EVENT ( ITEMERROR, SKIP )TEMPLATE ICOPY

DSN(IVDB2GL.&DB..&TS..D&DT..T&TI.) DISP (NEW,CATLG,DELETE) UNIT SYSALLDA SPACE(15,15) CYLCOPY TABLESPACE DSNDB01.SYSUTILX

SHRLEVEL CHANGE COPYDDN(ICOPY)//*//TERM2 EXEC DB2CMD,SYSTEM=DB2H,COND=EVEN//SYSTSIN DD * DSN SYSTEM(DB2H) -TERM UTIL(RAMA*) END//*


Appendix C. Creating a shadow catalog

This appendix contains four jobs that can be used to define a shadow catalog, unload data from DSNDB06 using UNLOAD utility, load into shadow catalog using LOAD utility, and run the RUNSTATS utility using the LISTDEF Wildcard.

The source for these jobs is available for download as described in Appendix E., “Additional material” on page 297.

DDL to define objects in shadow data baseThe DDL in Example C-1 was generated using the DB2 Administration Tool, GEN option. The generated DDL was edited to use STOGROUP instead of VCAT. The table definition’s DDL was modified to use the LIKE TABLE syntax to reduce the size the DDL data set. If you need to copy this DDL, please be aware of the following:

� The default PRIQTY and SECQTY for all the TS and IX values is 1 cylinder in size (3390 mod 3). You may need to change these sizes to suite your DB2 subsystem.

� You may omit the LOB table spaces SYSJAUXA and SYSJAUXB. These table spaces cannot be unloaded using the UNLOAD utility.

Example: C-1 DDL to define a shadow catalog in data base DSHADOW

//PAOLOR1D JOB (999,POK),REGION=5M,MSGCLASS=X,CLASS=A, // MSGLEVEL=(1,1),NOTIFY=&SYSUID //* //DB2BAT EXEC PGM=IKJEFT01,DYNAMNBR=20 //STEPLIB DD DSN=DB2V710G.RUNLIB.LOAD,DISP=SHR // DD DSN=DSN710.SDSNLOAD,DISP=SHR //SYSTSPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(DB2G) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP2) END //SYSIN DD DISP=SHR,DSN=PAOLOR1.TPCD.CNTL(DSHADOW)

C


Unload job to unload DSNDB06 with UNLOAD utilityExample C-2 contains the sample JCL to unload table spaces in DSNDB06. Please note that the table spaces in DSNDB06 must be explicitly defined in the INCLUDE statement of LISTDEF. Wildcard is not supported for the Catalog and Directory.

Example: C-2 Unload DSNDB06 using the UNLOAD utility

//PAOLOR1U JOB (999,POK),'BSDS PRINT',REGION=6072K, // CLASS=A,MSGCLASS=S, // NOTIFY=&SYSUID,TIME=1440 /*JOBPARM SYSAFF=SC63 //* //DELETE EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DELETE PAOLOR1.RAMA.DSNDB06.*.UNLOAD DELETE PAOLOR1.RAMA.DSNDB06.*.PUNCH SET MAXCC=0 //* //STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=UNLOAD, // UTSTATS='' //SYSIN DD * TEMPLATE ULDDDN DSN(PAOLOR1.RAMA.DSNDB06.&TS..UNLOAD) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) TEMPLATE PNHDDN DSN(PAOLOR1.RAMA.DSNDB06.&TS..PUNCH) UNIT(SYSDA) CYL DISP(NEW,CATLG,DELETE) VOLUMES(SBOX57,SBOX60) LISTDEF UNLIST INCLUDE TABLESPACE DSNDB06.SYSCOPY INCLUDE TABLESPACE DSNDB06.SYSDBASE INCLUDE TABLESPACE DSNDB06.SYSDBAUT INCLUDE TABLESPACE DSNDB06.SYSDDF INCLUDE TABLESPACE DSNDB06.SYSGPAUT INCLUDE TABLESPACE DSNDB06.SYSGROUP INCLUDE TABLESPACE DSNDB06.SYSGRTNS INCLUDE TABLESPACE DSNDB06.SYSHIST INCLUDE TABLESPACE DSNDB06.SYSOBJ INCLUDE TABLESPACE DSNDB06.SYSPKAGE INCLUDE TABLESPACE DSNDB06.SYSPLAN INCLUDE TABLESPACE DSNDB06.SYSSEQ INCLUDE TABLESPACE DSNDB06.SYSSEQ2 INCLUDE TABLESPACE DSNDB06.SYSSTATS INCLUDE TABLESPACE DSNDB06.SYSSTR INCLUDE TABLESPACE DSNDB06.SYSUSER INCLUDE TABLESPACE DSNDB06.SYSVIEWS UNLOAD LIST UNLIST PUNCHDDN PNHDDN UNLDDN ULDDDN NOPAD SHRLEVEL CHANGE ISOLATION UR

You will need to edit the SYSPUNCH output from the UNLOAD utility by replacing RESUME YES with RESUME NO REPLACE, if you plan to refresh your shadow data base frequently with fresh data from DSNDB06. Example C-3 contains the LOAD statements for the LOAD utility produced by the UNLOAD utility.


Example: C-3 Sample output of SYSPUNCH data set for SYSVIEWS

TEMPLATE U4681254 DSN(DB2V710G.RAMA.DSNDB06.&TS..UNLOAD) DISP(OLD,CATLG,CATLG) LOAD DATA INDDN U4681254 LOG NO RESUME NO REPLACE EBCDIC CCSID(00000,00000,00000) INTO TABLE "SHADOW "."SYSVTREE " WHEN(00001:00002 = X'001E') ( "NAME " POSITION( 00003) VARCHAR , "CREATOR " POSITION( *) CHAR(008) , "TOTLEN " POSITION( *) INTEGER , "IBMREQD " POSITION( *) CHAR(001) , "VTREE " POSITION( *) VARCHAR , "TYPE " POSITION( *) CHAR(001) ) INTO TABLE "SHADOW "."SYSVLTREE " WHEN(00001:00002 = X'001F') ( "IBMREQD " POSITION( 00003:00003) CHAR(001) , "VTREE " POSITION( 00004) VARCHAR ) INTO TABLE "SHADOW "."SYSVIEWS " WHEN(00001:00002 = X'0026') ( "NAME " POSITION( 00003) VARCHAR , "CREATOR " POSITION( *) CHAR(008) , "SEQNO " POSITION( *) SMALLINT , "CHECK " POSITION( *) CHAR(001) , "IBMREQD " POSITION( *) CHAR(001) , "TEXT " POSITION( *) VARCHAR , "PATHSCHEMAS " POSITION( *) VARCHAR , "RELCREATED " POSITION( *) CHAR(001) , "TYPE " POSITION( *) CHAR(001) ) INTO TABLE "SHADOW "."SYSVIEWDEP " WHEN(00001:00002 = X'0027') ( "BNAME " POSITION( 00003) VARCHAR , "BCREATOR " POSITION( *) CHAR(008) , "BTYPE " POSITION( *) CHAR(001) , "DNAME " POSITION( *) VARCHAR , "DCREATOR " POSITION( *) CHAR(008) , "IBMREQD " POSITION( *) CHAR(001) , "BSCHEMA " POSITION( *) CHAR(008) , "DTYPE " POSITION( *) CHAR(001) )

Tip: Save the SYSPUNCH data sets to avoid overwrite by UNLOAD utility. These data sets can be reused. You can also avoid unnecessary editing. Use a data set naming standard for the output data set from UNLOAD.

Appendix C. Creating a shadow catalog 291

Load data into shadow catalog using LOAD utilityThe LOAD JCL in Example C-4 was used to populate the table spaces in the shadow catalog.

Example: C-4 Load data from DSNDB06 using the LOAD utility

//PAOLOR1L JOB (999,POK),REGION=5M,MSGCLASS=X,CLASS=A, // MSGLEVEL=(1,1),NOTIFY=&SYSUID //* //LOAD EXEC DSNUPROC,SYSTEM=DB2G,UID='LOAD' //SYSUT1 DD DSN=PAOLOR1.TEMP,DISP=(NEW,DELETE), // UNIT=SYSDA,SPACE=(CYL,(100,100)) //SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(50,50)) //SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(50,50)) //SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(50,50)) //SORTWK04 DD UNIT=SYSDA,SPACE=(CYL,(50,50)) //SORTOUT DD UNIT=SYSDA,SPACE=(CYL,(100,100)) //SYSIN DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSDBASE.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSVIEWS.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSPLAN.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSSEQ.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSSEQ2.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSSTATS.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSSTR.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSUSER.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSCOPY.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSDBAUT.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSDDF.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSGPAUT.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSGROUP.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSGRTNS.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSHIST.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSOBJ.PUNCH // DD DISP=SHR,DSN=PAOLOR1.RAMA.DSNDB06.SYSPKAGE.PUNCH

RUNSTATS on shadow data baseThe JCL in Example C-5 was used to collect statistics on the shadow catalog to improve dynamic SQL.

Example: C-5 RUNSTATS on shadow catalog

//PAOLOR1R JOB (999,POK),'BSDS PRINT',REGION=6072K, // CLASS=A,MSGCLASS=S, // NOTIFY=&SYSUID,TIME=1440 /*JOBPARM SYSAFF=SC63 //* //STEP1 EXEC DSNUPROC,SYSTEM=DB2G,UID=PAOLOR1, // UTSTATS='' //SYSIN DD * LISTDEF DB01A INCLUDE TABLESPACE DSHADOW.* EXCLUDE TABLESPACE DSHADOW.SYSJAUXA EXCLUDE TABLESPACE DSHADOW.SYSJAUXB RUNSTATS TABLESPACE LIST DB01A TABLE INDEX(ALL) SAMPLE


Appendix D. UNICODE support

DB2 V7 supports UNICODE translation from the host code scheme to UNICODE 367. DB2 utilizes OS/390 services for the code translation. This appendix describes the procedures to install and activate the 1140 (Euro English) to UNICODE 367 conversion.

A full description of the OS/390 UNICODE support can be found at:

ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

Here we show only the CCSID 1140 to 367 conversion installation.

Generate the conversion tableThe JCL in Example D-1 creates the conversion table and output to DD SYSIMG. Please note that the ER option must be supplied to the CONVERSION command.

Example: D-1 JCL to create the conversion table

//CUNJIUTL JOB (POK,999),'UNICODE',MSGLEVEL=(1,1),MSGCLASS=T, // CLASS=A,NOTIFY=&SYSUID //******************************************************************* //* * //* LICENSED MATERIALS - PROPERTY OF IBM * //* * //* 5647-A01 * //* * //* (C) COPYRIGHT IBM CORP. 2000 * //* * //* STATUS = HUNI2A0 * //* * //******************************************************************* //* * //* IMAGE GENERATOR * //* CHANGED TO USE ER !!! * //******************************************************************* //CUNMIUTL EXEC PGM=CUNMIUTL //STEPLIB DD DSN=SYS1.LINKLIB,DISP=SHR //SYSPRINT DD SYSOUT=*

D


//TABIN DD DISP=SHR,DSN=SYS1.SCUNTBL //SYSIMG DD DSN=PAOLOR1.IMAGES(CUNIMG01),DISP=SHR //SYSIN DD * /******************************************** * INPUT STATEMENTS FOR THE IMAGE GENERATOR * ********************************************/ CASE NORMAL; /* ENABLE TOUPPER AND TOLOWER */ CONVERSION 1140,367,ER; /* EBCDIC EURO -> UNICODE */ CONVERSION 367,1140,ER; /* UNICODE -> EBCDIC EURO */ /*

The output of the job is shown in Example D-2. Please note that the job creates the conversion table indirectly, since no direct conversion from 1140 to 367 is currently available.

Example: D-2 Output from the generation job

CUN1000I OS/390 SUPPORT FOR UNICODE VERSION 2.8.0 CUN1001I PROCESSING STARTED ON 06/25/2001 AT 14:21:56 Source Listing ----+----1----+----2----+----3----+----4----+----5----+----6---- 1 2 /******************************************** 3 * INPUT STATEMENTS FOR THE IMAGE GENERATOR * 4 ********************************************/ 5 6 CASE NORMAL; /* ENABLE TOUPPER AND TOLOWER */ 7 8 CONVERSION 1140,367,ER; /* EBCDIC EURO -> UNICODE */ 9 CONVERSION 367,1140,ER; /* UNICODE -> EBCDIC EURO */ 10 Statement Report --+----1----+----2----+----3----+----4----+----5----+----6---- 1 CONVERSION 1140,367,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 01140-00367-ER */ /* 01140-01200-R using CUNRN5PH */ /* 01200-00367-X using CUNXPGB0 */ 2 CONVERSION 367,1140,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 00367-01140-ER */ /* 00367-01200-R using CUNRB0PG */ /* 01200-01140-E using CUNEPHN5 */ 3 CASE NORMAL; /* to-upper using CUNANUUP */ /* to-lower using CUNANULO */ CUN1014I INPUT READ 10 RECORDS CUN1015I STATEMENTS PROCESSED 3 CUN1016I STATEMENTS FLAGGED 0 CUN1017I GENERATED IMAGE SIZE 98 PAGES CUN1002I PROCESSING ENDED. HIGHEST RETURN CODE WAS 0


Activate the UNICODE tableThese are the steps to activate the UNICODE translation table:

� Copy the conversion table (the highlighted module in Example D-1 on page 293) into SYS1.PARMLIB as CUNIMG01.

� Create a new member in SYS1.PARMLIB, member name CUNUNI01. The contents of CUNUNI01 are shown in Example D-3.

Example: D-3 Contents of CUNUNI01 in SYS1.PARMLIB

/**********************************************************/ /* */ /* CUNUNIXX - UNICODE CONVERSION CONTROL PARAMETERS */ /* */ /**********************************************************/ /* ESTABLISH A NEW ENVIRONMENT */ /**********************************************************/ /* REQUIRED KEYWORD REALSTORAGE */ /* MAXIMAL USED PAGES OF REAL STORAGE, MIN=0 MAX=524287 */ /* WHERE 0 MEANS NO EXPLICITE LIMIT (=524287) */ /**********************************************************/ REALSTORAGE 51200; /* E.G. 200 MB */ /**********************************************************/ /* REQUIRED KEYWORD IMAGE WITH */ /* REQUIRED PARAMETER: MEMBER NAME */ /* THIS MEMBER MUST BE PLACED IN A DATA SET FROM THE */ /* LOGICAL PARMLIB CONCATENATION (DEF'D IN LOADXX) */ /**********************************************************/ IMAGE CUNIMG01;

� Activate the new UNICODE table with command T UNI=01. The SYSLOG output is reported in Example D-4.

Example: D-4 Activate the new UNICODE table

T UNI=01 IEE252I MEMBER CUNUNI01 FOUND IN SYS1.PARMLIB 537 CUN2036I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) WILL BE DELETED. ARE YOU SURE? (Y/N) R 537,Y IEE600I REPLY TO 537 IS;Y CUN2038I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) 282 WAS SUCCESSFULLY DELETED CUN2020I START LOADING CONVERSION IMAGE CUNIMG01 IEE252I MEMBER CUNIMG01 FOUND IN SYS1.PARMLIB CUN2022I LOADING CONVERSION IMAGE CUNIMG01 FINISHED: 401434 BYTES LOADED CUN2034I SET UNI COMMAND SUCCESSFULLY EXECUTED IEE536I UNI VALUE 01 NOW IN EFFECT

Appendix D. UNICODE support 295

� Verify the UNICODE table activation with command D UNI,ALL. The SYSLOG output is reported in Example D-5.

Example: D-5 UNI=ALL output in SYSLOG

D UNI,ALL IEF196I IEF237I 3D30 ALLOCATED TO SYS00108 IEF196I IEF285I SYS1.LINKLIB KEPT IEF196I IEF285I VOL SER NOS= Z01RE1. CUN3000I 14.26.27 UNI DISPLAY 303 ENVIRONMENT: CREATED 06/22/2001 AT 10.02.01 MODIFIED 06/25/2001 AT 14.24.25 IMAGE CREATED 06/25/2001 AT 14.21.56 SERVICE: CUNMCNV CUNMCASE STORAGE: ACTIVE 98 PAGES INACTIVE 98 PAGES SINCE 06/25/2001 AT 14.24.25 LIMIT 51200 PAGES CASECONV: NORMAL CONVERSION: 01140-00367-ER 00367-01140-ER


Appendix E. Additional material

This redbook refers to additional material that can be downloaded from the Internet as described below.

Locating the Web materialThe Web material associated with this redbook is available in softcopy on the Internet from the IBM Redbooks Web server. Point your Web browser to:

ftp://www.redbooks.ibm.com/redbooks/SG246289

Alternatively, you can go to the IBM Redbooks Web site at:

ibm.com/redbooks

Select the Additional materials and open the directory that corresponds with the redbook form number, SG246289.

Using the Web materialThe additional Web material that accompanies this redbook includes the following file:

File name Descriptionshadddl.zip DDL to create and populate a shadow DB2 catalog

System requirements for downloading the Web materialThe following system configuration is recommended:

Hard disk space: 8 MB minimumOperating System: Windows 95 or NT or 2000Processor: Intel 386 or higherMemory: 16 MB

E


ftp://www.redbooks.ibm.com/redbooks/


http://www.ibm.com/redbooks/


How to use the Web materialCreate a subdirectory (folder) on your workstation, and unzip the contents of the Web material zip file into this folder. Upload the file to your host S/390 system into a fixed block LRECL=80 sequential data set.


Special notices

References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's product, program, or service may be used. Any functionally equivalent program that does not infringe any of IBM's intellectual property rights may be used instead of the IBM product, program or service.

Information in this book was developed in conjunction with use of the equipment specified, and is limited in application to those specific hardware and software products and levels.

IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785.

Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact IBM Corporation, Dept. 600A, Mail Drop 1329, Somers, NY 10589 USA.

Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee.

The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk.

Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites.

The following terms are trademarks of other companies:

Tivoli, Manage. Anything. Anywhere.,The Power To Manage., Anything. Anywhere.,TME, NetView, Cross-Site, Tivoli Ready, Tivoli Certified, Planet Tivoli, and Tivoli Enterprise are trademarks or registered trademarks of Tivoli Systems Inc., an IBM company, in the United States, other countries, or both. In Denmark, Tivoli is a trademark licensed from Kjøbenhavns Sommer - Tivoli A/S.

C-bus is a trademark of Corollary, Inc. in the United States and/or other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and/or other countries.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries.


PC Direct is a trademark of Ziff Communications Company in the United States and/or other countries and is used by IBM Corporation under license.

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States and/or other countries.

UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.


Related publications

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM RedbooksFor information on ordering these publications, see “How to get IBM Redbooks” on page 302.

� DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-5129

� DB2 UDB Server for OS/390 and z/OS Version 7 Presentation Guide, SG24-6121

� DB2 UDB Server for OS/390 Version 6 Technical Update, SG24-6108

� DB2 Java Stored Procedures - Learning by Example, SG24-5945

� DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351

� DB2 for OS/390 Version 5 Performance Topics, SG24-2213

� DB2 for MVS/ESA Version 4 Non-Data-Sharing Performance Topics, SG24-4562

� DB2 UDB for OS/390 Version 6 Management Tools Package, SG24-5759

� DB2 Server for OS/390 Version 5 Recent Enhancements - Reference Guide, SG24-5421

� DB2 for OS/390 Capacity Planning, SG24-2244

� Developing Cross-Platform DB2 Stored Procedures: SQL Procedures and the DB2 Stored Procedure Builder, SG24-5485

� DB2 for OS/390 and Continuous Availability, SG24-5486

� Parallel Sysplex Configuration: Cookbook, SG24-2076-00

� DB2 for OS390 Application Design for High Performance, GG24-2233

� Using RVA and SnapShot for Business Intelligence Applications with OS/390 and DB2, SG24-5333

� IBM Enterprise Storage Server Performance Monitoring and Tuning Guide, SG24-5656

� DFSMS Release 10 Technical Update, SG24-6120

� Storage Management with DB2 for OS/390, SG24-5462

� Implementing ESS Copy Services on S/390, SG24-5680

Other resourcesThese publications are also relevant as further information sources:

� DB2 UDB for OS/390 and z/OS Version 7 What’s New, GC26-9946

� DB2 UDB for OS/390 and z/OS Version 7 Installation Guide, GC26-9936

� DB2 UDB for OS/390 and z/OS Version 7 Command Reference, SC26-9934

� DB2 UDB for OS/390 and z/OS Version 7 Messages and Codes, GC26-9940

� DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945

� DB2 UDB for OS/390 and z/OS Version 7 Programming Guide and Reference for Java, SC26-9932


� DB2 UDB for OS/390 and z/OS Version 7 Administration Guide, SC26-9931

� DB2 UDB for OS/390 and z/OS Version 7 Application Programming and SQL Guide, SC26-9933

� DB2 UDB for OS/390 and z/OS Version 7 Release Planning Guide, SC26-9943

� DB2 UDB for OS/390 and z/OS Version 7 SQL Reference, SC26-9944

� DB2 UDB for OS/390 and z/OS Version 7 Text Extender Administration and Programming, SC26-9948

� DB2 UDB for OS/390 and z/OS Version 7 Data Sharing: Planning and Administration, SC26-9935

� DB2 UDB for OS/390 and z/OS Version 7 Image, Audio, and Video Extenders, SC26-9947

� DB2 UDB for OS/390 and z/OS Version 7 ODBC Guide and Reference, SC26-9941

� DB2 UDB for OS/390 and z/OS Version 7 XML Extender Administration and Reference, SC26-9949

� OS/390 V2R10.0 DFSMS Using Data Sets, SC26-7339

� DB2 UDB for OS/390 Storage Management, IDUG Solutions Journal, Spring 2000, Volume 7, Number 1, by John Campbell and Mary Petras

� DB2 Administration Tools for OS/390 User’s Guide, Version 2 Release 1, SC27-0974-01

� DB2 Automation Tool for z/OS, V1R1, User's Guide, SC27-1189-00

� DB2 Change Accumulation Tool for z/OS V1R1 User's Guide, SC27-1191-00

Referenced Web sitesThese Web sites are also relevant as further information sources:

� http://ibm.com/software/data/db2/os390/ DB2 for z/OS and OS/390

� http://ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html UNICODE

� http://ibm.com/software/data/db2imstools/ DB2 for z/OS and OS/390 tools

How to get IBM RedbooksSearch for additional Redbooks or redpieces, view, download, or order hardcopy from the Redbooks Web site:

ibm.com/redbooks

Also download additional materials (code samples or diskette/CD-ROM images) from this Redbooks site.

Redpieces are Redbooks in progress; not all Redbooks become redpieces and sometimes just a few chapters will be published this way. The intent is to get the information out much quicker than the formal publishing process allows.

IBM Redbooks collectionsRedbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.





ronyms

AIX Advanced Interactive eXecutive from IBM

APAR authorized program analysis report

ARM automatic restart manager

ASCII American National Standard Code for Information Interchange

BLOB binary large objects

CCSID coded character set identifier

CCA client configuration assistant

CFCC coupling facility control code

CTT created temporary table

CEC central electronics complex

CD compact disk

CF coupling facility

CFRM coupling facility resource management

CLI call level interface

CLP command line processor

CPU central processing unit

CSA common storage area

DASD direct access storage device

DB2 PM DB2 performance monitor

DBAT database access thread

DBD database descriptor

DBID database identifier

DBRM database request module

DSC dynamic statement cache, local or global

DCL data control language

DDCS distributed database connection services

DDF distributed data facility

DDL data definition language

DLL dynamic load library manipulation language

DML data manipulation language

DNS domain name server

DRDA distributed relational database architecture

DTT declared temporary tables

EA extended addressability

Abbreviations and ac

© Copyright IBM Corp. 2001

EBCDIC extended binary coded decimal interchange code

ECS enhanced catalog sharing

ECSA extended common storage area

EDM environment descriptor management

ERP enterprise resource planning

ESA Enterprise Systems Architecture

ETR external throughput rate, an elapsed time measure, focuses on system capacity

FDT functional track directory

FTP File Transfer Program

GB gigabyte (1,073,741,824 bytes)

GBP group buffer pool

GRS global resource serialization

GUI graphical user interface

HPJ high performance Java

IBM International Business Machines Corporation

ICF integrated catalog facility

ICF integrated coupling facility

ICMF internal coupling migration facility

IFCID instrumentation facility component identifier

IFI instrumentation facility interface

IPLA IBM Program Licence Agreement

IRLM internal resource lock manager

ISPF interactive system productivity facility

ISV independent software vendor

I/O input/output

ITR internal throughput rate, a processor time measure, focuses on processor capacity

ITSO International Technical Support Organization

IVP installation verification process

JDBC Java Database Connectivity

JFS journaled file systems

JVM Java Virtual Machine

KB kilobyte (1,024 bytes)

LOB large object

303

LPL logical page list

LPAR logically partitioned mode

LRECL logical record length

LRSN log record sequence number

LVM logical volume manager

MB megabyte (1,048,576 bytes)

NPI non partitioning index

ODB object descriptor in DBD

ODBC Open Data Base Connectivity

OS/390 Operating System/390

PAV parallel access volume

PDS partitioned data set

PIB parallel index build

PSID pageset identifier

PSP preventive service planning

PTF program temporary fix

PUNC possibly uncommitted

QMF Query Management Facility

RACF Resource Access Control Facility

RBA relative byte address

RECFM record format

RID record identifier

ROT rule of thumb

RRS resource recovery services

RRSAF resource recovery services attach facility

RR repeatable read

RS read stability

RTS real time statistics

SDK software developers kit

SMIT System Management Interface Tool


Index

Numerics1140 20437 2045655-E62 135655-E63 135697-E98 13

AACCESSPATH 266ASCII 205AUTHID 75AVGROWLEN 75

BBUILD2 12, 175

CCARDF 71, 84CCSID 204CHANGELIMIT 70, 73, 238CHECK 226CHECK DATA 228

phases 228CHECK INDEX 226, 233CHECK LOB 229

phases 229check page 240CHECKP 150compression dictionary 84Concurrent Copy 246CONTINUE 180Control Center 89COPY 73, 232

CHECKPAGE 240copying to disk 242copying to tape 242FULL 241INCREMENTAL 241Lists 235parallelism 235SHRLEVEL CHANGE 242SHRLEVEL REFERENCE 242

COPY PARALLEL 49COPY YES 234COPYDDN 110copying Catalog and Directory objects 287copying indexes 232

recommendations 234COPYP 110COPYPAGESF 74COPYTOCOPY 12, 243

FROMCOPY 245

© Copyright IBM Corp. 2001

FROMLASTCOPY 245FROMLASTFULLCOPY 245FROMLASTINCRCOPY 245measurements 246

COPYTOCOPY OPTIONS 244core utilities 12CPAGESF 75Cross Loader 12, 129

binding the package 138declaring a cursor 137loading a partitioned table space 146loading with the cursor 138

DD UNI,ALL 296data set extents 80data set sizing 28DB2 Administration Tool 94DB2 Automation Tool 96DB2 Change Accumulation Tool 224DEADLINE 181degree of parallelism 118DELAY 180DFSMS 102DFSMSdss 248DFSORT 183DISCARD 156, 185, 191DISPLAY UTIL 237DISPLAY UTILITY command 181DRAIN 177DRAIN_WAIT 177DSN1CHKR 8DSN1COMP 8DSN1COPY 8, 193, 234DSN1LOGP 8DSN1PRNT 9DSN1SDMP 9DSNACCOR 88DSNJLOGF 7DSNJU003 7DSNJU004 8DSNRTSDB 86DSNRTSTS 86DSNTESS 86DSNTIPB 104DSNTIPL 217DSNU070I 138DSNU331I 138DSNU364I 63DSNU374I 181DSNU397I 63, 235DSNU427I 235DSNUM 75DSNUM, 75

305

DSNUTILS 89DSSIZE 11, 102dynamic utility jobs 11

EEBCDIC 204ENDEXEC 130ENFORCE CONSTRAINTS 149ER option 293ESS 80EVENT 25EXCLUDE 19EXEC phase 132EXEC SQL 130

commit scope 133EXTENTS 75externalizing in-memory statistics 87

FFARINDREF 72FAROFFPOS 71FASTSWITCH 12, 102, 172FREEPAGE 80

Hhistorical statistics 84HISTORY ALL 270HISTORY SPACE 77HISTORY tables 265HPGRBRBA 216

IICE046A 154IGNOREFIELDS YES 140INCLUDE 19INCURSOR 130Index recovery from Copy 219INDEXSPACESTATS 279inline copy 232INSERT processing 149ITEMERROR 32

JJOBNAME 75

KKEEPDICTIONARY 183

LLARGE 102LEAFDIST 70LEAFDISTLIMIT 70LEAFFAR 71, 75LEAFNEAR 71, 75LIMIT option 203LISTDEF 18

LISTDEF expansion 20LOAD 109LOAD and Inline Copy 110LOAD and Inline RUNSTATS 110LOAD partition parallelism 11LOBs AVGSIZE 82LOBs ORGRATIO 82Log Apply 216LOG NO 110Log records sorting 218LOGONLY 213LONGLOG 180

MMAXERR 197MAXRO 179Modify 221MODIFY RECOVER 268MODIFY STATISTICS 12, 267

NNEARINDREF 72NEAROFFPOS 71Non-partitioning Indexes 106NOSYSREC 182Note 271NPAGESF 74, 75NPI 106

OOFFPOS 150OFFPOSLIMIT 71Online LOAD RESUME 12

Restart 151Online LOAD Resume

performance 151online LOAD Resume 148

examples 152Online REORG

phases 167planning 166

Online REORG enhancements 12online utilities 4OPTION PREVIEW 24OPTIONS 35

PPAGESAVE 84Parallel Index Build 111Partition 123Partition parallelism with Parallel Index Build 123partitioning 102

impact on SQL 104impact on utilities 105reasons for 102

partitionsaltering 103

partitions and parallelism 105


PARTKEYU 104PCTFREE 80PERCDROP 81PIB 111pieces 11PIECESIZE 106PQ19897 187PQ42864 82PQ45184 221PQ45268 129PQ45835 30, 242PQ46759 129PQ46859 85PQ48447 85PQ48448 85PQ49114 188PQ50223 30, 207PQ50666 269PREFORMAT 127, 184PREVIEW 24, 35PRIQTY 80PSEUDO_DEL_ENTRIES 75

QQUIESCE 225

RReal Time Statistics 85

impact of utilities 280tables 275

REBUILD INDEX 232RECOVER

from CONCURRENT COPY 215without an image copy 215

RECOVER PARALLEL 49, 211restrictions 212

RECOVERYDDN 110redbook contents 4Redbooks Web site 302

Contact us xixREORG 161

BUILD2 175DEADLINE 181DISCARD 185DRAIN 176FASTSWITCH 172KEEPDICTIONARY 183LISTS 181LOG phase 171LONGLOG 180MAXRO 179NOSYSREC 182PREFORMAT 184REUSE 184SHRLEVEL CHANGE 164SHRLEVEL REFERENCE 164SORTDATA 182SORTKEYS 170, 182SWITCH 171

TIMEOUT 180UNLOAD EXTERNAL 187

REORG and Inline COPY 189REORG and Inline RUNSTATS 189REORG of index 188REORG of the DB2 Catalog 188REORG UNLOAD EXTERNA 193REORGP 103REPORT RECOVERY 222REPORTONLY 74, 162restartability 32restricted status 32RESUME YES 148, 149RETRY 177RETRY_DELAY 177REUSE 80, 128, 184RORG

SHRLEVEL NONE 163RUNSTATS 77, 253RUNSTATS and LOBs 262RUNSTATS option

FORCEROLLUP 261FREQVAL 258HISTORY 256KEYCARD 257LISTDEF 254REPORT 262SAMPLE 259UPDATE 255

RUNSTATS performance 271RUNSTATS statistics history 12RVA 80

SSECQTY 80shadow data sets 166SORTBLD 112, 170SORTDATA 182SORTKEYS 111, 170, 182SPACE 266space growth 84SPACEF 75, 84STACK 29stand-alone utilities 7star join 145STATISTICS option 110STATSINT 86STYPE 246SYSCOLDIST_HIST 265SYSCOLUMNS_HIST 265SYSCOPY 74SYSDBASE 76SYSHIST 76SYSIBM.SYSINDEXSPACESTATS 86SYSIBM.SYSTABLESPACESTATS 86SYSINDEXES 75SYSINDEXES_HIST 265SYSINDEXPART 75, 80SYSINDEXPART_HIST 265SYSLOBSTATS 81

Index 307

SYSLOBSTATS_HIST 265SYSSTATS 76SYSTABLEPART 80SYSTABLEPART_HIST 265SYSTABLES 75SYSTABLES_HIST 265SYSTABSTATS_HIST 265

TTABLESPACESTATS 277Templates 25

benefits 25how to use 26naming standards 27recommendations 29restrictions 30

Templates and partitions 34Templates and Wildcards 30

UUDT 141UNICODE 367 293UNICODE support 293UNLOAD 12, 191

converting data 204FROM TABLE 196FROMCOPY 196output data sets 192phases 192restarting 207SAMPLE option 202WHEN option 202

UNLOAD and image copy 194UNLOAD and SHRLEVEL 202UNLOAD EXTERNAL 187unloading compressed table space 197unloading from a table space 198unloading in parallel by partition 200UQ54731 188UQ55541 129UQ55542 129UQ56653 269utilities

changes across versions 9enhancements with V7 11packaging 12

UTRO 164UTRW 164UTUT 164

WWHEN option 197Wildcarding

benefits 18how to use 18

Wildcards 18restrictions 25

Wildcards and Templates 30


(0.5” spine)0.475”<

->0.873”

250 <->

459 pages

DB2 for z/OS and OS/390 Version 7 Using the Utilities Suite

®

SG24-6289-00 ISBN 0738423246

INTERNATIONAL TECHNICALSUPPORTORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information:ibm.com/redbooks

DB2 for z/OS and OS/390 Version 7Using the Utilities SuiteDescription of the new utilities and the recently enhanced functions

Recommendations for best performance and availability

Advice for operational scenarios

IBM has continuously enhanced the functionality, performance, availability, and ease of use of DB2 Utilities. Starting with DB2 for MVS/ESA Version 4, the progression of changes has increased. DB2 for z/OS and OS/390 Version 7 introduces new functions and a new way of packaging the utilities.

This IBM Redbook is the result of a project dedicated to the new DB2 Version 7 Utilities Suite product. It provides information on introducing Wildcarding in operational scenarios, optimizing concurrent execution of utilities for a shorter night window, information for triggering utilities execution, and considerations on partitioning.

It also describes the new functions of LOAD (Cross Loader and Online RESUME) and REORG, as well as the new UNLOAD and COPTYTOCOPY utilities, and it evaluates the impact of several options when using LOAD, REORG, RECOVER, COPY, and RUNSTATS.

This redbook concentrates on the recent enhancements. It implicitly assumes a basic level of familiarity with the utilities as provided by DB2 for OS/390 Version 6.

Back cover




utilities

Documents