the ims catalog - ims phoenix ug, june 19th, 2014

© 2011 IBM Corporation1

IMS Version 12

� IMS Catalog

Kenny Blackman [email protected]


IMS Version 12

Topics

– Catalog Metadata Access and Processing Enhancements• DBD/PSB changes• IMS Explorer• Data Converters• Data Structures • Data Arrays • Data Redefines • Data Mapping• DL/I GUR call• IMS 13


IMS Version 12

Catalog Metadata definition in DBD and PSB source� IMS 12 HALDB Catalog

– Contains information about IMS program resources, database resources, and relevant application metadata

– Database Metadata • describes a physical database

– Application Program Metadata• describes application program view of the data in the database

– Arrays and Structures – Field data types and data type conversion – Field redefines – Alternative Field maps for a segment

– ACBGEN• Used to auto-populate the catalog

– Benefits• consolidates the application program data requirements into trusted location

IMS Version 12

© 2011 IBM Corporation

Physical Catalog Structure

HEADER(RESOURCE HEADER)

SS

(SENSEG)

PCB

PSB

SF(SENFLD)

PSBVENDPSBRMK

(REMARKS)

PCBRMK

(REMARKS)

SFRMK

(REMARKS)

SSRMK(REMAKRS)

XDFLD

LCHILD

AREA CAPXDBD SEGMDSET

(DATASET)

DBD

MAP

(DFSMAP)

DBDRMK

(REMARKS)

CAPXSEGM

CASE(DFSCASE)

LCHRMK(REMARKS)

SEGMRMK

(REMARKS)

CMAR(DFSMARSH)

CPROP(PROPERTIES)

CFLD(FIELD)

DBDVEND

CFLDRMK(REMARKS)

CMARRMK(REMARKS)

DBDXREF

DSETRMK

(REMARKS)

AREARMK

(REMARKS)

FLD

(FIELD)

MAR(DFSMARSH)

FLDRMK

(REMARKS)

PROP

(PROPERTIES)

MARRMK(REMARKS)

MAPRMK(REMARKS)

CASERMK(REMARKS)

RESERVED RESERVED

LCH2IDX(INDEX NAME)

XDFLDRMK(REMARKS)

…RESERVED

DSG A DSG B DSG DDSG C

DFSCX000

Secondary

Index


IMS Version 12

PSB PCB statements changes

� PCB – EXTERNALNAME=

• An optional alias for the PCB label or the PCBNAME= parameter• Java application programs use the external name to refer to the PCB

REMARKS= • Optional user comments. A 1- to 256-character field. Added to

PSBGEN statements

6

Format of an IMS catalog record for a program specification block

Key value

PSBbbbbbAPSBNAME


IMS Version 12

DBD statement changes

� DBD – ENCODING=

• Specifies the default encoding of all character data in the database defined by this DBD.

• This value can be overridden in individual segments or fields.� SEGM

– EXTERNALNAME= • An optional alias for the NAME= parameter used by Java application programs to

refer to the segment.� FIELD

– CASENAME= • The name of the map case when alternative mappings are defined for the fields in

a segment– DATATYPE=

• Specifies the external data type of the field.– EXTERNALNAME=

• An optional alias for the NAME= parameter used by Java application programs to refer to the field.

� DFSMARSH – Specifies the DATATYPE converter routine to transform binary data to external data format

� DFSMAP – Defines alternative field mappings in a segment.

� DFSCASE – Defines a map case for a segment type that uses DFSMAP alternative field mapping.

REMARKS= • Optional user comments. A 1- to 256-character field. Added to DBDGEN

statements

8

Format of an IMS catalog record for a database description

Key value

DBDbbbbbADBDNAME


IMS Version 12

Specifying data types for application programs

� DATATYPE parameter in the FIELD statement– specify the data type that an application program expects in a field– data type converter transforms the field data from the binary data type

that IMS stores the physical data to data type on the DATATYPE

FIELD EXTERNALNAME=XDATE,

BYTES=8,

START=84,

DATATYPE=DATE


IMS Version 12

Specifying data types for application programs

� DFSMARSH statement.– define additional characteristics of your data type

FIELD EXTERNALNAME=XDATE,

BYTES=8,

START=84,

DATATYPE=DATE

DFSMARSH ENCODING=Cp1047,

INTERNALTYPECONVERTER=CHAR,

PATTERN='MMddyyyy'


IMS Version 12

Struct – Catalog Metadata

DBD

SEGMENT

FIELD NAME=ADDRESS_INFO,DATATYPE=STRUCT,START=5,BYTES=45

FIELD NAME=CITY,DATATYPE=CHAR,START=5,BYTES=15,PARENT=ADDRESS_INFO

FIELD

NAME=STREET,DATATYPE=CHAR,START=21,BYTES=25,PARENT=ADDRESS_INFO

FIELD NAME=ZIP,DATATYPE=CHAR,START=46,BYTES=5,PARENT=ADDRESS_INFO

******* COPYBOOK for Address (Structure)

02 ADDRESS-INFO.

04 CITY PIC X(15).

04 STREET PIC X(25).

04 ZIP PIC X(5).


IMS Version 12

Static Array – Catalog MetadataDBD

SEGMENT

FIELD EXTERNALNAME=COURSES,DATATYPE=ARRAY,START=5, X

BYTES=512,MINOCCURS=8,MAXOCCURS=8

FIELD EXTERNALNAME=COURSE_ID,RELSTART=1,BYTES=8,PARENT=COURSES

FIELD EXTERNALNAME=COURSE_TITLE,RELSTART=9,BYTES=48, X

PARENT=COURSES

FIELD EXTERNALNAME=INSTRUCTOR_ID,RELSTART=57,BYTES=8, X

PARENT=COURSES

******* COPYBOOK for COURSES (STATIC).

20 COURSES OCCURS 8 TIMES.

25 COURSE_ID PIC 9(8).

25 COURSE_TITLE PIC X(48).

25 INSTRUCTOR_ID PIC 9(8).


IMS Version 12

Dynamic Array – Catalog MetadataDBD

SEGMENTFIELD EXTERNALNAME=NUMOF_BKS,DATATYPE=INT,START=1,BYTES=4

FIELD EXTERNALNAME=BOOKS,DATATYPE=ARRAY,START=5,MAXBYTES=200 X

MINOCCURS=1,MAXOCCURS=5,DEPENDSON=NUMOF_BKS

FIELD EXTERNALNAME=ISBN,RELSTART=1,BYTES=10,PARENT=BOOKS

FIELD EXTERNALNAME=BOOK_TITLE,RELSTART=11,BYTES=22,PARENT=BOOKS

FIELD EXTERNALNAME=RETURN_DATE,RELSTART=33,BYTES=8,PARENT=BOOKS

******* COPYBOOK for BOOKS (DYNAMIC)

20 NUMOF-BKS PIC 9(4) COMP.

20 BOOKS OCCURS 1 TO 5 TIMES DEPENDING ON NUMOF-BKS.

30 ISBN PIC X(10).

30 BOOK-TITLE PIC X(22).

30 RETURN-DATE PIC 9(8).


IMS Version 12

DBD changes

�DFSMAP statement–Enables the alternate mapping of fields within a segment.–Defines a group of one or more map cases and relates the cases to a

control field. The control field identifies which map case is used in a given segment instance.

Policy SegmentControl

field

Car Insurance Policy Segment

01

Home InsurancePolicy Segment

02

Life InsurancePolicy Segment

03


IMS Version 12

15

Maps�Mapping support

– A Map is metadata that describes how a field (or set of fields) are mapped for a particular segment instance

– Metadata captures the various cases and for each case defines the set of fields to be used for that case

– Maps can be defined to the catalog– Maps are interpreted at runtime by the Universal drivers and the proper data

elements are returned based on the runtime case of the segment instance– Example: Insurance segment mapped multiple ways depending on value of

Policy Type control field

Policy Type

Property Type

Rooms Value Address Make Model Year Value Color

M - - - - Ford Escort 1989 2K Red

HSingle Family

5 500K555 Disk

Drive Way, 95141

- - - - -


IMS Version 12

Maps Metadata� DBD mapping support DBD

SEGM NAME=POLICY,EXTERNALNAME=CUSTOMERPOLICY************************************************************************ CONTROL FIELD:*********************************************************************** FIELD EXTERNALNAME=POLICYTYPE,DATATYPE=CHAR************************************************************************ DFSMAP STATEMENT:*********************************************************************** DFSMAP NAME=POLICYMAPS,DEPENDINGON=POLICYTYPE************************************************************************ DFSCASE STATEMENT 1:*********************************************************************** DFSCASE NAME=AUTOMAP,CASEID=AUTO,CASEIDTYPE=C,MAPNAME=POLICYMAPS FIELD EXTERNALNAME=AUTOMAKE,CASENAME=AUTOMAP, C BYTES=15, C START=5, C DATATYPE=CHAR FIELD EXTERNALNAME=MODEL, C CASENAME=AUTOMAP, C BYTES=15, C START=20, C DATATYPE=CHAR FIELD EXTERNALNAME=YEAR, C CASENAME=AUTOMAP, C BYTES=4, C START=35, C DATATYPE=CHAR************************************************************************ DFSCASE STATEMENT 2:*********************************************************************** DFSCASE NAME=HOMEMAP,CASEID=HOME,MAPNAME=POLICYMAPS, C FIELD EXTERNALNAME=DWELLING_TYPE,CASENAME=HOMEMAP, C BYTES=20, C START=5, C DATATYPE=CHAR FIELD EXTERNALNAME=ROOMS,CASENAME=HOMEMAP, C BYTES=5, C START=25, C DATATYPE=CHAR FIELD EXTERNALNAME=SQ_FOOT,CASENAME=HOMEMAP, C BYTES=6, C START=30, C DATATYPE=CHAR


IMS Version 12

UserTypeConverter Interface

� The IMS Catalog can store information on fields with a user

defined typeDBD

SEGMENT

FIELD DATATYPE=DECIMAL

DFSMARSH USERTYPECONVERTER=class://com.ims.PackedDateConverter

� Convert binary data stored by IMS to data type required by the application� Example of Catalog Metadata that contains a user defined type:

<field name="PACKEDDATEFIELD">

<startPos>40</startPos>

<bytes>5</bytes>

<marshaller encoding="">

<userTypeConverter>class://com.ims.PackedDateConverter</userTypeConverter>

<property name="pattern" value="yyyyMMdd"/>

<property name="isSigned" value="N"/>

</marshaller>

<applicationDatatype datatype=“DATE"/>

</field>


IMS Version 12

Partition Selection with Key RangeHigh key is specified for each partition

HALDB Partition Definition Utility or DBRC commands

key value is generated by the IMS catalog populate utility

Partitions are selected by using key from root segment

� Key value is created by concatenating the record type and the IMS member name of the resource.

� record type eight characters right-padded with blank characters.

� IMS member name is always eight characters long.

DBDbbbbbKDBDNAME PSBbbbbbYPSBNAME

Key range: A - M N - ZA - M N - Z

Sequence: 1 432

DBDbbbbbYDBDNAME PSBbbbbbKPSBNAME


IMS Version 12

Catalog Populate utility reportCATALOG DFSCD000

PARTITION DFSCD01NUMBER OF SEGMENTS INSERTED INTO THE

CATALOGINSERTED AVERAGE

SC SEGMENT SEGMENTS DSG PARENT SEGS/PARENT

1 HEADER 4228 A

2 DBD 2530 A HEADER 0.63 CAPXDBD 7 D DBD 0.0

5 DSET 2599 D DBD 1.07 AREA 139 D DBD 0.1

9 SEGM 16337 B DBD 6.510 CAPXSEGM 1 D SEGM 0.0

12 FLD 16426 C SEGM 1.014 MAR 16426 C FLD 1.0

17 LCHILD 2687 B SEGM 0.220 XDFLD 134 B LCHILD 0.0

33 PSB 1840 A HEADER 0.435 PCB 9190 B PSB 5.0

37 SS 75274 B PCB 8.239 SF 1105 B SS 0.0

41 DBDXREF 8886 D PSB 4.8

SEGMENT within existing header duplicates not inserted

DBD 71 0

PSB 72 0

ESTIMATED SPACE REQUIREMENT TO HOLD INSERTED SEGMENTS

DSG BLKSIZE BLOCKS

A 4096 596

B 4096 9343

C 4096 8214

D 4096 236

DSG RECORDS

L 8886

X 4230

SECONDARY

INDEX RECORDS

DFSCX000 8886


IMS Version 12

Application programming with the IMS Catalog

� IMS catalog resident PSBs for application programs– DFSCP000

• High-level assembler and COBOL applications– DFSCP002

• PL/I applications– DFSCP003

• PASCAL applications� The following PCBs are included to support different catalog

processing models:– DFSCAT00

• The primary PCB to access all data in the catalog database..– DFSCATSX

• Use this PCB to access the catalog database via the catalog secondary index.– DFSCATX0

• Use this PCB to process the catalog secondary index database.� All catalog processing is performed with PROCOPT=G.� GUR call

– application programs can use the Get Unique Record (GUR) DL/I call to retrieve catalog database record


IMS Version 12

Get Unique Record

� New “GUR” DL/I call– Get Unique Record– Restricted to use with IMS Catalog database

� Functions like a GU followed by a series of GNP calls � Returns the entire database record in one call

– Saves overhead of issuing GU & GNP to retrieve all the metadata for a catalog member

– Using an AIB token, the call can be continued if the I/O area is too small for entire catalog database record

� Data returned will be in XML format– Matches the XML schema

� Support added for DFSDDLT0 and IMS REXX


IMS Version 12

Get Unique Record Example

…Skipped output…


IMS Version 12

IMS Enterprise Suite V3.1 Explorer for Development

Generate SQL to

access IMS data

See database relationships

change DBD field attributes

Edit PSB sensitive

segments and attributes


IMS Version 12

IMS Enterprise Suite V3.1 Explorer for Development Runtime Catalog Access Support

DBDB

Catalog

DL/I or Type 2

Universal Drivers

(SQL/DLI/XML)

WAS z

DRDA/TCPIP

SQL/DLI

SQL/DLISCI

DLI

UsersIMS Explorer

Tools

Type 4

Universal

Drivers

(SQL/DLI/XML)

Applications

SQL/DLI

Dependent

Regions

(MPP,JMP,IFP

,BMP,JBP)

Type 2

Universal Drivers

(SQL/DLI/XML)

DL/I or

Type 2

Universal Drivers

(SQL/DLI/XML)

CICSDB2 z

SP

Type 4

Universal Drivers

(SQL/DLI/XML)

DRDA/TCPIP


IMS Version 12

Import IMS Source


IMS Version 12

Import COBOL/PLI

01 DEALER-SEGMENT .

03 DEALER-NUM PIC X(00004) . 03 DEALER-NAME PIC X(00030) . 03 ADDR PIC X(00032) . 03 ADDRESS-PARTS REDEFINES ADDR . 05 CITY PIC X(00020) . 05 STATE PIC X(00002) . 05 ZIP PIC X(00010) . 03 PHONE PIC X(00007) .


IMS Version 12

Updated IMS Source


IMS Version 12

Import from Catalog


IMS Version 12

IMS Enterprise Suite V3.1 Explorer for

Development� 1. Import DBDs/PSBs into IMS Explorer.� 2. Using IMS Explorer, update the metadata from a COBOL/PLI layout.� 3. IMS Explorer will generate new DBD source with the metadata.� 4. The "new" DBD will be sent through DBDGEN then ACBGEN.� 5. ACBGEN will update the Catalog.

PSBLIB

DBDLIB

ACBGENDFS3UACB Catalog

ACBLIB

INPUT PROCESS OUTPUT

LogsDFSDFxxxmember

IMS Explorer

DBD/PSBSource

DBD/PSBMetadataSource

PSB/DBD GENs

COBOL/PLISource


IMS Version 12

IMS Universal drivers and the catalog

�All IMS Universal drivers leverage the IMS catalog

–Direct access to IMS metadata in the catalog

–No longer require the separate Java metadata class

• Virtual deployment support

–No longer file-system dependent for metadata

• Virtual deployment support

–Metadata is trusted and up-to-date


IMS Version 12

Universal Drivers Variable Length Segments support

� Data portion of Variable Length Segment contains 2 byte LL field

� DBD specifies maximum and minimum number of bytes

DBD NAME=HOSPDBD, …

SEGM

NAME=WARD,PARENT=HOSPITAL,BYTES=(maxbytes,minbytes

)

� IMS Universal Drivers are now sensitive to the LL field of a Variable Length Segment– The IMS Universal DB resource adapter and IMS Universal JDBC

driver internally manage the LL field on behalf of the application

– Applications that use the IMS Universal DL/I driver are responsible for managing the LL field


IMS Version 12 • Native SQL COBOL and distributed applications (.NET/JDBC)

• Provides standard SQL keywords to easily access IMS data

� SELECT, INSERT, UPDATE, DELETE

� Uses Dynamic SQL programming model

� Converts SQL statements to DL/I calls

� Supports a subset of SQL keywords that are currently supported by IMS Universal JDBC driver

• Uses database metadata in IMS Catalog

� No need to generate metadata for use in applications

z/OS

IMS DB

DLI

Native

SQL

Catalog

MetadataSQL

DRDAJMP JBP

JAVA

MPP BMP IFP COBOL

Language

Inte

rface

IMS

JDBC

RYO

.NET

Language interface

SQL

ODBA / DRADistributed

planned

IMS 13 SQL Support

IMS 13

07- IMS Application:33

DataSetIMS Data Provider for Microsoft .NET

DataRelationCollection

SelectCommand

InsertCommand

UpdateCommand

DeleteCommand

IMSDataAdapter

IMS Connect

IMSCommand

SQL statement

IMSDataReader

DataRowCollection

DataColumnCollection

ConstraintCollection

DataTable

z/OS

ODBM

DRDA Target Server

IMSConnection

DRDA Client

IMS 13

Native SQL

Catalog

Metadata

ADO.NET Applications

IMS DB

IBM IMS Data Provider for Microsoft .NET Architecture

OM SCI

IMS 13

3407- IMS Application:

IMS 13 COBOL Native SQL

IMS Native SQL

DFSLI000

IMS DB

SQLTDLI

DLI

SQLIMSSQLCA

EXEC SQLIMS

(CALL SQLTDLI

USING SQL-

PARMLIST)

:

:

Retrieve IMS database PCB Schema metadata on first SQL call

Parse and validate SQL

Build and make DLI call to access IMS data

Perform aggregation on results data (if needed)

Map results data back to the application

IMS Catalog

Metadata

COBOL Application

IMSSQLCA + Data

IMS MPP,IFP,BMP

64-Bit Storage

IMS 13

07- IMS Application:35

SQL Parsing in IMS � Solution IMS Java intends to use IMS 13 Native SQL

– IMS Service Process

� Example with IMS SQL call handler

– How much money has my insurance company paid out in claims for the year 2011?

– SELECT SUM(CLAIMAMOUNT) FROM CLAIMS WHERE YEAR=2011

IMS DB

z/OSClient Application

IMS

JDBC

Driver

SQL Call

Claims$2432

$41…$6255+ ________

$797439097

Claims$2432

$41…$6255

IMS 13

Native SQL

$797439097

Total Claims

$797439097


IMS Version 12

IMS 13 DB Versioning

DBJK21

DBJK22

DBJK23

Application

IMS

DBJK21 V3

DBJK22 V3

DBJK23 V3

ACBLI B

DBJK21 V0,V1,V2,V3

DBJK22 V0,V1,V2,V3

DBJK23 V0,V1,V2,V3

IMS Catalog

DBJK21 V3

DBJK22 V3

DBJK23 V3

DBDLIB

PSBJK

PSBLIB

PSBGEN

DBLEVL=CURR

--------------------------

PCB

--------------------------

DBJK21 V1

DBJK22

DBJK23 V2

PSB=PSBJK Source

DBJK21 V3

DBD Source

DBJK22 V3

DBJK23 V3

DBDGEN

PSBGEN

DBJK21 V3

DBJK22 V3

DBJK23 V3

DBJK21 V3

DBJK22 V2

DBJK23 V1

DLI

Retrieve DBJK22 & DBJK23 from Catalog

Active

ACBGEN

��

��

��

��

Version “V3” of DBDs put into ACBLIB & Catalog

Database Versioning enabled -> data returned to app at V1, V2 & V3 levels

��

DBJK21

DBJK22

DBJK23

INIT

VERSION(DBJK21=3,DBJK22=2,DBJK23=1)

36

37

37

IMS Integration with QMF

IMS Open Database:

•IMS allows distributed access over TCP/IP using the IMS Universal Database Driver (JDBC)

•IMS Connect and Open Database Manager now work together as a DRDA server for IMS data

Workstation

Windows, Linux, and the Macz/OS

QMF

IMS

Universal

Database

Driver

TCPIP

TCPIP

IMS Connect

S

C

I

ODBM

S

C

I

DRA

CTL IMS DB

DL/I

PC

IMS

DRDA

TCPIP

IMS 12 Catalog Metadata IMS 11 Local file Metadata

jdbc:ims://host:port/class:

//dfsivp37.DFSIVP37DatabaseView:

dbViewLocation=C:/share/IMS Universal Drivers Metadata;fetchSize=0;

jdbc:ims://host:port/DFSCP000:dpsbOnCommit=true

38

DataPower to IMS DB

DataPower

IMS

JDBC

Driver

Routing/

data

transformation

SQL

sends /

receives

DRDA

sends /

receives

IMS DB

ODBMIMS

Connect

IMS

Catalog

IMS DB

IMS

Catalog

DRDA

DLI

Back

End

services


IMS Version 12

39

IMS catalog – intended support

PSBLIB

Catalog

IMS DB changes start with catalog

� IMS Explorer uses Data Definition Language (DDL) to update catalog

� IMS loads resource information from catalog

� ACBLIB/PSBLIB/DBDLIB updates will be the by-product of catalog updates

� Tools that use these libraries can continue to operate, but should migrate to catalog

� PSB and DBD source can still be optionally generated from PSBLIB and DBDLIB

IMS ExplorerDDL

ACBLIB

DBDLIB

PSB

source

DBD

source

IMS


IMS Version 12

40

Dynamic database - Data Definition Language

� SQL incorporates DDL to modify the schema of a database

� Authoring DDL is straight-forward with sophisticated tooling support in the industry

� SQL/DDL can be used to update/add metadata in the catalog without the need of a GEN

– Directly update the catalog

� IMS can be notified of such an update and load the new definitions

� It is our intention to offer this type of dynamic definition for IMS


IMS Version 12

The End

the ims catalog - ims phoenix ug, june 19th, 2014

Technology

datatype field externalname

data type field externalname

info field

courses field externalname

field statement

character field

books field externalname

bks field externalname