accelerate with ibm storage: ts7700 object store and ... · object storage 1. read data from object...

© Copyright IBM Corporation 2020

Accelerate with IBM Storage: TS7700

Object Store and DS8000 Transparent

Cloud Tiering

Eddie Lin, DS8000 Architect and Development

Lourie Goodall, TS7700 Architect and Development

Washington Systems Center - Storage

© Copyright IBM Corporation 2020 Accelerate with IBM Storage

Accelerate with IBM Storage Webinars

The Free IBM Storage Technical Webinar Series Continues in 2020...

Washington Systems Center – Storage experts cover a variety of technical topics.

Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome.

To automatically receive announcements of upcoming Accelerate with IBM Storage webinars, Clients, Business Partners and IBMers are welcome to send an email request to [email protected].

Located on the Accelerate with IBM Storage Site: https://www.ibm.com/support/pages/node/1125513

Also, check out the WSC YouTube Channel here: https://www.youtube.com/channel/UCNuks0go01_ZrVVF1jgOD6Q

2020 Upcoming Webinars:

April 21 – CSM 6.2.8 - What's New!Register Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=ef2d68c2c198e337c1e222998b32528e0

April 23 - TS7700 Security - Keeping Your Castle SafeRegister Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e4f9c3ec70aa0856171e3255ab0136993

May 12 - Artificial Intelligence (AI) Scalability for a DecadeRegister Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e6a684ae106025eeebb60ad03ec0de15b

May 21 - Storage Insights, Storage Insights Pro or Spectrum Control, which one is right for me?Register Here: https://ibm.webex.com/ibm/onstage/g.php?MTID=e32d1989e32fe468ef72c082519920fc1

mailto:[email protected]

https://www.ibm.com/support/pages/node/1125513

https://ibm.webex.com/ibm/onstage/g.php?MTID=ef2d68c2c198e337c1e222998b32528e0

https://ibm.webex.com/ibm/onstage/g.php?MTID=e4f9c3ec70aa0856171e3255ab0136993

https://ibm.webex.com/ibm/onstage/g.php?MTID=e6a684ae106025eeebb60ad03ec0de15b

https://ibm.webex.com/ibm/onstage/g.php?MTID=e32d1989e32fe468ef72c082519920fc1



Session objectives

• Introduction

• DS8000 Transparent Cloud Tiering

• TS7700 as an Object Store

• Q & A

3



Evolution of Mainframe Space Management

FlashCopy

Class

Transitions

Transparent

Cloud Tiering

TCT to TS7700

as an Object

Store

DFSMShsm wants OUT of the

business of physical data

movement

Easy Tier

Tape Tiering

Efficiency

5

The Transparent Cloud Tiering Advantage Cost SavingsSecurity

DFSMS

Primary Disk

Virtual or Physical Tape

1. Read Data from Tape into z/OS2. Write Data from z/OS to Disk3. Read Data from Disk to z/OS4. Data Available to User

Data Recall Steps

Efficiency

6

The Transparent Cloud Tiering Advantage IntegrationSecurity

Primary Disk

Object Storage

1. Read Data from Object Store into Primary Storage2. Data Available to Customer

Data Recall Steps

Direct data transfer from object storage to primary storage

• Eliminates data movement through IBM Z• Eliminates Tape Recycle Processing• Eliminates 16k Block Size restructuring• Eliminates Serial Access to Tape

• SSL/TLS Authentication

• TCT Encryption with Dataset Level Encryption Support

• Seamless integration into DFSMShsm and DSS

• Supports Off-Prem and On-Prem Object Storage and now TS7700 Virtual Tape



Transparent

Cloud Tiering

7

z/OS Transparent Cloud Tiering with DS8880 / DS8900 Current Key Use Case: DFSMShsm Dataset Migrate/Recall

• MIPS reduction via direct data transfer from DS8000 to Object Storage• Recall to volumes participating in Simplex, Flashcopy, Metro Mirror, Global Mirror, MGM Environments

• Includes support for Hyperswap• TCT Client Side Encryption• 1Gbit (Included) and 10Gbit (Additional) Ethernet Connectivity• Command Line and Automated HSM Migration – Dataset movement to Cloud by policy, size based override• Supports IBM Cloud Object Storage, Amazon S3 API, Openstack Swift, TS7700

IBM Cloud

Object Storage

IBM Z IBM DS8880

Transparent Cloud Tiering

IBM

TS7700

IBM Cloud


© Copyright IBM Corporation 2020 Accelerate with IBM Storage9

TCT CPU Savings Estimator

9

Based on approximations from preliminary internal IBM data measurements. Results will vary by

customer based on particular workloads, configurations and software levels applied.

Estimation for Daily PSM

CPU reduction

Total reported CPU compared

against an estimated CPU:

(.05 seconds * # data sets)

Remember, reduction is based

on the size of the data being

moved. Clients with mostly

small data sets will see modest

reductions.

ftp://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/HSM/zTCT/

ftp://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/HSM/zTCT/



Cloud Objects within the Object Store

10

• TCT stores datasets as individual objects, not as logical tape volumes.

• DFSMShsm considers objects a separate tier versus ML1, ML2 (Migration Level CLOUD)

• VOLSERs are modified to MIGRATC

• Datasets are a collection of objects : metadata and customer data are split

• Allows for future work with analytics, data sharing, and further tiering

DCSDMC01.GDG.V12S.G0001V00/APPMETA

DCSDMC01.GDG.V12S.G0001V00/DTPDSHDR

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/META

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/EXTENTS

VOLSER Z20022

VS.



Use Case : Targeted HSM Automatic Migration

11

• Management Class Enhancements

• Use new “Level 2 Days Non-Usage”

parameter with value of “0” to define moving

directly to CLOUD based on primary days

non-usage

• Use new SIZE based parameters to further

differentiate datasets to CLOUD



AB C D

L0

MLC

AutoMigration Window 2

• Runs 12 hours• Process large sized

IF DS Size < 1 GBNo Action

ELSECloud

ML2

AutoMigration Window 1

• Runs 4 hour• Skip small / Process medium sized

IF DS Size <= 1 MBNo Action

ELSE IF DS Size >= 1 GBNo Action

ELSEML2

Combine HSM Auto Migration and TCT’s minimized CPU footprint



Use Case : DSS Backups Migrated to TS7700

13

DFSMSdss

Source Storage Group

DSS Backup Copies

DFSMShsm TCT

▪ Step 1: Create DSS Disk backup with zEDC

▪ If source data set is already zEDC

compressed, backup will be same

size as source

▪ Otherwise, backup copy will be zEDC

compressed (VSAM, etc)

▪ Write backup copies to one or more

storage groups managed by HSM

▪ Step 2: Management Class and Storage

Group policies used by HSM to automatically

expire/migrate older backup copies to TS7700

▪ Backup copies with a short life can

expire on disk (HSM expiration)

▪ Backup copies with a long life can be

automatically migrated to TS7700 after

a policy based time period

X



Other TCT use cases

Flashcopy to a Storage Group, HSM Migrate to Cloud

14

Move very old ML2 data to MLC to eliminate recycle

• Create Point in Time Copy via DSS

FlashCopy

• eg. DB2 Image Copy

• Use HSM: MIGRATE

STORAGEGROUP(DB2IMGC)

CLOUD(MYCLOUD)

• Note: For now, manual recall

required before DSS restore

• ML2 data must be recalled and

then remigrated from ML2 to MLC

• Target very old datasets to avoid

additional recycles

Source

VolumesStorage Group

FlashCopy HSM



TCT in DS8000 Replicated Environments

15

Topology Minimum Code Level Support

Metro Mirror DS8880 R8.3• DFSMS APARs OA52386

• PTFs on z/OS V2R2 and z/OS V2R3

• GDPS APAR PI98596

DS8900 R9.0

Global Mirror DS8880 R8.5

DS8900 R9.0

MGM DS8880 R8.5

DS8900 R9.0

Multi-Target MM or MM/GM DS8880 R8.5

DS8900 R9.0

zGlobal Mirror (XRC) Not Supported

FlashCopy DS8880 R8.4• DFSMSdss APAR OA55043

• PTFs on z/OS V2R2 and z/OS V2R3

DS8900 R9.0



TCT Encryption for Object Storage

• Client Side Encryption• 256-bit AES Encryption

• Encryption happens in the DS8000, data remains encrypted

across the link and in the cloud

• Hardware accelerated encryption using POWER8 and 9

• Only customer data is encrypted

• Requires an external key manager (KMIP)• ISKLM 3.0.0.2+ (Distributed)

• Interlock with DFSMS dataset level encryption• If the dataset is already encrypted, DS8000 will not attempt

to encrypt again



TS7700 AS AN OBJECT STORE

17



Transparent Cloud Tiering with TS7700 as Object Store

18

• TS7700 as a target object store

• Supports TS7760/TS7770 Virtual Tape

and DS8880/DS8900 Disk systems

• Intermix existing FICON tape workloads

and TCT Object workloads to the same

TS7700

• Data flows from DS8000 to TS7700 via

GRID network links

• Utilizes logical partitions dedicated to

TCT objects

• Supports data redundancy via writing to

two TS7700 clusters within the same

GRID



Grid Cloud

Clo

ud

Clo

ud

TS7760C TS7760CTS7760T TS7760T

DS8880 DS8880 DS8880

Ethernet

Z System

FIC

ON

FIC

ON

• Co-existence with up to 8-way GRIDs

• TS7700 can be tapeless, tape attached,

or cloud attached.

• Objects cannot be migrated to tape or

cloud.

• Support multiple DS8Ks connect to a

TS7700

• Uses existing cache. MES of additional

cache not required

Transparent Cloud Tiering with TS7700 as Object Store



Logical Cache Partition for Objects

DS8000

FICON

Workload

Policy

Managed

TS7770T or TS7770C

Object

Workload

Cloud

• Utilizes existing TS7700 Logical Cache Partitioning

• Separates FICON logical volume data from object data

• Coexists with Tape or Cloud logical volume partitions

• Allocates from Cache Partition 0 (Resident only partition) and returns to CP0

Logical volume data

Object data

Resident Only

Partition

20

zHost



Parallel Access Objects

21

DCSDMC01.GDG.V12S.G0001V00/APPMETA

DCSDMC01.GDG.V12S.G0001V00/DTPDSHDR

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/META

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/EXTENTS

• Allows for parallelism of object retrieval and store• Eliminates serial access to tape

DS8000

TS7700

Z



Eliminates HSM Reclaim

22

Z

DS8000

TS7700

dksksksk

21%

dksksksk

20%

dksksksk

23%

dksksksk

21%

dksksksk

20%

dksksksk

23%

dksksnnnksk

Recall

Reclaim

Rewrite

HDELETE

dksksnnnks

k



Requirements for DS8000 TCT with TS7700 as an Object Store

• DS8880 and DS8900 supported (all models)

• Part of the zSynergy License Bundle

• DS8880 R8.5SP4.1 minimum level

• DS8900 R9.0SP1 minimum level

• Ethernet connections

• Note: DS8900 10Gb Ethernet CANNOT direct connect to TS7700 GRID adapters (SR vs. LR connectivity). DS8880 10Gb Adapters can direct connect.

• TS7760 (VEC) and TS7770 (VED) supported

• Minimum code level is R5.0

• Feature Code #5282

• TS7760T tape attached, TS7760C cloud attached, TS7760 tapeless supported

• Uses existing GRID adapters and will share TCPIP traffic with the GRID

• z/OS V2R3 and V2R4 supported

• APARs required to enable ”Tape Object” type in SMS Cloud Construct/Network Connection panel23



Easy Setup Steps

24

TS7700Activate the FC 5282 and enable DS8000 Object Store01

TS7700Enter DS8000 serial number and IP addresses02

TS7700Resize Object cache partition03

z/OSCreate Cloud Network Connection construct in the z/OS ISMF Cloud

panel using provider “TAPE-OBJECT”.04

z/OSCreate z/OS Management Class construct “Level 2 Days Non-

usage”=0 05

DS8000Enable Ethernet ports and enter TS7700 Grid IP address and

Network Connection name in DS8000 mkcloudserver command06



Object Cache Partition

• Object Cache Partition now visible on the TS7700 MI (tapeless)

• New object partition will now exist - created during Enable process

• Resize partition

• Object Count

25



Library Request CACHE2

Displays the partition type and cache utilization statistical information

26

> SHOWING RESULTS FOR COMMANDS: CACHE2

"TAPE VOLUME CACHE STATE V5 .0 "

"TOTAL SPACE INSTALLED/ENABLED: 191TB/ 191TB "

"TOTAL ADJUSTED CACHE SPACE USED: 1044GB "

"CACHE ENCRYPTION STATUS: CAPABLE "

"OVERCOMMITTED CACHE PARTITIONS: NONE "

"CACHE RESIDENT ONLY PARTITION "

" PRIVATE CACHE SPACE USED: 1042GB "

" SCRATCH CACHE SPACE USED: 0.0GB "

" CP ALLOC USED PIN PKP PRM COPY CPYT "

" 0 70.8TB 1042GB 0.0GB 1042GB 0.0GB 0.0GB 0 "

" FLASH COPY INFORMATION "

" INDEX ENABLED SIZE "

" 1 NO 0.0GB "

" 2 NO 0.0GB "

" 3 NO 0.0GB "

" 4 NO 0.0GB "

" 5 NO 0.0GB "

" 6 NO 0.0GB "

" 7 NO 0.0GB "

" 8 NO 0.0GB "

"TIERED CACHE PARTITIONS "

" CP TCO ALLOC USED PG0 PG1 PMIGR D_PMIGR COPY PMT CPYT "

" 1 NNY 121TB 2189GB 0.0GB 2189GB 0.0GB 0.0GB 0.0GB 0 0 "

" 2 NNN 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0.0GB 0 0 "






> EXECUTING COMMANDS: CACHE2



Library Request SETTING2, RSDOLOW/RSDOHIGH

• New settings RSDOLOW and RSDOHIGH

• LI REQ SETTING2 - Displays current settings

• LI REQ SETTING2,RSDOHIGH,<value GB> - High warning limit of resident data in the object partition

• LI REQ SETTING2,RSDOLOW,<value GB> - Low warning limit of resident data in the object partition

27

> SHOWING RESULTS FOR COMMANDS: SETTING2

"SETTING2 V4 .0 "

" SCRATCH CONTROLS "

" PFRLOCO = ENABLED "

"----------------------------------------------------------------------"

" ALERTS "

" PDRVSLOW = 0 PDRVSCRT = 0 "

" LMTDTHR = 0 "

" CAGALOW = 0 CAGAHIGH = 0 "

" CAGLOW = 0 CAGHIGH = 0 "

" RSDOLOW = 1000 RSDOHIGH = 10000 ”

"----------------------------------------------------------------------"

" CACHE CONTROLS "

" MAXLGMC = 128 "

" OVRSPLVL = 2000 "

"----------------------------------------------------------------------"

…..



Library Request STATUS,GRLNKACT

28

• "GRLNKACT STATUS V2 .0 "

• "CLUSTER INDEX: 1 LINK COUNT: 4 Time: Wed Feb 27 00:04:15 CUT 2019 "

• "GRID LINK ESTABLISHED SOCKET CONNECTIONS------------------------------"

• "LN INTF IP C0 C1 C2 C3 C4 C5 C6 C7 MQ RFA CLD "

• "L0 en8 9.11.219.164 0 0 55 0 0 0 0 56 111 0 0 "

• "L1 en10 9.11.219.165 0 0 0 0 0 0 0 0 0 0 0 "

• "L2 en9 9.11.219.169 0 0 0 0 0 0 0 0 0 0 0 "

• "L3 en11 9.11.219.170 0 0 0 0 0 0 0 0 0 0 0 "

• "LN INTF IP GGM OBJ "

• "L0 en8 9.11.219.164 0 2 "

• "L1 en10 9.11.219.165 0 0 "

• "L2 en9 9.11.219.169 0 0 "

• "L3 en11 9.11.219.170 0 0 "

• "NET ACTIVITY -----TCP RECV/SEND ADAPTER BUFFER ACTIVITY BYTES---------"

• "LN TxMBs RxMBs MQ_REC MQ_SND GFA_REC GFA_SND CLD_REC CLD_SND "

• "L0 100 100 0 0 80 80 0 0 "

• "L1 0 0 0 0 0 0 0 0 "

• "L2 0 0 0 0 0 0 0 0 "

• "L3 0 0 0 0 0 0 0 0 "

• "TOT 100 100 0 0 80 80 0 0 "

• "LN GGM_REC GGM_SND OBJ_REC OBJ_SND "

• "L0 0 0 20 20 "

• "L1 0 0 0 0 "

• "L2 0 0 0 0 "

• "L3 0 0 0 0 "

• "TOT 0 0 20 20 "

• "GRID LINK THROUGHPUT ESTIMATES-MB/s-----------------------------------"

• "DIR C1 C2 C3 C4 C5 C6 C7 GRD_TOT CLD GGM OBJ TOT "

• "Tx 0 50 0 0 0 0 30 0 0 0 20 100 "

• "Rx 0 30 0 0 0 0 50 0 0 0 20 100 "

<--C0 doesn’t show since it is the local

New command added in R4.2. Grid link point-in-time activity.

Add new columns for data for Objects



TS7700 Management Interface Changes

29

• Cache Utilization

• Object Count

• Historical Statistics

• Grid link throughput

Flash Copy Data



DS8000 Object Store - Object Data Resync/Recovery

30

In the event a TS7700 becomes

unavailable, DS8K will track

transactions not written to the

unavailable cluster and

automatically synchronize the

data once the cluster becomes

available.

XTODO

Object1Object2Object3Object4

Object1Object2Object3Object4



TCT, TS7700, GRID and DR Distance

31

• Planning to use synchronous writes for data redundancy?

• Planning on one of those TS7700 Clusters to be at the remote

site?

• Use existing IP Extension options to eliminate latency, just like

GRID to remote locations today

DS8000

Local TS7700

GRID Network (WAN)

Remote TS7700

IP Extension

IBM SAN 42B-R

Brocade 7840

IBM SAN 42B-R

Brocade 7840



Preview Announcement – TS7700 R5.0.1

TCT Compression to TS7700

• Compression will be done within the DS8900

• Uses a hardware compression engine within the POWER9

• Data compressed before transmission as well as when it lands in object store / TS7700

• Bandwidth optimization and storage savings

• Interlock with DFSMS if the dataset is already encrypted or compressed

TCT with Secure Data Transfer to TS7700

• TCT to TS7700 as an object store currently does not support encryption

• Uses the same protocol as Secure Data Transfer for the GRID

• Via embedded TS7700 data movement engine

• Encryption is only for data in flight – decrypted when landing in TS7700

• Uses hardware engine for encryption

• Does not require an external key server

32IBM Systems Technical University © Copyright IBM Corporation 2019


© Copyright IBM Corporation 2020 Accelerate with IBM StorageIBM Systems Technical University © Copyright IBM Corporation 2019 33

http://publibz.boulder.ibm.com/zoslib/pdf/OA51622.pdf

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.idak100/cloud23.htm

New - Available in DRAFT format for TCT and TCT to TS7700 as an Object Store

Other Links to publications:www.redbooks.ibm.com › Cloud

https://www.ibm.com/support/knowledgecenter/ST5GLJ_8.5.0/com.ibm.storage.ssic.help.doc/f2c_configuring_trans_

cloud_tiering.html

IBM TS7700 Series DS8000 Object Store User’s Guide Version 1.0,REDP-5583

http://www.redbooks.ibm.com/abstracts/redp5583.html

IBM DS8000 and Transparent Cloud Tiering,SG24-8381

http://www.redbooks.ibm.com/Redbooks.nsf/RedpieceAbstracts/sg248381.html

Publications

http://publibz.boulder.ibm.com/zoslib/pdf/OA51622.pdf

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.idak100/cloud23.htm

https://www.ibm.com/support/knowledgecenter/ST5GLJ_8.5.0/com.ibm.storage.ssic.help.doc/f2c_configuring_trans_cloud_tiering.html

http://www.redbooks.ibm.com/abstracts/redp5583.html

http://www.redbooks.ibm.com/Redbooks.nsf/RedpieceAbstracts/sg248381.html

Unleash the potential of your data.

Put smart to work today.

IBM Storage

accelerate with ibm storage: ts7700 object store and ... · object storage 1. read data from object...

Documents