ibm db2 tools update & ibm data virtualization manager for ... · pdf fileibm db2 tools...

© 2017 IBM Corporation

IBM Db2 Tools update &

IBM Data Virtualization Manager for z/OS

Jørn Thyssen

Solutions Advisor

Db2 for z/OS Tools Development

© 2017 IBM Corporation2

Legal Disclaimer

• © IBM Corporation 2017. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication,

it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice.

IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have

the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced

in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in

any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other

results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary

depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed.

Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance

characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime®

Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to

http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentat ion. All product names must

be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus

Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both.

Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:

UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme)

please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.

http://www.ibm.com/legal/copytrade.shtml


Contents

▪ IBM Data Virtualization Manager for z/OS

➢ High level overview, use cases, architecture

▪ IBM Db2 Tools update

➢ Strategy

➢ Highlights

NEW


Critical Business Issues

Speed of business and new

digital requirements forcing

need for real-time information

There is simply too much data for older

architectures

Data resides in silos and unfamiliar formats


Data Virtualization

Eliminating the Information Bottleneck


Modernization

Faster, easier delivery of

modern systems of

engagement

Optimization

Reduce the

cost/complexity of

accessing mainframe

data

Real-time

Analytics

Need for immediate insight

into your customer or

business

Data Virtualization Use Cases


IBM Data Virtualization Manager for z/OS

Virtualize z/OS data with other enterprise data sources in real-time to provide comprehensive

information that is readily consumable by analytics, digital applications and users

IBM Data

Virtualization

Manager for z/OS

• zIIP eligible / avoid MLC costs

• Optimized for z14, z13, EC12

• Supports relational sources

• >100X faster than IICF

• Support any API:

SQL, NoSQL, REST, SOAP, HTTP

• Python support

• Automated discovery though IBM ADDI

• Ships with industry mappings

• Exploits 64-bit storage

• Built-in MapReduce

• Built-in parallel I/O


Data Virtualization

Manager for z/OS

Data Virtualization Technology Accelerates Real Time Analytics

Predictive AnalyticsDiscovering patterns/meaning in data

Query AccelerationHigh-speed processing for complex Db2 queries

BI Solution with Access, Virtualization and Visualization

and Data Preparation

Common metadata maps

to share and reuse

Universal Data AccessData Virtualization for all enterprise data -

on and off mainframe

DB2 Analytics

Accelerator

QMF for z/OS

Machine Learning

for z/OS

Open Data Analytics

for z/OS

© 2017 IBM Corporation

DB2 z ToolsRoadmap, Investment and Future Directions


Agenda

▪Value and Benefits of

Continuous Delivery

▪Key Focus Areas

▪Simplification

▪Future Strategy and

Direction


16Continuous Delivery – IBM Db2 Tools


Db2 Tools – Continuous Delivery Details

▪ What is it?

➢ Monthly release of PMRs and New Function

➢ In general, no new releases ….. unless necessary

➢ No activation level or level management

➢ Identification of Tools maintenance that is required for a particular Db2 engine activation level

▪ Why?

➢ More responsive to customers

➢ More responsive to ‘DB2 Engine’ continuous delivery

➢ Improved quality and communication

▪ Internal Team Changes

➢ Move to agile development process

➢ Better align Dev/QA around customer requirements and true customer focus on new functions


Significant Development Improvements

▪ Cloning Tool

Log Apply Improvements

Improved instrumentation

Improved consistent clone from image copies

▪ Loader

Consistent Clone – Log Apply, Instrumentation

Improved messages

▪ Automation Tool – Utilities Enhancement Tool

Improved instrumentation (SPs)

Data Sharing Improvements

Some functions moved to utilities• LOAD replace share level reference

• CONSTANTIF

▪ Query Monitor

Improved error recovery

Intercept compatibility checks

Common Collector

▪ Log Analysis Tool

Improved messaging

Better error handling & recovery during long runs

▪ Recovery Expert

Log Apply – Page checks

Dropped Object Recovery

Data Sharing improvements


Key Areas of Focus

Roadmap 2017 – 2018


Performance and zIIP Enablement

▪ Lower cost of running tools across

product line

▪ Faster processing at lower overall CPU

cost

➢More zIIP exploitation, parallelism,

improved I/O methods & algorithms

➢Cloning Tool, Loader, Log Analysis,

Performance Tools, Recovery –

throughout year


Performance and Reliability

▪ Log Analysis Tool Performance

➢Move more processes to 64 bit, More zIIP exploitation

➢Usability & Reliability

• Audit All use case, further simplification of options

• Possible integration into Recovery Expert UI

▪Cloning Tool

➢Move more processes to 64 bit, zIIP exploitation

➢Use INDEX log apply in combination with TABLE SPACE log apply

➢High Performance Tablespace Clone (Expect up to 90% preformance improvement)

• UTS – only have OBID in header/spacemap page

▪Recovery Expert

➢Spread and manage recovery accross LPARs (Data Sharing)


DB2 Query Monitor / OMEGAMON - Performance

▪Shared Collector

➢Exploit CQC in OMEGAMON

➢ Identify best collection method at lowest cost and exploit

▪Collector Improvements

➢Reduce defaults of collected elements

➢More zIIP enablement

➢Exploit new z instructions – detect hardware and load specific modules

➢Looking to bring overhead down significantly for default install


Simplification

of

DB2 Tools


IBM Db2 Administration Solution

▪Usability & Ease-of-Use Improvements Across Solution More automated processes

Simplified presentation of options

▪Cloning Graphical Interface Data Server Manager (DSM)

integration

Simplification of Cloning Process

▪DevOps workflows Integrate into development

environments

Provision schema & data

Automate change management


IBM Db2 Cloning Tool – Simplification / Integration

▪Significant Simplification of tablespace clone process ➢Reduction of clone options

➢Specify source / target, main option – clone from current / from IC – end point

➢Automatcally choose best option for index processing (rebuild / log apply)

▪Rest Enable all of Cloning Tool batch functions➢Good bridge into future project

➢Allow for better integration into Administration Migrate and Alter functions


Recovery Expert - Simplification

▪Backend Rewrite Project

➢Fewer recovery plans - Improved UI Flow

▪Usability

➢Recovery Preview mode

➢Recovery Progress Monitor

➢Recovery Expert GUI in Data Server Manager

▪ Future

➢Better integration with Automation Tool

➢Recovery Time Estimates


Automation Tool and Utilities Enhancement Tool

▪Simplified Utilities Management

➢Setup hierarchy of utilities

➢Rest enable utilities generation (usable across tools)

▪ Integrate Performance Data

➢Better utilities (REORG) recommendations based on true performance analysis

➢Run fewer reorganizations - Less CPU – Fewer outages


Roadmap: Smarter Utility Decisions Based on Real World

Performance

▪ Pain Point

➢ DB2 Table/Tablespace Reorganizations can be expensive, and still cause a slight disruption

➢ Goal is to reduce reorganization executions

➢ Current reorganization indicators help, but there is room for improvement

▪ Benefits

➢ Execute reorganizations ONLY when real world SQL performance has degraded

➢ Cut data reorganizations by up to 90%. Eliminating CPU, memory, application impact & terabytes of disk space

▪ Details

➢ Baseline performance metrics are collected for designated DB2 application(s)

➢ Ability to detect real world performance degrading down to the object level in SQL queries

➢ RTS indicates disorganization of table data…..

➢ If performance is not degrading due to object disorganization, the reorg will not be recommended or executed


29


Strategy


31Future: Rest Enable Everything

▪Administration Tool

Provision Schema (Available)

Object Comparison (Available)

Provision Data

Schema Tear Down

Versioned Schema Management

▪Cloning Tool

High Speed data provision

▪Recovery

Recovery Service (Data Reset Service)

31


Analytics Solution – IDAA Loader

▪Additional Data Sources➢ IDMS – In Beta

➢Cloudera (3rd QTR)

➢Direct Support for MS SQL Server (DRDA)

➢JDBC support for many data sources

• Will expand to many data sources

• Performance will not be optimal

▪Accelerator Clone and Sync➢Ability to bring two accelerators in sync

➢Clone Use Cases

• Additional or accelerator upgrades

➢Reporting Use Cases

• Monitoring of IDAA High Availability configuration

• Alerting when ‘out of sync’

• Automated reload or refresh when out of sync occurs


34IBM Db2 Analytics Accelerator Loader V2.1 - Latest Enhancements

• PI70981: Backup & Recovery of Accelerator-only Tables

• PI75625: SMF base map fixes

• PI75624: Create unique key for SMF parent/child relationships

• PI67510: Support to load accelerator groups*

• PI63948/PI65565: Support for direct access to Oracle as a data source

• PI59666: HA Load from Db2 for z/OS or external source

• PI58663: Support for ADABAS as a data source

• PI56636/PI57068: Support for loading data from a remote server

• Maintenance PTFs:

ftp://public.dhe.ibm.com/software/dmtools/ptflist/DAAL210.html

*: An accelerator group in this context refers to a group name in SYSIBM.LOCATIONS that groups multiple accelerators

together


35

Recovery Expert is an intelligent recovery manager

• Reduces the complication of recovery!

– Analyzes your recovery resources

– Easy to Choose Recovery Points

– Create Detailed Recovery Plans

– Provides cost-based recovery estimates in real-time

– Detects and recovers related objects

– Detects objects not updated since last recovery point

and will not be recovered

– Utilizes new technologies and recovery strategies

– Generates the optimized JCL

– Provides reporting for review and control

• Minimizes the exposure & risk!

– Pro-actively create recovery plans to validate the

ability to recovery

– Health check functionality validates the ability to

recover to selected points in time


Recovery Expert New Features

▪ Log Based Dropped Object Recovery

➢ Complete end-to-end recovery of a dropped object from the log

➢ Recovery of DDL Schema, Data, DCL (Security), rebinding of packages and plans

➢ Automated recovery to drop point

▪ Redirected Recovery

➢ Restore to another system or different set of objects on same system

➢ Validate recovery point or test recovery time objective

▪ Transaction Recovery

➢ Generate and execute UNDO SQL for single transaction or time range

➢ Filters and visual aids to help identify work to back out: User, Package, URID, etc.

➢ Simple, visual UI to walk user through the process

▪ Mass Application Recovery

➢ Set priority of objects to restore high value assets first

➢ Useful when recovering large groups of objects


Db2 Log Analysis Tool – Released (V3.5)

▪ Simplified ISPF User Interface

– Task based – Reporting, Recovery, Replication

– Only present options that make sense per task

• Keep options to one screen

• Present options in ‘end user’ language

– Remove legacy options

– Supply best practice defaults

– Keep classic menus for existing power users



▪ Improve performance and use of 64 bit processing

– Process larger amounts of log more reliably

▪ Exploit more zIIP processing

– Reduced GP by 20% for many processes

▪ Improve catalog reporting

– More detail for DCL, DDL type changes in reports

– Improve performance

▪ Report on SYSCOPY events in range of processing

– Help users better understand log analysis path or processing

▪ Allow more parameters to be changed in batch job

– Reduce the need for ISPF interface



▪ Db2 Command Reports

– All Db2 Commands produced in a report

– Honor filtering for commands

• Userid, Authid, ….

▪ Eliminate need for ISPF interface

– Once JCL is generated once…

– Allow most all parameters to be changed in batch JCL


▪ Compare captured workloads to saved baselines

➢Determine whether application changes have degraded

performance.

➢ Identify which packages have SQL statements with

degraded performance

➢ Identify SQL statements that are new and could use

tuning

▪ Available Actions

➢Apply filters and review Comparison result

➢Review comparison result

➢Generate HTML comparison report or in csv

➢Generate new query workload for tuning & perform

analysis

IBM Db2 Query Monitor for z/OSPrevent Problems Before They Impact the Business


▪ Why did the Db2 Optimizer choose

that path?

▪ Query Monitor provide real world

values to optimizer for host

variables (distribution)

▪ Optimizer makes better decisions

knowing the most used values for

host variables in where clause!!

▪ End Result: CPU and Elapsed

Time Savings!!!!

Host Variable Collection & Selectivity OverrideIBM

Solution Exclusive!

SELECT * FROM

EMPLOYEES WHERE

SALARY BETWEEN ?

AND ?


Advantages of Query Common CollectorMinimum Resources / Minimum Overhead / Maximum Functionality

42


What‘s new in Db2 Automation for z/OS

▪ Further integration with IBM Db2 Autonomics Director for z/OS

– Db2 Automation Tool first tool to exploit Autonomics Director

– Can take actions supplied by any other tool (or user program) that participates in

Autonomics

• Example: IBM Query Monitor can advise when not to run a REORG

– Options were added to help fine-tune the maintenance workload by adjusting priorities

for actions, symptoms, and objects.

• Ensures most important jobs are run first when the maintenance window opens.

See the web cast by customer PennDot about their use of

Automation Tool and Autonomics Director.

• adding intelligence to the way they managed utilities

• reducing the maintenance window

• reducing CPU costs and improving availability


Addressing the Needs of DB2 Administration


Analytics Solution – Performance Tools

▪Better solution to store performance DB on IDAA

➢Phase 1

• Adjust PDB to be compatible with IDAA – Complete

➢Phase 2

• Integration with IDAA Loader

• Direct load of performance data to accelerator

ibm db2 tools update & ibm data virtualization manager for ... · pdf fileibm db2 tools...

Documents