an international mainframe consolidation project: strategy ... · =c proven™ professional...

=C Proven™ Professional Knowledge Sharing The article - “An international Mainframe consolidation project - strategy and technology

Copyright 2007 EMC Corporation. All rights reserved. of 24

”

An International Mainframe Consolidation Project: Strategy and Technology

EMC Proven Professional™ Knowledge Sharing November, 2007

Michael Zimmerman [email protected]

and

Richard Herbst [email protected]

mailto:[email protected]

mailto:[email protected]

=C Proven™ Professional Knowledge Sharing


The article - “An international Mainframe consolidation project - strategy and technology”

Table of Contents

Abstract ............................................................................................................................................................................ 3

Customer Profile .............................................................................................................................................................. 3

Introduction....................................................................................................................................................................... 3

Location Overview............................................................................................................................................................ 4

Task Setting ..................................................................................................................................................................... 5

Wide Area Network .......................................................................................................................................................... 7

Tape Migration ................................................................................................................................................................. 8

Data Migration................................................................................................................................................................ 10

Migration of STD Volumes ............................................................................................................................................. 10

Migration of R1 Volumes................................................................................................................................................ 13

Migration of R2 Volumes................................................................................................................................................ 18

Migration of BCV-Volumes............................................................................................................................................. 18

Data Migration Tests ...................................................................................................................................................... 19

Host connectivity ............................................................................................................................................................ 22

Cache ............................................................................................................................................................................. 22

Remote replication ......................................................................................................................................................... 23

Backend Directors.......................................................................................................................................................... 23

Parallel Access Volume ................................................................................................................................................. 23

Strategy – The way to win.............................................................................................................................................. 23

Relationship.................................................................................................................................................................... 23

Confidence ..................................................................................................................................................................... 24

Teaming ......................................................................................................................................................................... 24

Result ............................................................................................................................................................................. 24

Author Biography: .......................................................................................................................................................... 24

Disclaimer: The views, processes or methodologies published in this article are those of the authors. They do not necessarily reflect EMC Corporation’s views, processes or methodologies.




Abstract IBM dominates the Mainframe market particularly since they are able to bundle their solutions, making their offering very attractive to customers around the world. One of the largest outsourcing customers in Germany is consolidating all of their IBM Mainframes into one central location. This article describes EMC's first deal within this European Mainframe Consolidation project. It will present the strategy and technology used in the account. The Analysis, Concept and Migration components will be explained in detail. This article will demonstrate why working with EMC’s methodology is an advantage for the customer.

Customer Profile The customer is an international information technology services company with a primary presence in Europe. They transform their client’s vision into results by applying consulting, systems integration and managed operations. The company’s annual revenues are more than EUR 5.4 billion in 2006, and they employ over 50,000 people in 40 countries. The customers’ goal “is to exceed customers’ expectations as a permanent business partner and to deliver business value by proactively meeting their changing requirements with innovative, high-quality services and solutions. We are convinced that continuous quality improvement with our customers and suppliers is the base for our company’s successful future. We attract, develop and retain the best people for our company. We value people and empower them to build a winning team. Long term value for our shareholders is key to our success and a prerequisite for our future. It results from entrepreneurial behavior, creativity and teamwork at all levels.”

Introduction IT services and solutions costs are growing inexorably. Even outsourcing providers cannot ignore this pressure and are planning the European wide consolidation of their Mainframe environments. After a customer based study of their business, management chose Germany as the location for their “European Mainframe Hub.” Other Mainframe applications are operating in the Netherlands, France, Italy and the United Kingdom. The centralization will begin with the consolidation of the Mainframe-Hosts and Mainframe-Storage systems from the Netherlands to Germany. The customer decided on a European Mainframe platform based on the following considerations:

1. Cost Saving (infrastructure and personnel) 2. Stabilization of Market Position (Mainframe) 3. Optimization of the basis for negotiation for Mainframe host.



”

Location Overview The customer operates IT infrastructures in over 40 countries world-wide. Each location has an install base consisting of different Storage manufacturers. The preferred manufacturers are Hitachi Data Systems (HDS), Hewlett Packard (HP), International Business Machines (IBM) and EMC. IBM has been the exclusive Storage supplier in the Mainframe Business space in Germany. This project changes that.



”

Task Setting After deciding on the German location, the company requested proposals for Mainframe Systems storage. They also requested that each proposal include consolidation and migration plans. To offer a valid proposal, EMC analyzed the existing systems’ landscape (capacities, applications configurations, performance analysis of each individual system) and provided a first draft design. After numerous discussions with the company’s technical personnel, and co-ordinating with EMC specialists, a detailed Statement of Work was presented to the customer. Due to the extensive elaboration in the Pre-Sales phase, EMC was able to demonstrate similar experiences with Mainframe migration and consolidation. The inclusion of adequate references contributed to the manager’s technical accountability to the customer. The Way it Works - Technology The ‘state analysis’ illustrated the framework parameters that we had to consider with the migration. From the productive systems in the Netherlands, 20 customers are active on 23 LPARs (Logical partitions on Mainframe Servers). EMC Specialists in the Netherlands and Germany analyzed the productive systems. They pulled Performance reports at a representative time and EMC Performance Gurus evaluated them. On the basis of this information and additional details from the customer environment, we defined new targets. This example shows total throughput from/to host in Kilobytes, System: EMC Symmetrix DMX-2 “1000”:



”

During project planning, we included milestones to make the transition as smooth as possible. The migration included eight steps. Each of these steps included at least two tests to ensure the migration was secure (verified and eventually fine tuned the implemented procedures).

The overview of the system landscape (see above) illustrates the pure Storage infrastructure. The MIPS (million Instructions per second) amounts in the Netherlands are 3.000, whereas in Germany Mainframe systems already have an aggregated output of 12.500 MIPS.



”

Wide Area Network EMC specialists also addressed WAN (Wide Area Network). EMC helped the customer select the technology (IP [Internet protocol], DWDM [Dense Wavelength Division Multiplexing], Dark fiber, STM [Synchronous Transport Module]) and the choice of the carrier. Thanks to contacts with Longwave-carriers, we negotiated an excellent price on behalf of the customer. EMC finally placed the order for the WAN equipment; Ciena was the EMC SELECT Partner.



”

Tape Migration The data is kept on tapes for Backup and had to be transported from the Netherlands to Germany or migrated by Wide Area Connection. At the Germany site, the tapes were to be copied on the TS 7700 system that IBM provided. The possibilities and the production-accompanying procedure via “Copy Cross™” by EMC in interaction with the manufacturer “OpenTech” were addressed. Migration without off-line time was one of the requirements. EMC recommends using CopyCross (Software for MVS and z/OS) during data migration to drastically reduce complexity. With CopyCross, the Tape Data Migration was performed with the same technologies and procedures being used for DASD (direct access storage device) Data Migration.

CopyCross enables the customer to:

• concentrate on the data rather than the media

• have the same procedures for tape and DASD data to drive and control the migration

• have DASD and tape data available in Germany for test and production at the same point in time

• have the same procedures to perform validation tests during the data migration

• perform the same procedures for production cut-over from Netherlands to Germany

• reduce the risk of tape loss (data loss) by using the data migration links rather than transporting physical tapes.

The customer decided, despite these clear advantages, for the traditional way (due to management’s desire to minimize risk). This further helped with the finalization of the Storage of migration project.




Data Migration The customers’ online mainframe data is residing on EMC Symmetrix® storage systems in the Netherlands and needs to be migrated to the data centers in Germany. The EMC Data Migration Service provides for migration of data from existing storage (source data devices) to a new storage infrastructure (target data devices). To ensure successful data migration, the service includes Planning and Design to document the detailed requirements for the customer prior to the migration. EMC exploits its proven SRDF technology to perform the data migration tasks in a timely and effective manner. The migration services were divided into eight steps. The first step (stage zero) includes the migration of the customers’ test systems. This step also includes the verification of all migration tasks and their eventual fine tuning. The source systems’ SRDF ports in the Netherlands will be connected to fibre channel switches. These switches reduce the number of ports required on the Ciena long-distance equipment. They also provide the buffer-to-buffer credits needed for the improvement of data transmission. The following pages detail the technical solution steps.

Migration of STD Volumes Standard (STD) Volumes, volumes that are not SRDF (Symmetrix Remote Data Facility) protected, will be migrated using SRDF in data migration mode (SRDF/DM). The migration is performed in “Adaptive Copy Disk Mode”. In this mode, all tracks will be transmitted to their target volumes in parallel to production (FullSync). Tracks may be updated after they have been transmitted to the target. These updates are recorded as “invalid tracks“ in the track table belonging to the source volumes. Multiple updates to the same track are recorded once. After the completed FullSync, all “invalid tracks“are again transferred to their target. The reiterating transmission of “invalid tracks“ automatically reduces their number, the difference between the source and target volumes. The difference can be queried at any point in time in terms of “invalid tracks“. To complete the data migration, we change the SRDF mode from “Adaptive Copy“ to “Synchronous“. This enforces the transmission of the remaining “invalid tracks“. All new updates are then immediately transferred due to the synchronous mirroring option. The data migration is complete when the number of invalid tracks is zero for all volumes of this migration stage. You can verify this using query commands of the SRDF host component.



”

Finally the SRDF relation used during the data migration will be removed to make the data available in Germany for the production move. The following pictures illustrate the SRDF/DM operation. Intial situation:

The actual data (blue) resides in the Netherlands. The target volumes in Germany are empty (white).

Step 1:

The SRDF/DM relation is built using the CreatePair command of the SRDF host component. The data migration starts (shown by the blue tracks on the target R2 volume). As a prerequisite, the source and target volumes have been made capable for this procedure by assigning

the DynRDF attribute to these volumes by setting the binfiles of the source and target systems accordingly. Technically all tracks of source volumes are marked as „invalid“ in the associated track table (red „1“s) when establishing the SRDF relationship. The progress of the data migration will also be recorded in the track table (by „0“s).



”

Step 2: Updates may occur in the source volumes (orange tracks) while the data migration proceeds.

These updates are recorded in the track table. For the track that already had been transmitted, the „0“ in the track table is changed to „1“ (lower left corner of the track table in the given example). For the update to the track that was not yet transmitted, the „1“ remains a „1“.

Step 3:

This picture shows adddtional updates (yellow tracks) until the completion of the transmission of the last track. These updates are recorded in the track table accordingly.

Step 4: All tracks that have been changed during the last cycle will be transmitted periodically. If, for example, the SRDF links are not available due to network problems the same procedure is used, i.e. the updates are simply recorded in track tables. Their transmission resumes after the network problems have been resolved and the SRDF links have been restarted. SRDF/DM works on Symmetrix systems as a background task. Host I/O operations always run as foreground tasks and therefore have priority over data migration.This reduces the risk of any impact to production systems. Using track tables provides additional advantages. Bandwidth can be saved because tracks that are updated multiple times during a cycle are transmitted once in the next cycle. Performance will be optimized by a balanced utilization of the backend.



”

Shortly before the production move, the SRDF relationship will be switched from data migration (adaptive copy) mode to synchronous mode. This causes the last invalid tracks to be transmitted. Eventual further updates will be mirrored synchronously from the Netherlands to Germany.

Finally, the SRDF relationship will be dropped using the DeletePair command of the SRDF host component. The data is available at its target location. New updates will now occur on the target volume (purple tracks). Later, after production is started in Germany, we will perform a binfile change to remove the DynamicRDF attribute from the target volume.

Migration of R1 Volumes For R1 volumes, an SRDF mirror already exists in the source configuration between the two Netherland datacenters. We will create a second remote mirror in a data center in Germany (Concurrent SRDF). The second mirror will run in SRDF/DM mode (Adaptive Copy Disk mode) as described above for STD volumes. We must protect R1 volumes in the Netherlands by SRDF in their target location immediately after the production move. A special multi-hop procedure is required to avoid long synchronization times during the production move. This procedure deploys the advantages of the EMC SRDF and TimeFinder products.



”

We have to remove any existing BCV relationship to enable the Concurrent SRDF feature used for the migration of the R1 volumes (TimeFinder CONTROL DELINC command). We will rebuild the BCV relationship after the R1 volume has been made productive in its target system.

We will create the SRDF/DM relation in parallel to the synchronous SRDF mirror. All tracks of the source R1 volumes will be marked as ‘Invalid Tracks’ in relation to their target volumes in (R2“). The existing SRDF relation (R1 to R2’) remains unchanged. All steps regarding the data migration from the Netherlands to Germany will be performed in the same manner as described. This includes the maintenance of the track tables, eventual resynchronization procedures, etc.

http://dict.leo.org/ende?lp=ende&p=/gQPU.&search=synchronous



”

The R1 R2’- and the R1 R2“-relations are controlled independently. For example the synchronous mirror has its own R2’ Track Table. For clarity these internal tables have been committed from the pictures. Using the TimeFinder Establish command, we will create a second copy of the data transferred to the Netherlands within the same target storage system. The TimeFinder target volume is called BCV (Business Continuance Volume). This BCV is required to realize a synchronous SRDF mirror between the Germany data centers. RAID-5 protects all logical volumes. From the external view, the R1/BCV volumes act like BCVs. Internally they are realized as TimeFinder/Clone volumes, BCV behavior is important for the planned data migration. For the production move, it is important to have all data on locally protected (RAID-5) volumes. Both goals are achieved by using the TF/Mirror emulation provided by Enginuity 5x71.

Synchronization between the data centers in Germany will be started as soon as the R1/BCV volume is split off of the R2” volume. In the above picture this has been indicated by the blue R2 Germany volume. While the R1/BCV is split, updates may occur on the Netherlands’ production R1 volume. These updates are mirrored synchronously in the Netherlands and also transferred to Germany. Updates are indicated by red tracks in the above picture.



”

The tracks updated during the “Split-Off” will be incrementally copied to the R1/BCV volumes after a TimeFinder “Re-Establish” as shown in the picture below.

While the R2 volumes are established to the R1/BCVs there is no mirroring from R1/BCV to the R2 volume. The synchronization will start when the BCV is split off. For the data migration, the cycles used to establish and split the BCVs in Germany will become shorter and shorter. This also means that the number of updated tracks will decrease. The number of tracks that have to be synchronized between R1/BCV and R2 in Germany will decrease accordingly. For the final production move, only a fraction of the total capacity has to be mirrored to establish the disaster recovery environment in Germany. Illustration follows on next page.



”

For the final production move, the R1/BCV volumes will be split off the R2“ volumes. This causes the last tracks to be synchronized to the R2 volumes in Germany. The picture above shows some tracks already updated by the production in Germany (violet tracks). Production in Germany will use the R1/BCV and/or the R2 Germany volumes. The capacity required for the intermediate R2 volumes will mostly be satisfied by space that is currently not used, e.g. because it belongs to capacity of later migration stages or it is planned for BCVs of the current stage. R2 volumes will be on the site where capacity is available; for production this may be the R2 site. In this case, the mirror direction will be swapped right before the production start (or test) in Germany. Some cleanup tasks have to be performed after production starts in Germany. Illustration on following page.



”

• The source volumes will be made unavailable (Not-Ready-to-Host) in the Netherlands. • The TimeFinder relationship in the Germany data center will be removed (TF

CONTROL DELINC). If a BCV existed in the Netherlands it can now be established on the R1 or on the R2 side as needed.

• The BCV flag will be removed from the R1 volumes (binfile update); this flag was required for migration only.

• The SRDF relationship between the source volume in the Netherlands and its intermediate target volume (R2”) in Germany will be removed (SRDF DeletePair).

Migration of R2 Volumes The migration of R2 volumes is included in the migration of the R1 volumes as described above.

Migration of BCV-Volumes BCV-Volumes are not explicitly migrated from the Netherlands to Germany. The BCV volumes have to be recreated by TimeFinder Full-Establish in Germany after the production move.



”

Data Migration Tests SRDF is the only remote replication software that enables a differential resynchronization after updates on the source and on the target volumes. This allows tests at the target location during data migration without requiring a full synchronization. Data migration has to be suspended to perform tests at the target location. The target volumes must be made available to the LPARs in Germany.

Starting point: The data migration is in process. Source and target volumes are in sync, i.e. the number of invalid tracks is zero. During data migration, the target volumes are protected against updates outside of SRDF/DM. This is achieved by the volume attributes NRDY2HOST (Not-Ready to Host) and R/O (Read-Only).

NRDY2HOST makes the volume “invisible” to the server, i.e. this volume cannot be brought online. R/O also ensures that no host write operation can be issued against this volume. Data migration will be suspended for the test in the target location ( ).

Afterwards, the target volume will be made available to the MVS host ( ). Therefore the volume attributes RDY2HOST (Ready to Host) and R/W (Read/Write Enabled) will be set.

The specific characteristic of SRDF is the existence of track tables on either side, i.e. for the source and for the target volume. These track tables are maintained independently.



”

With the suspension of the remote mirroring (in our case the data migration) all updates to the R1 (source) volume are recorded in the associated R2 track table. The R1 volume owes these updates to the R2 volume. In the example, tracks 2 and 4 have been updated on the R1 volume (yellow).

The R2 volumes have been made available to the LPARs in the target location (Germany). Updates to the R2 volumes are recorded in its associated R1 track table. Theoretically, the R2 volume owes these updates to the R1 volume. In the data migration, it just records the differences between source and target volume from the R2 point of view. In the example, tracks 4 and 6 have been updated on the R2 volume (purple).

Data migration must be resumed after the tests in Germany are complete. For the incremental resynchronization, SRDF combines the track tables from the R1 side (production updates) with the track tables from the R2 side (test updates). The result of the logical “Inclusive OR” shows the difference between R1 and R2, i.e. which tracks have to be resynchronized. In our example tracks 2, 4 and 6 are different.



”

At first, the attributes of the target volumes will be set to resume the data migration ( ). Then the track tables will be merged (SRDF Refresh, ). Finally the data migration will be resumed ( ). The synchronization direction is R1 R2.

According to the merged track table, the new contents of tracks 2 and 4 (updated on source site) and the old contents of track 6 (updated on target site) will be transmitted. These actions will be accomplished thru SRDF host component commands.

Consistency of Data in Target Configuration Data consistency on the target site will be achieved by:

1. stopping all LPARs belonging to the actual migration stage in the Netherlands 2. transferring all data successfully to Germany (number of “invalid tracks” is zero) 3. logically suspending the SRDF/DM connection

The tests to be performed drive what level of data consistency is required on the target system. It might not be necessary to stop the Netherland’s LPARs for simple tests. Error messages must be accepted during the test. These steps are strongly recommended for a test including application tests on any kind of database. In either case, the differential resynchronization has to be performed as described in “Test during the data migration”.



”

For the target configuration, two storage systems Symmetrix DMX-3 “2500” will be installed in the two locations.

The Symmetrix DMX-3s will be configured according to the customers’ production and data migration needs. The configuration was chosen based on a performance trace (STP). These are the specifications for the EMC Symmetrix DMX-3 systems:

Host connectivity Each of the proposed Symmetrix DMX-3 possesses three (3) FICON directors, a total of 12 ports. With this configuration the systems are capable for a maximum six (6) to eight (8) FICON directors, i.e. to a maximum of 24 to 32 FICON ports.

Cache Each of the proposed Symmetrix DMX-3 possesses eight (8) 16GB Cache directors, a total of 128GB cache. The cache is mirrored and the usable cache size is 64GB. With this configuration, the systems are capable for a maximum 512GB cache, i.e. a maximum of 256GB usable cache.




Remote replication Each of the proposed Symmetrix DMX-3 possesses two (2) FibreChannel directors, a total of 16 FC ports. 4 ports are to be used for SRDF. A maximum of eight SRDF can be used with this configuration.

Backend Directors Each of the proposed Symmetrix DMX-3 possesses two (2) FibreChannel director pairs, a total of 16 copper fibre ports for the access to the physical 146GB drives. We can use a maximum of four backend director pairs with this configuration. We plan to further consolidate the additional backend directors within the scope of the project “European Mainframe Hub”.

Parallel Access Volume Parallel Access Volume (PAV) is not licensed for these EMC DMX-3 systems. We will reserve alias device numbers in each configured logical control unit (LCU) for later PAV licensing/ We documented all steps in a Statement on Work that was accepted by the customer and EMC. That creates the security of an approved project and the fulfillment of the customers’ expectation (Win-Win situation).

Strategy – The way to win The journey is not the reward, but rather the successful execution of the consolidation and migration project that achieves the customers’ goals. Until the Project Review Meeting, the customer continues to experience a difficult path. Prognostic savings cannot be proved until the project’s conclusion. It applies here to offer various alternatives to fit the customer’s individual profile.

Relationship The criticality of the relationship between customer and the company is often underestimated. We must maintain trusting relationships with both management and the technical staff involved.




Confidence All companies offer solutions. How does a customer select the right one? Partly through the understanding of concepts, whitepapers and reference reports; but the customers’ confidence in the IT partners’ ability is more important. Even if the customer only used a small part of EMC´s capabilities, the advantages we offer were apparent. Only through mature vision and a proven strategy is it possible to execute a successful business concept.

Teaming The external representation (“one face to the customer”) was improved with a customer-oriented Service unit, Professional Services, creative global Financial Services and Account Management (“Account Intelligents”) of the entire Sales Team.

Result This project proves that close co-operation and partnership with the customer brings rich results. The exclusive focus on the customer’s business needs and the technical expertise, combined with highly flexible financing, resulted in a collaboration with EMC. Much has been learned in this project, enabling further consolidation steps in other European Countries, again involving EMC Sales Personnel. The active Presales phase amounted to eight months. The contract was signed at the end of 2006 and the EMC project team is in the Netherlands and Germany working on the successful migration.

Author Biography: After training as an electronics engineer and successfully achieving a diploma in Communications Technology (specializing in data processing) at the Cologne University for Applied Sciences, I assumed a Presales Consultant Network Technology role, working for a National IT-Integrator for three and a half years. Since 2001, I have been working for EMC as a Presales Consultant for various International Enterprises in Germany.

an international mainframe consolidation project: strategy ... · =c proven™ professional...

Documents