Spectrum Scale 4.1 System Administration

Spectrum Scale

& Elastic Storage Server

Sizing & Engagement Planning

Session 2 of 2

© Copyright IBM Corporation 2015

Building a Spectrum Scale or Elastic Storage Server Solution

• Using the information from a Detailed Discovery Build a Spectrum Scale Solution

– What is the Use Case?

• Single or Multiple?

– What is the signature Use Case IO workload?

• Avg File / Object Size, and IO patterns?

– What are Key points of Integration?

• Any 3rd Party tools, or existing Hardware Requirements?

– Has the client defined Access Requirements?

• Authentication, Networking, or 3rd Party Solution tie in?

– Has the client defined Application and Clients for Data Access?

• Client HW, SW, OS, Applications or Appliance integrations?

– Has the client defined Performance Goals?

• Single or mixed workload performance demands.

• Has the client defined Capacity Goals?

– Total Capacity as well as capacity in Tiers.

• Has the client defined Tiers and Metadata workloads?

– All disk or Disk & Tape

• Has the client defined Distribution of Data Goals?

– Single Site, Multi-site, Global Distribution?

Has the client defined HA or DR Requirements?

Has the Client Defined a Budget to work with?

Has the client defined Training & Implementation Goals?© Copyright IBM Corporation 2015

To be truly successful

you have to hit the

Target with all of your

arrows!

Sizing & Architecting Solutions

• I like getting the bulk work done quickly first, leaving time for

the details.

• So I look at ways to break it down.

– DR Replication requirement (2site means 2X the config)

– HA for each site (could mean 2X Building block redundancy in each site)

– Number of Use Cases (could mean separate file systems for each Use Case)

– Number of sites/Locations

• Capacity (addressed by Each Site – Useable Capacity)

• Tiers (understand the tier & Tier Capacity)

• Clients (how many active concurrent clients max)

• Client Access Protocols (How will clients connect & how many / protocol)

• Client Authentication (make sure we will support the demand)

– Backup/no Backup (TSM or are they hoping for something else)

– 3rd party Solution Integration (almost always requires special TDA)

© Copyright IBM Corporation 2015

Elastic Storage Server Sizing Tool

© Copyright IBM Corporation 2015

Introduction to the Elastic Storage Server Sizing Guide(IBM Internal Use Only)

ess_capacity_perf_calc.xls by Indulis Bernstiens

Work with an SDI Team Specialist for working up the sizing

Introduction to Spectrum Scale license sizing

© Copyright IBM Corporation 2015

Work with an SDI Sales Rep or Specialist to Size your licensing requirements,

they have access to tools for sizing licenses for servers and clients.

GPFS 4.1 Quotation Estimator.xls (IBM Internal Use Only)

Spectrum Scale Protocol Node Sizing Tool

© Copyright IBM Corporation 2015

We will use an internal Protocol Node Sizing Tool after GA

Announcement for Sizing Protocol Nodes(IBM Internal Use Only)

Spectrum Scale protocol Node Sizing Guide.xls by Mike Taylor

Work with an SDI Team Specialist for working up the sizing

Review a sample Opportunity Discovery for feeding the Design

© Copyright IBM Corporation 2015

ACME Corporation Discovery DocumentACME-OpNum-ABC123-SpectrumScale-Pre-Sales-Discovery-Checklist-input.pdf

Site 1 New York Site 2 Philly

Work with Spectrum Scale SME’s to share

the discovery and validate the configs

Work with Spectrum

Protect (TSM)

specialist to Spec

out TSM Config)

Now that we know what we want let’s get to pricing

• Introduction to e-Config

– Work thru the slides or actually work thru an e-Config experience

© Copyright IBM Corporation 2015

ESS configuration questions, hints, and tips

Here are some hints and tips for ESS

ESS NSD server configurations:

There are 2 types of ESS NSD (disk) server node configurations:

o NSD servers with 128GB each for ESS models with 1 or 2 disk drawers

o NSD servers with 256GB each for ESS models with 4 or 6 disk drawers

o NSD1 "owns" the 1st disk drawer, which always has 2 x SSDs (either 785 GB if the rest of the drawer is 785 GB

SSDs, or 387GB if the rest of the drawer is 387 GB SSDs or disk). NSD2 "owns" the rest of the disk drawers

(from an e-config configuration aspect, not a setup or operational aspect!).

Do I need Spectrum Scale Entry, Standard, or Advanced? In almost all cases you need Standard. Advanced adds

encryption, please talk to an Elastic Storage GPFS specialist regarding this. Entry level removes quotas and other

features which are used in the ESS GUI, so it is not recommended.

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

How many HMCs do I need? Probably 1 per physical location that has ESS units. A physical

location would typically be a Data Centre.

How many Management servers do I need? The management server does 2 things, it is used to

reinstall ESS server images (using xCAT or PCMAE), and can also be used as a Spectrum Scale

quorum node (you need at least 3 x Spectrum Scale servers licensed with Spectrum Scale Server

licenses per Spectrum Scale cluster). So this can be a bit more complex.

o The Management server can be a quorum node for one Spectrum Scale cluster with a

single ESS

o The Management server can be a quorum node for one Spectrum Scale cluster with

multiple ESS units

o

If you have 1 or more ESS units in a single Spectrum Scale cluster at each site, then

you only need one Management node per site

If the customer requirements are for multiple Spectrum Scale clusters, or a "stretch

cluster", then contact a Spectrum Scale Elastic Storage expert to determine the

correct configuration!

All Spectrum Scale clusters need to meet quorum rules, you may need

to add Power8 servers to act as quorum nodes

It is possible to use LPARs and have a single Power8 server act as a Quorum node for multiple

Spectrum Scale clusters- configuration has to be correct to support multiple LPARs and networks!

Check with a Spectrum Scale Elastic Storage expert!

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

Who prices this- POWER or Storage?

o POWER brand if all ESS units are "GS" models (all components are POWER brand)

o STORAGE brand if all ESS units are "GL" models (as the 60 disk drawers are most of the cost)

o TBD if there is a mix!

6TB disks? Available Dec 2014 as an RPQ, shipping 1Q2015 (est.) Please use this link to a Notes DB

Notes://D03DBL02/85256E69007FDD48. Link When you have this link open, please select Create Request and then No for a

blank form. Include the basic configuration (number of GL6s, number of 1818-80Es). Please fill out the form and make sure to use

the customer number that will be used when ordering this solution.

"High density" multi-rack ESS configurations- these are not standard like they are for GSS, for more

information see the ESS High Density configurations page

Max Headroom: The IBM 7014-T42 racks are built in 2 sections, the top section can be removed if it has

to be moved through a low doorway or lift

o Additional paid-for IBM hardware engineer time is required, and should be planned for and priced if

this is required.

o Enterprise V2 racks don't have this facility.

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

3 x Infiniband FDR adapters (#EL3D), PCIe-3 x16, with dual QSFP+ ports to support Optical or

Copper cabling

Probably a Mellanox ConnectIB MCB194A-FCAT (User Manual here)

3 x 10 GbE PCIe2.0 adapters (#EL27) with Dual SFP+ sockets/ports for either Copper or optical

cabling

This currently forces copper cabling (copper cables plug into SFP+ ports on the adapter

and switch)

Could use Mellanox optical transceivers instead, do not use other SFP+ transceivers

without verifying compatibility with the Mellanox adapter!

3 x 10 GbE PCIe2.0 adapters (#EL2Z) with Dual SR ports (built in transceivers and "LC"

ports) for optical cabling

Probably a Mellanox MNPH29D-XTR with 2 x preinstalled SR transceivers

MFM1T02A-SR, LC connectors to standard 10 Gbit SR cables (up to 300m with OM-3

cables, up to 100m with OM-2 cables)

Maximum performance of ESS unit may not be achieved using 10 GbE network option,

especially if one ESS NSD server node is offline (planned or unplanned outages) < 6

GBytes/sec in this case

#ECBD (5m) and #ECBE (25m) cables are LC-LC optical OM2 standard cables

Now that we know what we want let’s get to pricing

• e-Config

© Copyright IBM Corporation 2015

Now that we know what we want let’s get to pricing

© Copyright IBM Corporation 2015

Additional Products tab

o Using Customer provided switches

Select the ESS just under Global Settings (top centre panel), Additional Products tab for the ESS, then Delete Product the

switches you do not need

Note that it is recommended to at least have a 7120-24L switch included, even if the customer has their own 1GbE network,

to "isolate" the ESS from any issues, and also to allow the installation of the ESS to be done independently of any interaction

with the customer's network.

The biggest cause of reported "Spectrum Scale problems" turn out to be customer network issues, so ensure the customer

understands their network must be reliable and redundant for Spectrum Scale traffic!

o If not using 10 GbE adapters

Delete Product the 7120-64C switch (this is a BNT G8264, now Lenovo)

Copy Product the 7120-48E and (this is a BNT G8052, now Lenovo)

e-config has a bug as it adds a 7120-48E and a 7210-64C which is incorrect. Change to 2 x 7120-48E switches. If

customer has own reliable 1GbE network which is easily available at implementation time, then remove both 7120-

48E switches.

This supports network switch redundancy via the use of VLAG if the 1GbE network is used for Spectrum Scale traffic

See Chapter 11 Virtual Link Aggregation Groups in the G8025 Manual or the G8264 Manual

o If using 10 GbE adapters and need to configure 10 GbE "leaf" switches

Copy Product the 7120-64C so you have 2 x 64-port 10GbE switches (with 40GbE uplinks)

o Redundant 1GbE switches (optional): If your 1GbE network is essential for operation as a reliable network (e.g. using 2 site

stretch cluster and 1GbE is the "common" network).

Select the 7120-48E and Copy Product so you have two 1GbE switches

Now that we know what we want let’s get to pricing

© Copyright IBM Corporation 2015

Management server

o Change network adapters from 10GbE if required: change network adapters to match the ESS NSD servers

Note that you may not need 3 x Network adapters, as long as your customer is not running multiple "subnets" or

networks, or has a shared 1GbE network

Spectrum Scale cluster design rules are that every node must be able to see every other node in the cluster, so if

using the Mgmt server for quorum, then every node in the cluster (and every node in other clusters using

Multicluster to get to this one) must be able to connect to the Quorum node over either 1GbE or a High Speed

network (10GbE, 40 GbE, Infiniband)

Misc

o Console: Change keyboard on 7316-TF4 to match your country

o HMC:

Redundant Power: Add redundant power supply to HMC

Additional memory: Is an additional 8GB memory required? (TBC)

Delete HMC? If the customer has POWER systems installed, and an HMC available (at HMC V8 level to support

Power8 servers) to connect to the ESS NSD and Management nodes, then the HMC can be removed from the

configuration

Select the ESS just under Global Settings (top centre panel), Additional Products tab for the ESS, then

Delete Product

To clear the error message, right click on the error and select "Have HMC"

Now that we know what we want let’s get to pricing

© Copyright IBM Corporation 2015

o Rack:

Change Power cords: modify 7014-T42 rack PDU cords to match your voltage and phase requirements

See http://www.redbooks.ibm.com/redbooks.nsf/RedbookAbstracts/tips0775.html for additional information, or the Power8 planning guide

Note: the IBM PDUs and Intelligent PDUs are "universal" and can adapt to different power feeds, they use a Souriau UTG connector, so if you need to change to a different power supply (phase or

connector) you do not need to order a new PDU, just order a new power cable.

Side Panels: Remove side panels #6098 from racks if not required. Typically if all the racks are adjacent you only need 2

(order with 1st rack)

Rack joiner: If you want to bolt the racks together use Suite Attach Kit #6086, 1 per "gap" between racks.

Water cooling: if customer will use water cooling to the rack rear door, add #6858 Rear Door Heat eXchanger (RDHX), on

the Products tab

Recommended:

Physical security: Add Rack Security Kit = locks (1 x #6580), this is inexpensive, is recommended and is hard to

order later!

Rack mounted tray: Consider adding a #6586 Modem Tray (Products tab) to the 1st rack, as this can be useful to put

a manual, DVDs, or temporarily hold a laptop etc. This takes up 1U in the rack.

Additional PDU to device power cords: Useful to attach a laptop or other device into a PDU temporarily e.g. for

service or other work, consider ordering 1 or 2 per site with the 1st rack.

Intelligent PDUs: Edit 7014-T42 rack, change PDUs to Intelligent PDUs (1 x #5889 + 3 x #7199)

This is relatively inexpensive (approx $2000 per rack to upgrade from normal PDUs), and allows remote

power cycling/control/monitoring as well as environmental monitoring

For environmental monitoring add 1 or more #7108 Environmental Monitoring Probe (Power tab) to remotely

monitor the temperature and humidity and status of two contact devices in the rack

Now that we know what we want let’s get to pricing

© Copyright IBM Corporation 2015

Output files are in list price, then they go to Pricers for discounts

Review

• How to manage Spectrum Scale opportunity Discovery

• How to work with external Resources to become more

successful

• How to Become more successful with RFP responses

• Building a power point to explain how the solution requirements

are addressed both Technically & Philosophically

• Work with e-Config to build solution configurations

© Copyright IBM Corporation 2015

Questions before Lunch Break

© Copyright IBM Corporation 2015