presentation sun stor edge 9990 system technical

StorEdge 9900 Series

Speakers NameTitle

Sun Confidential: Internal Only 2

Dialing-in the Right-Sized Storage!

The SE 9900 Combination Rated by importance to customer's application SLA (low-medium-high)

✔ Data Availability VERY HIGH✔IO Performance HIGH✔Service Costs (Installation and Management)LOW✔Data Scalability VERY HIGH

✔Data Protection VERY HIGH✔Implementation PriceMEDIUM

✔Flexibility of Server SupportVERY HIGH


Sun StorEdge 9990 SystemThe Ultimate Storage Platform● For the high end data center● Unparalleled performance,

availability, scalability, manageability, connectivity

● Application-oriented storage– Massive consolidation– High volume OLTP– Business intelligence, large scale– Regulatory compliance– Data and information lifecycle management– Service provider business models– E-mail archiving


Sun StorEdge 9990 SystemPerformance, Capacity, Scalability, Connectivity

● 81 GB per second aggregate system bandwidth (68 GB per second data and 13 GB per second control)

● Performance scales with capacity● 336 TB internal capacity (300 GB drives)● 1152 drive slots● 256 GB data cache● 12 GB shared memory● Up to 192 direct connections (FC)● 2 Gbps FC backend loops● Multi-platform connectivity– FC/FICON/ESCON/NAS

● 73 GB 15K, 146 GB 10K and 15K, 300 GB 10Kdual-ported FC drives


Scaled Down Version of the SE 9990 Excellent Performance for its Class● 12 GB/Sec Aggregate System Bandwidth

● Scalability● From 1 to 72TB (per Subsystem)● Does not grow into an SE9990

● Centralized Storage Management● Common Platform for Multi-Vendor Support● Virtualize and manage up to 16PB● Active Workload Analysis & Tuning

● Flexible Connectivity Options● FC, ESCON, FICON, NAS

StorEdge 9985 SystemThe Ultimate Entry System for the Data Center


SE9990Architecture


SE 9990 Architecture


Sun StorEdge 9990 SystemArchitecture — Third Generation Supercomputer Crossbar Switch●Supercomputer Crossbar Switch81 GB per second – 4 switches with a parallel switch fabric – 64 paths on both processor and cache sides– 128 concurrent

cache operations– Non-blocking


SE 9900 Family Adapter Design●Each Adapter actually consists of two PCBs●Even though they are one adapter, Two PCBs are actually placed in two different HW clusters●Sixteen slots in two HW clusters● A pair of PCBs are symmetrical which Means “active/active and load balanced.”

Adapter 1Adapter 2 BED = 8 ports

Adapter 1 primeAdapter 2 primeC1 C2


Sun StorEdge 9990 SystemArchitecture — I/O

● Total of 16 slots accommodate– Front-end directors (FEDs — AKA channel host adapters, or CHAs)– Back-end directors (BEDs — AKA array control processors, or ACPs)– Each FED or BED occupies two slots– Of the 16 slots, 4 are reserved for FEDs

● Up to 6 FEDs, with 16 or 32 FC ports, up to a max of 192 FC ports● Up to 96 ESCON and 48 FICON ports are supported● Up to 4 BEDs, each supporting sixteen 2 Gbps FC-AL connections to drives – Each BED supports up to 256 drives. End result is 64

FCAL loops


SE 9900 FamilyFED ●Each FED has two PCB, each PCB has 8 MPs for a total of 16 MPs per FED●Each MP has flash memory associated with it, where microcode is stored.● 32 port adapter = 2 port

per MP● 16 port adapter = 1 port

per MP

PCB = Printed Circuit BoardMP = Microprocessor


4 Gbit Front End Director● 255 MB/sec: GA October 2005● 355 MB/sec: Future (Estimated May 2006)● Expected configuration (subject to change)

● 32 and 16 port versions● Each port operates at 4 Gbit/sec


Non-Disruptive Upgrade of Microcode –

MP 1

MP 2

MP = Microprocessor w/Flash Memory; 16 MPs/FED@ 400MHz; Mps are dedicated

●Path control is transferred to MP partner ●Data path is kept online while flash memory updated●Dialogue asks if to continue if data path saturation exceeds 50%


DrivesDrives SparesSpares

Sun StorEdge 9990 100 (Entry)

Sun StorEdge 9990 600 (Enhanced)

Sun StorEdge 9990 1100 (Ultra)

5 – 256 1 – 4

97 – 512 1 – 8

129 – 1152 1 – 16

Sun StorEdge 9990 SystemServer Support/LDEVS/Virtual Ports/Sparing

● 625 server support example, assuming 10 switches– Each switch supports 144 ports = 1440 ports– 1440 ports divided by two for redundancy = 720 ports pairs– 720 port pairs less 95 port pairs (for back-end storage connectivity) =

625 ports to support servers

– Example only, actual implementation will vary according to workload

● 16,384 logical devices

● 24,576 virtual ports

Note: These are pricing reference configurations, not separate and distinct models

Sparing


SE 9900 NAS Blade Feature● Fits in the same slot of the FED● Eight 1Gbps Ethernet ports per blade● Up to 4 blades per SE 9990, for a total of 32 Ethernet ports● Blades are active-active failure● Supports CIFS and NFS● Works with ShadowImage and Truecopy, and other storage

mgmt software● Snapshot capability, Supports Active Directory and OpenLDAP● Symantec anti-virus agent; purchase anti-virus server

seperately● File Shared/blade: 256 for both CIFS and NFS


Battery Backup/Data Destage


SE 9990 Warranty

WarrantyWarranty Period 3yrs

Service Level 3 years

Yes

4hr response on-site support Telephone Support 24

X7Install Service Included in

Purchase Price


SE9985Architecture


15U

1U

12U

12U

Optional Disk Chassis

Initial Disk Chassis Required

SVP

Logic/Power Supply Box• Cache/Shared Memory Package: 2

slots • Mix Adapter - Includes 16-port FED

and 8-port BED: 2 slots • Optional Package (i.e. Fibre

Channel, NAS): 2 slots• Power supply• Battery

PrimaryRack

Phase 2 Architecture Completed

12U

12U

SecondaryRack

Opt

iona

l Dis

k C

hass

is

Primary/Secondary racks are identical components.Total available rack space = 42U (1U = 1.75-inches)

Additional Capacity, Cache and Mainframe Support


Physical Rack


Third Generation Crossbar Switch● Non-blocking supercomputer switch architecture for

outstanding performance and scalability● 32 concurrent cache operations (not including shared

memory)● 12 GB/sec aggregate

system bandwidth● Larger cache, improved

algorithms, fasterprocessors

● Ability to aggregate controllers, capacity

MIX PK CHA2 CHA3MIX PK

CSW0CACHE (SM)

CSW1CACHE (SM)

Hi-Star


Feature OverviewMin/Max ports 16/48 FC, 16 FICON, 16 ESCON 8 NASMinimum HDD 4 Data + 1 SpareMaximim HDD 240 HDDDrive Types 73 GB 15K, 146 GB 10K & 15K, 300 GB 10K

Max External Capacity 16PBMin/Max Data Cache 4GB/64 GB

Min/Max Shared Memory 2GB/6GBRack 19” Industry Standard Rack

ShadowImage Open System and Mainframe SupportTrueCopy Sync/Async Open System and Mainframe SupportUniversal Replicator Yes

Copy on Write Snapshot YesData Retention Utility Yes

Business Continuity Manager YesHiCommand Device Manager Yes

Volume Migration YesVirtual Partition Manager YesServer Priority Manager Yes

Universal Volume Manager YesFlashCopy V2 Yes



Key Differences● 6 slots instead of 16● Integration of functions ● Mixed PK = Mixed FED/BED Combo Adapter● Data Cache/Shared Memory/Cache Switch are now on the same board.● One Service Processor


CSW0CACHE (SM)

CSW1CACHE (SM)

Hi-Star

Options Slot

Integrated data cache, share memory, switch PCB


SE 9985 I/0

●Six slots in two HW clusters - Slots 1A & 2F = Mixed adapter - Slots 1C & 2D – Cache/Shared/

Switch - Slot 1B & 2E “Options Slot” for FED, NAS, FICON, ESCON● Mixed Adapter

●A pair of boards deployed into two HW clusters - Total of 16 FED and 8 BED ports

FED = 8 portsBED = 4 ports BED = 8 ports

FED = 8 portsBED = 4 ports

C1 C2


CSW0CACHE (SM)

CSW1CACHE (SM)

Hi-Star


SE 9985 I/0 -Board Summary

●FED = 16 or 32 ports●Mixed Adapter = 16 FED/8 BED●NAS Blade = 8 Gigabit Ethernet●FICON = 16 ports●ESCON 16 Ports


CSW0CACHE (SM)

CSW1CACHE (SM)

Hi-Star


Power● Single Phase 30 AMP and 50 AMP available

Worldwide● 60 Hz: 200V, 208V, 230 V at 30 Amps● 50 Hz: 200V, 220V, 240 V at 50 Amps


Minimum/ Maximums/RAID/Spares● MIN/MAX

● 4 GB to 64 GB Data Cache ● 2 GB to 6 GB Shared Memory● 3D +1 P to 240 HDD● FC Ports 16-48

● RAID● RAID5 (3D + 1P)● RAID5 (7D + 1P) – w/o 100 % data availability guarantee● RAID1 (2D + 2D)● RAID6 (6D + 2P)

● Global Spares: MAX = 16


Battery Backup ● If power failure is less than 20 milliseconds, full

operation is maintained (e.g. Disks keep spinning)● If longer than 20 milliseconds, the operation ceases

and battery maintains data in data cache and shared memory up to 36 hours.

● Manual de-stage of data to disk required


SE 9985 Warranty

Warranty Period 3yrs

Service Level

Return to factoryReturn to factory

Yes

4hr response on-site support Telephone Support 24

X71st year

2nd year3rd year

Install Service Included in Purchase Price

http://www.sun.com/service/warranty/network.html#diskstorage

Note: Warranty may change March-end 2006, so check this URL


Comparison Within SE 9900

Family


Visual ComparisonSE 9985 SE 9990


SE 9985 and SE 9990

● Similarities● Similar architecture● Same microcode● Runs same software

● Virtualization● Partitioning● Replication● Management, ect

● Same upgrade parts ● HDD, FED

● Differences● Mixed Adapter● # of SVPs ● Enclosure● Power/Batteries● Internal Layout● Expandability


Schedules● “Phase 2” Completed February 2006

● HDD capacity increased to 240 HDDs with 2nd rack● Data Cache increased to 64 GB● Mainframe Support; ESCON/FICON


SE 9985/SE 9990 ComparisonFeature SE 9985 SE 9990Packaging 19"Rack Multi-CabinetArchitecture Crossbar CrossbarAggregate internal bandwidth 12GB/s 81GB/sMaximum Raw Capacity (Internal) 72 TB 332 TBInternal and External 16 PB 32 PBMax # of HDDS 240 1152Max # of LUNs 8192 16384Max data cache 64 GB 256GB Partitions 8 32Mirrored cache boards Yes YesMax shared memory 6GB 12GBFC Connectivity Performance 2Gb/s or 4 Gb/s 2Gb/sor 4 Gb/sMax theoretical host support 6144 24576Fibre Channel Connectivity 48 192Multi-protocol support FC/NAS/FICON/ESCON

HDD Support

RAID RAID 1,10,5, 6 RAID 1,10,5, 6* RAID 6 and RAID 5 (7D+1P) are available with 300 GB at this time on the SE 9990* RAID 5 (7D + 1P) is available on the SE 9985 but the 100% data availbility guarantee is not provided

FC/NAS/FICON/ESCON73 GB 15K, 146GB 10K, 146 15K

300 GB 10K*73 GB 15K, 146GB 10K,146 GB

300 GB 10K


Summary Comparison: SE 9985 vs othersFeature SE 9985 SE 99970 SE 9980 SE9990

12GB/s 150% greater 25% less 85% less

72 TB 380% greater 48% less 79% less

Max # of HDDS 240 180% less 8 76% less 79% less Max # of LUNs 8192 50% less 50% less 50% lessMax data cache 64 GB Equal 50 % less 75.00%

6144 6144 8192 24576

Fibre Channel Connectivity 48 Equal 25% less 75% less

Multi-protocol support

HDD Support

Aggregate internal bandwidthMaximum Raw Capacity (Internal)

Max theoretical host support

FC/NAS/FICON/ESCON

FC/NAS/FICON/ESCON

FC/NAS/FICON/ESCON

FC/NAS/FICON/ESCON

73 GB 15K, 146GB 10K and

15K, 300 GB 10K

No 300 GB, no 146 GB 15K

No 300 GB, no 146 GB 15K

73 GB 15K, 146GB 10K and

15K, 300 GB 10K


SE 9985/SE 9970/80 ComparisonFeature SE 9985 SE 99970 SE 9980Packaging 19"Rack Multiple cabinets Multiple cabinetArchitecture Crossbar Crossbar CrossbarAggregate internal bandwidth 12GB/s 7.9GB/s 15.9GB/sMaximum Raw Capacity (Internal) 72 TB 18.5 TB 149.5 TBInternal and External 16 PB N/A N/AMax # of HDDS 120 128 1024Max # of LUNs 8192 16384 16384Max data cache 32 (64 later) 64 GB 128 GBMirrored cache boards Yes Yes YesMax shared memory 6GB 6GB 6GBFC Connectivity Performance 2Gb/s 1Gb/s 1Gb/sMax theoretical host support 6144 6144 8192Fibre Channel Connectivity 48 48 64Multi-protocol support FC/NAS FC/NAS/ESCON/FICON FC/NAS/ESCON/FICON

HDD Support

RAID RAID 1,10,5, 6 RAID 1,10,5 RAID 1,10,5

73 GB 15K, 146GB 10K, 300 GB 10K

73 GB 10K, 73 GB 15K, 146GB 10K

73 GB 10K, 73 GB 15K, 146GB 10K



Key Differences● 6 slots instead of 16● Integration of functions ● Mixed PK = Mixed FED/BED Combo Adapter● Data Cache/Shared Memory/Cache Switch are now on the same board.● One Service Processor


CSW0CACHE (SM)

CSW1CACHE (SM)

Hi-Star

Options Slot

Integrated data cache, share memory, switch PCB



Switches, data cache, shared memory, BED on seperate PCBs


Four Key Software Packages


Example:Universal Volume Manager ● Customer Topology

● SE 9900 and EMC and HP EVA● Use Universal Volume Manager (UVM) to virtualize

EMC, EVA, SE 3511● Use ShadowImage or Volume Migrator to copy or

migrate data to virtualized external storage.● Use HDS storage management software to

administer all storage ● Investments are protected● Consistent management and tools


Storage VirtualizationImplementation — Universal Volume Manager

Array Array GroupGroup

Mapped Mapped

HBA Port HBA Port

Target Port Target Port ELUN

HOSTHOST

Host ViewDevices

InternalDevices

StorEdge 9990 arrayExternal Storage

StorEdge 9990

Controller

External Port External Port

Target Port Target Port

VDEV VDEV VDEVVDEV

LUNLUN

LDEV LDEV

HDDHDD

LUNLUN LUNLUN LUNLUN LUNLUN LUNLUN LUNLUN

LDEV LDEV

LDEV LDEV

LDEV LDEV

LDEV LDEV LDEV LDEV

LDEV LDEV

LUNLUN LUNLUN

ELUN — LUN that is mapped to a LUN in anexternal storage device


Virtual Partitioning Manager● Purpose

● Dedicate/isolate specific resources to an application or customer● Disk, Data Cache, I/O● Data cache allocated in 4GB increments

● Facilitates dedicating resources to several applications or internal/external customers in a SE 9985 or SE 9990.● This is consolidation in context of this software package.

● While flexible enough to change resource over time, in practice, partitions are not expected to be changed frequently.


VSPM’s – Basic Concepts

Cache Logical Partition (CLPR)• Assign specific cache capacity and assign one or

more parity groups to use that dedicated cache

Storage Logical Partition (SLPR)• Assign one or more Target Ports to use

the Cache Logical Partitions • Administrators and User level authority

maintained in Access Control Lists

Assigned Port

AssignedCache

Partition

Storage LogicalPartitioning

Cache LogicalPartitioning

Assigned PG

PartitionedCache

xGB CacheMemory

Parity Group

Parity Group

A CB D

Port

Physical Configuration

…

…

…

…

…


Access Control – Four Levels

CLPR CLPR0CLPR2

SLPR2SLPR1

Super SA

/General SA

Super PSA

/General PSA

Super PSA

/General PSA

Copy operationConfiguration managementUser,

management

Copy operationConfiguration management

User ,

SLPR/CLPR management

No*3(OK)*1OKNoNoNoSuper PSA

OKOKOKOKOKOKSuper SA

No

OK

No

No

SLPR1~31

No

No

SLPR0

No

OK

No*3(OK)*2General PSA

OKOKGeneral SA

User Types

SI/

Volume Migration

SA:Storage Administrator

PSA:Partition Storage Administrator

*1: Depend on the specification by Storage Admin

*2: Depend on the specification by Partition Storage Admin

*3: Supported at 3rd GA


Universal Replicator (UR)● Conceptually: TrueCopy plus more capability:

● Disk Journaling● In case of WAN-link failure, data is written to disk,

and copy suspension and recovery is avoided● Need to plan for space to accommodate expected

outtage. Threshholds are settable● Remote site polls primary site SE 9990/85to pull

data off the production site, decreasing cycles consumed on at production site.

● UR is asynch; combine with TrueCopy Synch for data currency

● Use only with SE 9985 and SE 9990.


-Universal Replictor is more stable than TrueCopy Asynch (TCA) in addressing link issues. 26% of issue are link related:HDS/TRC analysis 04Q/2004

- Engineering tests suggest UR help manage expensive, limited, and unreliable links

- TCA could not maintain Remote Copy due to 45 seconds Link down

- UR could provide reliable Remote Copy during 5 minutes Link down

Response time(ms)

Time

Link down Link up5 minutes

TC Async

UR

TCA Suspend

(Cannot maintain remote copy after Suspend

After Suspend, TCA receives host I/O)

UR guarantees data consistency

during Link down

TCA starts host I/O throttle

to reduce input to cache over cache 30%

Universal Replicator maintains Remote Copy

during link down until JNL full

45 sec

Universal Replicator (UR)


Example: Universal Replicator● Before

● WAN-link fails; transactions overflow cache● Suspend copy and initiate re-synch operations

● After (e.g with Universal Replicator)● WAN-link fails and transactions are written to disk● WAN-link is restored, copy operation resumes, no copy

suspend, no re-synch operation● Pull technology polls central to transmit data according

to bandwidth capacity● Customer can by smaller pipe

● Benefits: Lower WAN costs and reduced operational disruptions


Tiered Storage Manager Details

● “Volume Migrator” is the underlying engine● Search capability enables collection of application data to be

grouped and migrated● Example:

Condition is: RAIDLevel IS RAID5 AND DriveType IS FC-AL AND ArrayGroup IS NOT 2-1

● Groups of volumes can be migrated at the same time(migration groups)

● A group of volumes can serve as migration targets(storage tiers)


Example Tiered Storage Manager

● Customer topology● SE9900 with SE3511 and EMC

● Use Tiered Storage Manager (TSM) to select several volumes and migrate them to specific target volumes

● Result: automated movement of data according to its value● Reduced DLM costs


Continuity Example


ShadowImage In-System Replication Up to 9 Full Duplicate Copies of Any Data Volume

9 Copies Total Open Systems

Non-stop Production

Secondary

Volume

Secondary

Volume

Secondary

Volume

Primary

Volume

Secondary

Volume

Secondary

Volume

Secondary

Volume

Secondary

Volume

Secondary

Volume

Secondary

Volume


SE 9900 ShadowImage Operations ● R/W operations continue during creation of S-Vol.

● R/W operations buffered in data cache● Pair status monitored to ensure completion of copy

operations● Meta data about changed/new data kept in differential

bitmap in shared memory to support resynch operations● R/W operations may also be performed on S-Vols. S-

Vols need not be fully populated before other copy software begin operations on S-Vol. Writes can be held in cache while other copy operations are executed against the S-Vol.

● TrueCopy Remote Replication and Universal Replicator use SI P-vol.


Copy On Write ● Copies Changes only

● Create Primary Volume (P-Vol) from the Production DB.● Create a Secondary Volume (S-Vol) from the P-Vol.● This is a Copy on Write Snapshot Pair

● The S-Vol is carved out of a pool of allocated physical storage called a “Pool-Vol”● Pool-Vol can dynamically grow; thresholds settable

● Cannot snap off of a snap (e.g. No S-Vol to S-Vol)● A snap can be taken off of a P-Vol

● TrueCopy (TC) and SI can operate off of a P-Vol. ● Use for write oriented applications: 60% W, 40% R is

rough limit.


TrueCopy to Remote DR site.● S-Vol copied by TC to Remote DR site may be

re-mounted as a P-Vol. ● COW Snaps can be taken off this P-Vol● Tape would use use a SI S-Vol created from P-Vol

in order to avoid write contention.● Updates to remote DR site occur realtime,

assuming cache write through


SnapBack Operations● Snapback

● Can now snapback to any version● Do not have to delete later versions


● The SE 9985 and SE 9990 represent the highest levels of:● Availability● Managability● Scalability● Connectivity● Performance

Summary

Thank You

Name of SpeakerTitle

presentation sun stor edge 9990 system technical

Technology

fc ports

internal only14

internal only13

gb drives

dedicatedsun confidential

nassun confidential

architecturesun confidential

port adapter