sdv-ccie-9-10[1]

© 2006 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 1

SAN DeviceVirtualization

Commands hidden in NX-OS 5.x as feature has been deprecated after NX-OS 4.x


About SAN Device Virtualization (SDV)

What is SAN Device Virtualization? Virtualization of Initiators/Targets

– Target Migration = Migrating from one target (Primary Target) to another (Secondary Target) for one or more initiators (servers).

– Server Migration = Migrating from one initiator to another for one or more targets (disk).

Main Focus of presentation on Target Migration, but applies to Server Migration also.

What is driving the need for Target Virtualization functionality?

–Target failures: hardware, logical corruption

–Data migration: technology refresh, workload balancing, storage consolidation

–Virtualization Ready: for future needs


About SDV continued

What are the benefits of the feature compared to manual migration?

– Reduce the amount of time for migration; hence reducing the downtime

– Ease of management; Reduces possibility of human errors.

– Easily scalable for larger number of initiators/targets.


Today's Deployment for Handling Target Failures

Designed for HA; Redundancy is the key.

Deploy two arrays: primary & secondary.

Use some type consistency technology such as EMC SRDF between primary and secondary to ensure that secondary is a mirrored copy of the production LUN.

When primary fails, manually bring secondary online. All I/O will now take place with the secondary.

–The time required to `use’ secondary is the problem.

SAN

Primary Target

Secondary Target

ASYNCReplication

I/O - Normal

I/O - After Primary Failure


The Challenges of Handling Target Failures The time required to make secondary target accessible can be quite

long. This impacts the application availability.– For a typical production environment of 2 clustered hosts, and a database size of

2TB, this takes over 4 hours.

This down-time consists of two components:

- Zoning Changes• All the initiators now have to be re-zoned with the Secondary Target

- Re-configuring certain initiators

• Since WWN and FCID of the secondary is different, some driver files have to be changed and server rebooted, which adds risk (Eg: HP-UX and AIX servers)

• Clustering (multiple initiators) compounds the problem; Procedure has to be repeated for each server of the cluster.

Primary Array fails

Zoning Changes Changes to InitiatorsSecondary Array online

Up to a couple of hours


The SDV Manager SDV is a conditional service running on the MDS Switch Supervisor

that can create virtualized devices.

SDV presents a Virtual (Proxy) target to initiator.– Initiators are presented with a Virtual Device (VD); the virtualized form of the real target.

– VD is created by configuration and is identified by a name.

– VD virtualizes a single real target at any point of time.

– VD registered with Name Server with a pWWN and consumes an FCID outside of the real domain.

– VD is ‘hosted’ by one of the switches in the fabric.

– Administrators Zones VD with (real) initiators using regular Zoning Config. Initiator can be a VD also.

– Initiators discover VD through NS queries.

– The initiators always perform SCSI I/O operation to the virtual address of the VD.

–At this point, either the initiator or target can be VD, but not both.


SDV Manager Contd..

SDV performs FCID translations on the frames destined to VD.

– In the forward direction, DID is rewritten to that of the real target.

– In the reverse direction, SID is rewritten to that of the VD.

The VD is hosted by one of the switches using a virtual domain.

The VD has an FCID and pWWN/nWWN assigned by the switch.

VD exists as long as its real counterpart is online. It inherits the FC-4 type and features from its real counterpart.


SDV Solution: The Big Picture

Configure a Virtual Target (VD) on MDS. Initiator is zoned with VD. VD is linked to a real (Primary) Target. For control traffic (PLOGI, PRLI etc.)

– MDS provides proxy functions. That is, it receives these ELS frames and re-transmits it to the real Target by performing NAT in header and payload.

For data traffic– MDS provides NAT of header VD FCID in

hardware

The Primary and Secondary Targets can be on different MDS switches.

This features can be enabled on MDS without requiring a change on servers and storage arrays.

The admin can perform the Target Migration from primary to secondary through a single step configuration.

Primary Target Secondary Target

WWN2 FCID2

WWN3FCID3

WWN1 FCID1

WWN4 FCID4

Virtual Target

Initiator


Logical Zones

• The {initiator, VD} is zoned. However the real target is not part of any zone. Hence according to zoning the reverse path frames from target to initiator would be dropped.

• To overcome this restriction a logical zone is created with {initiator, target} and Zone Server is notified.

• The purpose of this logical zone is to make sure that w.r.t. the targets it is zoned with the initiators, but w.r.t. the initiators they are not zoned with the target.

• The logical zone is not present physically in the active ZoneSet. Hence presence of this zone not known to others.

• Logical Zones known only to SDV switches. Hence solution is limited to those devices that are directly connected to SDV enabled switch.


Solution Step 1: Creating the VD

Primary Secondary

t1pwwn t1

t2pwwn t2

i1pwwni1

Virtual Target

VT

i2pwwni2

i3pwwni3

SDV

A VD defined by user configuration enumerating all the targets (primary and secondary)

When a VD is created• SDV picks a unique pwwn for the VD.• Registers the VD name with its pwwn as an Device Alias.

vtpwwnvt1


Solution Step 2: Zoning VD with Initiators

Primary Secondary

t1pwwn t1

t2pwwn t2

i1pwwni1

Virtual Device

vtpwwnvt1

i2pwwni2

i3pwwni3

When VD name is zoned with initiators and activated …

• SDV assigns FCID for VD.• SDV Registers VD with NS and sends SW-RSCN for VD once the primary target is online.• Inform ZS about presence of Logical Zone (dotted in the diagram).

SDV


Solution Step 3: Linking VD with target

Primary Secondary

t1pwwn t1

t2pwwn t2

i1pwwni1

Virtual Device

vtpwwnvt1

i2pwwni2

i3pwwni3

SDV

When VD is linked with real target

• ACL Rewrite entries are programmed to do the header NAT of data frames on access ports.• ACL Capture entries are programmed for payload NAT of Control frames (ELS) by the SUP.


Hosting the VD• Hosting involves injecting the virtual entities (VD) into the fabric.

• Hosting done by one switch; But it cannot use its local domain for the VD FCID because if this switch leaves the fabric OR the local domain changes, FCID would have to change.

• The hosting switch reserves a Virtual Domain used for assigning to VD. If the hosting switch goes down another switch could take over its role using the same virtual domain.

• Virtual Domain reservation to be used for the VD is through Domain Manager using the RDI mechanism.

• The route for virtual domain is advertised by hosting switch through FSPF.

• Hosting consists of: FCID allocation, NS registration, SW-RSCN generation, Injecting Zones and route advertisement.


Rewrite Entries & ELS Capture entries

• Rewrite entries are programmed in ACL TCAM for FC header fcid-rewrite of the data traffic.

• For each <initiator,target> pair two rewrite entries are needed. One in the forward direction and one in the reverse direction.• Each ACL Rewrite entry includes zoning, forwarding and rewrite information and avoids using FIB lookup/adjacency. The rewrite entry would overwrite the zoning entry.

• ELS capture entries are programmed for FC payload fcid rewrite of the control traffic.

• All the ELS frames to VD are punted to Supervisor.• The SDV module running on Sup does payload fcid-rewrite depending on the ELS type and forwards it on the egress interface.• Some of the ACC frames also need payload fcid-rewrite.


Frame translation with rewrite entries

t1pwwn t1

i1pwwni1

i1 vt1

i1 t1 t1 i1

vt1 i1

S_ID D_ID

ACL entries

Frame values Rewrite valuesS_ID D_ID S_ID D_ID

i1 vt1 i1 t1 i2 vt1 i2 t1 ………. t1 i1 vt1 i1 t1 i2 vt1 i2


Header and Payload rewrites

PLOGI trapped and punted to supervisor for payload FCID rewrite

Supervisor forwards frame to egress line card for header FCID rewrite

SCSI commands are forwarded to egress line card for FCID header rewrite


Configuration Enable the feature

(config)# sdv enable Create a VD

(config)# sdv virtual-device name disk vsan 1

(config-sdv-virt-dev)# pwwn 21:00:00:20:37:a9:d7:42 primary Zone VD with desired devices

(config)# zone name zone1 vsan 1

(config-zone)# member pwwn <initiator> (config-zone)# member pwwn <VD-pwwn assigned by MDS>

Activate ZoneSet (regular zoneset activation)

(config)# zoneset activate name <zs> vsan <> Link with current primary real target

(config-sdv-virt-dev)# link pwwn 21:00:00:20:37:a9:d7:42

(Use the same ‘link’ command to switch to secondary target if primary goes down)

(config-sdv-virt-dev)# link pwwn 22:00:00:20:37:a9:d7:42 Different pwwn that exposes

same luns

Real target PWWN

Device-Alias created with this name automatically

Real initiator PWWN

PWWN assigned by MDS to represent

target to host


Domain, FCID, and PWWN used for VDswitch# show sdv virtual-device name diskvirtual-device name disk vsan 1[ WWN:50:00:53:00:00:cd:c0:01 FCID:0x140654 Real-FCID:0x7c03e4 ] pwwn 21:00:00:20:37:a9:d7:42 primary

PWWN assigned by MDS. Use this for zoning with

real initiator

FCID used to represent the

real target

RTP9-CAE-POD2-9509# sh fcdomain domain-list

VSAN 1Number of domains: 2Domain ID WWN--------- -----------------------0x64(100) 20:01:00:05:30:00:49:1f [Local] [Principal]0x14(20) 50:00:53:07:ff:f0:00:71 [Virtual (SDM)]

Domain used to represent DID for the Virtual Devices hosted by this switch for this VSAN


Locking down the domain and FCID used for the VD

The hosting switch will reserve a DID for the VDs

It can not be the same DID assigned to the VSAN in the event that another switch has to assume the hosting of the VD.

switch(config-sdv-virt-dev)# ?Configure a Virtual-device: device-alias Add a device-alias to the Virtual-device do EXEC command end Exit from configure mode exit Exit from this submode link Link the Virtual-device to a real device no Negate a command or set its defaults pwwn Add a pwwn to the Virtual-device virtual-domain Configure the persistent virtual domain virtual-fcid Configure the persistent virtual fcid

sdv enablesdv virtual-device name disk1 vsan 1 virtual-domain 138 virtual-fcid 0x8a197f pwwn 21:00:00:04:cf:17:66:b7 primarysdv commit vsan 1

Configured under virtual device

Completed

Configuration


Multi-switch Fabric

Primary Secondary

I1 I2

SW1SW2

SW3

T1 T2

VD

• SVD enabled on one of the switches that acts as the hosting switch. Eg: SW1 is the hosting switch

• The frame rewrite has to happen only on this hosting switch. Eg: Even though the traffic between I2 and T1 need not go thru SW1, it goes thru SW1 (because of VD’s fcid)

• Drawback: Does not work if any switch in path from initiator to target is not SDV enabled. (logical zone unaware) • Other Drawbacks: - The hosting switch is the single point of failure. - Hosting switch failure recovery requires manual intervention. - Non optimal routing of data frames.


Comparison with IVR-NAT SDV IVR-NAT

Injects virtual devices in a VSAN for transparent migration so that initiators can communicate with a virtualized target.

Injects virtual devices in all the VSANs in the path from initiator to target, so that initiators communicate with targets across VSANs.

Scope within a VSAN. Scope across VSANs.

Does frame S_ID and D_ID rewrite. Does frame VSAN, S_ID and D_ID rewrite.

The VD created by user configuration.

All the Virtual devices required are created automatically.

Uses one virtual domain for all VDs in a VSAN.

Can use multiple virtual domains for virtual devices in a VSAN.

Regular Zoning Configuration used.

IVR Zoning Configuration used.


Q and A

sdv-ccie-9-10[1]

Documents