brocade fabric technology with the tegile ha2100ep ... equipment feature, or service offered or to...

May 2015

Brocade Fabric Technology with the Tegile HA2100EP Validation Test Report

Supporting Fabric OS v7.3.1


© 2015 Brocade Communications Systems, Inc. All Rights Reserved.

ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, The Effortless Network, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision and vADX are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of others. Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government. The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the accuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that accompany it. The product described by this document may contain open source software covered by the GNU General Public License or other open source license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable to the open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.


Contents Contents ........................................................................................................................................................ 3

Preface .......................................................................................................................................................... 5

Overview ................................................................................................................................................... 5

Purpose of This Document ........................................................................................................................ 5

Audience ................................................................................................................................................... 5

Objective ................................................................................................................................................... 5

Test Conclusions ....................................................................................................................................... 5

Related Documents ................................................................................................................................... 6

Document History ..................................................................................................................................... 6

About Brocade .......................................................................................................................................... 6

About Tegile .............................................................................................................................................. 6

Test Plan ........................................................................................................................................................ 7

Scope ......................................................................................................................................................... 7

Test Configuration ..................................................................................................................................... 7

DUT Descriptions ...................................................................................................................................... 8

DUT Specifications .................................................................................................................................... 8

Test Equipment ......................................................................................................................................... 9

Configure DUT and Test Equipment ......................................................................................................... 9

Step 1. Brocade FC Fabric Configuration .............................................................................................. 9

Step 2. Tegile Intelliflash HA2100EP Array Configuration .................................................................. 12

Step 3. Host Setup ............................................................................................................................... 15

Test Cases .................................................................................................................................................... 19

Fabric Initialization – Base Functionality ................................................................................................ 20

Storage Device – Physical and Logical Login with Speed Negotiation ................................................ 20

Zoning and LUN Mapping.................................................................................................................... 20

Storage Device Fabric IO Integrity ...................................................................................................... 21

Storage Device Multipath Configuration – Path Integrity .................................................................. 21

Fabric – Advanced Functionality ............................................................................................................. 22

Storage Device Bottleneck Detection Using MAPS FPI – w/Congested Host ..................................... 22

Storage Device Bottleneck Detection Using MAPS FPI – w/Congested Fabric ................................... 23

Storage Device – QOS Integrity ........................................................................................................... 25

Storage Device – FC Protocol Jammer Test Suite ............................................................................... 25

Stress and Error Recovery with Device Multipath .................................................................................. 26

Storage Device Fabric IO Integrity – Congested Fabric ....................................................................... 26

Storage Device Integrity – Device Recovery from Port Toggle ........................................................... 27


Storage Device Integrity – Device Recovery from Device Relocation ................................................. 28

Storage Device Stress – Device Recovery from Device Port Toggle – Extended Run ......................... 29

Storage Device Recovery – ISL Port Toggle (Sequential) .................................................................... 30

Storage Device Recovery – ISL Port Toggle (Entire Switch) ................................................................ 32

Storage Device Recovery – Director Blade Maintenance ................................................................... 33

Storage Device Recovery – Switch Offline .......................................................................................... 35

Storage Device Recovery – Switch Firmware Download .................................................................... 36

Storage Device Fibre Channel Routing (FCR) Internetworking Tests ...................................................... 37

Storage Device Internetworking Validation w/FC host ....................................................................... 37

Storage Device Internetworking Validation w/FCoE Using VDX FlexPort ........................................... 38

Storage Device Edge Recovery after FCR Disruptions ......................................................................... 39

Storage Device Backbone Recovery after FCR Disruptions ................................................................. 40

Optional/Additional Tests ....................................................................................................................... 42

Non-Disruptive Firmware Upgrade on Storage Device ....................................................................... 42

Synthetic I/O Workload Loop with Varying Block Sizes ...................................................................... 42

VMware I/O Analyzer Workload Test ................................................................................................. 43


Preface Overview

The Solid State Ready (SSR) program is a comprehensive testing and configuration initiative to validate the interoperability of Fibre Channel and IP flash storage with a Brocade network infrastructure. This program provides testing of multiple fabrics, heterogeneous servers, NICs and HBAs in a large port-count Brocade environment. The SSR qualification program will help verify seamless interoperability and optimum performance of solid state storage in Brocade FC and Ethernet fabrics.

Purpose of This Document

The goal of this document is to demonstrate the compatibility of Tegile Intelliflash HA2100EP arrays in a Brocade FC SAN fabric running FOS v7.3.1. This document provides a test report on the SSR qualification test plan executed on the Tegile Intelliflash HA2100EP array.

Audience

The target audience for this document includes storage administrators, solution architects, system engineers, and technical development representatives.

Objective

1. Test the Tegile Intelliflash HA2100EP array with Brocade FC fabrics, in single and routed configurations for different stress and error recovery scenarios, to validate the interoperability and integration of the Tegile array with Brocade FC fabrics.

2. Validate the performance of the FC fabric in a solid state storage environment for high throughput and low latency applications.

Test Conclusions

1. Achieved 100% pass rate on all the test cases in the SSR qualification test plan. The network and the storage were able to handle the various stress and error recovery scenarios without any issues.

2. Different I/O workload scenarios were simulated using Medusa and VMware I/O Analyzer tools and sustained performance levels were achieved across all workload types. The Brocade FC fabric handled both the low latency and high throughput I/O workloads with equal efficiency without any I/O errors or packet drops.

3. The results confirm that the Tegile Intelliflash HA2100EP array interoperates seamlessly with Brocade FC fabrics, and demonstrate high availability and sustained performance.

4. The Brocade Gen5 16Gb FC switches were able to handle the sustained throughput and latency performance requirements efficiently with fewer ISL trunks. Multiple ISL’s to different switches in the fabric should be setup for providing path redundancy through the fabric.

5. It is recommended to enable the Monitoring and Alerting Policy Suite (MAPS) heath monitor on all switches in the FC fabric to report fabric-wide events and traffic performance metrics. The additional MAPS feature of Fabric Performance Impact monitoring should also be enabled to detect bottlenecks in the form of timeouts and latency.

6. For optimal availability and performance, consideration should be given to multipath configuration on the host side. Enabling Windows MPIO on hosts will provide Round-Robin behavior by default,


Linux and VMware systems require a custom entry to the multipath configuration settings provided by Tegile to efficiently use all available paths and provide high availability.

Related Documents

References • Fabric OS Administrator's Guide • Monitoring and Alerting Policy Suite Administrator's Guide • Brocade SAN Design and Best Practices • Brocade SAN Fabric Administration Best Practices Guide • Network OS Layer 2 Switching Configuration Guide

Document History

Date Version Description May 2015 1.0 Initial version.

About Brocade

Brocade networking solutions help the world’s leading organizations transition smoothly to a world where applications and information reside anywhere. This vision is realized through the Brocade One™ strategy, which is designed to deliver key business benefits such as unmatched simplicity, non-stop networking, application optimization, and investment protection. Innovative Ethernet and storage networking solutions for data center, campus, and service provider networks help reduce complexity and cost while enabling virtualization and cloud computing to increase business agility. To help ensure a complete solution, Brocade partners with world-class IT companies and provides comprehensive education, support, and professional services offerings. To learn more, visit (www.brocade.com)

About Tegile Tegile SystemsTM is pioneering a new generation of flash-driven enterprise storage arrays that balance performance, capacity, features and price for virtualization, file services and database applications. Our hybrid arrays are significantly faster than legacy arrays and significantly less expensive than all solid-state disk-based arrays. Featuring both NAS and SAN connectivity, these virtual data storage systems are easy-to-use, fully redundant, and highly scalable. Additionally, they come complete with built-in snapshot, replication, near-instant recovery, and virtualization management features. Tegile’s patented IntelliFlashTM technology accelerates performance to solid state speeds without sacrificing the capacity or cost advantage of hard disk storage. Additionally it enables on-the-fly de-duplication and compression so usable capacity is far greater than its raw capacity.

http://www.brocade.com/downloads/documents/html_product_manuals/FOS_730_ADMIN_04/index.html

http://www.brocade.com/downloads/documents/html_product_manuals/FOS_730_MAPS_03/index.html

http://www.brocade.com/downloads/documents/best_practice_guides/san-design-best-practices.pdf

http://www.brocade.com/downloads/documents/best_practice_guides/san-admin-best-practices-bp.pdf

http://www.brocade.com/downloads/documents/html_product_manuals/NOS_501_LAYER2/index.html

http://www.brocade.com/


Test Plan The Tegile Intelliflash HA2100EP array is connected to a Brocade 16Gb FC fabric with the “active” and “backup” controller HBA ports balanced across the Brocade FC fabric.

Scope

Testing focusses on interoperability of the Tegile storage array and determining an optimal configuration for performance and availability. Testing covers various I/O stress and error handling scenarios. Performance is observed within the context of best practice fabric configuration; however absolute maximum benchmark reporting of storage performance is beyond the scope of this test. Details of the test steps are covered under “Test Case Descriptions” section. Standard test bed setup includes IBM/HP/Dell servers with Brocade/QLogic/Emulex HBA’s with two uplinks from every host to the Brocade FC fabric. IO generator tools used include Medusa Labs Test Tools and VMware I/O Analyzer.

Test Configuration

Test Configuration


DUT Descriptions

The following lists the devices under test (DUT) and the test equipment used. Storage Array DUT ID Model Vendor Description

Tegile HA2100EP

HA2100EP Tegile The HA2100EP array is a hybrid-flash array setup with dual controllers in a HA resource cluster. The controllers are in an active-passive configuration with 2x8Gb FC and 2x10GbE ports on each controller. The array supports FC, iSCSI, NFS and CIFS protocols.

Switches DUT ID Model Vendor Description

6510-1..9 BR-6510 Brocade 48 port 16Gb FC switch

DCX-1 DCX 8510-8 Brocade 8 slot 16Gb FC chassis

DCX-2 DCX 8510-4 Brocade 4 slot 16Gb FC chassis

VDX-1,2 VDX 6740 Brocade 48x10GbE and 4x40GbE ports switch

DUT Specifications

Storage Version Tegile Intelliflash HA2100EP array 2.1.3.2(141215)-338

Brocade Switches Version DCX 8510-8 FOS v7.3.1 DCX 8510-4 FOS v7.3.1 6510 + Integrated Routing, Fabric Vision Licenses FOS v7.3.1 VDX 6740 NOS v5.0.1

Adapters Version Brocade 1860 2-port 16Gb FC HBA/10GbE CNA driver & firmware version 3.2.5.1 Brocade 825 2-port 8GB FC HBA driver & firmware version 3.2.5.1 QLogic QLE2672 2-port 16GB FC HBA driver 8.07.00.16, firmware 7.04.01 QLogic QLE2562 2-port 8GB FC HBA driver 8.07.00.16, firmware 7.01.00 Emulex LPe 16202-X 2-port 16Gb FC HBA driver 10.2.370.8, firmware 1.1.55.0 Emulex OCe14102-UM 2-port CNA adapter driver 10.2.370.8, firmware 10.2.370.19

DUT ID Servers RAM Processor OS SRV-1 HP Proliant DL380p Gen8 16GB Intel Xeon E5-2620 Windows Server 2012 R2 SRV-2 HP Proliant DL380p Gen8 32GB Intel Xeon E5-2690 RHEL 6.5 x86_64 SRV-3 IBM System x3650 M4 16GB Intel Xeon E5-2620 RHEL 7 x86_64 SRV-4 IBM System x3550 M3 24GB Intel Xeon E5645 SLES 12.0 x86_64 SRV-5 HP Proliant DL360 Gen7 24GB Intel Xeon E5645 VMware ESXi 5.5 SRV-6 HP Proliant DL360 Gen7 24GB Intel Xeon E5645 VMware ESXi 5.5


SRV-7 IBM System x3630 M4 24GB Intel Xeon E5-2420 Windows Server 2012 R2 SRV-8 Dell Poweredge R710 8GB Intel Xeon L5520 Windows Server 2012

Test Equipment

Device/Software Tools Version Finisar 16Gb Analyzer/Jammer XGIG5K2001153 Medusa Labs Test Tools 6.0.1.148039 VMWare I/O Analyzer 1.6.2

Configure DUT and Test Equipment Some of the required and recommended configurations for the test bed systems are covered here.

Step 1. Brocade FC Fabric Configuration

1. Enable MAPS and Fabric Performance Impact (FPI) monitoring on all switches in the fabric. MAPS enables health monitoring on the switches to detect potential faults and create alerts. MAPS FPI allows fabric monitoring for performance impacts, including timeouts, latency, and throughput.

• MAPS requires “Fabric Vision License” to be installed. • Enable the desired MAPS policy using any of the available default policies or create custom policy. • Enable FPI monitoring. FPI monitoring requires “bottleneckmon” to be disabled.

<==========> root> mapsconfig --enablemaps -policy dflt_aggressive_policy WARNING: This command enables MAPS and replaces all Fabric Watch configurations and monitoring. Once MAPS is enabled, the Fabric Watch configuration can't be converted to MAPS. If you wish to convert your Fabric Watch configuration into MAPS policies, select NO to this prompt and first issue the "mapsconfig --fwconvert" command. Once the Fabric Watch configuration is converted into MAPS policies, you may reissue the "mapsconfig --enablemaps" command to continue this process.If you do not use Fabric Watch or need the configuration, then select YES to enable MAPS now. Do you want to continue? (yes, y, no, n): [no] y Enabled dflt_aggressive_policy policy. root> mapsconfig --actions raslog,fence,snmp,email,sw_marginal,sw_critical root> bottleneckmon --disable root> mapsConfig –enableFPImon root> mapsconfig --show Configured Notifications: RASLOG,SNMP,EMAIL,FENCE,SW_CRITICAL,SW_MARGINAL Mail Recipient: [email protected],[email protected] FPI Monitoring: Enabled Paused members :


=============== PORT : CIRCUIT : SFP : root> mapsdb --show 1 Dashboard Information: ======================= DB start time: Wed Dec 17 20:59:09 2014 Active policy: dflt_aggressive_policy Configured Notifications: RASLOG,SNMP,EMAIL,FENCE,SW_CRITICAL,SW_MARGINAL Fenced Ports : None Decommissioned Ports : None 2 Switch Health Report: ======================= Current Switch Policy Status: HEALTHY 3.1 Summary Report: =================== Category |Today |Last 7 days | -------------------------------------------------------------------------------- Port Health |No Errors |Out of operating range | Fru Health |In operating range |In operating range | Security Violations |No Errors |No Errors | Fabric State Changes |No Errors |In operating range | Switch Resource |In operating range |In operating range | Traffic Performance |In operating range |In operating range | FCIP Health |Not applicable |Not applicable | Fabric Performance Impact|In operating range |In operating range | 3.2 Rules Affecting Health: =========================== Category(Rule Count)|RepeatCount|Rule Name |Execution Time |Object |Triggered Value(Units)| ------------------------------------------------------------------------------------------------------------------------ Port Health(4) |1 |defALL_TARGET_PORTSLOSS_SIG|12/29/14 15:39:54|Port 16 |1 LOS | | |NAL_0 | | | |


|1 |defALL_TARGET_PORTSSTATE_CH|12/29/14 15:39:54|Port 16 |1 | | |G_0 | | | | |1 |defNON_E_F_PORTSLOSS_SIGNAL|12/29/14 15:39:54|Port 16 |1 LOS | | |_0 | | | | |1 |defALL_OTHER_F_PORTSLF_0 |12/29/14 15:31:48|Port 16 |1 | <==========>

2. Configure required zone sets to enable communication between the hosts and target.

<==========> root> cfgactvshow zone: ssr067161_Tegile 21:00:00:24:ff:48:b9:6a 21:00:00:24:ff:48:b9:6b 50:01:43:80:06:2d:08:28 50:01:43:80:06:2d:08:2a 10:00:8c:7c:ff:03:9b:00 10:00:8c:7c:ff:03:9b:01 <==========>

3. Configure Fibre Channel Routing (Integrated Routing license required). Detailed information on FCR

setup can be found in the Brocade Fabric OS Administrator’s Guide.

Example FCR configuration is shown below: <==========> root> fcrconfigure –bbfid 100 root> fosconfig --enable fcr root> portcfgexport [port#] -a1 –m[0/5] -f 10 --- {-m 0-Brocade 5-Brocade NOS fabric} Example output of exported devices root> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID in Fabric in Fabric ---------------------------------------------------------------------------- 10 21:00:00:24:ff:48:b9:6a 02f001 20 551a00 Imported 10 21:00:00:24:ff:48:b9:6b 02f101 20 541e00 Imported 10 52:4a:93:7d:f3:5f:61:00 02f201 20 550e00 Imported 10 52:4a:93:7d:f3:5f:61:01 02f401 20 540400 Imported <==========>


4. Configuring “FlexPort” on VDX 6740 switches. Flexport feature allows ports to transmit data as either 10G Ethernet or Fibre Channel, and to be changed from one type to other without requiring a reboot. Detailed information for FlexPort configuration can be found in the Network OS Layer 2 Switching Configuration Guide.

Example FlexPort configuration is shown below: <==========> VDX6740# show running-config hardware flexport hardware flexport 119/0/48 type fibre-channel ! flexport 120/0/48 type fibre-channel ! connector-group 119/0/6 speed HighMixed ! connector-group 120/0/6 speed HighMixed ! <==========>

Step 2. Tegile Intelliflash HA2100EP Array Configuration

1. Standard array setup with High-Availability configured between the dual controllers for the storage

pool.

HA Configuration for the Storage Pool

2. The FC target ports from both controllers on the array are connected to the Brocade FC fabric.


FC Target Connections

3. Create volume group “projects” based on host initiator group access. Default values for “Data

Compression” and “Deduplication” are accepted. No “Snapshots” or “Replication” is configured.

Project Configuration General Settings


Project Configuration – Mappings

Project Dataset


Step 3. Host Setup

1. Provision a minimum of two uplinks from the host to the FC fabric for redundancy and use native multipath tools to manage the available paths and load-balance across them.

2. Configuring the multipath settings allows for proper failover and load balancing across the available links. Multipath settings for Linux and VMware as recommended by Tegile are provided here.

• For Windows, the native MPIO settings are used and no special configuration is necessary.

Windows Disk MPIO Properties

• For Linux, add the following to /etc/multipath.conf

<==========> devices { device { vendor "TEGILE" product "ZEBI-FC" hardware_handler "1 alua" path_selector "round-robin 0" path_grouping_policy "group_by_prio" no_path_retry 10


dev_loss_tmo 50 path_checker tur prio alua failback 30 } Sample output: 3600144f0ec770e000000536d06730008 dm-2 TEGILE ,ZEBI-FC size=10G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 5:0:2:15 sdm 8:192 active ready running | |- 5:0:0:15 sdr 65:16 active ready running | |- 6:0:1:15 sdad 65:208 active ready running | `- 6:0:2:15 sdai 66:32 active ready running `-+- policy='round-robin 0' prio=1 status=enabled |- 5:0:3:15 sdd 8:48 active ready running |- 6:0:3:15 sde 8:64 active ready running |- 6:0:0:15 sdx 65:112 active ready running `- 5:0:1:15 sdw 65:96 active ready running <==========>

• For VMware, run the following command from VMware host CLI to setup default round-robin path-selection policy for Tegile devices.

<==========> esxcli storage nmp satp rule add -s VMW_SATP_ALUA –V "TEGILE" -M "ZEBI-FC" -c "tpgs_on” –P VMW_PSP_RR -e "Tegile Zebi FC" <==========>


VMware Path Selection Policy for Tegile

Tegile Devices Discovered on VMware Host


VMware Discovered Paths for Tegile Devices


Test Cases

1.0 FABRIC INITIALIZATION – BASE FUNCTIONALITY 1.0.1 Storage Device – Physical and Logical Login with Speed Negotiation 1.0.2 Zoning and LUN Mapping 1.0.3 Storage Device Fabric IO Integrity 1.0.4 Storage Device Multipath Configuration – Path integrity 1.1 FABRIC – ADVANCED FUNCTIONALTY 1.1.1 Bottleneck Detection using MAPS FPI – w/Congested Host 1.1.2 Bottleneck Detection using MAPS FPI – w/Congested Fabric 1.1.3 Storage Device – QOS Integrity 1.1.4 Storage Device – FC Protocol Jammer Test Suite 1.2 STRESS & ERROR RECOVERY WITH DEVICE MULTI-PATH 1.2.1 Storage Device Fabric IO Integrity – Congested Fabric 1.2.2 Storage Device Nameserver Integrity – Device Recovery with Port Toggle 1.2.3 Storage Device Nameserver Integrity – Device Recovery with Device

Relocation 1.2.4 Storage Device Nameserver Stress – Device Recovery with Device Port

Toggle 1.2.5 Storage Device Recovery – ISL Port Toggle (Sequential) 1.2.6 Storage Device Recovery – ISL Port Toggle (entire switch) 1.2.7 Storage Device Recovery – Director Blade Maintenance 1.2.8 Storage Device Recovery – Switch Offline 1.2.9 Storage Device Recovery – Switch Firmware Download 1.3 STORAGE DEVICE – FIBRE CHANNEL ROUTING (FCR) INTERNETWORKING

TESTS 1.3.1 Storage Device InterNetworking Validation w/FC host 1.3.2 Storage Device InterNetworking Validation w/FCoE using VDX FlexPort 1.3.3 Storage Device Edge Recovery after FCR Disruptions 1.3.4 Storage Device BackBone Recovery after FCR Disruptions 1.4 OPTIONAL/ADDITIONAL TESTS 1.4.1 Non-disruptive firmware upgrade on array 1.4.2 Synthetic I/O workload loop with varying block sizes 1.4.3 VMware I/O Analyzer workload tests


Fabric Initialization – Base Functionality

Storage Device – Physical and Logical Login with Speed Negotiation

Test Objective 1. Verify device login to switch and name-server with all supported speed settings. Procedure Test Execution: 1. Set switch ports to 4/8/16/Auto_Negotiate speed settings.

<==========> portcfgspeed <port> [4/8/16/0] <==========>

Result Validation: 1. Validate link states on the array and verify speed negotiation and device login at different speeds. 2. Check switch port status and verify the “actual” and “configured” link speed. Check name server for

device login <==========> # nscamshow # portshow X root> portshow 19 portIndex: 19 portName: port19 portHealth: HEALTHY portState: 1 Online Protocol: FC portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normal portSpeed: N16Gbps <==========>

Result 1. PASS. Test Passed. Speed negotiation, device login and connectivity verified.

Zoning and LUN Mapping

Test Objective 1. Verify host to LUN access exists with valid zoning. Procedure Test Execution: 1. Create FC zone on the fabric with the initiator and target WWNs. 2. Create Host Groups and LUNs on the array with access to initiator WWN.


Result Validation: 1. Verify LUNs are discovered on the hosts with host specific tools.

<==========> - Linux: Check output of ‘lsscsi’ - Windows: Check output of ‘Computer Management’ -> ‘Storage” -> ‘Disk Management’ - VMware: Check output at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ <==========>

Result 1. PASS. Test Passed. Host has read/write access to presented LUNs.

Storage Device Fabric IO Integrity

Test Objective 1. Validate single path host-to-LUN IO with write/read/verify testing. Include short device cable

pulls/port-toggle to validate device recovery. Procedure Test Execution: 1. Setup read/write I/O to LUN using Medusa. 2. Perform link disruptions by port-toggles, cable pulls. Result Validation: 1. Check Medusa I/O logs and verify I/O resumes after a short downtime. Medusa I/O may pause, but

should recover without errors. Result 1. PASS. Test Passed. I/O resumes after the disruption.

Storage Device Multipath Configuration – Path Integrity

Test Objective 1. Verify multi-path configures successfully. Each Adapter and Storage port to reside in different

switches. For all device paths, consecutively isolate individual paths and validate IO integrity and path recovery.

Procedure Test Execution: 1. Setup host with at least 2 initiator ports zoned with 2 target ports on array. 2. Setup multipath on the host and start I/O. 3. Perform sequential port toggles across initiator and target switch ports to isolate paths. Result Validation: 1. Check host multipath properties to verify the toggled path recovers.

<==========> - Windows: mpclaim –s –d - Linux: multipath –ll - VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>

2. Check the host and storage logs for any failures.


3. Check the switch error logs and switch port status after toggle. <==========> # errdumpall # portstatsshow X # portshow X root> portshow 19 portHealth: HEALTHY ………… portState: 1 Online ………….. portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normal portSpeed: N16Gbps <==========>

4. Check I/O logs and verify I/O continues without any errors. Result 1. PASS. Test Passed. I/O fails over to remaining active paths and recovers when disrupted path is

restored.

Fabric – Advanced Functionality

Storage Device Bottleneck Detection Using MAPS FPI – w/Congested Host

Test Objective 1. Verify congestion on host ports is detected. Verify storage device and fabric behavior during

congestion. Procedure Test Execution: 1. Enable MAPS monitoring and MAPS FPI on all switches. Fabric Vision license required. 2. Start I/O from single host initiator to multiple targets. 3. Monitor switch logs for Congestion and Latency (IO_PERF_IMPACT/IO_FRAME_LOSS) warnings. Result Validation: 1. Check switch error logs and MAPS dashboard for bottleneck warnings.

<==========> # errdumpall | grep IO_ root> errdumpall | grep IO_ 2014/12/17-11:56:00:672622, [MAPS-1003], 117148/115018, FID 128, WARNING, B6510_066_088, Port 16, Condition=ALL_F_PORTS(DEV_LATENCY_IMPACT==IO_PERF_IMPACT), Current Value:[DEV_LATENCY_IMPACT,IO_PERF_IMPACT, 30.0% in 10 secs], RuleName=defALL_F_PORTS_IO_PERF_IMPACT, Dashboard Category=Fabric


Performance Impact., actionHndlr.c, line: 755, comp:md, ltime:2014/12/17-11:56:00:671909 # mapsdb --show all root> mapsdb --show 1 Dashboard Information: ======================= Active policy: dflt_aggressive_policy ………………… 2 Switch Health Report: ======================= Current Switch Policy Status: HEALTHY 3.1 Summary Report: =================== Category |Today |Last 7 days | -------------------------------------------------------------------------------- Port Health |No Errors |Out of operating range | Fru Health |In operating range |In operating range | Security Violations |No Errors |In operating range | Fabric State Changes |No Errors |In operating range | Switch Resource |In operating range |In operating range | Traffic Performance |In operating range |In operating range | FCIP Health |Not applicable |Not applicable | Fabric Performance Impact|In operating range |In operating range | <==========>

Result 1. PASS. Test Passed. The bottlenecked ports are displayed on the MAPS dashboard and also a raslog

warning is created.

Storage Device Bottleneck Detection Using MAPS FPI – w/Congested Fabric

Test Objective 1. Create congestion on switch ISL port. Verify congestion in the fabric is detected. Verify storage device

and fabric behavior during congestion. Procedure


Test Execution: 1. Enable MAPS monitoring and MAPS FPI on all switches. Fabric Vision license required. 2. Isolate single ISL in the fabric. 3. Start I/O from multiple host initiator to multiple targets. 4. Monitor switch logs for Congestion and Latency (IO_PERF_IMPACT/IO_FRAME_LOSS) warnings. Result Validation: 1. Check switch error logs and MAPS dashboard for bottleneck warnings.

<==========> # errdumpall | grep IO_ root> errdumpall | grep IO_ 2014/12/17-11:56:00:672622, [MAPS-1003], 117148/115018, FID 128, WARNING, B6510_066_088, Port 16, Condition=ALL_F_PORTS(DEV_LATENCY_IMPACT==IO_PERF_IMPACT), Current Value:[DEV_LATENCY_IMPACT,IO_PERF_IMPACT, 30.0% in 10 secs], RuleName=defALL_F_PORTS_IO_PERF_IMPACT, Dashboard Category=Fabric Performance Impact., actionHndlr.c, line: 755, comp:md, ltime:2014/12/17-11:56:00:671909 # mapsdb --show all root> mapsdb --show 1 Dashboard Information: ======================= Active policy: dflt_aggressive_policy ………………… 2 Switch Health Report: ======================= Current Switch Policy Status: HEALTHY 3.1 Summary Report: =================== Category |Today |Last 7 days | -------------------------------------------------------------------- Port Health |No Errors |Out of operating range | Fru Health |In operating range |In operating range | Security Violations |No Errors |In operating range | Fabric State Changes |No Errors |In operating range | Switch Resource |In operating range |In operating range | Traffic Performance |In operating range |In operating range |


FCIP Health |Not applicable |Not applicable | Fabric Performance Impact|In operating range |In operating range | <==========>

Result 1. PASS. Test Passed. The bottlenecked ports are displayed on the MAPS dashboard and also a raslog

warning is created.

Storage Device – QOS Integrity

Test Objective 1. Verify storage device behavior and validate traffic characteristics with different QoS zones. Procedure Test Execution: 1. Setup initiator-target pairs with Low/Medium/High QoS zones in the fabric. 2. Start I/O from all hosts and verify I/O statistics. Result Validation: 1. Check I/O logs and verify I/O continues without any errors. 2. Check switch error logs and switch port status for any errors.

<==========> # porterrshow root> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 <==========>

Result 1. PASS. Test Passed. I/O from hosts in all QoS zones completed successfully without any errors.

Storage Device – FC Protocol Jammer Test Suite

Test Objective 1. Perform FC Jammer Tests including areas such as: CRC corruption, packet corruption, missing frame,

host error recovery, target error recovery.


Procedure Test Execution: 1. Insert Jammer device in the I/O path on the storage end. 2. Execute the following Jammer scenarios:

- Delete one frame - Delete R_RDY - Replace CRC of data frame - Replace EOF of data frame - Replace “good status” with “check condition” - Replace IDLE with LR - Truncate frame - Create S_ID/D_ID error of data frame

Result Validation: 1. Check the host and storage logs for any errors. 2. Check the switch logs and interface stats for any errors.

<==========> # errdumpall # porterrshow root> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 <==========>

3. Verify jammer operations and recovery with Analyzer. Result 1. PASS. Test Passed. Host and target were able to recover from the errors and continue I/O operations.

Stress and Error Recovery with Device Multipath

Storage Device Fabric IO Integrity – Congested Fabric

Test Objective 1. From all available initiators, start a mixture of READ/WRITE/VERIFY traffic with random data patterns

continuously to all their targets overnight. 2. Verify no host application failover or unexpected change in I/O throughput occurs. 3. Configure fabric & devices for maximum link & device saturation.


Procedure Test Execution: 1. Start FC I/O to the storage array from multiple hosts. 2. Setup a mix of READ/WRITE traffic. Result Validation: 1. Check the host and storage logs for any errors. 2. Verify the link congestion and check the switch logs for any errors.

<==========> # errdumpall # portperfshow # porterrshow root> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 <==========>

3. Check I/O generator tool logs to verify I/O runs without errors. Result 1. PASS. All I/O completed without errors. All validation checks passed.

Storage Device Integrity – Device Recovery from Port Toggle

Test Objective 1. With I/O running, perform a quick port toggle on every Storage Device & Adapter port. 2. Verify host I/O will recover. 3. Sequentially performed for each Storage Device & Adapter port. Procedure Test Execution: 1. Setup multipath on host and start I/O. 2. Perform multiple iterations of sequential port toggles across initiator and target switch ports.


Result Validation: 1. Check switch port status after toggle and for any errors in the switch error logs.

<==========> # errdumpall # portstatsshow X # portshow X root> portshow 19 portHealth: HEALTHY ………… portState: 1 Online ………….. portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normal portSpeed: N16Gbps <==========>

2. Check host multipath status on hosts to verify the toggled path recovers. <==========> - Windows: mpclaim –s –d - Linux: multipath –ll - VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>

3. Check host and storage error logs and verify I/O continues without errors. Result 1. PASS. I/O failed over and recovered successfully. All validation checks passed.

Storage Device Integrity – Device Recovery from Device Relocation

Test Objective 1. With I/O running, manually disconnect and reconnect port to different switch in same fabric. 2. Verify host I/O will failover to alternate path and toggled path will recover. 3. Sequentially performed for each Storage Device & Adapter port. 4. Repeat test for all switch types. Procedure Test Execution: 1. Setup multipath on host and start I/O. 2. Move storage target ports to different switch ports in the fabric.


Result Validation: 1. Check for any errors in the switch error logs and the switch port status at the new switch port.

<==========> # errdumpall # portstatsshow X # portshow X root> portshow 19 portHealth: HEALTHY ………… portState: 1 Online ………….. portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normal portSpeed: N16Gbps <==========>

2. Check host multipath status on hosts to verify the toggled path recovers. <==========> - Windows: mpclaim –s –d - Linux: multipath –ll - VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>


Storage Device Stress – Device Recovery from Device Port Toggle – Extended Run

Test Objective 1. Sequentially toggle each Initiator and Target ports in fabric. 2. Verify host I/O will recover to alternate path and toggled path will recover. 3. Run for 24 hours. Procedure Test Execution: 1. Setup multipath on host and start I/O 2. Perform multiple iterations of sequential port toggles across initiator and target switch ports. Result Validation: 1. Check switch port status after toggle and for any errors in the switch error logs.

<==========> # errdumpall # portstatsshow X # portshow X root> portshow 19


portHealth: HEALTHY ………… portState: 1 Online ………….. portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normal portSpeed: N16Gbps <==========>

2. Check host multipath properties for iSCSI hosts to verify the toggled path recovers. <==========> On Windows: mpclaim –s –d On Linux: multipath –ll On VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>


Storage Device Recovery – ISL Port Toggle (Sequential)

Test Objective 1. Sequentially toggle each ISL path on all switches. Host I/O may pause, but should recover. 2. Verify fabric ISL path redundancy between hosts & storage devices. 3. Verify host I/O throughout test. Procedure Test Execution: 1. Setup host multipath with links on different switches in the FC fabric and start I/O. 2. Ensure ISL redundancy by provisioning multiple ISL’s connected to different switches to provide

multiple paths through the fabric. <==========> # islshow root> islshow 1: 0-> 0 10:00:00:05:33:5b:7d:06 97 B6510_066_097 sp: 16.000G bw: 16.000G TRUNK CR_RECOV FEC 2: 1-> 1 10:00:00:27:f8:0f:b9:f0 91 B6510_066_091 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 3: 4-> 1 10:00:00:05:33:5b:10:e8 100 B6510_066_100 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 4: 34-> 34 10:00:00:27:f8:66:f3:81 92 B6520_066_92 sp: 16.000G bw: 32.000G TRUNK QOS CR_RECOV FEC 5: 40-> 40 50:00:53:31:39:65:ae:28 160 fcr_fd_160 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 6: 41-> 41 50:00:53:35:b1:d1:de:28 1 fcr_fd_1 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC <==========>

3. Perform multiple iterations of sequential ISL toggles across the fabric.


Result Validation: 1. Check FC fabric status after ISL toggle. Verify all nodes are online.

<==========> # fabricshow root> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083" The Fabric has 5 switches Fabric Name: SSR <==========>

2. Check the switch logs for any errors and verify I/O failed over to alternate ISL path in the fabric. <==========> # errdumpall # portperfshow # porterrshow root> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 <==========>

3. Check host and storage error logs and verify I/O continues without errors. Result 1. PASS. I/O re-routes to available paths in the fabric and recovers when the link is restored. All

validations checks passed.


Storage Device Recovery – ISL Port Toggle (Entire Switch)

Test Objective 1. Sequentially, and for all switches, disable all ISLs on the switch under test. 2. Verify fabric switch path redundancy between hosts & storage devices. 3. Verify switch can merge back in to the fabric. 4. Verify host I/O path throughout test. Procedure Test Execution: 1. Setup host multipath with links on different switches in the FC fabric and start I/O. 2. Ensure ISL redundancy by provisioning multiple ISL’s connected to different switches to provide

multiple paths through the fabric. <==========> # islshow root> islshow 1: 0-> 0 10:00:00:05:33:5b:7d:06 97 B6510_066_097 sp: 16.000G bw: 16.000G TRUNK CR_RECOV FEC 2: 1-> 1 10:00:00:27:f8:0f:b9:f0 91 B6510_066_091 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 3: 4-> 1 10:00:00:05:33:5b:10:e8 100 B6510_066_100 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 4: 34-> 34 10:00:00:27:f8:66:f3:81 92 B6520_066_92 sp: 16.000G bw: 32.000G TRUNK QOS CR_RECOV FEC 5: 40-> 40 50:00:53:31:39:65:ae:28 160 fcr_fd_160 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 6: 41-> 41 50:00:53:35:b1:d1:de:28 1 fcr_fd_1 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC <==========>

3. Perform multiple iterations of sequentially disabling all ISLs on a switch in the fabric. Result Validation: 1. Check FC fabric status after ISL toggle. Verify all nodes are online.

<==========> # fabricshow root> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083" The Fabric has 5 switches


Fabric Name: SSR <==========>


3. Check host and storage error logs and verify I/O continues without errors. Result 1. PASS. I/O failed over to alternate path and recovered once the switch merged back in the fabric. All

validations checks passed.

Storage Device Recovery – Director Blade Maintenance

Test Objective 1. Toggle each blade on the director in sequential order. 2. Include blade enable/disable, power on/off, and reboot testing. Procedure Test Execution: 1. Uplink edge switch ISL’s to different blades on the directors. 2. Setup host multipath with links on different switches in the FC fabric and start I/O. 3. Perform multiple iterations of sequential disable/enable, power on/off and reboot of all blades on

the 8510 directors.


Result Validation: 1. Check FC fabric status after the blade toggle. Verify all nodes are present in the fabric.



3. Check host and storage error logs and verify I/O continues without errors. Result 1. PASS. I/O failed over to alternate path and recovered once the blade recovered from the disruption.


Storage Device Recovery – Switch Offline

Test Objective 1. Toggle each switch in sequential order. 2. Include switch enable/disable, power on/off, and reboot testing. Procedure Test Execution: 1. Setup host multipath with links on different switches in the FC fabric and start I/O. 2. Perform multiple iterations of sequential disable/enable, power on/off and reboot of all the switches

in the fabric. Result Validation: 1. Check FC fabric status after the switch toggle. Verify all nodes are present in the fabric.


2. Check the switch logs for any errors and verify the toggled switch has recovered. <==========> # errdumpall # switchshow root> switchshow switchName: B6510_066_088 switchType: 109.1 switchState: Online switchMode: Native switchRole: Subordinate switchDomain: 88 switchId: fffc58 switchWwn: 10:00:00:27:f8:06:23:28 zoning: ON (SSR)


switchBeacon: OFF FC Router: ON FC Router BB Fabric ID: 100 Address Mode: 0 Fabric Name: SSR_2 <==========>

3. Check host and storage error logs and verify I/O continues without errors. Result 1. PASS. I/O failed over to alternate path and recovered once the switch merged back in the fabric.

Storage Device Recovery – Switch Firmware Download

Test Objective 1. Sequentially perform firmware maintenance procedure on all device connected switches under test. 2. Verify Host I/O will continue (with minimal disruption) through the “firmware download” and device

pathing will remain consistent. Procedure Test Execution: 1. Setup host multipath with links on different switches in the FC fabric and start I/O. 2. Sequentially perform firmware upgrades on all switches in the fabric. Result Validation: 1. Verify firmware upgrade completes successfully on each switch node and they merge back in the FC

fabric. <==========> # version root> version Kernel: 2.6.14.2 Fabric OS: v7.3.1 Made on: Thu Dec 11 14:30:38 2014 Flash: Wed Dec 17 09:53:59 2014 BootProm: 1.0.11 # fabricshow root> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"


The Fabric has 5 switches Fabric Name: SSR <==========>

2. Check I/O generator tool logs to verify I/O runs without errors throughout the firmware upgrade. 3. Check the switch logs for any errors and verify I/O resumes on the node after the firmware upgrade

is complete. <==========> # errdumpall # portperfshow <==========>

Result 1. PASS. I/O operations completed without any errors. I/O failed over to alternate path during the switch

reload after firmware upgrade and resumed after the switch was online. All validation checks passed.

Storage Device Fibre Channel Routing (FCR) Internetworking Tests

Storage Device Internetworking Validation w/FC host

Test Objective 1. Configure two FC fabrics with FCR. 2. Verify that edge devices are imported into adjacent name servers and hosts have access to their

routed targets after FC routers are configured.

Procedure Test Execution: 1. Setup FCR in an Edge-Backbone-Edge configuration. 2. Setup LSAN zoning and verify host access to target LUNs and start I/O.

Result Validation: 1. Verify name server and FCR fabric state.

<==========> # fcrfabricshow root> fcrfabricshow FC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82, Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83, Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name)


------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" # fcrproxydevshow root> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID in Fabric in Fabric ---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported ………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 Imported Total devices displayed: 20 <==========>

2. Verify I/O runs successfully without any errors. Result 1. PASS. I/O completed successfully. Both Edge fabrics have the corresponding proxy name server entries

for the host and target ports.

Storage Device Internetworking Validation w/FCoE Using VDX FlexPort

Test Objective 1. Configure a FC fabric with FCR connected to an FCoE fabric. 2. Verify that edge devices are imported into adjacent name servers and hosts have access to their

routed targets after FC routers are configured. Procedure Test Execution: 1. Add FCoE VCS fabric to FCR setup. 2. Setup LSAN zoning and verify host access to target LUNs and start I/O. Result Validation: 1. Verify name server and FCR fabric state.

<==========> # fcrfabricshow root> fcrfabricshow FC Router WWN: 10:00:00:05:33:5b:7d:06, Dom ID: 97, Info: 10.38.66.97, "B6510_066_097" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 18 110 10.38.66.120 10:00:50:eb:1a:62:8c:33 "VDX6740_066_120"


FC Router WWN: 10:00:00:05:33:5b:10:e8, Dom ID: 100, Info: 10.38.66.100, "B6510_066_100" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 8 110 10.38.66.119 10:00:50:eb:1a:62:83:7b "VDX6740_066_119" # fcrproxydevshow root> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID in Fabric in Fabric ---------------------------------------------------------------------------- 110 50:05:07:60:5e:80:76:52 02f001 120 612800 Imported 110 50:05:07:60:5e:80:76:71 02f101 120 642300 Imported 120 10:00:8c:7c:ff:00:48:00 03f101 110 781000 Imported 120 10:00:8c:7c:ff:00:48:01 03f001 110 771000 Imported Total devices displayed: 4 <==========>

2. Verify I/O runs successfully without any errors. Result 1. PASS. I/O completed successfully. Both Edge fabrics have the corresponding proxy name server entries

for the host and target ports.

Storage Device Edge Recovery after FCR Disruptions

Test Objective 1. Configure FCR for Edge-Backbone-Edge configuration. 2. With IO running, validate device access and pathing. 3. Perform reboots, switch disables, and port-Toggles on Edge connections to disrupt device pathing and

IO. 4. Verify path and IO recovery once switches and ports recover. Procedure Test Execution: 1. Setup FCR in an Edge-Backbone-Edge configuration. 2. Setup LSAN zoning and verify host access to target LUNs and start I/O. 3. Perform sequential reboots, switch disables and ISL port toggles on the switches in the edge fabric. Result Validation: 1. Verify FCR fabric state throughout the disruptions.

<==========> # fcrfabricshow root> fcrfabricshow FC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82, Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name)


------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83, Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" # fcrproxydevshow root> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID in Fabric in Fabric ---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported ………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 Imported Total devices displayed: 20 <==========>

2. Check the switch logs for any errors. <==========> # errdumpall # portperfshow <==========>

3. Check host and storage logs, and verify I/O runs without any errors. Result 1. PASS. I/O fails over to available switch path and recovers when disrupted switch is restored.

Storage Device Backbone Recovery after FCR Disruptions

Test Objective 1. Configure FCR for Backbone-Edge configuration. 2. With IO running, validate device access and pathing. 3. Perform reboots, switch disables, and port-Toggles on Backbone connections to disrupt device

pathing and IO. 4. Verify path and IO recovery once switches and ports recover.


Procedure Test Execution: 1. Connect array target ports to backbone fabric in an Edge-Backbone configuration. 2. Setup LSAN zoning and verify host access to target LUNs and start I/O. 3. Perform sequential reboots, switch disables and ISL port toggles on the switches in the backbone

fabric. Result Validation: 1. Verify FCR fabric state throughout the disruptions.

<==========> # fcrfabricshow root> fcrfabricshow FC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82, Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ----------------------------------------------------------------- 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83, Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ----------------------------------------------------------------- 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" # fcrproxydevshow root> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID in Fabric in Fabric ---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported ………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 Imported Total devices displayed: 20 <==========>

2. Check the switch logs for any errors. <==========> # errdumpall # portperfshow <==========>


3. Check host and storage logs, and verify I/O runs without any errors. Result 1. PASS. I/O fails over to available switch path and recovers when disrupted switch is restored.

Optional/Additional Tests

Non-Disruptive Firmware Upgrade on Storage Device

Test Objective 1. Perform firmware maintenance procedure on the storage device. 2. Verify Host I/O will continue (with minimal disruption) through the “firmware download” and device

pathing will remain consistent. Procedure Test Execution: 1. Setup host multipath with links on different switches in the FC fabric and start I/O. 2. Perform firmware update on all nodes of the storage array. Result Validation: 1. Check the I/O generator tools logs to verify I/O completes without any errors. 2. Check the host and storage logs for any errors throughout the I/O operations. 3. Check the switch error logs and port stats for any errors or I/O drops.


Result 1. PASS. I/O completed successfully throughout the firmware upgrade process.

Synthetic I/O Workload Loop with Varying Block Sizes

Test Objective 1. Validate Storage/Fabric behavior while running a workload simulation test suite. 2. Areas of focus may include random and sequential data patterns of various block sizes and database

simulation.


Procedure Test Execution: 1. Setup 4 standalone hosts with 2 multipathed initiator ports for I/O generation. 2. Use Medusa I/O tool for generating I/O and simulating workloads. 3. Run random and sequential I/O in a loop at block transfer sizes of 512, 4k, 8k, 16k, 32k, 64k, 128k,

256k, 512k, and 1m. Include a nested loop of 100% read, 100% write, and 50% read/write. 4. Run File Server simulation workload. 5. Run Microsoft Exchange Server simulation workload. Result Validation: 1. Check the I/O generator tools logs to verify I/O completes without any errors. 2. Check the host and storage logs for any errors throughout the I/O operations. 3. Check the switch error logs and port stats for any errors or I/O drops.


Result 1. PASS. All workload runs were monitored at the host, storage and fabric and verified they completed without any I/O errors or faults.

VMware I/O Analyzer Workload Test

Test Objective 1. Validate Storage/Fabric behavior while running a virtual workload simulation test suite. 2. Areas of focus include VM environments running de-duplication/compression data patterns and

database simulation.


Procedure Test Execution: 1. Setup an ESX cluster of 2 hosts with 4 worker VMs per host. 2. Use VMware I/O Analyzer tool for generating I/O and simulating workloads.

- Run random and sequential I/O at large and small block transfer sizes. - Run SQL Server simulation workload. - Run OLTP simulation workload. - Run Web Server simulation workload. - Run Video on Demand simulation workload. - Run Workstation simulation workload. - Run Exchange server simulation workload.

Result Validation: 1. Check the I/O generator tools logs to verify I/O completes without any errors. 2. Check the host and storage logs for any errors throughout the I/O operations. 3. Check the switch error logs and port stats for any errors or I/O drops.


Result 1. PASS. All workload runs were monitored at the host, storage and fabric and verified they completed without any I/O errors or faults.