inspeed

whitepaper

IntroducingInSpeed™TechnologyBreaking NewCost/PerformanceBarriers in StorageNetworking

byThomas Hammond-DoelDirector of Technical Marketing

Vixel CorporationJanuary 2002

rea l wor ld so lu t ions fo r s to rage network ing

www.v ixe l . com | 1 ·888·AT·V IXEL

whitepaper Introducing InSpeed™ Technology

2

InSpeed Fills A NeedExpertise Creating New Technology .............................................................. 3An Economical Foundation ............................................................................ 3

InSpeed Technology DefinedReduces Complexity ....................................................................................... 4Robust, Economical Solutions ....................................................................... 4True Switch Architecture ................................................................................ 4No Latency and Transparent Operation ......................................................... 4

InSpeed ImplementationsStorage Switch-On-A-Chip ............................................................................. 5

JBOD to SBOD Conversions ....................................................................... 5Storage Switch Blades.................................................................................... 6

Vixel’s Storage Switch Offerings ............................................................... 6

InSpeed ApplicationsMultiple Topologies ........................................................................................ 6RAID Applications ........................................................................................... 6InSpeed Performance Metrics ........................................................................ 7Interconnect Price Factor ................................................................................ 8

InSpeed Performance Comparisons in a Real-World SettingReal Configurations......................................................................................... 8Tuning Applications ........................................................................................ 9Conservative Numbers ................................................................................... 9

Real Performance Results .......................................................................... 9Interoperability .............................................................................................. 11

Affordable and Simple Storage SwitchingBenchmarks ................................................................................................... 11Powerful Solutions ........................................................................................ 12


3

InSpeed Fills A NeedThe world is becoming increasingly sensitive to complexity and cost. According to users whohave deployed SANs, a high performance technology that significantly reduces the complexityand cost metrics while maintaining interoperability is not only demanded now, but is arequirement for growth in the Fibre Channel-based SAN marketplace. If this new technologyprovides the same performance as Fibre Channel fabric switching, the makings for a paradigmshift in the implementations of small to medium scale SANs are at hand. This innovativetechnology was invented by Vixel and exists today. It is called InSpeed™ technology.

“On the other hand, Gartner Dataquest believes the emergence of modular externalhardware RAID controller-based storage systems that incorporate a fabric I/Oinfrastructure represents groundbreaking technology that will change the game inthe competitive large-capacity enterprise storage market.”

–Roger W. CoxGartner Group

Research Brief 11/01

Expertise Creating New Technology

Vixel is a technology leader in Fibre Channel Fabric switching and the world leader inaffordable SAN interconnect solutions. Many companies have tried to approach the problemof mitigating the cost and complexity of SANs, but have failed. The extensive experience Vixelbrings to understanding affordable SAN deployments, as well as Vixel’s patented technology,has led to a true breakthrough in storage switching and has eliminated the roadblocksencountered by other vendors.

An Economical Foundation

Vixel foresaw the pressing requirement and growing market demand for improvingperformance metrics such as input/output operations per second (IOPS) within NAS andRAID storage systems. As an outgrowth of these demands, Vixel developed InSpeedtechnology, a technology that has exceeded all expectations. The results are significant withmeasurable benefits that provide a compelling foundation upon which high performancestorage architectures can be built. With this technology, an interesting side effect was created.Designed with high-performance minimum complexity and low cost, it was developed inorder to meet the demanding requirements of embedded storage systems. However, due to itsflexible nature, this same technology can be aimed at many previously unconsideredenvironments. In fact, with the use of InSpeed technology the user takes advantage of FibreChannel based SAN connectivity for less than the price of IP solutions such as iSCSI for smallto medium scale SANs.


4

InSpeed Technology DefinedVixel’s InSpeed is a brand new technology. InSpeed technology is an advanced switchingarchitecture coupling a non-blocking crossbar switch with unique loop port logic resulting in asingle chip capable of handling multiple Fibre Channel devices in embedded, blade and boxenvironments. Actual implementations include embedded SERDES.

Reduces Complexity

Vixel’s InSpeed technology, publicly introduced on July 30, 2001 (http://www.vixel.com/pressroom/07302001.html), features the highest performance, lowest cost interconnectarchitecture for 1 Gb/s and 2 Gb/s storage networks. It reduces the complexity and simplifiesthe deployment and management of SANs. InSpeed technology allows storage traffic to beswitched at the arbitrated loop level, eliminating the need for attached devices to supportcomplex, ever changing and ambiguous fabric services standards. IT personnel can now deploySANs without a PhD in Fibre Channel standards and without the interoperability problemsintroduced by fabric services supported interconnect devices.

Robust, Economical Solutions

InSpeed technology advances the reach of Fibre Channel switch technology, allowing it to beembedded into environments that previously could not utilize switching technology. Theseareas include RAID, TAPE, NAS and next generation super-scalar storage systems. InSpeedtechnology is “future-proofed” by supporting both 1 Gb/s and 2 Gb/s Fibre Channel devicespeeds. Being the industries highest integrated Fibre Channel switch technology, Vixel InSpeedtechnology adds performance and system robustness that, for the first time, enables new levelsof solution scaling and economics.

True Switch Architecture

InSpeed technology enables the next generation of simple and affordable storage switching byimplementing a high performance non-blocking crossbar switch architecture. This is incontrast to traditional switch vendors use of the older, more error prone and non-scalablememory-mapped switching architecture. The resulting performance of an InSpeed-enabledASIC often exceeds Fabric services switching speeds - at a fraction of the cost and complexityof a Fabric services based switch.

No Latency and Transparent Operation

During initialization, switches based on InSpeed connect all devices together in a standard FC-AL2 arbitrated loop using standards-based protocols. Upon receipt of the Fibre Channelprimitive CLS at the end of the initialization process, InSpeed allows immediate transition tothe switching mode. When arbitration is attempted, it analyzes connections and enablesresponses according to its current configuration. InSpeed operation is completely transparentto attached devices. In fact, attached devices require significantly less software to run inInSpeed environments than in fabric services based switching environments. This results inquicker initialization, less server CPU cycles and less management by the IT staff. There is noneed for matching all the fabric services software revision levels or keeping up with all theambiguous and incomplete ANSI fabric services standards.


5

InSpeed ImplementationsThe following sections look at different uses of InSpeed technology. The general categories ofimplementation are in ASICs, blades and boxes.

Storage Switch-On-A-Chip

InSpeed technology provides storage switch-on-a-chip (ASIC) functionality for use in manydifferent applications. The first such availability of this technology is inside the VitesseVSC7192 ASIC. The VSC7192 can be used to turbo-charge RAID, TAPE, NAS and nextgeneration super-scalar storage systems. The VSC7192 can operate as an unmanaged stand-alone device or can intelligently monitor, correct and report system status. The VSC7192 isavailable today and has been adopted by several end-user customers.

JBOD to SBOD Conversions

A powerful application of InSpeedSwitch-On-A-Chip technology is toconvert a JBOD (Just a Bunch of Disks)to an SBOD (Switched Bunch of Disks),as shown in Figure 1. SBODs used inapplications accelerate performance anddramatically reduce FC-AL latency, as isshown in topologies 6, 7, and 8 of Table1. Network Appliance, for example, willembed InSpeed technology into theirJBODs to convert them into SBODs.This will effectively multiply thecapabilities of their NAS solutions.Other vendors are implementing systemswhere InSpeed is deeply embedded intotheir system architectures, providingthem with unprecedented performance.

Table 1IOPS & Price Performance Capabilities for RAIDS

Topology # Controller Interconnect Target IOPS IOPS $ FactorTechnology

1 1 Gb/s* Loop 1Gb/s JBOD 4000 100% 100%

2 1 Gb/s InSpeed 1Gb/s JBOD 5600 140% 130%

3 1 Gb/s Fabric 1Gb/s JBOD 6000 150% 540%

4 2 Gb/s InSpeed 1Gb/s JBOD 10000 250% 140%

5 2 Gb/s Fabric 1Gb/s JBOD 12000 300% 740%

6 2 Gb/s **Daisy Chain** 2Gb/s SBOD 13000 325% 110%

7 2 Gb/s InSpeed 2Gb/s SBOD 16000 400% 200%

8 2 Gb/s Fabric 2Gb/s SBOD 17000 425% 800%

* Reference Technology **Daisy Chain = No Interconnect Device

JBOD

Controller Cardor

Backplane

DISK0

DISK1

DISK2

DISK3

DISK4

DISK5

DISK6

DISK7

DISK8

DISK9

SBOD

InSpeed-basedController Card or

Backplane

DISK0

DISK1

DISK2

DISK3

DISK4

DISK5

DISK6

DISK7

DISK8

DISK9

X - b y - X S w i t c h M a t r i x

Figure 1JBOD vs. SBOD Architecture


6

Storage Switch Blades

Storage switch blades using InSpeed technology can be used to provide easy connection toRAIDs, or can be used to help consolidate multiple servers. Applications using storage switchblades range from aerospace to NEBS compliant telecommunications systems. Theirversatility, low real-estate, stand-alone capabilities, and low power consumption make the useof InSpeed technology in storage switch blades ideal. The InSpeed-based controller card shownin Figure 1 on page 5 is an example of such a blade that is resident in a JBOD chassis andconverting it into an SBOD.

Vixel’s Storage Switch Offerings

Vixel’s InSpeed enabled 2 Gb/s storage switches currently consist of 12-port, half-rackunmanaged storage switches, and 12-port, full-rack managed storage switches. These devicesare currently being designed into the products of major OEMs for a multitude of uses. Theresults of these designs will increase performance by being embedded in enterprise level RAIDand TAPE systems and will bring true affordable SANs to the small and medium scale ITshops. In addition, they will enable next generation super-scalar storage systems to offerscaling and performance never before offered or possible.

InSpeed Applications

Multiple Topologies

Applications of InSpeed technology include embedding it into RAID, NAS, and TAPE storagesystems, treed/daisy-chained JBODs or treed/daisy-chained SBODs, server and storageconsolidation in small and medium scale SANs, and connection of storage segments to largerfabric infrastructures.

RAID Applications

There are two main categories of RAID architectures today: monolithic and modular.Examples of monolithic architectures are the EMC Symmetric, the IBM Shark and HitachiLightning. Examples of modular architectures are in products from Compaq, Dell, LSI, mid-range IBM, and XIOtech.

Monolithic RAID architectures take advantage of InSpeed technology in the following areas:

• Provide fan out from the host/SAN side connection to the RAID• Provide connectivity between the front-end and backend modules• Provide connectivity from the RAID controllers to the drives• Provide drive isolation within attached JBOD cabinets to convert them into SBODs

InSpeed technology is currently being designed into these types of architectures.

Modular RAID architectures take advantage of InSpeed technology in the following areas:

• Provide connectivity between the RAID controller and the backend JBOD cabinets• Provide drive isolation and switching within a JBOD cabinet to convert it into an SBOD

Note that some mid to high-end NAS systems typically support Fibre Channel JBOD drivesin the backend. These systems can utilize InSpeed technology in the same way a modularRAID controller can utilize it.


7

2 Gb/s RAIDController


2 Gb/s SBOD 2 Gb/s SBOD

Topology 7(2 Gb/s InSpeed & SBOD)InSpeed-based

Switch

Topology 8(2 Gb/s Fabric & SBOD)Fabric Services

Switch

Topology 6DaisyChainSBOD



1 Gb/s JBOD 1 Gb/s JBOD

Topology 1(Reference)

Hub

Topology 3(1 Gb/s Fabric)

Fabric ServicesSwitch

Topology 2(1 Gb/s InSpeed)

InSpeed-basedSwitch



2 Gb/s JBOD 2 Gb/s JBOD

Topology 4(2 Gb/s InSpeed)

InSpeed-basedSwitch

Topology 5(2 Gb/s Fabric)

Fabric ServicesSwitch


2 Gb/s SBOD 2 Gb/s SBOD

Finally, high-end tape libraries can also utilize InSpeed technology in the same way as amodular RAID controller, simply by substituting tape controllers in the place of JBODdrawers.

InSpeedPerformanceMetrics

The following data showsthe enormous benefit ofutilizing InSpeed technologyin a modular RAIDenvironment. Topology 1 inFigure 2 shows a referencetopology using standardJBOD cabinets attached todual RAID controllers. Thetopology allows for eightJBODs consisting of tendrives each, or 80 drives.The topology shown isgreatly simplified, but isrepresentative of a largenumber of implementations.The reference topologyshown is one large arbitratedloop consuming 82 devices.In being conservative, thisconfiguration can operate at4000 IOPS, assuming singleframe coalescing, 4Ktransfers, and a 50%/50%sequential read/write.

Now change the reference(Topology 1 in Figure 2)by adding InSpeed into theJBOD to convert it into anSBOD or by adding anInSpeed-based storageswitch between the RAIDcontroller and backendJBOD. Both 1 and 2 Gb/sspeeds are included (seetopologies 2–8 in Figure2). The base metric used inthis review, IOPS, isgenerally increasing, withthe various combinations,as shown in Table 1 onpage 5.

Figure 2Reference Backend RAID Topologies


8

HBA#3

HBA#1

HBA#2

JBOD 1

HBA#4

– Interconnect -Loop, Private Loop, Fabric Servicesor InSpeed-based Storage Switch

JBOD 8

InterconnectPrice Factor

An interesting datumincluded in Table 1 onpage 5 is theinterconnect pricefactor. The effect of anestimated price of theinterconnect, whichconsists of theconcentrator and theJBOD/SBODconnection, is shown.Figure 3 graphicallydepicts the varioustopologies against the price factor to clearly demonstrate how InSpeed technology drasticallyimproves performance, but at only a fraction of the cost of a Fabric switch.

InSpeed Performance Comparisonsin a Real-World SettingThe following highlights the high performance of InSpeed technology in real-world settings.This environment compares InSpeed with some traditional SAN interconnect solutions suchas fabric switches, loop switches and hubs in a small to medium size SAN. The referenceconfiguration uses pure FC-AL mode. All scenarios use the configuration in Figure 4.

RealConfigurations

Table 2 shows the fivedifferent configurationsfor which data ispresented. The Loopmode of the FC Switchis used as the reference.The private loop modeof the Vixel 9200 2Gb/s Fabric switch isthe mode in which an estimated 75% of all Fabric switches operate.The Fabric mode of the Vixel 9200 Fabric switch uses FL_Ports for all 12 connected ports.

Table 2Real-World Configurations

Interconnect Technology Mode BandwidthHub Loop Shared

9200 Fabric Switch Private Loop Switching Switching

9200 Fabric Switch Fabric Switching

InSpeed-based Storage Switch HBAs On Tree Ports Switching

InSpeed-based Storage Switch HBAs On Device Ports Switching

HIGH PERFORMANCELOW PERFORMANCE

HIG

H C

OS

TLO

W C

OS

T

1 Gb/s Controller,Fabric, 1 Gb/s JBOD

1 Gb/s Controller,Loop, 1 Gb/s JBOD

1 Gb/s Controller,InSpeed, 1 Gb/s JBOD

2 Gb/s Controller,InSpeed, 2 Gb/s SBOD

2 Gb/s Controller,Daisy Chain, 2 Gb/s SBOD

2 Gb/s Controller,InSpeed, 2 Gb/s JBOD

2 Gb/s Controller,Fabric, 2 Gb/s JBOD 2 Gb/s Controller,

Fabric, 2 Gb/s SBOD

Figure 3Price/Performance Topologies for RAIDs

Figure 4Real-World Configuration


9

Tuning Applications

The Tree and Device ports for the two InSpeed-based Storage Switches, shown in Table 2 onpage 8, describe two of the three different modes in which the FC Switches may operate.These modes can be tuned to a particular application.

Conservative Numbers

Data comparing competitive Fabric switches to the Vixel 9200 has been collected, but thatinformation will not be presented here. In all cases, the Vixel 9200 generally outperforms allcompetitors in real-world implementations of multiple servers and one or more RAID devices.The numbers presented showing the performance of the InSpeed-based devices is even betterwhen compared to competitive products.

Real Performance Results

Figures 5, 6 and 7 show SCSI IOPS, MBps, and relative performance to arbitrated loop. The4k 50/50 Random Read/Write test case shows all devices performing about the same. This isdue to performance of the system being disk-bound rather than interconnect-bound.

Figure 5IOPs Comparison

4k50/50

RandomR/W

256kSeq

Writes

256kSeq

Reads

64kSeq

Writes

64kSeq

Reads

4kSeq

Writes

4kSeq

Reads

IO T

ES

T

2.125 Gb/s, 4 hosts (Emulex 950), 60 targets (Seagate 451),6 enclosures, 4 outstanding IOs

10

00

00

90

00

0

80

00

0

70

00

0

60

00

0

50

00

0

40

00

0

30

00

0

20

00

0

10

00

00

SCSI IOPs

InSpeed Loop

9200 Loop

9200 Fabr ic

InSpeed Tree

InSpeed Device


10

Figure 6MBps Comparison

Figure 7IOPs vs. Loop Performance Ratio

4k50/50

RandomR/W

256kSeq

Writes

256kSeq

Reads

64kSeq

Writes

64kSeq

Reads

4kSeq

Writes

4kSeq

Reads

TE

ST

CO

NF

IG2.125 Gb/s, 4 hosts (Emulex 950), 60 targets (Seagate 451),

6 enclosures, 4 outstanding IOs

500

MBps

InSpeed Loop 9200 Loop 9200 Fabric

InSpeed Tree InSpeed Device

10

0

15

0

20

0

25

0

30

0

35

0

40

0

45

0

50

0

55

0

60

0

65

0

70

0

75

0

4k50/50

RandomR/W

256kSeq

Writes

256kSeq

Reads

64kSeq

Writes

64kSeq

Reads

4kSeq

Writes

4kSeq

Reads

TE

ST

CO

NF

IG

2.125 Gb/s, 4 hosts (Emulex 950), 60 targets (Seagate 451),6 enclosures, 4 outstanding IOs

0.0

0

Relative Performance

1.0

0

2.0

0

3.0

0

4.0

0

5.0

0

6.0

0

7.0

0

InSpeed Loop

9200 Loop

9200 Fabric

InSpeed Tree

InSpeed Device


11

Two of the three write-only test cases show both private loop switching and fabric switchingare faster than an InSpeed-enabled storage switch. This is due to buffers available with privateloop and fabric services switches and the inherent slowness of hard drives when performingwrite operations. In all of the read-only tests, the InSpeed-enabled storage switch is faster thanthe private loop or fabric switches. In a real-world environment where the mixture of reads towrites distinctly favors reads, InSpeed-enabled storage switches are typically faster than privateloop or fabric services switches.

The relative performance chart (Figure 7 on page 10) shows that an InSpeed-enabled storageswitch can be 600% faster than arbitrated loop depending on the topology and traffic. Thisrelative performance is all at 2 Gb/s. When compared to 1 Gb/s, the results are almost twicethat shown. It should be noted that although InSpeed performs faster than a traditional fabricservices-based switch in real-world SAN deployments, it costs a fraction of the amount and ismuch less complex to deploy and manage than traditional fabric services-based switches.

Interoperability

As SANs have matured, interoperability issues have been slowly becoming less of an issue, buta SAN implementing fabric services has the largest set of outstanding issues. InSpeedtechnology does not participate in fabric services and is essentially transparent to the SAN.This means that SANs implemented with InSpeed technology do not have to implementcomplex and ever changing fabric services standards. Ease of integration because of nointeroperability issues is a powerful benefit to the system architect and end user. This isanother key point to emphasize. If a SAN implements fabric services, the level of special FibreChannel IT expertise to manage the SAN becomes huge and the needed type of personnel arein short supply. On the other hand, a SAN without fabric services can be managed withcurrent IT personnel with just a limited knowledge of Fibre Channel. This approaches the“holy grail” of SAN technology at IP network complexity and affordability.

Affordable and Simple Storage SwitchingThere is a distinct need for more affordable and less complex Fibre Channel SANs. This needis especially acute for small to medium scale IT deployments. Vixel’s InSpeed technology is anew solution showing up in areas where traditional Fibre Channel interconnect technology isdeemed too complex or too expensive. InSpeed-based deployments offer scalability, simplicityand performance at an exceptional value.

Benchmarks

When considering architectures that increase the IOPS – especially when concerned withmetrics such as the Transaction Processing Performance Council (TPPC - www.tpc.org)benchmarks – InSpeed technology provides a viable and exciting alternative to competitiveapproaches. The TPPC routinely sees vendors’ struggle for the difference of a few percentagepoints in their system performance. The ability to multiply performance of SAN deployments– be them RAID-based storage or NAS – demands consideration.


12

Powerful Solutions

Depending on the applications a vendor is deploying, the topologies provided in this paperclearly identify those areas where InSpeed can be used to add significant value. In particular, asthe industry progresses more towards 2 Gb/s systems, a system using Vixel’s InSpeedtechnology in conjunction with 2 Gb/s devices, will see increases over today’s IOPS ofhundreds of percent while adding only incremental cost to the overall solution. With thecombination of minimal cost increases and accelerated performance vendors find they havemore powerful solutions.

© Copyright 2002 Vixel Corporation. Contents Subject to Change.

inspeed

Documents