1© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1

Fibre Channel, FCoE, and FCIP: Oh My!Evaluating Storage Protocols in Distance SolutionsJ Metz, Ph.D – Sr. Product Manager

@drjmetz

Distance

What do I need to know about storage distance?

How far can I go?

Am I using the right tool for the job?

2© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Don't miss out on any of them!

Know your physical layer

Know your protocol

Know your limitations!

Three Major Things

3© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

4© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4

Let's Get Physical!

How to Differentiate?Optics

Fibre Channel Optics

Ethernet Optics-Yes, can transmit Fibre Channel too!

Speed

Power (i.e., Strength)-Translates to distance

5© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Distance Requires Specialized SolutionsOptics

SFP-Up to 4Gbps

SFP+-8 Gbps+

Longer distances often mean lower throughput speeds

km

1 km

5 km

10 km

40 km

100 km

FC-SR

Eth-SR

FC-MR

FC/Eth-LR

FC/Eth-ER

300m

860m

4 km

CWDM DWDM

10 km

40 km

100 km

2000+ km

6© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Single Mode Fibre

Multimode Fibre

Three Major TypesCables

Copper-based-Generally not suitable for distances greater than 30m-So... not a "long-distance solution"

Optical-SMF: "Single Mode Fibre"

-Small diameter core-Allows only one mode of light to be sent-Longer effective distance - 100 km or less-Need transceivers that have more power

-MMF: "Multi-Mode Fibre"-Larger diameter core-Allows multiple paths (modes) to be sent at the same time-Shorter effective distance - around 16 km or less-Generally uses cheaper equipment and easier to interconnect-Good for general purpose applications (e.g., campus, building deployments)-Often orange in color

7© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

The Dark And Mysterious ForceDark Fibre

Cabling laid out by telecommunication companies that isn't being used (dark vs. 'lit')

Leased lines to companies who wish to communicate over great distances

Follows same rules as we've already gone over

8© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

9© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

Use The Proper Protocol

It's All About the SourceFibre Channel (FC)

Fibre Channel is a source-based mechanism for lossless delivery

Buffer to buffer (B2B) credits keep FC frames in order

1 buffer = 1 frame

10

© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Mo' Buffers Mean Mo' DistanceFC Distance

Longer distances require more time to receive acknowledgements

(Very expensive to not use a pipe)

Ports can "donate" buffer credits to other ports

Can send more data before acknowledgement

11

© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

On the Receiving EndFibre Channel over Ethernet (FCoE)

FCoE is a receiver-based mechanism for lossless delivery

Receiving switch has a "high water mark"

Includes frames already "in the bucket" as well as frames in flight.

When threshold is reached, PAUSE frame is sent (IEEE 802.1Qbb)

Distance is proportional to receiver capability

12

© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Go Long! No, Even Longer!Fibre Channel over IP (FCIP)

Other alternatives include running over lossy links

-No buffer credits-No Priority Flow Control

Usually done over 1GbE links

Acceleration tools (like IOA) are available

Best of both worlds:-Uses IP infrastructure and management resources to connect and extend FC SANs-Keep high-performance SAN base; whilst increasing data sharing between SANS transparently

13

© 2013 Cisco and/or its affiliates. All rights reserved.Storage Distance by Protocol Cisco Public

Visit Cisco Booth D209Twitter: @ciscoDC, #vmworld

Facebook: http://www.facebook.com/CiscoDC

Youtube: http://www.youtubecisco.com/datacenter

Cisco DCC Blog: http://blogs.cisco.com/datacenter

Slideshare: http://slideshare.com/CiscoDataCenter

Community: : https://communities.cisco.com/community/technology/datacenter

Pinterest: http://pinterest.com/ciscosystems/data-center

LinkedIn: http://www.linkedin.com search “Cisco Data Center” group

Google +: http://goo.gl/irm4b

In Collaboration with Intel®

Intel, the Intel logo, Xeon and Xeon inside are trademarks of Intel Corporation in the U.S. and other countries.

15

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15