everything you wanted to know about storage (but were too … · 2019-12-21 · ethernet / fc...

November 1, 2016

J Metz, Cisco Dror Goldenberg, Mellanox

Chad Hintz, Cisco Fred Knight, NetApp

Everything You Wanted To Know about Storage (But Were Too Proud To Ask)

Part Mauve

Today’s Presenters

2

J Metz SNIA Board of Directors

Cisco

Chad Hintz SNIA-ESF

Board Cisco

Fred Knight Standards

Technologist NetApp

Dror Goldenberg VP Software Architecture

Mellanox

SNIA Legal Notice

!   The material contained in this presentation is copyrighted by the SNIA unless otherwise noted.

!   Member companies and individual members may use this material in presentations and literature under the following conditions:

!   Any slide or slides used must be reproduced in their entirety without modification !   The SNIA must be acknowledged as the source of any material used in the body of any document containing material

from these presentations. !   This presentation is a project of the SNIA. !   Neither the author nor the presenter is an attorney and nothing in this presentation is intended

to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney.

!   The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK.

3

About SNIA

4

Agenda

!   Channel vs. Bus !   Control Plane vs. Data Plane !   Fabric vs. Network

5

Host Network Storage

Fabric Network

Channel Busses

Control Plane Data Plane

Off We Go!


6

Bus vs. Channel

Channel Physical or logical data communication line

Bus Transfers multiple signals simultaneously

7

Get On The Bus

!   Communication system that transfers multiple signals simultaneously between components !   Control bus – send commands !   Data bus – move data !   Address bus – tell me where to find it !   Power bus – shock it to me!

!   The bus can be… !   Shared or point to point !   Parallel or serial !   Internal or external

8

Busses in a Server

9

Memory Bus DDR4

Cache Coherency Bus

Memory Bus DDR4

Peripheral Bus PCIe

PCI Switch

NVMe

Peripheral Bus PCIe

Chipset

Storage Bus SATA/SAS

SMBus/ i2c

Get On The Bus

!   Common bus examples !   Motherboard: PCIe (devices), DDR4 (memory), AGP (graphics) !   Storage: SCSI, SATA, SAS !   Peripherals: USB, Lightning, FireWire, Thunderbolt, i2c/smbus

!   What is not a bus? !   Ethernet, Fibre Channel, InfiniBand !   These are networks & fabrics

10

Faster Bus Performance

!   Faster data transfer on each lane !   Higher frequencies (faster signal transfer rate) !   More data per message (efficient encoding)

!   More lanes in parallel

11

Channel Details

!   Path for transferring data !   Physical—Wire, fiber or board trace !   Logical—Digital TV channel; VPN link !   Mainframe channel subsystem (CSS) !   SSD/DRAM data transfer path

!   Physical, electronic, or logical separation !   Different wires/fibers !   Different frequencies, multiplexing

12

More About Channels

!   Channel vs. bus? !   Bus can be comprised of multiple physical channels (a.k.a. lanes) !   Logical channels can operate over a bus or network

!   Channel vs. lane? !   Can be synonymous, i.e. “Channel” = “Lane” !   Or one channel can have multiple lanes

13

Bus vs. Network

!   Bus

!   Inside box !   Short distances !   Shared / point-to-point links !   Not typically shared by multiple

systems !   Byte/LBA addressing typically

!   Network

!   Outside the box !   Longer distances !   Usually point-to-point links !   Shared by multiple systems

!   L2/3 addressing typically

14 Both Bus and Network can use multiple channels!

Network – IB/Eth/FC

15

Processor Node

Network

Processor Node

Processor Node

Consoles

RAID Storage

Subsystem

Switch Switch

Switch Switch

Bus v Channel - Summary

!   A bus is a topology of interconnected devices

!   A channel is a dedicated path for traffic that can sit upon a topology

16

Next Up…


17

Control Plane vs. Data Plane

Data Plane The Movement

Control Plane The Plan

18

Network Centric

19

Where are we going? Control Plane-Network Centric

!   Networking (SAN or Data Networks) devices use the control plane to: !   Learn which paths are available

!   Using static configuration (static routes-data networks) or via learning (routing protocols SAN or Data)

!   Figure out the best path(s) and tell the data plane !   Any packet the device has to fully process (routing

protocols, bpdu, SSH/Telnet) 20

I know where to GO, Jump on! Data Plane

!   Networking devices use the data plane to: !   Transmit or Move the traffic

from one interface to another based on what we have learned from the Control Plane

21

Storage Centric

22

Storage Control/Data Plane

23

Traditional Storage

24

Control Plane and Data Plane together in one unit

Where are we going? Control Plane-Storage Centric

25

Hyper-Converged Centric

26

! Centralized Control Plane ! Tight integration of x86

servers for compute and storage, networking and virtualization in all-in-one appliance.

Hyper-convergence-Control Plane Centralized Controller

VM VM VM VM

Local Physical Storage Hypervisor

Controller VM VM VM VM


Controller VM VM VM VM


Data v. Control Planes - Summary

!   A Data Plane is the path by which data is moved from device to device

!   A Control Plane is the path by which the management of that data plane is transmitted

28

Last But Not Least…


29

What is a Network

!   Definition !   SNIA: An interconnect that enables communication among a

collection of attached nodes, consisting of optical, electrical, or wireless transmission media, infrastructure in the form of hubs, switches, and/or routers, and protocols that make message sequences meaningful.

!   The Internet is the most famous Network.

30

Typical Network

31

Typical Network

32

LAN WAN

What is a Fabric

!   Definition: !   A Fabric is specialized type of Network. !   Wikipedia: Switched Fabric or switching fabric is a network

topology in which network nodes interconnect via one or more network switches (including crossbar switches). Because a switched fabric network spreads network traffic across multiple physical links, it yields higher total throughput than broadcast networks, such as early Ethernet.

33

Typical Computer Fabric

34

!   It’s a fabric, what kind of fabric?

Typical Computer Fabric

35

!   This one is a tight, mesh fabric

Typical SAN Fabric

36

Wire vs. Protocol

!   Network !   Ethernet (copper, optical, wireless) – IEEE 802.x family !   Dialup / DSL / Broadband “wires” – ITU + others !   TCP, IP, + lots more protocol - IETF

!   Fabric ! Fibre Channel wire (copper, optical)

!   InfiniBand wire (copper, optical) ! Fibre Channel protocol – INCITS T11 family !   InfiniBand protocol – IBTA

37

Wire vs. Protocol

Network / Fabric Type

Protocols Media Governing Body

Ethernet TCP / IP (1) Copper, Optical, Wireless

IEEE 802.x (2)

IETF

Dialup / DSL / FIOS / Broadband

TCP / IP (1) Copper, Optical ITU + others

Fibre Channel FC (FC-SW, FC-GS, etc) Copper, Optical INCITS T11 (2)

InfiniBand IB Copper, Optical IBTA (2)

38

(1) POP, HTTP, iSCSI, DHCP, DNS, etc (2) Optical Internetworking Forum (OIF) defines optical cable standards

Connected Fabrics

39

Network

Fabric Domain 1 Fabric Domain 2

Key thoughts

!   Generally speaking: !   Networks are thought of as larger entities (WWW) !   Fabrics are thought of as smaller more specialized entities

!   LAN = Local Area Network (office/campus) !   WAN = Wide Area Network (internet) !   SAN = Storage Area Network (FC)

!   Specialized for storage ! iSAN = Storage Area Network (iSCSI)

40

Ethernet / FC differences

!   Generally speaking: !   Ethernet: More loosely coupled - network switches have less

knowledge !   Lots of end points that talk to each other !   Next hop routing (IS-IS, OSPF, BGP, EIGRP) !   Destination based flow control (backpressure) !   Higher layer corrects for packet loss

! Fibre Channel: More tightly coupled - fabric switches have more knowledge

!   Initiators talk to targets !   Distributed routing knowledge (FSPF) !   Source based flow control (credits) !   Virtually lossless

41

Fabric v. Network - Summary

!   Networks make up a series of interconnected devices and can consist of multiple topologies

!   Fabrics are a type of network with distributed intelligence across all devices

42

Summary


43

Host Network Storage

Fabric Network

Channel Busses

Control Plane Data Plane

Summary

!   Channel vs. Bus !   A bus is a topology of interconnected devices !   A channel is a dedicated path for traffic that can sit upon a

topology !   Control Plane vs. Data Plane

!   A Data Plane is the path by which data is moved from device to device

!   A Control Plane is the path by which the management of that data plane is transmitted

!   Fabric vs. Network !   Networks make up a series of interconnected devices and can

consist of multiple topologies !   Fabrics are a type of network with distributed intelligence

across all devices

44

Other Storage Terms Got Your Pride? This is a Series!

!   Check out – Chartreuse https://www.brighttalk.com/webcast/663/215131 !   The Basics: Initiator, Target, Storage Controller, RAID. Volume Manager and more

!   Future Topics/Colors (in no particular order): !   Vermillion (What-If-Programming-and-Networking-Had-A-Baby Pod)

!   Coherence/Cache Coherence, Storage APIs, Block, File, Object, Byte Addressable, Logical Block Addressing

!   Teal (Buffering Pod) !   Ring Buffers, Queues/Queue Depth/Buffering/Flow Control

!   Rosé (iSCSI Pod) !   iSCSI offload, TCP offload, Host-based iSCSI

!   Turquoise (Where-Does-My-Data-Go Pod) !   Volatile v. Non-Volatile v Persistent Memory, NVDIMM v. RAM v. DRAM v. SLC v. MLC v.

TLC v. NAND v. 3D NAND v. Flash v SSDs v. NVMe, NVMe (the protocol) !   More… !   Follow us @SNIAESF so you don’t miss any

45

After This Webcast

!   Please rate this Webcast and provide us with feedback !   This Webcast and a PDF of the slides will be posted to the SNIA

Ethernet Storage Forum (ESF) website and available on-demand ! www.snia.org/forums/esf/knowledge/webcasts

!   A full Q&A from this webcast, including answers to questions we couldn't get to today, will be posted to the SNIA-ESF blog: ! sniaesfblog.org

!   Follow us on Twitter @SNIAESF !   Need help with all these terms? Download the 2016 SNIA Dictionary

!   http://www.snia.org/education/dictionary

46

Thank You

47

everything you wanted to know about storage (but were too … · 2019-12-21 · ethernet / fc...

Documents