802.17 · pdf filenodeb reactive. data communication division cypress dvj _transit_01 november...

Data Communication DivisionCYPRESS

November 14, 2001, page 1dvj_transit_01

802.17 presentation

m Prepared for 802.17, November 2001m Dr. David V. James

Chief ArchitectNetwork Processing SolutionsData Communications Division110 Nortech ParkwaySan Jose, CA 95134-2307Tel: +1.408.942.2010Fax: +1.408.942.2099Base: dvj@alum.mit.eduWork: djz@cypress.com

Data Communication DivisionCYPRESS

November 14, 2001, page 2dvj_transit_01

Flow control

Data Communication DivisionCYPRESS

November 14, 2001, page 3dvj_transit_01

data[n]unprovisioned or

unused provisioned

overprovisioned,unbounded latency

Arbitration classes

Class-B1

Class-C

provisioned bandwidth,bounded latency Class-B0

provisioned bandwidth,low latency Class-A

guar

ante

ed

Data Communication DivisionCYPRESS

November 14, 2001, page 4dvj_transit_01

nodeA

Opposing arbitration

• Data packets flow in one direction• Arbitration control flows in the other*

nodeB nodeC packetcontrol

Data Communication DivisionCYPRESS

November 14, 2001, page 5dvj_transit_01

Supported topologies

n A physical ring

n Dual ringlets

n Duplex ringlet

Data Communication DivisionCYPRESS

November 14, 2001, page 6dvj_transit_01

Internal MAC arbitration signals

• Arbitration affects opposing run • My congestion affects upstream node• Downstream congestion affects me

dataPaths

arbitration

arbitration

dataPaths

Data Communication DivisionCYPRESS

November 14, 2001, page 7dvj_transit_01

External MAC arbitration signals

arbs

arbs

application

MAC receives informationMAC FIFOs are $$, latency++, inflexible

Application receives informationAllows reordering and run selection

Data Communication DivisionCYPRESS

November 14, 2001, page 8dvj_transit_01

B

Arbitration related components

• Distinct class-A & class-B/C paths • Load dependent policing

passBC

passA

policeApoliceBA

Precedence: Fa, A, B, C, Fbc

C BpoliceC

C BpoliceC

C BpoliceC

Data Communication DivisionCYPRESS

November 14, 2001, page 9dvj_transit_01

Class-A flow control(proactive and reactive)

Data Communication DivisionCYPRESS

November 14, 2001, page 10dvj_transit_01

Class-A flow control

m ProactiveMinimal (nonexistent?) passBC transit bufferLess available bandwidthEach station maintains constant classAp traffic

m ReactiveSignificant passBC transit bufferFull bandwidth utilizationEach station responds/regenerates throttle messages

m Interoperable?This is a bandwidth vs memory $$ tradeoff

Data Communication DivisionCYPRESS

November 14, 2001, page 11dvj_transit_01

nodeA

Proactive class-A partitions

• Data packets go source-to-destination• Residue returns destination-to-source

to provide subsistence for transmissions

nodeB nodeC

Data Communication DivisionCYPRESS

November 14, 2001, page 12dvj_transit_01

nodeA

Proactive class-Acompatibility options

• Reactive node trickles class-A bandwidth• Reactive node recycles class-A bandwidth

class-A => class-A’, thus preserving BW

nodeB Reactive

Data Communication DivisionCYPRESS

November 14, 2001, page 13dvj_transit_01

nodeA

Reactive class-A control

• Transmission of packets causes• Backup of passBC FIFO that• Returns flow-control information that• Provides consumable idle packets

nodeB

(1)

(2)(3)

(4)

Data Communication DivisionCYPRESS

November 14, 2001, page 14dvj_transit_01

nodeA

Reactive class-A compatibility

• Flow control passes upstream• Proactive stations pass these indications

nodeB

(1)

(2)(3)

(4)

Data Communication DivisionCYPRESS

November 14, 2001, page 15dvj_transit_01

Frame formats

Data Communication DivisionCYPRESS

November 14, 2001, page 16dvj_transit_01

headerCRC

payload

Frame formats

type

headerCRC

header

32 bits

compact headerCRC

payloadtype

headerCRC

complete

sourceMacLodstLo srcHi

destinationMacHiheader

Data Communication DivisionCYPRESS

November 14, 2001, page 17dvj_transit_01

Header formatw

rap

destStationID

71

ring

destStationID

71

flood

1 8

mac

s

1

clas

s

2

depthBC

4

timeToLive

Data Communication DivisionCYPRESS

November 14, 2001, page 18dvj_transit_01

headerCRC

payload

RPR control frames

802.17 discover

headerCRC

header

headerCRC

payload

802.17 congest

headerCRC

header

Data Communication DivisionCYPRESS

November 14, 2001, page 19dvj_transit_01

Topology discovery

Data Communication DivisionCYPRESS

November 14, 2001, page 20dvj_transit_01

Frame interchanges

m Triggered on state changem Triggered on state changem Also sent periodically

– Automatic fault recovery– Piggyback on heartbeat

m Also distributes stationID addresses– Previous: derived from topology and EUI-48 info– Bit map supportive “reclaiming” precedence

m Robust!– Context-less behavior (update rate only)– No addressing or timeouts required

Data Communication DivisionCYPRESS

November 14, 2001, page 21dvj_transit_01

Topology collection

• Prepend your macAddress(no duplicate copies present…)

Data Communication DivisionCYPRESS

November 14, 2001, page 22dvj_transit_01

Topology maintenance

• Prepend your macAddress• Strip duplicate macAddress & beyond

Data Communication DivisionCYPRESS

November 14, 2001, page 23dvj_transit_01

headerCRC

payload

Discovery formats

type17

complete

dstLo ??

DISCOV

headerCRC

header

32 bits

ring bridge

destinationMacHi

Data Communication DivisionCYPRESS

November 14, 2001, page 24dvj_transit_01

CRC processing

Data Communication DivisionCYPRESS

November 14, 2001, page 25dvj_transit_01

CRC processing

m Store&forward/Cut-through agnosticm Invalid data is effectively discarded

– store-and-forward discards– cut-through stomps the CRC

m Maximize error-logging accuracy– Separate header&data CRCs– “most” corruptions hit the data

Data Communication DivisionCYPRESS

November 14, 2001, page 26dvj_transit_01

payloadCRC

payload

Separate header and data CRCs

headerCRC

header

Data Communication DivisionCYPRESS

November 14, 2001, page 27dvj_transit_01

crcA data[n]core

Cut-through CRCs

• Corrupted packet remains corrupted• Error logged when first detected • if (crcA!=crc) {

errorCount+= (crcA!=crc^STOMP);crcB= crc^STOMP;

}

node crcB data[n]core

Data Communication DivisionCYPRESS

November 14, 2001, page 28dvj_transit_01

Distinct CRCs reduces discards

• Discard the corrupted packet

data head node

X

data head node

X(nothing)

• Discard the corrupted data

head

Data Communication DivisionCYPRESS

November 14, 2001, page 29dvj_transit_01

??

End-to-end CRC protected TTL

CRC checkCRC generate

zero dt zero

EXOREXOR

crcB ttl' header

t+= 1

crcA ttl header

CRC checkCRC check/stomp

Data Communication DivisionCYPRESS

November 14, 2001, page 30dvj_transit_01

Pre-emption(a physical layer decision)

Data Communication DivisionCYPRESS

November 14, 2001, page 31dvj_transit_01

Pre-emption

m Suspend class-B/C for class-A packetm Only one level is sufficient

– class-A is the latency critical traffic– more levels complicate hardware

m Physical layer dependent– marginal for high BW & small packets– distinctive “suspend” symbol required

Data Communication DivisionCYPRESS

November 14, 2001, page 32dvj_transit_01

Pre-emption fragments

• Packets can be suspended• The class-A packet can be stripped

• egress queues are store&forward• distinctive idle markers needed

data[n]partB data[n]partA node data[n]partB classAdata[n]partA node data[n]partB Idle' data[n]partA

Data Communication DivisionCYPRESS

November 14, 2001, page 33dvj_transit_01

Pre-emption compatibility

• Pre-emption mandates egress S&F

• Simplistic node has no such S&F

• Interoperability burden on elegant• boundary node has S&F bypass • cut-through in preemptive domain

preemptive domain

Data Communication DivisionCYPRESS

November 14, 2001, page 34dvj_transit_01

Limits of scalability

n The metro area

n Terrestrial

n Geosynchronous

n To the curb

n To the home

Data Communication DivisionCYPRESS

November 14, 2001, page 35dvj_transit_01

Lessons of the past…m Flow control mandates 2-out-of-3

– Low latency transmissions– Fair bandwidth allocation– High bandwidth utilization

m Feedback control systems– Low latency signaling– Control can pass class-B/C packets– Separate class-A queue is utilized

m Other observations– Local control => global perversions – Fairness is inherently “approximate”– Strange beating sequences DO OCCUR

Data Communication DivisionCYPRESS

November 14, 2001, page 36dvj_transit_01

Allowed transmissions

A,B,Cb,Cc,F----≥≥0

A,BA,B,Cb,F--pass≥≥1/4

AA,FA,Fpasssend≥≥1/2

A,FA,FA,Fsendsend≥≥3/4

HILOnoneHILO

transmissionswarnings

Data Communication DivisionCYPRESS

November 14, 2001, page 37dvj_transit_01

Arbitration summarym Dual levels

– Class-A, pre-emptive low latency – Class-B, less latency sensitive

m Jumbo frames– Affect asynchronous latencies– NO IMPACT on synchronous latency

m Cut-through vs store-and-forward– Either should be allowed– Light-load latency DOES matter

Data Communication DivisionCYPRESS

November 14, 2001, page 38dvj_transit_01

MAC interface links

Data Communication DivisionCYPRESS

November 14, 2001, page 39dvj_transit_01

LiteLink features

mByte-wide 10Gb/s link (Gb/pin/s)– 11 pins total for clock, control, and data

mSLVS-400 transmitter, matched impedance– Technology independent 0.8V driver supply– .2V and .6V low and high signal levels

mDifferential receiver, matched impedance– Near DC encoding (5/11 or 6/11 ones)– Termination derived reference voltage

mSource synchronous, DDR clock

Data Communication DivisionCYPRESS

November 14, 2001, page 40dvj_transit_01

Byte-wide coding properties

enco

der

data{0..7}

strobed

ecod

er

data{0..7}

strobe

error

2) About 1/2 asserted on each link1) Uncoded strobe

3) About 1/2 asserted on each group

4) Benefits of error detectionvia legal code checking

Data Communication DivisionCYPRESS

November 14, 2001, page 41dvj_transit_01

LiteLink signaling

(…)(common reference)

(…)

enco

ded

Dat

a5a

strobe

enco

ded

Dat

a5b