intelligent network interfaced device and system for accelerated

(12) United States Patent Boucher et al.

US006427173B1

(10) Patent N0.: (45) Date of Patent:

US 6,427,173 B1 *Jul. 30, 2002

(54)

(75)

(73) ( * )

(21) (22)

(63)

(60)

(51) (52) (58)

(56)

4,366,538 A

INTELLIGENT NETWORK INTERFACED DEVICE AND SYSTEM FOR ACCELERATED COMMUNICATION

Inventors: Laurence B. Boucher, Saratoga; Stephen E. J. Blightman, San Jose; Peter K. Craft, San Francisco; David A. Higgen, Saratoga; Clive M. Philbrick, San Jose; Daryl D. Starr, Milpitas, all of CA (US)

Assignee: Alacritech, Inc., San Jose, CA (US)

Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days.

This patent is subject to a terminal dis claimer.

Appl. No.: 09/464,283 Filed: Dec. 15, 1999

Related US. Application Data

Continuation-in-part of application No. 09/439,603, ?led on Nov. 12, 1999, and a continuation-in-part of application No. 09/384,792, ?led on Aug. 27, 1999, and a continuation of application No. 09/067,544, ?led on Apr. 27, 1998, now Pat. No. 6,226,680 Provisional application No. 60/061,809, ?led on Oct. 14, 1997, and provisional application No. 60/098,296, ?led on Aug. 27, 1998.

Int. Cl.7 ...................... .. G06F 15/172; G06F 15/16

US. Cl. ...................... .. 709/238; 709/230; 709/250

Field of Search ............................... .. 709/230, 238,

709/236, 228, 232, 250, 225 References Cited

U.S. PATENT DOCUMENTS

12/1982 Johnson et al. ........... .. 264/200

(List continued on next page.)

FOREIGN PATENT DOCUMENTS

WO98/19412 5/1998


NETWDRK

@2103

SEQUENCERS ' 2m CONFM;

RXSEQ mid? 2105 2104

OTHER PUBLICATIONS

Internet pages entitled: DART Fast Application—Level Net Working Via Data—Copy Avoidance, by Robert J. Walsh, printed Jun. 3, 1999.


Primary Examiner—Zarni Maung (74) Attorney, Agent, or Firm—T. Lester Wallace; Mark Lauer

(57) ABSTRACT

An intelligent network interface card (INIC) or communi cation processing device (CPD) Works With a host computer for data communication. The device provides a fast-path that avoids protocol processing for most messages, greatly accel erating data transfer and offloading time-intensive process ing tasks from the host CPU. The host retains a fallback processing capability for messages that do not ?t fast-path criteria, With the device providing assistance such as vali dation even for sloW-path messages, and messages being selected for either fast-path or sloW-path processing. A context for a connection is de?ned that alloWs the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message processing by the host. The device contains specialized hardWare circuits that are much faster at their speci?c tasks than a general purpose CPU. A preferred embodiment includes a trio of pipelined processors devoted to transmit, receive and utility processing, providing full duplex communication for four Fast Ethernet nodes. The device provides a fast-path that avoids protocol processing for most messages, greatly accelerating data transfer and offloading time-intensive processing tasks from the host CPU. The host retains a fallback processing capability for messages that do not ?t fast-path criteria, With the device providing assistance such as validation even for sloW-path messages, and messages being selected for either fast-path or sloW-path processing. A context for a connection is de?ned that alloWs the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message process ing by the host.

20 Claims, 21 Drawing Sheets

PROCESSOR 470

HOSTZU \ ***** reefer“

DRAM

2223 STATUS

1 Asa

BUFFER 1 2114

US 6,427,173 B1

US. PATENT DOCUMENTS WO PCT/US98/14729 1/1999 WO WO99/04343 1/1999

5,163,131 A 11/1992 ROW et al. ................ .. 395/200 W0 WO 99/65219 6/1999 5,212,778 A 5/1993 Dally et al. . 395/400 W0 W0 O1/O477O A2 7/2000 5,280,477 A 1/1994 Trapp 370/85.1 W0 WO 01/05107 A1 7/2000 5,289,580 A 2/1994 Latif et al. ...... .. 395/275 W0 W0 O1/O5116 A2 7/2000 5,303,344 A 4/1994 Yokoyama et al. ....... .. 395/200 W0 WO /01/05123 A1 7/2000 5,412,782 A 5/1995 Hausman et al. ......... .. 395/250 W0 WO 01/40960 A1 6/2001 5,448,566 A 9/1995 Richter et al. .... .. 370/94.1

5,485,579 A 1/1996 HitZ et al. .. .. 395/200.12 5,506,966 A 4/1996 Ban ......................... .. 395/250 OTHER PUBLICATIONS

5,511,169 A 4/1996 Suda ........................ .. 395/280

5,548,730 A 8/1996 Young et al. .. 395/280 Internet pages of InterProphet entitled: Frequently Asked 5,566,170 A 10/1996 Bakke et al. .. ....... .. 370/60 Questions, by Lynne Jolitz, printed Jun. 14, 1999. 5,588,121 A 12/1996 Reddin et a1. . .. 395/200.15 _ _ _ _

5,590,328 A 12/1996 Seno et a1_ _____ __ 395/675 Internet pages entitled: Technical White Paper—Xpolnt’s 5,592,622 A 1/1997 lsfeld et a1. __ 395/200,02 Disk—to—LAN Acceleration Solution for Windows NT 5,629,933 A 5/1997 Delp et al. ................ .. 370/411 Server, printed Jun. 5, 1997. 5,634,099 A 5/1997 Andrews et al. ..... .. 395/200.07 _ _ 5,634,127 A 5/1997 Cloud et a1_ _____ __ 395/680 Jato Technologies Internet pages entitled: Network Accel 5,642,482 A 6/1997 Pardillos _________ __ 395/2002 erator Chip Architecture, twelve—slide presentation, printed 5,664,114 A 9/1997 Krech, Jr. et al. .... .. 395/200.64 Aug. 19, 1998. 5,671,355 A 9/1997 Collins .................. .. 395/200.2 _ _ _ _

5,678,060 A 10/1997 yokoyama et a1_ _____ __ 709/212 EETIMES article entitled: Enterprise System Uses Flexible 5,692,130 A 11/1997 Shobu et a1. ........ .. 395/200.12 Spec, by Christopher Harrer and Pauline Shulman, dated 5,699,317 A 12/1997 Sartore et al. ....... .. 395/230.06 Aug. 10, 1998, Issue 1020, printed Nov. 25, 1998. 5,701,434 A 12/1997 Nakagawa ..... .. 395/484 _ _ _

5,701,516 A 12/1997 Cheng et a1_ __ 395/842 Internet pages entitled: lReady About Us and lReady Prod 5,749,095 A 5/1998 Hagersten .. 711/141 ucts, printed Nov' 25, 1998. 5751715 A 5 1998 Ch t l. . 370 455 . 5:752j078 A 41998 D551; :1_ ________________ __ 3952327 Internet ‘pages entitled: Srnart Ethernet Network Interface 5,758,084 A 5/1998 Silverstein et al. 395/200.58 Car, WhICh Berend OZCGH 1S developlng, Pnnted NOV- 25, 5,758,089 A 5/1998 Gentry et al. ........ .. 395/200.64 1998 5 758186 A 5 1998 H ~lt t l. ..... .. 395 831 . . 5’758’194 A 41998 Kirznrilaoflij _____________ __ 3952386 Internet pages entitled: Hardware Asslsted Protocol Process 5,771,349 A 6/1998 PicaZo, Jr. et a1. .... .. 395/188.01 ing, Which Eugene Feinberg is Working On, Printed NOV- 25, 5,790,804 A 8/1998 Osborne ......... .. .. 395/200.75 1998

5 794 061 A 8 1998 H t l .. 395 800.01 . . . . 5:802j580 A 41998 M25652; 7/11/149 Internet pages of XaQtl Corporation entitled: Glga POWER 5,809,328 A 9/1998 Nogales et a1_ _____ __ 395/825 Protocol Processor Product Preview, printed Nov. 25, 1998.

5,812,775 A 9/1998 Van Seters et al. 395/200.43 Internet a es of X Oint Technolo ies X Oint Com 5,815,464 A 9/1998 Purcell et al. ............ .. 395/163 b . psg p. d g ”' ”' "' p ' 5,878,228 A 3/1999 Bilansky et a1. ..... .. 395/200.57 We S1te( Pages) Pnnte Dec- 19,1997

5913928 A 6/1999 Wang et a1‘ """"" " 395/20033 Internet pages relating to iReady Corporation and the iReady 5,930,830 A 7/1999 Mendelson et al. ....... .. 711/171 I t tT M d 1 . t d N 2 1998 5,931,918 A 8/1999 Row et a1. ................ .. 709/300 n eme uner O u 6’ PH“ 6 0V‘ > '

2 gurayamatetlal ~~~~~~ Internet pages entitled: Asante and 100BASE—T Fast Eth , , onnery e a . ....... .. . -

5,941,969 A 8/1999 Rarn et a1. ................ .. 710/128 ernet’ pnnted May 27’ 1997'

5,941,972 A 8/1999 H9659 et a1~ ~~~~~~~~~~~~~~ ~~ 710/129 Internet pages entitled: A Guide to the Paragon XP/S—A7 5,950,203 A 9/1999 Stakuls et al' "" " 707/10 Supercornputer at Indiana University, printed Dec. 21, 1998. 5,991,599 A 11/1999 Radogna et al. .. 370/392 6,005,849 A 12/ 1999 Roach et al- - 370/276 Andrew S. Tanenbaurn, Computer Networks, Third Edition, 6,009,478 A 12/1999 Panner et al. .. ..... .. 710/5 ISBN ()_13_349945_6 (1996) 6,016,513 A 1/2000 Lowe ...... .. . 709/250

6,026,452 A 2/2000 Pitts .......................... .. 710/56 Richard Stevens, “TCP/IP Illustrated, vol. 1, The Protocols”, 6,034,963 A 3/2000 Minami et a1. ........... .. 370/401 pp. 325—326 (1994). 6,044,438 A 3/2000 Olnowich ....... .. 711/130 _ _

6,047,356 A 4/2000 Anderson et a1 711 /129 VT8501 Apollo MVP4 Docurnentatlon, VIA Technologies, 6,057,863 A 5/2000 Olarig ..... .. . 345/520 Inc., pp. i—iv, 1—11, cover and copyright page, revision 1.3 6,061,368 A 5/2000 HitZelberger 370/537 (Feb. 1, 2000). 6,065,096 A 5/2000 Day et a1. .... .. 711/114 _ _ _

6,141,705 A 10/2000 Anand et a1_ __ ____ __ 710/15 Internet pages entitled: Northrldge/Southrldge vs. Intel Hub

6,226,680 B1 * 5/2001 Boucher et al. 709/230 Architecture, 4 pages, printed Feb. 10, 2001. 6,246,683 B1 6/2001 Connery et a1. 370/392 _ _ _ _ _ _

6,247,060 B1 6/2001 Boucher et a1. .......... .. 709/238 Glgablt Ethernet Technlcal Bnef, Achlevlng End—I0—EI1d 6,334,153 B2 * 12/2001 Boucher et a1. .......... .. 709/230 Performance~ Alteon Networks, Inc, First Edition, 5911

FOREIGN PATENT DOCUMENTS

WO 00/13091 WO98/50852

11/1998 11/1998

1996.

Internet pages directed to; Technical Brief on Alteon Eth ernet Gigabit NIC technology, www.alteon.corn, 14 pages, printed Mar. 15, 1997.

US 6,427,173 B1

Internet web pages from “Adaptec.corn” website directed to the Adaptec, AEA—7110C iSCSI Host Bus Adapter and about Adaptec’s IP storage activities, 11 pages, downloaded and printed Oct. 1, 2001. Internet web pages from “iSCSIhba.corn” website directed to JNI iSCSI HBAs including the “FCE—3210/6410”, 10 pages, downloaded and printed Oct. 1, 2001. Internet web pages from the “ISCSI Storage.corn” website that mention an ErnuleX HBA, 2 pages, downloaded and printed Oct. 1, 2001. Internet web pages from the “iSCSIhba.corn” website that mention QLogic HBAs including the “SANblade 2300 Series”, 8 pages, downloaded and printed Oct. 1, 2001. “Two Way TCP Traf?c over Rate Controlled Channels: Effects and Analysis”, IEEE Transactions on Networking, vol. 6, No. 6, pp. 729—743 (Dec. 1998). Internet pages from IReady News Archives entitled “iReady Rounding Out management team with two key executives,” 2 pages, (printed Nov. 28, 1998). Toshiba, “Toshiba Delivers First Chips to Make Consurner Devices Internet—Ready Based on iReady’s Design”, 3 pages, Press release Oct. 14, 1998, downloaded Nov. 28, 1998. Internet pages entitled “iReady Products” from website http://www.ireadyco.corn/products.htrnl, 2 pages, printed Nov. 25, 1998. Iready News Archives. Toshiba, iReady shipping Internet Chip, 1 page printed Nov. 28, 1998. Internet site www.interprophet.corn, 17 pages, printed Mar. 1, 2000. The I—1000 Internet Tuner Features, iReady Corporation, 2 pages, date unknown. Internet pages from website http://www.ireadyco.corn/ about.htrnl, 3 pages, downloaded Nov. 25, 1998. IReady News Archives, “Revolutionary Approach to Con surner Electronics Internet Connectivity Funded”, San Jose, CA. 2 pages, Nov. 20, 1997, downloaded and printed Nov. 2, 1998.

IReady News Archives, “Seiko Instruments Inc. (SII) Intro duces World’s First Internet—Ready Intelligent LCD Mod ules Based on IReady Technology”, 2 pages Oct. 26, 1998, downloaded and printed Nov. 2, 1998. NEWSwatch—IReady Internet Tuner to Web Enable Devices, Nov. 5, 1996. 2 pages, printed Nov. 2, 1998. David Larnrners, EETirnes, Jun. 13, 1997, “Tuner for Toshiba, Toshiba taps iReady for Internet tuner,” 2 pages, printed Nov. 2, 1998. Internet pages entitled: Comparison of Novell Netware and TCP/IP Protocol Architectures, by Janique S. Carbone, 19 pages, Jul. 16, 1995, downloaded and printed Apr. 10, 1998. 60/053,240 (US. Provisional Application), by JolitZ et al. (listed ?ling date Jul. 18, 1997). Zilog Product Brief entitled “Z85C30 CMOS SCC Serial Cornrnunication Controller”, Zilog Inc., 3 pages (1997). US. application No. 08/964,304, by Napolitano, et al., entitled “File Array Storage Architecture”, ?led Nov. 4, 1997.

Article by D. HitZ, et al., “File System Design For An NFS File Server Appliance”, 13 pages, 1996. Adaptec Press Release, “Adaptec Announces EtherStorage Technology”, 2 pages, May 4, 2000, printed Jun. 14, 2000. Adaptec article, “EtherStorage Frequently Asked Ques tions,” 5 pages, printed Jul. 19, 2000. Adaptec article, “EtherStorage White Paper,” 7 pages, printed Jul. 19, 2000. CIBC World Markets article by J. Berlino et al., “Cornput ers; Storage”, 9 pages, dated Aug. 7, 2000. Merrill Lynch article by S. Milunovich, “Storage Futures”, 22 pages, dated May 10, 2000. Internet—draft of J. Satran, et al., “SCSI/TCP (SCSI over TCP)”, 38 pages, dated Feb. 2000. Article by S. Taylor, “Montreal Start—Up Battles Data Stor age Botttleneck,” 2 pages, dated Mar. 5, 2000.

* cited by eXarniner

U.S. Patent Jul. 30, 2002 Sheet 1 0f 21 US 6,427,173 B1

CPU

STORAGE

REMOTE HOST

FIG. 1

5 3

@K 2) MA 0 w

8 \

6 4

4 4

K REM Kvm RYc KN mAAmom WWMMSWA UmEmNTT K rm// w 4 8 %

M 3

A

K 50

CONTEXT

INIC/ CPD

52

FIG. 2


RECEIVE PACKET FROM NETWORK \_/\47

BY CPD

V

VALIDATE PACKET, SUMMARIZE \_/\57 HEADERS

61

FAST PATH SEND PACKET TO CANDIDATE” STACK FOR SLOW

' PATH PROCESSING

65 w

SEND PACKET TO MATCH WITH STACK FOR SLOW

CCB? PATH PROCESSING

YES 69 K v

SEND TO DESTINATION CREATE CCB FOR IN HOST VIA MESSAGE FAST-PATH

1 51

FIG. 3


REMOTE HOST

1 52" \\l _ - - - _ _ _ _ _ _ _ _ _ _ _ 1

FA T-P ' S ATH l > SOURCE/DEST I 159 168 JV " i

' APPLICATION ' K 166 "T. I

\ | |

150 \L170\ 1% 164% TRANSPORT : |_ W _ _ _ _ _ _ _ _ _ _ _ _ _ ' ' _| I |

' I 162 I l | \K : PROCESSOR I l NETWORK I I S 1 160 l 1 { HARDWARE LOGIC } SLOW-PATH I DATA LINK : ' | ' \ I

U.S. Patent Sheet 5 0f 21 Jul. 30, 2002

MEDIA ACCESS

US 6,427,173 B1

172 CONTROLLER N

ASSEMBLY N 174 178 176 REGISTER S S

PACKET —~—> FLY BY

SEQUENCER ' CONTROL v { SEQUENCER

MULTIPLEXOR N 180 +

3182 S183 SRAM

SRAM ' CONTROL ‘

A

v

DRAM CONTROL ¢

& 186 #

DRAM 188 QUEUE N 184'“ MANAGER

FIG. 7

U.S. Patent

178

Jul. 30, 2002 Sheet 6 0f 21

PACKET

SEQUENCER

MAC N191

SEQUENCER

NETWORK N 192

SEQUENCER

TRANSPORT N 194

SEQUENCER

SESSION N 195

SEQUENCER

180

S

US 6,427,173 B1

174

8 CONTROL ———>

ASSEMBLY REGISTER

MULTIPLEXOR

FIG. 8


202»\_2l I l 1 300 318 i I \y /\/ UPPER LAYER <—{—-—— ' | ' | l

UPPER LAYER INTERFACE I 330% I ' |

325% TRANSPORT TRANSPORT my 316 ' | l

322 N‘, NETWORK NETWORK wig 314 ' l |

320/\:/ DATA LINK DATA LINK ’\L/ 312

306 ’\/ INIC MINIPORT DRIVER

I 240 ’\/

4T INIC 4 210 I

INIC MEMORY

FIG. 10


_

_ 6

_ D. 1U &D.

C TV A C

U“ M mR 4 _ 2 _ 8 4 n 6 m 2%

n w 2

_ J j m“ m w _ C 4 &C4 _ C T . UAYA‘LVMW “ M X R

_ m

\J M m _ 4

m _J 2 J _ B M &

B

_ O A V! T _ Au.‘ A MW _ M XR

210 400\\ 240 ____________L__

M AC-A

1/410 XMT& 418 RCV-A

EXTERNAL MEMORY CTRL

|||||J

/480

I 47 REG FILE WCS ROM

PCI BUS INTERFACE UNIT ||||||||||||||||||||||||||||||||||||||||||||||||||||||||L FIG. 13


CLOCK /

I f f 1 l l l

: 49L CONTROLS FOR FIRST REGISTER SET

/ /| 500 ’\ 490

1

: V \ V V V V

1%» FIRST REGISTER SET I RAM FILE L REGISTER / >- — —————————————————————————————— — — — — — — — —

l 498 I x v v v v v } INSTRUCTION DECODER } AND

,/ OPERAND MULTIPLEXER 560\ 496

l V X V V V V V |

H—:> SECOND REGISTER SET I l \ \ L_ ____________________________________________ __

// E V V V V , STACK : EXCHANGE ARITHMETIC LOGIC UNIT QUEUE

' K J J 608 60% 602 503

\l v v v v v 0

TI» THIRD REGISTER SET |

'\ 501 / 470 L‘; ____ “<1 ____ “Kn-“bx ____ “Y ____ ""

U.S. Patent

U2 .GE

US 6,427,173 B1 n. M m

32m

O3 wow 0% 26 N8

m % \ \ \ \ a \ \ x

m. :55

, div @EEQVE v52

0 / - IV V

m 9:5 M25 % 55 a?

m 236

|||||||||||||||||||||| Ell lilll I} {I ll -llll ,

/

/


FIG. 16

745

J 768 7% /

772 7 /

764

750

755

K790 / 792

/ 796 7

/ 788

760

FIG. 17


Proc DZQ QZD XMT RCV 806 802 R Seq Seq Seq Seq 844 Wrlte

eq Req Req Req Req Data

808 8 10 804 /815 846

ARBITER \ MUX

v K REGISTER A

I v v Addr DIn Addr 825 Out In

QRAM DOut

820 V V

REGISTER B

828 v '

QALU

822\ ' 821

r___|____________._ _~__________J_1 1 v v v v | l v v 1! Ir I v

| Q Q Q Q l I Sram Sram Bofiy Body I srafn Empty Fun In Out I | Req Addr Wnte Read Wnte

: RD Y I | Req Req | Data

FIG. 18


MRU

6

MRU

R13 R14 R15 13 4

R13 R14 R15

R8 R9 12 10

R8 R9 12

R7 2

FIG. 19A

R7

R2 7

R2

' FIG. 19B

R1 1

R1 0

LRU

R0 9

LRU

R 0 0\

O 0

9

MRU

R13 R14 R15 6

LRU

1O 9 R8 R7 R2 R1 0

FIG. 19C

MRU

R13 R14 R15 10 12

R7 R2 R1

LRU 0 0

FIG. 19D

intelligent network interfaced device and system for accelerated

Documents