au-5_bae_protocol analysis in a complex enterprise

8/15/2019 AU-5_Bae_Protocol Analysis in a Complex Enterprise

1/15

SHARKFEST '09 | Stanford University | June 15–1! "009

#roto$o% Ana%ysis in a &o(%e)Enter(rise* T+e ,(ortan$e of -T+e Art ofRe$o.nition/

June 16th, 2009

Hansan. aeSenior VP| Citi (f.k.a Citigroup)hbae@n!.rr.!o"

SHARK #$S% '09Stanfor& 'nierit June 1*+1, 2009


2/15


Cha--enge

/ it turn out, ie &oe "atter

Citi bran!h net3ork pan *,0004 -o!ation inthe 'S

Citi net3ork infratru!ture in!-u&e 50,0004&ei!e

500,000 uer -o!ate& in oer 100 !ountrie.

u"ber of erer in ue i "in& nu"bing--arge

Co"p-ian!e7Se!urit 8uag"ire

oing a fu-- pa!ket !apture i &i:!u-t.

%oo- in ue in!-u&e etVC; an& t &i-o& e& at "arket


3/15


4/15


5/15


6/15


/!t >V =in&o3 %a-e

Ca-- !enter erer are not ab-e to keep up 3ith !a-- o-u"e after a&ata !enter "igration

%he erer are not getting the &ata fat enough + !auing aba!k-og. =hat i"p-e !hange !an in!reae the throughputB

%he path after the "igration i -onger b *0 ".

Deon Dearne& >f -aten! i !auing a prob-e", -ook for ;#C1525 re-ate&

prob-e".

Fno3 3hat aEe!t a tranfer throughput. KuEer tearing,3in&o3 ie, or pa!ket -o.

'e the graphi!a- p-ot to oo" in on the prob-e" T o -et -ookat the 3in&o3 ie. Shou-& 3e -ook at the re!eie or en&3in&o3B

/rgue our !ae. >f oure right, oure right Kut ou ha&better be right. Iou earn our G!re&H oer ti"e, but ou !anb-o3 it in one hot

'e the graphi!a- too- aai-ab-e in =irehark. Pi!ture R>SR

?8S-o3.p!ap ?8S-o3Print.t@t


7/15SHARKFEST '09 | Stanford University | June 15–1! "009

/!t >V =in&o3 %a-e

Use STAT,ST,&S! ,2 3RA#H to 4rin. u( t+is .ra(+/ odify t+e +i.+%i.+ted itesto 4rin. u( t+is vie6



/!t V / 'er Co"p-aint

S"ith Karne #inan!ia- Conu-tant are !o"p-aining of -o3page -oa& ti"e for their ho"e page. %he prob-e" ipora&i! an& ran&o" but happen enough that it i"pa!tingtheir pro&u!tiit.

%he prob-e" i 3i&e+prea&, not eai-

repro&u!ib-e.3here &o ou tartB =hat &o ou &oBG=ho ou gonna !a--BH

=hat !o""on in the prob-e"B Lo"e pageN ue of -oa&ba-an!erN !o""on ba!ken& ererN aEe!ting "anuer.

=hat the ob of a -oa& ba-an!erB

=here hou-& 3e take the tra!eB

=hat Gba& thingH !an happen if ou are uing a -oa&ba-an!er 3ith Sour!e /% !onAgure&B



/!t V / 'er Co"p-aint (!ont)



/!t V / 'er Co"p-aint (!ont)DKProb-e"6e3.p!ap




Deon Dearne& Start b -ooking at 3hat infratru!ture i in !o""on for

a-- uer eperien!ing the prob-e".

=hat !ontitute a %CP pa!ketB 2+%up-eB U+%up-eB

;e"e"ber that eMuen!e nu"ber are nothing "orethan the nu"ber of bte tranferre&. /!kno3-e&ge"enti nothing "ore than an in&i!ation of ho3 "u!h of the&ata ou re!eie&. Iou re!eie o"ething outi&e of3hat epe!te&, o"ething 3ent horrib- 3rong

=hen ou hae a 22,000 uer bae, haing a ephe"era-port range of 102U+*000 !an be ehaute& Mui!k-.

So"eti"e, ou hae to reort to turning oE Gre-atieeMuen!e nu"berH for ana-i. %hi i epe!ia-- true3hen -oa& ba-an!er T or an &ei!e that /% T i in the&ata path.




Deon Dearne& (!ont) (%urn oE re-atie eMuen!enu"ber)

Frames 1-8 contain the orderly close o a connections

Frame " which occurs appro# 1$ seconds later is an attempt o a

%new& client to open a connection to the L' (Frame 1) is the L'

translated re*uest to the web ser+er,

Frame 11 is an acnowledgement or the prior connection .his

occurs/ because the 0eb ser+er still has this socet in F-0.

(Frame 12 is the translated re*uest 4 L' to client,

Frames 13 and 1$ is the R5. generated by the client/ and the

translated re*uest/ respecti+ely

Frames 16-18 contain a connection creation .his is allowed to

occur because o the R5. 7owe+er/ this causes the client to

pause or appro# 3 5econds

DK%CPLan&)hake.p!ap



/!t V> / Iou Dog >t

/fter a &ata !enter "igration, an app-i!ation 3a no -ongerab-e to upport the pro&u!tion tra:!. %he ne3 &ata !enter3a eparate& b 11" roun& trip -aten!. Kefore the "oe,both erer 3ere -o!ate& in the a"e C

atura--, Art in!-ination 3a to b-a"e the net3ork

/fter a--, the prob-e" tarte& after the "igration. %he app-i!ation generate a 5 bte Ga-ertH "eage

fo--o3e& b another "a-- pa!ket 3ith the a!tua- &ata.

=hat hou-& be the Art prob-e" that !o"e to our"in&B

=hat -ooke& -ike a -a"+&unk turne& out be Muite!o"p-i!ate&

>n the /r", 3e ha& a aing Ke, Fno3, o. >t app-ie topa!ket ana-i.

/t the en& of the &a, in &epth kno3-e&ge of ho3 %CPhou-& 3ork a--o3e& u to An& the prob-e".

C?o(eJKothSi&eDook/t91,.p!ap

C?o(eJ


14/15


/!t V> / Iou Dog >t (!ont)

Deon Dearne&ag-e an& e-ae& /!kno3-e&g"ent &ea&-o!k i er

!o""on 3hen %CP i ue& to hutt-e "a-- a"ount of&ata.

%hi !an be a Gki--erH 3hen tra&ing progra" areino-e&.

%urning on app-i!ation -ee- -ogging !an he-p, but &ontforget to turn it oE

Fno3 3hat i"pa!t ou !an hae if ou &e!i&e to -og. #oru router+o!ke, it eMuia-ent to &oing a G&ebug ipopfH on a pro&u!tion ba!kbone router. Lint not a goo&i&ea. >t a e-f !orre!ting error T if ou &o it on!e, ou--neer &o it again

>f ou kno3 ho3 %CP rea-- 3ork, ou !an argue ourpoint 3ith !oni!tion be!aue &eep &o3n ini&e, oukno3 oure right.


15/15


/ppen&i >P ue& in theea"p-e

/C% > >C?PKLe3Rp!ap 192.16.1.1 an& 192.16.1.2*U are erer on the a"e 3it!h.

C. 5low557Loging2pcap

192.16.1.1 i the !-ient. 12.16.*0.*0 i the h erer. 192.16.*.* an&192.16.200.200 are >S4 erer.

C. 5lowFtpnonpcap

1)1)1)1) is the tp ser+er 1"219811 client is pulling the ile rom the ser+er

C. : ; o the L' and 1=2192)2) is the > used by the L' or source .&ing when talingto the real web ser+er

C. : ?C;o+e@Apcap

1"219811)2 and 1=2191126 are two ser+ers in+ol+ed in the transer 'oth send data

independently o one another

#%ease eai% e at +4ae7ny$/rr/$o if you 6ou%d %i8e t+e-T+e Too% isio a$ro/
mailto:[email protected]:[email protected]

au-5_bae_protocol analysis in a complex enterprise

Documents