ban tuning and benchmarking eap session clustering
TRANSCRIPT
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
1/43
Benchmarking and tuning JBoss
EAP session clustering
Bela BanJBoss06.29.2012
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
2/43
What's this talk about ?
Performance of a clustered webapp on EAP 6 Default configuration --> performance
Tuned configuration --> performance
Along the way
How to configure an EAP 6 cluster
et some tuning ad!ice" #est practices
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
3/43
Architecture of an EAP 6 cluster
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
4/43
Architecture of an EAP cluster
HTTP sessionclustering
HTTP sessionclustering
InfinispanInfinispan
JGroupsJGroups
HTTP sessionclustering
HTTP sessionclustering
InfinispanInfinispan
JGroupsJGroups
Replication
JBoss instance JBoss instance
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
5/43
Let's take a look at the clustering
modes replicationand
distribution
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
6/43
!ode"replication
All the data is stored on all cluster nodes
$rites are sent to all nodes
E!ery node updates its local cache
%eads are always local
&ew nodes ac'uire the initial state from the oldest node
(lients can access any node for reading or writing
)cala#ility is limited #y cluster si*e and data si*e
+, nodes with +,,B state each. e!ery node needs +B
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
7/43
K1K1
K2
K3
K
!al1
!al2
!al3
!al
K1K1
K2
K3
K
!al1
!al2
!al3
!al
K1K1
K2
K3
K
!al1
!al2
!al3
!al
#eplication mode write
Replication
"lients
#o$%cluster
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
8/43
#eplication mode read
K1K1
K2
K3
K
!al1
!al2
!al3
!al
K1K1
K2
K3
K
!al1
!al2
!al3
!al
K1K1
K2
K3
K
!al1
!al2
!al3
!al
Replication
"lients
#o$%cluster
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
9/43
!ode"distribution
Data is only stored on & cluster nodes /say &012 A consistent hashon a 3ey 4id5 determines the 1
ser!ers for 4id5
Eample. cluster is 7A"B"("D"E"89
Hash/4id52 0 :; : Primary owner 0 B" #ac3up owner 0 (
(rash of B" new !iew is 7A"("D"E"89
--> Primary owner 0 D" #ac3up owner 0 E
--> ( needs to transfer 4id5 to D and E and remo!e it locally
=nowing the 3ey" we always find the right ser!er/s2
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
10/43
!ode"distribution
A write updates the & owners of a 3ey A read as3s the primary owner
Possi#ly stores the result in its + cache
+ caches re'uire in!alidation on updates
Distri#ution uses less space than replication
?f all owner nodes of a gi!en 3ey crash at the sametime" the data is lost
The cost of shipping the modifications is constant ood for large clusters
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
11/43
$istribution mode write
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K1
K
!al1
!al
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K1
K2
K
K&
!al1
!al2
!al
!al&
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K2
K3
K&
!al2
!al3
!al&
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K3 !al3
Replication
"lients
#o$%cluster
'rite(K2)
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
12/43
$istribution mode read
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K1
K
!al1
!al
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K1
K2
K
K&
!al1
!al2
!al
!al&
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K2
K3
K&
!al2
!al3
!al&
K1
K2
K3
K
K&
!al1
!al2
!al3
!al
!al&
K3 !al3
Replication
"lients
#o$%cluster
rea$(K&)
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
13/43
!ode"distribution and session clustering
JBoss always stores a session on the current node
%e'uires session stic3iness" or else we ha!e a lot ofremote reads /ena#le + in this case2 @
To do so" it gra#s a 3ey whose consistent hash mapsto the current node
That 3ey acts as the ession?d
Ad!antages
&o remote reads
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
14/43
%&nc ersus as&nc
)ync #loc3s the caller /)essionsetAttri#ute/22 until the
changes ha!e #een applied (onfirmation using ac3s
Async ships the changes in the #ac3ground" the callerreturns immediately
Async is faster than sync
%ecommendation. use stic3y sessions
)ync always in!ol!es networ3 round trips
C com#inations. repl-sync" repl-async" dist-sync" dist-async
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
15/43
(onfiguration of an EAP cluster
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
16/43
httpd ) mod*cluster +httpd,conf-
*a+Re,uestsPer"-il$ 0er/eri#it 20
T-rea$sPer"-il$ &0*a+"lients 1000
If*o$ule #anager#o$ule isten 4000 !irtualHost 54000 ocation 7
8r$er $en:allo' ;llo' fro# all 7ocation
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
17/43
EAP configuration +standalone*ha,.ml-
Address of httpd mod-cluster.
su=sste# +#lns>?urn@=oss$o#ain#o$cluster1.1? #o$%cluster%config pro+%list>?cluster09:8000? a$/ertise>?false?77su=sste#
su=sste# +#lns>?urn@=oss$o#ain#o$cluster1.1? #o$%cluster%config pro+%list>?cluster09:8000? a$/ertise>?false?77su=sste#
(ache configuration.
su=sste# +#lns>?urn@=oss$o#aininfinispan1.2? cac-e%container na#e>?web? $efault%cac-e>?repl-sync? transport stacA>?udp?7 replicate$%cac-e na#e>?repl-async? #o$e>?;C"?7 replicate$%cac-e na#e>?repl-sync? #o$e>?C"?7 $istri=ute$%cac-e na#e>?dist-async? o'ners>?2? #o$e>?;C"?7 $istri=ute$%cac-e na#e>?dist-sync? o'ners>?2? #o$e>?C"?7 7cac-e%container7su=sste#
su=sste# +#lns>?urn@=oss$o#aininfinispan1.2? cac-e%container na#e>?web? $efault%cac-e>?repl-sync? transport stacA>?udp?7 replicate$%cac-e na#e>?repl-async? #o$e>?;C"?7 replicate$%cac-e na#e>?repl-sync? #o$e>?C"?7 $istri=ute$%cac-e na#e>?dist-async? o'ners>?2? #o$e>?;C"?7 $istri=ute$%cac-e na#e>?dist-sync? o'ners>?2? #o$e>?C"?7
7cac-e%container7su=sste#
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
18/43
EAP configuration +standalone*ha,.ml-
Transport.
su=sste# +#lns>?urn@=oss$o#[email protected]? $efault%stacA>?u$p? stacA na#e>?udp? transport tpe>?DEP? socAet%=in$ing>?@groups%u$p?7 protocol tpe>?PICG?7 protocol tpe>?FE8"K? socAet%=in$ing>?@groups%u$p%f$?7 protocol tpe>?p=cast.C;K;"K?7
protocol tpe>?DCI";T2?7 protocol tpe>?p=cast.T;B
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
19/43
Webapp configuration +web,war-
$EB-?&8we#ml.
...$istri=uta=le7...
...$istri=uta=le7...
$EB-?&8#oss-we#ml
replication%config cac-e%na#eweb.repl-async7cac-e%na#e7replication%config
replication%config cac-e%na#eweb.repl-async7cac-e%na#e7replication%config
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
20/43
Performance numbers
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
21/43
/est enironment
HP D6, F dual soc3et" C core EG6C, &ehalem"166 H*" 1B %A
+B networ3
J. +6,I+
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
22/43
Perf test setup
-ttp$#o$%cluster
-ttp$#o$%cluster @=oss%01
@=oss%01 @=oss%02@=oss%02 @=oss%03@=oss%03@=oss%01@=oss%01
Perf testclient
Perf testclient
@=oss%0@=oss%0
@=oss%01@=oss%01 @=oss%06@=oss%06 @=oss%0@=oss%0@=oss%01@=oss%01@=oss%0&@=oss%0& @=oss%04@=oss%04
10.1.+.+
192.164.+.+
10.1.+.+
"luster
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
23/43
/est drier
Each LclientL is a thread in the /same2 test dri!er C,, clients eecute +,,L,,, HTTP re'uests
M,N of the re'uests are reads" +,N are writes
Each client does the following /num-re's0+,,,,,2.
% "reate a session (10 attri=utes: eac- -as a 1K =te =uffer)% '-ile nu#%re,s 0 % 'it- a 90J c-ance rea$ a ran$o# attri=ute (1%10): or: % 'it- a 10J c-ance 'rite a ran$o# attri=ute % $ecre#ent nu#%re,s% Eestro t-e session
% "reate a session (10 attri=utes: eac- -as a 1K =te =uffer)% '-ile nu#%re,s 0 % 'it- a 90J c-ance rea$ a ran$o# attri=ute (1%10): or: % 'it- a 10J c-ance 'rite a ran$o# attri=ute % $ecre#ent nu#%re,s% Eestro t-e session
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
24/43
Base line
24&00
29000
29&00
30000
30&00
31000
31&00
32000
32&00
Con%replicate$
tatic in$e+.-t#l
tatic 'e=7static.@sp
ocal (not replicate$)
Co$es
Re,uests7sec
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
25/43
(omments
(lustering adds some o!erhead" so we wonLt gethigher than these num#ers
(a ,L,,, re'uests sec is what apache httpd canhandle for a single test dri!er /C,, threads2
$e could run multiple httpds and multiple test dri!ersto increase total throughput" #ut thatLs not what wewant to measure we want to measure the difference#etween local and clustered sessions
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
26/43
0$P
0
&000
10000
1&000
20000
2&000
30000
3&000
DEP non%opti#ie$
Con%replicate$
repl%sncrepl%asnc
$ist%snc
$ist%asnc
Co$es
Re,uests7sec
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
27/43
/(P ersus 0$P
0
&000
10000
1&000
20000
2&000
30000
3&000
T"P non%opti#ie$
Con%replicate$
repl%sncrepl%asnc
$ist%snc
$ist%asnc
Co$es
Re,uests7se
c
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
28/43
(omments
$ith increasing cluster si*e" D?)T outperforms %EP
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
29/43
(omments
$ith mode0%EP" T(P is faster than KDP initially" #utperf degrades with increasing cluster si*e
?n a +6 node cluster" a LmulticastL in!ol!es sending thesame message to +G nodes
T(P gets slower than KDP at :-+1 nodes $ith D?)T" at : nodes" T(P and KDP perform the
same
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
30/43
-
8/12/2019 Ban Tuning and Benchmarking Eap Session Clustering
31/43
Effect of changing the replication granularit&
0
&000
10000
1&000
20000
2&000
30000
3&000
replication%granularit>;TTRIBDT