1/18

Improving Load Balance via ResourceExchange in Large-Scale Search Engines

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang Liu

College of Computer ScienceNankai University

June 2020

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

2/18

Background

A simpli�ed architecture of a search engine.

Queries

Leaf Machine

Answers

...

Broker

[Index Shards]

Leaf Machine Leaf Machine

[Index Shards] [Index Shards]

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

3/18

BackgroundLoad rebalancing is NP-hard.E.g., Google machine reassignment problem.

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

4/18

Motivation�e transient resources a�ect shard reassignment!

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

5/18

MotivationWe use the exchangeable machines to facilitate load balancing.

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

6/18

Shard Reassignment Problem

ML the set of leaf machinesME the set of exchangeable ma-

chinesM ML ∪MER the set of resourcesS the set of shardscm[r] the capacity of machine m for

resource rum[r] the utilization of resource r by

machine mutl[r] the expected utilization of re-

source rds[r] the demand for resource r by

shard sX Xs,m = 1, shard s is assigned

to machine m; Xs,m = 0, oth-erwise

Vm if machine m will be returned

�e objective function:

min∑r∈R

∑m∈M

Vm |um[r]− utl[r]| .

,where the um[r] and utl[r] are de�ned asfollows:

um[r] =∑

s∈S Xs,mds[r]cm[r]

.

utl[r] =∑

s∈S ds[r]∑m∈ML

cm[r].

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

7/18

Shard Reassignment Problem

Hard constraints:1 Single principle constraint2 Capacity constraint3 Transient constraint4 Con�ict constraint5 �ota constraint

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

8/18

SRA overview

Shard Reassignment Algorithm (SRA)

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

9/18

�e core procedures of SRA in details�e �owchart:

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

10/18

Experiment1 Input se�ings (synthetic and real ) and baseline (noisy strategy

based local search (NLS), SRA without EX machines)

Table: Summary of the synthetic input se�ingsC1 C2 C3 C4 C5 C6 C7 C8

r1 range S S L L S S L Lr2 range S L S L S L S Lmachines Homogeneous Heterogeneous

the No. shards (α = 0.2) 14074 6281 6285 6121 9358 4572 4651 3882

Table: Summary of the real datasets DS1 to DS4Instance ID DS1 DS2 DS3 DS4Machine type Hetero Hetero Hetero Hetero

Number of indexes 78 64 123 60Number of shards 3072 2753 2936 3057

Number of machines 1816 1220 1987 1976Machine mean Utl. (r1) 0.50 0.74 0.66 0.42Machine mean Utl. (r2) 0.39 0.66 0.65 0.29

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

11/18

Experiment2 Load balancing on synthetic datasets

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

12/18

Experiment

3 Load balancing on real datasets

DS1 DS2 DS3 DS40

2

4

6

8

10·1032876

1286

5503

3152

1746

1174

5442

1894

1694

1122

5229

1843

GlobalIm

balance(×

103) NLS SRAnoEX SRA

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

13/18

Experiment4 Load balancing under di�erent runtimes (unit: minutes)

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

14/18

Experiment

5 Global imbalance and di�erentNo. EX. Machines onsynthetic datasets

1 2 3 4 5 6 7 8 9 100

0.5

1·103

proportion of exchangeable machines (%)

GlobalIm

balance(×

103)

C1 C2 C3 C4

C5 C6 C7 C8

6 Global imbalance and di�erentNo. EX. Machines on realdatasets

1 2 3 4 5 6 7 8 9 100

2

4

6·103

proportion of exchangeable machines (%)

GlobalIm

balance(×

103) DS1 DS2 DS3 DS4

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

15/18

Experiment

7 Global imbalance under di�erent No. leaf machines

GlobalIm

balance(×

103)

1000 2000 50000

5

10

15·103

68% 46%

23%

C1

1000 2000 5000

63% 53%

25%

C5

NLS SRAnoEX SRA

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

16/18

Experiment

8 Comparing with the lower bound

C5 C6 C7 C8

0

1

2

·102

17% 19%

28%

6%

GlobalIm

balance(×

102) Relaxation NLS SRAnoEX SRA

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

17/18

Conclusion

Use exchangeable machines and the SRA to handle the shardreassignment problem.�ere are two trade-o�s.We have not deploy the SRA on a real search engine.

Kaiyue Duan, Yusen Li, Trent Marbach, Gang Wang, Xiaoguang LiuImproving Load Balance via Resource Exchange in Large-Scale Search Engines

18/18