Download - Scaling Saved Searches at eBay Kleinanzeigen
Scaling Saved SearchesServing real time push-notifications for millions
saved searches
466382733
André Charton, Christiane Lemke
Who we are?
ebay
kleinanzeigen
≠
ebay
local platforms
some numbers
19M ads
12M searches/day
0.6M new ads/day
6M saved searches
36.000.000.000
theoretical
matches
p r o c e s s
i t !
make it run,
make it fast,
make it stable,
make it better.Special thx fly to Ramkumar Aiyenger – how see some project in the same way.
hat?
ow?
* * 0/6 * * ?
r e a l t i m e ?
s c a l a b l e ?
C a n
w e
d o
b e t t e r ?
2 0 1 5
src=https://www.esciencecenter.nl/img/main/logo-elastic.png
Percolator
Traditionally you design documents based on your data, store
them into an index, and then define queries via the search
API in order to retrieve these documents. The percolator
works in the opposite direction. First you store queries into an
index and then, via the percolate API, you define documents
in order to retrieve these queries.
src=https://www.elastic.co/guide/en/elasticsearch/reference/current/search-percolate.html
H o w
m a n y
p u s h s
p e r d a y ?
C r e a t e
d e l a y s
f r o m
s i z e
0 - 100: RT
101 - 1000: 1h
1001 - 10000: 2h
> 10000: 6h
a s k
s o l r
s k i p
o n
o v e r l o a d
S e t u p
S e t u p
cloud
cloud
2 data centers
cloud
2 data centers
10 + 3 machines
cloud
2 data centers
10 + 3 machines
replication x1
shards x80
...
l i f e t i m e
o f
a
s e a r c h
s l e e p ...
Z Z Z Z Z Z
Z Z Z Z Z Z
e l a s t i c
f a s t
o n
i n d e x i n g
r e m o v e
s l e e p i n g
s e a r c h e s
o n l y 3 0 %
s e a r c h e s
a r e
o n l i n e
d e s k t o p
a v o i d
d b - r e a d
p e r
s e a r c h
b l o o m
f i l t e r
i n
c o o k i e
src=https://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/Bloom_filter.svg/2000px-Bloom_filter.svg.png
a p p s
d e e p l i n k
o n
r e s u l t
s i z e
s t o r e
s e a r c h e s
l o c a l
b a c k e n d
s y n c
o n
a c t i o n s
S a v e d
S e a r c h
S t a b l e ?
S t a b i l i z e
e l a s t i c
Boost your percolator!Tips & Tricks
“This indeed seems like a
large application of
percolate.”Elastic support, June 2015
Performance linear with number of queries
1.Consider using other systems.
1.Consider using other systems.
“It is worth noting that simple exist matches on a field are probably not a
great application for percolator. This doesn’t utilize any text matching
capability or complex boolean.”
Anything,
anywhere!Everything
for free!
1.Consider using other systems.
2. Optimise your data structure.
2. Optimise your data structure.
2. Optimise your data structure.
3. Filter, filter, filter!
3. Filter, filter, filter!
“The filter only works on the metadata fields.
The query field isn’t indexed by default.”
{
"query":
{...},
"category": CARS
}
{"doc" : {...},
"filters": [
{"or": {"filters":[
{"term":{
"category":CARS
}},
{"term":{
"category":ALL
}}
]}}
]
}
{
"query":
{...},
"category": ALL
}
3. Filter, filter, filter!
… what else can we filter?
3. Filter, filter, filter!
4. Consider prioritising.
Josh:
Flysurfer kite in Hamburg for less
than 300 Euro
Christiane:
Bicycle in Berlin
4. Consider prioritising.
4. Consider prioritising.
Josh:
Flysurfer kite in Hamburg for less
than 300 Euro
Christiane:
Bicycle in Berlin
Josh:
Flysurfer kite in Hamburg for less
than 300 Euro
Christiane:
Bicycle in Berlin
4. Consider prioritising.
HIGH PRIORITYLOW PRIORITY
4. Consider prioritising.
5. Use multipercolate.
5. Use multipercolate.
6. Use parallel multipercolate.
6. Use parallel multipercolate.
● Scenario: 2 shards, replication factor 1 (Shards a1, a2, b1, b2)
-> Queries sent to (a1 OR a2) AND (b1 OR b2)
● Open issue: https://github.com/elastic/elasticsearch/issues/13177
“Currently, to utilise all of your shards, you would need to consider
sending multiple mpercolate requests in parallel.”
6. Use parallel multipercolate.
Outcome
Reduced percolation time:
Outcome
Doubled the number of push notifications:
S t a b i l i z e
e l a s t i c
S t a b l e ?
S t a b i l i z e
p l a t f o r m
6 0 0 0 0 0 0
s e a r c h e s
6 0 0 0 0 0
a d s / d a y
u s i n g
k a f k a
T h a n k
y o u