semantics of caching with spoca: a stateless, proportional

Context of the problemRequirements for routing

Hashing algorithmGeographic distribution

Experiments

Semantics of Caching with SPOCA: A Stateless,Proportional, Optimally-Consistent Addressing

Algorithm

Piotr Skowron

November 6, 2011

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional, Optimally-Consistent Addressing Algorithm



Experiments

Abstract

This presentation is about:

how Yahoo! efficiently serves millions of videos from its videolibrary,

what architecture they use to ensure efficient caching (and sothe significant improvement in the quality of service),

how their new algorithm improved disk cache misses from 5%to less than 1% and increased memory cache hits from 45% to80% (thus improving overall cache hits from 95% to 99.6%).




Experiments

The way it works

clients access videos using web browsers,

clients connect to front-end servers which serve the videocontent,

the front-end servers cache content, but are not thepermanent repository,

videos are stored in a storage farm that is accessible throughfront-end servers.




Experiments

The way it works – diagram




Experiments

Local or remote serving

The videos are spread around the world so when the user isaccessing a video we have one of the following options:

retrieve the content from the storage farm /,

if it is cached in the disks of a front-end server, the contentcan be served more efficiently ,,

in the best case the content may be cached in the memory ofa front-end servers ,.

In case of videos the difference between caching in memory andthe caching in disks is small.




Experiments

Local or remote serving

Retrieving a content from the storage farm not only causessignificantly longer delivery, but also causes more load on theback-end infrastructure. This causes:

higher cost of upgrading networking components,

higher number of the servers in the storage farm required, inorder to handle more load.




Experiments

Why caching is difficult in case of the video?

Why caching is difficult in case of the video?

videos are large – a typical front-end server can hold 500unique videos in memory and 100,000 on disk,

the demand is high – users make over 30,000,000 requests perday for over 800,000 unique videos; there is over 20,000,000of unique videos in the library,

the ratio: total/unique requests in low.




Experiments

The traditional VIP (Virtual IP)




Experiments

The traditional VIP (Virtual IP)

With VIP each piece of popular content will end up on multipleservers → this redundant caching is highly inefficient compared tocaching where each piece of content is kept at the single server.

On the other hand remembering which front-end server hostswhich content is expensive.

The question of a day is: how to increase caching efficiencythrough intelligent routing without remembering content locationin a database.




Experiments

Stateless

There should be no need of keeping a data catalog associatingeach content file with a particular front-end server.




Experiments

Consistent

The requests for a particular content file should all be directed tothe same server. For stateless addressing the inputs are a filenameand a list of currently available servers; the output is a server fromthe list. Consistency means that the same input always producesthe same output.




Experiments

Proportional

The requests should be partitioned between the front-end serversproportionally to the servers weights.

Example

A newer server might have twice the capacity of an older one, andtherefore should service twice as large portion of the content library.

Remark

The proportionality requirement rules out the use of adistance-based consistent hashing algorithm, although suchalgorithms are consistent and stateless.




Experiments

Proportional

Proportionality must be ensured also in case of adding or removinga server. The requests must not be distributed only between thenearest servers, therefore the distance-based hashing algorithms donot pass their exam.




Experiments

Optimally-consistent

When a front-end server enters or leaves the pool as few aspossible files are redistributed to the other servers.

Example

Suppose that a pool has 3 front-end servers of weight 100 and twoservers of weight 200. If the new server of weight 200 is added tothe pool it must be assigned 2

9 of the files in the content library.But also, more specifically, for each of the other servers it musttake over 2

9 of the files that server was handling.




Experiments

Optimally-consistent




Experiments

Proportionality once again

It must be taken into account that proportional distribution of thefiles does not necessarily means the proportional distribution of therequests.




Experiments

Hashing algorithm

The routing function uses a hash function to map file namesto the points in a hash space.

Each front-end server is assigned a portion of hash spaceproportional to its capacity.

Not every point in the hash space maps to a front-end server,so when the hash of the name of a requested video maps tounassigned space, the result of the hash function is hashedagain until result lands in an assigned portion of the hashspace.




Experiments

Unassigned space hit




Experiments

Leaving server




Experiments

Entering server




Experiments

Hashing algorithm

The percentage of the assigned space is between 1%-25%dependently on the anticipation of future growth.

If the percentage of the assigned space is 1% then SPOCAwill have to generate on average 100 pseudo-random numbersto distribute tone request – this is efficient as linearcongruence generators are very simple.

By starting from 1% of the assigned space the system maygrow to almost 100 times bigger, without worrying of runningout of room.




Experiments

How to deal with popular files

The hash of each filename, seed, is saved in the memory for aconfigurable length of time, T .

When T elapses the seed is thrown away from the memory.

If a request arrives for a file for which the request router has asaved seed, that file is deemed to be popular, and thegeneration of pseudo-random numbers starts from the savedseed rather than from the filename.




Experiments

So the SPOCA algorithm looks like that




Experiments

Geographic distribution

The separate problem is how to take into consideration geographicdistribution of the content. The second question of a day,therefore, is: when it is profitable to serve data from the storingcluster, home locale, and when it is profitable to cache it near tothe user, at nearest locale.

The solution: we want to serve unpopular data from the homelocal, while the popular data should be fetched to the nearestlocale. This is done by the independent Zebra protocol.




Experiments

Bloom filters

Zebra tracks popularity of the files using Bloom filters.

Bloom filter

A Bloom filter, conceived by Burton Howard Bloom in 1970, is aspace-efficient probabilistic data structure that is used to testwhether an element is a member of a set. False positives arepossible (but rarely), and false negatives are not.

Unfortunately it is not possible to remove content from the bloomfilter, therefore Zebra uses 17 Bloom filters instead of the singleone. Each bloom filter represents requests for a given interval, onthe order of hours. When the oldest filter expires, it is removed,and the new filter is added.




Experiments

Data growth




Experiments

Back-end servers traffic without SPOCA




Experiments

Back-end servers traffic with SPOCA




Experiments

Estimations of the savings in the storage costs




Experiments

The impact of the szie popularity window (T) on cachingefficiency

Here the popularity window was decreased from 300s to 240s.

Decreasing the window increases cach hits, however decreasing ittoo much can result in overloading the servers.




Experiments

Related work




Experiments

Thank You

Thank You :)


semantics of caching with spoca: a stateless, proportional

Documents