hashcache: cache storage for the next billion · •hashcache: storage engine w/ plug-in indexing...

HashCache: Cache Storage for the Next Billion

1

Anirudh Badam

KyoungSoo Park Vivek S. Pai Larry L. Peterson

Princeton University

Next Billion Internet Users

2


• Schools, urban middle class in developing regions

2



• Affordable hardware

• OLPC and Intel Classmate

2

$200



• Affordable hardware

• OLPC and Intel Classmate

• Expensive Internet

• $1500+ per month per Mbps

• Unlikely to improve in the near future

2

$200

$1500per month

Bandwidth Saving

3

Bandwidth Saving

• Connectivity is a precious resource

3

Bandwidth Saving


• Saving bandwidth important

• Disk caches reduce network bandwidth requirement

3

Internet

Cache

Bandwidth Saving


• Saving bandwidth important

• Disk caches reduce network bandwidth requirement

• Good news - disk is very cheap

• $100/TB

3

Internet

Cache

Why Large Caches?

4

Why Large Caches?

• Larger bandwidth savings

• Refreshes cheaper than re-fetches

• Overnight prefetch, content push from peers

4

Why Large Caches?




• Good offline behavior

• Preload websites

• Enables local search

4

Why Large Caches?




• Good offline behavior

• Preload websites

• Enables local search

• Save even on dynamic content

• WAN Acceleration = packet caching

4

What is the Cost?

5

What is the Cost?

• In-memory data structures

• Hash table avoids seeks for misses

• Cache replacement (LRU, etc)

5

70 seeksper sec

What is the Cost?




• RAM index size per TB

• Open Source (Squid) - 10 GB

• Commercial (Tiger) - 5 GB

5

70 seeksper sec

What is the Cost?




• RAM index size per TB

• Open Source (Squid) - 10 GB

• Commercial (Tiger) - 5 GB

• Can not use laptops for cache

• 2 servers, $2K each = 20 laptops

5

70 seeksper sec

x 20

Our Solution

6

• HashCache: storage engine w/ plug-in indexing

• 6 schemes in paper, 3 shown here

• New Web proxy using HashCache engine

Our Solution

6




• Efficiency and Performance:

• Range of policies trading speed and memory

• 20-50x less RAM for Squid-like speed

• 6-10x less RAM vs Tiger (Commercial)

Our Solution

6




• Efficiency and Performance:

• Range of policies trading speed and memory

• 20-50x less RAM for Squid-like speed

• 6-10x less RAM vs Tiger (Commercial)

• Can now use cheap laptops vs servers

• Even for TB-sized caches

Our Solution

6

Performance: req/sec/disk

0

1

2

3

4

5

0 70 140 210 280 350

HashCache

Current

SquidTigerG

igab

ytes

/Dol

lar

Our Solution

6

Better

A Brief History of Cache

7


7

URL

http://www.example.com/index.html



7

URL

ddddhhhhbbbb




7

URL

ddddhhhhbbbb} } }

DirectoryLevel 1

DirectoryLevel 2

Actual File




7

URL

ddddhhhhbbbb} } }

DirectoryLevel 1

DirectoryLevel 2

Actual File

In-memoryHashtable




7

URL

ddddhhhhbbbb} } }

DirectoryLevel 1

DirectoryLevel 2

Actual File

In-memoryHashtable

Squid




7

URL




7

URL

Filesystem

CircularLog




7

URL

hash_value

Filesystem

CircularLog




7

URL

hash_valueoffset

Memory

hash_value

Filesystem

CircularLog




7

URL

hash_valueoffset

Memory

hash_value

Filesystem

CircularLog

Tiger




• Open Source Implementation - Squid

• Multiple seeks for hit, miss and write

• Dependent on default filesystems

• Commercial/High Performance - Tiger

• One seek for hit

• Custom file layout

7

Index Element Sizes

8

FunctionalityImplementation

ChoiceSquid(Bits)

Tiger(Bits)

ExistenceIdentification

HashtableChaining Pointers 96 96

Hash 160 32

Replacement Policy LRU List Pointers 64 64

LocationInformation

Disk Offset,Version Number,

etc0 40

OtherExpiration Date, Size, HTTP header info etc 240 0

Total 560 232

Index Element Sizes

8


ChoiceSquid(Bits)

Tiger(Bits)



Hash 160 32


LocationInformation


etc0 40


Total 560 232

Index Element Sizes

Focusing mainlyon reducing thesize of the index

8


ChoiceSquid(Bits)

Tiger(Bits)



Hash 160 32


LocationInformation


etc0 40


Total 560 232

Index Element Sizes

8

Revisiting the Index...

9


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

9

Cache sizelimited by

memory size


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

9


memory size


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

Performancedepends ondisk seeks

9


memory size


In-memoryIndex

Application Cache

Manager

ReliableFilesystem


reduce thedependency

9


memory size


In-memoryIndex

Application Cache

Manager

ReliableFilesystem


reduce thedependency

optimizefor seeks

9


• Eliminate(?) in-memory index

In-memoryIndex

Application Cache

Manager

ReliableFilesystem

9



• Need membership and location information

In-memoryIndex

Application Cache

Manager

ReliableFilesystem

9




• Use disk as hash tableIn-memoryIndex

Application Cache

Manager

ReliableFilesystem

9




• Use disk as hash table

• On disk data structures for key lookup

• Store the object as values for the keys

In-memoryIndex

Application Cache

Manager

ReliableFilesystem

9


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

HashCache Engine

9

ApplicationLogic

Application Cache

Manager

ReliableFilesystem

In-memoryIndex


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

HashCache Engine

9

ApplicationLogic

ReliableFilesystem

In-memoryIndex


In-memoryIndex

Application Cache

Manager

ReliableFilesystem

HashCache Engine

9

ApplicationLogic

In-memoryIndex

HashCache: Basic Policy

10

URL


10



URL


data

10



URL


hash_value

data

H Bits

10



URL


hash_value

data

H Bits

N contiguousblocks

Filesystem(Disk Table)

10



URL


hash_value

% N

t

data

H Bits

N contiguousblocks


10



URL


hash_value

% N

t

data

t th block

H Bits

N contiguousblocks


10



URL

CircularLog

head


hash_value

% N

t

data

t th block

H Bits

N contiguousblocks


10



URL

CircularLog

head


hash_value

% N

t t th block

H Bits

N contiguousblocks


10



• Advantages

• No index memory needed

• Tuned for one seek for most objects


10

• Advantages

• No index memory needed

• Tuned for one seek for most objects

• Disadvantages

• One seek per miss

• No collison control

• No cache replacement policy


10

Collision Control

11

Collision Control

Chaining

11

Collision Control

• Does not transition well to disk-based

• Multiple seeks per operation

• Walking hash bin list

• Global replacement policy crosses bins

Chaining

11

Collision Control

Chaining

11

Collision Control

• Fixed locations where each object can be found

• Allocated contiguously, read together

Set AssociativityT-Ways

11

Collision Control



• Seek time dominates short read

Set AssociativityT-Ways

11

Collision Control



• Seek time dominates short read

• Eliminate global cache replacement policiesSet Associativity

T-Ways

11

Reducing Seeks

12

Reducing Seeks

• In-memory hash table

• Too much memory for pointers

Bin Pointers 32

Chaining Pointers 64

Hash 32

Total (bits) 128

In-memory Hash Table

12

Reducing Seeks



• Disk is already a hash table

• Pointers not needed

• Large bitmap with the same layout as the disk

Disk Table

12

Reducing Seeks



• Disk is already a hash table

• Pointers not needed

• Large bitmap with the same layout as the disk

• Just store hash per URLDisk TableIn-memoryBitmap

H Bits Disk Block

12

Reducing Seeks

• Original hash of the URL: 64 bits

12

64 OriginalHash

Reducing Seeks


• Eliminate bits for (same) bin # (228 objs, 8-way, #bins=225 (S))

12

64 OriginalHash

39 64 - log(S)

Reducing Seeks


• Eliminate bits for (same) bin # (228 objs, 8-way, #bins=225 (S))

• Shrink hash size: Just to eliminate most false positives (8 bits)

12

64 OriginalHash

39 64 - log(S)

8 low FP hash

Cache Replacement

13

64 OriginalHash

39 64 - log(S)

8 low FP hash

Cache Replacement

13

• Large disks: 10-100+ million objects

• Global caching relevant when disk size working set

• When disk >> working set, local policies global policies

!

!

64 OriginalHash

39 64 - log(S)

8 low FP hash

Cache Replacement

13

• Large disks: 10-100+ million objects

• Global caching relevant when disk size working set

• When disk >> working set, local policies global policies

• Local replacement benefits

• 3 bits per URL

• Performed on contiguous objects

• False positives limited by set size

!

!

64 OriginalHash

39 64 - log(S)

8 low FP hash

11 hash + rank

HashCache: SetMem Policy

14


URL

14




URL

data

14




URL

hash_value

% S

t

data

14




URL

hash_value

% S

t

data

Filesystem

head

Memory

14




URL

hash_value

% S

t

data

Filesystem

head

Memory

11 Bits

tth set

tth set

14




URL

hash_value

% S

t

Filesystem

head

Memory

11 Bits

tth set

tth set

14




URL

hash_value

% S

t

LRULRU

Filesystem

head

Memory

11 Bits

tth set

tth set

14



• Advantages

• No seeks for most misses

• 1 seek per read, 1 seek per write

• Good hash + replacement in 11 bits


14

• Advantages

• No seeks for most misses

• 1 seek per read, 1 seek per write

• Good hash + replacement in 11 bits

• Disadvantages

• Writes still need seeks


14

Further Reducing Seeks

Disk Table

15


• Storing objects by hash can produce random reads & writes

Disk Table

15



• Restructure on-disk table

Disk Table

15




• Store only hash, rank, offset

15

11 hash rank





• Move all data to log

15

11 hash rank

43 hash

rank

offset






• Benefits

• Group writes amortize seeks

• Scheduling related writes enables read prefetch

• Both reads & writes < 1 seek

15

11 hash rank

43 hash

rank

offset






• Benefits




15

0 HC-Basic

43 HC-Log

11 HC-SetMem






• Benefits




15

560 Squid

232 Tiger

0 HC-Basic

43 HC-Log

11 HC-SetMem

HashCache: Log Policy

16


URL

16


URL

data

16


URL

hash_value

% S

t

data

16


URL

hash_value

% S

t

data

CircularLog

Filesystem

head

LRU

Memory16


URL

hash_value

% S

t

data

CircularLog

Filesystem

head

LRU

Memory

tth set43 Bits

16

Implementation

17

Implementation

• HashCache Storage Engine with plug-in policies

17

Implementation


• HashCache Web proxy using storage engine

17

Implementation



• Multiple apps on same box, sharing memory

17

Implementation




• 20,000 lines C code for the proxy and 1000 lines for the indexing policies

17

Implementation





• Event Driven implementation with non-blocking I/O

17

Implementation






• Design similar to that of Flash Web Server. Helpers for I/O and DNS lookups

17

Implementation






• Design similar to that of Flash Web Server. Helpers for I/O and DNS lookups

• Balances load across multiple disks easily and makes scaling obvious

17

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies

• Compare all variants of HashCache with Squid and Tiger

18

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies

• Compare all variants of HashCache with Squid and Tiger

18

ExperimentName Setting Configuration Comparision

Low End Small School using Laptop

1.4 GHz256 MB

60 GB SATA

HashCache vsSquid vs Tiger

High End ISP with High-End Server

2 GHz3.5 GB

5x18 GB SCSI

HashCache-Log vsSquid vsTiger

Large DiskLarge

School withMini-Tower

1.4 GHz2 GB

2x1TB USB

HashCache-Log vsHashCache-SetMem

0

15

30

45

60

HC-Basic Squid HC-SetMem HC-Log Tiger

Hit

Rat

e

Max

Hit Rate Comparison

19

Low End Configuration


0

75

150

225

300

Perf

orm

ance

req

/sec

HC-Basic Squid HC-SetMem Tiger HC-Log


0

75

150

225

300

0% RAM for Index

75% 5%

50%20%

Open Source and Commercialcould index only 18 GB

HashCache couldindex 60 GB

Perf

orm

ance

req

/sec

HC-Basic Squid HC-SetMem Tiger HC-Log

0

750

1,500

2,250

3,000

Perf

orm

ance

req

/sec

Squid TigerHashCache-Log

High End Configuration

21

5x18 GB Disk

0

750

1,500

2,250

3,000

Perf

orm

ance

req

/sec

Squid TigerHashCache-Log

40% RAMfor Index

4% 18%

High End Configuration

21

5x18 GB Disk

Index Efficiency

Squid

Tiger

HC-Log

HC-SetMem

0 1,500 3,000 4,500 6,000

Index Efficiency

Max Disk for 1GB RAM

Squid

Tiger

HC-Log

HC-SetMem

0 1,500 3,000 4,500 6,000

Index Efficiency

Max Disk for 1GB RAM

40 reqs/sec

300

300

65

Large Disk Configuration

23

1 TB Disk

0

75

150

225

300

HC-Basic HC-SetMem HC-Log

Perf

orm

ance

req

/sec

Large Disk Configuration

23

1 TB Disk

0

75

150

225

300

HC-Basic HC-SetMem HC-Log

Perf

orm

ance

req

/sec

150 MB

600 MB

0 MBIndex

Conclusions & Status

• New Storage Engine & Web Cache

• From no RAM per object to tiny no. of bits/obj

• 6-10x better than Tiger, 20-50x vs Squid

• Enables large disk w/ only laptop-class machine

• More policies, details in paper

• Suitable for developing-world usage

• Current deployments: Ghana, Nigeria

• Working w/ school supplier on new deployments

24

Thank [email protected]

http://tinyurl.com/hashcache

25

mailto:[email protected]

mailto:[email protected]

http://blah

http://blah

hashcache: cache storage for the next billion · •hashcache: storage engine w/ plug-in indexing...

Documents