a novel method to extend flash memory lifetime in flash based dbms
TRANSCRIPT
A novel method to extend flash memory lifetime
in flash-based DBMS
Zhichao Liang(Renmin University of China)[email protected]
Outline
• Motivation• Our work• Experiment evaluation• Conclusion
Outline
• Motivation• Our work• Experiment evaluation• Conclusion
SSD Write-lifetime
60GB X 5000 =300TB
60GB X 5000 =300TB
40TB(60GB drive)
40TB(60GB drive)
37TB(80GB drive)
37TB(80GB drive)
Maximum amount of data can be written to
SSD
Maximum amount of data can be written to
SSD
Extending SSD Lifetimes with Disk-Based Write Caches by Gokul Soundarajan. FAST 10
Extending SSD Lifetimes with Disk-Based Write Caches by Gokul Soundarajan. FAST 10
Original sin
• Erase before write >>> Out-place update• Limited erase-program cycles >>> Wear out• The abuse of write can be poison for flash
- small write >>> storage utilization declines- random write >>> frequent erase
Behave the write pattern !Behave the write pattern !
Free space management
• To maximize the storage space, free space management methods are employed in DBMS
• The commonly usedmethods are space maps and free lists
4
2043
4 2
…
Heap pages
Binary tree
Free_list_header
Data page Data page Data page Data page
Towards Effective and Efficient Free Space Management by Mark L. McAuliffe. ACM SIGMOD Record
Towards Effective and Efficient Free Space Management by Mark L. McAuliffe. ACM SIGMOD Record
The endemic
• Both space maps and free lists are based on the assumption the in-place update is supported
• In-place update is impossible on flash, so …
Outline
• Motivation• Our work• Experiment evaluation• Conclusion
The overview
• The framework of our methodDBMS buffer management
DBMS storage management
Flash device
Storage space
Shared memory
Write buffer
Data Pages
Append Only
Read
Write
Insert
update
Main memory Data file Underlying device
Append in file and memory
• Append Only(AO): we replace the traditional methods used in disk-based DBMS with append only- no complicated data structures and useless searches- the simplest and fastest algorithm
• Write Buffer: write buffer is a stand-alone area in main memory- PagesForRelation & RelationForBuffer- flushing out until pages are full or a checkpoint occurs
Write buffer (cont.)
• For insert and update operation
Write
ReadShared memory
Insert
update
Write buffer (cont.)
• The advantages- reduces the number of small write- reduces the amount of data to write and the number of write operation to execute- the number of random write declines
• The disadvantages- more logging data to read when the system crashes- destroy the data distribution- delay the erase operation indirectly
Clustering and Non-clustering Effects in Flash Memory Databased by Duck-Ho Bae. 20th
DEXA
Clustering and Non-clustering Effects in Flash Memory Databased by Duck-Ho Bae. 20th
DEXA
Merge operation
• Write buffer&AO affect storage in another wayOriginal page 1st update
To be erased
To be erased
To be erased
2nd update
Traditional method
Our method
Merge operation (cont.)
• Obsolete Address Log (OAL) and merge operation are introduced to compact the pages with “holes”PageId Offset Length
2 48 256
17 56 700
2 4012 120
5 569 400
2 345 3000
Page 2 3376
Page 5 400
Page 17 700
(48, 256)
valid valid
(345, 3000)
valid (4012, 120)valid valid valid
Write buffer
Outline
• Motivation• Our work• Experiment evaluation• Conclusion
Experiment evaluation
• Experiment environment: a flash simulator with a plain FTL Algorithm
• Comparison: Traditional method(LRU for buffer management) + Our method(LRU for shared memory)
• Test case: four different trace file
Tracefile1 R/W=2/8
Tracefile1 R/W=2/8
Tracefile2 R/W=4/6
Tracefile2 R/W=4/6
Tracefile3 R/W=6/4
Tracefile3 R/W=6/4
Tracefile4 R/W=8/2
Tracefile4 R/W=8/2
A Flexible Simulation Environment for Flash-aware Algorithms by Peiquan Jin. CIKM 09
A Flexible Simulation Environment for Flash-aware Algorithms by Peiquan Jin. CIKM 09
Experiment evaluation (cont.)
• Page read I/O • Page write I/O
Experiment evaluation (cont.)
• Total latency • Tendency of ratio
Outline
• Motivation• Our work• Experiment evaluation• Conclusion
Conclusion
• Contributions- we analyze and point out the traditional free space management is useless on flash - we propose to use AO & write buffer to reduce the small write and random write
• Future work- Implement the method in real system- More standard benchmark test
Q & A