the solid state storage revoluon · 2008-10-30 · the solid state storage revoluon andy...
TRANSCRIPT
![Page 1: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/1.jpg)
The Solid State Storage Revolu1on
Andy Bechtolsheim Sun Microsystems
![Page 2: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/2.jpg)
A Quick History of the Hard Disk
• 1956: IBM introduces 350 Storage Unit, first Hard disk (5 MB)
• 1962: IBM introduces Model 1301 Advanced Disk File (25 MB) • 1970: IBM introduces Model 3330 DAS Disk Drive (100 MB)
• 1980: IBM introduces Model 3380 DAS Disk Drive (2500 MB) • 2002: IBM sells its disk business to Hitachi
• 2004: 3.5” 7200 RPM Disks store 250 GB (250,000 MB)
• 2006: 50 Year Anniversary of the Hard disk drive • 2008: 3.5” 7200 RPM Disks store 1 TB (1,000,000 MB)
![Page 3: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/3.jpg)
![Page 4: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/4.jpg)
![Page 5: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/5.jpg)
![Page 6: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/6.jpg)
Hard Disk $/GB Cost Evolu1on $/GB
TIme
![Page 7: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/7.jpg)
IOPS Per Hard Disk
7200 RPM 10K RPM 15K RPM 125 IOPS 180 IOPS 250 IOPS
300
200
100
0
![Page 8: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/8.jpg)
Hard Disk Evolu1on 1998‐2008
• Great improvements in: – Density (average 60% / year growth) – Cost per GB (tracking density)
• Some improvements in: – Transfer rate (higher bit density) – Interface speed (SAS‐SATA I/II/III)
• No significant improvements in: – IOPS per disk (more or less same) – Reliability per disk (more or less same)
![Page 9: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/9.jpg)
Hard Disk Summary
• HDD have made enormous improvements in capacity, cost per GB and reliability per GB
• Very limited improvements in performance
• IOPS per GB are con1nuously declining • In the meanwhile, server performance is increasing at 50‐60% per year, leading to a major CPU versus I/O performance “crisis”
![Page 10: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/10.jpg)
In the meanwhile…
![Page 11: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/11.jpg)
A New Development: Flash Memory
• 1985: Toshiba shows first Flash chip (256 Kbit) • 1995: First Mul1‐level Flash chip (32 Mbit)
• 2002: Samsung delivers 1 Gbit chips (120 nm)
• 2005: Samsung delivers 4 Gbit chips (70 nm)
• 2006: Flash Market hits $12B in revenue
• 2008: Toshiba delivers 16 Gbit chips (56 nm)
![Page 12: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/12.jpg)
A 16 Gbit Flash Chip
![Page 13: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/13.jpg)
Gartner Flash Forecast (Aug 2008)
![Page 14: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/14.jpg)
The NAND Flash Price Free Fall
![Page 15: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/15.jpg)
SLC and MLC FLash
• SLC: Single‐level Flash: 1 Bit / T – Most Robust Flash Technology – > 500,000 Write Cycles – Ideal for enterprise applica1ons
• MLC: Mul1‐level Flash: 2‐3‐4 Bits/T – Most Cost‐effec1ve Flash Technology – > 10,000 Write Cycles – Ideal for consumer applica1ons
![Page 16: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/16.jpg)
Flash Reliability
• SLC is the most reliable – 5 Year Life cycles with con1nuous writes – Gradual capacity reduc1on over 1me – No mechanical failure modes
• MLC does not do a lot of writes – Even with RAID device does not last – Only usable in read‐mostly applica1ons – There are some of these in enterprise
![Page 17: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/17.jpg)
Flash Performance
• Random Read Access Time: 100 us (10K IOPS) – This is per device or device channel – Typical controller has 4 channels – Typical controller read performance is 30K IOPS
• Write cycles are considerably slower – Erase cycles are painfully slow – Controller hides these with erase‐ahead – Typical controller write performance is 10K IOPS
![Page 18: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/18.jpg)
Flash Performance Roadmap
• New “Enterprise Flash” device category – Higher reliability – Higher throughput – Lower access 1mes
• Next‐genera1on Flash Controllers – Faster I/O Interfaces, more channels
– Faster CPU clock rates – Larger DRAM Caches
![Page 19: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/19.jpg)
Role of Flash Memory Controller
• FMC is Key for Reliability and Performance • Func1ons of FMC: – Error correc1on – Wear leveling – Bad Block mapping – Scrubbing – Write caching – Read caching
• Significant Improvements Ahead
![Page 20: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/20.jpg)
Typical Flash Memory Controller
SAS/SATA Interface
FMC DRAM
Flash Flash Flash Flash
uCode
![Page 21: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/21.jpg)
Flash vs Disk Performance
• Flash is more than 100X faster than disk • Flash consumes less power than hard disk
• Flash is more reliable than hard disk
• Flash is physically smaller than hard disk
• Flash is a lot cheaper per IOPS than Disk • Disk is a lot cheaper per GB than Flash
![Page 22: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/22.jpg)
Flash as a Disk Cache
• Flash as a Cache is a very promising Model – Most stored data is not ac1vely being accessed
– Working set is typically a few % of total capacity – Caching the ac1vely used data in Flash makes the en1re disk storage appear as fast as Flash
• Significant gain for I/O intensive workloads • Key is to not have to change the applica1on
![Page 23: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/23.jpg)
Flash Cache in Solaris ZFS
• ZFS already supports a DRAM memory cache – Cache efficiency is limited to size of main memory
• Trivial to extend mem‐cache to FLASH – Flash can be much larger than main memory
• Flash cache divided into two segments: – Readzilla for Read Caching – Logzilla for Transac1onal Commits
• Delivers significant cost‐performance gains
![Page 24: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/24.jpg)
ZFS Hybrid Pool Example
![Page 25: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/25.jpg)
Sun Project Lightning Flash
• What is the speed limit with Flash? • Are mul1‐million IOPS achievable?
• What storage fabrics can support this?
• How much space/power/cooling will this take?
![Page 26: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/26.jpg)
Lightning Flash First Results
• Speed limit is HBAs, not Flash – HBAs were not designed for Flash I/O rates
• One million IOPS achievable today – Requires lots of PCI slots and HBAs
• Exis1ng HDD arrays are problema1c for SSDs – There are simply not enough I/O channels
• SSDs significantly improve IOPS power efficiency – Almost a factor of 100X compared to hard disk
![Page 27: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/27.jpg)
Other Flash Use Models
• Use Flash as boot device – No‐brainer, just a ques1on of cost – Saves space and power
• Make Flash part of the memory hierarchy – Unfortunately this is not applica1on transparent – Flash is ~ 1000X slower than DRAM
• Replace exis1ng disk access protocols – Memory transfers more efficient than SCSI – Will be required to scale to Mul1‐million IOPS
![Page 28: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/28.jpg)
Gartner HDD and Flash Forecast
![Page 29: The Solid State Storage Revoluon · 2008-10-30 · The Solid State Storage Revoluon Andy Bechtolsheim Sun Microsystems A Quick History of the Hard Disk • 1956: IBM introduces 350](https://reader033.vdocuments.us/reader033/viewer/2022042001/5e6dbf4c16f1b9735b64f663/html5/thumbnails/29.jpg)
In Conclusion
• Flash is ~ 100X faster than Hard disks • Flash is ~ 100X more IOPS power efficient
• Flash prices are dropping about 50% / year • This makes Flash suddenly very interes1ng
• Using Flash as cache looks very promising
• Flash Performance @ Cost of Hard Disk
• Hard Disks are not going away any1me soon