WMPI 2006, Austin, Texas © 2006 John C. Koob

An Empirical Evaluation of Semiconductor

File Memory as a Disk Cache

John C. Koob

Duncan G. Elliott

Bruce F. Cockburn

VLSI Design Lab

ECE Department

University of Alberta

Edmonton, Alberta

Canada

WMPI 2006, Austin, Texas Slide 2 John C. Koob, University of Alberta

Outline

Motivation

Extended Storage

File Memory

Experimental Platform

Cost/Performance Analysis

Conclusions

WMPI 2006, Austin, Texas Slide 3 John C. Koob, University of Alberta

Motivation

Source: Computer Architecture, Hennessy & Patterson, 2003

WMPI 2006, Austin, Texas Slide 4 John C. Koob, University of Alberta

Access Time Gap Problem

Use Extended Storage Cheaper per bit than main memory Faster than disk Slower than main memory Potential for power savings

How to fill the access time gap?

WMPI 2006, Austin, Texas Slide 5 John C. Koob, University of Alberta

Historical Systems

Extended storage first appeared in expensive systems

IBM 3090 mainframe Main memory: 0.5 GB Extended storage: 4 GB Terminology: Expanded Storage

Image courtesy of www.ibm.com

WMPI 2006, Austin, Texas Slide 6 John C. Koob, University of Alberta

Historical Systems

Extended storage first appeared in expensive systems

Cray Y-MP supercomputer Main memory: 1 GB of 15-ns SRAM Extended storage: 4 GB of 50-ns DRAM Terminology: Solid State Disk

Image courtesy of the Charles Babbage Institute

WMPI 2006, Austin, Texas Slide 7 John C. Koob, University of Alberta

Recent Research

Compressed caching (1999-2003) Compression can reduce paging costs Adaptive sizing of compressed page cache

Multi-level main memory (WMPI 2004) 30% of memory must run at DRAM speed Remaining memory can be slower

WMPI 2006, Austin, Texas Slide 8 John C. Koob, University of Alberta

Extended Storage Today?

Emerging technology may prompt

a return to extended storage

Semiconductor file memory Up to 5 times slower than DRAM Cheaper per bit than DRAM

MEMS probe-based storage 5 times faster than disk 10 times more expensive than disk

WMPI 2006, Austin, Texas Slide 9 John C. Koob, University of Alberta

What is File Memory?

File memory leverages current DRAM technology

DRAM design constraints increase costs per bit 100% of nominal capacity must be functional Contiguous address space

File memory relaxes DRAM’s design constraints Bad block marking to improve yield Address space is not contiguous Improve density at the expense of performance

(e.g. multi-level DRAM or hardware compression)

WMPI 2006, Austin, Texas Slide 10 John C. Koob, University of Alberta

Feasibility of File Memory

A precedent for file memory exists

in the non-volatile memory market

NOR Flash memory Limited capacity Moderate reliability Random-access supported

NAND Flash memory High capacity Low reliability bad block marking Restricted to sequential access

Contiguous Memory

Non-Contiguous Memory

WMPI 2006, Austin, Texas Slide 11 John C. Koob, University of Alberta

Extended Storage Disk Cache

To evaluate file memory as extended storage: Require an experimental platform Modify Linux 2.4.18 OS kernel

ESDC Design Summary High memory support Page cache containment Configurable performance CPU caching issues Performance metrics

WMPI 2006, Austin, Texas Slide 12 John C. Koob, University of Alberta

Postmark Results using File Memory

WMPI 2006, Austin, Texas Slide 13 John C. Koob, University of Alberta

Postmark Results Analysis

Need 39% more file memory for equivalent performance

WMPI 2006, Austin, Texas Slide 14 John C. Koob, University of Alberta

Summary of Postmark Results

0 256 512 768 1024

Additional Memory Capacity (MB)

2

3

4

5

File

Me

mo

ry S

low

do

wn

Equivalent Performance Comparison

File Memory

DRAM

WMPI 2006, Austin, Texas Slide 15 John C. Koob, University of Alberta

Conclusions

Use file memory for extended storage Leverage DRAM cell technology Relax DRAM design constraints Use bad block marking

Preliminary evaluation of ESDC File memory can be up to 4 times slower than DRAM Performance improved even with no page cache

Ongoing research Evaluate hierarchies with file memory and page cache

WMPI 2006, Austin, Texas Slide 16 John C. Koob, University of Alberta

Selected References

Bray. Bonnie. www.textuality.com/bonnie, 1996.Castro et al. Adaptive compressed caching. Symp. on Comp. Arch.

And High Performance Computing, Nov. 2003.Ekman and Stenstrom. A case for multi-level main memory. WMPI

2004.Hennessy and Patterson. Computer architecture: A quantitative

approach. Third Edition, 2003.Katcher. PostMark: A new filesystem benchmark. TR3022, Network

Appliance, Oct. 1997.Koob et al. Test and characterization of a variable capacity multilevel

DRAM. In Proc. VLSI Test Symp., pp. 189-197, May 2005.Uysal et al. Using MEMS-based storage in disk arrays. FAST 2003,

pp. 89-101.

WMPI 2006, Austin, Texas Slide 17 John C. Koob, University of Alberta

Configurable Performance

How to model different file memory access times? Use multiple page copies Gives accurate file memory slowdown ratios

Problem: Repeated page copies would be cached

Solution: Disable CPU caches for ESDC

– Use IA-32 memory type range registers (MTRRs)

WMPI 2006, Austin, Texas Slide 18 John C. Koob, University of Alberta

Experimental Setup

Experimental Platform Processor 2.4 GHz Pentium 4 Memory 2 GB DDR SDRAM Hard disk 18-GB Seagate SCSI Disk buffer 4-MB

Experimental Suite PostMark – benchmark for many small files Bonnie – file system benchmark Kernel compilation – Linux kernel build

WMPI 2006, Austin, Texas Slide 19 John C. Koob, University of Alberta

Postmark Results for Original Hierarchy