challenges in storage systems - dtc.umn.edu in storage systems: ... disk-to-disk backup with tape...

Tag line, tag line

Challenges in Storage Systems: A NetApp perspective

Deepak Kenchammana-HosekoteAdvanced Technology GroupNetApp

© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08)

Agenda

�What we do (context)�What we see happening (trends)�What we are doing about it (initiatives)�How you can help us.


NetApp Fact Sheet

� Founded in 1992 � Fastest growing storage company� 4 consecutive years of 30%+ growth� #6 Best Company To Work For� Headquartered in Sunnyvale

– Other engineering sites in: Pittsburgh, RTP, Boston, Bangalore India

� 75,000+ systems installed worldwide� 111 Petabytes shipped Q1 FY08� #1 in NAS PB shipped� #1 in NAS market share

'03

'04

'05

'06

'03

'04

'05

'06

FY07 $2.8 billion‘07


Key NetApp Ideas

� Purpose-built appliance; does one thing well– One management model & one system to learn

� Simplicity– Simple on the outside ≠ simple on the inside– Easy to use, easy to set up

� Combination of then new research ideas– RAID [Patterson88] – Log-structured file system [Rosenblum92]

� All built from commodity components– Off-the-shelf x86 CPUs, memory, etc– NVRAM is main exception

© 2008 NetApp. All rights reserved. Presentation to DISC (05/13/08) 5

Scale from 1 TB to >6,000 TB

NetApp DataFort™Data encryption for audio, video, and image storage

Data Security

Information Server IS1200Information classification and management for unstructured data

Information Lifecycle Mgmt

Data ONTAP®Operating System (FC SAN, IP SAN, NAS)

Virtual Tape Library

NearStore VTLDisk-to-disk backup with tape library emulation

Media Center Storage

FAS6000 seriesFAS3000 seriesModular enterprise-class storage; massive scalability

Storage Virtualization

V-SeriesDynamic virtualization for heterogeneous storage

StoreVault™Entry-level nearline archives under $3K

Entry NAS Direct Attach/Small Archive NAS

FAS2000 SeriesExpandable NAS for small to medium nearline applications


Data ONTAP® from 30,000 feet…

� StorageRAIDWAFL

Pro

toco

ls

NetworkStack

Client

Client

Client

Client

Client NVRAM Disks

Net

wor

k Filer

…looks a lot like an operating system, but…– Optimized for efficient data movement

� Specialized interfaces between components

– Robust HW error recovery


WAFL file system

volinfo

Inodefile

File A File B

� No fsck. Ever! � Writes buffered in NVRAM

� Data never overwritten� Metadata stored in files


Writes

IA IB

A1 A2 B1 B2

Superblock

Inode file

File blocks

I0

Contains inodesfor regular files and dirs

File A File B


Writes (2)

IA IB

A1 A2 B1 B2 B2’

I0

Superblock

Inode file

File blocks

IB’

I0’


Snapshots

IA IB

A1 A2 B1 B2’

IB’

I0 I0’

S0

B2

Inode file

File Blocks

Snapshotsuperblock


Snapshots™ enable many features

� Efficient application recovery (SnapDrive)– MS Exchange Server & Oracle RDBMS

� Consistent image for backups (SnapMirror)– Only changed data need be mirrored remotely

� Compliant data repository (SnapLock)– Read-only online data

� Efficient creation of clones– Copy-on-write snapshot; cloning DBs very useful

� Quickly tell what’s changed (SnapDiff API)– Compare inode files of two snapshots


Beyond a Single Controller

� Scaling up– What if one controller is not big enough?– Want a namespace that spans multiple controllers

� Scaling out– Add capacity & performance with more controllers– “pay as you go” model

� But what happens to manageability?– Filers should only scale in performance and capacity;

management should be just the same“ One filer is easy to use. 100 filers are a pain to use.”

Answer: Clustered ONTAP


Clustered ONTAP Architecture

D-blade

N-blade

SpinNP

Protocol

DistributedVolume Location

Database

Clients

D-bladeD-blade

N-blade

…

NFS, CIFSiSCSI, FC

N-blade

WAFL

RAID

Storage


Global Namespace in a Cluster

Namespace Root

BA C

R

C2

D

D1 D2

Clustered ONTAP® System

C D1 D2 C2 D C1

B A

C1

R


Advanced Technology Group

� Under office of the CTO; 3yrs old– 25 Members and growing; 4 sites worldwide

� Pursue long term projects– Explore technology driving our strategic direction– Create opportunities beyond current product horizon

� University research investments– Coordination of research funding– Leverage investments through hiring, internships

Goal : Investigate new technologies, influence and create products


The Advanced Technology Group (2)

� University collaborations– Wisconsin-Madison, Carnegie Mellon, MIT, UCSC,

UCSD, Harvard, UIUC, Waterloo, Duke, Berkeley,…� Actively publish

– SIGMETRICS 2007 Best student paper– 6 papers with NetApp authors in FAST 2008 incl. Best

student paper– 2 papers in USENIX 2008


Latent Sector Errors Study [Sigmetrics07]

� How individual sector errors affect data integrity� Examined >70,000 systems with over 1.3 million disks� Are our defense mechanisms good enough?

� Findings: 3.45% of 1.53 million disks with ≥ 1 LSE � 8.5% of SATA vs. 1.9% of FC disks

SATA

On average 77% of Latent Sector Errors discovered b y VERIFY


A Comprehensive Study of Failure Characteristics [FAST 2008]

How to best improve resiliency to HW failures? – Ex: RAID group layout

Shelf enclosure model has strong impact on failures

Failures are not independent

The AFR for disks and storage subsystems does not increase with disk size.


Agenda

�What we do (context)�What we see happening (trends)�What we are doing about it (initiatives)�How you can help us.


Three Categories of Innovation

� New invention– Create an entirely new technology– Launch new industries

� Monotonic improvement– Technologies that ratchet ever upward

� Exponential improvement– Monotonic improvement that follows an exponential curve

� E.g. DRAM, Processor Mips, Flash Memory, Disk size– Observation 1: The steep part of the curve can be as enabling

as a new invention– Observation 2: The steep part of the curve can break old

solutions� E.g. disk size : disk IOPs ratio

– Observation 3: Suddenly becomes disruptive when applied to new areas, displace existing technology� E.g. tape archive replaced by disk archive� E.g. primary disk storage replaced by flash memory


Technology Drivers

� Performance� Capacity

� Security� Power

� Management and Complexity� New applications and value-add

� Global regulatory requirements


Enabling New Applications and Invention

� Server virtualization� Ubiquitous computing

– Network computing– PDA’s as clients– Globally distributed data service

� Increasing modularity and scale– Reducing visible complexity– Self-diagnosing, self-repairing servers– Federating services across administrative and

organizational boundaries– End-to-end integrity and security


The Storage Business

� Primarily about providing “containers”– LUNs– Files– Directories– Volumes

� Value-add primarily in:– Container virtualization– Container management– Container reliability– Container access performance– Archive, backup– Data security and integrity


Key Initiatives

� Building a better container� Scaleout

� Data Management� Data Protection and Retention

� The expanded vision…


Building a better container

� Increased virtualization– Fine-grained container hierarchy– Further disassociation of logical from physical– Dynamic, fine-grained data selection

� Alternative and parallel hierarchies of containers

� Simplified and improved manageability– Manage relatively few large-scale “datasets” composed

of many fine-grained logical containers


Building a better container (2)

� Make copies smart: – Leverage archive, backup as active assets.

� Compression and Dedup– Improve storage utilization

� In-place data encryption– Robust and secure key management

� Challenge is to combine these


Scaleout

� Multi-controllers– Tightly integrated hardware– Embedded clustering– Incrementally expandable– Scales up and down

� Increasing modularity in the core software– Separate key components of the data path from each

other and from software infrastructure– Exploit user space for value-added software

� Increase absolute numbers of containers of all types� Striping file system


Scaleout (2)

� Exploit scaleout technology to provide– Caching– Heterogeneous storage

� Hybrid storage hierarchies� Heterogeneous federations� Heterogeneous storage backends

� Storage and System resiliency– Self-diagnosing and self-repairing subsystems– Policy-driven fault recovery


Data Management

� Powerful data management within the single system image cluster

� Layered on-box and off-box data and storage management tools– Spans multiple clustered and unclustered systems– Heterogeneous participants

� Abstract data set model simplifies management– Pushes complexity into the underlying software layers


Data Protection and Retention

� Low (to zero) Recovery Point Objective (RPO), Recovery Time Objective (RTO) for mirrors

� Smart copies– Multi-use mirrors

� Advanced server and virtual server level DR� Integration with virtualized computing environments


The Expanded Vision

� Global Data Service: Federations– Many systems all interoperating to provide

� DR� Remote caching and Vserver presence� Archive and backup� A hierarchy of flexible virtualized containers independent of

the underlying hardware

– Connected across wide geographies� Weak interconnects� Different administration and trust zones

– Ultimately multi-vendor solutions� Requires standards


The Database Business

� Primarily about adding value to the “contents” of containers– Indexing– Query processing– Imposes structure on data to facilitate analysis by

reusable tools

� Storage systems do not generally impose any discernable structure aside from block or extent size– Containers are blobs


Containers and Contents

� Add value to stored content– Extracting structure and information from “unstructured”

blobs of data– Indexing– Query processing– Extended Attributes and Metadata

� E.g. “file provenance”

� Tighter integration of database and storage systems� Add value beyond providing containers

– Produce information from data


Tooling to meet challenges� Software engineering challenge is enormous

– Not unique to NetApp– How can we build ever larger systems quickly enough to meet

accelerating market needs?– How can we make all this software reliable?– How can we prove its reliability?

� Big challenges in:– Software tools– System test and validation– Support infrastructure, diagnosis and resolution– Globally distributed development

� Are the programming languages and tools up to the task?– Facing a lot of inertia– MP is now the norm– Moving from C to C++ is not the answer

� We have many of the same problems we help our customers solve


How you can help us

� We are seeking collaboration and leverage in the academic community– Equipment donation– Funded research– Internships– Full-time hires

� We would like to share real world data– System logs, traces, …

� Hope to energize you to look at some of these very interesting problems

Tag line, tag line

We’re hiring!

Questions?

challenges in storage systems - dtc.umn.edu in storage systems: ... disk-to-disk backup with tape...

Documents