zfs on the server and the desktop: n+1 ways to better store your data

ZFS on the server and the desktop

N+1 ways to better store your data

Presented at NLUUG 21-11-2013

About

Matthias van der Heide

Storage Network Engineer

Inprove IT BV an EraStor member

@matthiasvdh on Twitter

Contents

• ZFS Intro

• Features

• N+1 ways

• Why use it in DevOps

• Where to get it

What is ZFS?

• “Zettabyte File System”

• Sun, now Oracle

• Thank Sun for OpenSolaris and releasing ZFS

ZFS Features

• Yes, it is a file system

• It is also volume manager

• Virtually unlimited capacity

• Transactional

• Multiple types of redundancy

• Virtually unlimited snapshots, clones

• Smart caching

• Versioned on-disk format

Copy-on-write

Checksums

NL 08 IN GB 00 00 00 05 55

ARC

• Adaptive Replacement Cache

• Combines MRU and MFU

• Uses ghost lists

• Consumes (a lot of) memory, extend to SSD

ZIL

• ZFS Intent Log

• In memory

• On the pool

• Optionally on a seperate log device (SSD)

Transaction Groups (TXG)

Open • Accept new writes

Quiescing • Wait for I/O to complete

Syncing • Write the data to disk

N+1 ways

• Designing your pool

• Testing your pool

• Tuning the filesystem

• Maintenance

N=1: Designing your pool

• Your performance and redundancy needs determine your pool design

• You can change it, only slightly, afterwards

• Consider a system with 4 HDDs

• Now what do I do?

Redundancy types

• A vdev is the building block of the zpool

• Everything is striped over vdevs

• Options: – Leaf (1)

– Mirror (2..n)

– RAID-Z1 (n+1)

– RAID-Z2 (n+2)

Options

• Stripe over all drives

• Stripe over mirrors

• RAID-Z1

• RAID-Z2

Fun options

• Dual RAID-Z1

• 4-way mirror

• HDD as slog device

N=2: Testing your pool

• Simple: time dd

• Iometer

• Vdbench

• FIO

N=3: Tuning your filesystem

• Block size

• Log bias

• Access time

• Sync on/off

• Caching on/off

N=4: Maintenance

• ZFS will tell you when stuff is broken

• Check data on all reads

• Regular scrubs

Scrubs

• Scans the pool for data integrity

• Checks checksums

• Restores data if needed

• Can take a while on large pools

Scrub

Scrub time

0,00

10,00

20,00

30,00

40,00

50,00

60,00

70,00

80,00

90,00

100,00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71

Estimate accuracy

Estimate accuracy

Resilvers

• When replacing a device

• When adding a device back after removal

Simple detach/attach

After being offline

New device

Future work on ZFS

• Feature flags enables development of features seperately

• Persistent L2ARC

• Larger block support

• Dedupe tables on SSD

Why use it in DevOps

• ZFS performs

• ZFS scales

– From your laptop to your workstation to your cloud

• ZFS replicates

– Using zfs send/receive you can deploy copies of your VMs to other hosts

– Keep simple backups of your repos and machines

• Use snapshots to boot into new versions of your operating system

Where to get it?

• FreeBSD

• Mac

• ZFS on Linux

• Illumos (EraStor OS)

• NexentaStor (based on OpenSolaris/Illumos)

• OpenZFS - http://open-zfs.org

Links

• Open ZFS Project http://open-zfs.org

• ARC Paper http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.13.5210

• NOSIG http://www.nosig.nl/

• Erastor http://www.erastor.eu