jim harris principal software engineer data center groupspdk+state+of+… · ocf ocf cache spdk...

Jim Harris

Principal Software Engineer

Data Center Group

SPDK, PMDK & VTune™ Amplifier Summit 2

By Edmund Garman from Salem, Oregon, USA - Waikiki, Honolulu PanoramaCC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=39574339

SPDK, PMDK & VTune™ Amplifier Summit 3

Agenda• Community Growth

• Feature Growth

• Looking Forward

SPDK, PMDK & VTune™ Amplifier Summit 4

SPDK Patch Commit StatisticsVersion Range Total

CommitsNon-Intel Committers

Non-IntelCommit %

v1.0 .. v16.12 1360 13 2.6%

v16.12 .. v18.01 2696 31 4.6%

v18.01 .. v19.01 3083 51 14.7%

SPDK, PMDK & VTune™ Amplifier Summit 5

SPDK Patch Commit StatisticsVersion Range Total

CommitsNon-Intel Committers

Non-IntelCommit %

v1.0 .. v16.12 1360 13 2.6%

v16.12 .. v18.01 2696 31 4.6%

v18.01 .. v19.01 3083 51 14.7%

2018+65% Non-Intel Committers

3x Non-Intel Commits

SPDK, PMDK & VTune™ Amplifier Summit 6

Community Growth

+30% YoY+218% since Jan’17

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

QoS

Linux nbdRDMA

RDMA

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

7

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

QoS

Linux nbdRDMA

TCP

RDMA

TCP

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

8

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 9

NVMe/TCPNVMe TP ratified November 2018

SPDK added TCP transport for

NVMe driver

NVMe-oF target

Supports alternative TCP stack implementations

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

10

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 11

CryptoHardware Accelerators

• Intel Quickassist

Optimized Software

• ISA-L

DPDKFramework

NVMe bdev

NVMe Controller

Namespace

Logical Volume Store

lvolbdev

lvolbdev

lvolbdev

cryptobdev

cryptobdev

cryptobdevKEYKEYKEY

cryptodev

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

12

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 13

CinderProvision ephemeral storage using SPDK

• NVMe-oF target

• Logical volumes

• Dynamic configuration via JSON-RPC

Kubernetes integration in progress

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

14

Open ChannelFTL

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 15

Open Channel BLOCK FTLHost FTL enabling smart data placement

• Based on OC2.0 specification

Block FTL support added to bdev nvme module

Long term goal: Zoned Namespace API

With ZNS/OC adapters

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

Recently Added Features

iSCSI

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

16

Open ChannelFTL

OCF

OCFCache

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 17

Open Cas FrameworkBlock Caching Library

Open Sourced in Q4’18

SPDK integration added v19.01

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

Recently Added Features

iSCSI

spdk-cli

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

18

Open ChannelFTL

OCF

OCFCache

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 19

Spdk-cli

SPDK, PMDK & VTune™ Amplifier Summit 20

Looking ForwardUsage Models

Rack Scale Architecture

Smart NICs

New SPDK Features

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

SPDK Architecture

iSCSI

spdk-cli

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

21

CompressOpen Channel

FTL

OCF

OCFCache

SPDK 18.04

SPDK, PMDK & VTune™ Amplifier Summit 22

Block compression

22

DPDKFramework

compressbdev

compressdev

PMDK

lvolbdev

DPDK Framework for compression algorithms

Accelerators or Optimized SW

PMDK for persistent memory access

For compression metadata

SPDK logical volumes for storage

For realizing capacity savings

SPDK, PMDK & VTune™ Amplifier Summit

Drivers

StorageServices

StorageProtocols

iSCSI Target

NVMe-oF*Target

SCSI

vhost-scsiTarget

NVMe

NVMe Devices

Blobstore

FabricsInitiator

Intel® QuickDataTechnology

Driver

Block Device Abstraction (bdev)

Ceph RBD

Linux AIO

Logical Volumes

3rd Party

NVMe

PCIe

In Progress

vhost-blkTarget

BlobFS

Integration

RocksDB

Ceph

Tools

fio

PMDKblk

virtio(scsi/blk)

VPP TCP/IP

QEMU

Cinder

QoS

Linux nbdRDMA

Crypto

TCP

RDMA

TCP

SPDK Architecture

iSCSI

spdk-cli

vhost-nvmeTarget

virtio

virtio-PCIe

vhost-user

nvme-cli

Added since 18.04

23

CompressOpen Channel

FTL

OCF

OCFCache

SPDK 18.04

io_uring

SPDK, PMDK & VTune™ Amplifier Summit 24

Io_uring

Linux logo by: lewing@isc.tamu.edu Larry Ewing and The GIMP

New Linux kernel interface for asynchronous I/O

Targeted for Linux 5.1

Significant efficiency improvements compared to Linux AIO

SPDK io_uring bdev module

SPDK, PMDK & Vtune™ Summit 25

Future WorkSmart Scheduling

Block Indirection

Zoned Namespaces

Replication and Parity

Even better performance!

SPDK, PMDK & Vtune™ Summit 26

Call to ActionTake advantage of talks and demos to learn more about SPDK

and PMDK and VTuneTM Amplifier

Join the SPDK Community

SPDK, PMDK & Vtune™ Summit 28

Backup