unikernels: in search of a killer app and a killer ecosystem

Unikernels: in search of a killer app and a killer ecosystem

RomanShaposhnik,DirectorofOpenSource@Pivotal,@rhatr

Unikernels•  “Unikernels:libraryopera?ngsystemsforthecloud”cameoutin2013

•  A“library”opera?ngsystem•  Akernelthatcanonlysupportoneprocess•  An‘executable’thatneedsvirtualiza?ontorun– Qemu,VB,VMWare,Xen,PublicCloud

Anykernels•  Programmingdisciplineforkernelcodereuse•  “TheDesignandImplementa?onoftheAnykernelandRumpKernels”byAnVKantee

•  Capabili?es– NetBSDfilesystemsasLinuxprocesses– User-spaceTCP/IPstack

•  Buildingblocksfor…anykernels

AnVKantee:Back-AlleyDoctorofNetBSD

Whatunikernelsareavailable•  MirageOS–  EmergedfromXen,OCamlspecific,research

•  Clive– Gospecific,Plan9lineage,research

•  RumpKernels(broughttoyoubyA.Kantee)–  Rumprununikernel,“sta?clinking”downtothekernel

•  OSv

UniK:UnikernelBuilds&Deployment•  Anopensourcetool–  heps://github.com/emc-advanced-dev/unik

•  AfamiliarDocker-likeCLI•  Abstractsawaydetailsofvirtualiza?onbackends•  IntegrateswithDocker&CloudFoundry•  PluggablesupportforUnikernels– OSv&rump

Interac?veIntermission:Disaggrega?ontrend

TheraiseofthePaaS:CloudFoundry

CloudFoundry

No,butseriously?

myApp $cfpush …

service#N

service#1

…

App#N

App#1

…

Cloud-na?veappsAKA12factor.net•  Codebase•  Dependencies•  Config•  Backingservices•  Build,deploy,run•  Statelessprocesses

•  Portbinding•  Concurrency•  Disposability•  Dev==prod•  Logs==streams•  Adminprocesses

> cd /path/to/my/app > tree . ├── README.md ├── app.groovy ├── application.properties ├── manifest.yml

> cat manifest.yml --- applications: - name: cf-spring memory: 512M instances: 3 random-route: true

> cf push my-app

Using manifest file /Users/verney/workspace/cf-sample-app-spring/manifest.yml

Creating app cf-spring in org pivot-jules / space test as jules@verne.io...

OK

Uploading cf-spring...

Uploading app files from: /Users/vereny/workspace/cf-sample-app-spring Uploading 1M, 44 files Done uploading OK

Runtime Container

Droplets

Staging Container

App Source Code

Buildpack

DropletFile System (‘Stack’)

> cf scale my-app –i 8

Anatomyofadroplet

Hardware

“Stuff”

[Java]VirtualMachine

μservicecode

Howarewedoingittoday?

JailedFS,net,etc.

Hardware

[Java]VirtualMachinelibFS,libC,libJVM

μservicecode

Applica?on-specificsta?clinking

OCI“runc”image

Common,sharedkernel

Isthereabeeerway?

vHardware

Hardware

[Java]VirtualMachinelibFS,libC,libJVM

μservicecode

Applica?on-specificsta?clinking

TinyVMimageAKAunikernel

Hardware-assistedvirtualiza?on

Imageby@GrahamDumpleton

OSvfromCloudiusSystems•  Aunikernelfor“POSIX”andmemorymanagedplaqorms(JVM,Go,Lua)

•  Anykernel’ish–  E.g.ZFS

•  RunsontopofKVM,Xen,VirtualBox,VMWare•  LookslikeanapptothehostOS•  Small,fastandeasytomanageatscale

OSvmanifesto•  Runexis?ngLinuxapplica?ons•  Runexis?ngLinuxapplica?onsfaster•  Makeboot?me~=exec?me•  ExploreAPIsbeyondPOSIX•  Leveragememorymanagedplaqorms(JVM,Go)•  Stayopen

What’sinside?

singleaddressspacein“kernelmode”

“kernelthreads”“userthreads”

diskZFS vir?oC++kernelcode

dynamiclinker

libjvm.soifconfig.so

TCP/IP

iface

Anythingitcan’tdo?•  A100%replacementforaLinuxkernel– Nofork()ing

•  Noprocessisola?on•  Theleastamountofdevicedriversever

Virtualiza?onvs.performance•  Network-intensiveapps:–  unmodified:25%gaininthroughput47%decreaseinlatency

–  non-POSIXAPIsuseforMemcached:290%increaseinperformance

•  Compute-intensiveapps:–  YMMV

VanJacabson’snetchannelssocket

TCP

IP

iface

socket

TCP

IP

iface

lock

lock

lock

Tradi?onalTCP/IPstack

appthreadkernel(IRQ)

send/recv

socket

TCP

IP

iface

channel

classifier

iface

lock

OSvTCP/IPstack

appthreadkernel(IRQ)

send/recv

MemorymanagementinUNIX

OSMemory

ProcessMemory

JVMHeap

ProcessMemory

JVMHeap

MemorymanagementinOSv

OSMemory

ProcessMemory

JVMHeap

JVMbalooning(nomore-Xmx)

JVMHeap

OSobject

TurbochargingJVMGC

object1 object2

TurbochargingJVMGC

object1 object2

TurbochargingJVMGC

object1 object2

CPUMMUassistedtrackingtable

Whyshoulditworkthis?me?•  Unikernels/exokernelsbackin’90•  JVM-on-bare-metal(Azul,BEA,etc.)backin‘00•  Thingstheydidn’thavebackthen– HW-assistedvirtualiza?on(KVM,XEN,etc.)–  Elas?cinfrastructureorientedarchitectures–  CloudFoundry(PaaS)

No,reallyweneedPaaS

No,reallyweneedPaaS

Elas?c,nextgenera?ondatacenter•  Commodity,rack-provisionedHardware•  JeOS(CoreOS,SmartOS,Xen+JeOS)–  aglorifieddevicedriver:anything2vir?o–  op?onally:awaytovirtualizea“dom0”kernel

•  Docker++asthenewELFformat– witheithernokernelorunikernelinside

•  CloudFoundrytorulethemall

FinallykillingDevOps•  Ops(IT)maintainsthebareOS•  Devsmaintaintheμservices•  PaaSmapsμservicestoimagesandorchestrates

FinallykillingDevOps•  Ops(IT)maintainsthebareOS•  Devsmaintaintheμservices•  PaaSmapsμservicestoimagesandorchestrates

Andonemorething…

Ques?ons?

By@cloud_opinionImaginenoplaqormsIwonderifyoucanNoneedforxAASAbrotherhoodofbaremetalImaginethereisnoVMIt'seasyifyoutryNohostbelowusAboveusonlyapps