2017 feb-10 snowcamp.io-unikernels

Unikernels-Thenextbiglittlething?Snowcamp.io,Grenoble,France,Feb2017

MikeBright, @mjbright

OverviewWhatareUnikernels?

Whydoweneedthem?Domainsofapplication

Unikernelimplementations

Clean-slateorlegacyTooling

ContainersandUnikernels

Demo

Conclusions

Unikernels

@mjbright

Whythistalk?

@mjbright

Whythistalk?Curiosityabout

WhatwecanexpecttoseefromUnikernels(andDocker...)Whotheplayersare

@mjbright



1990's:Firstunikernels-ExokernelandNemesis(Univ.Cambr)

@mjbright




Jan2014:ACM-"Unikernels:RiseoftheVirtualLibraryOperatingSystem",AnilMadhavapeddyandDavidJ.Scott

@mjbright

https://queue.acm.org/detail.cfm?id=2566628





Mar2015:Linux.com-7UnikernelProjectstoTakeOnDockerin2015,LarsKurth

@mjbright


https://www.linux.com/news/7-unikernel-projects-take-docker-2015






Nov2015:DockerConEuropedemoCoolHack"Unikernels,MeetDocker!"

@mjbright



https://blog.docker.com/2016/01/unikernel/







Jan2016:DockerBlog-UnikernelSystemsJoinsDocker

@mjbright












Oct2016:TheNewStack-DebunkingUnikernels,IditLevineDell-EMC

@mjbright





http://thenewstack.io/utilizing-unikernels-within-internet-things/








Oct2016:TheNewStack-DebunkingUnikernels,IditLevineDell-EMC

In2017?

MirageOS3willbereleasedDockerConUSandEU

@mjbright





http://thenewstack.io/utilizing-unikernels-within-internet-things/

WhatareUnikernels?"LibraryOS"

WhatareUnikernels?"LibraryOS"ApplicationsbuiltwithonlytheOScomponentstheyactuallyrequire,e.g.TCP

Stack,DNS,DHCP,NAT,F/w,Diskaccess.

@mjbright

WhatareUnikernels?"LibraryOS"ApplicationsbuiltwithonlytheOScomponentstheyactuallyrequire,e.g.TCP

Stack,DNS,DHCP,NAT,F/w,Diskaccess.

Singleprocess(*)applications(nothreads,forkingormulti-user)(*)

Smallsize(fewlinesofcode)andveryfasttoboot

Smallattacksurface(potentiallysecure)

Highperformance-nocontextswitches!

Noshell

@mjbright

WhyareUnikernelsneeded?Thinkforamoment-WhatOSdoyourun?

Onwhathardware?

ModernOSprovideamazingbackwardscompatibilityandfeatures

@mjbright


Onwhathardware?


Butanappusesatinyfractionofthosefeaturesconsumingresources,increasingtheattacksurface(linuxkernel~25MLOC)

@mjbright


Onwhathardware?


Butanappusesatinyfractionofthosefeaturesconsumingresources,increasingtheattacksurface(linuxkernel~25MLOC)

Unikernelsprovideanalternative

Butaretheyapanacea?

@mjbright

The2familiesofUnikernels

UnikernelImplementations-2familiesThereare2mainclassesofUnikernels


Clean-SlateTheClean-Slateapproachemphasizessafetyandsecurity.Samelanguageforapplicationand"LibraryOS"components.

OneexampleofthisapproachisMirageOS(writteninOcaml)


Clean-SlateTheClean-Slateapproachemphasizessafetyandsecurity.Samelanguageforapplicationand"LibraryOS"components.

OneexampleofthisapproachisMirageOS(writteninOcaml)

LegacyTheLegacyapproachfavoursbackwardcompatibilityofexistingapplicationsbasedonPOSIX-compatibilities.

OneexampleofthisapproachisOSvforwhichthereareimplementatonsofTomcat,Jetty,Cassandra,OpenJDK

WewillseemoreUnikernelimplementationslater...

ApplicationdomainsforUnikernels

Inwhatdomainsmighttheybeused?CloudComputing

Small(kB/MB)immutableentitieswithfastboottimes(100'sms).

Possibilityofon-demandservers,µ-services

Potentiallygreatersecurity(<LOC)

@mjbright





NFV(NetworkFunctionalVirtualization)

CloudbutstricterrequirementsonresponsetimesDecouplesoftwarefromthehardware,decompose/chainfunctionsEricsson,NEC,Ciscoareactiveinthisdomain

@mjbright







IoT/Embedded/NetworkSwitches

Forlow-resource,potentiallysecureelements(baremetalorµ-vmm?)Buildupthe"app"insteadofstrippingdownthe"OS"

@mjbright







IoT/Embedded/NetworkSwitches

Forlow-resource,potentiallysecureelements(baremetalorµ-vmm?)Buildupthe"app"insteadofstrippingdownthe"OS"

HPC

Greaterperformancepossible(butmaybehardwork)@mjbright

Inwhatdomainsmighttheybeused?-NFV/SDNNano-servicesbootupin10-20msecondemandandareremovedwhentherequestcompletes.

PresentedbyEricssonResearch,Jan2016atSCALE14x.

UnikernelsmeetNFV

EricssonResearchBlog

Unikernels.orgBlog

https://www.youtube.com/watch?v=3jGClCBXuTg

https://www.ericsson.com/research-blog/sdn/unikernels-meet-nfv/

http://unikernel.org/blog/2016/unikernel-nfv-platform

IETFdraftonContainersforNFVexpiredJan2017Takenfrom:draft-natarajan-nfvrg-containers-for-nfv-03.txt

4.2.InstantiationTimes

Measurementoftimetobootimage,uptothe1stRSTpacket(toaSYNflood).

|--------------------------------------+|TechnologyType|Time(msecs)||--------------------------------------+|standardvm.xen|6500||standardvm.kvm|2988||Container|1711||tinyx.kvm|1081||tinyx.xen|431||unikernel.osv.kvm|330||unikernels.minios.xen|**31**|+-----------------------+--------------+

Note:

Theseunikernelsincludejustoneapplication-iperf.Tinyxis"TinyfiedLinux"running4.4.1kernel-busybox+sshd+iperfStandardVMisDebianrunning4.4.1kernel+iperfDockercontainerincludingiperf

https://www.ietf.org/archive/id/draft-natarajan-nfvrg-containers-for-nfv-03.txt

IETFdraftonContainersforNFVexpiredJan20174.3.Throughput

TCP/IPthroughputwasmeasuredusingiperffromguesttohost(toavoidphysicalmediumlimitations)

|---------------------------------------------------------------+|Technology|Throughput(Gb/s)|Throughput(Gb/s)||Type|Tx|Rx||-----------------------+-------------------+-------------------+|standardvm.xen|23.1|24.5||standardvm.kvm|20.1|38.9||Container|45.1|43.8||tinyx.kvm|21.5|37.9||tinyx.xen|28.6|24.9||unikernel.osv.kvm|**47.9**|**47.7**||unikernels.minios.xen|**49.5**|32.6|+-----------------------+-------------------+-------------------+

Note:

ThroughputdependsnotjustonguestefficiencyXenisoptimizedforTxbutnotRx(similartoClickOSexperience)

IETFdraftonContainersforNFVexpiredJan20174.4.RTT

Averageround-triptime(RTT)measuredfromanexternalserverusingapingflood.

+-----------------------+--------------+|TechnologyType|Time(msecs)||--------------------------------------+|standardvm.xen|34||standardvm.kvm|18||Container|**4**||tinyx.kvm|19||tinyx.xen|15||unikernel.osv.kvm|9||unikernels.minios.xen|**5**|+-----------------------+--------------+

IETFdraftonContainersforNFVexpiredJan20174.5.ImageSize

Wemeasureimagesizeusingthestandard"ls"tool.

+-----------------------+------------+|TechnologyType|Size(MBs)||------------------------------------+|standardvm.xen|913||standardvm.kvm|913||Container|61||tinyx.kvm|3.5||tinyx.xen|3.7||unikernel.osv.kvm|12||unikernels.minios.xen|**2**|+-----------------------+------------+

IETFdraftonContainersforNFVexpiredJan20174.6.MemoryUsage

"top"and"xl"(onXen)usedtomeasurememoryusage:

+-----------------------+-------------+|TechnologyType|Usage(MBs)||-------------------------------------+|standardvm.xen|112||standardvm.kvm|82||Container|**3.8**||tinyx.kvm|30||tinyx.xen|31||unikernel.osv.kvm|52||unikernels.minios.xen|8|+-----------------------+-------------+

Note:

OSvpre-allocatesmemory,e.gforbuffersBestresultisDockerasithasnoOSfunction

IETFdraftonContainersforNFVexpiredJan2017SowhatconclusionscanwedrawinthecaseofNFV?


Wellitdependsofcourse!!



Itdependsuponyourapplications',yourorganizations'criteria:

Serviceagility/elasticity:spinup/downtimesMemoryconsumptionSecurity/IsolationManagementframeworksCompatibilitywithapplications



Itdependsuponyourapplications',yourorganizations'criteria:

Serviceagility/elasticity:spinup/downtimesMemoryconsumptionSecurity/IsolationManagementframeworksCompatibilitywithapplications

ThesearestillearlydaysforUnikernelsforCloudComputing.

Hybridapproachesmaybeappropriate.

UnikernelImplementations...inmoredetail


TheClean-Slateapproachemphasizessafetyandsecurity.SamelanguageforapplicationandLibraryOScomponents.

MirageOS(Ocaml)HalVM(Haskell)LING(Erlang)


TheClean-Slateapproachemphasizessafetyandsecurity.SamelanguageforapplicationandLibraryOScomponents.

MirageOS(Ocaml)HalVM(Haskell)LING(Erlang)

TheLegacyapproachfavoursbackwardcompatibilityofexistingapplicationsbasedonPOSIX-compatibilities.

Manyapplicationshavebeenported

OSv(Tomcat,Jetty,Cassandra,OpenJDK,...)Rumprun(MySQL,PHP,Nginx)RuntimejsClive(Go)

UnikernelImplementationsTechnology Description

ClickOScnp.neclab.eu

Forembeddednetworkh/w.~5MBimages,boots<20ms,45μsdelay,100VMs=>10Gbps

Clivelsub.org

WritteninGo.Fordistributedandcloud.

DrawbridgeMS

Researchprototype.Picoprocess/containerwithminimalkernelAPIsurface,andWindowslibraryOS.

Graphenegraphene

Securing"multi-process"legacyapps-addsIPC.

HaLVMgalois.com

PortofGHC(GlasgowHaskellCompiler)suite.WriteappsinHaskelltorunonXen.

IncludeOSincludeos.org

ResearchprojectforC++codeonvirtualhardware.

LINGerlangonxen.org

Erlang/OTPrunsonXen.

MirageOSmirage.io

Clean-slatelibraryOSforsecure,high-perfnetworkapps.Morethan100MirageOSlibrariesplusOCamlecosystem.

OSvosv.ioCloudius

RunLinuxbinaries(w.limitations),supportsC/C++,JVM,Ruby,Node.js

RumprunFreeBSD-RunsPOSIXs/wonBMorVM(Xen).

@mjbright

http://cnp.neclab.eu/

http://lsub.org/ls/clive.html

https://www.microsoft.com/en-us/research/project/drawbridge/

http://graphene.cs.stonybrook.edu/

http://localhost:9002/2017-Feb-10_Snowcamp.io_Unikernels/galois.com

http://www.includeos.org/

http://erlangonxen.org/

https://mirage.io/

http://osv.io/

Clean-Slate

https://mirage.io/

OCaml-Based

MirageOS"LibraryOS"componentsarewritteninOcaml.

ML-derivedlanguagesarebestknownfortheirstatictypesystemsandtype-inferringcompilers.

OCamlunifiesfunctional,imperative,andobject-orientedprogrammingunderanML-liketypesystem.

OCamlhasextensivelibrariesavailable

(Unisonsyncutility)

Unikernelimplementations-MirageOS/Ocaml

https://mirage.io/

https://en.wikipedia.org/wiki/OCaml

Clean-Slate

https://mirage.io/

OCaml-Based

MirageOSUnikernelsarebasedontheMirage-OSUnikernelbase(OSlibrary).

ThemiragetoolisusedtobuildUnikernelsforvariousbackends:

XenHypervisor(PV)Unix(LinuxorOS/Xbinaries)Browser(viaOcaml->JScompiler!!)MirageOS3(/Solo5)willsupportkvm(/ukvm)EvenanexperimentalBMbackendforRaspberryPi

Unikernelimplementations-MirageOS-2

@mjbright

https://mirage.io/

Clean-Slate

https://mirage.io/

OCaml-Based

MirageOSUnikernelsarebasedontheMirage-OSUnikernelbase(OSlibrary).

ThemiragetoolisusedtobuildUnikernelsforvariousbackends:

XenHypervisor(PV)Unix(LinuxorOS/Xbinaries)Browser(viaOcaml->JScompiler!!)MirageOS3(/Solo5)willsupportkvm(/ukvm)EvenanexperimentalBMbackendforRaspberryPi

Buildingapplicationsforunixorxen

mirageconfigure-tunixmake./mir-console

mirageconfigure-txenmake****xencreate./mir-console.xen

Unikernelimplementations-MirageOS-2

@mjbright

https://mirage.io/

Clean-Slate

https://mirage.io/

BNCPinata:http://ownme.ipredator.se/

Networkingapplications

e.g.CyberChaff"falsenetworkhosts"

PayGarden,SeanGrove

"Babystepstounikernelsinproduction"

Toopainfultocreate/configureAMIimagesonAWSSolo5allowstocreateKVMimagesdeployableonGCE

Unikernelimplementations-MirageOS-UseCases

@mjbright

https://mirage.io/

https://www.youtube.com/watch?v=i9eu9e7gN0Q

Unik[EMC-Dell]:"TheUnikernelCompilationandDeploymentPlatform"(+imagehub)

rumprun:Python,Node.jsandGoOSv:Java,Node.js,CandC++IncludeOS:C++MirageOS:OCaml

Solo5[IBM]:Analternativeunikernel-baseforMirageOS

Providesqemu/KVMsupportforMirageOSIscurrentlybeingintegratedintoMirageOS3beta

ukvm[IBM]:AnalternativeVMMonitor

a"libraryhypervisor"

capstan:OSvbuildtool(+imagehub)

UnikernelTooling

@mjbright

https://github.com/emc-advanced-dev/unik

https://developer.ibm.com/open/openprojects/solo5-unikernel/

http://osv.io/capstan/

UnikernelToolingMirageOSjitsu:"Just-In-TimeSummoningofUnikernels"

ADNSserverthatstartsunikernelsondemand.

TestedwithMirageOSandRumprununikernels.

https://github.com/mirage/jitsu

@mjbright



UnikernelsandContainers:Myguess...SowhataboutContainers?...andwhydidDockerbuyUnikernelSystems?

@mjbright


UnikernelSystemsareinvolvedinMirageOS/Xen

Unikernelsalreadyusedasspecificfunctionsin"DockerforMac"

@mjbright




No-brainer:Providebuild/ship/runtoolsforUnikernels

build:toolstofacilitatebuildingUnikernelstest:runUnikernelsincontainerstofaciltatetestinghttps://github.com/mato/docker-unikernel-runner

ship:DockerregistryextendedtoprovideUnikernelimagesrun:DockerSwarmorchestratestasksincl.Unikernels

@mjbright

https://github.com/mato/docker-unikernel-runner




No-brainer:Providebuild/ship/runtoolsforUnikernels

build:toolstofacilitatebuildingUnikernelstest:runUnikernelsincontainerstofaciltatetestinghttps://github.com/mato/docker-unikernel-runner

ship:DockerregistryextendedtoprovideUnikernelimagesrun:DockerSwarmorchestratestasksincl.Unikernels

SecureContainerdeploymentsthroughhybridsolutions

Securefront-endsmadeofunikernelse.g.forOCamlMediaWiki(http2https,tlstunnel,...)

Containersforbackend

Surprises?...@mjbright@mjbright

https://github.com/mato/docker-unikernel-runner

Demo

DeferPanic-UnikernelIaaS-https://deferpanic.com/

runtime.js-Node.jsUnikernel-https://github.com/runtimejs/example-web-server

4unikerneldemos-LookMa,noOS!-https://github.com/technolo-g/lookma

https://deferpanic.com/

https://github.com/runtimejs/example-web-server

https://github.com/technolo-g/lookma

ConclusionsMuchworkneedstobedone

tomakethemeasytobuild,deploy,debugWewillseeeasiertousesolutions

WhateverDockerplantosurpriseuswithUnikwillfacilitatebuilding,deployingmultipletechnologiesSolo5willallowmixingoftechnologies

Severaldisparatetechnologiestoday

butsomeeffortstosynergize

Unikernelsareaninterestingcomplimentarytechnologytocontainers

Wecanexpecthybridsolutions

2017willbeaninterestingyearforUnikernels

@mjbright

ResourcesScoop.it

Unikernelswww.scoop.it/t/unikernels

YoutubePlaylist

youtube.com/.../unikernels

Wikipedia en.wikipedia.org/wiki/Unikernel

unikernels.org unikernels.org

mirageos.iomirageos.io

mirage.io/docs/papers

OReilly"Unikernels"

Freedownload

@unikernel @unikernel

github.com/ocamllabs ocamllabs

github.com/mirage MirageOS

@mjbright

http://www.scoop.it/t/unikernels

https://www.youtube.com/playlist?list=PLCDlZzVd_jn8heLw_Q10gOaEflLZKyf81

https://en.wikipedia.org/wiki/Unikernel

https://unikernels.org/

https://mirageos.io/

https://mirage.io/docs/papers

http://unikernel.org/blog/2016/unikernel-ebook

https://twitter.com/unikernel

https://github.com/ocamllabs

https://github.com/mirage

ThankyouQ&A

2017 feb-10 snowcamp.io-unikernels

Engineering