DEPLOY,MANAGE&SCALEKUBERNETESWITH

RANCHER

BANGALORECONTAINERCONFERENCE7TH APRIL2017VISHALBIYANI

RANCHER&INFRACLOUD

VISHALBIYANICTO&FounderatinfraCloudtechnologies(www.infracloud.io )

2004Java,PLM,JSP,Servlets

2004-2009eMatrix PLM,J2EE,Database,architecture,Shellandwhatnot

2010- 2013:Spring,Maven,Jenkins,ElasticSearch,CloudFoundry,GoogleAppEngine,APIs,CI

2013:Puppet,Chef,Ansible,CD/CI,DevOpsCoach,Docker,APIMgmt,Microservices,Infraascode

Now:Containers,Kubernetes,Mesos,Salt,Scale,Distributed

https://twitter.com/vishal_biyanihttps://www.vishalbiyani.com

infraCloud isaRancherconsultingpartnerhttp://rancher.com/partners-index/

RancherhaspublishedaFREE eBookon

“ScalinganddeployingKubernetes”

http://info.rancher.com/deploying-scaling-kubernetes-ebook

Theaveragecompany

QUINTUPLESitsDockerusage

within9MONTHS1

Thereare460KDockerizedapps,a3100%GROWTHover2years2

Dockercontainershavebeen

downloadedmorethan4BILLION

times3

THEMOMENTUMOFCONTAINERADOPTIONISUNDENIABLE…

4

1 Datadog,June20162 Coscale,July20163 Docker,November2016

…BUTRUNNINGCONTAINERSINPRODUCTIONSTILLISN’TEASY

5

⬆ numbertools+⬆ change=

⬆complexityAppCatalog Helm,…Orchestration Compose,Kubernetes,Marathon,Scheduling Swarm,Kubernetes,Mesos,…Monitoring cAdvisor,Sysdig,Datadog,…

AccessControl LDAP,AD,GitHub,…Registry DockerHub,Quay.io,…Engine Docker,Rkt,…

Security Notary,Vault,…

Network VXLAN,IPSEC,HAProxy,…

Storage Ceph,Gluster,Swift,…

DistributedDB Etcd,Consul,MongoDB,…

⬆ density+⬇ lifespan=⬆volatility

ACOMPLETECONTAINERMANAGEMENTPLATFORMTHATMAKESITEASYTO…

6

INNOVATEWITHCONTAINERS

byempoweringdeveloperswithfastaccesstothelatesttools

SIMPLIFYAPPLICATIONDEVELOPMENT

withapowerful,yeteasytouseinterfaceandapplicationcatalog

RUNCONTAINERS

withthemostcompletesetofcontainerandinfrastructuremanagementcapabilities

Enterprise ready✔ Open platform for

innovating✔ Easy to use

interface✔ Multi-tenancy✔ Role based access✔ 24X7 support✔ And more….

DOYOUWANTTOMANAGEALLTHIS?

8

AppCatalog

Orchestration

Scheduling

Monitoring

AccessControl

Network

Storage

DistributedDB

Registry

Engine

Security

Helm,…

Compose,Kubernetes,Marathon,

Swarm,Kubernetes,Mesos,…

cAdvisor,Prometheus,Datadog,…

LDAP,AD,GitHub,…

Nexus,Artifactory,DTR…

Docker,runC,Rocket…

Notary,Vault,…

VXLAN,IPSEC,HAProxy,…

Ceph,Gluster,Swift,…

Etcd,Consul,MongoDB,…

…or this?

CHALLENGES:KUBERNETESONLYIMPLEMENTATIONS

• CreatingaKubernetesenvironmentthat iscustomizedtoDevOpsneeds• AutomatingthedeploymentofmultipleKubernetesclusters• ManagingthehealthofKubernetesclusters• AutomatingtheupgradeofKubernetesclusters• Deployingmultipleclustersonpremisesoracrossdisparatecloudproviders• Ensuringenterprisereadiness,includingaccessto 24×7support• Customizingthen repeatedlydeployingmultiplecombinationsofinfrastructureservices(e.g.storage,networking,DNS,loadbalancer)

• DeployingandautomatingupgradesforKubernetesadd-onssuchasDashboard,HelmandHeapster

RUNNINGCONTAINERSINPRODUCTIONISHARD,RANCHERMAKESITEASY

10

Develop Build Package Test Deploy/Upgrade Operate

DockerHub

GAININGSIGNIFICANTMOMENTUM

GAMarch2016

>20milliondownloads

5,000GitHubstars

100+enterprisecustomers

WORKSHOPAGENDA

• InfrastructureSide• HowtomodifyandmaintainmultipleKubernetesconfigurationseasily

• Configureseparatedata,cluster&workernodes

• ConfigureKubernetescloudproviders

• NFS&EBSconfiguration

• ConfiguringNetworktypes:IPSec&VXLan

• ApplicationSide• DeployapplicationswithHelmchart

• AutocreationofdisksandELBinaction

• CustomRegistry

• AutoScalingofhosts

• Hostsupgrades

SETUPWITHDIGICAL OCEAN

• UsethepromocodeDOBCC.Itwillgiveyou$15worth ofcreditsonDigitalOceanplatform.Pleasenotethefollowing:

• a)Youcansignupforanaccount@ https://cloud.digitalocean.com/registrations/new.Theabovepromocodewilladdcreditsonlytonew DigitalOcean accounts.

• b)Addingapaymentoption(credit/debitcardorPaypal)ispartofthesignupworkflow.Toverifytheauthenticityofthecard, sometimesthepaymentgatewaydoesanauthorizationchargeofaround$1butthischargegetsreversedimmediatelyafterthecardhas beenverified.

• c)Oncetheabovepromocodeisapplied,$15increditswillbeaddedtoyouraccountwhichcanbeusedforanythingonthe

ENVIRONMENTTEMPLATES

• Creatingandcustomizingtemplatesfordifferentrequirementsinanorganization

• Youcanhavedifferentstorage,networkingandotherrequirementsindifferentunits/projects

• YoumightwantatrueHAsetupforPre-prod/prodwhereasasimplesetupforDevelopmentenvironment

• Rancherenablesthiswithtemplatestacks- officialaswellascommunitysupported.

• Youcancreatemultipleenvironmenttemplatesandcanlaunchenvironmentsbasedontemplate

DEMO

RESILIENCYPLANES

• Objective:Achieveseparationbetweendata,Orchestrationandcomputenodes.

• Data- UsedbyEtcdtostorealldata• Recommendedminimum3

• Orchestrate- forKubernetes• Recommendedminimum2(ForHA)

• Compute- foractualworkload• 1ormore

• Youcannotchangeanodetypefromoneresiliencyplanetoother etcd=true

orchestrate=true

compute=true

1 2 3

1 2

1 N

CLOUDPROVIDERCONFIGURATION

• Kubernetescloudproviders:interfacetounderlyingcloudprovider

• Usefulforthingssuchas:Loadbalancer,Nodemanagement,Networksetc.

• Ranchercomesbuiltwithtwocloudproviders:Rancher&AWS• AWSprovidercanbeusedforELB,EBSandNodemanagement• RancherproviderisusefulforNodes&HAProxybasedloadbalancers

DNS- USINGDIGITALOCEAN

• EnablesquickandeasyintegrationwithDNS(AWSRoute53,DigitalOceanDNSetc.)

• EachserviceoftypeLoadBalancer- getstheloadbalancerautoprovisionedandDNSrecordcreated.

• DNSrecordiscustomizable

RANCHERNETWORKSERVICESVXLan(Overlay)

• Unencryptedtrafficbetweenhosts

• Goodifunderlyingnetworkissecure

• FasterConfigurableMTU

IPSec(Overlay)• Encryptedtrafficbetween

hosts,MTUconfigurable• Goodforpublicclouds• Relativelyslowdueto

encryptionoverhead

Morepluginscomingfor• Calico

• Weaveetc.

NetworkManager• InterfacetoCNIplugin&

respondstoadd/removecontainerevents

• Takescareofpartmapping(InitialCNIdidnothaveit)

RancherDNS• DNSServicewithincluster,

communicateswithupstreamDNS

• Providesservicediscoveryincluster

Rancher- Metadata• Metadataagentrunsonall

hosts• ProvidesServiceDiscovery

locally

NetworkingUnderthehood

Allthreecomponentsareopensource

RANCHERHEALTHCHECK

• Healthcheckstackisoneofinfrastructurestacks• LaunchedasasetofcontainersandutilizedHAProxyinternallytovalidatehealthofcontainers

• Containersarecheckedforhealthfrommultiplehealthcheckcontainers

• Ifevenoneofhealthcheckcontainersrespondpositiveonaservice-thenitisgood

• Ifallofhealthcheckcontainersrespondnegativeonaservice,thenitisassumeddown

PORTAINER

• WedeployedPortaineraspartofthestack,whichisasimpleUIforcontainers.

• TheDashboardisreachableathttp://rancher-server:8080/r/projects/1a5/portainer/ (JustopentheKubernetesdashboardUIandchangetheURL)

• Portainerissimpleutilityandshowscontainersonahost• ThisshowshoweasyitcanbetodeploycustomutilitiesstacksontopofRancher

POWERFULCOMPOSITION

• Everystackisarancher-compose+docker-compose• Youcancustomcreatecompletestack,uploadandhaveanewenvironmenttemplate

AWSCLOUDPROVIDERBASEDENVIRONMENT

• CreateaAWScloudproviderbasedKubernetesenvironmenttemplateandanenvironment

• CreateRolesforinstanceprofilesfortheKubernetesmaster&agent- thisenablestheinstancestoattachdisksorcreateELBandsoon

• Create4hosts- onemaster&3nodesandinstalldockeronthem

• AddthehostsmanuallytotheRancherenvironment• Seetheenvironmentbuildup

WALKTHROUGHOFINFRASTACKS

• Healthcheckstackforhealthchecks

• IPSecnetworkingforencryptedoverlaytraffic

• IngresscontrollerforLBandIngressmanagement

• SupportingNetworkservices- NWmanagerandmetadata

• Portainerasautility

• Schedulerframeworkforadditionallyscheduling

WALKTHROUGHOFINFRASTACKS

• Kubernetesstackforallcorecomponents

• Controllermanagercontrolnodes,endpointsetc.

• Kubernetes- APIServer• Ingresscontrollerforingress&LBmanagement

• CoreScheduler

SAMPLEAPPLICATIONDEPLOYMENT

• WewilluseaHelmcharttodeployWordPressstack- whichcontainstheWordPressapp&MySQLDB

• MySQLDBneedsapersistentdisk- whichbeautoprovisionedforus.

• WealsoneedaLoadBalancer- whichwillbeautocreated.• Wewon’tuseDNSlikewedidinlastexample,butthatispossibletoo.

HELMONMYMACHINE

• Configure~/.kube/configfile- verifywithkubectl• ‘helminit’- initializesalldirectoriesandstandardrepo• helmsearchWordPress• helminstall--namebcc-releasestable/wordpress

OH,WAIT,WHATISHELM?

• HelmisapackagemanagerforKubernetes• Tiller- RepoServer• Chart- apackage• HelmistheclientforTiller• Chartsareinarepo(TypicallysomeGitrepo)

• Achart- issetofmanifests• Thevaluescanbedefaultedtooroverriddenasinputfromuser• Achartisreleasedasareleasesothatitcanbetracked.

ISWORDPRESSDEPLOYED?

• DeploymentsforWordPresscreated

• Servicescreated• Volumesautocreated• ELBautocreated

MOREVALIDATIONS

• PV&PVCcreatedusingthedefaultstorageclass

• Andwecanreachourblog:

WordPresshelmchart- codewalkthrough

HOSTEVACUATION

• Youwanttoupgradeahostforsomesecuritypatchesorsomechange

• Butwithoutdisruptingnormaloperations• Evacuationhelpsyoureschedulepodstootherhosts,gracefully!

CUSTOMREGISTRYADDITION

• YoucanuseDockerhuboranyprivateregistry• Hostdockercfgisautopopulated- soimagescanbepulledfromthoseregistries

RECEIVERHOOKS

• Likewebhooks- canbeusedtoinvokeactionsinRancher

• Canbetiedtolet’ssaymonitoringsystem

• Possibletoachieveauto- hostscaling&serviceupgradeasoftoday.

• Moreactions&“Kind”ofhookscomingsoon

ANDITCOMESWITHANAPI

• RancherhasacomprehensiveAPI- andallactionscanbedoneviaAPI

• APIiswelldocumented,hasinbrowseraccessibilityandisexhaustive

• RancheralsocomeswithaCLI

REFERENCE/EXTRA

INGRESS:LOADBALANCERS

• Foraningressyouneedaloadbalancer.

• Ranchercreates/updates/managesRancherloadbalancersbasedoningresslifecycle,usingrancheringresscontroller.

• Thisalsomakesusageofingresseasieroutsideacloudprovider.

• Rancherloadbalancerssupport

• Host/pathbasedrouting

• TLS

• Advancedtargetingandschedulingofloadbalancers.