container-based virtualization: docker · corso di sistemi distribuiti e cloud computing ... server...

20
Container-based virtualization: Docker Università degli Studi di Roma Tor VergataDipartimento di Ingegneria Civile e Ingegneria Informatica Corso di Sistemi Distribuiti e Cloud Computing A.A. 2018/19 Valeria Cardellini Case study: Docker Lightweight, open and secure container-based virtualization Containers include the application and all of its dependencies, but share the kernel with other containers Containers run as an isolated process in userspace on the host operating system Containers are also not tied to any specific infrastructure Valeria Cardellini - SDCC 2018/19 1

Upload: donhi

Post on 28-May-2019

215 views

Category:

Documents


0 download

TRANSCRIPT

Container-based virtualization: Docker

Università degli Studi di Roma “Tor Vergata” Dipartimento di Ingegneria Civile e Ingegneria Informatica

Corso di Sistemi Distribuiti e Cloud Computing A.A. 2018/19

Valeria Cardellini

Case study: Docker •  Lightweight, open and secure container-based

virtualization –  Containers include the application and all of its

dependencies, but share the kernel with other containers –  Containers run as an isolated process in userspace on the

host operating system –  Containers are also not tied to any specific infrastructure

Valeria Cardellini - SDCC 2018/19

1

Docker internals •  Docker is written in Go language

•  With respect to other OS-level virtualization solutions, Docker is a higher-level platform that exploits Linux kernel mechanisms such as cgroups and namespaces –  First versions based on LXC –  Then based on its own libcontainer runtime that uses Linux

kernel namespaces and cgroups directly –  libcontainer (now included in opencontainers/runc): cross-

system abstraction layer aimed to support a wide range of isolation technologies

•  Dockers adds to LXC –  Portable deployment across machines –  Versioning, i.e., git-like capabilities –  Component reuse –  Shared libraries, see Docker Hub hub.docker.com

Valeria Cardellini - SDCC 2018/19

2

Docker engine

•  Docker Engine: client-server application composed by: –  A server, called daemon

process –  A REST API which specifies

interfaces that programs can use to control and interact with the daemon

–  A command line interface (CLI) client

Valeria Cardellini - SDCC 2018/19

3

See https://docs.docker.com/engine/docker-overview/

Docker architecture •  Docker uses a client-server architecture

–  The Docker client talks to the Docker daemon, which builds, runs, and distributes Docker containers

–  Client and daemon communicate via sockets or REST API

Valeria Cardellini - SDCC 2018/19

4

Docker image •  Read-only template containing instructions for creating a

Docker container –  Described in text file called Dockerfile, with simple, well-defined

syntax –  The Build component of Docker –  Enables the distribution of applications with their runtime

environment •  Incorporates all the dependencies and configuration necessary for

it to run, eliminating the need to install packages and troubleshoot –  Target machine must be Docker-enabled

•  The Docker Image –  Can be pulled and pushed towards a registry –  Image names have the form [registry/][user/]name[:tag] –  Default for tag is latest

Valeria Cardellini - SDCC 2018/19

5

Docker image: Dockerfile •  Images can be created from a Dockerfile and a context:

–  Dockerfile: instructions to assemble the image –  Context: set of files (e.g., application, libraries) –  Often, an image is based on another image (e.g., ubuntu)

•  Example of a Dockerfile

6 Valeria Cardellini - SDCC 2018/19

Docker image: build

•  Build an image from a Dockerfile

$dockerbuild -t hello-world .

Valeria Cardellini - SDCC 2018/19 7

$dockerrun[OPTIONS]PATH|URL|-

Docker image: layers •  Layered image

–  Each image consists of a series of layers –  Docker uses union file systems to combine these layers into a

single unified view •  Layers are stacked on top of each other to form a base for a

container’s root file system •  Based on the copy-on-write (COW) principle

Valeria Cardellini - SDCC 2018/19

8

Docker image: layers •  Layering pros

-  Enable layer sharing and reuse, installing common layers only once and saving bandwidth and storage space

-  Manage dependencies and separate concerns -  Facilitate software specializations -  See https://docs.docker.com/storage/storagedriver/

Valeria Cardellini - SDCC 2018/19

9

Docker image: storage

•  Containers should be stateless. Ideally: –  Very little data is written to the container’s writable layer –  Data should be written on Docker volumes –  Nevertheless: some workloads require to write data to the

container’s writable layer

•  The storage driver controls how images and containers are stored and managed on the Docker host

•  Multiple choices for the storage driver -  Including AuFS and OverlayFS (both operate at file level), Device

Mapper, Btrfs and zfs (that operate at block level) -  Storage driver’s choice can affect the performance of the

containerized applications -  See https://dockr.ly/2FstUe6

Valeria Cardellini - SDCC 2018/19

10

Docker container and registry •  Docker container: runnable instance of a Docker

image –  Run, start, stop, move, or delete a container using Docker API

or CLI commands –  The Run component of Docker

•  Docker registry: stateless server-side application that stores and lets you distribute Docker images -  Provides an open library of images -  The Distribute component of Docker -  Docker-hosted registries: Docker Hub, Docker Store (open

source and enterprise verified images) Valeria Cardellini - SDCC 2018/19

11

-  Docker containers are stateless: when a container is deleted, any data written that is not stored in a data volume is deleted along with the container

Docker: run command

•  When you run a container whose image is not yet installed but is available on Docker Hub

Valeria Cardellini - SDCC 2018/19 12

Courtesy of “Docker in Action” by J. Nickoloff

State transitions of Docker containers

Valeria Cardellini - SDCC 2018/19 13

Courtesy of “Docker in Action” by J. Nickoloff

Commands: Docker info

•  Obtain detailed info on your Docker installation $dockerinfo

E.g., to know the used storage driver (e.g., AuFS)

14

Valeria Cardellini - SDCC 2018/19

Commands: image handling

•  List images on host (i.e., local repository) $dockerimages or $dockerimagels-  To list every image, including intermediate image layers: $dockerimages–a or $dockerimagels–a–  Options to list images by name and tag, to list image digests

(sha256), to filter images, to format the output, e.g., $dockerimages--filterreference=ubuntu

•  Inspect an image –  Display detailed information, including image layers $docker[image]inspectimageid

•  Remove an image $dockerrmiimageid or $dockerimagermimageid

15 Valeria Cardellini - SDCC 2018/19

Can also use imagenameinstead of imageid

Command: run

•  Most common options --nameassign a name to the container –ddetached mode (in background) –iinteractive (keep STDIN open even if not attached) -tallocate a pseudo-tty--exposeexpose a range of ports inside the container -ppublish a container's port or a range of ports to the host –vbind and mount a volume –eset environment variables --linkadd link to other containers

•  The “Hello World” container $dockerrunalpine/bin/echo'Helloworld'

-  Alpine: lightweight Linux distro with reduced image size

16

$dockerrun[OPTIONS]IMAGE[COMMAND][ARGS]

Valeria Cardellini - SDCC 2018/19

Commands: containers management •  List containers

–  Only running containers: $dockerps•  Alternatively,$dockercontainerls

–  All containers (even stopped or killed containers): $dockerps-a

•  Container lifecycle –  Stop running container $dockerstopcontainerid

–  Start stopped container $dockerstartcontainerid

–  Kill running container $dockerkillcontainerid

–  Remove container (need to stop the container before attempting removal) $dockerrmcontainerid

17

Can also use containernameinstead of containerid

Valeria Cardellini - SDCC 2018/19

Commands: containers management •  Inspect a container

–  Most detailed view of the environment in which a container was launched

$dockerinspectcontainerid

•  Copy files from and to docker container $dockercpcontainerid:pathlocalpath$dockercplocalpathcontainerid:path

18

Valeria Cardellini - SDCC 2018/19

Examples of using Docker •  Run a nginx Web server inside a container

-  Also bind the container to a specific port $dockerrun–d–p80:80--namewebnginx

•  Send HTTP request through Web browser -  First retrieve the hostname of the host machine

•  Send HTTP request through an interactive container using the Docker internal network$dockerrun-i-t--linkweb:web--nameweb_testbusybox

/#wget-O-http://web:80/

/#exit

•  To not use --link, let us define a bridge network $dockernetworkcreatemy_net

$dockerrun-d–p80:80--nameweb--net=my_netnginx

$dockerrun–i-t--net=my-net--nameweb_testbusybox

/#... Valeria Cardellini - SDCC 2018/19

19

--link: legacy flag to manually create links between the containers

wget: -OFILESavetoFILE('-'forstdout)

Examples of using Docker •  Sending an HTTP request through an Alpine Linux

container with curl installed and set as entrypoint$dockerrun--rmbyrnedo/alpine-curlhttp://…

•  Checking the logs of the container $dockerlogscontainerid_or_name

Valeria Cardellini - SDCC 2018/19

20

Examples of using Docker •  Running Apache web server with minimal index page

–  Define container image with Dockerfile •  Define image starting from Ubuntu, install and configure Apache •  Incoming port set to 80 using EXPOSE instruction

Vale

ria C

arde

llini

- S

DC

C 2

018/

19

21

FROMubuntu

#Installdependencies

RUNapt-getupdate

RUNapt-get-yinstallapache2

#Installapacheandwritehelloworldmessage

RUNecho'HelloWorld!'>/var/www/html/index.html

#Configureapache

RUNecho'./etc/apache2/envvars'>/root/run_apache.sh

RUNecho'mkdir-p/var/run/apache2'>>/root/run_apache.sh

RUNecho'mkdir-p/var/lock/apache2'>>/root/run_apache.sh

RUNecho'/usr/sbin/apache2-DFOREGROUND'>>/root/run_apache.sh

RUNchmod755/root/run_apache.sh

EXPOSE80

CMD/root/run_apache.sh

Examples of using Docker

–  Build container image from Dockerfile $dockerbuild-thello-apache.

–  Run container and bind $dockerrun-d-p80:80hello-apache

•  Option –p: publish container port (80) to host port (80)

Valeria Cardellini - SDCC 2018/19 22

Examples of using Docker

23

Valeria Cardellini - SDCC 2018/19

$dockerps

CONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES

2b2b39d05fe0hello_world"/bin/sh-c/root/ru…"3secondsagoUp2seconds0.0.0.0:80->80/tcppedantic_austin

•  Stopping and removing a container $dockerps$dockerstopcontainerid_or_name$dockerps-a$dockerrmcontainerid_or_name

•  Stopping and removing all container $dockerstop$(dockerps-a–q)$dockerrm$(dockerps-a–q)

Examples of using Docker

24

Valeria Cardellini - SDCC 2018/19

•  Running a Python web app in Docker –  See https://docs.docker.com/get-started/part2/ –  Define container image with Dockerfile

•  Define image with Python runtime and Python app •  Incoming port set to 80 using EXPOSE instruction

–  Write app code in Python using Flask and Redis packages and create file requirements.txt to specify those packages needed by app

•  Redis: in-memory data store •  Redis is used to keep the counter of the number of visits to web app

–  Build container image $dockerbuild-tfriendlyhello.

–  Run container and bind $dockerrun-d-p4000:80friendlyhello

•  Option –p: publish container port (80) to host port (4000)

Multi-container Docker applications

•  How to run multi-container Docker apps? •  Docker Compose

–  Deployment on single host

•  Docker Swarm –  The native orchestration tool for Docker –  Deployment on multiple hosts

•  Kubernetes

Valeria Cardellini - SDCC 2018/19 25

Docker Compose •  To coordinate the execution of multiple containers,

we can use Docker Compose –  See https://docs.docker.com/compose/

•  Docker Compose –  Not bundled within Docker installation (on Linux) –  Can be installed following the official Docker documentation

https://docs.docker.com/compose/install/

•  Allows to easily express the containers to be instantiated at once, and the relations among them

•  Runs the composition on a single machine (i.e., single Docker engine)

•  Use Docker Swarm to deploy containers on multiple nodes

26

Valeria Cardellini - SDCC 2018/19

Docker Compose •  We specify how to compose containers in a easy-to-

read file, by default named docker-compose.yml

•  To start the Docker composition (background -d):

•  To stop the Docker composition:

•  By default, Docker Compose looks for the docker-compose.ymlfile in the current working directory

–  Change file using -fflag

27

$docker-composeup-d

$docker-composedown

Valeria Cardellini - SDCC 2018/19

Docker Compose •  Different versions of the Docker compose file format

–  Latest: version 3 is supported from Docker Compose 1.13

28

Docker compose file format: https://docs.docker.com/compose/compose-file/ Valeria Cardellini - SDCC 2018/19

Docker Compose: example •  Simple Python web app running on Docker Compose

–  Two containers: Python web app and Redis –  Use Flask framework and maintain a hit counter in Redis –  See https://docs.docker.com/compose/gettingstarted/

•  Steps: –  Write Python app –  Define Python container image with Dockerfile –  Define services in docker-compose.yml file

•  Two services: web (image defined by Dockerfile) and redis (image pulled from Docker Hub)

–  Build and run your app with Compose $docker-composeup–d

–  Send HTTP requests using curl (now counter is increased) –  Stop Compose

$docker-composedown Valeria Cardellini - SDCC 2018/19

29

Docker: Swarm mode •  Docker includes the swarm mode for natively

managing a cluster of Docker Engines, which is called swarm –  See https://docs.docker.com/engine/swarm/

•  Task: running container part of a swarm service •  Basic features of swarm mode:

–  Scaling: allows to declare the number of tasks for each service

–  State reconciliation: Swarm monitors the cluster state and reconciles any differences w.r.t. the expressed desired state

–  Multi-host networking: allows to specify an overlay network among services

–  Load balancing: allows to expose the ports for services to an external load balancer. Internally, the swarm lets you specify how to distribute service containers among nodes

30

Valeria Cardellini - SDCC 2018/19

Docker: Swarm mode •  A swarm consists of multiple Docker hosts which run in swarm

mode •  Node: an instance of the Docker engine

–  Manager node dispatches tasks to worker nodes –  Worker nodes receive and execute tasks

•  Load balancing –  The swarm manager can automatically assign the service a

(configurable) PublishedPort –  External components can access the service on the PublishedPort.

All nodes in the swarm route ingress connections to a running task

31 Valeria Cardellini - SDCC 2018/19

Commands: Swarm cluster •  Create a swarm: manager node

•  Create a swarm: worker node

•  Inspect status

32 Valeria Cardellini - SDCC 2018/19

$dockerswarminit--advertise-addr<MANAGER-IP>

Swarminitialized:currentnode(<nodeid>)isnowamanager.

Toaddaworkertothisswarm,runthefollowingcommand:

dockerswarmjoin--token<token><manager-ip>:port

$dockerswarmjoin--token<token><manager-ip>:port

$dockerinfo

$dockernodels

ID HOSTNAME STATUS AVAILABILITY MANAGERSTATUS<nodeid>* controller Ready Active Leader<nodeid> storage Ready Active

Commands: Swarm cluster •  Leave the swarm

If the node is a manager node, you will receive a warning about maintaining the quorum. To override the warning, pass the --forceflag

•  After a node leaves the swarm, you can run dockernoderm command on a manager node to remove the node from the node list

33 Valeria Cardellini - SDCC 2018/19

$dockerswarmleave

$dockernodermnode-id

Commands: manage services •  Deploy a service to the swarm (from the manager node)

•  List running services

34 Valeria Cardellini - SDCC 2018/19

$dockerservicecreate-d--replicas1\

--namehelloworldalpinepingdocker.com

$dockerservicels

ID NAME MODE REPLICAS IMAGE PORTS<serviceid> helloworld replicated 1/1 alpine:latest

Commands: manage services •  Inspect the service

•  Inspect the container

35 Valeria Cardellini - SDCC 2018/19

$dockerps<cont.id1>

#Managernode

CONTAINERID IMAGE COMMAND CREATED STATUS ...NAMES<cont.id1> alpine:latest "pingdocker.com"2minago Up2minhelloworld.1.iuk1sj…

#Workernode

CONTAINERID IMAGE COMMAND CREATED STATUS ...NAMES<cont.id2> alpine:latest "pingdocker.com"2minago Up2minhelloworld.2.skfos4…

$dockerserviceinspect--pretty<SERVICE-ID>

$dockerserviceps<SERVICE-ID>

ID NAME IMAGE NODE DESIREDSTCURRENTST ERROR PORTS<cont.id1>helloworld.1 alpine:latest controller Running Running…<cont.id2>helloworld.2 alpine:latest storage Running Running…

Commands: manage services •  Scale the service

The swarm manager will automatically enact the updates

•  Apply rolling updates to a service

•  Roll back an update

•  Remove a service

36 Valeria Cardellini - SDCC 2018/19

$dockerserviceupdate--imageredis:3.0.7redis

$dockerserviceupdate--replicas2helloworld

$dockerservicerm<SERVICE-ID>

$dockerservicerollback[OPTIONS]<SERVICE-ID>

$dockerservicescale<SERVICE-ID>=<NUMBER-OF-TASKS>

Docker Swarm: example •  Let’s use Docker Swarm to scale Python web app on the same

host and load balance the traffic among the container replicas –  See https://docs.docker.com/get-started/part3/

•  Steps: –  Define services in docker-compose.yml file

•  Only one service called web •  Set replication degree to 5 •  Set resource limits to be used by each single replica •  Set restart policy to on-failure so that a failed container is

automatically restarted •  Docker supports 4 restart policies

»  no»  on-failure»  unless-stopped»  always

•  Instruct web’s containers to share port 80 via a load-balanced network called webnet

•  Define webnet network (load-balanced overlay network) Vale

ria C

arde

llini

- S

DC

C 2

018/

19

37

Docker Swarm: example –  Start the swarm $dockerswarminit

–  Deploy the stack $dockerstackdeploy-cdocker-compose.ymlgetstartedlab

–  Send multiple requests to Python web app and see how they are served by different replicas chosen in a round-robin fashion

–  Take down the app and the swarm $dockerstackrmgetstartedlab

$dockerswarmleave--force

38 Valeria Cardellini - SDCC 2018/19