Lecture 2virtualization

4.9.2019 ssss 1

Content

• General course related matters• Recap of DevOps• Virtualization – what and why?• Virtualizations techniques• Full/hypervisor virtualization• Containers• Containers in practice: Docker• Summary

4.9.2019 ssss 2

General course related matters

4.9.2019 ssss 3

Numbers

• Sign-ups in ROCK: 96• Sign-ups in plus: 71• With UTA ID: 13 !

• Responses to initial questionary: 53• Final number <50?

6.9.2019 ssss 4

DevOps

4.9.2019 ssss 5

Virtualization – what and why?

4.9.2019 ssss 6

Use case 1: run ”foreign” software

4.9.2019 ssss 7

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application VM tech

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

Use case 2: isolate

4.9.2019 ssss 8

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

VM technology

Use case 3: scale

4.9.2019 ssss 9

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

VM technology

Customer 1 Customer 2

Levels of virtualization

• Hardware virtualization• Operating system virtualization• Desktop virtualization• Application virtualization• Network virtualization

4.9.2019 ssss 10

Network virtualization

• Network, its HW and protocols, is simulated with software so that it looks like a different network to applications• Different from OSI layer models

• Is VPN a virtual network?

4.9.2019 ssss 11

Application virtualization

• Applications are compiled to machine-independent ”machine ” code• Applications are run with a virtual machine• Benefits• Same code can be run on different CPUs• Increased safety. Why?

• Problems• Performance

4.9.2019 ssss 12

Example: java

09.11.2015 TIE-02300/K.Systä 13

Virtual machine

Hardware

Bytecode

Compiler

class

Programminglanguage

X = Integer.parseInt("1234");

Libraries (APIt)

Program

IDE

Java security

09.11.2015 TIE-02300/K.Systä 14

Virtual machine

Hardware

Bytecode

Compiler

class

Programminglanguage

X = Integer.parseInt("1234");

Libraries (APIt)

Program

IDE

$ javap -c testCompiled from "test.java"class test {int X;

test();Code:

0: aload_0 // this1: invokespecial #1 // Method java/lang/Object."<init>":()V4: return

void foo();Code:

0: aload_01: ldc #2 // String 12343: invokestatic #3 // Method java/lang/Integer.parseInt:(Ljava/lang/String;)I6: putfield #4 // Field X:I9: return

}4.9.2019 ssss 15

Desktop virtualization

• When you run the ”linux-desktop” (linux-desktop.cc.tut.fi) on your windows machine

• Piirretään kuva luennolla (eri mahdollisia tapoja)

4.9.2019 ssss 16

Hardware virtualization

• Hypervisor• HW or SW based

4.9.2019 ssss 17

VM tech

Machine instructions

Operating system

Run-time / libraries

Application

Machine instructions

Operating system

Run-time / libraries

Application

5.9.2019 ssss 18

Mika Kaaretkoski,Master thesis, 2018

F. Rodrguez-Haro et al., “A summary of virtualization techniques,” Pro-cedia Technology, vol. 3, pp. 267 { 272, 2012, the 2012 IberoamericanConference on Electronics Engineering and Computer Science. [Online]. Accessed:4.8.2018 Available: http://www:sciencedirect:com/science/article/pii/S2212017312002587

Guest

Host

Virtual Machine Mngr(Hypervisor)

Type-1, native or bare-metal hypervisors

• Examples of virtualization that uses hardware assisted are Kernel-based Virtual Machine (KVM), VirtualBox, Xen, Hyper-V, and VMware products• Hardware-assisted = • Pros & cons

+ efficient+ provides service to all the guests in equivalent way- requires cleaning of the existing system clean- I/O device drivers must be available for installation in the VMM

5.9.2019 ssss 19

Different approached(source of pictures: VMWARE)

5.9.2019 ssss 20

Type-2 or hosted hypervisors

• A guest operating system runs as a process on the host.

• VMware Workstation, VMware Player, VirtualBox, Parallels Desktop for Mac and QEMU

• Emulation or binary translation

• Pros & cons+ Virtualizing SW use device drivers and other lower-level services of host- Loss of efficiency because more layers of SW involved

5.9.2019 ssss 21

Containers

4.9.2019 ssss 22

Basic of containers• Lightweight virtualization Guests share OS kernel with host

• In Linux (LXC) basically Separate namespaces

• IPC – inter-process communication

• Network

• Mount – Filesystem

• PID – Processes

• User

• UTS – hostname and domains

• Cgroup –

• Nice tutorial (with commercial at the end)

https://www.youtube.com/watch?time_continue=2&v=n-JwAM6XF88

5.9.2019 ssss 23

The dominant way to manage container: DockerSo dominant that we go details of one technology

4.9.2019 ssss 24

From docker.com

5.9.2019 ssss 25

Use case example• Your application needs• Certain version of nodejs• Set of libraries (certain versions)• Mongo database

5.9.2019 ssss 26

• Your system has• Wrong version of nodejs• Mongo serving another application

• Solution• Create a docker image (container)• Install the image• Run the image

https://www.katacoda.com/courses/container-runtimes(thanks to your fellow student)

5.9.2019 ssss 27

Let investigate a bit

5.9.2019 ssss 28

I could not resist

5.9.2019 ssss 29

Typical set-up

5.9.2019 ssss 30

Host

Repository

Image

Image

Image

Download

Containe

r

Containe

r

Containe

r

Containe

r

Typical set-up

5.9.2019 ssss 31

Host

Cont

aine

r

Cont

aine

r

Cont

aine

r

Cont

aine

r

Volume Volume

Volumes are for- Persistent data- Data sharing

Docker commands (subset of)

5.9.2019 ssss 32

docker build Build an image from a Dockerfile

docker container Manage containers

docker commit Create a new image from a container’s changes

docker exec Run a command in a running container

docker image Manage images

docker inspect Return low-level information on Docker objects

docker ps List containers

docker run Run a command in a new container

docker stop Stop one or more running containers

docker swarm Manage Swarm

docker volume Manage volumes

Dockerfile

#This is a sample ImageFROM ubuntuMAINTAINER demousr@gmail.comRUN apt-get updateRUN apt-get install –y nginxCMD [“echo”,”Image created”]

5.9.2019 ssss 33

AnotherFROM node:6.10.0-alpine

# Never run processes as root!USER root

# Copy application itself:COPY . /homeWORKDIR /home# Set port on which to run the node process:ENV PORT=8893# Expose port 8893:EXPOSE 8893

CMD node pinger.js

5.9.2019 ssss 34

If I want to pre-install?

#This is a sample ImageFROM ubuntuMAINTAINER demousr@gmail.comRUN apt-get updateRUN apt-get install –y nginxCMD [“echo”,”Image created”]

5.9.2019 ssss 35

?

Vagrant

4.9.2019 ssss 36

Vagrant intro

• A way to create and distribute development environments as virtual machine (full VMs – not containers)• If time lets look: https://www.vagrantup.com/intro/index.html

5.9.2019 ssss 37

Vagrant vs Docker(https://www.vagrantup.com/intro/vs/docker.html)

• Vagrant is a tool focused on providing a consistent development environment workflow across multiple operating systems. Docker is a container management that can consistently run software as long as a containerization system exists.

• Containers are generally more lightweight than virtual machines, so starting and stopping containers is extremely fast. Docker uses the native containerization functionality on macOS, Linux, and Windows.

• Currently, Docker lacks support for certain operating systems (such as BSD). If your target deployment is one of these operating systems, Docker will not provide the same production parity as a tool like Vagrant. Vagrant will allow you to run a Windows development environment on Mac or Linux, as well.

• For microservice heavy environments, Docker can be attractive because you can easily start a single Docker VM and start many containers above that very quickly. This is a good use case for Docker. Vagrant can do this as well with the Docker provider. A primary benefit for Vagrant is a consistent workflow but there are many cases where a pure-Docker workflow does make sense.

• Both Vagrant and Docker have a vast library of community-contributed "images" or "boxes" to choose from.

5.9.2019 ssss 38

Next plus exercise – will be published today

• Run though this tutorial: <https://docs.docker.com/get-started/>Requires that you have a Linux. VM strongly recommended• Then, after all steps, return output of• docker image ls• docker container ls

• And answer to additional questions.• Deadline is “by next weekend”.

6.9.2019 ssss 39

Material

• Mika Kaaretkoski, Master thesis, 2018

• F. Rodrguez-Haro et al., “A summary of virtualization techniques,” Pro-cediaTechnology, vol. 3, pp. 267 { 272, 2012, the 2012 Iberoamerican Conference on Electronics Engineering and Computer Science. [Online]. • VMWare Inc., Understanding Full Virtualization, Paravirtualization, and

Hardware Assist, 2008. <https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/VMware_paravirtualization.pdf>• Nice tutorial about docker; <https://docs.docker.com/get-started/>

6.9.2019 ssss 40

Rought plan – subject to change

• 30.08: Intro to the course and DevOps

• 06.09: Recap (from earlier courses) on virtualization technologies;

• 13.09: Cloud and scalability

• 20.09: Continuous Deployment –what & why

• 27.09: Continuous Deployment –tools and techniques

• 04.10: Security issues; recap on internet networking (from earlier courses)

• 11.10: Guest lecture on AWS

• 23.10: Issues on cloud-SW: isolation, dependency management etc,

• 30.10: Cloud-native architectures: microservices

• 06.11: Cloud-native architectures: FaaS/serverless

• 13.11: About automation: testing and beyond

• 20.11: Hosting and deployment issues

• 27.11: Guest lecture on Kubernetes

• 04.12: Recap

5.9.2019 ssss 41