microservice deploymentce.sharif.edu/courses/96-97/.../root/...deployment.pdf · deployment ci...
TRANSCRIPT
Microservice
Deployment
Software Engineering II
Sharif University of Technology
MohammadAmin Fazli
Deployment
Topics
Continuous Integration & Microservices
Continuous Delivery
Deployment Artifacts
Custom Images
Environments
Service Configuration
Service-to-Host Mapping
Physical to Virtual
Reading:
Building Microservices-Sam Newman-Chapter VI
2
Deployment
Continuous Integration
With CI, the core goal is to keep everyone in sync with each other, which we achieve by making sure that newly checked-in code properly integrates with existing code.
A CI server detects that
The code has been committed
Checks it out
Carries out some verification
Making sure the code compiles
Making sure tests pass
As part of this process, we often create artifact(s) that are used for further validation, such as deploying a running service to run tests against it.
3
Deployment
Continuous Integration
Benefits
Fast Feedback about code quality
Automatic binary artifacts’ creation
All the code is version controlled and all artifacts can be recreated
Traceability from deployed artifacts to the code
Traceability over what tests were run over codes and artifacts
CI checks: Jez Humble’s three questions he asks people to test if they really understand what CI is about
Do you check in to mainline once per day?
You need to make sure your code integrates.
Do you have a suite of tests to validate your changes?
Check if our code semantically integrates
When the build is broken, is it the #1 priority of the team to fix it?
Please avoid piling up non-integrated changes
4
Deployment
CI & Microservices
When thinking about microservices & CI, we need to think
about how our CI builds map to individual microservices.
1st solution: One repository+Monolithic build
5
Deployment
CI & Microservices
1st solution:
Benefits:
Fewer repositories to worry about
Simpler build
Downsides:
A one-line change to a single service cause all the other services get verified
and built and can be very time consuming
If this one-line change to a service breaks the build, no other changes can be
made to the other services until that break is fixed.
If a change cause break, which team is in charge?
6
Deployment
CI & Microservices
2nd Solution: One code repository + Individual Build
Check in/Check out process can be simpler as we have one repository to worry about
Can be easily get into the habit of checking in source code for multiple services at once which can slip into making changes that couple services together.
7
Deployment
CI & Microservices
3rd Solution: Individual Repository + Individual Build
When making a change, we run only the build and tests I need to.
We get a single artifact to deploy.
Alignment to team ownership is more clear.
If you own the service, you own the repository and the build.
Making changes across repositories can be more difficult in this
world8
Deployment
Continuous Delivery
The build pipeline concept gives us a nice way of tracking the
progress of our software as it clears each stage, helping give us
insight into the quality of our software.
Ex: Running fast and slow tests together does not get us a fast
feedback. If fast tests fail waiting for slow tests is useless. So it is
better to run them in different stages.
CD builds according to the pipeline concept.
Continuous delivery is the approach whereby we get constant
feedback on the production readiness of each and every check-in,
and furthermore treat each and every check-in as a release
candidate.
9
Deployment
Continuous Delivery
In CD we must model the process of getting our software from check-in to production and know where any given version of the software is.
Different software stages must be modeled both manual or automated
We really want a tool that embraces CD as a first-class concept.
Tools that fully support CD allow you to define and visualize these pipelines, modeling the entire path to production for your software.
By modeling the entire path to production for our software, we greatly improve visibility of the quality of our software, and can also greatly reduce the time taken between releases
10
Deployment
Platform Specific Artifacts
Most technology stacks have some sort of first-class artifact, along with tools to support creating and installing them.
Rubby gems, Java Jars, Python eggs
Depending on the technology stack, these artifacts may not be enough by themselves.
Example: sometimes they need Apache/Nginx installation and configuration
We may need some way of installing and configuring other software that we need in order to deploy and launch our artifacts.
This is where automated configuration management tools like Puppet and Chef can help.
These artifacts are specific to a certain technology stack. This may make deployment more difficult when we have a mix of technologies in play
Automation can go a long way toward hiding the differences in the deployment mechanisms of the underlying artifacts.
11
Deployment
Operating System Artifacts
One way to avoid the problems associated with technology-specific
artifacts is to create artifacts that are native to the underlying
operating system.
Rpm packages for CentOS, MSI for windows, deb for Ubuntu
Advantages of using OS-specific artifacts:
From a deployment point of view we don’t care what the underlying
technology is. We just use the tools native to the OS to install the package.
The OS tools can also help us uninstall and get information about the
packages too,
The OS tools may even provide package repositories that our CI tools can
push to
Downsides:
Difficulty in creating the packages
There is a great overhead when deploying on different OSes
12
Deployment
Custom Images
Preparing machines for a service is very time consuming even if automated configuration tools (Puppet, Chef, Ansible) are used.
Installing a JVM needs some minutes
Automatic tools can be smart and will avoid installing software that is already present. This does not mean that running the scripts on existing machines will always be fast, unfortunately, as running all the checks takes time.
Moreover, we don’t want to allow for too much configuration drift. So we should have many machine preparations and it is a real drag.
Configuration Drift is the phenomenon where running servers in an infrastructure become more and more different as time goes on, due to manual ad-hoc changes and updates, and general entropy.
13
Deployment
Custom Images
One approach to reducing this spin-up
time is to create a virtual machine image
that bakes in some of the common
dependencies we use.
When we want to deploy our software,
we spin up an instance of this custom
image, and all we have to do is install the
latest version of our service.
Because you build the image only once,
when you subsequently launch copies of
this image you don’t need to spend time
installing your dependencies, as they are
already there.
14
Deployment
Custom Images
Drawbacks:
Building images can take a long time. This means that for developers
you may want to support other ways of deploying services to ensure
they don’t have to wait half an hour just to create a binary
deployment.
Some of the resulting images can be large. This could be a real
problem if you’re creating your own images.
Moving a huge image around a network is a real problem
The tool chain required to build such an image varied from platform
to platform.
Building a VMWare image is different from building an AWS AMI, a Vagrant
image, or a Rackspace image.
Some of the tools are multi-platform like packer.io which has support for
VMWare, AWS, Racksapce Cloud, Digital Ocean, Vagrant and …
15
Deployment
Immutable Servers
By storing all our configuration in source control, we are trying to ensure that we can automatically reproduce services and hopefully entire environments at will.
But once we run our deployment process, what happens if someone comes along changes things independently of what is in source control?
Configuration Drift-the code in source control no longer reflects the configuration of the running host.
Immutable Servers Pattern: we can ensure that no changes are ever made to a running server. Instead, any change, no matter how small, has to go through a build pipeline in order to create a new machine.
We can disable SSH
16
Deployment
Environments
Different environments in CD stages:
Development Environments and Unit Testing
One for Slower Tests
One for UAT
One for Performance Testing
One for Production
17
Deployment
Environments
Our microservice should be the same throughout, but
the environment will be different.
Separate distinct collection of configuration and hosts
Differences can be more than just some configurations
Ex. our production environment for our service might consist of multiple
load-balanced hosts spread across two data centers, whereas our test
environment might just have everything running on a single host
As you move from development environment to build server
to UAT environment all the way to production, you’ll want to
ensure that your environments are more and more
production-like to catch any problems associated with these
environmental differences sooner.
18
Deployment
Service Configuration
Our services need some configuration
Ex. Username and password for database connection in different
environments
Configuration that changes from one environment to another
should be kept to an absolute minimum.
The more your configuration changes fundamental service
behavior, and the more that configuration varies from one
environment to another, the more you will find problems only
in certain environments, which is painful in the extreme.
19
Deployment
Service Configuration
Handling service configurations in CD
Building one artifact per environment, with configuration inside the artifact itself. Let’s imagine I build a Customer-Service-Test and Customer-Service-Prod
artifacts. If my Customer-Service-Test artifact passes the tests, but it’s theCustomer-Service-Prod artifact that I actually deploy, can I be sure that I haveverified the software that actually ends up in production?
There is the additional time taken to build these artifacts.
We need to know at build time what environments exist.
Problems with sensitive data: I don’t want information about production passwords checked in with my source code, but if it is needed at build time to create all those artifacts, this is often difficult to avoid
Creating one single artifact, and manage configuration separately
This could be a properties file that exists for each environment, or different parameters passed in to an install process.
Using a dedicated system for providing configuration
Useful when dealing with a large number of microservices
20
Deployment
Service-to-Host Mapping
How many services per machine?
Solutions
Multiple services per host
Application containers
Single service per host
Platform as a service
Host: A generic term which defines an operating system onto
which we can install and run services
They can be physical or virtual machines
21
Deployment
Multiple Services per Host
Benefits
It is simpler. If more services are packed on to a single host, the host
management workload doesn’t increase as the number of services
increases.
It has less cost. Even if you have access to a virtualization platform
that allows you to provision and resize virtual hosts, the
virtualization can add an overhead that reduces the underlying
resources available to your services.
This model is also familiar to those who deploy into some form of an
application container.
22
Deployment
Multiple Services per Host
Challenges
It can make monitoring more difficult.
What can we do to trace CPU for one service?
If one service is under significant load, it can end up reducing the resources available to other parts of the system.
Deployment of services can be somewhat more complex too, as ensuring one deployment doesn’t affect another leads to additional headaches.
If we use Puppet to prepare a host, but each service has different (and potentially contradictory) dependencies
This model can also inhibit autonomy of teams.
Efforts to target scaling to the service most in need of it can be complicated.
This model can limit our deployment artifact options.
23
Deployment
Application Containers
The idea is that the application container your services live in
gives you benefits in terms of improved manageability, such as
clustering support to handle grouping multiple instances
together, monitoring tools, and the like.
Like IIS for .NET applications and Servlet container for Java
applications
24
Deployment
Application Containers
Downsides
They inevitably constrain technology choice.
They also force the implementation technology
Some of them have ability to manage clusters to support shared in-
memory session state, which is in contradiction with scalability
Their monitoring abilities are not sufficient
Attempting to do proper lifecycle management of applications on top
of platforms like the JVM can be problematic, and more complex than simply restarting a JVM.
Analyzing resource use and threads is also much more complex, as
you have multiple applications sharing the same process.
They add resource overheads
25
Deployment
Single Service per Host
Benefits:
Simpler monitoring
Simpler remediation
Reduced single points of failure
Easy Scaling one service independent from others
This opens up the potential to use alternative deployment techniques
such as image-based deployments or the immutable server pattern
Downside
We have more servers to manage
It has more cost
26
Deployment
Platform as a Service
Using a platform as a service (PaaS), you are working at a higher-level abstraction than at a single host.
Taking technology specific artifacts like java WAR files and Ruby gems and automatically provision and run them.
Some of them handle scaling the system up and down
27
Deployment
Automation
One of the pushbacks against the single-service-per-host setup
is the perception that the amount of overhead to manage
these hosts will increase.
It is true if everything is done manually
2X servers, 2X work
Many things can be automated
Automation enables developers for self-service-provision
Ideally, developers should have access to exactly the same tool chain
as is used for deployment of our production services so as to ensure
that we can spot problems early on.
28
Deployment
Move from Physical to Virtual
One of the key tools available to us in managing a large
number of hosts is finding ways of chunking up existing
physical machines into smaller parts.
Traditional Virtualization
Vagrant
Linux Containers
Docker
29
Deployment
Traditional Virtualization
Virtualization allows us to slice up a physical server into separate hosts, each of which can run different things.
Type 2 virtualization: A hypervisor run on the OS whose jobs are:
It maps resources like CPU and memory from the virtual host to the physical host.
It acts as a control layer, allowing us to manipulate the virtual machinesthemselves.
Technologies like AWS, VMWare, VSPhere, Xen, KVM and …
In type 1, VMs run directly on hardware.
The problem is that the hypervisor here needs to set aside resources to do its job.
30
Deployment
Vagrant
Vagrant is a very useful deployment platform, which is normally used for dev and test rather than production.
Vagrant provides us with a virtual cloud on your laptop.
It allows us to define a set of VMs in a text file, along with how the VMs are networked together and which images the VMs should be based on.
This text file can be checked in and shared between team members.
We can spin up multiple VMs at a time, shut individual ones to test failure modes, and have the VMs mapped through to local directories so you can make changes and see them reflected immediately.
Downside:
Running lots of VMs can tax the average development machine. If we have one service to one VM, you may not be able to bring up your entire system on your local machine.
31
Deployment
Linux Containers
In Linux processes can spawn other processes
The Linux kernel’s job is to manage the tree of processes
For Linux users, there is an alternative to virtualization. Rather
than having a hypervisor to segment and control separate
virtual hosts, Linux containers instead create a separate
process space in which other processes live.
Many forms of this technology
Solaris Zones
OpenVZ
LXC is the most popular
Available on most modern Linux
distributions
32
Deployment
Linux Containers
We don’t need a hypervisor.
More available resources
Although each container can run its own operating system distribution, it has to share the same kernel
Because the kernel is where the process tree lives
Ex. Our host operating system could run Ubuntu, and our containers CentOS, as long as they could both share the same kernel.
Linux containers are much faster to provision than full-fat virtual machines.
Due to the lighter-weight nature of containers, we can have many more of them running on the same hardware than would be possible with VMs.
33
Deployment
Docker
Docker is a platform built on top of lightweight containers. It handles much of the work around handling containers for you.
In Docker, you create and deploy apps, which are synonymous with images in the VM world, albeit for a container-based platform.
Docker manages the container provisioning, handles some of the networking problems for you, and even provides its own registry concept that allows you to store and version Docker applications.
The Docker app abstraction is a useful one for us, because just as with VM images the underlying technology used to implement the service is hidden from us.
Rather than using Vagrant to host multiple independent VMs,each one containing its own service, we can host a single VM in Vagrant that runs a Docker instance.
We then use Vagrant to set up and tear down the Docker platformitself, and use Docker for fast provisioning of individual services.
34
Deployment
Docker
Docker is a simple PaaS for a single machine
Different technologies for Docker:
CoreOS: A stripped-down Linux OS that provides only the essential
services to allow Docker to run. This means it consumes fewer
resources than other operating systems, making it possible to
dedicate even more resources of the underlying machine to our
containers.
Kubernetes & CoreOS Cluster Technology: Help to manage services
across multiple Docker instances
Deis: Heroku like PaaS on top of Docker
Container as a Service (CaaS): Docker with an appropriate
scheduling layer sits between IaaS and PaaS solutions
35
Deployment
Deployment Interface
Ex: we’re developing locally and want to deploy our catalog service into our local environment.
Ex: Once I’ve checked in, our CI build service picks up the change and creates a new build artifact, giving it the build number b456.
Ex: our QA wants to pull the latest version of the catalog service into an integrated test environment to do some exploratory testing, and to help with a showcase.
36
Deployment
Environment Definition
Ex: An
environment
definition file
37
Deployment
Environment Defintion
1: We varied the size of the instances we used to be more cost
effective. You don’t need a 16-core box with 64GB of RAM for
exploratory testing!
2: Being able to specify different credentials for different
environments is key. Credentials for sensitive environments
were stored in different source code repos that only select
people would have access to.
3:We decided that by default if a service had more than one
node configured, we would automatically create a load
balancer for it.
38
Deployment
Environment Definitions
39
1: This was the name of the Puppet file to run.