the rise of microservices - containers and orchestration
TRANSCRIPT
The Rise of Microservices: Containers and Orchestration
Andrew Morgan ([email protected])
26th September 2016
2
Agenda
1. Why Microservices2. What are Microservices3. Best practices4. Containers5. Orchestration6. MongoDB & Microservices7. Business benefits8. When to use Microservices9. Use Cases10.Resources
3
Microservices - The Attraction
Web Scale
Speed
Iterate
Agile
ReactIsolate
Maintain
Empower
4
1990s & Earlier
Monolithic
Cross-Team
Tight Coupling
Small Change
Huge Impact
System Retests
5
2000s
SOA
Coordination
Looser Coupling
System Retests
6
2010s
Microservices
DecoupledIndependent
DevIsolated Impact
7
Microservices Example
TwitterIngestGoogle+
Ingest
Snapchat Ingest
Facebook Ingest
Feed merge
8
Microservices Example
TwitterIngest
Snapchat Ingest
Facebook Ingest
Feed merge
9
Microservices Example
TwitterIngest
Snapchat Ingest
Facebook Ingest
Feed merge
10
Microservices Example
TwitterIngest
Ingest
Snapchat Ingest
Facebook Ingest
Feed merge
11
Microservices Example
TwitterIngest
Ingest
Snapchat Ingest
Facebook Ingest
Feed merge
Snapchat IngestSnapchat
IngestSnapchat IngestSnapchat
Ingest
12
Alternate Microservices Example
13
Conway’s Law
"Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.”– Melvin Conway
14
Development Teams
15
Containers – Powering Microservices
Real world shipping containers• Same container efficiently transports goods by road,
rail, and sea
• Contents remain untouched through all those
environments; no repacking needed
• Ubiquitous and standardized
• Simple to use – open, fill, close
• Contents of each container safe from the others
• Know how much space the container consumes
16
Containers – Powering MicroservicesSoftware containers• Build an image containing the full application stack only
once
• Spin up many containers from the same image in
multiple environments
– Laptop, data center, cloud
– Development, QA, production, support
• Simple to use & efficient
• Contents of each container isolated from the others
– Storage, memory, namespace
• Constrain resources available to each container
– Storage, memory, CPU, IO
VM VMVM
VMs Containers
Bare Metal
Host Operating System
Hypervisor
Guest OS
Libraries
Apps
Service
Guest OS
Libraries
Apps
Service
Guest OS
Libraries
Apps
Service
Container ContainerContainer
Bare Metal
Host Operating System
Docker Engine
Libraries
Libraries
Apps
Libraries
Apps
Service ServiceService
18
DockerThe most popular container technology
• Simple to use and has a rich ecosystem
• 100,000+ images available from Docker Hub
– Including mongo hub.docker.com/_/mongo/
– Syncs with GitHub projects
• Define new images built upon base images
• Define interfaces between containers
• LINUX, (and now) Windows, and OS X
• Runs on bare metal, VMs, and cloud. Cloud
providers supply the Docker infrastructure (e.g.
Google Container Engine)
19
docker run -d mongo
21
22
Containers and MicroservicesMicroservices built by combining multiple
containers
• Build sophisticated services from many small,
focused processes (containers)
– Well defined APIs between components
– Each component can use different libraries,
middleware & programming languages
• Modular, decoupled architecture simplifies
maintenance and enables reuse
• Fault tolerant
• ScalableCmglee
Connecting the Microservices – Apache Kafka
24
Orchestration
Automated deployment, connecting, and
maintenance of multiple containers
• Provision hosts
• Instantiate containers
• Reschedule failed containers
• Link containers through defined interfaces
• Expose services to the outside world
• Scale out and back in
25
Orchestration – Kubernetes
Created by Google, feature-rich and widely
adopted
• Automated container deployment and ‘replication’
• On-line scale out/in
• Rolling upgrades
• HA – automatic rescheduling of failed containers
• Exposure of network ports to external apps
• Load balancing over groups of containers providing
a service
• Provided as a service by Google Compute Engine
26
Orchestration – Apache MesosDesigned to scale to 10,000s of physical
servers; used by Twitter, Airbnb & Apple
• Developer writes code to turn application into a
framework to run on Mesos
• Less feature rich than Kubernetes; considers many
functions such as load balancing, rescheduling, and
scaling to be a higher level function
– Project exists to run Kubernetes as a Mesos
framework
• Foundation for distributed systems
– Apache Aurora, Chronos, Marathon
27
Choosing an Orchestration Framework
Factors to consider…• Integration with existing DevOps frameworks?
• Number of hosts?
• Bare metal, VMs, or cloud deployment?
• Automated High Availability?
• Grouping and load balancing?
• Existing skills?
• Install your own orchestration framework or use as a
service?
28
Security
Containers provide opportunities to
improve security• Containers provide isolation; resources can only be
accessed from outside through explicitly provided
APIs
• Resources can be rationed
• A container’s role can be very narrow – remove
anything not required
• Images and containers should be kept current;
rolling upgrades with Kubernetes or Aurora
• Typically log into container as root so restrict access
29
Orchestrating MongoDBOrchestrating MongoDB containers requires
special treatment as it’s a distributed,
stateful application…• State should survive rescheduling; use Kubernetes’
persistent volumes abstraction
• Replica Set members must communicate with each
other; expose external IP addresses/ports which
survive rescheduling
• Replica Set must be initialized from exactly one
member
• MongoDB must still be monitored and backed up –
MongoDB Cloud Manager
BenefitsCD
Scale
Test
IsolatePerform
Available
TTM
Org
Cost
32
When to use Containers & Microservices
Why MongoDB is a good fit for Microservices
Flexible data model
Redundancy
Monitoring &
automationScalability
Simplicity
MongoDB & Microservices in the Wild
35
Resources• Case Study – FuboTV
https://www.mongodb.com/blog/post/leaf-in-the-wild-leading-soccer-streaming-service-fubotv-scales-its-business-with-mongodb-docker-containers-and-kubernetes
• Case Study – Square Enixhttps://www.mongodb.com/blog/post/leaf-in-the-wild-square-enix-scales-tomb-raider-hitman-absolution-deus-ex-and-more-on-mongodb
• “Enabling Microservices – Containers &
Orchestration Explained” white paperhttps://www.mongodb.com/collateral/microservices-containers-and-orchestration-explained
• “Microservices: The Evolution of Building Modern
Applications” white paperhttps://www.mongodb.com/collateral/microservices-the-evolution-of-building-modern-applications
MongoDB Europe 2016Old Billingsgate, London
15th November
mongodb.com/europeandrewmorgan20 – 20% discount