service discovery and registration in a microservices architecture

Copyright © PLUMgrid, Inc. 2011-2016

what, why and how?

Service Discovery and Registration in a Microservices Architecture


About us

2

PLUMgrid

KhaliqFawad

@fawadkhaliq

Mesosphere

SanchezFernando

@fernandosanchez


• What is Service Discovery and Registration?• A bit of history: the world in year 2000 (load balancers, physical servers and

CCIEs)• Evolution: Virtualization brings shuffling and unpredictability• Today: Containers demand automation• What are Microservices and how do they affect Service Registration / Discovery?

• Service Registration/Service Discovery Patterns and Tools• Patterns• Tools: ZooKeeper, etcd, Consul, Mesos-DNS, Minuteman etc.

• Scope inside OpenStack?• The need for SR/SG in Openstack (as in any stack)• Current picture• Desired future state

• Q/A

Agenda

Part 1 - A bit of history- DISCLAIMER

- Trying to provide context about the “what” and the “why”.

- Historically inaccurate

- Obviously and shamelessly biased

- A bit blurry…

- DISCLAIMER (2)

- This goes WAY BACK to provide context. I felt a bit OLD making this presentation

- If you know what a v92 modem is, you probably will also feel old listening to it.

A bit of history (1) : Service Registration in the early days

(Very) early days: applications living on a single physical server

“Service Discovery”? = How do I find my service “by name” when it really is an IP address? Well… DNS!

1:1 relationship between the Application and the backend implementing it


Early days: applications living on MANY physical servers

1:N relationship between Service IPs and backend Servers

Physical App Server “churn” was still “manageable”

“Service Discovery”?: DNS + Virtual IPs + Virtual Hosts

Need a Load Balancer!

“Service Registration” = When I add a new server to my service, how does it get added to the list of available ”Service Backends”?

** Add a new backend to the Service IP/ Virtual IP in the load balancer (“Service Access Point”)

** Many times, added manually by a guy who has a vendor certification.

** Including “healthchecks” for each backend


- Advent of the 3-tier architecture (web, application, dB)

- Need load balancing in each layer so that each can scale up/down

- With their respective healthchecks

- So each layer now has a “Service Registration / Service Discovery” need!

A bit of history (4) : Virtualization FTW- Enter server virtualization: Web/App/dB Servers can be

created and destroyed in minutes.

- “Automated” Service Registration becomes an evident need and options start to appear (we’ll discuss each in the 2nd part):

- “Sidecar process” in workloads to register into centralized “backend database”?

- “External Orchestrator” registers backends upon creation?

- “API Gateways” with client or server-side discovery?

- Applications can scale up and down more dynamically, even automatically depending on load

Enter Containerization!

Private Copy

Shared

App 3

Libraries

Guest Operating

System

Infrastructure

App 1

Libs

Host Operating System

Start / stop time One minute to few seconds Milliseconds

Workload density 10 - 100x1x

App 2

Libraries

Guest Operating

System

App 1

Libraries

Guest Operating

SystemApp 2

Libs

App 3

Libs

Virtual Machines Containers

Host Operating System

Hypervisor

Infrastructure

App 4

Libs

App 5

Libs

Today: Containers and Container Orchestration

- Containers multiply the potential complexity: can be created/destroyed in milliseconds, with endpoints being one or more orders of magnitude more numerous than VMs

- Impossible to have manual procedures: need automatic discovery of backends

- All options mentioned for VMs may still be considered, with their pros and cons:

- Sidecar, Centralized Orchestration, Client/Server discovery…

From Containers to Microservices: What and Why?

“The Monolith” and its issues

- Traditional applications were typically developed as a single unified codebase

- Which tends to grow along time

- Making it harder to maintain, troubleshoot, evolve

Microservices: divide and conquer!

- Divide applications into independent pieces (typically functional areas)

- Interconnect the pieces with networking and (REST) APIs

Microservices: Divide and Conquer

Traditional Architecture

Many functions in a single process

Siloed teams

REST

API

s

Microservices Architecture

Cross-functional teams organized around capabilities

Scales individually

Each element of functionality defined as “microservices”

Scales monolithically

Microservices Challenge: SD/SR on steroids!

REST

API

sDependencies Between Microservices Running Multiple Microservices-based Apps

SD/SR Evolution: from tiered architecture to usvcs

14

Before Micro-Services

10’s of services with well-defined endpoints

100’s of backends implementing them, with long duration

Secure Infrastructure with DMZ and Firewalling.

After Micro-Services

1000’s of services requiring new VIPs to talk to others

100000’s of backends that are created and destroyed constantly, sometimes in milliseconds

1000’s of services requiring Auth handshakes

W W W

A AA


Service Registration/ServicePatterns and Tools


Let’s talk about some terminologies that will help us better classify the tools

● Self registration

● Third-party registration

● Client-side discovery

● Service-side discovery

Service Registration/Service Discovery Patterns


Self registration

Self Registration

Microservice

Microservice

Service Registry

When going up/down

When going up/down


Third-party registration

Third Party Registration

Microservice

Microservice

Service Registry

starts service

detects service crash

Microservice Service Managerstop service


Client-side discovery

Client-side Discovery

Client Service Registry

MicroserviceAPI Gateway

12

3

4


Server-side discovery

Server-side Discovery

Client

Service Registry

Microservice

API Gateway1

23

4


Service Discovery Tools


Key-value store based on ephemeral nodes

Consensus using ZAB

Clients handle failure or load balancing themselves

HA: Non-quorum nodes return error on read/write

Allows watch on changes

Written in Java. Provide Java and C language bindings.

Heavyweight for simple architectures

Zookeeper/A/C1

/C2/C3/C4

/B/C5/C6/C7

/C/C8

ZKCluster

Container Host A

config

C1 C3 R/W

C2 C4 NGINX

Container Host B

C6

C7 R/WC5

NGINX

Container Host C

R/WC1

NGINX

Register C1

Watch Lookup C1


Distributed K/V store based on directories

Consensus using RAFT

Service definition queried using JSON-based HTTP APIs

Supports TTLs on keys

Clients handle failure or load balancing themselves

Allows watch on changes

Supports TLS/SSL

etcd

Key , Value---------------------C1 , 1.2.3.4:80C2 , 1.2.3.4:88C3 , 1.2.3.5:80C4 , 1.2.3.5:88C5 , 1.2.3.6:80C6 , 1.2.3.6:88

etcdCluster

Register C1

Look up C1

Container Host A

config

C1 confd

C2 NGINX

1.2.3.4

Container Host B

C4

confdC3

NGINX

config

1.2.3.5

Container Host C

confdC5

NGINXC6

config

1.2.3.6

Watch


Distributed key, value(KV) store based on data directories

Provides comprehensive service health checking using both in-built solutions as well as user provided custom solutions.

Provides REST based HTTP API for interaction.

Security: TLS and ACLs

Service database can be queried using DNS.

Does dynamic load balancing.

Supports single data center and can be scaled to support multiple data centers.

consul

Monitoring Service Locks

Key-value Configuration Store

Host Clustering & Failure Detection


Distributed service for announcement and discovery of services built on top of etcd

Translates etcd keys and values to the DNS

Sky DNSdockerdocker-skydns-adaptor

etcd

skydns

microservice 1

microservice 2

microservice 3

Update DNS entries

Read configuration

HTTP Client

Readconfiguration 1 2 Call

microservice


Translates names to the IP address and port on the machine currently running each application

Syncs with Mesos Master periodically

Updates the DNS records to reflect the latest state

Is stateless

Supports SRV records

Round robin for tasks with the same name

Central DNS

Mesos-DNS

Mesos-DNS

Record Generator

Mesos Master Cluster

State

DNS Resolver

External DNS Servers

Mesos Slave

DNS Lookup

DNS Records

DNS Lookup

DNS Records


Works with DC/OS

Distributed DNS proxy in all members of the cluster

Dual-dispatching of DNS queries to multiple DC/OS

masters to avoid SPOF!

Able to optimally route queries to nodes in a cluster

Spartan

DC/OS Master

DC/OS Svc. Registry (DNS)DC/OS Master


DC/OS Svc. Registry (DNS)

Corporate/External DNS Server

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

rService 1

Service 2

Service 3

External Service


HAproxy based

Ingests state of running Marathon applications

Regenerates HAproxy config

Central LB for east-west

Potentially hundreds of Microservices are talking to each other

If all that traffic goes through a centralized Load Balancer, it quickly becomes a bottleneck

Marathon-lb

DMZ

marathon-lb(HA Proxy)



Internet

Autoscale Controller

AppAppAppApp

Marathon

Mesos Agents (worker nodes)


Works with DC/OS

Distributed Load Balancer

Distributed database of VIPs and endpoints

Provides health-checking and endpoint availability

update

Minuteman

DC/OS Master



DC/OS Svc. Registry (DNS)

Corporate/External DNS Server

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

r

DC/OS Agent

Cont

aine

r

Cont

aine

r

Cont

aine

rService 1

Service 2

Service 3

External Service


HAproxy on each client host

Uses Zookeper as Service Directory

Synapse: updates HAproxy

Nerve: provides backend registry to Zookeeper

SmartStackProvider

Application

Zookeeper

Publication

Consumer

Application

HA Proxy SynapseNerve

API

Discovery


Service Registration/ServiceIn OpenStack


● Container first class citizens in OpenStack

● Microservices use-cases are emerging

● Use of service discovery and service registration tools is unavoidable

● OpenStack as an infrastructure layer as well as platform layer, service

discovery and service registration is a need.

Need for Service Discovery/Registration in OpenStack?


Magnum: containers infrastructure management service

Kolla: containerized deployment of OpenStack

Murano: application catalog for OpenStack

Kuryr: brings OpenStack Networking (Neutron) and Storage to containers

Microservices architecture facilitators in OpenStack


Approach-1:

Containers and ecosystem entirely managed by container technologies like Docker, Kubernetes, Mesos running atop OpenStack infrastructure

Approach-2:

Containers and ecosystem is partly managed by container technologies and partly/completely managed/supported by OpenStack components: Neutron, Nova, Keystone etc.

Two approaches in OpenStack


Approach-1 (Docker, Mesos, Kubernetes managed containers)

● Service discovery and service registration provided by the container orchestration platform via the options discussed above: etcd, mesos-dns, skydns etc

● OpenStack does not participate in service registration and discovery

Approach-2 (Hybrid scenarios with OpenStack tools are used)

● No off-the-shelf proper solution exists today

● Although we have all the tooling to achieve the results.

● Afterall, it boils down to “register”, “lookup” and load balancing

Current Picture


1. Load Balancer: Octavia

2. Designate: DNS

3. Kuryr+Neutron: Networking

4. Service Discovery: A new OpenStack project?

One of the possible options is #4, which makes use of existing OpenStack

tooling to provide service registration and discovery for services and

microservices inside OpenStack.

Future/Desired Future State


Questions?

Thank you!

service discovery and registration in a microservices architecture

Technology