load balancing applications with nginx in a coreos cluster

#nginx #nginxplus

Load Balancing Applications with NGINX in a CoreOS Cluster

1

Kevin Jones - Engineer, NGINX, Inc. @webopsxMichael Pleshakov - Engineer, NGINX, Inc. @plshkv

#nginx #nginxplus2

Links

https://goo.gl/DY0QIt

#nginx #nginxplus3

• Quick Overview of Our Deployment Plan• Quick Overview of CoreOS, etcd and fleet• Discuss Using etcd as a Service Discovery Tool• Discuss Using fleet as Application Deployment Scheduler• Discuss Using NGINX as a Software Load Balancer• Tie It All Together With…. A Live Demo!

Our Objectives… wait CoreOS party bus?!

#nginx #nginxplus4

What Exactly is Service Discovery?

https://www.nginx.com/blog/service-discovery-in-a-microservices-architecture/

• Used for tracking dynamic assigned IP addresses and port numbers of services

• Tracking credentials, protocols, version or environment details of services

#nginx #nginxplus5

The Sidekick Service Discovery Model

#nginx #nginxplus6

CoreOS Clustered Deployment

#nginx #nginxplus7

#nginx #nginxplus8

• Open Source Project• Easy to Setup• Run Services as Containers• Stable & Reliable Update System• Secure

Why we like CoreOS…

#nginx #nginxplus9

• Built in Cluster Management! (fleet)

• Built in Service Discovery Tool! (etcd)

Most Importantly…

#nginx #nginxplus10

fleet

#nginx #nginxplus11

What is fleet?

• Present your CoreOS cluster as a single init system• Schedule deployment units across a cluster• Deploy containers on arbitrary hosts• Distribute services as ephemeral units across a cluster

of CoreOS machines• Maintain set number of instances and re-schedule on

failure• Remote configuration using fleetctl

#nginx #nginxplus12

core@core-01 ~ $ fleetctl list-machinesMACHINE IP METADATA20f5eff1... 172.17.8.103 -23a36596... 172.17.8.102 -6ee835cb... 172.17.8.101 -fd546c18... 172.17.8.104 -

Use fleetctl To Manage Your Cluster

https://coreos.com/fleet/docs/latest/using-the-client.html

#nginx #nginxplus13

core@core-01 ~/unit-files $ cat backend@.service

[Unit]Description=Backend ServiceAfter=docker.serviceRequires=docker.service

[Service]TimeoutStartSec=0ExecStartPre=-/usr/bin/docker kill backendExecStartPre=-/usr/bin/docker rm backendExecStartPre=/usr/bin/docker pull nginxExecStart=/usr/bin/docker run --name backend -p 8080:80 nginxdemos/helloExecStop=/usr/bin/docker stop backend

[X-Fleat]Conflicts=backend@*.service

Unit Files

https://coreos.com/fleet/docs/latest/launching-containers-fleet.html

#nginx #nginxplus14

Conflict - Tells fleet not to schedule more than one Unit on the same machine.

EnvironmentFile - Imports the CoreOS environment variables from a specified file.

ExecStart - Executes a command at the launch of the Unit.

MachineOf - Tells fleet to schedule the Unit on the the same server of the specified Unit.

BindsTo - Links the two Units so they stop at the same time.

ExecStop - Executes a command and the stop of the Unit.

After - Tells fleet to schedule deployment after the specified Unit.

Unit File Configurations Used…

#nginx #nginxplus15

#nginx #nginxplus

What is etcd?

16

• Reliable distributed key/value storage• Written in Go• Simple interface (HTTP+JSON)• Secure (SSL client cert auth)• Fast

https://github.com/coreos/etcd

#nginx #nginxplus17

core@core-01 ~/unit-files $ etcdctl ls /services/backend/services/backend/172.17.8.104:8080/services/backend/172.17.8.103:8080/services/backend/172.17.8.102:8080

Use etcdctl To Manage Your Key Store

#nginx #nginxplus18

Or…. A REST API!! YAY!

core@core-01 ~/unit-files $ curl 127.0.0.1:2379/v2/keys/services/backend {"action":"get","node":{"key":"/services/backend","dir":true,"nodes":[{"key":"/services/backend/172.17.8.104:8080","value":"server","expiration":"2016-02-27T22:28:24.021550862Z","ttl":72,"modifiedIndex":4438,"createdIndex":4438},{"key":"/services/backend/172.17.8.103:8080","value":"server","expiration":"2016-02-27T22:28:29.225960364Z","ttl":77,"modifiedIndex":4463,"createdIndex":4463},{"key":"/services/backend/172.17.8.102:8080","value":"server","expiration":"2016-02-27T22:29:12.812515389Z","ttl":120,"modifiedIndex":4569,"createdIndex":4569}],"modifiedIndex":1010,"createdIndex":1010}}

#nginx #nginxplus19

#nginx #nginxplus20

confd - http://www.confd.io/

Method 1: Configuration Management

confd is a lightweight configuration management tool focused on:

• keeping local configuration files up-to-date using data stored in etcd, consul, dynamodb, redis, vault, zookeeper or env vars and processing template resources.

• reloading applications to pick up new config file changes

#nginx #nginxplus21

Method 2: NGINX Plus API

core@core-01 ~/unit-files $ curl 127.0.0.1:8081/upstream_conf?upstream=backendserver 172.17.8.103:8080; # id=2server 172.17.8.102:8080; # id=7server 172.17.8.104:8080; # id=8

Dynamic Reconfiguration API (upsteam_conf) - HTTP based API to manage NGINX upstream servers

#nginx #nginxplus22

How Can NGINX Plus Help?

Active Health Checks - Ability to perform regular expression match against the body of the response, specific HTTP status code and specific header response

location @healthcheck { internal; proxy_pass http://backend; proxy_connect_timeout 1s; proxy_read_timeout 1s; health_check interval=1s;}

#nginx #nginxplus23

Well How About Open Source NGINX?

Passive Health Checks - If the response from a particular server fails with an error, NGINX will mark this server as failed, and will try to avoid selecting this server for subsequent inbound requests for a while.

• fail_timeout - Sets the time during which the specified number of failed attempts should happen and still consider the server unavailable. In other words, the server is unavailable for the interval set by fail_timeout. (default 10 seconds)

• max_fails - sets the number of failed attempts that should happen during the specified time to still consider the server unavailable. (default 1)

#nginx #nginxplus

Live Demo!

24

#nginx #nginxplus25

https://goo.gl/DY0QIt

Thank you for coming!