cloud native: designing change-tolerant software
TRANSCRIPT
@cdavisafc
CLOUD-NATIVEDESIGNING CHANGE-TOLERANT SOFTWARECornelia Davis, Sr. Director of Technology, Pivotal
@cdavisafc
TEXTMajor Outage on 20 September 2015
Airbnb, Nest, IMDB and many more experienced downtime
… Including Netflix
Outage lasted 5+ hours
“brief availability blip” experienced a
@cdavisafc
TEXT
ME?
Developer (wasn’t Ops)
Web architectures for >10 years
Cloud-native for 5 years
Cloud Foundry for 5 years
More recently
Discount code 39% off!: ctwintitx
@cdavisafc
▸ Cloud-Native App (yup, the microservices)
▸ Cloud-Native Services (yup, some are apps)
SERVICE APP
SERVICE APP
SERVICE APP
SERVICE APP
APP
@cdavisafc
▸ Cloud-Native App (yup, the microservices)
▸ Cloud-Native Services (yup, some are apps … and other are other types of services)
SERVICE APP
SERVICE APP
SERVICE APP
SERVICE APP
DATA DATA
DATA
APP
@cdavisafc
▸ Cloud-Native App (yup, the microservices)
▸ Cloud-Native Services (yup, some are apps … and other are other types of services)
▸ Cloud-Native Data (a distributed data fabric)
SERVICE APP
SERVICE APP
SERVICE APP
SERVICE APP
DATA DATA
DATA
?
APP
@cdavisafc CLOUD-NATIVE SOFTWARE
▸ Cloud-Native App (yup, the microservices)
▸ Cloud-Native Services (yup, some are apps … and other are other types of services)
▸ Cloud-Native Data (a distributed data fabric)
▸ The Cloud-Native Collective (implicit and explicit connections)
APP
SERVICE APP
SERVICE APP
SERVICE APP
SERVICE APP
DATA DATA
DATA
?
@cdavisafc
AGENDA (THE REST OF IT)
▸ Cloud-Native App
▸ Cloud-Native Services
▸ Cloud-Native Data
The Cloud-Native Collective
@cdavisafc
CLOUD-NATIVE APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
APP APP
APP
APP
APPSCALE HORIZONTALLY
APP APP
APP
APP
APP
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APPAPP
APP
APP APPAPP
APP
APP APP
APP
APP
APPSCALE HORIZONTALLY
APP APP
APP
APP
APP
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APPAPP
APP
APP APPAPP
APP
APP APP
APP
APP
APPSCALE HORIZONTALLY
▸ Need an abstraction to treat set of instances as one logical unit
▸ Load Balancing - DNS? Something else?
▸ Dynamic!!!
▸ IP Addresses of instances always changing
APP APP
APP
APP
APP
ROUT
ER
ROUT
ER
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUT
ER
APP (INSTANCE 1)
ValidTokens: [UserToken]
/login
token=UserToken
token=UserToken
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUT
ER
APP (INSTANCE 1)
ValidTokens: [UserToken]
/login
token=UserToken
token=UserToken
APP (INSTANCE 2)
ValidTokens: []
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUT
ER
APP (INSTANCE 1)
ValidTokens: [UserToken]
/login
token=UserToken
token=UserToken
APP (INSTANCE 2)Unauthorized ValidTokens: []
token=UserToken
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you think!) the instance you are sticky to will disappear
▸ Do make your instances completely stateless
ROUT
ER
APP (INSTANCE 1)
ValidTokens: [UserToken]
/login
token=UserToken
token=UserToken
APP (INSTANCE 2)Unauthorized ValidTokens: []
token=UserToken
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you think!) the instance you are sticky to will disappear
▸ Do make your instances completely stateless
▸ Externalize state
ROUT
ER
APP (INSTANCE 1)
ValidTokens: [UserToken]
token=UserToken
token=UserToken
APP (INSTANCE 2)
token=UserToken
STATE STORE
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Property Files are the abstraction used to externalize configuration
▸ They do not hold actual values!
▸ Values coming from the environment are supplied via environment variables
▸ Other configuration is controlled just like source code and injected
RUNTIME CONTEXT WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S) DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp= ${CF_INSTANCE_IP} specialization= ${SPECIALIZATION:
Science}
APP CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController { @Value("${hostIp}") private String hostIp; @Value("${specialization}") private String specialization; ... }
SYSTEM PROPERTIES PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Other configuration is controlled just like source code and injected
RUNTIME CONTEXT WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S) DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp= ${CF_INSTANCE_IP} specialization= ${SPECIALIZATION:
Science}
APP CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController { @Value("${hostIp}") private String hostIp; @Value("${specialization}") private String specialization; ... }
SYSTEM PROPERTIES PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29
APP PROPERTIES STORED AND VERSIONED IN A
REPOSITORY SUCH AS GIT
specialization =Sports
CONFIG SERVER ROLE IS TO INJECT MANAGED APP
PROPERTIES INTO THE APP
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have rippling effects through the collective
INVOICE APP ordersvcIP =10.24.1.35
ORDER SERVICE APP 10.24.1.3510.24.63.116
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have rippling effects through the collective
▸ The application must broadcast lifecycle change events
▸ (Note: I strongly recommend you use a framework to help you with this!)
▸ And an app must be able to absorb that configuration at the right time (run time?)
INVOICE APP ordersvcIP =10.24.1.35
ORDER SERVICE APP
Some “magic” happens
10.24.63.116
Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …
I’ll adapt to these changes! Orders Service new IP: 10.24.63.116
@cdavisafc
CLOUD-NATIVE SERVICE
LET’S TALK ABOUT AUTONOMY
A major benefit of cloud-native architectures is independent:
▸ Application Scaling
▸ Team Scaling
▸ Development Cycles
▸ Experimentation
▸ Resilience
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making deployments a non-event
SERVICE APP
SERVICE APP
APP
SERVICE APP
APP
… AND PARALLEL DEPLOYS
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making deployments a non-event
SERVICE APP
SERVICE APP
APP
SERVICE APP
APP
… AND PARALLEL DEPLOYS
SERVICE APP
1.0.0 2.0.0
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Let’s dig into the “magic” (it’s not really magic)
INVOICE APP orderSvcCoords...
ORDER SERVICE APP 10.24.63.116
Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …
I’ll adapt to these changes! Orders Service new IP: 10.24.63.116
Some “magic” happens
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing table
▸ Messaging pattern to decouple service from router
▸ Must handle lost broadcast messages
▸ Request pass through router INVOICE APP orderSvcCoords =orderSvc.example.com
ORDER SERVICE APP 10.24.63.116
Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …
ROUTER
OrderService: 10.24.63.116, … CustomerService: 10.24.3.1, …
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing table
▸ Messaging pattern to decouple service from router
▸ Must handle lost broadcast messages
▸ Request pass through router INVOICE APP orderSvcCoords =10.24.1.13
ORDER SERVICE APP 10.24.63.116
Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, … CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes! Orders Service new IP: 10.24.63.116
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing table
▸ Messaging pattern to decouple service from router
▸ Must handle lost broadcast messages
▸ Request pass through router INVOICE APP orderSvcCoords =10.24.63.116
ORDER SERVICE APP 10.24.63.116
Here I am! IP Address: 10.24.63.116 Version: 3.4.239 …
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, … CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes! Orders Service new IP: 10.24.63.116
@cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING
▸ App must look for and leave breadcrumbs
▸ Use a framework to help you do this!!!
▸ Tools can then read and correlate logs
SERVICE APP
SERVICE APPAPP
SERVICE APP
SERVICE APP
SERVICE APP
@cdavisafc
CLOUD-NATIVE DATA
DATA APIS (SERVICES)
▸ Microservices do not access data layer directly
▸ Except for those that implement the data API
▸ A surface area to:
▸ Implement access control
▸ Implement throttling
▸ Perform logging
▸ Other policies…
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned microservices…
V1 V2
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned microservices…
… andV1 V2
PARALLEL DEPLOYS
@cdavisafc
CLOUD-NATIVE DATA
“Caching at Netflix: The Hidden Microservice” https://www.youtube.com/watch?v=Rzdxgx3RC0Q
@cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES
▸ Supports Polyglot persistence
▸ Independent availability, backup/restore, access patterns, etc.
@cdavisafc
CLOUD-NATIVE
▸ Cloud-native Apps
▸ Scale out
▸ Statelessness
▸ Externalize configuration
▸ Implications from changes in the application lifecycle
▸ Services
▸ Versioned services (autonomy)
▸ Service Discovery
▸ Distributed tracing
▸ Data
▸ Breaking the Data monolith
▸ Data APIs
▸ Caching
▸ Polyglot Persistence
▸ Event sourcing