an illustrated guide to microservices (ploneconf 10 21-2016)
TRANSCRIPT
An Illustrated Guide to Microservices
Philip LombardiEngineering
datawire.io
Background...
1. Philip Lombardi @ Datawire.io (twitter: @TheBigLombowski)
2. Datawire.io is building a Microservices Development Kit to enable developers to build resilient microservice applications.
3. Check us out after meetup: https://datawire.io
2
datawire.io
This is a Python talk… really...
1. Datawire is a first-class Python shop.
2. We love Python so much, we use it to write code in other languages (yes, we have a compiler written in Python).
3. Python & Microservices complement each other really well.
4. Time permitting I’ll do some demo’s with Python microservices...
3
datawire.io 4
What’s a Microservice?
datawire.io
Common Microservice Definitions
It’s a service that is...
● Small● Self contained● Narrow in scope● Bounded context● Independent● Loosely coupled
Correct, but incomplete...
5
datawire.io
Common Microservice Definitions (as applied to Legos)
A piece of plastic that is…
● Small● Self contained● Narrow in scope● Bounded context● Independent● Loosely coupled
What’s missing…?
6
datawire.io
A (simpler) Microservices Definition
A Microservice is a unit of business logic.
A Microservice application is a distributed composition of business logic via services.
7
datawire.io
Combine to build AWESOME!
8
datawire.io
What do you get when you combine Microservices?
Topologies!
9
datawire.io 10
Death Star Topology
datawire.io
Wait, why do I want a death star?
Maybe you don’t, but your CXO does
11
Organizational Scale
AgilityVelocity
datawire.io
Topologies
12
datawire.io
Topologies have always been a thing
● Topologies aren’t new, they have always been the bread and butter of distributed systems design
● In fact we’ve all been stuck living in the same topology for a while now
13
datawire.io
Topologies are powerful
Information flows through topologies like water through a pipe or current on a wire
Probably the most important factor in the performance of a distributed system
14
datawire.io
Topologies are the purview of the Elite
Architects define the topology
Developers get to plug in business logic in a few predefined places
15
datawire.io
Microservices bring topology to Developers
Developers define the topologyBusiness logic is distributed rather than being centralized
The topology changes and grows much more quickly
16
datawire.io
Topologies can solve a lot of problems
IntegrationOperational Scale
Reliability
17
datawire.io
Operational Scale (Linear Topology)
18
Ingest Source of Truth Transform Present
Template for many data driven businesses…
datawire.io
Operational Scale (Linear Topology)
19
Ingest Source of Truth Transform Present
Template for many data driven businesses…
Network Disk Disk + CPU Network + CPU
datawire.io
Scale Components Independently
20
Ingest Source of Truth Transform Present
Template for many data driven businesses…
Network Disk Disk + CPU Network + CPU
datawire.io
Integration (Fan Out Topology)
21
datawire.io 22
Reliability
Software Bug
Centralized Business Logic can be a Single
Point of Failure
Operational Failure
datawire.io 23
Reliability
Software Bug
Decentralize Business Logic to Isolate Failures
datawire.io 24
Organizational Scale
Software Bug
Decentralized Business Logic Means● Multiple Teams● Smaller Codebases● Ramp up Developers Faster
datawire.io
Can topologies give me a death star?
They help, but not quite… you need to build topologies quickly!
25
Organizational Scale
AgilityVelocity
datawire.io
Microservices vs SOA(Service Oriented Architecture)
26
datawire.io
Common Explanations
Both are topologies of services
● Fine grained SOA● Integration pattern vs application components● Organizational factors...
We need a better definition…
● These explanations ignore the process that yields the topology
27
datawire.io
Architecture and Development are different processes
Both can yield topologies of services, but SOA won’t build your death star. For that you need agile topologies
Microservices: Service Oriented Development
Architecture:
● Lots of up front thinking● Slow iteration cycle● Months/years until you get feedback
Development:
● Experimental● Rapid iteration cycle● Minutes/hours until you get feedback
28
datawire.io
You need a new Mindset and new Tooling
So, how do I build Agile Topologies?
29
datawire.io
Mindset
30
datawire.io
What’s the new Mindset?
The role of architecture changes
● An architect’s job is to enable developers, not confine them● Architecture teams -> developer infrastructure, developer happiness, …
31
Up Front Thinking Experimentation
datawire.io
Tooling
32
datawire.io
What tools enable experimentation & rapid iteration?
You need three things when you decentralize your business logic:
1. A way to make your topology resilient to software bugs○ Where does my catchall go?○ Minimize impact of bugs reaching production○ Graceful degradation instead of catastrophic failure
2. A robust tool for deployment pipelines that enables both service and system level testing○ Where do your integration tests go?
3. Good visibility into the state of the running system○ Where does my printf go?
33
datawire.io
How do you build these tools?
It’s all about how you manage your topology
● Transition from DNS -> Discovery● Central load balancers -> Smart Endpoints
34
datawire.io
Routing table is (relatively) static
Routing policy is global
Traditional Topology Management
35
Client
DNS
Load Balancer
Serverresolve
traffic
datawire.io
Routing table is updated in realtime
Routing policy is local
Microservices Topology Management
36
Client
DiscoveryServer
heartbeatsroutes
Smart Endpoint
datawire.io
Demo
37
datawire.io
Summary
Useful definitions:
● Microservice: A node in an Agile Service Topology● Microservices: Service Oriented Development
How to build a death star (trifecta of velocity, agility, and organizational scale):
● Mindset: architecture -> experimentation● Tooling: Discovery + Smart Endpoints
With these tools you can get started in half an hour instead of spending six months rearchitecting
38
datawire.io
To learn more
Contact me:
● [email protected]● Twitter: @TheBigLombowski
Jobs
● https://www.datawire.io/careers/○ Python, Java|Scala|Kotlin, Go and Kubernetes Developers Wanted!○ Hiring for DevOps | SRE role!
Try
● https://github.com/datawire/mdk● https://www.datawire.io
39
datawire.io 40