surviving in a microservices environment

Surviving In A Microservices Environment

Steve Pember CTO, ThirdChannel

Gr8Conf US, 2017

@svpember

@svpember

Microservice Blog Posts & Presentations• Microservices: Yay!

• Microservices: Boo!

• Smashing The Monolith (and here’s how we did it)

• Microservices + Technology X

• Microservices + Methodology Y

Microservices have many lessons to offer

What even are Microservices?

@svpember

Why Choose Microservices?• Reduce Coupling!

• Right Tool for the Job

• Continuous Delivery

• Smaller codebases are easier to reason about

• Easy to replace old services

• Efficient Scaling

• Can move quickly (once you’re up and running)

Help?

@svpember

Microservice Topics• Infrastructure

• Architecture

• Team Communication

• Miscellaneous Advice

@svpember

Microservice Topics• Infrastructure

@svpember

Infrastructure• How do we manage the logs?

Centralized Logs are your #1 Priority

@svpember

@svpember

@svpember

@svpember

@svpember

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

You’d best be monitoring your platform.

–Johnny Appleseed

“Type a quote here.”

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

@svpember

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

@svpember

@svpember

@svpember

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

• Do we have any coding conventions?

Check Style? Coverage?

@svpember

@svpember

@svpember

@svpember

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

• Do we have any coding conventions?

• Can I generate a Service template?

wget https://github.com/thirdchannel/base-template/archive/release.zip

(This is not a real url, but you get the idea)

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

• Do we have any coding conventions?

• Can I generate a Service template?

• How do we share code?

Whoops! Gotcha!

@svpember

Sharing Code• It’s OK to reimplement functionality within each service

• Setup your own internal Artifactory!

• DO share infrastructure libraries (e.g. communications)

• NEVER share Domain or business logic libraries

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

• Do we have any coding conventions?

• Can I generate a Service template?

• How do we share code?

• How do we manage our (multiple) environments?

@svpember

Test & Develop in Isolation

@svpember

Infrastructure• How do we manage the logs?

• How about metrics / telemetry?

• How do we deploy our code?

• How/where are our builds done?

• Do we have any coding conventions?

• Can I generate a Service template?

• How do we share code?

• How do we manage our (multiple) environments?

• Do our Devs get time to work on Infrastructure?

@svpember

Microservice Topics• Infrastructure

• Architecture

@svpember

Architecture• Do we have an overall design vision?

Does Anyone Know What We’re Doing?!

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

@svpember

@svpember

Be Careful When Choosing New Tech

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

Integration Tests are the Best Tests

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

• How do we test the platform as a whole?

@svpember

Test Environment• Run periodically (e.g. nightly)

• Each service generates fixture data

• Service data reset after EACH test

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

• How do we test the platform as a whole?

• How do our services communicate?

HTTP vs Async Events

@svpember

HTTP• Well Established

• Built In libraries

• Existing structure for response codes (2**, 4**, 5**, etc)

• Synchronous

• Increases coupling

• Requires services to know which others require their data

• Has dependency on Service Discovery mechanism

• Susceptible to Data Loss

@svpember

Events• Asynchronous

• Pub-sub / Fire and forget

• Loose Coupling

• Prevents Data Loss

• Allows for Reactive systems

• No existing structure for response error handling

• Dependency on Message Broker technology

• Can be difficult for Junior folks to understand

Ensure you have Circuit Breaker or a Dead Letter Mechanism

@svpember

@svpember

<3 RabbitMq• Intelligent broker; dumb consumers

• Highly nuanced & robust routing

• Backpressure

• Dead letters /retries

• Message Ordering

• Multiple deliveries

• Ack / Nack/ Reject … re-enqueue

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

• How do we test the platform as a whole?

• How do our services communicate?

• How/where is our data persisted?

@svpember

Locality of Reference• Spatial: “How close together is our data?”

@svpember

Locality of Reference• Spatial: “How close together is our data?”

• Temporal: “How often is our data accessed?”

@svpember

@svpember

Beware the ‘Mini-Monolith’

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

• How do we test the platform as a whole?

• How do our services communicate?

• How/where is our data persisted?

• Do we follow an overall architectural style?

Make Sure Everyone Is Aware

Applies to Both Intra- and Inter-Service

It all Started With =>

@svpember

Domain Driven Design• Ubiquitous Language

• Entities

• Aggregates

• Bounded Contexts

• No Direct communications across Boundaries

@svpember

@svpember

… And CQRS

@svpember

@svpember

Allows for interesting Architectures

While we’re at it, go Reactive

@svpember

Architecture• Do we have an overall design vision?

• What technologies do we use?

• How much freedom do we have in choosing new technologies?

• How do we test an individual service?

• How do we test the platform as a whole?

• How do our services communicate?

• How/where is our data persisted?

• Do we follow an overall architectural style?

• When do we create new services?

DDD to the rescue

@svpember

New Service?• Initially: design platform around Bounded Contexts, create services from

inner Contexts

• Don’t create services ‘just because’

• Do make an effort to identify boundaries

• Ensure a service has/will have proper context boundaries before creating

• If two services need to communicate synchronously or frequently, good candidates for MERGING

# of services < # of developers

@svpember

Microservice Topics• Infrastructure

• Architecture

• Team Communication

@svpember

Team Communication• How are our teams structured?

Conway’s Law is REAL

–Johnny Appleseed

“Type a quote here.”

DBA

Engineers

QA

UX

@svpember

@svpember

Team Communication• How are our teams structured?

• Who owns which Service?

Teams should be Owners

@svpember

Team Communication• How are our teams structured?

• Who owns which Service?

• What’s our process for merging code?

@svpember

Team Communication• How are our teams structured?

• Who owns which Service?

• What’s our process for merging code?

• What’s our process for releasing code?

@svpember

@svpember

Team Communication• How are our teams structured?

• Who owns which Service?

• What’s our process for merging code?

• What’s our process for releasing code?

• What’s our process for monitoring code in production? Dealing with bugs?

@svpember

–Johnny Appleseed

“Type a quote here.”

@svpember

Team Communication• How are our teams structured?

• Who owns which Service?

• What’s our process for merging code?

• What’s our process for releasing code?

• What’s our process for monitoring code in production? Dealing with bugs?

• Do we add more process if something goes wrong?

Be Reluctant to Add New Process

High Freedom / High Responsibility

@svpember

Microservice Topics• Infrastructure

• Architecture

• Team Communication

• Miscellaneous Advice

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

Any idea what these

do?

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

• New Feature -> walk backwards from what the user will see

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

• New Feature -> walk backwards from what the user will see

• Release when a feature is ready

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

• New Feature -> walk backwards from what the user will see

• Release when a feature is ready

• If a service A has another service B as a dependency => release B first

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

• New Feature -> walk backwards from what the user will see

• Release when a feature is ready

• If a service A has another service B as a dependency => release B first

• How to bootstrap a new service?

@svpember

Miscellaneous Advice• Don’t get cute with the naming of services

• New Feature -> walk backwards from what the user will see

• Release when a feature is ready

• If a service A has another service B as a dependency => release B first

• How to bootstrap a new service?

• Don’t release Friday afternoons

In Summary

Most of the Challenge is In infrastructure

<3 Microservices

Please share your survival tips!

Any Questions? Anything I missed?

Thank you!

@svpember

@svpember

Images• Bad Odds (Jon Snow): https://www.reddit.com/r/gameofthrones/comments/4owm95/s6e9_megathread_gifwallpaperscreenshot_requests/

• Say Anything / John Cusak: http://www.glamour.com/story/happy-birthday-john-cusack-and

• Wilderness Survival Guide: https://www.kobo.com/us/en/ebook/the-wilderness-survival-guide-1

• XKCD: Code Review: https://xkcd.com/1513/

• Monolithic vs Microservices: @alvaro_sanchez

• Picard FacePalm: https://www.flickr.com/photos/30418788@N03/8381426895

• Council of Ricks: https://www.reddit.com/r/rickandmorty/comments/3exy00/it_looks_like_there_might_some_sort_of_council_of/

• Drago (break you): https://giphy.com/gifs/GWD5nSpiHxs3K

• Wrecking Ball: https://www.flickr.com/photos/rhysasplundh/5202454842/in/photostream/

• Rocky Stairs: http://pixgood.com/rocky-stairs.html

• Technical Difficulties: https://www.youtube.com/watch?v=EC15BmzsdhM

• Grafana example: https://i.imgur.com/KOqcD6L.png

• Zipkin example: https://blog.buoyant.io/2016/05/17/distributed-tracing-for-polyglot-microservices/

• Assembly Line: http://www.solidsmack.com/culture/humans-need-apply-new-short-film-explores-future-robots-manufacturing-automation/

@svpember

Links• Eric Evans: DDD and Microservices: https://www.youtube.com/watch?

v=yPvef9R3k-M