highway to heaven - microservices meetup munich
TRANSCRIPT
![Page 1: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/1.jpg)
Highway to heavenBuilding microservices in the cloudMunich microservices meetup, June 2015
![Page 2: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/2.jpg)
Agenda
Meet AutoScout24Shifting gears How we build our servicesHow we organize ourselves
![Page 3: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/3.jpg)
AutoScout24 - 10.000ftC
![Page 4: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/4.jpg)
Baseline AutoScout24 IT
Highly optimized, but of last decade
IT platform supported growth for >6 years
Microsoft oriented stack
Enterprise IT setup - MTBF over MTTR
Proven agile and lean principles
C
![Page 5: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/5.jpg)
From monolith to microservices...
...via swimlanes
C
![Page 6: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/6.jpg)
New CEOC
![Page 7: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/7.jpg)
Do you attract talent?C
![Page 8: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/8.jpg)
C
![Page 9: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/9.jpg)
Project TatsuW
![Page 10: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/10.jpg)
Microservices
AWS
JVM / Linux
DevOps culture
Product
Five challengesW
![Page 11: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/11.jpg)
Shiny new cut
Cut to the core
Add new business value
New user experience
W
![Page 12: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/12.jpg)
Objectives
Mobile first, user centricSophisticated user experienceReduce time to marketAttract talentsReduce costs in IT production
W
![Page 13: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/13.jpg)
Technical transformationW
![Page 14: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/14.jpg)
Change the wheels while drivingW
![Page 15: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/15.jpg)
Strategic GoalsGoals of the business side
Architectural PrinciplesHigh-Level Principles
Design and Delivery PrinciplesTactical measures
Reduce Time to MarketSpeed, Fast Feedback
Cost Transparency Collect metrics to allow decisions cost vs. value.
Support Data-Driven DecisionsListen to users and validate hypothesis.Provide as many relevant metrics & data as possible.
You build it, you run itResponsibility to run and maintain a product stays with the building team. Fast feedback from live and customers helps us to continuously improve.
Organized around Business CapabilitiesBuild teams around products not projects. Follow the domain and respect bounded contexts. Inverse Conway Maneuver
Shared NothingAvoid shared infrastructure and tight coupling as much as possible. Don’t create the next monolith.
Macro and Micro ArchitectureClear separation. Autonomous micro services within the rules and constraints of the macro architecture.
AWS FirstFavor AWS service over managed service, over self-hosted OSS, over self-rolled solutions.
Data-Driven/ Metric-DrivenCollect metrics from processes and applications. Analyze, alert and act on them.
Eliminate Accidental ComplexityStrive to keep it simple. Focus on essential complexity. No silver bullet.
Event Sourcing and PublishingKeep history of state changes and publish application events.
Autonomous TeamsMake fast local decisions. Be responsible. Know your boundaries. Share findings.
Continuous DeliveryDeliver changes reliable, often and fast.
Infrastructure As CodeAutomate everything: Reproducible, traceable and tested.Immutable servers over snowflake servers.
DevOps CultureDevelopers and Ops work together in collaborative teams as engineers. No silos.
Be BoldGo into production early. Value monitoring over tests. Recover and learn. Optimize for MTTR not MTBF.
Security & Data PrivacySecurity must be first class citizen and everybody’s concern. Keep data-privacy in mind.
C
![Page 16: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/16.jpg)
Principles
Organized around business capabilities
You build it, you run it
Be bold
Macro and micro architecture
Event sourcing and publishing
C
![Page 17: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/17.jpg)
Event Sourcing and data pumps
One way data highway
Event Sourcing - store history of all changes
C
![Page 18: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/18.jpg)
SQS + S3
Kinesis + S3
Kinesis + DynamoDB
SQS + DynamoDB
Proxy + DynamoDB
DynamoDB
EvolutionW
![Page 19: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/19.jpg)
Classified Events - OverviewC
![Page 20: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/20.jpg)
Classified Events - BoundariesC
![Page 21: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/21.jpg)
DynamoDB as the new Atom Feed
Learn to be come a cloud native
DynamoDB as service contract
The return of the integration database?
The price is right?
W
![Page 22: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/22.jpg)
DynamoDB as the new Atom Feed
Learn to be come a cloud native
DynamoDB as service contract
The return of the integration database?
The price is right?
Event Sourcing rocks
C
![Page 23: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/23.jpg)
Watchlist evolution
DynamoDB as JSON document store
Scheduled user journey test
Capacity exceeded
Requests throttled
Latency spikes for real users
C
![Page 24: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/24.jpg)
From documents to events
Refactoring toward deeper insight
From CRUD to sync
Offline first
Writes are expensive
USD 20 over USD 500
C
![Page 25: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/25.jpg)
How many layers to estimate a price
Evolving architecture
R backend and Play application
Single Play application
Play backend and Play web server
Long feedback cycles
Frequency of change
W
![Page 26: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/26.jpg)
Shared infrastructure
Shared nothing
Availability over shared nothing
Convenience offerings
No side effects
Fast local decisions over committees
W
![Page 27: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/27.jpg)
Shared nothing?
How many environments?Use over re-useRe-use only after hardeningHow to share
Copy n’paste, OSS, library
W
![Page 28: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/28.jpg)
Unified log processing
AWS EC2AWS Kinesis
Event Streams
Unstructured Log Stream
Service A
Service B
System Log
ELB Log Stream
ELB Access Logs S3 Collector
Collector
Collector
ElasticSearch
Kibana
JSON
JSON
JSON
Collector
in progress
System Metrics Stream
Collectd
???
AWS CloudWatch
C
![Page 29: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/29.jpg)
Unified log processingC
Optimised for imaginary use cases?
JVM metrics ... err not used
Application events over log files
What are the options?
![Page 30: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/30.jpg)
How to build autonomous teams
Do not fall back into old behavioursBeware of Mandelbrot teamsPager duty so that you run itPart-time ops not workingNot all T-shapes are the same
WolfCW
![Page 31: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/31.jpg)
Infrastructure guild
Agree on things to do
Share learnings
Delegate implementation to teams
Empty backlog should be normal
How about infrastructure product teams?
Mind the Shirky Principle
C
![Page 32: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/32.jpg)
Team over infrastructure guildC
![Page 33: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/33.jpg)
Focus sliders
Product over platform
platform
product
time
W
![Page 34: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/34.jpg)
Focus sliders (cont.)
Cash stack meets shiny new stack
One company
Lights on in cash stack
Feature freeze
Where to build new features?
Ease of integration helps business people
C
![Page 35: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/35.jpg)
[email protected] @[email protected] @wolfwolf
![Page 36: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/36.jpg)
AttributionsBlue sky, white-gray clouds by nature protector Natubico, www.vivism.info [CC BY-SA 3.0]
http://commons.wikimedia.org/wiki/File%3ABlue_sky%2C_white-gray_clouds.JPGA Danish Perspective by NASA [Public domain] http://commons.wikimedia.org/wiki/File%3AA_Danish_Perspective.jpg
http://commons.wikimedia.org/wiki/File%3ANASAComputerRoom7090.NARA.jpgGREG
EINRADAmazon16 by Neil Palmer/CIAT [CC BY-SA 2.0] https://www.flickr.com/photos/ciat/5641594952
BERGSTEIGERBarber in Cameroon by James Emery from Douglasville, United States (Daddy Joe_1355) [CC BY 2.0]
http://commons.wikimedia.org/wiki/File%3ABarber_in_Cameroon.jpgWide objectives by Kivela (Own work) [Public domain] href="http://commons.wikimedia.org/wiki/File
%3AWide_objectives.jpgTransformer Fire Barrier by GerryS1 (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File
%3ATransformer_Fire_Barrier.jpg
![Page 37: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/37.jpg)
Attributions (cont)Alonso Renault Pitstop Chinese GP 2008 by Bert van Dijk (Pitstop F1 ING Renault) [CC BY-SA 2.0]
http://commons.wikimedia.org/wiki/File%3AAlonso_Renault_Pitstop_Chinese_GP_2008.jpgPrinciple of Panchasheel by Prakash Adhikary (Own work) [CC BY 3.0] http://commons.wikimedia.org/wiki/File
%3APrinciple_of_Panchasheel.JPGTraffic Jam by Doo Ho Kim [CC BY-SA 2.0] https://www.flickr.com/photos/titicat/3049591547
Pellets by The original uploader was Richard Mayer at German Wikipedia [GFDL or CC-BY-SA-3.0] http://commons.wikimedia.org/wiki/File%3APellets.jpg
Pipes and Valves by Uwe Hermann [CC BY-SA 2.0] https://www.flickr.com/photos/73628542@N00/6272975359Size variation in Coccinella undecimpunctata (2127991716) by Gilles San Martin from Namur, Belgium [CC BY-SA 2.0]
http://commons.wikimedia.org/wiki/File%3ASize_variation_in_Coccinella_undecimpunctata_(2127991716).jpgMille crêpe by Laitr Keiows (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File%3AMille_cr
%C3%AApe.jpgCountry Energy power line replacement 01 by Bidgee (Own work) [CC BY-SA 3.0]
http://commons.wikimedia.org/wiki/File%3ACountry_Energy_power_line_replacement_01.jpg
![Page 38: Highway to heaven - Microservices Meetup Munich](https://reader030.vdocuments.us/reader030/viewer/2022032505/55c6d3eebb61eb02438b45e4/html5/thumbnails/38.jpg)
Attributions (cont)Sharing Sucks (4536747557) by eyeliam from Portland, United States [CC BY 2.0]
http://commons.wikimedia.org/wiki/File%3ASharing_Sucks_(4536747557).jpg7Line 9184 (8263568241) by Metropolitan Transportation Authority of the State of New York (7Line_9184 Uploaded by
tm) [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3A7Line_9184_(8263568241).jpgEngland rugby team 1905 by Russell & Sons (The Graphic) [Public domain or Public domain]
http://commons.wikimedia.org/wiki/File%3AEngland_rugby_team_1905.jpgWandergeselle by Sigismund von Dobschütz [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File
%3AWandergeselle_02.JPGFaber-Rechenschieber 5304 by User:Karl Gruber (Own work) [CC BY-SA 4.0] http://commons.wikimedia.org/wiki/File
%3AFaber-Rechenschieber_5304.JPGWheel clamps Texas by Richard Anderson from Denton, United States (Boots.) [CC BY-SA 2.0]
http://commons.wikimedia.org/wiki/File%3AWheel_clamps_Texas.jpgGuadalupeNOLA15Oct07Thanks by Infrogmation of New Orleans (Photo by Infrogmation) [GFDL or CC BY-SA 3.0]
http://commons.wikimedia.org/wiki/File%3AGuadalupeNOLA15Oct07Thanks.jpgAtariBasic by Calin99 (Own work) [GPL] http://commons.wikimedia.org/wiki/File%3AAtariBasic.png