4finance microservices 2015.06 prague jug
TRANSCRIPT
@mgrzejszczak
Microservices at 4financeIT
08/06/2015, Prague
@mgrzejszczak
About meSoftware Architect atAuthor of "Mockito Instant", "Mockito Cookbook" booksCo-author of the Groovy core’s @Builder AST Co-founder of the Warsaw Groovy User GroupCo-author of "micro-infra-spring" libCo-author of “spring-cloud-zookeeper” and “spring-cloud-sleuth”
Twitter: @MGrzejszczakBlog: http://toomuchcoding.blogspot.comHomepage: http://marcin.grzejszczak.pl
@mgrzejszczak
Agendashort intro to microserviceshow we did it in 4financeITmicroservice challenges
@mgrzejszczak
Agendashort intro to microserviceshow we did it in 4financeITmicroservice challenges
@mgrzejszczak
Conway’s Law
Conway, Melvin E. (April 1968), How do Committees Invent?, Datamation 14 (5): 28–31, retrieved 2009-04-05
Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations
— M. Conway
@mgrzejszczak
A single codebase
Conway’s Law in practice
@mgrzejszczak
A single codebase
Conway’s Law in practice
@mgrzejszczak
Conway’s Law in practiceConcept:
one team
two countries
one codebase
@mgrzejszczak
Conway’s Law in practiceReality:
two teams
two countries
one codebase
@mgrzejszczak
Conway’s Law in practiceEffect:
two different solutions
solving same stuff
one codebase
@mgrzejszczak
Conway’s Law - siloed teams
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Conway’s Law - cross functional teams
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Business flow
AccountingBack officeFront office
@mgrzejszczak
Common problematic code flow
Dto Service
Impl Entity
Accounting Accounting
Accounting Accounting
Front office Front office
Front office Front office
Back office Back office
Back office Back office
monolith
many programmers
big organization
@mgrzejszczak
Common problematic code flow
Looks familiar?
http://www.foodnetwork.com/topics/spaghetti-recipes.html
@mgrzejszczak
Code flow
REST REST
Autonomous
Business oriented
PolyglotLeightweight
Front officebounded context
JARS
Back officebounded context
JARS
Accountingbounded context
JARS
@mgrzejszczak
Small and lightweight
single application as a suite of small applications
each running in its own process
communicating with lightweight mechanisms
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Business oriented
http://www.ops.fhwa.dot.gov/publications/ics_guide/
@mgrzejszczak
Business orientedbuilt around business capabilities
products not projects
cross business not technical boundaries
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Polyglot
@mgrzejszczak
Autonomous
@mgrzejszczak
Autonomous
independently deployable / upgradeable
deployed via fully automated deployment machinery
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Autonomous
bare minimum of centralized management (do not share your data, minimum set of standards)
embrace failure (resilience - recover fast)
extract from http://martinfowler.com/articles/microservices.html
@mgrzejszczak
Deployment - provisoning (Ansible)
http://labs.qandidate.com/blog/2013/11/21/installing-a-lamp-server-with-ansible-playbooks-and-roles/
@mgrzejszczak
Deployment - pipeline (Jenkins)
@mgrzejszczak
Deployment - job scheduling (Rundeck)
http://rundeck.org/
@mgrzejszczak
Fail fast
@mgrzejszczak
Agendashort intro to microserviceshow we did it in 4financeITmicroservice challenges
@mgrzejszczak
Explain one simple truth
@mgrzejszczak
Microservices are complex!who can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
@mgrzejszczak
So we wrote a framework...
@mgrzejszczak
… having management on our side
@mgrzejszczak
Microservices are complex!who can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
The big picture
@mgrzejszczak
The big picture
HEY GUYS LET’S TELL ZOOKEEPER WHERE WE ARE
@mgrzejszczak
The big picture
MY IP: …MY PORT: ….
@mgrzejszczak
The big pictureI WOULD LOVE TO TALK TO SERVICE X
@mgrzejszczak
The big pictureWHERE IS SERVICE X ? GIVE ME ANY INSTANCE
@mgrzejszczak
The big pictureTHERE YOU GO!IP: ….PORT: ….
@mgrzejszczak
The big pictureGREAT NOW I CAN SEND HIM A REQUEST
@mgrzejszczak
Microservice descriptor{
"pl": {
"this": "some/path/my-name",
"dependencies": {
"an-alias": {
"path": "some/path/his-name"
}
}
}
@mgrzejszczak
Microservice descriptor{
"pl": {
"this": "some/path/my-name",
"dependencies": {
"an-alias": {
"path": "some/path/his-name"
}
}
}
REALM
@mgrzejszczak
Microservice descriptor{
"pl": {
"this": "some/path/my-name",
"dependencies": {
"an-alias": {
"path": "some/path/his-name"
}
}
}
MY NAME IN ZOOKEEPER
@mgrzejszczak
Microservice descriptor{
"pl": {
"this": "some/path/my-name",
"dependencies": {
"an-alias": {
"path": "some/path/his-name"
}
}
}
HIS NAME IN ZOOKEEPER
@mgrzejszczak
Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics
@mgrzejszczak
How can I communicate easily?serviceRestClient.forService('an-alias')
.post()
.onUrl('/api/whatever/123')
.body('''{"some":"json"}''')
.withHeaders()
.contentTypeJson()
.andExecuteFor()
.anObject()
.ofType(String)
@mgrzejszczak
Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
Where do I take my props from?
@mgrzejszczak
Where do I take my props from?
@mgrzejszczak
Where do I take my props from?
@mgrzejszczak
Where do I take my props from?
@mgrzejszczak
Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
your microservice publishes its API by itself!
What’s my API?
@mgrzejszczak
What’s my API?
@mgrzejszczak
What’s my API?
RESTCONTROLLERS
@mgrzejszczak
What’s my API?
METHOD TYPES
@mgrzejszczak
What’s my API?
METHOD URLS METHOD NAMES
@mgrzejszczak
What’s my API? - Try me out!
@mgrzejszczak
What’s my API? - Try me out!
ENTER PARAMETERS
SEND A REQUEST
@mgrzejszczak
Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
How to be part of a bigger system: logs?You’re not going to grep your logs from 10 servers, right?
@mgrzejszczak
Correlation ID
a header value in a request
in each microservice this header is the same for the same business action
you can see a flow of messages between microservices in a chronological order
@mgrzejszczak
How to be part of a bigger system: logs?
FIRST SERVICE
REQUEST
No correlationId
CorrelationId set to X
CorrelationId set to X
RESPONSE
SECOND SERVICE
REQUEST
CorrelationId set to X
CorrelationId set to X
RESPONSE
ANOTHER SERVICE
REQUEST
CorrelationId set to XCorrelationId
set to X
RESPONSECorrelationId set to X
YET ANOTHER SERVICE
REQUEST
CorrelationId set to XCorrelationId
set to X
RESPONSE
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
TIMESTAMP IN ISO8601 FORMAT
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
LOGGING LEVEL
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
CORRELATION ID
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
THREAD NAME
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice application
CLASS FULLY QUALIFIED NAME (SHORTENED)
@mgrzejszczak
How to be part of a bigger system: logs?Log example:
2014-09-10 14:34:40.448+0200 |
INFO |
538898f2-543f-48a3-ad9e-3d2961b2b73d |
qtp326276239-140 |
c.o.c.v.w.a.AbstractBackofficeApplication |
Initializing new backoffice applicationLOG MESSAGE
@mgrzejszczak
How to be part of a bigger system: logs?
@mgrzejszczak
How to be part of a bigger system: logs?
SAME CORRELATION ID
@mgrzejszczak
How to be part of a bigger system: logs?
DIFFERENT MICROSERVICES
@mgrzejszczak
View the logs in Kibana
@mgrzejszczak
Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics
@mgrzejszczak
How do I expose my metrics?just have to update your metrics:
// treat this code more of a pseudocode ;)Counter transferBalanceRequests = metricRegistry.counter(‘transfer.requests.balance’)
void doSomethingUponTransferBalanceRequest() {
// do sth with the request for transfer balance…
// and update the counter metrictransferBalanceRequests.inc()
}
@mgrzejszczak
How do I view my metrics?
@mgrzejszczak
How do I view my metrics?
@mgrzejszczak
Ok, so how do I start?But, but, but… it sounds so complicated!
@mgrzejszczak
Perform 3 steps...You just have to:
add ONE dependency - 4finance micro-infra-spring to your project
add ONE annotation - @EnableMicroservice
create ONE file - microservice.json
@mgrzejszczak
One dependency with Spring Boot
And for Spring-Boot….
@mgrzejszczak
It’s even easier with Spring Boot!You just have to:
add ONE dependency - 4finance micro-infra-spring-boot to your project
create ONE file - microservice.json
@mgrzejszczak
Ok, so how do I start?
Question:what should I do to start working on a new microservice?
@mgrzejszczak
Ok, so how do I start?
Answer:
Clone a repo
@mgrzejszczak
Ok, so how do I start?
Without UI:https://github.com/4finance/boot-microservice
With UI:https://github.com/4finance/boot-microservice (branch boot-microservice-gui)
@mgrzejszczak
Ok, so how do I start?those repos:
contain completely set up microservice
are written in Groovy and built with Gradle
are JDK8 compliant
contain an example of a business scenario
@mgrzejszczak
Where are we?
Majority of microservice related features are there…
We are moving our stuff to Spring-Cloud
We have automated / standardized plenty of things already
NOW
Move more stuff to Spring-Cloud
Leave 4financeIT related stuff in micro-infra-spring
Automate more
Standardize more stuff
NEXT
New templates for other JVM languages
Scale dynamically our production servers (Mesos, Marathon etc.)
Sky is the limit...
FUTURE
@mgrzejszczak
Working closely with Pivotal
@mgrzejszczak
Working closely with Pivotal
@mgrzejszczak
3 steps to eternal fame1. Get account at Github
2. Pick an issue, fix it and create a pull request
3. Once merged profit from eternal fame
@mgrzejszczak
The setup - overview
@mgrzejszczak
Agendashort intro to microserviceshow we did it in 4financeITmicroservice challenges
@mgrzejszczak
Code reusedo not abstract everything
sometimes copy paste gives you code decoupling
no - copy paste is not a solution to all problems ;)
@mgrzejszczak
Too many technology stackspick a right tool for the job but don’t exaggerate
do you really want to support a microservice written in Brainfuck or Whitespace?
@mgrzejszczak
Managing dozens of servicesmonitoring, monitoring, monitoring!
Zabbix, Nagios, Kibana, Graphite etc.
do not write nanoservices - who will support it?
@mgrzejszczak
Microservice envythose guys wrote a microservice and I’m diving in the abyss of a monolith
I will write a microservice per feature!
one bounded context - 100 microservices with duplicated data - WRONG!
@mgrzejszczak
Global changes (big features)you’ll have to work piece by piece and communicate
you can use feature toggling and API versioning
easier A/B testing - load balancing via Service Discovery
@mgrzejszczak
Testingcontract changes - how to test that?
how do I know what’s the API of other services?
Consumer Driven Contracts to the rescue!Accurest - https://github.com/Codearte/accurestPact - https://github.com/DiUS/pact-jvm
@mgrzejszczak
Management issueshave to invest time and effort to build foundations
have to invest in infrastructure and devops
feature delivery pace will decrease for some time
@mgrzejszczak
Questions?