from no services to microservices
TRANSCRIPT
![Page 1: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/1.jpg)
From No Services to Microservices
A journey of a FinTech application into the cloud, starting as a monolith and growing to MicroServices. Why going functional is so good for finance
technology, and how Scala helped us build a better application.
João Cavalheiro - [email protected]
![Page 2: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/2.jpg)
Fintech in a Nutshell• Fintech is mostly about using technology to
make financial services more efficient.
• Fintech companies are in many cases startups, and have the purpose of disrupting existing financial systems.
• Popular areas are payments & lending, blockchain and market monitoring and tracking
![Page 3: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/3.jpg)
Developing Fintech Software
• Complex business models
• Data security
• Code stability and adaptability
• Quality of service (QOS)
Your programming language, team skills and toolkit must be able to deal with all the above
![Page 4: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/4.jpg)
Avoid• Non-Agile development processes
• Growing the team too quickly
• Start writing code before you understand the business model
![Page 5: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/5.jpg)
Go Functional
• Intent over implementation
• Immutability - correct code by design
• Transparent concurrency
• Use conventions and patterns
![Page 6: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/6.jpg)
Use Scala (you should)
• Focus on easiness of refactoring instead of elite-looking code
• Do not abuse implicits!
• Part of our team was moving from Java, so..
![Page 7: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/7.jpg)
Leverage on Java Ecosystem
• Lots of useful Java libraries out there. You can use them in your Scala project
• Don’t feel guilty for using OO design in your first days in Scala
• All JVM concepts can be applied
![Page 8: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/8.jpg)
Why Starting with a Monolith can be Good
• Fintech companies sometimes grow really fast
• The business model is a moving part
• Logical option is to invest time on a carefully designed monolith
• It would be simpler to move to microservices later
![Page 9: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/9.jpg)
Monolith Design
• Logical option is to invest time on a carefully designed monolith
![Page 10: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/10.jpg)
Monolith Design
• Should be adjusted to business and product requirements
• Sometimes is just better to focus initially on your monolith functionality and gradually peel off microservices at the edges
• You can also sacrifice your currently architecture and go for a complete system redesign
![Page 11: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/11.jpg)
Our Application• Software-as-a-Service
• Core micro-finance system
• Combines administration, security, accounting, loan portfolio management and deposit tracking
• Currently in production with 20 paying customers (MFI banks) and over 400.000 end clients.
![Page 12: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/12.jpg)
Initial Tech Stack• Using Scala and Play! framework
• PostgresSQL database
![Page 13: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/13.jpg)
The Data Model
• Data model is event-based, every operation over an account is an event
• Repayment schedules are stored in the database, one row per instalment
• We also keep track of all non-transaction actions on the system
![Page 14: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/14.jpg)
The Hit on The Wall• We reached 200.000 lines of code on our
monolith play! Application.
• It was getting hard to keep development under control
• Bug density was increasing with the product complexity
• And we had a few demanding requirements…
![Page 15: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/15.jpg)
Business talks• Operations, including transactions, can be
backdated and reverted, any moment in time
• It is in fact, possible to change the past
• We could not rely on a simple snapshotting model. The snapshots in the past could change, and they would affect all the subsequent snapshots as well.
![Page 16: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/16.jpg)
Headed for Microservices
• Services that are themselves very simple, focusing on doing one thing well
• Use different languages/frameworks and tools for each service
• Integrate new team members much easier
![Page 17: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/17.jpg)
The First Microservice• Our immediate need was to find a dynamic
snapshot solution to accommodate transaction backdating and have the updated detailed status of each account, for at least the last 30 days
• This could never operate on top of current application stack and database
• We had a great candidate for the first microservice
![Page 18: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/18.jpg)
Landscape
![Page 19: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/19.jpg)
CQRS• The concept at the heart of it is to use a different
model to create/update information than the model you use to read information
• It adds complexity, but we were going to use it only on specific portions of the system
• It allowed us to define separate materialised query views that implement the aggregation strategy for account transactions
![Page 20: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/20.jpg)
The outcome
• The view model It’s eventually consistent, but reports are in most cases statistical, and therefore “almost-real-time” is acceptable
• Denormalized = Efficiency lookup
• All seemed great!
![Page 21: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/21.jpg)
The Dark Side
![Page 22: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/22.jpg)
The doubts
• Can we share a single database?
• How do we manage dependencies?
• Multiple code bases, multiple git repos & branching models
![Page 23: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/23.jpg)
The Next Service and experience with AKKA HTTP
• The need to add document management to our product and to do it with a scalable solution
• We needed something lighter than Play!
• Spray is no longer actively developed
• AKKA-http?
![Page 24: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/24.jpg)
AKKA-http
• It was a moving target. Api stability was not proven
• It doesn’t work right out of the box
• Should we fall back to Play?
![Page 25: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/25.jpg)
AKKA-http
• With Akka-http you can choose and control each component of your solution
• Add new components with a smaller risk
• Easier to find and fix problems
• But, if you are short on time, under pressure, and you don’t know Spray or Akka-http at all, consider Play
![Page 26: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/26.jpg)
It worked in DEV…
• AKKA-http proved to be a good solution
• Microservices were helping
• Performance was improved
• But, someone was not happy…
![Page 27: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/27.jpg)
“… OPS problem now”• Microservices come with operations overhead:
• Monitoring
• Deployment
• Security
• Seriously consider to have DEVOPS
![Page 28: From no services to Microservices](https://reader034.vdocuments.us/reader034/viewer/2022051706/58f9b9e21a28abda618b4569/html5/thumbnails/28.jpg)
What we Learned• Moving to microservices is not a silver bullet,
and the increased complexity means there are a lot of areas which need to be carefully considered
• All in all, microservices are not a free lunch, but in many cases it’s a worthwhile investment
• Scala made it easier!