![Page 1: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/1.jpg)
MODERN SOFTWARE ARCHITECTURES
Ricard Clau PHPUKConference 2015
![Page 2: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/2.jpg)
HELLO WORLD• Ricard Clau, born and grown up in Barcelona
• Server engineer at Another Place Productions
• Not doing much PHP these days
• Open-source contributor and occasional speaker
• Twitter (@ricardclau) / Gmail [email protected]
![Page 3: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/3.jpg)
WE WILL TALK ABOUT• Technical teams and Software architecture evolution
• SOA, MicroServices, Distributed systems
• Real time stream processing, data storage
• Different languages for different problems
• Where does PHP fit in this new world?
![Page 4: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/4.jpg)
WHY THIS TALK?• The way we build applications has changed
• Lots of people over-engineer for no reason
• An industry full of stupid trends and hypes
• A bit of a rant talk
![Page 5: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/5.jpg)
APPLICATIONS EVOLUTION
![Page 6: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/6.jpg)
LOTS OF PROJECTS START…
• Classic LAMP Stack
• PHP monolithic codebase
• MySQL (or any other RDBMS)
• Millions of £ generated
• Nothing fundamentally wrong!
![Page 7: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/7.jpg)
TRAFFIC STARTS TO GROW…• Nginx / Varnish
• Cache servers
• NoSQL databases
• Search text engines
• Queue systems
![Page 8: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/8.jpg)
BUT SOMETIMES…• Over-engineered systems for no reason
• Hard to maintain and develop with
• NIH Syndrome, reinventing the wheel
• New problems from complex design
• Ultimately harming the company!
![Page 9: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/9.jpg)
LIVING IN THE CLOUD• Pay only for what you use
• Save money in operations
• Embrace failure every piece can fail
• Provisioned vs on-demand models
• Many people waste a lot of money
![Page 10: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/10.jpg)
TEAMS EVOLUTIONThere is always some resistance!
![Page 11: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/11.jpg)
DEVOPS PHILOSOPHY• Infrastructure as code
• Automation
• Communication
• Responsibility
• No more Dedicated Ops?
![Page 12: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/12.jpg)
FULL-STACK DEVELOPERS• Very useful in startups
• Impossible to be good at everything
• It is good to have traversal skills
• Look beyond PHP!
![Page 13: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/13.jpg)
ARCHITECTS / PLATFORM TEAMS• Not a big fan myself
• They need to be building features
• “Wisdom committee” idea
• Founding engineers may NOT be the best suited for these roles
![Page 14: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/14.jpg)
DISTRIBUTED SYSTEMSMessage passing, embracing failure
![Page 15: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/15.jpg)
DEFINITION AND MOTIVATIONS• A collection of independent computers that appears to its users as a
single coherent system thanks to a middleware
• Concurrency, horizontal scalability, resilience, fault tolerance…
• Some things don’t fit in just one box, cannot be computed with the biggest instance available or users are spread all over the world and need low latencies
![Page 16: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/16.jpg)
DISTRIBUTED SYSTEMS ARE HARDDon’t blindly follow the trends!
![Page 17: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/17.jpg)
CHOOSE YOUR STORAGE: CHAOS!
![Page 18: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/18.jpg)
CAP THEOREM• A shared-data system cannot guarantee simultaneously:
• Consistency: All clients have the same view of the data
• Availability: Each client can always read and write
• Partition tolerance: The system works well even when there are network partitions
![Page 19: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/19.jpg)
“During a network partition,
a distributed system must
choose between either
Consistency or Availability”
![Page 20: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/20.jpg)
Consistency
Availability
Partition Tolerance
Mostly RDBMS(MySQL, PostgreSQL,
DB2, SQLite…)
Special nodes (Zookeeper, HBase, MongoDB, Redis…)
All nodes same role (Cassandra, Riak, DynamoDB…)
![Page 21: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/21.jpg)
SOA & MICROSERVICESDoing one thing, and doing it right… right?
![Page 22: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/22.jpg)
SOA PRINCIPLES• Self-contained units of functionality
• Share contract and schema
• Can evolve independently, be reused
• We need some orchestration
• Stop thinking applications, think business processes
![Page 23: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/23.jpg)
WHY DOES SOA USUALLY FAIL?• It’s hard to explain the business value
• Strong impact in the organisation
• Sometimes, we do SOA “on the cheap”
• People not skilled / experienced enough
• New complexities are added
![Page 24: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/24.jpg)
MICROSERVICES• Small units of software following SRP
• Replaceable, upgradeable, independent
• Encapsulated, composable, client friendly
• Fast startup / shutdown, testable
• SOA integrates different apps, Microservices architect a single app
![Page 25: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/25.jpg)
HOW MICRO SHOULD THEY BE?• No rule of thumb
• Some people abuse from nanoservices
• A microservice is NOT a function
• Many services add complexity and inter-call latencies
![Page 26: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/26.jpg)
ORCHESTRATION• Responsible for interoperation
• Also to make a client perceive a single system
• Can become extremely complex
• Proper frameworks need to emerge
• Some people move their complexity here!
![Page 27: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/27.jpg)
SERVICE DISCOVERY• Directory of services, registering and finding them
• Sounds pretty much like DNS, right?
• Hard to make DNS highly available
• DNS was mostly designed for standard ports
• DNS was not optimised for real-time changes
![Page 28: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/28.jpg)
TESTING MICROSERVICES• Most of the times, unit testing adds small value
• You need to do integration tests and load tests
• Testing network connectivity problems can get tricky
• Most of the times you need end-to-end tests! Hard to maintain!
![Page 29: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/29.jpg)
COMMUNICATION BETWEEN SERVICES
AsyncWorkers
OfflineWorkers
Queue
Message Bus
ProtobufThrift
JSON
![Page 30: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/30.jpg)
Proxies
ELB
Server
Database
Booking Cinema
Validation
Audit
Stats
Seating
Fraud
Payment
Users
Mails
Analytics
![Page 31: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/31.jpg)
TIMEOUTS AND RETRIES: MADNESS!• Should we always retry on failure? Idempotence problem
• Servers down? Database locks? Database down? Network issue?
• We can have 2 timeouts: establishing connection but also waiting for a response. And we have a chain of those!
![Page 32: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/32.jpg)
OTHER PROBLEMS• One microservice can take all your system down
• Health-checks, constant monitoring and instrumentation
• Availability goes down in a chain of calls (0.99^3 = 0.97)
• It’s up to the team to decide in every situation / problem if breaking into microservices is worth the hassle
![Page 33: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/33.jpg)
DATA ANALYSISAll businesses need it!
![Page 34: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/34.jpg)
QUERY VS PROCESSING• SQL is great because we can query by any field
• There is no standard in NoSQL databases
• NoSQL systems are more limited, only keys (some allow secondary indexes) or complex graph syntax
• We sometimes need processing for complex queries
![Page 35: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/35.jpg)
MAP-REDUCE
![Page 36: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/36.jpg)
HADOOP VS SPARK• Techniques to extract subsets of the data (MAP) and operate them
in parallel before aggregating (REDUCE)
• Not real time, Hadoop the most popular
• Apache Spark opens a new paradigm for near real-time
• You need other languages for these techniques
![Page 37: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/37.jpg)
REAL TIME?• Pseudo real-time (up to 60 seconds) is usually enough
• Unless you are building a video game with social features :)
• What if we have a distributed application between regions?
• Do we prefer latency to one region or the replication hassles?
![Page 38: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/38.jpg)
FIREHOSE• Twitter : real-time stream of tweets
• Technologies like Kafka, Amazon Kinesis, RabbitMQ or NSQ allow us to create a firehose of events from our system to process
• Be VERY careful with the different trade-offs
![Page 39: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/39.jpg)
MODERN STREAM PROCESSING
Service CService BService A Service D
Credits to @alexanderdean from Snowplow
Email MKT
CRM
Analytics
3rd party
Unified log
Eventstream
Keeping few days
Low latency Streaming APIs / web hooks
Own data center
High latency Archive
“Big”Data
HadoopSpark Workers “Live” analysis Monitoring
Low latency Pseudo real-time
API
![Page 40: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/40.jpg)
WHAT ABOUT US?Is there any hope?
![Page 41: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/41.jpg)
PHP• Libraries for everything
• Community and documentation
• Created for the web
• Easy to scale horizontally
• Facebook, Etsy, Youporn, Yahoo…
• Slow
• Language weirdos and WTFs
• Lack of threading
• Not great reputation
• Catching up slowly
![Page 42: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/42.jpg)
DON’T USE PHP!• Heavy math calculations, massive data processing
• Long-running CLI processes, queue workers
• Intensive threading, forking, message passing
• High concurrency scenarios with no cacheable requests
• Not ideal either for writing DSLs or using websockets
![Page 43: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/43.jpg)
ERLANG• Extreme lightweight / concurrency
• Maturity / Stability / Tooling
• OTP framework
• Hot code swaps
• RabbitMQ, Riak, Whatsapp, Ejabberd…
• A bit alien syntax
• Steep learning curve
• String processing
• Hard to find developers
![Page 44: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/44.jpg)
GOLANG• Lightweight / easy concurrency
• Feasible learning curve
• Easy deploy, fast compile
• Hype and momentum
• Docker, NSQ, used by many…
• Immature libraries
• No exceptions, error bubbling
• Hard to recover panics in go-routines
• Not great for business logic
![Page 45: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/45.jpg)
SCALA• Runs on the JVM
• Mature, stable, libraries, fast
• Tooling
• Big hype in the Java community
• Akka, SBT, Play Framework…
• Weird objects model
• Perhaps too much magic
• Incompatibilities between minor versions
• Slow compilation
![Page 46: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/46.jpg)
LOOK BEYOND PHPIt will surely make you a better developer
![Page 47: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/47.jpg)
THE FUTURE• The next decade will be fascinating for the industry
• The internet of things will bring new challenges
• Architectures… and life… are full of tradeoffs
• PHP is not enough to address these needs
• But… PHP is not going anywhere
![Page 48: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/48.jpg)
DON’T BLINDLY FOLLOW THE TRENDS
![Page 49: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/49.jpg)
READ CAREFULLY THE DOCS
![Page 50: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/50.jpg)
CHOOSE THE RIGHT TOOL
![Page 51: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/51.jpg)
“A complex system that works is invariably found to
have evolved from a simple system that worked. The
inverse proposition also appears to be true: a
complex system designed from scratch never works
and cannot be made to work. You have to start over,
beginning with a simple system”
John Gall, systems theorist
![Page 52: Modern software architectures - PHP UK Conference 2015](https://reader038.vdocuments.us/reader038/viewer/2022110310/55a5147b1a28ab866b8b45f7/html5/thumbnails/52.jpg)
QUESTIONS?•Twitter: @ricardclau•E-mail: [email protected]•Github: https://github.com/ricardclau
•Please rate the talk at https://joind.in/13372