scaling symfony - from single to multi-node cluster
TRANSCRIPT
Scaling SymfonyFrom single to multi-node cluster
Antoni [email protected]
THEORYScaling types
1. Vertical Scaling
2. Horizontal Scaling
THEORYScaling types
Real life:Vertical ! horizontal
SCALABILITY PATH0. Caching
HTTP Cache1. Browser cache (private)2. Proxy cache (shared)
0
1st Tier Cache – Reverse ProxyCaches HTTP responses. Most performant, close to the user.E.g. Varnish
1
$response = new Response;$response->setMaxAge(600); // For user’s browser$response->setSharedMaxAge(600); // For reverse proxy$response->setVary([’User-Agent’]); // Reverse proxy should cache
// different responses for UA
@Cache(smaxage=”15”, // Configure using annotationsvary={”User-Agent”})
2nd Tier Cache – in-applicationCaches database query results, long-term operations.E.g. Redis, Memcache (mostly in-memory)
2
App DB
AppDb
Single serverwith Application (PHP, Symfony) and Database (MySQL)
Separated serversservers separated by their rolesBeware of ping between app and DB "
SCALABILITY PATH1. Separating servers
$ htopF4 (filter): mysql
collectd
SCALABILITY PATH1. Separating servers When?
# parameters.ymlparameters:
database_host: localhost…
# parameters.ymlparameters:
database_host: db.pixers…
SCALABILITY PATH1. Separating servers How?
Ask your *ops to set-up new db server1Change database host in configuration2
App1App1 AppN…
Load BalancersDistribute requests to backend servers, e.g. HAProxy, Varnish
Application serversEach has the same application (clones)
SCALABILITY PATH2. Adding application servers
lb1 lb2
DNS
DNS ServerDistributes requests to Load Balancers
DNS Load Balancing1
Load Balancing softwaree.g. HAProxy, Varnish, nginx2
$ dig a microsoft.com...;; ANSWER SECTION:microsoft.com. 924 IN A 104.40.211.35microsoft.com. 924 IN A 104.43.195.251microsoft.com. 924 IN A 191.239.213.197microsoft.com. 924 IN A 23.96.52.53microsoft.com. 924 IN A 23.100.122.175
SCALABILITY PATH2. Adding application servers How?
Session handlingLosing users’ session? Sticky sessions?3
Saving filesUsing distributed filesystem (Amazon S3, CEPH or just NFS)4$ composer require knplabs/knp-gaufrette-bundle ~0.3
$ composer require oneup/flysystem-bundle @stable
$ composer require snc/redis-bundle 2.x-dev
Symfony\Component\HttpFoundation\Session\Storage\Handler\MemcachedSessionHandler
SCALABILITY PATH2. Adding application servers How?
db2master
db1master
Master-Master replicationWrites to db1 or db2Reads from db1 or db2
+ High-availability+ Better performance in writes and reads- Generating IDs (auto_increment_offset)- Failover mechanism?
Master-Slave replicationWrites to db1Reads from db1 or db2
+ High-availability+ Better performance in reads+ Easier than master/master- Failover mechanism?
SCALABILITY PATH3. Scaling database - Replication
db2slave
db1master
Multiple databasesData partitioned by type
+ More space for data+ Better performance in reads- Not transparent for application (99% cases)- Beware of JOINs (app-side)
SCALABILITY PATH3. Scaling database - Sharding
db products
db orders
App
SCALABILITY PATH3. Scaling database – Sharding in SaaS
db tenant2
db tenant1
App
Multiple databasesData partitioned by tenants
db core
SCALABILITY PATH3. Scaling database How?
Sharding
+ Dynamic connections (SaaS/Multitenant)+ Multiple entity managers
1
Replication - Doctrine2# app/config/config.ymldoctrine:dbal:connections:default:slaves:slave1: # ...slave2:
SCALABILITY PATHFinal architecture?
SCALABILITY PATH4. Going SOA
Core Application
PrintAPI PhotoAPI
SCALABILITY PATH4. Going SOA When?
Pros:+ New business opportunities+ Better scalability as services are stateless+ Exposing API by SOAP or REST+ Easier development (dev-team per service)+ Technology independent (Symfony app can use Node.js service)
Cons:- Platform complexity – harded to maintain- Communication overhead- Security?
1
2
SCALABILITY PATH4. Going SOA How?
Service-side (e.g. Search application)1
Client-side (e.g. Core application)2$ composer require guzzlehttp/guzzle
$ composer require eightpoints/guzzle-bundle(we don’t use it)
$ composer require friendsofsymfony/rest-bundle(we don’t use it in every project)
$ composer require jms/serializer-bundle
$ composer require nelmio/api-doc-bundle
Contact me at:[email protected]
linkedin.com/in/antoniorfintwitter.com/antoniorfin
www.pixersize.com
Thank you!Questions & Answers