Архитектура приложений с использованием mysql, Петр...
DESCRIPTION
Доклад Петра Зайцева на HighLoad++ 2014.TRANSCRIPT
Architecture and Design of MySQL Powered Applications
Peter ZaitsevCEO, PerconaHighload++ 2014Moscow, Russia31 Oct 2014
2
About Percona
Open Source Software for MySQL
Ecosystem•Percona Server•Percona XtraDB Cluster (PXC)•Percona Xtrabackup•Percona Toolkit
Solutions to succeed with MySQL
•Consulting•Support •Managed Services
3
About Presentation
What's in your Toolbox ?
What Questi
ons you
should
ask ?
Common
MySQL Architecture Patter
ns
4
LiveJournal Architecture 10 Years Ago
5
10 Years ago we see
Load Balancers
FrontEnd vs BackEnd Web Server Separation
MemcacheD (Caching)
Global Database for Shared Data
Read Write Splitting
Sharding “Database Clusters”
6
History Lesson
MySQL is quite Mature
Battle tested Architecture Patterns
Most Problems have proven solutions
7
YOUR TOOLBOX NOWWell… certain things have changed
8
Very Powerful Hardware
•16GB to 512GBLots of Memory
•1000 IOPS to 100.000 IOPS•10ms latency to 0.1ms IO Latency
Very Fast Storage
•4 Cores to 32 CoresMany CPU Cores
9
Reasonable Software
•MySQL •Linux Kernel•File Systems•Etc..
Can use this
Hardware Pretty
Well
10
Cloud
Dynamic Scalability
Throw Away Server Mentality
Agility
Less involved Operations
11
Automation
Development
Testing
Deployment
Operations
12
New Languages and Frameworks
More Agile
Less “system” code
More Client Side Development
Less Database Dependence
ORM
13
Multiple “Clients”
Desktop Web Browsers
Mobile Web Browsers
“Apps”
API
14
Not Only “MySQL”
•Memcache, Redis•RabbitMQ•Cassandra, MongoDB•Hadoop•Column Store Engines, Vertica
Right Tools for
Right Job!
15
More HA Options
Mature MySQL Replication
Percona XtraDB Cluster, Galera
MySQL Cluster
Proprietary cloud HA
16
Ready solutions for Replication Management
MHA
PRM
Continuent Tungsten
17
Making Sharding Better
•Clustrix•MemSQL•ScaleDB
Not Quite MySQL
•ScaleArc, ScaleBase, Tesora•MySQL Proxy, MaxScale, ProxySQL
Proxy Solutions
•Vitess•JetPants•MySQL Fabric
OpenSource Frameworks
18
ARCHITECTURE QUESTIONS
19
Basics
Make some Right decisions Early
Your first Architecture will not be perfect
Some poor choices are very expensive to fix
20
Consultants are like Lawyers
You can waste a lot of money with them
Getting right advice on the right topics will save you a lot of trouble in the end
21
Dimensions to consider
Application Scale
High Availability
Team Experience
22
Approach to Scaling
Avoid Over-Engineering
Give yourself some runway
Do Some Capacity Planning
23
High Availability
Real High Availability is Really Expensive
Medium Availability is getting more affordable
People are often leading cause of downtime
24
Team Experience
Chose Solutions which your Team can Support
• Percona Toolkit, Percona Cloud Tools• Systematically find and review new queries
Often Developers know little about Databases
Right Development, Operational Process Is a Key
Tools to Support your Process
25
Simplicity
•Less complexity lower costsUse fewer
components when possible
•Monitor•Troubleshoot
Each components needs experience
to
•“Follow the Crowd”•Unless you have a reason not to
There is safety in numbers
26
World of “Small” Applications
•ReallyMajority of
Applications are Small
•Especially with support of caching
May well run on single database
instance
•Active intranet site for company with 200K employees•Ecommerce site supporting $10M+ in Sales
Such “Small” Applications can be
significant
27
Lets do Some Math
Modern MySQL Instance Handle 100
K QPS
20 Queries per Use
r Interaction
5K Use
r Interactions Per Second
144M User
Interaction
s a Day
(Considering 3x peak
to avera
ge)
10M Daily Activ
e users (with Medium
Engagement)
28
PRACTICAL CHOICESBasic Architecture Patterns
29
Baseline
Single MySQL Instance
No Caching
No HA
No Supplemental Technologies
30
Simple High Availability
HA with DBaaS (Amazon RDS)
HA with MySQL Replication
HA with Percona XtraDB Cluster
31
What do you need to Scale ?
Reads
Writes Data Size
32
Load Management Tricks
Move the load
In Time (Doing Batch Processing at Night) In Space (Use Slave for Reports)
33
Queueing
Used by Most high volume architectures
• RabbitMQ• Redis• Gearman
Deal with Load Spikes
Increase Reliability
Scale parts of the system which need to
Many solutions
34
Scaling Reads
Replication (and Read Write Splitting)
Caching (Memcache, Redis etc)
Percona XtraDB Cluster (Read Anywhere)
35
Scaling Writes
•Using Different Servers for Different Parts of Application
Functional Partitioning
•Horizontal Partitioning over many servers
Sharding
36
Scaling Data Size
•Depends on the Application and Needs•10TB good rule of thumb
How Much We Can
Practically Store in
Single MySQL Instance
these days
37
How Is it Possible
“Online” Schema Change
MySQL 5.6Pt-online-schema-change
(Percona Toolkit)
38
Summary
MySQL is Mature. Solutions Available for Many Problems
Make few choices right in the beginning
Hardware and Software advances allow to go a long way with simple architectures