set up & operate tungsten clusters
DESCRIPTION
In this live virtual course, you learn how to get from a single database server to a scalable cluster, or from a brittle MySQL replication system to a transparent, manageable Continuent Tungsten cluster. We discuss the benefits of leveraging Continuent Tungsten clustering with MySQL, and walk you through the steps to implement a Continuent Tungsten cluster in Amazon EC2. We cover the prerequisites, installing and configuring Continuent Tungsten, and best practices that are part of most production installations and proof-of-concepts. Course Topics: - Configuring MySQL and the OS for a proper Continuent Tungsten installation - Installing a cross-site Continuent Tungsten cluster - Online maintenance by performing a schema upgrade on the master database server with no application downtime (switch operation) - Automatic master failover upon a MySQL database server failure - Recovery of a failed master to a fully operational cluster member with a single command (recover operation) - Disaster recovery by switching database operations to another data center or AWS region (geo-clustering, cross-site 'switch' operation) - Multi-master operations across multiple geographic regions We also demonstrate basic operations, such as adding and removing a cluster node, basic monitoring and troubleshooting, and discuss the basic failure scenarios. Learn how to quickly configure and provision highly optimized Continuent Tungsten deployments in the cloud or on-premises.TRANSCRIPT
©Continuent 2014
Tungsten University: Set Up and Operate
Continuent Tungsten Jeff Mace, Director of Professional Services
©Continuent 2014
Introducing Continuent
2
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
©Continuent 2014
Single Master Clustering
3
©Continuent 2013
Application Application
Replicator Replicator Replicator
MySQL Client API MySQL Client API
MasterSlave Slave
Existing MySQL Replication
4
©Continuent 2013
Application Application
Replicator Replicator Replicator
MySQL Client API MySQL Client API
MasterSlave Slave
Step 1: Replace MySQL Replication
5
©Continuent 2013
Application Application
Replicator Replicator Replicator
MySQL Client API MySQL Client API
Manager Manager Manager
MasterSlave Slave
Step 2: Add Manager Process
6
©Continuent 2013
Application Application
Replicator Replicator Replicator
Manager Manager Manager
Tungsten Connector Tungsten Connector
MasterSlave Slave
Step 3. Add Intelligent Connector
7
©Continuent 2013
Data Service
Application
Replicator Replicator Replicator
Application
Manager Manager Manager
Monitoring and control
Monitoring and control
MasterSlave Slave
Step 4. Connector Connectivity and Communication
Tungsten Connector Tungsten Connector
8
©Continuent 2013
Application Application
NYC
©Continuent 2014
Installation
10
©Continuent 2014
Setup Servers
$> git clone https://github.com/continuent/continuent-vagrant $> cd continuent-vagrant $> git submodule update --init !$> cp ./examples/BlankMySQL/default.pp ./manifests $> cp ./examples/Vagrantfile.6.vbox ./Vagrantfile $> cp ~/continuent-tungsten-2.0.2-463.tar.gz . $> cp ~/tungsten-replicator-2.2.1-403.tar.gz . !$> ./launch.sh
11
Follow these steps to create 6 servers to use for testing. The README shows you how to launch these servers into EC2 or VirtualBox.
©Continuent 2014
Install From Staging
localhost $> vagrant ssh db1 db1 $> sudo su - tungsten db1 $> wget -O install-sor.sh http://bit.ly/R9vkLW db1 $> bash ./install-sor.sh
12
Follow these steps to unpack the software and install a System-of-Record topology to all 6 servers. This uses a staging directory to manage the configuration on all servers.
©Continuent 2014
Operations
13
©Continuent 2014
Checking Continuent Tungsten
db1 $> . /opt/continuent/share/env.sh !# Checks the health of Continuent Tungsten # and reports errors db1 $> tungsten_monitor !# Check replication status across all servers db1 $> multi_trepctl !db1 $> cctrl -multi cctrl /> ls !cctrl /east> use east cctrl /east> ls
14
©Continuent 2014
Master Failover & Recovery
# Force automatic failover to a new master cctrl /east> service db1/mysql stop cctrl /east> ls db1 $> tungsten_monitor !# Recover the db1 server as a slave in the cluster cctrl /east> recover db1 $> tungsten_monitor !# Make db1 the master server again cctrl /east> switch to db1 cctrl /east> ls
15
©Continuent 2014
Connectivity
db1 $> mysql -hdb2 -P3306 -uapp_user -psecret -e'SELECT @@hostname' !cctrl /east> switch to db2 db1 $> mysql -hdb2 -P3306 -uapp_user -psecret -e'SELECT @@hostname' !db1 $> tpm connector --samples
16
©Continuent 2014
Online Maintenance
db1 $> mysql -hdb2 -P13306 -utungsten -psecret \ -e'ALTER TABLE …’ # Repeat applying the change directly to db2 through db6 !cctrl /east> switch to db2 !# Apply the change directly to db1 which is now a slave db1 $> mysql -hdb1 -P13306 -utungsten -psecret \ -e'ALTER TABLE ...' !cctrl /east> switch to db1
17
©Continuent 2014
Global Switch
db1 $> cctrl -multi cctrl /> use us cctrl /us> ls !# Switch operations to the west service cctrl /us> switch to west cctrl /us> ls !db1 $> mysql -hdb2 -P3306 -uapp_user -psecret -e'SELECT @@hostname'
18
©Continuent 2014
Configuration Update
19
db1 $> cd /opt/continuent/software db1 $> cd continuent-tungsten-2.0.2-463 !db1 $> ./tools/tpm configure defaults \ --repl-backup-method=xtrabackup-full db1 $> ./tools/tpm update
Execute these steps on db1 and they will update the configuration on all servers.
©Continuent 2014
Reset the servers
localhost $> cd continuent-vagrant localhost $> ./scripts/clean.sh -y
20
This will remove the installed Continuent Tungsten software for the next demo.
©Continuent 2014
Multi-Master Clustering
21
©Continuent 2014
Normal Multi-Master Configuration
22
master
slave
master
slaveeast1
east1
west
west1
US-West-1 US-East-1
©Continuent 2014
Failed master
23
master
master
master
slave
masterXeast1
east1
west
west1
US-West-1 US-East-1
©Continuent 2014
Failed Slave
24
master
slave
master
slaveX east1
east1
west
west1
US-West-1 US-East-1
©Continuent 2014
Failed Multi-Master Replicator
25
master
slave
master
slaveeast1
east1
west
west1X
US-West-1 US-East-1
©Continuent 2014
Full-Site Failure
26
master
slave
master
slaveeast1
east1
west
west1XUS-West-1 US-East-1
©Continuent 2014
Installation
27
©Continuent 2014
Install from INI
28
localhost $> vagrant ssh db1 db1 $> sudo su - tungsten db1 $> wget -O install-msmm.sh http://bit.ly/1vGLsoE db1 $> bash ./install-msmm.sh
Follow these steps to unpack the software and install a Multi-Site Multi-Master topology to all 6 servers. This uses an INI file on each server to manage configuration.
©Continuent 2014
Operations
29
©Continuent 2014
Monitor Replication Status
db1 $> . /opt/continuent/share/env.sh !db1 $> multi_trepctl --by-service !db1 $> multi_trepctl --role=master heartbeat !db1 $> multi_trepctl --by-service !db1 $> cctrl cctrl /east> switch db1 $> multi_trepctl --by-service !
30
©Continuent 2014
Configuration Update
31
db1 $> vi /etc/tungsten/tungsten.ini # Add ‘repl-backup-method=xtrabackup-full’ to [defaults] !db1 $> /opt/continuent/tungsten/tools/tpm update !
Repeat these steps on each server to update the configuration.
©Continuent 2014
Wrapping Up
32
©Continuent 2014
Useful Links
• https://github.com/continuent/continuent-vagrant
• https://docs.continuent.com/continuent-tungsten-2.0/operations.html
• https://docs.continuent.com/continuent-tungsten-2.0/deployment-msmm-monitoring.html
33
©Continuent 2014
More Tungsten University Webinars!
34
• Set Up and Operate Real-Time Data Loading into Hadoop (29 May 2014, 10am PDT)
• Set Up and Operate Oracle Replication (5 June 2014, 10am PDT)
Send feedback to: [email protected]
©Continuent 2014
Continuent Web Page: http://www.continuent.com
!
Tungsten Replicator 2.0: http://code.google.com/p/tungsten-replicator
Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: [email protected]