introduction to tungsten replicator
DESCRIPTION
An Introduction to Continuent Tungsten's replicator product from Percona Live 2013TRANSCRIPT
![Page 1: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/1.jpg)
©Continuent 2012.
Introduction to Tungsten Replicator
Neil Armitage, Cluster implementation Engineer, Continuent
1
Monday, 22 April 13
![Page 2: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/2.jpg)
©Continuent 2012.
What is Tungsten replicator?
2
• Drop in replacement for native MySQL replication.
• Fully OpenSource under GPL v2
• Heterogeneous Replication
• Flexible topologies
• Parallel replication
Monday, 22 April 13
![Page 3: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/3.jpg)
©Continuent 2012.
Tungsten replicator
• Global transaction ID
• Multiple masters
• Multiple sources (Fan In)
3
Monday, 22 April 13
![Page 4: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/4.jpg)
©Continuent 2012.
Tungsten Replicator in a nutshell
binlog THL
slavemaster
host1 host2
THL
trep_commit_seqnotrep_commit_seqnotrep_commit_seqnoorigin seqno eventid
trep_commit_seqnotrep_commit_seqnotrep_commit_seqnoorigin seqno eventid
global transaction ID
4
Monday, 22 April 13
![Page 5: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/5.jpg)
©Continuent 2012.
star-schema
master-slave Heterogeneous
fan-in slave all-masters
MySQL
Oracle
Oracle
MySQL Oracle
Oracle
MySQL MySQL
Monday, 22 April 13
![Page 6: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/6.jpg)
©Continuent 2012.
Why Use Tungsten Replicator?
6
Monday, 22 April 13
![Page 7: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/7.jpg)
©Continuent 2012.
A failover scenario1: MySQL native replication
7
Monday, 22 April 13
![Page 8: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/8.jpg)
©Continuent 2012.
1. one Master, two slaves
• Loading the “employees” test database
8
Monday, 22 April 13
![Page 9: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/9.jpg)
©Continuent 2012.
2. Master goes away
* Stop replication* Slaves are updated at di!erent levels
# 2select count(*) from titles 333,145
# 3select count(*) from titles 443,308
9
Monday, 22 April 13
![Page 10: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/10.jpg)
©Continuent 2012.
3. Look into Slave #2 binary logs
• "nd the last transaction
10
Monday, 22 April 13
![Page 11: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/11.jpg)
©Continuent 2012.
4. Look into Slave #3 binary logs
1. "nd the transaction that was last in slave #2
2. Recognize that last transaction in the log of slave #3 (This can actually take you a LOOOONG TIME)
3. Get the position immediately after this transaction
4. (e.g. 134000 in "le mysql-bin.000018)
11
Monday, 22 April 13
![Page 12: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/12.jpg)
©Continuent 2012.
5. promote Slave #3 to master
* in slave #2
CHANGE MASTER TO master_host=‘slave_3_IP’, master_user=‘slavename’,master_password=‘slavepassword’,master_log_file=‘mysql-bin.000018’,master_log_pos=134000;
12
Monday, 22 April 13
![Page 13: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/13.jpg)
©Continuent 2012.
A failover scenario1I: Tungsten Replicator
13
Monday, 22 April 13
![Page 14: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/14.jpg)
©Continuent 2012.
1. one master, two slaves
• loading the ‘employees’ test database
14
Monday, 22 April 13
![Page 15: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/15.jpg)
©Continuent 2012.
2. Master goes away
* Stop replication* Slaves are updated at di!erent levels
# 2select count(*) from titles 333,145
# 3select count(*) from titles 443,308
15
Monday, 22 April 13
![Page 16: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/16.jpg)
©Continuent 2012.
3. no need to !nd the last transaction
# simply change roles
trepctl -host slave3 setrole -role master
trepctl -host slave2 setrole \ -role slave -uri thl://slave3
trepctl -host slave3 online State: ONLINE
trepctl -host slave2 online State: GOING-ONLINE:SYNCHRONIZING
16
Monday, 22 April 13
![Page 17: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/17.jpg)
©Continuent 2012.
4. Check that the slave has synchronized
# new masterselect seqno from tungsten.trep_commit_seqno;78
# new slaveselect seqno from tungsten.trep_commit_seqno;64
17
Monday, 22 April 13
![Page 18: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/18.jpg)
©Continuent 2012.
4. ... and we’re done
# new masterselect count(*) from employees.titlescount(*)443308
# new slave: count(*)443308
18
Monday, 22 April 13
![Page 19: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/19.jpg)
©Continuent 2012.
Filters
19
Monday, 22 April 13
![Page 20: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/20.jpg)
Replicator Pipeline Architecture
THL SlaveDBMS
Transaction History Log
MySQLBinlog
ApplyExtract Extract
PipelineTungsten Replicator Process
Stage
ApplyExtractAssign Shard
IDApply
StageStage
Monday, 22 April 13
![Page 21: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/21.jpg)
Replicator Pipeline Architecture
THL SlaveDBMS
Transaction History Log
MySQLBinlog
ApplyExtract Extract
PipelineTungsten Replicator Process
Stage
ApplyExtractAssign Shard
IDApply
StageStage
filter filter
Monday, 22 April 13
![Page 22: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/22.jpg)
©Continuent 2012.
Restrict replication to some schemas and tables
21
./tools/tungsten-installer \ --master-slave -a \ ... --svc-extractor-filters=replicate \ "--property=replicator.filter.replicate.do=test,*.foo" \ ... --start-and-report
# test="test.*" -> same drawback as binlog-do-db in MySQL# *.foo = table 'foo' in any database# employees.dept_codes,employees.salaries => safest way
Monday, 22 April 13
![Page 23: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/23.jpg)
©Continuent 2012.
Exclude some schemas and tables from replication
22
./tools/tungsten-installer \ --master-slave -a \ ... --svc-extractor-filters=replicate \ "--property=replicator.filter.replicate.ignore=test,*.foo" \ ... --start-and-report
# test="test.*" -> same drawback as binlog-ignore-db in MySQL# *.foo = table 'foo' in any database# employees.dept_codes,employees.salaries => safest way
# DO NOT MIX .do and .ignore! # (you can do it, but it may not do what you mean)
Monday, 22 April 13
![Page 24: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/24.jpg)
©Continuent 2012.
Change name of replicated schema
23
-a --svc-applier-filters=dbtransform \ --property=replicator.filter.dbtransform.from_regex1=stores \ --property=replicator.filter.dbtransform.to_regex1=playground
# from_regex1=stores -> name of the schema in the master# to_regex1=playground -> name of the schema in the slave
# WARNING: requires "USE schema_name" to work properly.
Monday, 22 April 13
![Page 25: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/25.jpg)
©Continuent 2012.
parallel replication
24
Monday, 22 April 13
![Page 26: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/26.jpg)
Replicator Pipeline Architecture
THL SlaveDBMS
Transaction History Log
MySQLBinlog
shard.listfile
ApplyExtract Extract
PipelineTungsten Replicator Process
StageApplyExtract
ApplyExtract
ApplyExtract
ParallelQueue
Assign Shard
IDApply
StageStage
“channels”
Monday, 22 April 13
![Page 27: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/27.jpg)
©Continuent 2012.
Parallel replication facts
✓Sharded by database
✓Good choice for slave lag problems
❖Bad choice for single database projects
26
Monday, 22 April 13
![Page 28: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/28.jpg)
Parallel Replication test
binary logs
MySQL slave
Tungsten slave
OFFLINE
STOPPED
replicator alpha
direct: alpha(slave)
Concurrent sysbenchon 30 databasesrunning for 1 hour
TOTAL DATA: 130 GBRAM per server: 20GB
Slaves will have 1 hour lagMonday, 22 April 13
![Page 29: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/29.jpg)
measuring results
binary logs
MySQL slave
Tungsten slave
ONLINE
START
replicator alpha
direct: alpha(slave)
Recording catch-up time
Monday, 22 April 13
![Page 30: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/30.jpg)
MySQL native replication
slave catch up in 04:29:30
Monday, 22 April 13
![Page 31: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/31.jpg)
Tungsten parallel replication
slave catch up in 00:55:40
Monday, 22 April 13
![Page 32: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/32.jpg)
Parallel replication made simpler
FROM HERE ....Monday, 22 April 13
![Page 33: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/33.jpg)
Parallel replication made simpler
TO HEREMonday, 22 April 13
![Page 34: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/34.jpg)
Parallel replication made simpler
Monday, 22 April 13
![Page 35: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/35.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
34
Monday, 22 April 13
![Page 36: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/36.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
34
Monday, 22 April 13
![Page 37: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/37.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
34
Monday, 22 April 13
![Page 38: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/38.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
34
Monday, 22 April 13
![Page 39: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/39.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
✓Native replication can continue on other slaves
34
Monday, 22 April 13
![Page 40: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/40.jpg)
©Continuent 2012.
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
✓Native replication can continue on other slaves
❖Failover (either native or Tungsten) becomes a manual task
34
Monday, 22 April 13
![Page 41: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/41.jpg)
©Continuent 2012.
Checking parallel replication
trepctl status
trepctl status -name tasks
trepctl status -name shards
trepctl status -name stores
35
Monday, 22 April 13
![Page 42: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/42.jpg)
©Continuent 2012.
Tungsten GTID vs MySQL 5.6 GTID
• What is GTID
• How it works in Tungsten
• How it works (or not) in MySQL 5.6
36
Monday, 22 April 13
![Page 43: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/43.jpg)
©Continuent 2012.
without global transaction ID
37
slave
master
slave
A
B C
commitcommitcommitcommit
binlog
position
binlog
position position
binlog
Monday, 22 April 13
![Page 44: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/44.jpg)
©Continuent 2012.
with global transaction ID
38
slave
master
slave
A
B C
commitcommitcommitcommit
id#200
id#200id#200Monday, 22 April 13
![Page 45: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/45.jpg)
©Continuent 2012.
Tungsten and global transaction ID:activation
(none)active by default
39
Monday, 22 April 13
![Page 46: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/46.jpg)
©Continuent 2012.
Tungsten and global transaction ID:status
trepctl statusProcessing status command...NAME VALUE---- -----appliedLastEventId : mysql-bin.000002:0000000000001442;0appliedLastSeqno : 6appliedLatency : 0.862clusterName : defaultcurrentEventId : NONEcurrentTimeMillis : 1354304680923dataServerHost : qa.r4.continuent.com
40
Monday, 22 April 13
![Page 47: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/47.jpg)
©Continuent 2012.
Tungsten and global transaction ID:seeing transactions
thl list -seqno 6SEQ# = 6 / FRAG# = 0 (last frag)- TIME = 2012-11-30 20:44:35.0- EPOCH# = 0- EVENTID = mysql-bin.000002:0000000000001442;0- SOURCEID = qa.r1.continuent.com- SQL(0) = insert into test.v1 values (1, 'inserted by node #1') /* ___SERVICE___ = [cookbook] */
41
Monday, 22 April 13
![Page 48: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/48.jpg)
©Continuent 2012.
Tungsten and global transaction ID:changing master connection
trepctl offlinetrepctl online -seqno 105
42
Monday, 22 April 13
![Page 49: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/49.jpg)
©Continuent 2012.
Tungsten and Global transaction ID:crash-safe slave tables
mysql -e 'select * from tungsten_cookbook.trep_commit_seqno\G'*************************** 1. row *************************** task_id: 0 seqno: 6 fragno: 0 last_frag: 1 source_id: qa.r1.continuent.com epoch_number: 0 eventid: mysql-bin.000002:0000000000001442;0 applied_latency: 0 update_timestamp: 2012-11-30 20:44:35 shard_id: testextract_timestamp: 2012-11-30 20:44:35
43
Monday, 22 April 13
![Page 50: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/50.jpg)
©Continuent 2012.
Tungsten and Global transaction ID:crash-safe tables and parallel replication
mysql -e 'select seqno, source_id, shard_id,update_timestamp from tungsten_cookbook.trep_commit_seqno'+-------+----------------------+----------+---------------------+| seqno | source_id | shard_id | update_timestamp |+-------+----------------------+----------+---------------------+| 7 | qa.r1.continuent.com | db1 | 2012-11-30 20:54:14 || 8 | qa.r1.continuent.com | db2 | 2012-11-30 20:54:14 || 9 | qa.r1.continuent.com | db3 | 2012-11-30 20:54:14 || 10 | qa.r1.continuent.com | db4 | 2012-11-30 20:54:14 || 11 | qa.r1.continuent.com | db5 | 2012-11-30 20:54:14 || 12 | qa.r1.continuent.com | db6 | 2012-11-30 20:54:14 || 13 | qa.r1.continuent.com | db7 | 2012-11-30 20:54:14 || 14 | qa.r1.continuent.com | db8 | 2012-11-30 20:54:14 || 15 | qa.r1.continuent.com | db9 | 2012-11-30 20:54:14 || 16 | qa.r1.continuent.com | db10 | 2012-11-30 20:54:14 |+-------+----------------------+----------+---------------------+
44
Monday, 22 April 13
![Page 51: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/51.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDactivation
mysqld --log-slave-updates \ --gtid-mode=on \ --enforce-gtid-consistency
45
Monday, 22 April 13
![Page 52: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/52.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDseeing transactions
#121203 11:15:49 server id 1 end_log_pos 344 CRC32 0x45b25c8f GTID [commit=yes]SET @@SESSION.GTID_NEXT= '7A77A490-3D3A-11E2-8CC9-7DCF9991097B:2'/*!*/;# at 344#121203 11:15:49 server id 1 end_log_pos 423 CRC32 0x873c8fac Query thread_id=3 exec_time=0 error_code=0SET TIMESTAMP=1354533349/*!*/;BEGIN/*!*/;# at 423#121203 11:15:49 server id 1 end_log_pos 522 CRC32 0xb4bf4372 Query thread_id=3 exec_time=0 error_code=0SET TIMESTAMP=1354533349/*!*/;insert into t1 values (1)
46
Monday, 22 April 13
![Page 53: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/53.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDstatus
show slave status\G*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 127.0.0.1 Master_User: rsandbox Master_Port: 13233 Connect_Retry: 60 Master_Log_File: mysql-bin.000002 Read_Master_Log_Pos: 1837 Relay_Log_File: mysql_sandbox13234-relay-bin.000005 Relay_Log_Pos: 2047 Relay_Master_Log_File: mysql-bin.000002... Retrieved_Gtid_Set: 46E13434-3B28-11E2-BF47-6C626DA07446:1-7 Executed_Gtid_Set: 46E13434-3B28-11E2-BF47-6C626DA07446:1-7
47
Monday, 22 April 13
![Page 54: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/54.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDchanging master connection
CHANGE MASTER TO master_log_file='mysql-bin-000003', master_log_pos='1234'
# No global transaction ID is used
48
Monday, 22 April 13
![Page 55: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/55.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDcrash-safe slave table
select * from slave_relay_log_info\G********************* 1. row ******************** Number_of_lines: 7 Relay_log_name: ./mysql_sandbox13234-relay-bin.000005 Relay_log_pos: 2047 Master_log_name: mysql-bin.000002 Master_log_pos: 1837 Sql_delay: 0Number_of_workers: 5 Id: 1
# NO Global transaction ID is used!
49
Monday, 22 April 13
![Page 56: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/56.jpg)
©Continuent 2012.
MySQL 5.6 and global transaction IDcrash-safe slave table + parallel
select * from mysql.slave_worker_info\G Id: 12 Relay_log_name: ./mysql_sandbox13234-relay-bin.000007 Relay_log_pos: 4299 Master_log_name: mysql-bin.000002 Master_log_pos: 7155 Checkpoint_relay_log_name: ./mysql_sandbox13234-relay-bin.000007 Checkpoint_relay_log_pos: 1786Checkpoint_master_log_name: mysql-bin.000002 Checkpoint_master_log_pos: 4642 Checkpoint_seqno: 9 Checkpoint_group_size: 64 Checkpoint_group_bitmap: ?
# NO Global transaction ID is used!
50
Monday, 22 April 13
![Page 57: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/57.jpg)
©Continuent 2012.
Installation
51
Monday, 22 April 13
![Page 58: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/58.jpg)
©Continuent 2012.
Installation
• System Requirements
• Validate "rst
• Deploying from a single location
52
Monday, 22 April 13
![Page 59: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/59.jpg)
©Continuent 2012.
Installation - tools
• tools/ tungsten-installer
• tools/ con"gure-service
• tools/update
• (Using the cookbook recipes, you hardly see them)
53
Monday, 22 April 13
![Page 60: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/60.jpg)
©Continuent 2012.
Tungsten in practiceInstallation
54
Monday, 22 April 13
![Page 61: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/61.jpg)
©Continuent 2012.
Installation
• Check the requirements
• Get the binaries
• Expand the tarball
• Run cookbook
55
Monday, 22 April 13
![Page 62: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/62.jpg)
©Continuent 2012.
REQUIREMENTS
• Java JRE or JDK (Sun/Oracle or Open-jdk)
• Ruby 1.8 (only during installation)
• ssh access to the same user in all nodes
• MySQL user with all privileges
56
Monday, 22 April 13
![Page 63: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/63.jpg)
©Continuent 2012.
binlog
THL
THL
slave
slave
master
host1host2
host3
THL
57
master-slave
Monday, 22 April 13
![Page 64: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/64.jpg)
©Continuent 2012.
binlogTHL
slavemaster
relay log
host1host2
host3
THL
slave
relay log
58
direct
Monday, 22 April 13
![Page 65: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/65.jpg)
©Continuent 2012.
Demo
• Install Master Slave using the Cookbook
59
Monday, 22 April 13
![Page 66: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/66.jpg)
©Continuent 2012.
Overview of Tungsten cookbook
60
Monday, 22 April 13
![Page 67: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/67.jpg)
©Continuent 2012.
tungsten cookbook
tungsten-replicator-2.0.7-xx | +--/cluster-home +--/cookbook +--/tools +--/tungsten-replicator
61
Monday, 22 April 13
![Page 68: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/68.jpg)
©Continuent 2012.
tungsten cookbook
tungsten-replicator-2.0.7-xx | +--/cookbook | +--COMMON_NODES.sh +--USER_VALUES.sh +--NODES_MASTER_SLAVE.sh +--show_master_slave.sh +--test_master_slave.sh +--clear_cluster_master_slave.sh...
62
Monday, 22 April 13
![Page 69: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/69.jpg)
©Continuent 2012.
tungsten cookbook
tungsten-replicator-2.0.7-xx | +--/cookbook | +--COMMON_NODES.sh +--USER_VALUES.sh +--NODES_STAR.sh +--show_star.sh +--test_star.sh +--clear_cluster_star.sh...
63
Monday, 22 April 13
![Page 70: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/70.jpg)
©Continuent 2012.
Master - to - master
• Bi-directional installation
• Operational steps:
• install a master service in both servers
• install the corresponding slave service in the other server
64
master-alpha
slave-bravomaster-bravo
slave-alpha
Monday, 22 April 13
![Page 71: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/71.jpg)
©Continuent 2012.
BI-DIR: the painless way
• edit cookbook/COMMON_NODES.sh
• edit cookbook/USER_VALUES.sh
• remove two nodes
• edit the variables in cookbook/NODES_ALL_MASTERS.sh
• cookbook/install_all_masters.sh
65
Monday, 22 April 13
![Page 72: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/72.jpg)
©Continuent 2012. 66
master-alpha
slave-bravo
slave-charlie
slave-delta
master-bravo
slave-alpha
slave-charlie
slave-delta
master-charlie
slave-alpha
slave-bravo
slave-delta
master-delta
slave-alpha
slave-bravo
slave-charlie
all-masters
A
B
CDD
A
B
C
Monday, 22 April 13
![Page 73: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/73.jpg)
©Continuent 2012.
Multiple masters
• fan-in
• Steps:
• install a master service in each node
• install a slave service for each master in the fan-in node
• or :
• cookbook/install_fan_in.sh
67
Monday, 22 April 13
![Page 74: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/74.jpg)
©Continuent 2012.
Master - to - master
• star schema
• Steps:
• install a master service in each server
• in the hub, install a slave service for each spoke
• in each spoke, install a slave service for the hub, using bypass option
• cookbook/install_star.sh
68
Monday, 22 April 13
![Page 75: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/75.jpg)
©Continuent 2012
For more information download the slides from Mondays Tutorial
69
Monday, 22 April 13
![Page 76: Introduction to Tungsten Replicator](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b67b2a4a795962088b4613/html5/thumbnails/76.jpg)
©Continuent 2012 70
Continuent Website:http://www.continuent.com
Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator
Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.blogspot.comhttp://flyingclusters.blogspot.com
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]
Monday, 22 April 13