oracle goldengate for ms sql server 2008 extract …...oracle goldengate for ms sql server 2008 page...

Oracle GoldenGate for MS SQL Server 2008


Objective

Upon completion of this lesson, you will be able to configure GoldenGate to provide SQL

Server to SQL Server transactional data management using log-based extraction.

During this lesson, you will learn how to:

Prepare your user environment

Prepare the database

Configure and start the change capture process of database operations

Configure and execute the initial data load process

Configure and start the change delivery process of database operations

SQL Server configuration

The following diagram illustrates a SQL Server to SQL Server configuration.

Windows server <target>

MS SQL Server

<dsn>

Target

<schema>

Windows server <source>

MS SQL Server

<dsn>

Source

<schema> Network

Extract

Replicat

Collector

Trails

Manager Manager

Log


Overview of Tasks

Prepare the GoldenGate Environment

The GoldenGate application must be installed on both the source and target systems. A

checkpoint table and Manager Service will be implemented.

In order to execute this lesson, the SQL Server source database must be configured to support

log-based extraction, and an ODBC data source must be created and configured correctly. The

source and target tables will be created with scripts provided in the GoldenGate installation.

For log-based SQL Server extraction, additional logging must be enabled to provide enough

information to reconstruct update operations.

Configure Initial Data Load

Initial database synchronization is almost always required. To initially load data across

actively changing databases, you have limited choices. GoldenGate provides the ability to

perform initial data synchronization while your application remains active. There are several

techniques available, including sending data to files to be loaded by Replicat or by Microsoft

BCP/DTS bulk-load utility, or sending data directly to Replicat.

This lesson demonstrates extracting the data with Extract and sending the data directly to the

Replicat process. This is known as the Direct Load method.

Configure Change Capture

The Extract process is configured to capture change data directly from the SQL Server

transaction logs and distribute the changes to a series of files known as a GoldenGate trails.

Configure Change Delivery

Once the tables have been initially loaded with data, the Replicat process is configured to

deliver the captured change data into the target database.

<source> server <target> server

Network

Initial

ExtractInitial

Replicat

MS SQL Server

<dsn>

Target

<schema>

MS SQL Server

<dsn>

Source

<schema>

Prepare the Environment

Exercise 1.


Objective

The goals of this exercise are to:

Create the database

Create SQL Server logins and users

Create ODBC data source.

Set up transaction logging.

Create tables and populate source data.

Prerequisites

This lab assumes that the GLOBALS parameters have been created with a <mgr service> name

and that the Manager service has been added. This should have been done when installing

GoldenGate for Windows.

Prepare the SQL Server 2008 source environment

Install Manager

1. Configure Manager process on the source

Execute the following commands on the <source> system.

Note! The port numbers must be unique if you are installing GoldenGate more than once on

the same server.

Shell> ggsci

GGSCI> EDIT PARAMS MGR

In the parameter file, enter the following parameter, then save and close the file.

PORT <port>

Start Manager.

GGSCI> START MANAGER

Verify the results:


GGSCI> INFO MANAGER

Create the database

2. Create the database

From Start>Programs, run SQL Server Management Studio.

Enter the name of the local SQL Server instance, then click connect.

Note: These instructions assume SQL Server Authentication which requires the

more complex, double layer, of authentication. SQL Server Authentication is an

option only if it is permitted at the server level, and only Windows Authentication is

enabled by default. If you are using Windows Authentication, you can leave out

most of the login and password entries.

This displays the main SQL Server Management Studio dialog box. Right, right-click Database and select the New Database option.


The New Database dialog box appears. In the Name box, type the <database> from

the Lab Preparation worksheet.


Click OK to add the database.

The main SQL Server Management Studio dialog box reappears.

3. Create SQL Server logins and users

Right click Security and select the New > Login.


The New Login dialog box appears.

For Name, type the <login>.

Select Authentication,


If you selected SQL Server Authentication, type the <password> in the two

Password fields.

Under Defaults, select your <database> for Database, and leave Language set to the

default.

Do not click OK. Instead, from Select a page in the upper left-hand box, double click

Server Roles.

Under Server Role, check the box for the System Administrators role.

Click OK to exit.

The next step creates a schema that you will associate with the new user.

While you have the database selected, click on the New Query button in the main

menu.

The right side of the dialog displays the SQL Query interface. Type in:


create schema <schema_name>;

And press the Execute button from the toolbar.

Now you need to associate the new user with the login and the schema that you have created

with the new database.

In the SQL Query interface, type in:

create user <user_name>

for login <select login_name>

with default_schema=<schema_name>;



4. Create the ODBC system data source name

The GoldenGate Extract process connects to a SQL Server database through an ODBC (Open

Database Connectivity) connection. Both the Extract and Replicat components of GoldenGate

require that a system data source name (DSN) be established, which stores the information

about how to connect to the SQL Server.

Click Start > Settings > Control Panel.

Double-click Administrative Tools.

Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator

dialog box.

Click the System DSN tab, and then click the Add button. The Create New Data

Source dialog box appears.


Select the SQL Server Native Client 10.0 driver and then click Finish. The Create a

New Data Source to SQL Server dialog box appears.

For Name, type <dsn>.


Optionally type in a description, type for this source DSN.

For Which SQL Server do you want to connect to, select local if SQL Server is on

the local system, or enter the server name at your SQL Server location.

Click Next.

Answer the question about how SQL Server should verify the login by selecting the option that you entered when creating the <login>. Then type <login> in Login ID and

<password> in Password.

Click Next.

Make certain the default database is set to the database that you created; otherwise

select Change the default database to: and select it. Leave the other settings to their

defaults to use ANSI.


Click Next.

Leave the next dialog box set to the defaults, and click Finish.

In the confirmation, click Test Data Source to test the connection.

Close the confirmation dialog box and the Create a New Data Source box.

You can leave the SQL Server Management Studio running on the source system,

because you will be using it later.

5. Create the practice tables and insert source data

Execute the following commands on the <source> system only.

Go to SQL Server Management Studio, select your database and select New Query

from the toolbar.

If the correct database is not displayed in the drop down box, click on the arrow and

select it from the list.

Click the File > Open and navigate to the demo_mss_create.sql script.

Click Open to open the script in the New Query window.

Execute the script by clicking the Execute Query button on the toolbar.


Verify the results using the following commands in the input window:

sp_help tcustmer

go

sp_help tcustord

go

Click the X button to close the input tab. Click New Query whenever you need to run

another SQL command.

Following the steps you used to execute the table create script, run the

demo_mss_insert.sql script to insert source data into the source tcustmer and

tcustord tables.

Note! Run this on the source only.

Verify the results as you did before.

Leave the SQL Management Studio running on the source system.


Prepare for transaction logging

To support GoldenGate extraction, the following are required. Execute these steps on the <source> system only.

Log truncation and non-logged bulk copy must be turned off.

The SQL Server database must be set to the full recovery model (this is the default for

SQL Server 2008).

At least one full database backup must be done before GoldenGate processes are

started for the first time.

Additional log data must be enabled so GoldenGate can reconstruct update

operations.

6. Turn off log truncation and bulk copy

In SQL Management Studio on the source system perform the following steps to turn off log

truncation and bulk copy.

Clear the input window and then use the following command to check the status of

the log truncation option. Type the command in the input window (upper window),

then click the F5 key to execute it.

exec sp_dboption '<database>', 'trunc. log on chkpt.'

The following response appears in the results (lower) window with a status of either

ON or OFF.

If log truncation is ON, turn it off by executing the following command.

exec sp_dboption '<database>', 'trunc. log on chkpt.', false

The output should now show a CurrentSetting of OFF’.

Check the status of the non-logged bulk copy option by executing the following

command:

exec sp_dboption '<database>', 'select into/bulkcopy'

If non-logged bulk copy is on, disable it with the following command:

exec sp_dboption '<database>', ' select into/bulkcopy ', false

7. Execute backup

Still in SQL Server Management Studio:


In the console tree, right-click the database name and select Tasks > Backup.

The Back Up Database dialog box displays. If the backup destination is correct, skip

the next two bulleted steps and initiate the backup as explained in the last step. If it is

not correct, change the backup destination and click ADD.

The Select Backup Destination appears, where you can select File Name and then

click the navigation (...) button to navigate to a directory and specify the backup file

name.

Click OK to close the Select Backup Destination dialog box. The backup file is

added to the Destination list box in the SQL Server Backup dialog box.

Click OK to start the backup. This procedure ensures that no transaction log

information is lost when GoldenGate starts up.

8. Set up capture of additional log data

Perform the following steps on the source system to configure the database to log full before

and after images for each update operation.

From the source operating system's command shell, run GGSCI.

Log into the database with the following command:

GGSCI > DBLOGIN SOURCEDB <dsn>, USERID <login>, PASSWORD

<password>

Issue the following commands to force the logging of the full before and after image

for updates.

GGSCI > ADD TRANDATA <owner/schema>.tcustmer

GGSCI > ADD TRANDATA <owner/schema>.tcustord

Verify that supplemental logging has been turned on for these tables.

GGSCI> INFO TRANDATA <owner/schema>.tcust*

Prepare the SQL Server 2008 target system

Install Manager

1. Configure Manager process on the target

Execute the following commands on the <target> system.

Note! The port numbers must be unique if you are installing GoldenGate more than once on

the same server.

Shell> cd <install location>


Shell> ggsci

GGSCI> EDIT PARAMS MGR

In the parameter file, enter the following parameter, then save and close the file.

-- GoldenGate Manager Parameter file

PORT <port>

Start Manager.

GGSCI> START MANAGER

Verify the results:

GGSCI> INFO MANAGER

Steps to create the database

2. Create the database

From Start>Programs, run SQL Server Management Studio.

Enter the name of the local SQL Server instance, then click connect.

Note: These instructions assume SQL Server Authentication which requires the

more complex, double layer, of authentication. For only Windows Authentication,

you can leave out most of the login and password entries.

This displays the main SQL Server Management Studio dialog box. Right, right-click Database and select the New Database option.

The New Database dialog box appears. In the Name box, type the <database> from

the Lab Preparation worksheet.

Click OK to add the database.

The main SQL Server Management Studio dialog box reappears.

3. Create SQL Server logins and users

Right-click Security and select the New > Login.

The New Login dialog box appears.

For Name, type the <login>.

Select Authentication,


If you selected SQL Server Authentication, type the <password> in the two

Password fields.

Under Defaults, select your <database> for Database, and leave Language set to the

default.

Do not click OK. Instead, from Select a page in the upper left-hand box, double click User Mapping.

In the upper Users mapped to this login area, activate the checkmark next to your

database.

In the lower Database role membership for <database> area, check the box for the

db_owner role.

Click OK to exit.

The next step creates a schema that you will associate with the new user.


While you have the database selected, click on the New Query button in the main

menu.

The right side of the dialog displays the SQL Query interface. Type in:

create schema <schema_name>;


Now you need to associate the new user and the schema that you have created with the new

database.

In the SQL Query interface, type in:

alter user <user_name>

with default_schema=<schema_name>;


4. Create the ODBC system data source name

The GoldenGate Extract process connects to a SQL Server database through an ODBC (Open

Database Connectivity) connection. Both the Extract and Replicat components of GoldenGate

require that a system data source name (DSN) be established, which stores the information

about how to connect to the SQL Server.

Click Start > Settings > Control Panel.

Double-click Administrative Tools.

Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator

dialog box.

Click the System DSN tab, and then click the Add button. The Create New Data

Source dialog box appears.

Select the SQL Server Native Client 10.0 driver and then click Finish. The Create a

New Data Source to SQL Server dialog box appears.

For Name, type <dsn>.

For a description, type "GG Training Data source" when creating this source DSN.

This field is optional.

For Which SQL Server do you want to connect to, select the server name.

Click Next.

Answer the question about how SQL Server should verify the login by selecting the option that you entered when creating the <login>. Then type <login> in Login ID

and <password> in Password.


Click Next.

Make certain the default database is set to the database that you created; otherwise select Change the default database to: and select it. Leave the other settings to their

defaults to use ANSI.

Click Next.

Leave the next dialog box set to the defaults, and click Finish.

In the confirmation, click Test Data Source to test the connection.

Close the confirmation dialog box and the Create a New Data Source box.

5. Create the practice tables


Go to SQL Server Management Studio, select your database and select New Query

from the toolbar.



Click the File > Open and navigate to the demo_mss_create.sql script.

Note: Just run the "create" script. Do not run the insert script. The data will be loaded

into the target database using the Oracle GoldenGate Direct Load Method - the steps

are detailed in the next section.

Click Open to open the script in the New Query window.


Verify the results using the following commands in the input window:

sp_help tcustmer

go

sp_help tcustord

go

Initial Data Load using Direct Load Method

Exercise 2.


Objective

These steps configure initial load groups that copy source data and apply it to the target

tables.

Steps to configure initial load

1. Add the initial data load extract batch task group

Execute the following commands on the <source> system to add an Extract process called

EINI<unique id>1.

Execute the following command in GGSCI to create the batch task.


Shell> ggsci

GGSCI> ADD EXTRACT EINI<unique id>, SOURCEISTABLE

Verify the results:

GGSCI> INFO EXTRACT EINI<unique id>

2. Configure the initial data load Extract parameter file


Execute the following command in GGSCI to open the editor.

GGSCI> EDIT PARAMS EINI<unique id>

Add the following lines to the parameter file:

--

-- GoldenGate Initial Data Capture

-- for TCUSTMER and TCUSTORD

--

EXTRACT EINI<unique id>

SOURCEDB <dsn>, USERID <login>, PASSWORD <password>

RMTHOST <target>, MGRPORT <port>

RMTTASK REPLICAT, GROUP RINI<unique id>

TABLE <owner/schema>.TCUSTMER;

TABLE <owner/schema>.TCUSTORD;

1 The process names used in lab exercises, for example EINIBD, are made up of 1) one character for the

GoldenGate process (E for Extract, R for Replicat); 2) three or four to describe the process type (INI for

initial data load, ORA for capture from or delivery to an Oracle database, etc.) and 3) two characters to

create a unique identifier (usually your initials).


3. Add the initial data load Replicat batch task group


Execute the following command in GGSCI to create the batch task.

GGSCI> ADD REPLICAT RINI<unique id>, SPECIALRUN

Verify the results:

GGSCI> INFO RINI<unique id>*, TASKS

4. Configure the initial data load Replicat parameter file



GGSCI> EDIT PARAMS RINI<unique id>

Add the following lines to the parameter file

-- GoldenGate Initial Data Load Delivery

--

REPLICAT RINI<unique id>

ASSUMETARGETDEFS

TARGETDB <dsn>, USERID <login>, PASSWORD <password>

DISCARDFILE ./dirrpt/RINI<unique id>.txt, PURGE

MAP <owner/schema>.TCUSTMER, TARGET <owner/schema>.TCUSTMER;

MAP <owner/schema>.TCUSTORD, TARGET <owner/schema>.TCUSTORD;

5. Execute the initial data load process


Execute the following command to start the initial data load process.

GGSCI> START EXTRACT EINI<unique id>

Verify the results:

GGSCI> VIEW REPORT EINI<unique id>


Verify the results:

GGSCI> VIEW REPORT RINI<unique id>


In your SQL query facility, enter the following commands to verify the content of the

tables:

SELECT * FROM ‘<owner/schema>.TCUSTMER’

SELECT * FROM ‘<owner/schema>.TCUSTORD’


Exercise 3.


Objective


Add the Extract process that will capture changes.

Add the trail that will store the changes.

Start the Extract process.

Steps to configure change

1. Add the Extract checkpoint group

Execute the following commands on the <source> system to create the Extract group.

GGSCI> ADD EXTRACT EMSS<unique id>, TRANLOG, BEGIN NOW

Verify the results:

GGSCI> INFO EXTRACT EMSS<unique id>

2. Create the Extract parameter file



GGSCI> EDIT PARAM EMSS<unique id>

Add the following lines to the parameter file.

--

-- Parameter file to capture

-- TCUSTMER and TCUSTORD Changes

--

EXTRACT EMSS<unique id>

TRANLOGOPTIONS MANAGESECONDARYTRUNCATIONPOINT

SOURCEDB <dsn>, USERID <login>, PASSWORD <password>

RMTHOST <target>, MGRPORT <port>

RMTTRAIL ./dirdat/<trail id>

TABLE <owner/schema>.TCUSTMER;

TABLE <owner/schema>.TCUSTORD;

Note: Record the two characters selected for your <trail id>: ______. You will need this in

the next step and when you set up the Replicat.


3. Define the GoldenGate extract trails

Execute the following commands on the <source> system to create a trail.

GGSCI> ADD RMTTRAIL ./dirdat/<trail id>, EXTRACT EMSS<unique id>,

MEGABYTES 50

Verify the results:

GGSCI> INFO RMTTRAIL *

4. Start the Extract process

On the <source> system, issue the following command in GGSCI to start the Extract process.

GGSCI> START EXTRACT EMSS<unique id>

Verify the results:

GGSCI> INFO EXTRACT EMSS<unique id>, DETAIL


Exercise 4.


Objective


Set up the checkpoint table on the target system.

Create a named group that includes the Replicat process and the checkpoint tables.

Configure the Replicat group by adding parameters.

Start the Replicat group.

Set up checkpoints

1. Edit the GLOBALS file on the target system


Edit the GLOBALS parameter file to add the checkpoint table.


Shell> ggsci

GGSCI> EDIT PARAMS ./GLOBALS

In the text editor, add the following line:

CHECKPOINTTABLE <owner/schema>.ggschkpt

Record the checkpoint table owner and name, then save and close the file.

Table owner ____________________ name ___________________

Note: You could name the table anything you want, but for training purposes we are using ggschkpt.

2. Activate the GLOBALS parameters

For the changes to the GLOBALS configuration to take effect, you must exit the session in

which the changes were made. Execute the following command to exit GGSCI.

GGSCI> EXIT


3. Add a Replicat checkpoint table


This step adds the checkpoint table that you specified when you created the GLOBALS

parameter file. It will be created in the <dsn> database.

Run GGSCI on the target.


Shell> ggsci

Execute the following commands in GGSCI.

GGSCI> DBLOGIN SOURCEDB <dsn>, USERID <login>, PASSWORD

<password>

GGSCI> ADD CHECKPOINTTABLE

Configure delivery

4. Add the Replicat checkpoint group

Execute the following commands on the <target> system.to create the Replicat group named

RMSS<unique id>.

GGSCI> ADD REPLICAT RMSS<unique id>, EXTTRAIL ./DIRDAT/<trail id>

Note: Refer to your Extract set up for the correct two-character <trail id>.

5. Create Replicat parameter file

Execute the following command on the <target> system to edit the Replicat

parameter file.

GGSCI> EDIT PARAM RMSS<unique id>

Add the following lines to the parameter file.

REPLICAT RMSS<unique id>

TARGETDB <dsn>, USERID <login>, PASSWORD <password>

HANDLECOLLISIONS

ASSUMETARGETDEFS

DISCARDFILE ./DIRRPT/RMSS<unique id>.DSC, PURGE

MAP <owner/schema>.TCUSTMER, TARGET <owner/schema>.TCUSTMER;

MAP <owner/schema>.TCUSTORD, TARGET <owner/schema>.TCUSTORD;

4. Start the Replicat process



Execute the following command in GGSCI to start the Replicat process.

GGSCI> START REPLICAT RMSS<unique id>

Verify the results:

GGSCI> INFO REPLICAT RMSS<unique id>

Discussion points

Search in the Windows/UNIX Reference Guide for the information on the following

questions.

1. When to use HANDLECOLLISIONS

When would you use HANDLECOLLISIONS? What does it do?

_________________________________________________________________________

_________________________________________________________________________

2. When should you use ASSUMETARGETDEFS?

_________________________________________________________________________

_________________________________________________________________________

3. What is the purpose of the DISCARDFILE?

_________________________________________________________________________

________________________________________________________________________

Generate Activity and Verify Results

Exercise 5.

Generate Activity & Verify Results


Execute miscellaneous update, insert, and delete operations on the source system.

Verify the delivery of the changes to the target

Turn off the error handling used for initial load.

Verify your source SQL Server 2008 results

Steps to generate database operations

1. Generate inserts, updates, and deletes


Go to SQL Server Management Studio, select your database and press New Query.



Click the File > Open and navigate to the demo_mss_misc.sql script and click Open

to open it in the input window.


Verify and record processing statistics with the following command in GGSCI.

GGSCI> SEND EXTRACT EMSS<unique id>, REPORT

GGSCI> VIEW REPORT EMSS<unique id>

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

___________________________________________________________________

Verify the data by executing the following commands in the New Query window of

SQL Server Management Studio on the source:

select * from TCUSTMER;

select * from TCUSTORD;

Generate Activity and Verify Results

Verify your target SQL Server 2008 results

3. Verify your results on the SQL Server 2008 target

Verify and record processing statistics on the <target> with the following command

in GGSCI.

GGSCI> SEND REPLICAT RMSS<unique id>, REPORT

GGSCI> VIEW REPORT RMSS<unique id>

Verify the data by executing the following commands in the New Query window of

SQL Server Management Studio on the source:

select * from TCUSTMER;

select * from TCUSTORD;

Turn off error handling

4. Turn off initial load error handling for the running delivery process


GGSCI> SEND REPLICAT RMSS<unique id>, NOHANDLECOLLISIONS

5. Remove initial load error handling from the parameter file

GGSCI> EDIT PARAMS RMSS<unique id>

Remove the HANDLECOLLISIONS parameter.

Exercise Name

oracle goldengate for ms sql server 2008 extract …...oracle goldengate for ms sql server 2008 page...

Documents