nate locklin ◦ database analyst, ppg industries ◦ nate@locklin.orgnate@locklin.org steve tirone...

Three Best Practices with SSIS

Nate Locklin◦ Database Analyst, PPG Industries◦ nate@locklin.org

Steve Tirone◦ Data Warehouse Analyst, Amerinet◦ stephen.tirone@gmail.com

About Us

Survey:◦ What is SSIS?◦ Use it occasionally◦ Use it every day◦ I wrote the book on advanced SSIS

Given that we didn’t know, we wanted to present something generally useful, hence◦ “Three best practices to keep in mind when

developing and deploying SSIS packages”

SSIS Experience?

Open BIDS (Business Intelligence Development Studio)

Available in VS2005, VS2008, but not VS2010 (yet)

Incidentally, the Import/Export wizard in Management Studio runs SSIS under the hood

First, a Baseline Intro

Several tabs at the top◦ Control Flow: your top level workflow workspace◦ Data Flow: details of data transfer operations◦ Event Handlers: where you can handle raised

events ◦ Package Explorer: like a site map of your package

Start by dragging Tasks to your Control tab Most commonly used tasks are probably:

◦ Data Flow◦ Execute SQL

Constructing your Package

Let’s import the DJIA stock prices◦ downloaded from Yahoo Finance into an Excel

spreadsheet

Quick Example

Demo: Quick Package

Two often used log providers◦ Text Files◦ SQL Server (sysssislog in 2008, sysdtslog90 in

2005) – created when first used Many events possible to log

◦ OnError and OnTaskFail events most common◦ Can reduce detail to Computer, Source, and

Message◦ Here you will find all the error messages:

http://msdn.microsoft.com/en-us/library/ms345164.aspx

Best Practice #1: Logging

Demo: Logging

Properties of objects can be changed at run-time

No need to change the package itself to promote from test to production◦ Connection string is all that’s needed to change

Several formats:◦ XML configuration file most commonly used

Best Practice #2: Configurations

Demo: Configurations

Checkpoints let a package start where it left off previously, such as when an error occurs in the middle of your package◦ Lets you fix the error, then restart the package,

without it re-running the parts already done

Best Practice #3: Checkpoints

Set the following properties of the Package◦ CheckpointFileName = <some name>◦ CheckpointUsage = IfExists◦ SaveCheckpoints = True

Set the following properties on all the tasks:◦ FailPackageOnFailure = True

Setting up for Checkpoints

Demo: Checkpoints

Most importantly: ◦ “A package can be restarted only at the control

flow level. You cannot restart a package in the middle of a data flow.”

Also:◦ “When a package is restarted the Foreach Loop

containers and its child containers are run again. If a child container in the loop runs successfully, it is not recorded in the checkpoint file, instead it is rerun.”

Checkpoint Caveats

RetainSameConnection◦ Using lots of connections? Use this, please!◦ Can be found as a property of the Connection Mgr

Packet size◦ Consider increasing (32,767) for larger data

Row Per Batch Size◦ Sometimes speed is enhanced when setting the

“Rows per batch” to something other than blank, like 1000, 5000 or 10,000 (particularly when run through BIDS over the network)

Other Design Considerations

Property of the Connection Manager

Retain Same Connection

On All page of the Connection Manager

Packet Size

Package protection level

Default is Encrypt with User Key, but you want Encrypt with Password (all or sensitive)

Deployment Considerations

Know your environment (64 bit vs 32 bit)◦ Might get

[Connection manager "Excel Connection Manager 1"] Error: SSIS Error Code DTS_E_OLEDB_EXCEL_NOT_SUPPORTED: The Excel Connection Manager is not supported in the 64-bit version of SSIS, as no OLE DB provider is available.

◦ Or Class not registered

The 64 bit issue may be addressed by telling Integration Services to use 32 bit runtime

Deployment Considerations

In the SQL Agent job step:

64/32 bit Options

Property of the PROJECT:

64/32 bit Options (BIDS)

Great flexibility in error handling if you “program” the OnError handler

Error Handling

Implement Logging Use Configuration Files for flexibility Checkpoints can be useful Know your environment Keep small design considerations in mind

Summary

Nate Locklin◦ Database Analyst, PPG Industries◦ nate@locklin.org

Steve Tirone◦ Data Warehouse Analyst, Amerinet◦ stephen.tirone@gmail.com

Slides are at http://tinyurl.com/SSISBestPractices

Questions?

Future BI Topics?◦ Integration Services (SSIS)◦ Reporting Services (SSRS)◦ Analysis Services (SSAS)

SQL Saturday?◦ BI Edition?◦ Volunteers to help coordinate?

Email Gina or Steve

Another Survey! Yeah!