copyright © 2003 americas’ sap users’ group data archiving: 101 terry knowles – flowers foods...

Copyright © 2003 Americas’ SAP Users’ Group

Data Archiving: 101

Terry Knowles – Flowers Foods

Rob Jackson – Owens Corning May 19, 2003

Introduction:

Terry Knowles – Flowers Foods

Archiving / DART Project Manager – 2 years

Over One Terabyte Archived

Rob Jackson- Owens Corning

Archiving Project Manager - 6 years

Chairperson ASUG Archiving SIG

Over One Terabyte Archived

Purpose of this Presentation

A “101” Primer for those new to Data Archiving.

A general overview of some of the common terms and processes used in a Data Archiving project.

Some lessons learned by two “seasoned” Project Mangers.

The basics of what you’ll need to know if you are going to, or currently leading, and Archiving initiative.

WOW! Finally, A “HOW TO” Book!!

“Archiving Your SAP Data”

SAP Press

Available at the ASUG Bookstore or SAP Press

Required Reading!!

What is Data Archiving?

Data archiving is the process of migrating data from the on-line transactional database to a different form of storage, while maintaining access to all or part of the data as required by Business processes.

( of course the storage system may also be the “bit bucket” in some instances)

What is Document Archiving?

Scanning Incoming Documents and Storing them in an Archive System.

Archiving Outgoing Documents ( in a PDF or similar format) and Storing them into an Archive System.

A lot of Vendors at the Vendor Fair will be more than happy to show you this functionality.

Benefits of Data Archiving:

Storage Costs.

System Availability.

Resource Usage.

Records Retention.

System Performance.

Speed up your Upgrade.

Identifies Open-ended Business Processes.

A Common Statement......

“Hard disks are inexpensive! If I need more database space, I’ll add another disk to my storage system.”

Q.C. System Staging System

Production System

D.R.S. System

Typical Environment results in DB growth X 4 !!!

COPIES from Production

Database Growth: With and Without

Database Volume Forecast

0,0

200,0

400,0

600,0

800,0

1000,0

1200,0

1400,0

Timeline

Vo

lum

e (

GB

)

Volume withArchiving

VolumewithoutArchiving

Nip it in the Bud…..

Data Prevention

Is the data still required?

Is the data still required?

Data remains in the

database

Data remains in the

database

N

Deactivate the creation

of the data

Deactivate the creation

of the data

N

Summarize data

Summarize data

Y

Can the data be

archived?

Can the data be

archived?

N

Y

Archive data

Can the data be summarized?

Can the data be summarized?

Y

Data Archiving Basics:

The Parts and Pieces of Archiving:

ADK: The “Basis” of Archiving

ADK (Archive Development Kit).

Central Archive Administration.

The Software Layer between the SAP Applications and the Archive.

A Toolkit you can use for a Customer Specific Archiving Requirements.

Archive Development Kit – ADK

Development tool for Data Archiving solutions

SAP AG Application developers integrate archiving functionality into the applications.

Customer Archiving solutions for customer-specific tables.

Archive files

ADK Adjustment of code page, number format, structure changes, compression, file handling, job scheduling

Database

Application Application datadata

mySAP.comApplications

ADK

SARA: Archiving “Console”

SARA

Executes programs:

Prepare

Archive

Delete

Post-processing

Reporting (analyze)

Reload (exception)

Storage

Archiving Session Display

Central Archiving Management : SARA

The Business Object

Business Object

Usually a “Transaction” in SAP, such as:

PO’s

Financial Documents

Production Orders

Etc.

Business Objects consist of Data from many different related tables. For Example, for an FI Document:

BKPF, BSEG, BSIK, BSAS, BSAK, etc.

Usually has DEPENDANCIES!

Ex: Production Orders to Sales Orders etc.

The Archiving Object

The “Archiving Object”

Each ARCHIVE OBJECT is required to have a silly name.

Examples:

FI_DOCUMNT – Financial Doc’s.

MM_MATBEL – Material Doc’s.

SD_VBRK – Sales Billing Doc’s.

MM_ACCTIT – MM/FI Interface Data

The “Archiving Object”:

Contains programs used when archiving a specific “Business Object”.

Pre-processing

Write

Delete

Post-processing.

Analyze (Sequential Read)

Customer Specific Customizing.

Retention times, storage parameters, etc.

Handles checking dependencies in most cases.

Archiving Object

Customizing

Programs

Data

The Basic Archiving Process

Prepare

Archive

Delete

Post-processing

Storage

Reporting

Display Access

Reload (exception)

Simple Archiving Process

ArchivingprogramDatabase

Deletogram

Archivefile

Archivefile

Deleteprogram

WriteProgram

The Typical Archiving Run

DatabaseWrite

Program

Archive-File

DeleteProgram

DeleteProgram

DeleteProgram

CurrentArchivefile

Archive-File

Archive-File

Archive-File

Storage of Archive Files

File System

Tape

HSM System

ArchiveLink Interface via Third-party software

Process with ArchiveLink

ArchivingprogramR/3

DatabaseArchive

fileArchive

file

Deleteprogram

Delete Program

ArchiveLink

InterfaceThird Party Software

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

Archivefile

ArchiveLink Third Party Software Providers

Several to meet at the Vendor Fair.

Complete list at :

http://www.sap.com/global/scripts/softwarepartnerdir/directory.asp?softcat

(Choose Archiving and Imaging)

Well that was easy….

Oh, you wanted to still see the Archived Data???

Access to Archived Data

Using PRINTLISTS(in combination with an optical storage system)

REPORTING from the application

Accessing SINGLE OBJECTS from the application

Implementing the ARCHIVE INFORMATION SYSTEM – AS (SAP Note 99388)

Implementing the DOCUMENT RELATIONSHIP BROWSER – DRB (SAP Note 217404)

THIRD PARTY SOLUTIONS. Stop by the Vendor Fair, they’ll be more than happy to show you!

Archive Information System

Browsing through archive files

Configurable retrieval based on infostructures

Ad-hoc access to archive files

Using database tables for fast access

Direct access to single data objects

Transportable down to SAP Release 3.1I via OSS note 99388

Archive Information System

Archive Information System

Archive Retrieval Configurator

Archive files

create read

Archive Explorer

Archive Information

Structure

Access to Archived Data w/A.S.

Archive Information System – Access

Show scanned documents

Document Relationship Browser

Simultaneous access to archived and online data

Start via transaction ALO1 or in AS

User authorizations can be checked

Must configure A.S. first, which is used as a basis for the archived data indexes.

Can Transport down to Release 3.1I OSS note 217404 (and others)

All common archiving objects are connected

SD: Sales activities, Sales orders, Delivery documents, Billing documents

FI: Accounting documents

CO: Controlling documents

MM: Purchase requisitions, Purchase orders, Material documents, Logistics invoice verifications

DRB – Access : Two Ways

Access via AS

Access viatransaction ALO1

DRB – Access via AS

“Related Items”

DRB – Access via AS

DRB – Access via ALO1

DRB – Access via ALO1

DRB – Access via ALO1

Project Management Stuff

Project Risks:

Lack of Business Commitment and/or Management Sponsorship

Inexperienced Project Manager

Limited functionality in older versions of SAP (especially <4.0)

“Glitzy” projects take priority.

B.W., CRM, Web Portals, etc.

Archiving Data that is still required to be on-line.

Lack of Archiving Skills on Project Team.

Management Commitment

Clearly Defined Project Charter.

C-Level Approval of Project.

Visibility of Management during the Project.

Attending meetings

E-mail updates

Etc.

Business Commitment

Recruit your legal department

Compliance with records retention policies is important to the business

Build confidence

Strong leadership

Broad and repeated communication

Attention to detail

Engage the business people

Signed project charters must clearly outline business involvement

Gaining Experience

Start with easy, low-risk archive objects

Build experience with these, then move on to harder areas

Go slow…start with just a few objects

Older Versions of SAP

Archiving is less developed in older versions of SAP

More OSS notes are required

Stick to the most commonly used archive objects

Do it anyway – it will help your upgrade

Especially: IDOCs, MM_ACCTIT, WORKITEMS

Managing the Project…..

Building your Project Team

Who Should Lead the Project???

Strong Project Manager

Technically Oriented

Well Sponsored

Build your Team

BASIS / DBA person(s).

For each one of your “chosen objects” you will need a functional person with expertise in that corresponding module.

Business process expert(s) for each module.

Document retention person.

Security person.

Auditors; Internal, External?

Build your Team

Tax Experts:

IRS Federal tax

State and Local tax

Asset management

Foreign (if necessary)

Create a Project Charter

Identify a Specific Goal.

Part of this goal should be to develop a process for engaging additional Archive Objects.

Documentation Repository

Shared Network Drive.

Kinds of Documentation you may expect:

Transport “Cookbook”

OSS Research Log.

White Papers, Training material, Presentations.

“Object Specific” items.

Matrix of Object Details

Object specific Customizing Details

Testing data, retention times, etc.

Technical Evaluation

Technical Evaluation

Analyze SAP database

Map large tables to archive objects

Research archive objects

Test Test Test !!!!

Map configuration to Business Requirements.

Include Retrieval Requirements.

Include Retention Time Customizing.

In SAP

In Archive System

Database Analysis

Get top 50 largest tables from your DBA

Don’t forget the indices!

Use the “DB Tables” button on SARA or DB15 to map large tables to archive objects.

Depends upon your version of SAP.

Estimate sizes (it’s not worth the extra time to be exact)

DB02: Tables & Indices

SARA: DB Tables

DB15

Research Pre-requisite: Read the Book First!

Read SAP Online Help

SAP Library Cross-Application Components Archiving Application Data

– Drill down by module

– Identify dependencies

• Search OSS

– Use archive object name

– Use program names

Common Objects

IDOC – EDI/ALE IDOCs

WORKITEM – Workflow documents

MM_ACCTIT – MM/FI documents

FI_DOCUMNT – Financial documents

FI_SL_DATA (formerly GLX-OBJECT) – Special ledger

MM_MATBEL – Material movements

SD_VBRK – Billing documents

PP_ORDER – Production orders

Easy Objects

IDOC – EDI/ALE IDOCs

WORKITEM – Workflow documents

MM_ACCTIT – MM/FI documents

Payment Media

Transaction F110, Payment Run -> Reorganization

Start by deleting payment proposals

Testing

Once data is gone, it’s gone…

Run data in test mode

Run archive program but not delete program

Need Production copy

Better performance estimates

Identification of data cleansing issues

Take advantage of system refreshes

Can run destructive tests prior to refresh

The ISSUES…..

Technical and Business Issues

Technical Issues to Think About…..

Did you modify SAP?

Do the Archive Object programs run with acceptable performance?

Are all of the objects within the selection criteria being archived?

Your Business Processes must be “technically” complete.

Technical Issues to Think About…..

Do you have any Bolt-ons?

Do you have Z-tables that belong to the Business Object you are archiving?

Inventory the SAP objects within the scope of the project.

Use number ranges and/or custom ABAPs to count the number of objects in a time period.

Ex: How many FI documents in CC 1000 for 1996 and 1997?

OSS Notes: 317219 (spec. ledger) 138688 (CO data)

Business Issues to Think About….

How is Archiving going to impact the way you do business?

Are you going to have to re-engineer (again) some of your business processes?

How will you handle the times when you absolutely must display archived data?

What can be a realistic “on-line” time for the chosen objects?

Business Issues to Think About….

How will you handle a request from a Business partner to reconcile an account?

Do I need any of this data in the Business Warehouse?

What interdependencies do these Archive Objects have with other Business Objects?

Tax and Legal Issues

Tax Issues

You will need to satisfy ALL taxing authorities. I.R.S.

State

Local

Foreign?

DART Tool. U.S. focused tool for Tax Relevant Data Retention.

New version for multiple countries.

Tax Issues

Your Tax Department MUST analyze the DART tool to make sure it will satisfy their requirements.

You will probably need to add fields and/or tables to the DART extracts.

Pay particular attention to the State and Local Use tax.

Tax Issues

Do you have a Retention Agreement with the I.R.S.?

Although it is possible to build a DART extract from an Archive file, it is not a “best practice”.

Stay current with DART patch levels.

Retention Issues

Does your company have a retention schedule? If not, create one.

Map your archive objects to the retention schedule.

Maintain flexibility in your retention process.

Make sure you can identify specific media that archive files from a specific fiscal year reside so they may be physically destroyed.

Legal Concerns

Check with your Legal Department to verify compliance with all legal issues.

Your business may require retention of certain data beyond what would be considered “normal”.

Different Countries have very different retention laws.

Auditing Issues

New A.I.S. system.

DART?

What are your Auditing Requirements?

Internal needs?

External needs?

Security/Risk Management

Display Access

DART files

Managing the risk associated with deleting data from the database:

Variants set up wrong.

Check and Balance procedure between I.S. and the Business Data Owners.

HELP!!!

Book: ARCHIVING YOUR SAP DATA

Available HERE!

Authored by Helmut Stefani

SAP Press

Excellent Source of Information

334 pages

Nice picture of Georg too!

WWW.ASUG.COM Archiving Discussion Forum

http:service.sap.com/data-archiving

Documentation

Managing SAP Archiving Projects

Data Prevention Checklist

Guide to Data Archiving in mySAP Retail

Data Archiving and the Euro

http://service.sap.com/data-archiving

Media Center Literature

Presentations

SAP Data Archiving – Introduction

Data Archiving Projects

Data Archiving and mySAP Retail

http://service.sap.com/data-archiving Media Center Presentations

Copyright © 2003 Americas’ SAP Users’ Group

Thank you for attending!

Please remember to complete and return your evaluation form following this session.

Session Code: [102]