images services 4.1.1

1© 2008 IBM Corporation

Pete Stryjewski, Senior Managing Consultant

Images Services 4.1.1Integral SDS and Storage Overview


Information Management Software | Enterprise Content Management

IS Release and Other Information

IS 4.1.1 was released January 2008

IS 4.1.2 was release November 2008

Plasmon currently in receivership.

– Looking for a equity partner

– Support through Kodak continues

– Currently there is media supply

MSAR “WORM” being fast tracked.



New Storage Features for IS 4.1.1

Integral Single Document Storage (ISDS)

– Second generation connectivity to Protected Storage Devices

– Tightly integrated into “commit path”

Read only MSAR support

– Full surfaces can be located on “read-only” file systems.

– File system based Protected Storage devices can enforce retention at the volume level.

– New enhancements in the pipeline – may be accelerated



What is Integral SDS?

Integral SDS is…

– New IS server-based architecture

– Reuses robust OSAR/MSAR processes

– Plug-in device drivers model

– SDS references stored in MKF locator database

– In Image Services transactional path

– New, self-describing SDS object format



Integral SDS Benefits

Robust

– Increased system stability and reliability

– SDS references are tamper proof from IS client application

Extensible

– As new WORM compliant storage devices come on the market, “plug-in” model allows for development of new drivers in parallel

Improved HA/DR

– No longer need to monitor NLS and script any failover. Can now leverage IS internal processes to start, stop and fail over



Integral SDS Benefits

Improved performance

– MKF faster; frees up relational database resources

– Server-side efficient, integrated process

– Scalable: up to 254 devices and 99 “workers” per

– Fully integrates with IS commit/migration process

Improved administration and ease of use

Improved supportability

– Documentation, Education, Error handling part of IS



Integral SDS Capabilities

Native support of SDS devices

More IS functionality supported with SDS

– Multiple storage configuration support

– All CFS-IS use cases when IS configured to use SDS

Backwards compatibility with NLS: Immediate or day forward support from NLS to Integral SDS



Integral SDS Capabilities

Native support of SDS Devices

– Plug-in Model - new drivers for

• Network Appliance/IBM N-Series Snaplock

• Sun StorageTek Compliance(5x20 NAS with Compliance Archiving Software)

• EMC Centera

• IBM Total Storage Tivoli DR (DR550)

– Standard IS Tools

• Configure through fn_edit, Xapex• Doc_tool to monitor performance and manage devices

• Error handling with RSVPs, Info Messages, SNMP



Backward Compatibility with NLS: Do nothing with NLS SDS documents –

– Still can be retrieved

– SDS_tool can still be used to manage these objects

– SDS information is not secure in the DOCTABA

– Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCTABA table

Copy SDS info from DOCTABA to DOCS

– The SDS info (CLIP_ID) is copied from DOCTABA to the DOCS table

– Can still use SDS_tool to manage NLS SDS objects. It uses the DOCTABA sds info

– Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table



Backward Compatibility with NLS (continued):

Move SDS info from DOCTABA to DOCS

– Can not use SDS_tool to manage object

– New Integral SDS tools also may not be used

– Beware: Objects can not be managed and is read-only

– Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table

Copy NLS object to new SDS unit

– The new Integral SDS sds_copy tool can be used to copy NLS SDS objects to a new Integral SDS unit

– New Integral SDS tools can be used to manage these new objects

Note

– All the above modes and Integral SDS can co-exist on a single system



Device Specific Concerns

Centera (all platforms)

– An “Advanced Retention Management” (ARM) License is required to be purchased from EMC.

– Provided Event Based Retention and Litigation Holds

SnapLock/N-Series (Windows)

– Requires “domain” user to access mounted volume (like MSAR currently requires)

IBM DR550

– Retention cannot be increased



New SDS fields in the MKF Document Locator (DOCS) table:

sds_id: signed long; -- SDS UNIT ID

sds_obj_date: datetime; -- unix time at which SDS object was

sds_obj_id_len: unsigned byte;

sds_obj_id: hex byte [sds_obj_id_len:256];

sds_obj_blob_sz: unsigned long;

sds_orig_doc_id: unsigned long -- This could be a foreign document.

nonunique key; -- Nil value would indicate local document.

sds_retent_typ: unsigned byte; -- Possible values

-- None = 1

-- infinite = 2

-- chronological = 3

-- EBR + EBR supported device = 4

-- EBR + non-EBR supported device = 5

sds_retent_value: datetime; -- This retention expiration date.

-- Not set for none, infinite, and EBR type

-- Always set for chronological & virtual EBR

sds_held: unsigned byte; -- If set to 1, it is held.

sds_migrating: unsigned byte; -- If set to 1, it is migrating to SDS



Document Retention

Really for the first time “retention matters”

– ‘deldocs’ requires DOCTABA AND ISDS object retention to be met. This can be over-ridden.

– Individual tools (IDM desktop, WAL apps), can remove the “pointer” information and leave an orphaned object.

– ISDS objects (having met their retention) can be deleted via DOC_tool

Backfile conversions should have proper retention set prior to migration.



SDS Retention

Image Services retention has always been stored in the DOCTABA table

With Integral SDS retention enabled, the retention stored in the DOCTABA is propagated to the SDS object and recorded in the DOCS table

Retention of an the SDS object is inherited from the DCL when SDS objects are created as a result of committal

Retention of an the SDS object is inherited from the DOCTABA when SDS objects are created as a result of migration (MSAR=>SDS migration)

The sds_update_retention tool may be used to sync up SDS object with the DOCTABA (EBR object is triggered). Note it is not automatically done in 4.1.1.



Other Integral SDS features: Retrieval re-direction

– SDS to MSAR/optical (primary/tranlog)

– MSAR/optical (primary/tranlog) to SDS

SDS is integrated into auto-restart write request infrastructure

– When Image Services is re-cycle, the write request(s) are restarted automatically

– Writes to SDS are guaranteed to happen

SDS_worker reads FBC objects directly from cache

– SDS_worker knows how to process FBC objects efficiently.

SDS_worker reads from MSAR surfaces directly when migrating from MSAR to SDS.

– When this is done, SDS_worker does not use page cache.

SDS always create documents with checksums.

– If a checksum was not created during capture (no checksum in the long descriptor), Integral SDS will calculate checksum and write it at end of the SDS BLOB object



Lab Services Packages

Integral SDS Installation Package - PS0064

– “Day forward” configuration

Integral SDS Migration Mentoring Package – PS0065

– “Backfile” historical document migration (OSAR/MSAR)

– Conversion from NLS to ISDS



Questions?

Pete Stryjewski - Lab Services Platform Services

– [email protected]

Craig Werner – Lab Services Sales Manager

– [email protected]

– (Office) 916-641-445

images services 4.1.1

Technology