Structured Approach to Implementing Information and Records Management (IDRM) Systems

Alan McSweeney

April 7, 2023 2

Objectives

• Provide an overview of a structured approach to analysis, design and specification of Information, Document and Records Management (IDRM) systems

• Provide introduction to generic reference models for IDRM that can assist in solution definition

April 7, 2023 3

Information, Document and Records Management (IDRM)• Information Management tends to be IT related – backup/restore of

information under the control of IT systems− IT systems tend by their nature to be centralised

• Records Management tends to be associated with the management of paper records− Records tends to be managed in multiple locations across an organisation, local

and central filing systems− Records are seen as red tape - a relic of the old process-driven way of doing

business - a cost to be reduced, if not eliminated− Filing, dusty files and basements

• Document Management tends to associated with systems to manage paper (and other) records electronically, either at a departmental or enterprise level

• There is a division between IT information and its management and non-IT managed business records

• Spread of electronic systems has lead to a tendency towards ad hoc or substandard IDRM practices and processes

• Need to bridge the gap:− They all share the same management requirements− Common business goals and requirements

• Take an integrated approach

April 7, 2023 4

Information, Document and Records Management (IDRM)

• Information, document management and records management, retention and deletion procedures are becoming increasingly important−Driven by many regulations −Standards around correct storage and retrieval of

business records and electronic communications

April 7, 2023 5

IDRM Challenges

• Budgets - Systems must be implemented in the context of limited budgets

• Information Growth - The annual rate of growth of information to be managed in over 50%

• Compliance and Regulations - There are multiple compliance standards regulations to be adhered to

• Management and Control - Information must be managed from creation to deletion with as easily and automatically as possible

• Protection and Recovery - While being managed the information must be protected and recoverable

April 7, 2023 6

General Approach to IDRM

• An effective approach to IDRM supported by processes and systems:− Creates a culture of compliance to business information

management− Ensures quality, enforces standards and guidelines − Eliminates risk due to lost or unsaved documents and

information

• Designs IDRM systems and associated processes that allow you to actively managing information during its entire lifecycle, according to its value:− From creation to deletion− Consistent with business importance− With automated management − Aligned with business needs

April 7, 2023 7

General Information Management Lifecycle

• Define solution to implement lifecycle

• Define specific information lifecycle for different information types being analyses

Create, Acquire, Update, Capture

Store, Manage, Replicate and

Distribute

Protect and Recover

Archive and Recall

Delete/Remove

April 7, 2023 8

IDRM Structured Analysis and Design Approach

• Analyse your information, document management and records management requirements

• Design and specify appropriate a solution and associated processes

• Depending on your requirements, this can encompass hardware, software, processes, all of which are aligned with the needs of the business

• Use a structured approach to enable the selection and implementation of IDRM solutions and processes that are based on a sound understanding of an organisation’s requirements

• Methodology is designed to ensure that any IDRM solution is strictly based on the business needs of the organisation

• Ensure solutions meet requirements

April 7, 2023 9

General IDRM Analysis and Design Methodology

April 7, 2023 10

General IDRM Analysis and Design Methodology

• Steps 1 to 4 make certain that the IDRM analysis and design is focused on the key business requirements− Preliminary investigation − Analysis of business activity− Identification of recordkeeping requirements− Assessment of existing systems

• Steps 5 to 6 focus on selecting the optimum approach− Identification of strategies for recordkeeping− Design of IDRM system

• Steps 7 and 8 are concerned with the implementation of this optimum approach− Implementation of IDRM system − Post-implementation review

• The process can be used in its entirety or a subset of it can be followed to produce interim results

April 7, 2023 11

General IDRM Analysis and Design Methodology

• Practical−Best practice implementation

• Comprehensive− Includes all information in all formats

• User-based− Identifies specific business needs and legal

requirements

• Flexible−Eight-step process can be applied at different levels

April 7, 2023 12

IDRM Analysis Benefits

• Understand the business, regulatory and social context in which they operate, and establish a business case for reviewing their IDRM practices

• Analyse business activities and environmental factors to identify their IDRM requirements

• Assess the extent to which existing organisational strategies (for example, policies, standards, technology) satisfy these requirements

• Redesign existing strategies or design new strategies to address unmet or poorly satisfied requirements

• Implement, maintain and review these strategies

April 7, 2023 13

IDRM Analysis Outcomes

• The IDRM analysis approach enable the development of IDRM systems and processes that are based on a sound understanding of an organisation’s IDRM requirements

• These systems and processes enable organisations to:− Conduct business in an orderly, efficient and accountable manner− Deliver products and services in a consistent manner− Support and document policy formation and managerial decision making− Provide consistency, continuity and productivity in management and

administration− Facilitate the effective performance of activities through an organisation− Provide continuity in the event of a disaster− Meet legislative and regulatory requirements including archival, audit and

oversight activities− Provide protection and support in litigation, including the management of

risks associated with the existence of or lack of evidence of organisational activity

− Protect the interests of the organisation and the rights of employees, clients, and present and future stakeholders

− Support and document current and future research and development activities, developments and achievements, as well as research, provide evidence of business, personal activity

− Establish business, personal and cultural identity− Maintain corporate, personal or collective memory

April 7, 2023 14

IDRM System and Process Objectives

• In order to be full and accurate, information (or all types) must be authentic, reliable, complete, unaltered and useable and the systems that support them must be able to protect their integrity over time− Authentic - it must be possible to prove that a record is what it purports

to be and that it has been created or sent by the alleged person and at the time purported. Information need to be protected against unauthorised addition, deletion, alteration, use or concealment and the creation, receipt, transmission of records needs to be controlled to ensure that records creators are authorised and identified

− Reliable - it must be possible to trust the information content as an accurate representation of the transaction to which it attests. It should be created and captured in a timely manner by an individual who has direct knowledge of the event or generated automatically by processes routinely used by the organisation to conduct the transaction

− Complete and Unaltered - it must be possible to protect information against unauthorised alteration and to monitor and track any authorised annotation, addition or deletion

− Usable - it must be possible to locate, retrieve, render and interpret information and understand the sequence of activities in which it was created and used for as long as such evidence is required

− System Integrity - it must be possible to implement control measures, such as access monitoring, user verification, authorised destruction, security and disaster recovery to prevent unauthorised access, destruction, alteration or removal of information and to protect them it accidental damage or loss

April 7, 2023 15

IDRM Processes

• There are fundamental processes that are common to IDRM systems− Capture/Acquire/Create – formally determine that information should

be made and kept− Registration – formalise the capture of information into a designated

system by assigning a unique identifier − Classification and Indexing – identify the business activities to which

a record relates and then link it to other records to facilitate description, control, retrieval, disposal and access

− Access and Security – assign rights or restrictions to use or manage particular information

− Appraisal – identify and link the retention period of information to a functions-based disposal authority at the point of capture and registration

− Storage – maintain, handle and store information in accordance with their form, use and value for as long as they are legally required

− Use and Tracking – ensure that only those employees with appropriate permissions are able to use or manage information and that such access can be tracked as a security measure

− Disposal – identify information with similar disposal dates and triggering actions, review any history of use to confirm or amend the disposal status, and maintain a record of disposal action that can be audited

April 7, 2023 16

Step 1 - Project Initiation and Initial Analysis and Investigation

• Activities−Determine the scope of the analysis−Collect information −Document research −Prepare a report

• Issues−Defining the boundaries of the analysis

• Outputs− List of the sources used−Report containing the major findings of preliminary

investigation and making recommendations on the scope, conduct and feasibility of the proposed IDRM project

−Project plan

April 7, 2023 17

Step 2 - Analyse Information-Related Business Processes• Activities

− Collect information from documentary sources and interviews− Identify and document each business function, activity and transaction

• Assigning terms to functions and activities• Defining the scope of functions and activities• Assigning dates to functions and activities

− Develop a business classification scheme − Identify organisational stakeholders − Assess risk − Record your findings − Validate the analysis

• Issues− Defining the scope of the analysis− Defining the scale of functions and activities− Maintaining the currency of the analysis

• Outputs− Document the sources used− Document organisation’s functions, activities and transactions− Highlight business-critical functions using risk assessment− Prepared a business classification scheme showing the organisation’s functions,

activities and transactions in a hierarchical relationship− Validate findings with stakeholders

April 7, 2023 18

Step 3 - Define Information and Records Management Requirements• Activities

− Locate information sources − Identify regulatory, business and external requirements for IDRM− Document identified requirements for IDRM − Assess the risk of not meeting requirements − Document results

• Issues− Drawing on other IDRM projects and experiences− Maintaining a history of IDRM requirements − Using IDRM requirements − IDRM requirements for housekeeping functions

• Outputs− Identified and documented all sources of organisation’s IDRN

requirements− Documented organisation’s identified requirements in a structured and

traceable form;− Investigation of risk and feasibility factors associated with

requirements where necessary− Documented assessment of these factors

April 7, 2023 19

Step 4 - Analyse Existing Information and Records Management Systems

• Activities− Identify existing information systems− Analyse existing systems− Document results

• Issues− Assessing housekeeping functions− Duplicate systems

• Outputs− Benchmark to assess organisation’s existing systems and

practices− Identified, surveyed and documented relevant systems− Assessment whether the systems have the capacity to function

as IDRM systems − Determined whether the systems currently create, capture and

manage evidence consistent with your prioritised IDRM requirements

− Prepared a report describing the strengths and weaknesses of the

− existing practices

April 7, 2023 20

Step 5 - Define Information and Records Management Strategy• Activities

− Investigate the range of strategies available• Policy tactics• Design tactics• Implementation tactics• Standards tactics

− Identify appropriate tactics• Policy tactics• Design tactics• Implementation tactics• Standards tactics

− Assess factors that support or inhibit tactics• Types of activities and transactions• Corporate culture• Systems and technological environment• Assessing risks

− Check tactics against identified weaknesses− Adopt an overall strategy

• Issues− Agree where IDRM strategies ends and designing systems to incorporate those strategies begins− Get high-level commitment to the remaining design and implementation process

• Outputs− Investigation of the range of tactics potentially available to organisation− Assessment and documentation of the organisational constraints that may affect the adoption of these

tactics− Selection of the most appropriate combination of tactics− Establishment of the links between the selected tactics and the requirements they are intended to address− Development of an overall design strategy to bring the tactics to fruition (including a framework for

change management)− Support for recommended strategy or decision not to proceed, and the rationale behind it

April 7, 2023 21

Step 6 - Design Information and Records Management System and Processes• Activities

− Establish and maintain recordkeeping policies− Assign recordkeeping responsibilities− (Re)design work processes− Produce design documentation− Design electronic or hybrid systems for records creation, capture and control− Develop guidelines and operating procedures− Conduct regular design reviews− Develop initial training plan− Prepare initial system implementation plan

• Issues− Determining the limits of user participation in the design process− Determining where design stops and implementation begins

• Outputs− Feedback from staff and other stakeholders on relevant system components (such as

policies, processes, roles and responsibilities, guidelines and procedures)− Documentation of any changes to requirements and design components arising from

consultation− Prepared detailed design descriptions, logical and physical models, and technical design

specifications for electronic system components− Preparation of a system implementation plan− Preparation of a an initial training plan− Management endorsement of the design process

April 7, 2023 22

Step 7 - Select/Build and Implement Information and Records Management System• Activities

− Select implementation strategies and techniques• Documentation of policies, procedures and practices covering all aspects of IDRM systems• Communication about new or improved systems to staff both before and during the implementation

phase• Training• Conversion• Regulation and review• Quality systems• Change management

− Plan the implementation process − Manage the implementation − Develop a maintenance

• Issues− Decide to begin the implementation process and stop refining the design− Issues affecting implementation such as time lags, cost overruns and user acceptance, can be

minimised by conducting risk and feasibility assessments before the design and implementation (Step G) phases are begun

• Outputs− Representative of senior management to act as sponsor− Establishment of a project team and/or steering committee that includes relevant

stakeholders and experts− Selection of an appropriate mix of implementation strategies and techniques based on risk,

cost-benefit and feasibility assessments− Identification of all the activities associated with each of the strategies− Development of a project plan to manage the implementation process− Implemented and tested systems, processes and practices according to the project plan− Reports on the implementation process to the project sponsor, steering committee and staff

April 7, 2023 23

Step 8 - Validate System and Process Operation After Implementation• Activities

− Plan the evaluation• Appropriateness• Effectiveness• Efficiency

− Collect and analyse performance data• Information creation and retrieval• Management• Disposal• Training• Learning from the process

− Document and report on your findings− Take corrective action− Make arrangements for ongoing review

• Outputs− Identification and documentation of performance indicators for the system and process− Identification of methods to collect performance data for the initial and subsequent reviews− Collection of performance data− Assessment of whether the system is satisfying IDRM requirements and working within organisational

constraints− Production of substantiating documentation− Assessment of design and implementation process and documented costs and benefits− Review the conversion strategy, process and audit trail− Verification of the existence, currency and comprehensiveness of technical, user and training

documentation (including policies, procedures, information disposal authorities)− Preparation a report for management on the status of the system and recommendations for improvement

(including staffing issues)− Initiation of remedial action endorsed by management− Establishment an ongoing cycle of reviews of recordkeeping

April 7, 2023 24

Benefits of Structured Approach

• You establish a business case for reviewing IDRM• You understand the business processes and activities

that give rise to IDRM requirements• You define the IDRM requirements of the organisation• You assess the extent to which existing organisational

IDRM procedures meet these requirements• You get redesigned existing processes or new

processes designed to address partially of completely unfulfilled requirements

• You will get a detailed design before embarking on (an expensive) implementation

• You get a solution implemented to meet the agreed business requirements

• The operation of the system is reviewed to ensure it meets the requirements

April 7, 2023 25

Open Archival Information System (OAIS) Reference Model

• Establishes a common framework of terms and concepts• Identifies the basic functions of an OAIS• Defines an information model• Adopted as ISO 14721:2003• Provides an idealised architecture and framework for IDRM

design and implementation− OAIS is a reference model and conceptual framework) and not a

detailed blueprint for system design− Informs the design of system architectures, the development of

systems and components− Provides common definitions of terms and a common language,

means of making comparisons− Basis for defining solution for acquisition/development and

implementation• OAIS is focussed on information storage and management

rather than processes that use the information such as workflow

• A useful tool in IDRM design and implementation

April 7, 2023 26

Reference Model

• A framework that provides− For understanding significant relationships among the entities of

some environment, and − For the development of consistent standards or specifications

supporting that environment.

• A reference model− Is based on a small number of unifying concepts − Is an abstraction of the key concepts, their relationships, and their

interfaces both to each other and to the external environment− Can be used as a basis for education and explaining standards to

users and other stakeholders

• Does not specify an implementation• OAIS reference model divide IDRM into a number of

functional areas such as ingest, storage, access, and preservation planning

April 7, 2023 27

Open Archival Information System (OAIS)

• Open−Reference Model standard(s) are developed using a

public process and are freely available

• Information−Any type of knowledge that can be exchanged− Independent of the forms (i.e., physical or digital) used

to represent the information−Data are the representation forms of information

• Archival Information System−Hardware, software, and people who are responsible

for the acquisition, preservation and dissemination of the information

−Additional OAIS responsibilities are identified later and are more fully defined in the Reference Model detail

April 7, 2023 28

OAIS Reference Model

• Provides definitions of terms that need to have well-defined meanings:−Archival Storage, Content Data Object, Designated

Community (key term), Ingest, Metadata, Representation Information, etc.

• High level concepts, e.g.−The environment of an OAIS (Producers, Consumers,

Management)−Definitions of information, Information Objects and

their relationship with Data Objects−Definitions of Information Packages, conceptual

containers of Content Information and Preservation Description Information

April 7, 2023 29

OAIS Purpose, Scope, and Applicability

• Framework for understanding and applying concepts needed for long-term IDRM − Long-term is long enough to be concerned about changing

technologies−Starting point for model addressing non-digital information

• Framework for comparing architectures and operations of existing and future data stores

• Basis for development of additional related standards

• Addresses a full range of archival functions• Applicable to all data stores and those organisations

dealing with information that may need IDRM

April 7, 2023 30

Functional Model

• Six entities− Ingest: services and functions that accept SIPs from Producers;

prepares AIPs for storage, and ensures that AIPs and their supporting Descriptive Information become established within the OAIS

− Archival Storage: services and functions used for the storage and retrieval of AIPs

− Data Management: services and functions for populating, maintaining, and accessing a wide variety of information

− Administration: services and functions needed to control the operation of the other OAIS functional entities on a day-to-day basis

− Preservation Planning: services and functions for monitoring the OAIS environment and ensuring that content remains accessible to the Designated Community

− Access: services and functions which make the archival information holdings and related services visible to Consumers

April 7, 2023 31

OAIS Reference Model – High Level

• Producer provides the information to be preserved

• Management sets overall OAIS policy• Consumer retrieves and acquires information of

interest

OAIS(IDRM)

Management

Producer Consumer

April 7, 2023 32

OAIS Reference Model – Medium Level

Administration

Ingest

ArchivalStorage

Access

DataManagement

Descriptive Information

PRODUCER

CONSUMER

MANAGEMENT

Queries

Result Sets

DescriptiveInformation

Preservation Planning

Orders

SIP

SIP

SIP

DIP

DIP

AIP AIP

• SIP = Submission Information Package• AIP = Archival Information Package• DIP = Dissemination Information Package

April 7, 2023 33

OAIS Reference Model – Detail

April 7, 2023 34

OAIS Reference Model

• Information Model− Information Object (basic concept):− Data Object (bit-stream)− Representation Information (permits “the full interpretation of

Data Object into meaningful information”)• Information Object Classes− Content Information− Preservation Description Information (PDI)− Packaging Information− Descriptive Information

• Information Package− Container that encapsulates Content Information and PDI− Packages for submission (SIP), archival storage (AIP) and

dissemination (DIP)− AIP = “... a concise way of referring to a set of information that

has, in principle, all of the qualities needed for permanent, or indefinite, Long Term Preservation of a designated Information Object”

April 7, 2023 35

OAIS Responsibilities

• Negotiates and accepts Information Packages from information producers

• Obtains sufficient control to ensure long-term preservation

• Determines which communities (designated) need to be able to understand the preserved information

• Ensures the information to be preserved is independently understandable to the Designated Communities

• Follows documented policies and procedures which ensure the information is preserved against all reasonable contingencies

• Makes the preserved information available to the Designated Communities in forms understandable to those communities

April 7, 2023 36

OAIS Information Definition

• Information is defined as any type of knowledge that can be exchanged, and this information is always expressed (i.e., represented) by some type of data

• In general, it can be said that “Data interpreted using its Representation Information yields Information”

• In order for this Information Object to be successfully preserved, it is critical for an archive to clearly identify and understand the Data Object and its associated Representation Information

April 7, 2023 37

OAIS Archival Information Package

• Archival Information Package (AIP)−Content Information

• Original target of preservation• Information Object (Data Object & Representation

Information)

−Preservation Description Information (PDI) • Other information (metadata) “which will allow the

understanding of the Content Information over an indefinite period of time”

• A set of Information Objects

April 7, 2023 38

OAIS Archival Information Package (AIP)

• Content Information− Document as an electronic file together with its format description

• Preservation Description Information (PDI)− How the Content Information came into being, who has held it,

how it relates to other information, and how its integrity is assured

ArchivalInformation

Package (AIP)

ContentInformation

PreservationDescriptionInformation

(PDI)

PackagingInformation

PackageDescriptor

Further described by

Delimited byDerived from

How to find Content information and PDI onsome medium

Informationsupporting usersearches for AIP

April 7, 2023 39

Information Package

• Content− Content Data Object - Physical Object or Digital Object− Representation Information

• Preservation− Provenance− Context− Reference− Fixity

April 7, 2023 40

OAIS Preservation Description Information (PDI)

PreservationDescriptionInformation

Reference Information

ProvenanceInformation

ContextInformation

FixityInformation

April 7, 2023 41

OAIS Reference Model in Wider ContextBusiness Process

Services

Network Infrastructure

Data Models

Behavior Models

Application

Interface

Data

Encoding

Fram

ew

ork

Refe

ren

ce M

od

el

Desi

gn

Workflow

Operations

Messages

Transport

Data }OAIS

• Provides wider framework for IDRM design and implementation

April 7, 2023 42

Summary

• Structured IDRM methodology yields benefits in ensuring appropriate analysis, design and specification is performed ensuring requirements are captured and any solution complies with them− Consistency− Speed− Drives Delivery− Ensures Acceptance− Increases Productivity− Increases User Confidence− Speed− Accuracy− Provides Audit Trail− Maximises Cost Savings− Risk Management and Reduction− Provides for Change Management− Provides for Formal Project Closure

• Standards and reference models provide a mechanism for designing solutions and comparing products from vendors

April 7, 2023 43

More Information

Alan McSweeneyalan@alanmcsweeney.com