drambora auditors training: chapel hill, june 09 1 session 0: drambora chapel hill

95
DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

Upload: gilbert-shields

Post on 01-Jan-2016

218 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

DRAMBORA Auditors Training: Chapel Hill, June 091

Session 0: DRAMBORA

Chapel Hill

Page 2: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

2 DRAMBORA Auditors Training:

DRAMBORA

• Digital Repository Audit Method Based on Risk Assessment

• Jointly developed by the Digital Curation Centre (DCC) and DigitalPreservationEurope (DPE)

• First released in March 2007• DRAMBORA provides:

– A methodology for conducting repository self-assessments

– An on-line tool to facilitate the assessment and document its results – DRAMBORA Interactive

– Reporting tools on the outcomes of the assessment

Page 3: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

3 DRAMBORA Auditors Training:

DRAMBORA• Is about self-assessment and is aimed

towards validating appropriateness of repository's efforts

• Is designed to help the repository to:– develop an organisational profile: describing

and documenting mandate, objectives, activities and assets

– identify and assess the risks that impede their activities and threaten their assets

– manage the risks to mitigate the likelihood of their occurrence

– establish effective contingencies to alleviate the effects of the risks that cannot be avoided

Page 4: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

4 DRAMBORA Auditors Training:

DRAMBORA

• Supports:– Validation

• “Are my efforts successful?”– Preparation

• “What must I do to satisfy external auditors?”

– Anticipation • “Are my proposals likely to succeed?”

– Risk management in several other areas: records management, appraisal, paper collection management, etc.

Page 5: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

5 DRAMBORA Auditors Training:

Outcomes and results

• Following the successful completion of the self-assessment, organisations will have:– Established a comprehensive and

documented self-awareness of their mission, aims and objectives, and of intrinsic activities and assets

– Constructed a detailed catalogue of pertinent risks, categorised according to type and inter-risk relationships

– Created an internal understanding of the successes and shortcomings of the organisation

– Provided the organisation with a tool for continuous management of risks

– Prepared the organisation for subsequent external audit

Page 6: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

6 DRAMBORA Auditors Training:

Digital repository

• An increasing range of content collections are referred to as repositories in a variety of areas

• Widespread use of a term goes hand in hand with increasing diversity of meanings

• In real life, not all repositories are alike, created for the same purpose or delivering a similar range of services

• Not all of them even aim to preserve the content they are holding

Page 7: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

7 DRAMBORA Auditors Training:

Digital repository

• Repositories form an intersection of interest for different communities of practice: digital libraries, research, learning, e-science, publishing, commercial data exploitation, records management, preservation

• Within these communities the motivation for creating repositories differs, and the key services that repositories might provide range over many functional areas

Page 8: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

8 DRAMBORA Auditors Training:

The Call for Repository Certification

“A critical component of the digital archiving infrastructure is the existence of a sufficient number of trusted organizations capable of storing, migrating, and providing access to digital collections…

A process for certification of digital archives is needed to create an overall climate of trust about the prospects of preserving digital information.”

Task Force on Archiving of Digital Information, Preserving Digital Information, 1996.

Page 9: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

9 DRAMBORA Auditors Training:

Trust, Trustworthiness and Safe Stewardship

• Evolution of the Digital Preservation (specifically Repository) Landscape:– Defining the problem

• Preserving Digital Information• Trusted Digital Repositories: Attributes & Responsibilities

– Practical Responses to the problem• repository software [DSPACE, ePrints, Fedora];• metadata schema [PREMIS];• reference models [OAIS];

• This work focuses on determining the success of the solutions we propose or have already deployed

• “Stewardship is easy and inexpensive to claim; it is expensive and difficult to honor, and perhaps it will prove to be all too easy to later abdicate” Lynch (2003)

Page 10: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

10 DRAMBORA Auditors Training:

Chronology of repository audit work• 2002: Trusted Repositories Attributes & Responsibilities

• 2002: Reference Model for an Open Archival Information System

• 2005: RLG/NARA Draft Audit Check-list for Repository Certification released for public comment

• 2006-2007: CRL and DCC Pilot Repository Audits• Dec 2006: Catalogue of Criteria for Trusted Digital

Repositories published (in English) by nestor• Feb 2007: Digital Repository Audit Method Based on Risk

Assessment (DRAMBORA) published by DPE/DCC• Mar 2007: Trustworthy Repositories Audit & Certification

(TRAC) Criteria and Check-list published by CRL and OCLC• 2007: Birds of a Feather group of audit checklist

standardisation• Mar 2008: DRAMBORA Interactive released• May 2008: Data Seal of Approval by DANS• Nov 2008: Version 2 of the nestor repository criteria

Page 11: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

11 DRAMBORA Auditors Training:

The Problem

• How do we know that our data are secure

• Trust– How is it established?– How is it maintained?– How is it secured?– What happens when it is lost?– How can it be verified?

Page 12: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

12 DRAMBORA Auditors Training:

…be trusted

• Processes: – Workflows– Operation (management of integrity,

authenticity, intelligibility, and accessibility

– Automation (e.g. ingest, management, publication)

– Documentation of procedures– Auditability

• Architecture and Implementation• Organisation………..[and more]

Page 13: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

13 DRAMBORA Auditors Training:

Repository Operation

• Change will be a feature of repositories– Storage technologies– Services, close down of some and

initiation of others– Workflows– Verification mechanisms– Migration, refreshing, emulation

Page 14: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

14 DRAMBORA Auditors Training:

Audit as a starting point• Independent measuring of repositories is

seen as essential aim• Taken as axiomatic that audit is a

mechanism for establishing the trustworthiness of a repository

• Internal audit– Self assessment– Internal Audit Service

• External audit– Financial auditing– Operational auditing– IT/EDP systems and services audit

Page 15: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

15 DRAMBORA Auditors Training:

Audit Challenges• What do we want to achieve or pursue with

audit?• What should be audited (e.g. repository

level, data set capability)? Under what circumstances?

• Who should do the audits (e.g. specialised bodies or not)?

• What are the requirements for auditing organisations?

• What framework(s) do we need in relation to the different business contexts to conduct an audit?

• What processes and steps are necessary to conduct a proper audit?

• What steps should the audit process encompass?

• Should an audit be followed by (deliver) certification?

Page 16: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

16 DRAMBORA Auditors Training:

Audit / Review

• What information is needed for an audit?– Policies defined by the repositories– Inner workings of the repositories (workflows)– Chain of custody/process repositories

• Who will perform the audits?• Information necessary for audit and

monitoring must created and maintained– Important for the understanding of the context

of the holdings for future users

Page 17: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

17 DRAMBORA Auditors Training:

DRAMBORA: Pre-History DCC Pilot Audits

The UK Digital Curation Centre engaged in a series of pilot audits of a TRAC draft in diverse environments (2006-2007)

• 6 UK, European and International organisations: national libraries, scientific data centres, cultural heritage archives

• Rationale– establish evidence base– establish list of key participants– refine metrics for assessment– contribute to global effort to conceive audit

processes– establish a methodology and workflow for audit

Page 18: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

18 DRAMBORA Auditors Training:

General conclusions• Need to describe evidence base

– To contribute towards consistency– To create a mechanism that ensures

conclusions can be validated and replicated– Practical, applicability depends on

identification of objective means to demonstrate compliance

– Efforts must probe for evidence of concrete processes, structures and functionality

– Documentary, testimonial, and observational evidence

• Need to establish ‘preservation pressure points’ including uncertainties and risks– Risk awareness is low within the community

Page 19: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

19 DRAMBORA Auditors Training:

Planning-based

Resource-based

Capacity-based

Commitment-based

Documentation-based

Primary Documentation

Unsubstantiated

Stakeholder Testimony

Secondary

Documentation

Corroborated

Stakeholder Testimony

Supervised Practical

DemonstrationAuditor Directed

Demonstration

Evidence base of an audit

Page 20: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

20 DRAMBORA Auditors Training:

Documentary Evidence

• Sometimes mere presence will be encouraging, other times content will require scrutiny

• Several example documents– Risk Register– Repository Mission Statement– Example Deposit Agreements (including legal

arrangements)– Job Descriptions– Organisational Chart– Staff Profiles/CVs/Resumes– Annual Financial Reports– Business Plan– Policy Documents

Page 21: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

21 DRAMBORA Auditors Training:

Documentation (continued)

– System Procedure Manuals– Technical Architecture– Maintenance Reports– Results of Other Audits– Other Documentation Records

• Document and records management processes provide insights

• Privacy concerns must be addressed• Evaluation methods must be refined

Page 22: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

22 DRAMBORA Auditors Training:

Testimonial Evidence

• Useful means to:– highlight where omissions exist in

documentation– validate whether documented aspirations are

realised in reality

• Roles for interview:– Repository Administrators– Hardware and Software Administrators– Repository Function-specific Officers– Depositors– Information Seekers

Page 23: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

23 DRAMBORA Auditors Training:

Observation of Practice Evidence

• Less objectively quantifiable, but nevertheless important

• Especially appropriate in terms of procedure and workflow

• Might include– walkthroughs– testing and measurement of

characteristics of objects after preservation action

– deposit and assessment of test objects (perhaps incrementally over several audits)

Page 24: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

24 DRAMBORA Auditors Training:

Conclusions from TRAC pilots

• Existing methods are:– too static – ‘one size fits all’ approach– too much fixed on the OAIS reference

model– too little emphasis on evidence in the

auditing process

• Audit results should help to manage the repository better continuously, not just give a one-time evaluation

Page 25: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

25 DRAMBORA Auditors Training:

DRAMBORA Assessment Principles

• It should be a self-audit that repositories do themselves, based on the provided tools

• Self-audit could be a preparatory step for taking an external audit

• It should be flexible and be valid for repositories of all shapes and sizes and of different contexts

• It should be assessing how well the repository is managing the risks it is facing when it does what it does

• It should offer advice on how to overcome the risk situations and what other repositories have done in similar situations

Page 26: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

26 DRAMBORA Auditors Training:

Linking Risk and Trust

• Are repositories capable of:– identifying and prioritising the risks that impede

their activities?– managing the risks to mitigate the likelihood of

their occurrence?– establishing effective contingencies to alleviate

the effects of the risks that occur?

• If so, then they are likely to engender a trustworthy status – if they can demonstrate these capabilities

Page 27: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

27 DRAMBORA Auditors Training:

DRAMBORA: The Beginning• Easy to say establish evidence and

recognise risk, but how do you do this and then take advantage of this knowledge

• Digital Repository Audit Method Based on Risk Assessment (DRAMBORA)

• Provides mechanisms to facilitate internal self-assessment & reporting– Validates appropriateness of repository's efforts– Provides means to generate appropriate

documentation

• External certification less of a priority currently, and less immediately viable

Page 28: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

28 DRAMBORA Auditors Training:

DRAMBORA: Ancient History

• Follows lessons learned from DCC pilot audits

• A collaborative exercise between DCC and DigitalPreservationEurope (DPE)

• Audit methodology released in March 2007

• http://www.repositoryaudit.eu• Four public tutorials (London, the Hague,

Arlington, JISC Repositories Conference)• A test period within the DPE project

Page 29: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

29 DRAMBORA Auditors Training:

Testing DRAMBORA 1.0• National Archives of Scotland,

Edinburgh, UK • National Library of the Czech

Republic• National Central Library of

Florence, Italy• International Institute for Social

History, Amsterdam, The Netherlands

• Netarkivet (Danish Internet Archive), Denmark

• Ludwig Boltzmann Institute in Linz, Austria, in cooperation with the Ars Electronica Center

• E-LIS repository managed by CILEA, Rome, Italy

• Lithuanian Museum of Ethnocosmology, Lithuania

Page 30: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

30 DRAMBORA Auditors Training:

What DRAMBORA users learned…

• “Good, visible and persuading documentation of risks might help to improve conditions for their successful management. And, of course, as soon as you have the truly trusted repository, you need the good documentation and certification to prove it”

• “We discovered some points of weakness in the repository and also learned to stop fretting about the stuff we actually do very well”

• “Assessment will be continued and the risk register will be an integral part of the repository once it becomes operational”

• “We originally planned to use TRAC for both our internal and later external audit. We also looked at NESTOR. […] we believe that regular self audits using DRAMBORA will make the external audit easier and cheaper”

Page 31: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

31 DRAMBORA Auditors Training:

DRAMBORA: Renaissance

• The pilot audits validated the methodology's effectiveness - audit as a standalone process has demonstrable value

• Development of DRAMBORA Interactive as an on-line tool to support and guide the audit

• Another round of pilot audits within the DPE to test the on-line tool and to provide feedback for improving it

Page 32: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

32 DRAMBORA Auditors Training:

DRAMBORA Interactive

www.repositoryaudit.eu

Page 33: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

33 DRAMBORA Auditors Training:

Testing DRAMBORA 1.0 and 2.0

DELOS Digital Preservation Cluster• MBooks Michigan-Google Digitization Project, US• CERN Document Server, Switzerland• Kungliga Biblioteket, Stockholm• Gallica, National Library of France

Among other users• British Library, London, UK • US Geological Survey• European repositories and archives • American universities• National Diet Library Japan

Page 34: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

34 DRAMBORA Auditors Training:

DRAMBORA and DELOS• To allow comparisons

between peer organisations, profiles of repository types need to be developed

• An attempt at a typical digital library risk profile included in the DELOS report “Investigation of the potential application of the DRAMBORA toolkit in the context of digital libraries to support the assessment of the repository aspects of digital libraries”

• Support for peer comparisons should be built into the DRAMBORA Interactive system

Page 35: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

35 DRAMBORA Auditors Training:

DRAMBORA in Japan

• Grant awarded by the Great Britain Sasakawa Foundation, DRAMBORA

• The toolkit was presented at a HUSCAP repository meeting, University of Hokkaido; Digital Repository Foundation workshop, Library Fair 2008, Yohokama; Digital Archive meeting of the National Archives of Japan in Tokyo; Digital Library meeting of the National Diet Library in Kyoto.

• DRAMBORA applicability in Japanese libraries archives and data centres/repositories has been investigated and discussed with hosting organisations, and further developments on this research will be available soon.

Page 36: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

36 DRAMBORA Auditors Training:

DRAMBORA: Present and Future

• Stats March 2009:– 1278 downloads of the toolkit in its .pdf

('printed') form– 2139 downloads of accompanying forms and

other materials in .doc and .xls format– 139 registered repositories within the

DRAMBORA interactive online systemAccreditation of DRAMBORA Auditors

• Development of training materials to support self-assessment

• Discussion with other working groups developing repository audit checklists

Page 37: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

37 DRAMBORA Auditors Training:

DRAMBORA Conclusions• In isolation, or combined with objective

guidelines, DRAMBORA offers benefits to repositories both individually and collectively

• DRAMBORA Interactive is offering more than just increased usability

• We need to train audit facilitators and guide the assessment process in the system

• The concept of ‘trust’ in repositories is evolving through this work and discussions with other groups involved in similar work

Page 38: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

38 DRAMBORA Auditors Training:

International Coherence

• Practical aspects of the collaboration:– Definition of shared set of Digital Repository principles– Active cross pollination of ideas and personnel– Ongoing pilot audit programmes of CRL, DCC, DPE and

nestor– Mutual review of documentary outputs

• Implicit aspects of the collaboration– Pursuing a shared vision with a culture of shared

action– A successful international program of assessment requires

community consensus, rather than competing strategies for repository audit and certification.

• Some differences reflect geopolitical obstacles – but the commonalities are much more noteworthy

• Efforts are ongoing to overcome such impediments

Page 39: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

39 DRAMBORA Auditors Training:

Benefits of TRAC, nestor and DRAMBORA

• Support the planning of new repositories or repository functions

• Facilitate organisational self-awareness of repository successes and shortcomings

• Engender trust from depositors, users, funders and other stakeholders

• Provide a necessary precursor to the viability of widespread preservation repository services

• Present an intellectual foundation upon which formal certification services can be built

Page 40: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

40 DRAMBORA Auditors Training:

Beneficiaries• Repository administrators or those

planning to develop repository infrastructures

• Those considering utilising repository services– Information creators– Information depositors– Information end-users

• Funding bodies• Repository management• Accredited certifying agencies

Page 41: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

41 DRAMBORA Auditors Training:

The Bottom Lines

• Mechanisms for measuring and demonstrating success are vital

• Recent years have seen an increased sense of the importance of performance and capability measurement

• This consortium represents an international consensus corresponding to a unified international drive of activity

• Integrated not simply by consortium agreements, but by a shared will in favour of delivering that which is required and expected by the international community

Page 42: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

42 DRAMBORA Auditors Training:

URLs

Page 43: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

43 DRAMBORA Auditors Training:

Key premises• Repositories will be of many types

and sizes, and that preservation requirements must be scaled to the needs and means of a particular repository’s identified communities

• Repositories can be organisations whose core business is acquisition, preservation and dissemination; but

• Repositories can also form part of a larger organisation with a very different mission

Page 44: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

44 DRAMBORA Auditors Training:

Digital preservation repository

• In January 2007 the Center for Research Libraries (CRL) hosted a meeting of projects (DCC/DPE/nestor/CRL) developing repository audit checklists.

• This meeting resulted in the development of a common set of criteria to which all digital preservation repositories, regardless of their mission, business model and source of funding, should adhere

Page 45: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

45 DRAMBORA Auditors Training:

Digital preservation repository core criteria

© H

AT

II UofG

lasgow, 2007

• An intellectual context for the work:– Commitment to digital object maintenance

– Organisational fitness

– Legal & regulatory legitimacy

– Effective & efficient policies

– Acquisition & ingest criteria

– Integrity, authenticity & usability

– Audit trail and metadata

– Dissemination

– Preservation planning & action

– Adequate technical infrastructure

http://www.crl.edu/content.asp?l1=13&l2=58&l3=162&l4=92

Page 46: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

46 DRAMBORA Auditors Training:

Ten Characteristics of Repositories

• Commits to continuing maintenance of digital objects for its identified community(ies).

• Demonstrates organisational fitness (including financial, staffing, structure, processes) to fulfil its commitment.

• Acquires and maintains requisite contractual and legal rights and fulfils responsibilities.

• Has effective and efficient policy framework.• Acquires and ingests digital objects based upon stated

criteria that correspond to its commitments and capabilities.• Maintains/ensures the integrity, authenticity and usability

of digital objects it holds over time. • Creates and maintains requisite metadata about actions

taken on digital objects during preservation as well as about the relevant production, access support, and usage process contexts before preservation.

• Fulfils requisite dissemination requirements.• Has strategic programme for preservation planning and

action.• Has technical infrastructure adequate for continuing

maintenance and security of digital objects.

Page 47: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

47 DRAMBORA Auditors Training:

Using the core characteristics

• The 10 core characteristics of digital preservation repositories provide:– The scope of what an assessment

should be looking at– The structure for carrying out the

repository assessment and presenting its results

• The characteristics can be divided into:– Operational and support functions /

criteria

Page 48: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

48 DRAMBORA Auditors Training:

Kind of risks

• Assets or activities fail to achieve or adequately contribute to relevant goals or objectives

• Internal threats pose obstacles to success of one or more activities

• External threats pose obstacles to success of one or more activities

• Threats to organisational assets: information (databases, data files, contracts, agreements, documentation, policies and procedures), software assets, physical assets, services and utilities, processes, people, intangibles such as reputation and cultural value

Page 49: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

49 DRAMBORA Auditors Training:

Example Risk

• Loss of Trust or Reputation– One or more stakeholder communities have doubts

about the repository's ability to achieve it's business objectives

• Example manifestation– Irrecoverable loss of digital objects provoke

community concerns about competence– public statement about cut in funding raises concerns

about viability of repository's continued operations

Page 50: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

50 DRAMBORA Auditors Training:

“Commitment to preservation”Repository commits to continuing maintenance of digital objects for its

identified community(ies)

Mandate or mission statement of the repository:

• To provide a cost-effective, long-term preservation repository for digital materials in support of teaching and learning, scholarship, and research in Scotland

• To collect, list and preserve STM e-thesis as well as making them available to the public

• To focus and strengthen the National Library's efforts to create digital content, and to collaborate with others to ensure that citizens have barrier-free access to the record of their heritage

Page 51: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

51 DRAMBORA Auditors Training:

Uses of mandates

• Mandate should help to define:– who the repository services are for– who are the target users of the content– what kind of content needs to be collected – how long is the content to be kept– what kind of services should the

repository offer

Page 52: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

52 DRAMBORA Auditors Training:

Typical risks

Risk:• Mandate of the repository is not

formalisedDescription:• Repository cannot relate its

activities to its mandate or mission statement and thus verify the efficiency of its strategic planning

Page 53: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

53 DRAMBORA Auditors Training:

Typical risks

Risk:• Lack of definition of service levels

Description:• Repository’s ability to deliver

preservation services based on the quality of the submitted content has not been formalised, leading to false perception of security for depositors and end-users

Page 54: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

54 DRAMBORA Auditors Training:

Typical risks

Risk:• Failure of succession planning and

sustainability

Description:• Does the budget and organisational

structure support continuing commitment to the mission statement

• Lack of appropriate exit strategy or plan for succession can lead to a false perception of security for depositors and end-users

Page 55: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

55 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 56: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

56 DRAMBORA Auditors Training:

“Digital object maintenance”

Repository maintains the integrity, authenticity and usability of digital

objects it holds over time• Preserve original files exactly as submitted,

with demonstrated integrity, viability and authenticity

• Ensure measures are in place to protect and demonstrate the authenticity of the digital objects

Page 57: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

57 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 58: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

58 DRAMBORA Auditors Training:

Typical risks

Risk:• Delivered digital object cannot be traced to

a received object

Description:• The MD5 hash code does not include a

time stamp leading to mistrust of digital objects

• Audit trail not implemented in the digital archive software

• Lack of administrative metadata in the digital archive

Page 59: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

59 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 60: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

60 DRAMBORA Auditors Training:

Typical risks

Risk:• Lack of an automated mechanism for

updating already ingested digital objects

Description:• Several versions of the same digital

object are being stored in parallel without version control function, which may result in the wrong version of the digital object being delivered to the user

Page 61: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

61 DRAMBORA Auditors Training:

Typical risks

Risk:• Identifier collision

Description:• Human error at ingest or other

processing results in two digital objects with the same ID, which leads to problems at dissemination and storage management

Page 62: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

62 DRAMBORA Auditors Training:

“Preservation Planning & Action”

Repository has a strategic program for preservation planning and action

• Regularly reviews technological developments for preservation

• Has adopted a strategic preservation planning approach• Has mechanisms to monitor effectiveness of

preservation action• Has awareness of services and registries

Page 63: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

63 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 64: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

64 DRAMBORA Auditors Training:

Typical risks

Risk:• Preservation planning is not

formalised Description:• Preservation is not part of anyone’s

job description or list of duties at the digital library. No unit is tasked with making decisions and planning for preservation as it is too specific a task and there is a general lack of resources to develop preservation tools and services

Page 65: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

65 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 66: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

66 DRAMBORA Auditors Training:

Typical risks

Risk:• Lack of a formalised review of

technology trends

Description:• Technology trends for development

of digital library services and preservation are not included in overall technology watch plan

Page 67: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

67 DRAMBORA Auditors Training:

Typical risks

Risk:• Lack of knowledge of a

comprehensive list of file formats in the digital library collection

Description:• A comprehensive list of file formats in

the digital library's care is not easily available and makes it difficult to plan for preservation activities

Page 68: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

68 DRAMBORA Auditors Training:

Typical risks

Risk:• Identified file formats are not

compatible with users’ expectations

Description:• Users do not like or cannot use some

of the dissemination file formats chosen by the digital library

Page 69: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

69 DRAMBORA Auditors Training:

Typical risks

Risk:• Insufficient number of copies of

digital objects is kept

Description:• A system failure or storage media

error may result in loss of digital objects

Page 70: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

70 DRAMBORA Auditors Training:

Typical risks

Risk:• Preservation strategies fail to

preserve the significant properties of digital objects

Description:• Adopted preservation strategies may

result in loss of some of the significant properties, e.g. semantic value of particular objects

Page 71: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

71 DRAMBORA Auditors Training:

“Organisational fitness”

Demonstrates organisational fitness (including financial, staffing, structure,

processes) to fulfill its commitment

• What makes an organisation work?– Budget

– Staff

– Organisational structure

– Business model and processes

Page 72: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

72 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 73: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

73 DRAMBORA Auditors Training:

Typical Risks• Loss of key member(s) of staff

– Individuals with roles, responsibilities or aptitudes vital to the achievement of business objectives part company with the repository, rendering achievement of those objectives less straightforward

• Example manifestation– Repository head systems administrator,

the sole individual with knowledge of the system's root password, leaves the organisation to work elsewhere

Page 74: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

74 DRAMBORA Auditors Training:

What risks can you think of from your own experience?

Page 75: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

75 DRAMBORA Auditors Training:

Typical risks

Risk:• Organisational inability to cope with a

shift in the scale and scope of repository services

Description:• Repository cannot fulfil its mission or

offer its core services

Page 76: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

76 DRAMBORA Auditors Training:

Typical risks

Risk:• False perception of success

Description:• Repository believes that its efforts are

much more or less effective than is actually the case

Page 77: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

77 DRAMBORA Auditors Training:

Typical risks

Risk:• Obsolescence / stagnation of staff skills

Description:• Staff members are no longer equipped to

deal with emerging concepts, technologies or their skills are generally outdated. Appropriate training is not available or is inaccessible

Page 78: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

78 DRAMBORA Auditors Training:

Typical risks

Risk:• Insufficient communication between

units responsible for repository services

Description:• Repository does not offer the services

that it could and it takes longer to develop these because of communication inefficiencies

Page 79: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

79 DRAMBORA Auditors Training:

Topics for discussion

1. Self-assessment as a useful method2. DRAMBORA methodology3. Risk approach to repository assessment4. Further developments of DRAMBORA

Interactive5. Trust in digital preservation repositories6. Certification of repositories

Page 80: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

80 DRAMBORA Auditors Training:

Why assess repositories with DRAMBORA

• Aligned with international efforts• Evidence-based approach using risk as a

metric• Repository level management • Self-assessment• Identify, prioritise and manage risks,

verifying compliance, checking effectiveness and identifying opportunities for improvements

• DRAMBORA interactive interface to facilitate the collection of information necessary to conduct a risk-analysis assessment, its analysis and reporting

• Working towards automating the process that DRAMBORA encapsulates

Page 81: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

81 DRAMBORA Auditors Training:

Benefits of DRAMBORA

Following the successful completion of the self-audit, organisations can expect to have:

1. Established a comprehensive and documented self-awareness of their mission, aims and objectives, and of intrinsic activities and assets

1. Constructed a detailed catalogue of pertinent risks, categorised according to type and inter-risk relationships

1. Created an internal understanding of the successes and shortcomings of the organisation

1. Prepared the organisation for subsequent external audit

Page 82: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

82 DRAMBORA Auditors Training:

How long will this take?

• Unfortunately, this depends

• Has your organisation already defined:– Objectives?– Constraints?– Policy?– Risks?

• If so, it will be a straightforward process

• If not… why not?

• DRAMBORA is not just for Christmas!

Page 83: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

83 DRAMBORA Auditors Training:

“DRAMBORA is too…”

• “Hard”?• “Time consuming”?• “Involved”?• The audit process is

undoubtedly a challenging one, but intended at its most fundamental level to reflect good repository management

• The tool makes it easier, but it’s not trivial, and remains a highly interactive process

Page 84: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

84 DRAMBORA Auditors Training:

Limits of self-assessment

• DRAMBORA is fundamentally 'bottom-up'

• Comparability and reproducibility of results are compromised

• More concerning is that improvement in self assessment is limited by one's own horizons

• How can repositories comment on unanticipated risks? When they are unaware of available opportunities?

Page 85: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

85 DRAMBORA Auditors Training:

Our experience of Self-Assessment

• Most of the pilot audits have been facilitated by an expert who has training for DRAMBORA

• Is improvement in ‘bottom-up’ self assessment limited by one's own horizons?

• How can repositories comment on unanticipated risks? When they are unaware of available opportunities?

• What damage can dishonest auditors do?• Are comparability and reproducibility of

results compromised?

Page 86: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

86 DRAMBORA Auditors Training:

Assessment Methodology

• The workflow and stages of the assessment are understandable, and have become more user-friendly in DRAMBORA Interactive

• Flexibility in defining the scope and steps of the assessment is valued by users

• Estimate of time required to complete some stages of the assessment: depends on the ‘preparedness’ of the organisation

Page 87: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

87 DRAMBORA Auditors Training:

Risk Assessment• Risk appears to be an easily understood

concept for repositories• We have had many discussions with users

about the risk impact and probability scores and scales, and have modified them slightly

• Any risk assessment leaves some room for interpretation – keep the purpose of the assessment in mind

• We have no fixed benchmark on the number of risks or their severity

Page 88: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

88 DRAMBORA Auditors Training:

Risk Assessment

• The 80 or so example risks in DRAMBORA to prompt thinking... are sufficient?

• DRAMBORA Interactive enables repositories to align their objectives, activities, strengths and shortcomings with other peer repositories' responses

• This will ultimately be collated as a series of repository profiles encapsulating key roles, responsibilities, functions and risks

Page 89: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

89 DRAMBORA Auditors Training:

Repository Profiles

• To allow comparisons between peer organisations, profiles of repository types need to be developed– An attempt for a typical digital library risk

profile included in the DELOS report “Investigation of the potential application of the DRAMBORA toolkit in the context of digital libraries to support the assessment of the repository aspects of digital libraries”, 2008

• Support for peer comparisons should be built into the DRAMBORA Interactive system

Page 90: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

90 DRAMBORA Auditors Training:

Service Classification• We want to identify and describe classes of

repositories in terms of their common services and characteristics

• Services are critical, with performance understood in terms of those services

• Auditors can space their own efforts within the context of comparable repositories

• They can reflect and inform the perspective of best practice that exists within their own particular 'repository-sphere'.

Page 91: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

91 DRAMBORA Auditors Training:

Trust in Repositories

• Strong link between the organisational context of the repository and its users’ expectations

• Different focus on preservation in archives and data centres

• The concept of ‘trust’ is varying from one user community to another

• Linking ‘trust’ to services that a repository is offering is more meaningful than to a whole institution or unit within an organisation

Page 92: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

92 DRAMBORA Auditors Training:

Trust in Repositories

• Is auditing repositories the answer to the problem?– do we have to look beyond repositories

given the current fundamental changes in the web-environment?

– how to address cross-organisation repositories?• Inter-repository transfer/ exchange/

networks

– what should be the scope?• individual repositories or the web and/or

services or all?

Page 93: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

93 DRAMBORA Auditors Training:

Certification• What do we certify?• Is it already possible or realistic given the immature

state of digital preservation?• What does it prove?

– what further certainty does it provide?• Who will take the responsibility for mis-judged

assessments?• Will it ‘generate trust’?• No infrastructure for repository certification yet• Could it relate to other certification processes, such

as ISO 9000, ISO 27000?• Should the trust relationship be certified between the

repository and different stakeholders individually?• DPE is accrediting auditors for self-audit with

DRAMBORA.

Page 94: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

94 DRAMBORA Auditors Training:

Conclusion• In isolation, or combined with objective

guidelines, DRAMBORA offers benefits to repositories both individually and collectively

• DRAMBORA Interactive is offering more than just increased usability

• We need to train audit facilitators and guide the assessment process in the system

• The concept of ‘trust’ in repositories is evolving through this work and discussions with other groups involved in similar work

Page 95: DRAMBORA Auditors Training: Chapel Hill, June 09 1 Session 0: DRAMBORA Chapel Hill

95 DRAMBORA Auditors Training:

Contacts

http://www.repositoryaudit.eu/

[email protected]