autoarchive ?:the ads and the swordarm project

Your Name

AutoArchive?:The ADS and the SWORDARM project

Catherine Hardman - Archaeology Data ServiceUniversity of York

White Rose/RoaDMap 24th May 2012

http://archaeologydataservice.ac.uk


Archiving Digital DataAfter 15+ years experience of digital archiving within archaeology, we are seeing a move toward some organisations submitting digital archives to the ADS as a norm.

Southampton's Designated Archaeology Collections Programme

Wessex Archaeology Image Archive


The origins of the SWORDARM project

These, and other, projects have highlighted the need for an automated system for file upload and metadata deposition if the process of a

scalable and sustainable digital archive is to become a reality.

… a semi-automated, project-based system, that allows for the deposition of archaeological data and metadata through the use of a SWORD-style protocol. It will develop a costing module that will make the charging process more transparent for the community. It will streamline the ADS infrastructure making its procedures and, perhaps more significantly, make the future of the digital repository more sustainable in the long-term.


What will the automated ingest system do?

1. Allow users to upload files directly to the ADS repository2. Building on existing structure it will link files to existing metadata from the

online recording form for fieldwork projects (OASIS)3. Allow users to add additional and new metadata for projects4. Semi-automate the accessioning of files and associated metadata within the

ADS5. Collections Management System, streamlining the archiving process6. Allow users to set up and manage ‘accounts’ that will allow for the creation of

sample costing’s, will allow them to make informed decisions over the selection and retention strategies of the archive and provide for the semi-automated billing of charges.


Managing deposits: current workflow

CollectionManagement

System

Archiving

Dissemination

Digital Archivist

Manual input of metadata and data conversion

Problems • Protracted

Physical movement of data and metadata is not ideal

Archive or metadata often incomplete = negotiation

• InefficientDuplication of effort

• Inflexible for user


Managing deposits: SWORDARM workflow

PCMSSword-Arm

‘Clearing House’ for

data and metadata

• Virus check• Checksum• Creation of file

level metadata (DROID)

• Allocation of DOI

• Simple automated conversions

• Account Management

• Upload data • Management of

the deposit• Creation of

collection and file level metadata

• Costing module• Licensing

Sword-Arm System

Once submitted the archive

is passed

to the archivist for

validation

CollectionManagement

System

Archiving

Dissemination


Why SWORD ?SWORD, or Simple Web-service Offering Repository Deposit, Protocol

• is a light-weight Atom publishing protocol, developed by JISC, that has been designed to facilitate the deposition of resources into remote scholarly repositories; this interoperable standard has consequently been used to alleviate the problem for digital archives in develop such web-based clients

• developed out of an awareness of the trend within the digital community to standardise the outputs of digital repositories through OAI-PMH, DC metadata etc. BUT little or no standards for getting data into these repositories

• Is currently utilised by DSpace, EPrints, Fedora, and Intralibrary repositories

http://swordapp.org/http://www.ariadne.ac.uk/issue54/allinson-et-al/

http://swordapp.org/

http://www.ariadne.ac.uk/issue54/allinson-et-al/

http://www.ariadne.ac.uk/issue54/allinson-et-al/


SWORD Protocol

SWORD Protocol

But…its initial development stemmed from a requirement for the deposit of scholarly communications outputs into repositories – these typically being small text-based items

the problem…much archaeological data is neither simple or text-based

as a result…while the SWORD protocol can be used for many deposits… e.g. Grey Literature, and other text-based files…for more complex and large datasets a SWORD-style protocol will be implemented


Metadata generation


Cost Module

Sand BoxAllowing users to ‘test’ various

costing options

SWORD ARMFormal ‘quote’ for archiving

based on a per file basis

Cost ModuleUnregistered Registered

• Cost control • Depositor control of archive allowinginvolvement in selection and retention of files• Manage multiple depositions• Pricing based upon ‘economies of scale’• Financial encouragement for use, making it ‘cheaper’


Cost Control

… we have tried to keep things simple with a start up fee covering elements of management and administration for each project and then a range of prices usually per file; the more complex the file type the more expensive to deposit. So we’d charge £1 for a .csv file and up to £6 for an .au audio file that takes a lot more time to check, document and preserve. Experience has taught that the only circumstance where we can really benefit from economies of scale is when dealing with image files and that too is taken into consideration within the charging model.

ADS BlogMarch 21st 2012

http://archaeologydataservice.ac.uk/blog/sword-arm/



Outcomes

• Extend the use of the SWORD protocol• Allow the ADS to enhance its data management structures, semi-automating

metadata capture, file upload, and account management• Provide an exemplar of the integration of a charging module within data management

infrastructures• Help researchers to deposit their research data• Provide case studies from four UK HE institutions for deposit in a repository using the

SWORD client• Embed the outputs within a preservation infrastructure and facilitate the policy

objectives of research councils and non HE public bodies to enable project-based charging and deposit.

Workshops: August 2012 – Testing of demo version – University of York March 2013 – Testing of full version by institutional managers


Some Useful Links

SWORD-ARMInformation on the SWORD-ARM project

http://archaeologydataservice.ac.uk/research/swordarm

Keep up to date with SWORD-ARM through the project bloghttp://archaeologydataservice.ac.uk/blog/sword-arm/

The SWORD websiteFor more information on the SWORD protocol http://swordapp.org/








autoarchive ?:the ads and the swordarm project

Documents