integrating dspace with duracloud duraspace webinar: 30 nov 2011 tim donohue duraspace
DESCRIPTION
Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace. Basis for DSpace Integration. DSpace AIP Backup & Restore (1.7 +) (Initial DuraCloud use case: Backup & Restore) DSpace Curation Task System (1.7 +) DSpace Replication Task Suite (1.8 add-on). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/1.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Integrating DSpace with DuraCloudDuraSpace Webinar: 30 Nov 2011
Tim DonohueDuraSpace
![Page 2: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/2.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
1. DSpace AIP Backup & Restore (1.7 +) (Initial DuraCloud use case: Backup & Restore)
2. DSpace Curation Task System (1.7 +)
3. DSpace Replication Task Suite (1.8 add-on)
Basis for DSpace Integration
![Page 3: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/3.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Primary Use Cases– Backup & Restore of DSpace Content
• All content or just partial (Community/Collection/Item)
– Migration/Export of DSpace Content• All content or just partial (Community/Collection/Item)
– DuraCloud Integration• Offsite backup & restore of content
Intro to Archival Info Pkgs (1.7+)
![Page 4: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/4.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Backup DSpace (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
![Page 5: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/5.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore All (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
![Page 6: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/6.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore a Collection (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Temporary Database
Temporary Folder?
![Page 7: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/7.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore a Collection (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Temporary Database
Temporary Folder?
![Page 8: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/8.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Backup via Archival Info Pkgs
Package for each Community,
Collection & Item
AIP backup
![Page 9: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/9.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore All via Archival Info Pkgs
AIP backup
Package for each Community,
Collection & Item
![Page 10: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/10.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore a Collection via AIPs
AIP backup
Collection AIP
Items in Collection
1 2
![Page 11: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/11.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
What’s in an AIP?
METS(DIM / MODS / PREMIS /
METSRights)
LicenseContent Files
or Logos
*Also a BagIt version (Replication Suite add-on)
Other Filesin Bundles
(optional)
Archival Information Package (AIP)
![Page 12: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/12.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
What’s in an AIP?
Descriptive Metadata: DIM & MODS Tech/Preservation Metadata: PREMIS
Related Object AIPs
METS(DIM / MODS / PREMIS /
METSRights)
LicenseContent Files
or Logos Other Filesin Bundles
(optional)
Rights Metadata: METSRights
![Page 13: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/13.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
The “Site” AIP
Top-Level Community
AIPsMETS
(DIM / MODS / PREMIS / METSRights)
Special AIP for site-wide info/metadata:(e.g. Group Memberships, EPeople)
![Page 14: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/14.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore All In-Archive Content (Files + Metadata)Restore All People & GroupsRestore All Permissions / Access RightsRestore Community / Collection Logos, Metadata,
Rights & Item TemplatesRestore Community / Collection / Item HierarchyRestore In-Process / Incomplete ItemsRestore Collection OAI-PMH/ORE Harvest SettingsRestore all configuration files (dspace.cfg, etc.)
What can AIPs restore?
![Page 15: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/15.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
AIP Use Case: Migrate Content
34
Items in Collection
One DSpace Install Another DSpace Install
Collection AIP
12
![Page 16: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/16.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
“Manual” AIP backup to DuraCloud
Local “Watch” Folder
1
2
DuraCloud Sync
Tool
./dspace packager -d
java -jar synctool.jar
[1]
[2]
This two step route is
required for DSpace 1.7.x
![Page 17: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/17.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
“Manual” AIP restore from DuraCloud
Package for each Community,
Collection & Item
Local Folder
1
2
DuraCloud Retrieval
Tool
java -jar retrievaltool.jar
./dspace packager -r
[1]
[2]
This two step route is
required for DSpace 1.7.x
![Page 18: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/18.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
New: DSpace Replication Suite add-on
![Page 19: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/19.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Enables a basic ‘microservices’ approach to curating DSpace objects
• Anyone can build a task & share it• Currently tasks must be written in Java
– Experimental support for non-Java tasks in 1.8• “Frees” admin tasks from Command Line
– Can now run from Admin UI or CLI– Can also “queue” tasks for later processing
Intro to DSpace Curation System
![Page 20: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/20.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• In DSpace 1.8.0, ANY curation task run across your entire DSpace site will report a “NullPointerException” error
• THIS DOES NOT MEAN YOUR TASK FAILED!• Check dspace logs to see if task did succeed• This bug will be fixed in 1.8.1
– https://jira.duraspace.org/browse/DS-1077
DSpace 1.8.0 WARNING
![Page 21: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/21.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• A set of curation tasks geared towards ‘replicating’ (backup/restore/audit) content• Compatible with DSpace 1.8.1 or above
– Not recommended on 1.8.0 (see previous slide)• “Wraps” DSpace AIP Backup & Restore tool• Provides configurable AIP storage plugins for filesystem or DuraCloud• Provides optional BagIt packaged AIPs
DSpace Replication Suite Add-on
![Page 22: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/22.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
The Suite of TasksTransmit AIP(s)Verify AIP(s) existFetch AIP(s)Audit against AIP(s)Remove AIP(s)Restore Missing Object(s) from AIP(s)Replace Existing Object(s) from AIP(s)Read Odometer (I/O upload/download stats)Estimate storage size of AIP(s) (rough estimate)All tasks can be configured to store AIPs in an existing
DuraCloud account. Local/Mounted filesystem storage is also supported.
![Page 23: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/23.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
Backup AIP to DuraCloud
Local Temp Folder(Temporary Cache)
ORCommand line Curation Tools
1
Replication Task Suite:• One step process: Generate and Upload AIP• Via UI or CLI
1
![Page 24: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/24.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
Restore AIP from DuraCloud
Local Temp Folder(Temporary Cache)
ORCommand line Curation Tools
Replication Task Suite:• One step process: Retrieve and Restore AIP• Via UI or CLI• Also ‘auditing’ tools
1
1
![Page 25: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/25.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
DSpace Replication Suite Demo
![Page 26: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/26.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Cannot yet take advantage of DuraCloud streaming capabilities (AIPs are zip files)
Cannot yet take advantage of DuraCloud transformation services (AIPs are zip files)
Known Limitations
![Page 27: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/27.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Early December (likely by Mon, Dec 5)• This release is for early adopters to try out
add-on & provide feedback• Download from Replication Task Suite page:
https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite
• Install & Configuration instructions also on above page
“Early Access” Release
![Page 28: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/28.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• MIT : Richard Rodgers & Wendy Bossons – Developed Curation Task Framework– Developed initial Replication Suite tasks (BagIt
versions)• @mire : Mark Diggory
– Gave feedback throughout early development
In Large Thanks to…
![Page 29: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/29.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Replication Task Suite:– https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite
• AIP Backup & Restore:– https://wiki.duraspace.org/display/DSDOC18/AIP+Backup+and+Restore
• Curation Task System:– https://wiki.duraspace.org/display/DSDOC18/Curation+System
For More Information
![Page 30: Integrating DSpace with DuraCloud DuraSpace Webinar: 30 Nov 2011 Tim Donohue DuraSpace](https://reader035.vdocuments.us/reader035/viewer/2022070502/568143f4550346895db07f05/html5/thumbnails/30.jpg)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Package : http://www.flickr.com/photos/halfbisqued/2353845688/• Harddrive & Terminal icons: http://tango.freedesktop.org/Tango_Desktop_Project• Folder icon: http://www.openclipart.org/detail/13740• Database icon: http://www.openclipart.org/detail/68413• Zip Pkg icon: http://veryicon.com/icons/system/capital-icon-suite-mac/zip-10.html• File icons: http://veryicon.com/icons/system/rhor-v2-part-3/• Checkmark & Delete icons: http://veryicon.com/icons/system/on-stage/• Tools Icon: http://veryicon.com/icons/system/azullustre/
Photo/Icon Acknowledgments