the new drs: plan for metadata migration
DESCRIPTION
Harvard Library and Library Technology Services review the new standard in digital preservation at the LibraryTRANSCRIPT
![Page 1: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/1.jpg)
The New DRS: Plan for Metadata Migration
Harvard Library & Library Technology ServicesFebruary 26, 2014
![Page 2: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/2.jpg)
AgendaWelcome and introduction …... Franziska Frey, Tracey Robinson
Migration challenges .…………... Randy Stern
Creating the plan ………………..... Kate Bowers
Walkthrough of plan …………….. Andrea Goethals
Imaging Services:
minimizing disruption……….… Bill Comstock
Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals
Q & A ……………………………….…... All
![Page 3: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/3.jpg)
WELCOME & INTRODUCTIONFranziska Frey, Tracey Robinson
![Page 4: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/4.jpg)
The DRS Advisory Group…
…provides oversight and guidance during the rollout phase of the DRS2 project and ensures that the user community of active DRS depositors and content owners contributes to decisions about the rollout.
Amy BensonKate BowersBill ComstockFranziska Frey (chair)
Andrea GoethalsWendy GogelTracey Robinson Randy Stern
![Page 5: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/5.jpg)
Why a New DRS?
• Upgrade to best-in-breed technologies• Adopt digital preservation best practices
and standards• Preserve metadata better• Improve collection management• Support preservation planning & activities• Improve access to content & metadata• Support more formats & genres
![Page 6: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/6.jpg)
Preservation Capability Before and After the DRS2 Project
Level One Level Two Level Three Level Four
Storage & Geographic Location
File Fixity and Data Integrity
Information Security
Metadata
File Formats
= already compliant = will be compliant after the DRS2 project
Based on the NDSA Levels of Digital Preservation
![Page 7: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/7.jpg)
Evolution of the DRS
2000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20122001
Current DRS in
production
New DRS in productionDRS enhancements
New DRS infrastructuredevelopment
2013 2014 2015
New DRS metadata migration
& user adoption
![Page 8: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/8.jpg)
New DRS - Completed
2009 2010 2011 2012
convened DRS
Advisory Group
software in production
2013 2014 2015
users trained,phase 1
hardware in production
migrated content to new hardware
InfrastructureDevelopment
Metadata Migration
& User Adoption
Fedora assessment
DuraCloud pilot test
early release beta 1 beta 2
beta 3
first object deposited to the new
DRS
![Page 9: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/9.jpg)
New DRS - Upcoming
2009 2010 2011 2012 2013 2014 2015
InfrastructureDevelopment
Metadata Migration
& User Adoption
metadata migration tools
created
metadata migrated
users moved
![Page 10: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/10.jpg)
MIGRATION CHALLENGESRandy Stern
![Page 11: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/11.jpg)
Why “Metadata” Migration?
Why not“content” migration?
![Page 12: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/12.jpg)
Pre-migration
DRS Content
Current DRS
Database
![Page 13: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/13.jpg)
Post-migration
DRS Content
Current DRS
Database New DRS Database
New DRS Index
New DRS Object Descriptors
![Page 14: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/14.jpg)
New DRS Data Model
• Not a simple metadata conversion• A new DRS object is a logical intellectual
entity that unifies multiple DRS files– Still image objects - archival and production
masters, and deliverables including thumbnails – Audio objects - archival and production
masters and deliverables– PDS objects - page image and text files
![Page 15: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/15.jpg)
Object Descriptors
• METS files generated for each object– Standards-based internal schemas (PREMIS,
MODS, MIX, etc.)
• Metadata gathered from multiple sources– Current DRS database– Every content file– HOLLIS records
![Page 16: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/16.jpg)
Technical Challenges
• Many formats– Images, audio, text, digitized books, web
sites, documents, biomedical image stacks, opaque files
• Unique migration rules per format – technical metadata, roles, relationships
• Large (>5000 file) PDS documents• 45+ million DRS files
![Page 17: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/17.jpg)
Technical Challenges
• At 1 sec/file, 45 million files would take 520 days!
• We are designing the migration software tools for parallel processing
• We are configuring multiple servers to run the migration
![Page 18: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/18.jpg)
![Page 19: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/19.jpg)
CREATING THE PLANKate Bowers
![Page 20: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/20.jpg)
Formulating a Migration Strategy
• Analysis of:– DRS content
• Technical (relationships, etc. for building objects)• Volume and type by repository
– metadata for mapping– user activity in the DRS– survey of highest volume, active users– training and testing registration lists
![Page 21: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/21.jpg)
Migration Strategy Factors
• Combines needs of users with technical requirements
• User sequencing will be based on:– Current deposit & administrative activity – Level of preparation (training and participation
in beta testing)
![Page 22: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/22.jpg)
WALKTHROUGH OF PLANAndrea Goethals
![Page 23: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/23.jpg)
Migrating Content in 5 Stages
Migrate 1st : Tier 1 contentMigrate 2nd: Tier 2 contentMigrate 3rd: Tier 3 contentMigrate 4th: Tier 4 contentMigrate 5th: Tier 5 content
![Page 24: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/24.jpg)
Migrating Content in 5 Stages
Migrate 1st : Tier 1 contentMigrate 2nd: Tier 2 contentMigrate 3rd: Tier 3 contentMigrate 4th: Tier 4 contentMigrate 5th: Tier 5 content
simpler objects
more complex objects
![Page 25: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/25.jpg)
Migrating Content in 5 Stages
Migrate 1st : Tier 1 contentMigrate 2nd: Tier 2 contentMigrate 3rd: Tier 3 contentMigrate 4th: Tier 4 contentMigrate 5th: Tier 5 content
dependenciesbetween tiers
dependencieswithin tiers
![Page 26: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/26.jpg)
Migrating Content in 5 StagesTier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
![Page 27: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/27.jpg)
Migrating Content in 5 StagesTier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
![Page 28: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/28.jpg)
Migrating Content in 5 StagesTier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one timeTier 2: Migrate one DRS owner code at a time
![Page 29: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/29.jpg)
Tier 2: Sequence by DRS Owner Code
• Migrate just your unit’s PDS document and still image content
• Minimize the amount of time the content you manage the most is in 2 different systems
![Page 30: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/30.jpg)
DRS Owner Codes to be Migrated
DIV.LIBR
FHCL.HOUGH
FHCL.MUSI
GSD.LIBR
RAD.ARCH
RAD.SCHL
FHCL.JUD
FHCL.FAL
FHCL.MAPS
FMUS.MCZ
HLS.LIBR
HUL.ARCH
HUAM.MUSE
HBS.BAKR
VIT.BERE
HUL.PRES
FMUS.GRAY
HPPM.PIRC DOAK.RESLIB
DOAK.MUS
FCOR.REISCH
FMUS.ARN
HLNC.LIBRARB.AAHOD
DOAK.LIBR
FCOR.FORST
FCOR.WOLBACH
FMUS.FARL
FMUS.HUH
FMUS.ORC
FMUS.PEAB
HMS.COUNT
HPRE.WARD
HUAM.SARDISHUL.GGL
HUL.OIS
FCOR.CARP
FCOR.HCO FCOR.URI
FHCL.CAB
FHCL.COLL
FHCL.DAVIS
FHCL.ENV
FHCL.FUNG
FHCL.GOV
FHCL.LITTFHCL.MED
FHCL.SLV
FHCL.TOZ FHCL.YENCH
FMUS.SEM
FMUS.WARE
GSE.GUTMN
KSG.LIBR
![Page 31: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/31.jpg)
Timing
• Current estimates: – Building & testing migration tools: Now– Begin Tier 1 content: Spring 2014– Begin Tier 2 content: Summer 2014
• Units will be contacted about their Tier 2 migration schedule
![Page 32: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/32.jpg)
After Your Tier 2 Migration
• You, and anyone depositing on your behalf, will begin depositing only to the new DRS
• All of your management tasks will be done only in the new DRS
![Page 33: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/33.jpg)
IMAGING SERVICES: MINIMIZING DISRUPTION
Bill Comstock
![Page 34: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/34.jpg)
Minimizing Disruption
• Testing by Imaging Services• Uninterrupted services• Migration sequencing• Participating as a “pioneer”
![Page 35: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/35.jpg)
Testing by Imaging Services
Alpha and beta testing:• Depositing processes• DRS content maintenance tools
– Searching and assembling content for download
– Editing PDS objects
![Page 36: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/36.jpg)
Uninterrupted Services
Providing services before and after your migration • Content needs to be deposited• Content needs to be searched • Content needs to be assembled
• may need to be edited• may need to be downloaded
![Page 37: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/37.jpg)
Migration Sequencing
• We will synchronize deposits with your migration– start depositing for you in the new DRS
after your Tier 2 content is migrated
![Page 38: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/38.jpg)
Imaging Services as Pioneers
As pioneers, we:• Learn to use the new tools• Refine the new depositing workflows• Identify bugs• Suggest improvements• Create a group of local experts that can support those
that follow
We’ll wear the scars so that you can stay pretty!
![Page 39: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/39.jpg)
WRAP-UP AND NEXT STEPSKate Bowers and Andrea Goethals
![Page 40: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/40.jpg)
Nine Pioneers
• Limited number of first depositors to new DRS• Factors
– New DRS-ready content from new systems • EAS (Electronic Archiving Service), ACORN
(Weissman Preservation Center conservation treatments), DASH (for ETD)
– Prepared and trained staff– No content to migrate
• HUA example: opaque objects
![Page 41: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/41.jpg)
First Deposit in the New DRS
大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language
![Page 43: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/43.jpg)
http://hul.harvard.edu/ois/systems/drs/drs2.html
![Page 44: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/44.jpg)
Coming Attractions
• Open meetings– Technical aspects brown bag (March)– Digital preservation & DRS intro (Summer)
• Training and instruction– Refresher training– New training– Onsite assistance
![Page 45: The New DRS: Plan for Metadata Migration](https://reader038.vdocuments.us/reader038/viewer/2022110306/555069efb4c90524138b467b/html5/thumbnails/45.jpg)
Q & AThanks!