building an audio preservation system at indiana university using standards and best practices
DESCRIPTION
Casey, Michael, Jon Dunn, and Jenn Riley. “Building an Audio Preservation System at Indiana University Using Standards and Best Practices.” April 14, 2008.TRANSCRIPT
![Page 1: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/1.jpg)
Building an Audio Preservation System at
Indiana University Using Standards
and Best Practices
Mike Casey, Archives of Traditional Music
Jon Dunn, Digital Library Program
Jenn Riley, Digital Library Program
April 14, 2008
![Page 2: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/2.jpg)
The Problem
• Numbers• Degradation• Obsolescence
![Page 3: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/3.jpg)
April 9, 2023
Plus many more!
Audio/Video at IUB
• AAAMC• Music Library• University Archives• ATM• HPER• Radio/TV• Center for the Study of History and
Memory• Kinsey Institute• Athletics• Emeriti House• Office of University Marketing• Wells Library
• AISRI• Office of Dean of the Faculties• Lilly Library• Alumni Relations• School of Journalism• School of Music• School of Law• Traditional Arts Indiana• Department of History• Department of Anthropology• Department of
Folklore/Ethnomusicology• Black Film Center/Archive
![Page 4: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/4.jpg)
By the Numbers
• ATM: 110,000 (mostly) audio recordings• Wells Library, Kent Cooper Room: 20,000 videos• Music Library: 137,000 audio recordings
*2,000 lacquer discs
*8,000 DATs
*50,000 open reel tapes• CSHM: 3,200 audio recordings• AAAMC: 5,900 audio recordings
![Page 5: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/5.jpg)
![Page 6: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/6.jpg)
![Page 7: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/7.jpg)
![Page 8: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/8.jpg)
![Page 9: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/9.jpg)
![Page 10: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/10.jpg)
![Page 11: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/11.jpg)
![Page 12: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/12.jpg)
Obsolescence
• Audio formats• Equipment (playback machines, test devices)• Repair parts• Playback expertise• Repair expertise• Tools• Supplies
![Page 13: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/13.jpg)
Preservation in the Analog Domain
• Life expectancy critically important• Predicting when a recording will fail• Quest for the eternal carrier• Target preservation format-mastering-quality
open reel tape• Standards set in the mid-1980’s-ARSC/AAA
![Page 14: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/14.jpg)
The New Paradigm
• Eternal sound carriers never available• Maintaining equipment long-term
unmanageable
Therefore, classical preservation strategy is hopeless
![Page 15: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/15.jpg)
The New Paradigm
• Preserve the content, not the carrier• The eternal file, not the eternal carrier• Use digital mass storage systems
Longevity of carriers in mass storage systems of minor importance
![Page 16: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/16.jpg)
Standards and Best Practices
• Ensure Quality• Provide Philosophical/Ethical Foundation• Encourage Sustainability• Foster Interoperability• Provide a Migration Path
![Page 17: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/17.jpg)
Preserving Digital Information
• Advantage: Digital information may be copied without degradation
• Disadvantage: Digital information requires active management in order to remain accessible
![Page 18: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/18.jpg)
Risks of Digital Information: Bit Loss• Degradation of physical media
– Optical, magnetic• Damage or theft of physical media• Media obsolescence
– Ability to read physical media– Ability to read logical media format
![Page 19: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/19.jpg)
Risks of Digital Information: Semantic Loss• Even if the bits are intact, can a file still be
understood?
• File format obsolescence• Loss of context
– Insufficient metadata
![Page 20: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/20.jpg)
Risks of Digital Information: Integrity• How do we know whether or not information
has been altered, whether intentionally or unintentionally?
![Page 21: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/21.jpg)
Methods of Mitigating Risks
• Migration– Migration of data to new physical media– Migration of data to new file formats
• Replication– Multiple copies of data in multiple locations
• Validation– Retain checksums for files, routinely retrieve
files and compare against checksums
![Page 22: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/22.jpg)
Scaling Digital Preservation
• Migration, replication, and validation require:– Automated processes– Ongoing monitoring, management, and
planning– Ongoing funding for technology refresh
![Page 23: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/23.jpg)
Digital Repositories
• Centrally-managed systems for storage (and delivery) of digital information
• Leverage economies of scale for storage and management costs
• Support preservation integrity functions (migration, replication, validation)
• Much easier to manage than many little pockets of digital information
![Page 24: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/24.jpg)
OAIS: Open Archival Information System• ISO Standard 14721:2003• Origins in space science community• Conceptual framework for an archival system
dedicated to preserving and maintaining access to digital information over the long term
• Basis for much work on digital preservation within the library and archive community
![Page 25: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/25.jpg)
OAIS Reference Model
![Page 26: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/26.jpg)
Preservation Packages in OAIS
• Preservation package– Digital content plus metadata
• SIP: Submission Information Package• AIP: Archival Information Package• DIP: Dissemination Package
![Page 27: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/27.jpg)
From OAIS to Trusted Digital Repositories• 2002 OCLC-RLG task force report:
– Trusted Digital Repositories: Attributes and Responsibilities
• What are the attributes of a trusted repository?– OAIS compliance– Administrative responsibility– Organizational viability– Financial sustainability– System security– Procedural accountability
![Page 28: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/28.jpg)
Trusted Digital Repositories: Auditing and Certiciation• Digital Repository Audit Method Based on Risk
Assessment (DRAMBORA)– http://www.repositoryaudit.edu/
• Trustworthy Repositories Audit & Certification (TRAC): Criteria and Checklist– OCLC/NARA/CRL report– http://www.crl.edu/PDF/trac.pdf
![Page 29: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/29.jpg)
Archives of Traditional Music
• Established 1948 • 110,000 recordings• 1890s to present• Field—30%• World music traditions• Endangered/extinct world languages
![Page 30: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/30.jpg)
Sound DirectionsDigital Preservation and Access for Global Audio Heritage
• Collaboration between Harvard University and Indiana University
• Phase 1 an R&D project funded by NEH• Focus on preservation
![Page 31: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/31.jpg)
Sound DirectionsDigital Preservation and Access for Global Audio Heritage
Project Partners
• Archives of Traditional Music, Indiana University• Archive of World Music, Harvard University• Harvard College Library Audio Preservation Services• Digital Library Program, Indiana University• Office for Information Systems, Harvard University
![Page 32: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/32.jpg)
Sound DirectionsDigital Preservation and Access for Global Audio Heritage
Objectives
• Research best practices in areas without standards or best practices
• Develop best practices to meet existing and emerging standards
• Test existing and emerging standards/best practices with a real world project
![Page 33: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/33.jpg)
Sound DirectionsDigital Preservation and Access for Global Audio Heritage
Results
Publication—
Sound Directions: Best Practices for Audio Preservation Development of audio preservation system Software tools Preservation of field collections
![Page 34: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/34.jpg)
![Page 35: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/35.jpg)
Sound DirectionsDigital Preservation and Access for Global Audio Heritage
Project Future
• “Preservation” Phase funded by NEH• Increase throughput• Simultaneous transfer• Indiana automation• Release ATMC• Develop new access system for field collections
![Page 36: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/36.jpg)
Migration decision
Workflow management
Workflow management / scheduling
Cleaning or physical restoration as needed
System / ProjectPlanning & DevelopmentFundingPersonnel / VendorEquipment
Software ToolsCreation / maintenance of software and scripts
Selection for PreservationAssess research valueEvaluate conditionConsider political, technical, and other issuesEstablish priorities
Digitization Analog playbackA/D conversionCreation of Preservation Master FilesLocal filenames
Digitization Technical metadata Structural metadata ChecksumsQuality controlLocal storage solution
Post-Transfer ProcessingQuality controlGeneration of derivativesMarking areas of interest in filesSignal processing (if appropriate)
Preliminary Work / Pilot Project
Exploratory transfers and metadata collectionQuality controlReassessment of digitization plan
Collection SetupGather and assess documentationEvaluate collection needs / condition Assess cataloging / descriptive metadata issuesDevelop digitization planAssess and calibrate equipment
Ingestion into / Copy to Long-Term Storage
SolutionPreservation packages
Periodic EvaluationData integrity checkingFormat obsolescence analysis
MigrationNew carrierNew format
![Page 37: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/37.jpg)
Common sense definition of a system:
• Set of interacting units or elements• Forms an integrated whole• Performs a function
![Page 38: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/38.jpg)
A few basic principles…
• Each element/part affects the whole• Whole is greater than sum of parts• Inputs and outputs• Equifinality
![Page 39: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/39.jpg)
What should we preserve?Selection for Preservation
• Analysis of research value• Evaluation of preservation condition and risk
![Page 40: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/40.jpg)
Data Collection/Analysis
Research ValueScore
Condition/Risk(FACET)
Score
Combined SelectionScore
Collection Ranking
Curatorial Review
Selection forPreservation
![Page 41: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/41.jpg)
FACET
• Software tool—point-based, collection level• Analyzes data on condition of field formats• Returns a risk assessment score
![Page 42: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/42.jpg)
FACET Package
• Software• Formats document• Procedures manual• FACET worksheets
![Page 43: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/43.jpg)
![Page 44: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/44.jpg)
![Page 45: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/45.jpg)
Where should preservation work be done?
• In-house or outsource?• Issues: studio space, technical expertise,
amount of work, future location of expertise• Critical listening spaces• Development of preservation studio
![Page 46: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/46.jpg)
![Page 47: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/47.jpg)
Who should do preservation transfer work?
• Audio engineer• Importance of analog playback stage• Audio examples
![Page 48: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/48.jpg)
Who and Where Best Practices
• Use audio engineers in the workflow where their skill is required
• Critical listening environment• Use cleanest, most direct signal path to
converter• Instant comparison from playback machine and
post A/D converter• Test/calibration chain
![Page 49: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/49.jpg)
What is the target preservation format?
• Digital file• Broadcast Wave Format (BWF or BWAV)
Preservation involves a long-term responsibility to the digital file
![Page 50: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/50.jpg)
What do we look for in a file format?
• Disclosure• Adoption• Transparency• Self-documentation• External dependencies• Impact of patents• Technical protection mechanisms
http://www.digitalpreservation.gov/formats/sustain/sustain.shtml
![Page 51: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/51.jpg)
Broadcast Wave Format
• Audio file format based on .wav files• EBU 1996 for the exchange of files• Non-proprietary • Recommended by IASA, AES, NARAS, Sound
Directions for preservation• “Chunk” for metadata residing with the file• Time stamp
![Page 52: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/52.jpg)
Broadcast Wave Format
Metadata elements include:
• Description of the sound sequence• Name of the originator• Date/time• Coding history (signal chain components)• Format independent, sample accurate time
stamp• “Catastrophic” metadata
![Page 53: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/53.jpg)
![Page 54: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/54.jpg)
How do we define the files we create?
• What is in them?• How are they created?• What do they represent?
![Page 55: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/55.jpg)
Preservation (Archival) Master Files
Best Practice Documents
• Unmodified• No subjective alterations or improvements• Preserve history, not re-write it• As true to the original source as possible
![Page 56: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/56.jpg)
Preservation (Archival) Master Files
• Complete, unaltered stream from playback machine
• Carrier of raw material from transfer• No editing, signal processing, data reduction,
gain manipulation, announcements (slates)• 24 bit, 96 kHz
![Page 57: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/57.jpg)
Preservation (Archival) Master Files
Best Practices
• Define purpose of every digital file• Written guidelines on characteristics of files• Written guidelines on “technical” and content
edits• Maintain common reference timeline
![Page 58: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/58.jpg)
Data Integrity
Data integrity checking “Checksums” MD5 hash or algorithm
A7F1DAD8A7BF5E88EF44495E19683B18 *atm_01007_cass6936_010101_pres_20080228.wav
![Page 59: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/59.jpg)
Data Integrity
• All files with enduring value• As soon as possible• Critical metadata stored in database and in
preservation package• Verify before trusting
A7F1DAD8A7BF5E88EF44495E19683B18 *atm_01007_cass6936_010101_pres_20080228.wav
![Page 60: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/60.jpg)
How do we make the preserved content understandable and manageable?
• Descriptive Metadata• Administrative—Technical Metadata• Administrative—Digital Provenance• Administrative—Rights Management• Structural Metadata
![Page 61: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/61.jpg)
Audio Technical Metadata Collector (ATMC)
• Enter/edit technical and structural metadata• Audio object and process history metadata• Enter/edit audio object evaluations• Parse files to collect metadata
![Page 62: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/62.jpg)
![Page 63: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/63.jpg)
![Page 64: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/64.jpg)
![Page 65: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/65.jpg)
![Page 66: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/66.jpg)
![Page 67: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/67.jpg)
Quality Control and Assurance
• Quality control vs. quality assurance• QC at ATM: aural, visual, software tools• Collection setup—preliminary transfers• Role of permanent staff• QA at ATM
![Page 68: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/68.jpg)
How do we store the data immediately after capture?
• Local, interim storage• Backup copies at each stage• ATM NAS• Additional redundant copy
![Page 69: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/69.jpg)
Director
Project DevelopmentSelection for Preservation
Archivist
SelectionPreview CollectionsQC Documentation
Librarian
Cataloging Issues
Associate Director
Project ManagementSelection—Format IssuesScheduling Coordination
QCR&D
Audio Engineer
Preservation TransferPreservation Master FilesTechnical MD Collection
ChecksumsBWAV MD
ADL’sSignal Processing
Project Assistant
Content DivisionProduction Masters
QCADL’s
Workflow ManagementCollection SetupIngestion Process
ProgrammerSoftware/ScriptDevelopment
Digital Library Program
Preservation Repository ServicesDeliverables
Access System
![Page 70: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/70.jpg)
The Role of Metadata in Digital Preservation
![Page 71: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/71.jpg)
What is metadata?
• “The stuff we need to know in order to discover and manage data over the long term”
• Here’s a better definition:
“Metadata is structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource.”
NISO. “Understanding Metadata.” 2004. <http://www.niso.org/standards/resources/UnderstandingMetadata.pdf>
![Page 72: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/72.jpg)
Metadata standards
• Standards define mutually agreed-upon:– Definitions of key terms– “Fields” of data to record– Rules for structuring data in these fields
• In this area, generally expressed in XML • Allow us to benefit from community experience• Promote preservation by providing for more
predictable data
![Page 73: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/73.jpg)
Evaluating metadata standards
• Good fit for the type of material I have?• Supports my access/management/preservation
needs?• Are there existing tools to help me create it?• Has it been used before in similar situations?• Who maintains it?• How quickly are the standards in this
environment changing?
![Page 74: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/74.jpg)
Creating metadata
• Generally not done by humans encoding data directly in the storage format
• Instead:– Humans use tools designed for specific
purposes– Derived computationally from the digital
resource itself
![Page 75: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/75.jpg)
Technical metadata
• Tracks properties of a digital file necessary for its rendering and processing
• Can also include data about the circumstances of creation of a digital file
• Often format- or media-specific• Much can be generated automatically from
digital file
![Page 76: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/76.jpg)
Digital provenance metadata
• Tracks the history of a set of related digital files– Can include the methodology by which the
“master” file was created from an analog source (overlap with technical metadata)
– What transformative processes have been applied to the file
– Relationship of “derivative” files to the “master”
![Page 77: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/77.jpg)
Structural metadata
• Documents relationships within and between digital files– Locating the same intellectual content on
multiple representations– Noting points of interest within a single
resource– Grouping and sequencing multiple files that
make up a logical whole
![Page 78: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/78.jpg)
Rights metadata
• Covers legal, moral/ethical, financial rights over resources– Rights holders– Copyright status– Conditions on access– Usage fees/royalty payments
• Can be in human- or machine-readable format
![Page 79: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/79.jpg)
Descriptive metadata
• Like “cataloging”• Allows users and collection managers to find
and identify resources of interest• Factual information such as creator, date
created, running time (overlap with technical metadata)
• Constructed information such as title• Subjective information such as topic, genre
![Page 80: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/80.jpg)
Preservation metadata
• Some overlap with technical and process history metadata
• Catch-all for all the metadata we need to support the preservation process that’s not recorded elsewhere
• Most important feature: tracking events that occur during the preservation process
![Page 81: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/81.jpg)
Preservation Packages
![Page 82: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/82.jpg)
Types of preservation packages
• According to OAIS:– Submission information package (SIP)– Archival information package (AIP)– Dissemination information package (DIP)
• The AIP is what is stored (potentially broken up into pieces) in the IU repository
• Metadata Encoding and Transmission Standard (METS) used to wrap various pieces together
![Page 83: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/83.jpg)
Information representation
• Repository needs two simultaneous views of the content it manages– Physical files– Functions the repository needs to support
![Page 84: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/84.jpg)
Technical metadata
Audio Engineering Society, Core Audio Schema Draft. AES X098-B/SC-03-06.
Also record for analog source object!
![Page 85: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/85.jpg)
Digital provenance metadata
Audio Engineering Society, Audio Processing History Draft. AES X098-C/SC-03-06.
![Page 86: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/86.jpg)
Structural metadata (1)
Audio Engineering Society, Audio Decision List. AES 31-3andMetadata Encoding and Transmisson Standard (METS), <structMap> section
![Page 87: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/87.jpg)
Structural metadata (2)
Audio Engineering Society, Audio Decision List. AES 31-3andMetadata Encoding and Transmisson Standard (METS), <structMap> section
![Page 88: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/88.jpg)
Rights metadata
• For field audio collections, the ATM knows:– Collector– Terms of deposit governing access
• This area still under develop for the IU repository
• No decision yet on metadata format; need more thorough analysis of the functions this metadata needs to support
![Page 89: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/89.jpg)
Descriptive metadata (1)
MARCXML
![Page 90: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/90.jpg)
Descriptive metadata (2)
METS reference to external Word document
![Page 91: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/91.jpg)
Preservation metadata
• Still under investigation for IU repository, for all formats of material
• Will need to implement before any preservation events occur
• Will likely use PReservation Metadata Implementation Strategies (PREMIS) data dictionaries and schema
![Page 92: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/92.jpg)
Need to share
• Copies in multiple repositories can help ensure preservation
• Sound Directions did a test exchange of content between IU and Harvard– Different repository architectures– Different preservation package structures
• ...demonstrated how different levels of preservation are possible
![Page 93: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/93.jpg)
Two Repositories Supported by the Digital Library Program• IUScholarWorks Repository
– “Institutional Repository”• For preserving and providing access to IU’s
research output: articles, papers, etc.
– Based on DSpace software• IU Digital Library Repository
– General-purpose digital content repository– Based on Fedora software
![Page 94: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/94.jpg)
Fedora
• Flexible Extensible Digital Object Repository Architecture
• Open source digital repository software developed by Cornell and the University of Virginia
• Supported by new organization: Fedora Commons
• Basis for IU Digital Library Repository
![Page 95: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/95.jpg)
Moving Content to a Digital Repository – Idealized Workflow
Master audio filesin MDSS
Delivery audiofiles on streaming server
Metadata records ondisk
ATMC/AudioWorkstation Upload
preservationpackage
Temporary Server Disk
Storage
FedoraRepository
Validate and ingest
![Page 96: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/96.jpg)
IU Massive Data Storage System(MDSS)• Hierarchical storage management
– Some storage on hard disks– Much more storage on automated tape
• Managed by UITS Research Technologies• Servers in Bloomington and Indianapolis
connected via I-Light high-speed fiber link• Total capacity: 2+ petabytes• Need to build Fedora-MDSS connection
![Page 97: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/97.jpg)
Repository Status
• Fedora is running in production– Supporting access to image and text
collections– Experiments with loading audio and video
• Need to improve tools for ingest and retrieval to support audio projects
• Not yet a true preservation repository
![Page 98: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/98.jpg)
Toward a Preservation Repository
• Need to add:– File integrity validation– Integration with MDSS – replication of data– Eventually, file format obsolescence
monitoring and migration• Self-audit and/or external certification as
Trusted Digital Repository– DRAMBORA, TRAC
![Page 99: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/99.jpg)
Access Systems
• Variations2– variations2.indiana.edu– Provides access to cataloged commercial
recordings from the Music Library and ATM
• Need access system to provide discovery and delivery of field collections and other types of archival audio collections
![Page 100: Building an Audio Preservation System at Indiana University Using Standards and Best Practices](https://reader036.vdocuments.us/reader036/viewer/2022081602/547bf5a8b37959892b8b4eee/html5/thumbnails/100.jpg)
Questions?
• [email protected]• [email protected]• [email protected]
• www.dlib.indiana.edu/projects/sounddirections/