digital libraries and music jon dunn slis l631 music librarianship seminar april 7, 2003
TRANSCRIPT
Digital Libraries and Music
Jon Dunn
SLIS L631
Music Librarianship Seminar
April 7, 2003
Outline Digital Libraries
Music content Variations Variations2 Special topics:
Music information retrieval Open Archives Initiative
What is a digital library? DL as collection/information system
“a collection of information that is both digitized and organized” -- Mike Lesk, National Science Foundation
“networked collections of digital text, documents, images, sounds, scientific data, and software”-- President’s Information Technology Advisory Council report
DL as organization: “an organization that provides the resources, including the
specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities”-- Digital Library Federation
Applications of music DLs Education
Electronic reserves Online instructional tools
Research Better access to special collections New capabilities for analysis, searching
Commercial Professionals
E.g. music/film/video production Consumers
Online music catalogs, digital distribution
What is a music digital library? What does it contain? How is this content acquired? How is this content accessed? How can the content be used once located? What is the purpose? Who are the users? How is content protected?
Music DL features Content
Selection, digitization, storage, delivery Metadata (cataloging) Search capabilities
for content and metadata Interfaces
User interfaces, programmatic interfaces Access control
Basic Representations of Music
Audio (e.g., CD, MP3): like speech
Time-stamped Events (e.g., MIDI file): like unformatted text
Music Notation: like text with complex formatting
Digital Audio
Time-stamped Events
Music Notation
Content Formats Audio MIDI Scores
Images Structured file format
(Video)
Digital audio Sampling
Sample rate, sample size, number of channels
Compression Perceptual audio coding
File formats Standards
Digital audio file formats Uncompressed –all basically the same
WAV - Microsoft/IBM AIFF - SGI/Apple AU/SND - NeXT/Sun
Compressed MPEG-1 layers 1-3, MPEG-2 AAC RealAudio, Windows Media, QuickTime
Each supports various compression options
Digital audio file sizes Uncompressed audio
44.1 kHz, 16 bit, stereo (CD quality) 650 MB for one hour 1.4 Megabits/second
Compressed MP3: 58 MB for one hour, 128 Kilobits/second AAC: 29 MB for one hour, 64 Kilobits/second RealAudio, Windows Media Audio, QuickTime Qdesign
Music: down to 20 Kilobits/second or less
Digital Audio
audiosampling
quantizationnoise
Barlow, Multimedia Systems, p. 77.
CD Audio Sample rate:
44.1 kHz (44,100 samples/second) Sample size:
16 bits Number of channels:
2 (stereo) Bitrate
44100 samples/second * 16 bits/sample * 2 channels = 1.4112 megabits/second (plus file format/network overhead)
Masking Effect
Barlow, Multimedia Systems, p. 73.
From Research and Creative Activity, September 1999
Digital audio delivery Delivery options
Download Streaming
e.g. RealAudio, Windows Media, QuickTime Streaming
Encrypted download e.g. LiquidAudio, a2bmusic, Windows Media
Scores Score image
File format: TIFF, JPEG, GIF, PDF, … Resolution Grayscale vs. bitonal (black and white)
Score notation Many proprietary formats No common standard
Attributes of notated musical information
Pitch Duration Tempo Dynamic level Articulation Part (sometimes
implying timbral definition) Selfridge-Field, Beyond MIDI, p. 9.
Difficulties in representing CMN
Grammar of CMN is open-ended Which is more critical: graphical
appearance or semantic meaning? Much left open to interpretation
Style differences, e.g. interpretation of rhythms
Music Notation File Formats www.music-notation.info lists over 50
different music notation formats, most for CMN
MIDI Musical Instrument Digital Interface
Originally a hardware interface spec Communication of real-time events between
musical devices Standard MIDI File (SMF)
Stores time-stamped MIDI event information e.g note on/off, key pressure, aftertouch, pitch bend,
control change, program change. Each event accompanied by parameters
e.g. note on includes pitch, duration, dynamic range Spec maintained by industry group
MIDI Manufacturers’ Assocation
Limitations of MIDI MIDI does not represent many musical
attributes Graphical notation elements
Rests, stem direction, enharmonic distinctions, staff systems, page layout, etc.
Sound elements Timbre, full stylistic expression
Extensions exist but not widely used
Creating Notation Content Transcription
Music notation editor ASCII data entry
Recognition OMR: Optical Music Recognition
Music notation editor example:Finale
OMR: Optical Music Recognition Commercial packages
Musitek MidiScan/SmartScore Version included with Finale
Neuratron PhotoScore Version included with Sibelius
OMR: A long way from OCR
Scanned into Finale: Only 5 easy edits needed.
Here's the original:
Taken from http://www.codamusic.com/finale/scanning.asp
OMR: Optical Music Recognition Research projects
CANTOR University of Waikato, New Zealand
Adaptive OMR Johns Hopkins University, USA Example:
http://mambo.peabody.jhu.edu/omr/demo/ others:
http://www.cs.waikato.ac.nz/~davidb/omr/
Score Images Scanned images still useful
Historical editions, manuscripts Preservation, improving access
Impracticality of large-scale OMR Music presents challenges for scanning
Variations Digital library of music sound recordings
and scores Online since 1996 Accessible in Music Library and other
select locations - copyright Used daily by large student population
Original Concept Burroughs and Fenske, 1990 VARIATIONS name
Theme and Variations Variety of information formats for music
Networked access for the music student or scholar to sound recordings, scores, textual materials, video recordings
Focus on Audio High demand portion of collection Fragile formats Lack of previous work; uniqueness
Focus on Audio Reserves Half of sound recording use from
reserves Problems with existing practices
Cassette tape dubs, analog distribution systems
Concentrated use of a few items at any given time
Variations System Digitization Storage Access
Design and Development Developed by Music Library with assistance from
UITS and Library Information Technology Integrate rather than develop from scratch Partnership with IBM Funding: School of Music, Libraries, UITS, IBM Online in April 1996
Digitization Formats
Analog: LP, cassette tape, reel-to-reel tape Digital: CD, DAT
Capture at CD quality 44.1 kHz, 16 bit, stereo, 700MB for one hour
Compress to MPEG-1 layer 2 (“MP2”) 200 MB for one hour
Create “track file”
Digitization Hardware and Software Windows PCs Sound capture card Microtest Disc-to-Disk CD capture
software Sonic Foundry Sound Forge XP audio
editor
Music Library Digitizing Lab
Storage Tivoli Storage Manager software IBM RS/6000 AIX server IBM Tape Library Dataserver
Contains three tape drives 10 terabyte (10,000 gigabyte) capacity
IBM 3494 Tape Library
IBM 3590E tapes:
20GB each
Access Discovery
How does the user find the desired recording?
Playback How is audio delivered to the user? How does the user navigate within a given
recording?
Collection Currently: 6900 titles, 8000 hours of
audio 5.6 TB uncompressed 1.6 TB compressed
Opera, songs, instrumental music, jazz, rock, world music
Discovery Varies based on purpose of access
Reserves Course reserve lists Faculty-created course home pages (incl.
Oncourse) General use
Links from IUCAT library catalog(856 fields in MARC bib records)
Playback Streaming server
IBM RS/6000, 150 GB disk IBM VideoCharger server software Software to connect VideoCharger with
TSM (locally written) Client
IBM VideoCharger client software Variations Player (locally written)
Navigation via track files
Network Originally ATM
25/100/155 megabits per second Now switched Ethernet
10/100/1000 megabits per second Variations audio stream requires 384
kilobits/second Up to 150 streams
Variations Demonstration
Variations2 Four-year project
Started October 1, 2000 Funding from NSF and NEH through Digital
Libraries Phase 2 (DLI2) program Large interdisciplinary team of investigators Faculty: Music, Information Science, Law,
Computer Science Librarians and technologists: Libraries, University
Information Technology Services Bloomington and Indianapolis campuses
Project goals Establish a digital music library testbed
system supporting multiple formats: audio, video, score images, score notation
Develop multiple interfaces for specific user applications in the music library and the classroom
Conduct research in metadata, usability, copyright, and networking
Partners: “Satellite Sites” United States
University of Illinois at Urbana-Champaign University of Massachusetts at Amherst Northwestern University
United Kingdom Kings College - London Loughborough University University of Oxford
Japan Waseda University
Evaluation…potential for co-development
The Variations2 System Integrated access to music in all formats
Digital audio recordings Score images Score notation Video
Delivery to wide range of users Faculty: teaching, course design, research Students: coursework, independent study Music librarians, other library users
Extensible Multiple user interfaces Staged development
Variations2Version 1.0 Features Infrastructure
Data/metadata repositories, authentication, logging
Search and retrieval interface Based on new data model
Presentation/navigation of audio and scanned scores
Bookmarking
Variations2 Version 1.0 Technical Environment Client and server developed in Java Windows and Mac OS X client platforms,
Unix (AIX/Linux) server Audio streaming: QuickTime for Java, Darwin
Streaming Server Database: IBM DB2, DB2 Text Information
Extender Image compresssion: DjVu from AT&T Labs
and Lizardtech XML/MARC/Z39.50 tools: Saxon, Xerces,
Jafer, James
Variations2 1.0System Architecture: Layer View
General user
Searchtool
Soundplayer
Scoreviewer
Digitaltime liner
General purposelibrary application
Multimedia MusicTheory Teaching
application
Oncourse
Theory student orinstructor
Non-majormusic studentor instructor
Soundplayback
Imageretrieval
Notationretrieval
Search
Metadata Audio VideoScore
imagesScore
notation
Applications
AccessComponents
Repositories
User InterfaceComponents
and others...
and others...
Catalogingtools
Cataloging/adminsistration
application
Cataloger ordigitization technician
Variations2 1.0Communications
Variations2Client
Variations2LibraryServer
DarwinStreaming
Server
ApacheHTTP
Server
Variations2KerberosServer
IUKerberosServer
DB2Database
HTTP
RTSP/RTP
Java RMI
KerberosJDBC
Demonstration
Usability Usability = ease of use + usefulness Established baseline
Usability test of existing Variations system Satisfaction study of Variations users Contextual inquiry
Evaluation of usability of Variations2 Prototype interviews Usability tests of preliminary versions Pilot studies
Data gathering through satisfaction survey and automated usage logging
Future Versions Continuing 6-month development cycle
Version 2: Spring 2003 Version 3: Fall 2003 etc.
Features to be added include: Support for music notation Support for additional image and audio formats Support for new and emerging streaming technologies Support for video
Future Versions Features to be added (continued):
Support for supplemental recording materials (e.g., liner notes, booklets)
Improved browsing interface User interface support for synchronized navigation and
playback Instructional authoring, classroom presentation, and
instructional delivery interfaces Structure diagramming/visualization tools (e.g., Digital
Timeliner) Web browser interface OnCourse integration Access control based on intellectual property requirements Improved cataloging/administrative interface
Variations2 Version 2 Demo
Music Information Retrieval Areas of research:
Indexing and search of music content Audio, MIDI, notation
Feature detection Genre, style, form, instrumentation, …
Music IR:Inherent Difficulties in Music No analogue to ‘words’
No easy units on which to index or do synonym lookup, etc. Problems of representation
Graphical vs. logical aspects of music Polyphony
Multiple voices, chords Cross-voice matching Music is not linear
Polyphony
Polyphony Almost all music IR work to date
focused on pitch matching in monophonic music
Music IR: More problems Query specification
What would a musical query look like? “Query by humming”
Music perception People do not always perceive pitch correctly
What type of matching? Exact pitch or intervals Melodic contour Exact rhythm “Rhythmic contour”
What to index? Entire works Themes
Music IR: Yet more problems Variety of users, probably with very different needs,
including: General public looking for pop music Music students, scholars
However, no formal assessment of user needs No standard query sets, relevance judgements, or
test collections Problems of copyright in building test collections
More information on Music IR ISMIR: International Symposium/Conference on
Music IR 2000: Plymouth, Massachusetts, USA 2001: Bloomington, Indiana, USA 2002: Paris, France October 26-30, 2003: Baltimore, MD, USA
http://www.ismir.net/
OAI: Open Archives Initiative Original problem: searching across e-
print archives Distributed searching hard
e.g. Z39.50 Varying search semantics, capabilities Network, server problems
Solution: metadata harvesting
Metadata harvesting Extract metadata from various sources Build services on local copies of metadata
user
. . .
search for “Mozart”
local copy ofmetadata
metadataharvested offline
metadataharvested offline
metadataharvested offline
metadataharvested offline
all searching, browsing, etc. performed on the metadata hereIndividual repositories can
still support direct userinteraction
Data providers
Service provider
OAI roles Data Providers
Repositories of digital content and metadata Support harvesting of metadata via the OAI
protocol Service Providers
Harvest metadata from data providers using the OAI protocol
Implement user interface to data Usually for searching, but other services also possible
Can be selective
OAI protocol Originally developed in 1999 (“Santa Fe Convention”) Original focus on E-prints Has grown into general metadata harvesting protocol
OAI-PMH: OAI Protocol for Metadata Harvesting Version 1.0: January 2001 Version 1.1: June 2001
Conform to XML Schema 1.0 Version 2.0: June 2002
Transition period through December 2002 Currently 53 registered OAI data providers
OAI protocol Carried over HTTP Requests: HTTP GET or POST Responses encoded in XML
Format defined via XML schema Metadata in simple (unqualified) Dublin
Core (and potentially other formats)
Dublin Core elements Coverage Description Type Relation Source Subject Title
Contributor Creator Publisher Rights Date Format Identifier Language
OAI verbs
Verb Function
Identify description of archive
ListMetadataFormats metadata formats supported by archive
ListSets sets defined by archive
ListIdentifiers OAI unique ids contained in archive
ListRecords listing of N records
GetRecord listing of a single record
OAI resources Web site, mailing lists Repository explorer Data/service provider toolkits
www.openarchives.org
Becoming an OAI data provider Make digital content available on web Translate metadata into Dublin Core
Crosswalks exist for MARC Can also make other formats available, e.g. MARC XML
Choose a unique identifier system Set up OAI data provider server software
See tools list at www.openarchives.org Depending on tool, uses its own database or operates over
existing database
Other technical concerns for scores Areas for standardization/agreement
Within-score navigation User interface, supporting metadata
Image file format MARC-DC metadata crosswalk
Not essential to OAI model, but enables more consistent user experience
Packaging scores for exchange between libraries, e.g. for e-reserves, cooperative preservation
Can METS play a role?
Examples of OAI service providers UIUC Cultural Heritage Repository
http://dlc.grainger.uiuc.edu/ UMich OAIster
http://www.oaister.org/ RLG Cultural Materials
http://www.rlg.org/culturalres/ UCLA/JHU/IU Sheet Music Harvester
http://digital.library.ucla.edu/sheetmusic/