digital libraries and music jon dunn slis l631 music librarianship seminar april 7, 2003

Digital Libraries and Music

Jon Dunn

SLIS L631

Music Librarianship Seminar

April 7, 2003

Outline Digital Libraries

Music content Variations Variations2 Special topics:

Music information retrieval Open Archives Initiative

What is a digital library? DL as collection/information system

“a collection of information that is both digitized and organized” -- Mike Lesk, National Science Foundation

“networked collections of digital text, documents, images, sounds, scientific data, and software”-- President’s Information Technology Advisory Council report

DL as organization: “an organization that provides the resources, including the

specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities”-- Digital Library Federation

Applications of music DLs Education

Electronic reserves Online instructional tools

Research Better access to special collections New capabilities for analysis, searching

Commercial Professionals

E.g. music/film/video production Consumers

Online music catalogs, digital distribution

What is a music digital library? What does it contain? How is this content acquired? How is this content accessed? How can the content be used once located? What is the purpose? Who are the users? How is content protected?

Music DL features Content

Selection, digitization, storage, delivery Metadata (cataloging) Search capabilities

for content and metadata Interfaces

User interfaces, programmatic interfaces Access control

Basic Representations of Music

Audio (e.g., CD, MP3): like speech

Time-stamped Events (e.g., MIDI file): like unformatted text

Music Notation: like text with complex formatting

Digital Audio

Time-stamped Events

Music Notation

Content Formats Audio MIDI Scores

Images Structured file format

(Video)

Digital audio Sampling

Sample rate, sample size, number of channels

Compression Perceptual audio coding

File formats Standards

Digital audio file formats Uncompressed –all basically the same

WAV - Microsoft/IBM AIFF - SGI/Apple AU/SND - NeXT/Sun

Compressed MPEG-1 layers 1-3, MPEG-2 AAC RealAudio, Windows Media, QuickTime

Each supports various compression options

Digital audio file sizes Uncompressed audio

44.1 kHz, 16 bit, stereo (CD quality) 650 MB for one hour 1.4 Megabits/second

Compressed MP3: 58 MB for one hour, 128 Kilobits/second AAC: 29 MB for one hour, 64 Kilobits/second RealAudio, Windows Media Audio, QuickTime Qdesign

Music: down to 20 Kilobits/second or less

Digital Audio

audiosampling

quantizationnoise

Barlow, Multimedia Systems, p. 77.

CD Audio Sample rate:

44.1 kHz (44,100 samples/second) Sample size:

16 bits Number of channels:

2 (stereo) Bitrate

44100 samples/second * 16 bits/sample * 2 channels = 1.4112 megabits/second (plus file format/network overhead)

Masking Effect

Barlow, Multimedia Systems, p. 73.

From Research and Creative Activity, September 1999

Digital audio delivery Delivery options

Download Streaming

e.g. RealAudio, Windows Media, QuickTime Streaming

Encrypted download e.g. LiquidAudio, a2bmusic, Windows Media

Scores Score image

File format: TIFF, JPEG, GIF, PDF, … Resolution Grayscale vs. bitonal (black and white)

Score notation Many proprietary formats No common standard

Attributes of notated musical information

Pitch Duration Tempo Dynamic level Articulation Part (sometimes

implying timbral definition) Selfridge-Field, Beyond MIDI, p. 9.

Difficulties in representing CMN

Grammar of CMN is open-ended Which is more critical: graphical

appearance or semantic meaning? Much left open to interpretation

Style differences, e.g. interpretation of rhythms

Music Notation File Formats www.music-notation.info lists over 50

different music notation formats, most for CMN

MIDI Musical Instrument Digital Interface

Originally a hardware interface spec Communication of real-time events between

musical devices Standard MIDI File (SMF)

Stores time-stamped MIDI event information e.g note on/off, key pressure, aftertouch, pitch bend,

control change, program change. Each event accompanied by parameters

e.g. note on includes pitch, duration, dynamic range Spec maintained by industry group

MIDI Manufacturers’ Assocation

Limitations of MIDI MIDI does not represent many musical

attributes Graphical notation elements

Rests, stem direction, enharmonic distinctions, staff systems, page layout, etc.

Sound elements Timbre, full stylistic expression

Extensions exist but not widely used

Creating Notation Content Transcription

Music notation editor ASCII data entry

Recognition OMR: Optical Music Recognition

Music notation editor example:Finale

OMR: Optical Music Recognition Commercial packages

Musitek MidiScan/SmartScore Version included with Finale

Neuratron PhotoScore Version included with Sibelius

OMR: A long way from OCR

Scanned into Finale: Only 5 easy edits needed.

Here's the original:

Taken from http://www.codamusic.com/finale/scanning.asp

OMR: Optical Music Recognition Research projects

CANTOR University of Waikato, New Zealand

Adaptive OMR Johns Hopkins University, USA Example:

http://mambo.peabody.jhu.edu/omr/demo/ others:

http://www.cs.waikato.ac.nz/~davidb/omr/

Score Images Scanned images still useful

Historical editions, manuscripts Preservation, improving access

Impracticality of large-scale OMR Music presents challenges for scanning

Variations Digital library of music sound recordings

and scores Online since 1996 Accessible in Music Library and other

select locations - copyright Used daily by large student population

Original Concept Burroughs and Fenske, 1990 VARIATIONS name

Theme and Variations Variety of information formats for music

Networked access for the music student or scholar to sound recordings, scores, textual materials, video recordings

Focus on Audio High demand portion of collection Fragile formats Lack of previous work; uniqueness

Focus on Audio Reserves Half of sound recording use from

reserves Problems with existing practices

Cassette tape dubs, analog distribution systems

Concentrated use of a few items at any given time

Variations System Digitization Storage Access

Design and Development Developed by Music Library with assistance from

UITS and Library Information Technology Integrate rather than develop from scratch Partnership with IBM Funding: School of Music, Libraries, UITS, IBM Online in April 1996

Digitization Formats

Analog: LP, cassette tape, reel-to-reel tape Digital: CD, DAT

Capture at CD quality 44.1 kHz, 16 bit, stereo, 700MB for one hour

Compress to MPEG-1 layer 2 (“MP2”) 200 MB for one hour

Create “track file”

Digitization Hardware and Software Windows PCs Sound capture card Microtest Disc-to-Disk CD capture

software Sonic Foundry Sound Forge XP audio

editor

Music Library Digitizing Lab

Storage Tivoli Storage Manager software IBM RS/6000 AIX server IBM Tape Library Dataserver

Contains three tape drives 10 terabyte (10,000 gigabyte) capacity

IBM 3494 Tape Library

IBM 3590E tapes:

20GB each

Access Discovery

How does the user find the desired recording?

Playback How is audio delivered to the user? How does the user navigate within a given

recording?

Collection Currently: 6900 titles, 8000 hours of

audio 5.6 TB uncompressed 1.6 TB compressed

Opera, songs, instrumental music, jazz, rock, world music

Discovery Varies based on purpose of access

Reserves Course reserve lists Faculty-created course home pages (incl.

Oncourse) General use

Links from IUCAT library catalog(856 fields in MARC bib records)

Playback Streaming server

IBM RS/6000, 150 GB disk IBM VideoCharger server software Software to connect VideoCharger with

TSM (locally written) Client

IBM VideoCharger client software Variations Player (locally written)

Navigation via track files

Network Originally ATM

25/100/155 megabits per second Now switched Ethernet

10/100/1000 megabits per second Variations audio stream requires 384

kilobits/second Up to 150 streams

Variations Demonstration

Variations2 Four-year project

Started October 1, 2000 Funding from NSF and NEH through Digital

Libraries Phase 2 (DLI2) program Large interdisciplinary team of investigators Faculty: Music, Information Science, Law,

Computer Science Librarians and technologists: Libraries, University

Information Technology Services Bloomington and Indianapolis campuses

Project goals Establish a digital music library testbed

system supporting multiple formats: audio, video, score images, score notation

Develop multiple interfaces for specific user applications in the music library and the classroom

Conduct research in metadata, usability, copyright, and networking

Partners: “Satellite Sites” United States

University of Illinois at Urbana-Champaign University of Massachusetts at Amherst Northwestern University

United Kingdom Kings College - London Loughborough University University of Oxford

Japan Waseda University

Evaluation…potential for co-development

The Variations2 System Integrated access to music in all formats

Digital audio recordings Score images Score notation Video

Delivery to wide range of users Faculty: teaching, course design, research Students: coursework, independent study Music librarians, other library users

Extensible Multiple user interfaces Staged development

Variations2Version 1.0 Features Infrastructure

Data/metadata repositories, authentication, logging

Search and retrieval interface Based on new data model

Presentation/navigation of audio and scanned scores

Bookmarking

Variations2 Version 1.0 Technical Environment Client and server developed in Java Windows and Mac OS X client platforms,

Unix (AIX/Linux) server Audio streaming: QuickTime for Java, Darwin

Streaming Server Database: IBM DB2, DB2 Text Information

Extender Image compresssion: DjVu from AT&T Labs

and Lizardtech XML/MARC/Z39.50 tools: Saxon, Xerces,

Jafer, James

Variations2 1.0System Architecture: Layer View

General user

Searchtool

Soundplayer

Scoreviewer

Digitaltime liner

General purposelibrary application

Multimedia MusicTheory Teaching

application

Oncourse

Theory student orinstructor

Non-majormusic studentor instructor

Soundplayback

Imageretrieval

Notationretrieval

Search

Metadata Audio VideoScore

imagesScore

notation

Applications

AccessComponents

Repositories

User InterfaceComponents

and others...

and others...

Catalogingtools

Cataloging/adminsistration

application

Cataloger ordigitization technician

Variations2 1.0Communications

Variations2Client

Variations2LibraryServer

DarwinStreaming

Server

ApacheHTTP

Server

Variations2KerberosServer

IUKerberosServer

DB2Database

HTTP

RTSP/RTP

Java RMI

KerberosJDBC

Demonstration

Usability Usability = ease of use + usefulness Established baseline

Usability test of existing Variations system Satisfaction study of Variations users Contextual inquiry

Evaluation of usability of Variations2 Prototype interviews Usability tests of preliminary versions Pilot studies

Data gathering through satisfaction survey and automated usage logging

Future Versions Continuing 6-month development cycle

Version 2: Spring 2003 Version 3: Fall 2003 etc.

Features to be added include: Support for music notation Support for additional image and audio formats Support for new and emerging streaming technologies Support for video

Future Versions Features to be added (continued):

Support for supplemental recording materials (e.g., liner notes, booklets)

Improved browsing interface User interface support for synchronized navigation and

playback Instructional authoring, classroom presentation, and

instructional delivery interfaces Structure diagramming/visualization tools (e.g., Digital

Timeliner) Web browser interface OnCourse integration Access control based on intellectual property requirements Improved cataloging/administrative interface

Variations2 Version 2 Demo

Music Information Retrieval Areas of research:

Indexing and search of music content Audio, MIDI, notation

Feature detection Genre, style, form, instrumentation, …

Music IR:Inherent Difficulties in Music No analogue to ‘words’

No easy units on which to index or do synonym lookup, etc. Problems of representation

Graphical vs. logical aspects of music Polyphony

Multiple voices, chords Cross-voice matching Music is not linear

Polyphony

Polyphony Almost all music IR work to date

focused on pitch matching in monophonic music

Music IR: More problems Query specification

What would a musical query look like? “Query by humming”

Music perception People do not always perceive pitch correctly

What type of matching? Exact pitch or intervals Melodic contour Exact rhythm “Rhythmic contour”

What to index? Entire works Themes

Music IR: Yet more problems Variety of users, probably with very different needs,

including: General public looking for pop music Music students, scholars

However, no formal assessment of user needs No standard query sets, relevance judgements, or

test collections Problems of copyright in building test collections

More information on Music IR ISMIR: International Symposium/Conference on

Music IR 2000: Plymouth, Massachusetts, USA 2001: Bloomington, Indiana, USA 2002: Paris, France October 26-30, 2003: Baltimore, MD, USA

http://www.ismir.net/

OAI: Open Archives Initiative Original problem: searching across e-

print archives Distributed searching hard

e.g. Z39.50 Varying search semantics, capabilities Network, server problems

Solution: metadata harvesting

Metadata harvesting Extract metadata from various sources Build services on local copies of metadata

user

. . .

search for “Mozart”

local copy ofmetadata

metadataharvested offline

metadataharvested offline

metadataharvested offline

metadataharvested offline

all searching, browsing, etc. performed on the metadata hereIndividual repositories can

still support direct userinteraction

Data providers

Service provider

OAI roles Data Providers

Repositories of digital content and metadata Support harvesting of metadata via the OAI

protocol Service Providers

Harvest metadata from data providers using the OAI protocol

Implement user interface to data Usually for searching, but other services also possible

Can be selective

OAI protocol Originally developed in 1999 (“Santa Fe Convention”) Original focus on E-prints Has grown into general metadata harvesting protocol

OAI-PMH: OAI Protocol for Metadata Harvesting Version 1.0: January 2001 Version 1.1: June 2001

Conform to XML Schema 1.0 Version 2.0: June 2002

Transition period through December 2002 Currently 53 registered OAI data providers

OAI protocol Carried over HTTP Requests: HTTP GET or POST Responses encoded in XML

Format defined via XML schema Metadata in simple (unqualified) Dublin

Core (and potentially other formats)

Dublin Core elements Coverage Description Type Relation Source Subject Title

Contributor Creator Publisher Rights Date Format Identifier Language

OAI verbs

Verb Function

Identify description of archive

ListMetadataFormats metadata formats supported by archive

ListSets sets defined by archive

ListIdentifiers OAI unique ids contained in archive

ListRecords listing of N records

GetRecord listing of a single record

OAI resources Web site, mailing lists Repository explorer Data/service provider toolkits

www.openarchives.org

Becoming an OAI data provider Make digital content available on web Translate metadata into Dublin Core

Crosswalks exist for MARC Can also make other formats available, e.g. MARC XML

Choose a unique identifier system Set up OAI data provider server software

See tools list at www.openarchives.org Depending on tool, uses its own database or operates over

existing database

Other technical concerns for scores Areas for standardization/agreement

Within-score navigation User interface, supporting metadata

Image file format MARC-DC metadata crosswalk

Not essential to OAI model, but enables more consistent user experience

Packaging scores for exchange between libraries, e.g. for e-reserves, cooperative preservation

Can METS play a role?

Examples of OAI service providers UIUC Cultural Heritage Repository

http://dlc.grainger.uiuc.edu/ UMich OAIster

http://www.oaister.org/ RLG Cultural Materials

http://www.rlg.org/culturalres/ UCLA/JHU/IU Sheet Music Harvester

http://digital.library.ucla.edu/sheetmusic/