dietrich schüller audiovisual archiving visions, challenges, strategies nijmegen, 29 th november...

Dietrich Schüller

Audiovisual Archiving

Visions, Challenges, StrategiesNijmegen, 29th November 2004

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES

History of linguistic research closely related to sound recording

Audiovisual archives were initiated by research bodies

Vienna 1899

Paris and Berlin 1900

St. Petersburg 1908…

mushroomed with the advent of battery operated audio recorders in the mid-1950s

with truly portable video recorders in the 1980s video recordings became increasingly important

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCESDietrich Schüller

Audio (and upcoming video) documents - sources proper for ethno-linguistic research

total amount audio and video carriers worldwide estimated to be 100 +100 million hours

amount of ethno-linguistic recordings unknown


Visions: Future Archive usage scenarios

– the topic of this workshop


Preconditions for any visionary outlook• physical existence of information• linguistic data gathering did not start yesterday, but

already 1890 - hitherto accumulated research documents to be included

• considerable amounts of relevant recordings outside the world of linguistic research institutions


Challenges• carrier degradation• format and system obsolescence• analogue-to-digital conversion• digital preservation


Strategies• spotting and ingesting language resources outside

linguistic archives/institutions• spotting and safeguarding unattended collections• labour division: archives vs dedicated data bases


Carrier degradation

Mechanical audio carriers

cylinders brittle, mouldy, extremely vulnerable

shellac-78 rpm fairly stable, fragile

instantaneous discs highly endangered

vinyl (LP) discs fairly stable, vulnerable


Instantaneous disc 1990 2001



Magnetic tape - audio and video

most endangered:

acetate tape historical audio

pigment binder break down post 1975 audio and video

MP / ME tape R-Dat, Video/Hi8, DV


Compact Discs - Digital Versatile Discs

replicated CDs + DVDs fairly stable ?

recordable CDs + DVDs highly endangered

DVDs: higher data density - lower data security


Format obsolescence and availability of equipment audioformat obsolete equipment

now sooncylinders x specialist only

coarse groove discs x specialist only

micro groove discs (vinyls) x fading out

quarter inch tape x fading out

micro cassettes x fading out

compact cassette ? ?

R-Dat x fading out

MiniDisc ? ?


Format obsolescence and availability of equipment video professionalformat obsolete equipment

now soon2" quadruplex x specialist only1" x specialist onlyUmatic x fading outM II x specialist onlyBetacam SP ? ?D 1 x fading outD 2 x fading outD 3 x fading outD 5 x ?DigiBeta x ?IMX ? ?


Format obsolescence and availability of equipment video homeformat obsolete equipment

now soonVCR x specialist onlyVideo 2000 x specialist onlyBetamax x specialist onlyVHS ? ?S-VHS ? ?Video8 x ?Video Hi8 x ?Digital 8 ? ?DV ? ?


Traditional paradigm of preserving documents: preserve the original

Around 1990 shift of preservation paradigm:

Long term preservation of

deteriorating original documents

machines of all formats and their spare parts

is hopeless!

Audio (and video) preservation must be based on subsequent digital (= lossless) copying of contents


Viable solution: Digital Mass Storage Systems (DMSSs) – robotic storage systems

self-checking

self-regenerating

self-migrating

DMSS prerequisite for remote access to audiovisual collections

Radio Archives introduced DMSSs since 1992

National and Research Archives following


Storage requirement per hour

preservation access

audio 0.63 - 2GB <0.1GB

video ~ 80GB (CCIR 601) ~ 2GB (DVD)

~22GB (MPEG50,

DVCPRO50)

film (35mm) 3.6 TB (!)

photographs 40-100MB/image ~ 1MB/image


Intermediary solutions- manual imitation of DMSS

produce carriers free of uncorrectable errors and with a minimum of correctable errors – check each carrier and keep the error status record

check error status at regular intervals, ideally of each carrier, at least of representative samples of each batch

copy contents to new carrier before signal becomes irretrievable (refreshment)

copy to new systems before hard- and/or software becomes obsolete (migration)


Problems of intermediary solutions

Original digital target formats R-Dat and CD-R

R-Dat close to obsolescence

CD-R (and DVD-Rs) presently unreliable

Labour intensive

Documents of both formats are already endangered


Present development

Avoid consumer market media (CD-Rs, DVD-Rs) and industrial (=proprietary) audio and video formats (R-Dat, DV, DigiBeta, IMX… )

Go to computer (IT) environment and openly defined file formats: Wave for audio, MXF - linear and MJPEG 2000 lossless compressed - for video

Small, scalable approaches successfully under way


PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCESMichael Risnyovszky

Workstation with firewire/usb2

attached desktop RAID

Small, scalable approaches towards DMSSs successfully under way

To start with:

A single LTO drive or a small autoloader for making backups


Using 2 or more workstations

File server

Network switch

Workstation Workstation LTO autoloader


Further expanded

File server with SCSI-attached drive enclosure(s)

1Gigabit server backboneSeveral workstations

Library with 1 or 2 LTO drives


1Gigabit Server BackboneSeveral workstations

Library with 2 LTO drives, could be connected to the SAN or directly attached to one of the file servers

A small DMSS

2 or more file servers Independent, highly scalable storage unit in a SAN environment

Predominant technical task ahead:

Feeding analogue and digital single carriers into a safe digital environment

time consuming

labour intensive

in demand of well maintained equipment and expert knowledge


Transfer principles• complete, unmodified transfer of originals to digital• no data reduction (“compression”) - unless original

was data reduced

Digital resolutions for analogue originals audio

48 kHz/24 bit minimum (radio)

96/24 standard (heritage and research archives

192/24 upcoming for mechanical originals

“the worse the signal, the higher the resolution”

Keep originals as long as possible


Transfer factors

cylinders, instantaneous discs open ended

78 rpm discs 1: 5 ?

vinyl discs depending on condition

magnetic tape audio 1:3 minimum

including content description 1:6

….or: 1 hour of original tape material takes one day !

magnetic tape video 1: 8–20 (!)


Transfer may take many years, even decades

Consequently

storage/handling of collections must be optimised to retard deterioration

strategies must be developed to ensure future availability of replay equipment - present greatest challenges

audio: low speed quarter inch tape machines

video “professional” player for half inch home formats


Transfer contents is technically demanding

Use modern replay equipment aligned to original (=historical) format parameters

speed

track configurations

equalisation

Generally, do NOT use original recording equipment



Quarter inch audio tape track formats 1


Quarter inch audio tape track formats 2

Butterfly head


S/N vs track width


Tape flux frequency response for different equalizationtime constants (DIN 45 513) and recordings speeds (consumer formats)


Tape equalisations 1


Many (most?) audio research documents need azimuth adjustment

Replay machines must be kept in good condition

spare part supply becomes an ever increasing problem

engineering knowledge and skills fading out

….transfer is more than pressing the replay button!


Predominant financial challenge 1:

cost of equipment

audio:

digital mass storage systems $ 100,000++

“intermediary solutions” $ 20,000+

future “personal” DMSS $ 30,000??

video: “video file archiving” $ 80.000+


Predominant financial challenge 2:

keeping digital documents alive requires ongoing devotion and financial input

professional, automated preservation: 8-12 $/GB/year

manual solutions cheaper, but less secure and only viable up to the low TB range


Predominant strategic aims 1: Spotting and ingesting language resources outside linguistic archives/institutions

• broadcast archives• national/regional/municipial archives• parliamentary archives• oral history archives /collections


Predominant strategic aims 2: Safeguarding the documents of endangered /extinct languages

• 80% of audiovisual materials related to ethno-linguistics are outside proper archival custody

• private collections and small research/cultural units, unaware of necessary preservation measures – or lacking financial and personal resources

• unique sources of the last 50 years are left unattended and will rot away within 10-30 years

• spotting and preserving these collections must be organised on a great scale!


Generally, small collections are lacking

awareness

expertise

analogue replay equipment

digital workstations

financial means for keeping digital data alive

Viable solutions will require cooperative solutions

Greatest problem: mistrust


EU funded Project TAPE objectives:

spotting small collections outside professional archival care, in cooperation with respective NGOs

awareness raising

developing training programmes and materials

organising training events

Similar projects on regional and international scales needed


Predominant strategic aims 3:

Labour division between • archives (in the narrower sense)• content related data bases


Archives in the narrower sense • repositories of raw materials• focus on high standard ingest/transfer and

preservation • metadata coarsely structured and restricted to

otherwise unobtainable secondary information

Sources for………


Data bases (“archives” in the wider sense)• content/region etc related• specifically structured• time and place (school) oriented • structurally between raw material and publication• not responsible for high standard ingest/transfer and

preservation• legal and moral rights clearance viable for dedicated

corpora


Two-tier structure allows• Optimising preservation and access to specialised

corpora by labour division• Cost of transfer from archives to be born by data

bases


Urgent recommendations

Specialised data bases intensify contacts to av-archives (in the narrower sense)

Coordinate activities with• IASA Technical Committee• IASA Cataloguing and Documentation Committee


Summary• access to information is the aim of archiving • however, access is based on the physical existence

of information• technical and logistic challenges to preserve

information are not trivial – specifically not at the transition from analogue to digital


IASA Technical Committee

Standards, Recommended Practices and Strategies

IASA-TC 03 The safeguarding of the Audio Heritage: Ethics, Principles and Preservation StrategyVersion 2, September 2001http://www.iasa-web.org/iasa0013.htm

IASA-TC 04 Guidelines on the Production and Preservation of Digital Audio Objects, edited by

Kevin BradleyPublication on IASA website forthcomingPrint copies available from Ilse Assmann [email protected]

PHONOGRAMMARCHIV - ÖSTERREICHISCHE AKADEMIE der WISSENSCHAFTEN

Dietrich Schüller

http://www.iasa-web.org/iasa0013.htm




mailto:[email protected]







Thank you!

[email protected]

http://www.pha.oeaw.ac.at/

http://www.iasa-web.org/


dietrich schüller audiovisual archiving visions, challenges, strategies nijmegen, 29 th november...

Documents

video mp

video carriers

s video recordings

fragileinstantaneous

portable video recorders

research bodiesvienna

tape rdat

accumulated research