preservation and handling of media & digital...
TRANSCRIPT
1
Besser-Northwestern Library, 7/14/09
Preservation and Handling of Media & Digital Works
Howard Besser, Director Moving Image Archiving & Preservation Program
NYU’s Tisch School of the Arts http://www.tisch.nyu.edu/preservation
Besser-Northwestern Library, 7/14/09
Preservation and Handling of Media & Digital Works
• The problems posed by Digital Works, and the need for Digital Curation and Stewardship
• Obsolescence & Reformatting, and how Audiovisual collections have been doing this for a long time
• NYU’s MIAP Program & our work with Libraries and other collections
Besser-Northwestern Library, 7/14/09
The Short Life of Digital Info: Digital Longevity Problems-
• Digital Preservation • The Viewing Problem • The Scrambling Problem
• Digital Curation • The Inter-relation Problem • The Custodial Problem • The Translation Problem
Digital Preservation
Besser-Northwestern Library, 7/14/09
Besser-Northwestern Library, 7/14/09
The Viewing Problem
• Digital Info requires a whole infrastructure to view it
• Each piece of that infrastructure is changing at an incredibly rapid rate
• How can we ever hope to deal with all the permutations and combinations
Besser-Northwestern Library, 7/14/09
Viewing Problem
• Requires new file formats and new physical strata at regular intervals
• Needs a serious Managed Environment • Main InterPARES finding--the need for
complete lifecycle management – archivist needs to be incolved when record is
created and throughout active life
2
Besser-Northwestern Library, 7/14/09
The Scrambling Problem�
Dangers from: Compression to ease storage & delivery Encryption to enhance digital commerce
Digital Curation
Besser-Northwestern Library, 7/14/09
Besser-Northwestern Library, 7/14/09
The Inter-relation Problem
-Info is increasingly inter-related to other info
-How do we make our own Info persist when it points to and integrates with Info owned by others?
-What is the boundary of a set of information (or even of a digital object)?
Besser-Northwestern Library, 7/14/09
The Custodial Problem
In the past, much of survival was due to redundancy
How do we decide what to save? Who should save it? Mellon-funded E-Journal Archives
How should they save it?-
Besser-Northwestern Library, 7/14/09
The Custodial Problem:�How to save information?
Methods for later access Refreshing Migration Emulation
Issues of authenticity and evidence
Besser-Northwestern Library, 7/14/09
The Translation Problem
Content translated into new delivery devices changes meaning – -A photo vs. a painting – -If Info is produced originally in digital form in
one encoded format, will it be the same when translated into another format?
– Behaviors
3
Besser-Northwestern Library, 7/14/09
The Translation Problem�Thinking of the Future (1/2)
• Screens will be different resolutions and different aspect ratios
• CRTs won’t exist • A decade or 2 from now, today’s user
interfaces will look like arrow-key navigation looks like today
Besser-Northwestern Library, 7/14/09
The Translation Problem�Thinking of the Future (2/2)
• Today’s streaming media are small windows, slow speeds
• As bandwidth increases, viewers will expect higher quality streams
• Creators may need to consider how they’ll be able to deliver higher-bandwidth streams – Delivery Derivatives vs. Masters encoded w/standards – May also want to re-edit the piece to take advantage of
changes in technology, viewer expectations, society-
Besser-Northwestern Library, 7/14/09
Screen Formats
Besser-Northwestern Library, 7/14/09
Responding to serious Longevity Problems
Previous formats required little ongoing intervention (remote storage facilities, Iron Mtn); digital formats require intense ongoing management
Need for: Preservation Repositories Preservation Metadata
Digital Stewardship definitions
• “acquire, manage, organize, preserve, and provide access to massive amounts of data for use and re-use by a variety of interdisciplinary and heterogeneous communities over time”
• “Preservation + Use”
Besser-Media Art Conservation, 2/7/09 2
Digital Stewardship:�
Harvard Library OIS definition • the management of digital objects over the long term through careful digital
asset management practices. Collection managers and DRS staff must work together to manage stored digital objects throughout all phases of the objects' life cycle. This section describes the major areas of digital stewardship in the life cycle of a digital object:
• Assessment and selection phase: Collection manager performs a curatorial assessment of materials intended for DRS storage.
• Acquisition and creation phase: Collection manager (in consultation with HUL analysts) selects digital formats and defines technical specifications and workflow processes for creation of objects and related metadata.
• Deposit phase: DRS ensures a successful deposit by validating each package of digital objects and related metadata that is submitted to DRS.
• Archive and preservation phase: DRS staff perform periodic checks to ensure the usability of digital objects over time. This includes periodic reports to collection managers about their objects.
Besser-Media Art Conservation, 2/7/09 2
4
Digital Stewardship • Includes much of what cultural institutions have always
done (select, acquire, preserve, provide access, maintain context)
• But also adds some of the activities that audiovisual collections have always done (maintain knowledge of the original equipment and formats, manage process of reformatting when older hardware is difficult to maintain
• Also includes important notions from the digital world about ongoing management of re-formatting information, technical metadata, contextual info, repository issues, …
Besser-Media Art Conservation, 2/7/09 2
Digital Curation
Besser-Media Art Conservation, 2/7/09 2
DCC Digital Curation Life-cycle�http://www.dcc.ac.uk/
Besser-Media Art Conservation, 2/7/09 2
Digital Curation
• Planning & risk management • Open flexible file formats • Format registries • OAIS repositories • Extensive Metadata • Individuals & organizations-
Besser-Media Art Conservation, 2/7/09 2
Organisation to Engage & Collaborate
Besser-Media Art Conservation, 2/7/09
Industry
research collaborators
standards bodies
testbeds�& tools
communities of practice: users
community support & outreach
research
development co-ordination
service definition & delivery
management & admin support
Associates Network
curation �organisations �eg DPC
a centre of expertise in data curation and preservation
OCLC October 2006 2 Besser-Media Art Conservation, 2/7/09
Digital Preservation Players
• Collection staff (need to reach agreement on SIP/DIP and acceptable AIP transformations) – preservation/conservation staff – metadata staff – access staff
• Repository staff • Agreement negotiators
2
5
Besser-Media Art Conservation, 2/7/09
Preservation Repositories:�too difficult for small institutions
• Too complex for small institutions to manage • Will be done through partnering (small museum or dance
company with University) or through consortia (museum association, state-wide organization, …) or through service bureaus (OCLC)
• Archive or museum will direct what is needed, but digital repository will carry out the actual work (as defined in SIP/DIP/AIP agreement)
2 Besser-Media Art Conservation, 2/7/09 2
Storage Media
• Removable media (like CDs) is not a long-term answer
• The long-term answer requires ongoing management, and involves regular migration or emulation. This solution is only viable with storage on spinning disks-
Besser-Media Art Conservation, 2/7/09 2
Storing on CDs becomes a big problem over time
Besser-Media Art Conservation, 2/7/09 2
Consumers replace their CDs with a hard disk (& so should you)
Besser-Media Art Conservation, 2/7/09 2
Plain DVDs are no longer the latest format
Besser-Northwestern Library, 7/14/09
Obsolescence
6
Besser-Northwestern Library, 7/14/09
Difficult Materials become obsolete relatively quickly
• The physical carriers decay or become obsolete
• The technology required to view the carriers changes frequently
• The encoding formats needed to decode the content shift
Besser-Northwestern Library, 7/14/09
Obsolete or deteriorated Physical Carriers
Besser-Northwestern Library, 7/14/09
Obsolete Carriers & Info Techn
Besser-Northwestern Library, 7/14/09
Obsolete Carriers
Besser-Northwestern Library, 7/14/09
Obsolete Carrier viewing Technology?
Besser-Northwestern Library, 7/14/09
Kodak stops making some films
7
Besser-Northwestern Library, 7/14/09
List of old Audio Formats Format Description Years in UseWax Cylinder Records 2- or 4-minute formats, wax
or wax compound1888– 1929
Recordable Disc Records(Direct or Acetate Discs)
7”, 12”, or 16”, recorded at33 or 78 revolutions perminute (rpm). Generallyvinyl on a paper, glass ormetal base
1929– 1960s
Recording Wire Spooled wire, usually in 15-to 30- minute lengths, onedirection only
c. 1945– 1955
Open reel recording tape 1/4”– 2”, 3”– 10 1/2" reels,1 7/8– 30 inches per second(IPS) speeds
c. 1945– Present
Compact Cassette 1/8” tape in hard case, 1 7/8IPS format
1965– Present
Microcassette/Minicassette Very small 2-4 cm cassettetapes
1977– Present
Digital disk, MP3, and otherdigital recorders
Audio recorded directly indigital files to optical disksor internal hard drives
2000– Present
Besser-Northwestern Library, 7/14/09
Old Video Formats (www.vidipax.com)
Besser-Northwestern Library, 7/14/09
Keep Old Video Equipment--NARA
Besser-Northwestern Library, 7/14/09
Keep Old Video Equipment--NARA
Besser-Northwestern Library, 7/14/09
Keep Old Audio Equipment-NARA
Besser-Northwestern Library, 7/14/09
Reformatting and Ethics
8
Besser-Northwestern Library, 7/14/09
Re-formatting is not a new idea
Besser-Northwestern Library, 7/14/09
What is Reformatting?
• A form of copying • Usually copied onto a medium having
different physical characteristics than the original physical strata
• Examples – Document on acidic paper onto non-acidic
paper – Newspaper microfilming
Besser-Northwestern Library, 7/14/09
History of Conservation & Preservation Reformatting
• In ancient times, in the library of Pamphilus at Caesaria, badly damaged papyrus manuscript pages were replaced with parchment (which was stronger)
-Saint Jerome • The Bible was hand-copied for millenia • 1964 - US Newberry Library (Paul Banks) began 1st US
institutional preservation program • 1987 - US NEH begins funding massive microfilming of
brittle paper (mainly newspapers)
Besser-Northwestern Library, 7/14/09
Why do we Reformat?-
Besser-Northwestern Library, 7/14/09
Brittle Newspapers (Australia Battye Library)
Besser-Northwestern Library, 7/14/09
Film Decay (LC Dayton)
9
Besser-Northwestern Library, 7/14/09
Why do we Reformat? • Because we cannot sustain the original object (its physical
characteristics are deteriorating too fast) • Because continued access and handling of the original
object will rapidly decay its physical characteristics (so we create a surrogate for users and store the original in very good conditions, away from users)
• Because viewing the work requires some kind of technology, and we can’t keep that technology working very far into the future-
Besser-Northwestern Library, 7/14/09
Edison
Besser-Northwestern Library, 7/14/09
Metal sound recording Disks�Casa Rui Barbosa
Besser-Northwestern Library, 7/14/09
Paper print (LC Dayton)
Besser-Northwestern Library, 7/14/09
Record Turntables
Besser-Northwestern Library, 7/14/09
Slide Projector
10
Besser-Northwestern Library, 7/14/09
Limitations of Reformatting
• Authenticity issues (more later) • User behaviors (newspaper, book, video
game, …) • Users mistaking the reformatted work for
the original
Besser-Northwestern Library, 7/14/09
Critiques of Reformatting
Mainly User Behaviors
• Can’t view outside the library
• Only sequential access • Viewing and studying
is awkward • …
Besser-Northwestern Library, 7/14/09
But unless we Reformat, we totally lose some kinds of works�
(particularly audiovisual works like film) • 50% of all titles produced before 1950 have vanished (approximate
number as of late 1970s) • This reflects full-length features; survival rates are much lower for
other types (studio newsreels, shorts, docs, independent, …), and these “orphans” are particularly in peril
• Fewer than 20% of features from 1920s survive in complete form; survival rates of 1910s is <10% (& none of these are negatives)
-Film Preservation 1993: A Study of the Current State of American Film Preservation, Vol 1: Report, June 1993, Report of the Librarian of
Congress (http://www.loc.gov/film/study.html)
Besser-Northwestern Library, 7/14/09
And sometimes we have to reformat because of technology changes
• We don’t have video players to play tapes made 25 years ago
• We don’t have 8-inch floppy disk drives, syquest drives, zip drives
• We don’t have Windows 3 operating systems
• But this is something that conservators have always dealt with…
Besser-Northwestern Library, 7/14/09
Authenticity Issues with reformatting-
• Is the work what it purports to be? • Commercial reformatting examples • Media archivists and interpretation • Beyond Film & Video
Besser-Northwestern Library, 7/14/09
Commercial Reformatting Issues:�PR at 1967 re-issue of GWTW
• “Spherical Blow-Up” • “In the Splendor of 70mm. Wide screen and full
stereophonic sound!” • “For the thousands who remember its unparalleled drama,
action and romance! For the new thousands to whom the wonders will be revealed for the first time! Breathtaking spectacle, inspired acting by the greatest cast ever assembled! The screen's most exciting love story! The most-talked about picture ever made!”
• http://www.imdb.com/title/tt0031381/taglines
11
Besser-Northwestern Library, 7/14/09
But most people didn’t know that 70mm widescreen is different shape than 35mm
normal
Besser-Northwestern Library, 7/14/09
Change in Aspect Ratio forces cutting
Besser-Northwestern Library, 7/14/09
Pan and Scan example
• 7 Brides for 7 Brothers, Stanley Donen, MGM, 1954 from http://en.wikipedia.org/wiki/Pan_and_scan
Besser-Northwestern Library, 7/14/09
This meant eliminating part of the frame
Besser-Northwestern Library, 7/14/09
Eliminating even in famous scenes
Besser-Northwestern Library, 7/14/09
But the re-release wasn’t governed by artistic concerns
• Intellectual Property is owned by MGM, not by Fleming or Selznick
• MGM is motivated maximizing profit, not in maintaining artistic integrity
• Bigger is always better
• Not radically different than another blow to artistic integrity/originality in Atlanta 20 years later…
12
Besser-Northwestern Library, 7/14/09
Atlanta was also home of 1980s Colorization movement
Besser-Northwestern Library, 7/14/09
Sometimes even the Director wants to go back and change their original film
Besser-Northwestern Library, 7/14/09
Star Wars 1977 vs 2004
Besser-Northwestern Library, 7/14/09
Someone needs to maintain the integrity of artistic works
• This means preserving original versions – even when commercial interests want to replace
the older version with something new and fancy
– even when the “artist” wants to use more recent technological developments to “improve” their work
Besser-Northwestern Library, 7/14/09
Media Archivists
• Preserve original works • Provide access to older versions • Maintain the integrity of the “original” in any
restoration process • Champion works that do not have commercial
entities pushing for their preservation and distribution
• Try to make sure that works are viewed within their original context-
Besser-Northwestern Library, 7/14/09
Viewing Context (images like in Dayton-Hudson)
13
Besser-Northwestern Library, 7/14/09
Be concerned about © • For preservation you may need to re-format, but with
recent changes in copyright laws, you may not have the right to re-format
• Intellectual property rights are very difficult, particularly considering that most films and videos have extensive underlying rights that you could never get prior permission for (stock footage, historical footage, music composition, music performance, …) [“Eyes on the Prize”]
• And even if you have the right to re-format for preservation, you might not have the right to show what you have preserved
Besser-Northwestern Library, 7/14/09
Possible endless need for reformatting implies
• Possible loss with each generation • Requires managed environment • Can lead to © violations
Besser-Northwestern Library, 7/14/09
Managed Environment
• More than temperature & humidity control • Periodic monitoring of the works • Periodic monitoring of the technical
environment for viewing the works (software, systems, hardware)
• Trusted repositories
Besser-Northwestern Library, 7/14/09
Media Repositories-
Besser-Northwestern Library, 7/14/09
Univ of GA�Walter J Brown Media Archives
Besser-Northwestern Library, 7/14/09
Univ of GA�Walter J Brown Media Archives
14
Besser-Northwestern Library, 7/14/09
Univ of GA�Walter J Brown Media Archives
Besser-Northwestern Library, 7/14/09
Univ of GA�Walter J Brown Media Archives
Besser-Northwestern Library, 7/14/09
Univ of GA�Walter J Brown Media Archives
Besser-Northwestern Library, 7/14/09
Hogan Jazz Archive�Tulane Univ Library
Besser-Northwestern Library, 7/14/09
Cinemateca Brasileira (video storage)
Besser-Northwestern Library, 7/14/09
Hampton Collection (interviews)
15
Besser-Northwestern Library, 7/14/09
Various Formats Intermixed (Hampton)
Besser-Northwestern Library, 7/14/09
MIAP Projects (Paper Tiger)
Besser-Northwestern Library, 7/14/09
MIAP Projects (Paper Tiger)
Besser-Northwestern Library, 7/14/09
Orphan Works
Besser-Northwestern Library, 7/14/09
Home Movies--Emanuel Goldberg
Besser-Northwestern Library, 7/14/09
Amateur Film-- UGA�Walter J Brown Media Archive
• Athens GA, 1947 • Shows contrast btwn White and African
-American neighborhoods • http://www.libs.uga.edu/media/collections/homemovies/housing.html
16
Besser-Northwestern Library, 7/14/09
Wolfson Florida Moving Image Archive
• http://youtube.com/watch?v=vhWa307owQ0
Besser-Northwestern Library, 7/14/09
Palmour Street (1957)
• Gainesville, GA • http://www.youtube.com/watch?v=VpgWZceHX30 • Educational film from Southern Educational
Film Production Service • Prelinger Archives
Besser-Northwestern Library, 7/14/09
Physical Deterioration
Besser-Northwestern Library, 7/14/09
Hampton Collection (atmosphere cntrl)
Besser-Northwestern Library, 7/14/09
Academy-Atmosphere
Besser-Northwestern Library, 7/14/09
IPI Preservation Index�temperature/humidity, Years until noticeable deterioration
% RH
Temperature Cº
2º 7º 13º 18º 24º 29º 35º 20 1250 600 250 125 60 30 16
30 900 400 200 90 45 25 12
40 700 300 150 150
70 35 18 10
50 500 250 100 50 25 14 7
60 350 175 80 40 20 11 6
70 250 125 60 30 16 9 5
80 200 100 50 25 13 7 4
17
Besser-Northwestern Library, 7/14/09
Temperature & Humidity for Tape Storage
• Variance of less than 2ºC and 5% RH per 24 hours
• Ideally 8ºC and 25% RH • Other options
– 20°C (68°F) and 20-30% RH – 15°C (59°F) and 20-40% RH – 10°C (50°F) and 20-50% RH
• Never store below 8ºC
amianet.org 2003, & ISO 18923
Oral/Visual Histories-
Besser-Northwestern Library, 7/14/09
Besser-Northwestern Library, 7/14/09
New distribution:�
iTunes-U UCB Oral Histories
Besser-Northwestern Library, 7/14/09
iTunes-U UCB Campus Events
Besser-Northwestern Library, 7/14/09
iTunes-U Duster Video
Besser-Northwestern Library, 7/14/09
Do we trust iTunes to preserve these?
18
Besser-Northwestern Library, 7/14/09
The Re-Mix generation
Besser-Northwestern Library, 7/14/09
Dueling Videos: Scholar Creates Remix of Another Academic's YouTube Hit�
http://chronicle.com/wiredcampus/article/2707/dueling-videos-scholar-creates-spoof-of-another-academics-youtube-hit
Besser-Northwestern Library, 7/14/09
Chronicle report on Center for Social Media Re-Mix project
• http://chronicle.com/wiredcampus/article/2638/video-report-a-legal-defense-of-videos-riffing-on-popular-culture
Besser-Northwestern Library, 7/14/09
Some Re-Mix Videos • Baby Got Book
http://www.godtube.com/view_video.php?viewkey=97759aa27a0c99bff671 • Bush vs Zombies http://www.youtube.com/watch?v=IoXgRtDysLY • George Bush Don’t Like Black People
http://revver.com/video/71633/george-bush-dont-like-black-people/ • Victory in Iraq http://revver.com/video/79501/victory-in-iraq/
• Fox News Edits a Democrat to Make Him Look Worse http://www.youtube.com/watch?v=oGqPxn7njqM
• xx
Besser-Northwestern Library, 7/14/09
Born-digital images
• Where is the “original”?
Besser-Northwestern Library, 7/14/09
Born-digital works are both easier and harder to preserve than analog works
• +With a born-digital work, we don’t have to worry so much about the “original artifact” (there really isn’t one)
• -We know that digital works face a whole range of obsolescence problems, so they must be reformatted at least once per decade
19
Besser-Northwestern Library, 7/14/09
What can you do now?�For both Film & Video
• Label elements as well as you can • Try to keep things at a low humidity and
temperature • Limit the number of formats as much as
possible • Save important production elements
Besser-Northwestern Library, 7/14/09
Risk Management • We can’t say definitively that we can make every digital work persist • What we CAN say is that the more a digital work conforms to
standards and best practices, the greater the likelihood that we can assure persistance
• Our preservation repositories can even accept deposits of non-conforming works, but the less they conform, the less likely that they’ll be salvageable
• Persistance is most likely for works that share standards, metadata, and best practices
Besser-Northwestern Library, 7/14/09
Reformatting Best Practices (still images) • Think about users (and potential users),
uses, and type of material/collection • Scan at the highest quality that does not
exceed the likely potential users/uses/material
• Do not let today’s delivery limitations influence your scanning file sizes; understand the difference between digital masters and derivative files used for delivery
• Many documents which appear to be bitonal actually are better represented with greyscale scans
• Include color bar and ruler in the scan
• Use objective measurements to determine scanner settings (do NOT attempt to make the image good on your particular monitor or use image processing to color correct)
• Don’t use lossy compression • Store in a common (standardized)
file format • Capture as much metadata as is
reasonably possible (including metadata about the scanning process itself)
Besser-Northwestern Library, 7/14/09
So, with electronic works, the focus should be less on stable temperature�
(Helsinki underground vaults)
Besser-Northwestern Library, 7/14/09
And less on the construction of Vaults (Helsinki underground vaults)
Besser-Northwestern Library, 7/14/09
Paradigms Shifts needed Old New
Physical preservation
atmospheric cntrl ongoing mgmt
What to save? artifact idea + ancillary material & documentation
Cataloging Individual work in hand
FRBR
Later access Artifact & documentation
Restaging, ancillary material & documentation
20
Preservation and Handling of�Media & Digital Works�
Howard Besser�NYU Moving Image Archiving & Preservation Program�
http://www.tisch.nyu.edu/preservation
• http://besser.tsoa.nyu.edu/howard/Talks/ • http://www.ptvdigitalarchive.org/
• http://www.dcc.ac.uk/
• http://www.digitalpreservation.gov/ • http://www.amianet.org/ • http://www.clir.org/pubs/reports/pub54/ • http://sunsite.berkeley.edu/Longevity/ • http://www.imagepermanenceinstitute.org/ • http://www.screensound.gov.au/screensound/screenso.nsf/ Besser-Northwestern Library, 7/14/09