george the magnificent prepare to be amazed!. on choosing a preservation file format for video:...
TRANSCRIPT
George the Magnificent
Prepare to Be Amazed!
On Choosing A Preservation File Format for Video: “TIFFs are too big to store”, or “We Used JPEGs and Nobody Died”
George Blood
Digital Preservation Interest GroupAmerican Library Association, Preservation and Reformatting SectionSan Francisco 2015
“What should we do?”
https://vimeo.com/3750507
1. Use as High a data rate as possible
Less Bad Practice
Proprietary:-ProRes-WMV-Silverlight-DV50
Proprietary vs. Non-Proprietary
Non-Proprietary:-v210 (uncompressed)-DV (IEC 61834)-D10 (SMPTE 356M), -MPEG4 (ISO/IEC 14496)
Proprietary vs. Non-Proprietary
1. Use as High a data rate as possible2. Use Non-proprietary codec
Less Bad Practice
Understand loss
One broken bit in a single frame of “losslessly” compressed video
Courtesy of David Rice
Bit Rot
Understand loss
http://georgeblood.com/Movies/4-Artifacts_Null_Quad_Split_h264.mov
Zero loss is unacceptable!
Except what we throw away!
1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away
Less Bad Practice
1975 - 1 reel, 1 hour of quad video $350 $1,550 in 2016 dollars
2015 - 1 cartridge, 1 hour of Digital Betacam $28 obsolete format
2015 - 100GB quality HDD storage $102 copies $20
2015 – 100GB in the Cloud $1/mo.Google or Glacier, and falls over time
It's not a TDR but neither are your hard drives on a shelf!
Understand Storage Costs
High Performance Storage
• 2007 GBAVF cost for 2TB $100,000, 2 FTE
• 2015 GVAVF cost for 100TB, $100,000, 2 FTE
managing 250TB of storage, 10TB/day, etc.
David Rosenthal – “Cost of storage is rising because we're storing more.”
http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/mow/VC_Rosenthal_et_al_27_B_1330.pdf
Understand Storage Costs
1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs
Less Bad Practice
If we're able to digitize it, though compressed, and it’s saved and accessible, isn’t it preserved?
If it's worth retaining access to it, but we accept there's some loss because the quality is poor to start with - that is, image quality isn't what we're concerned about saving, it's the intellectual content - if we don't call it preservation, what do we call it? Janet Gertz
Recast the question:
1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs5. Understand your actions - preserving the
intellectual essence
Less Bad Practice
• Are we there yet?• How will I know my institution is ready?• Isn’t my preservation space already “special”?• Can I afford it?• How often will it be a problem?
JPEG2000/MXF, AS-07
10-bit uncompressed 100 GBDV25 14DV50 27ProRes (HQ) 33JPEG2000 38
File Sizes
1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs5. Understand your actions - preserving the
intellectual essence
Less Bad Practice
6. Exert yourself
"advocate for archives”-make good choices-Who was ever fired for doing a better job?
- Advocate for uncompressed
- Adopt JPEG2000/MXF
Less Bad Practice
On Choosing A Preservation File Format for Video: “TIFFs are too big to store”, or “We Used JPEGs and Nobody Died”
George Blood
Digital Preservation Interest GroupAmerican Library Association, Preservation and Reformatting SectionSan Francisco 2015