george the magnificent prepare to be amazed!. on choosing a preservation file format for video:...

George the Magnificent

Prepare to Be Amazed!

On Choosing A Preservation File Format for Video: “TIFFs are too big to store”, or “We Used JPEGs and Nobody Died”

George Blood

Digital Preservation Interest GroupAmerican Library Association, Preservation and Reformatting SectionSan Francisco 2015

“What should we do?”

https://vimeo.com/3750507

1. Use as High a data rate as possible

Less Bad Practice

Proprietary:-ProRes-WMV-Silverlight-DV50

Proprietary vs. Non-Proprietary

Non-Proprietary:-v210 (uncompressed)-DV (IEC 61834)-D10 (SMPTE 356M), -MPEG4 (ISO/IEC 14496)

Proprietary vs. Non-Proprietary

1. Use as High a data rate as possible2. Use Non-proprietary codec

Less Bad Practice

Understand loss

One broken bit in a single frame of “losslessly” compressed video

Courtesy of David Rice

Bit Rot

Understand loss

http://georgeblood.com/Movies/4-Artifacts_Null_Quad_Split_h264.mov

Zero loss is unacceptable!

Except what we throw away!

1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away

Less Bad Practice

1975 - 1 reel, 1 hour of quad video $350 $1,550 in 2016 dollars

2015 - 1 cartridge, 1 hour of Digital Betacam $28 obsolete format

2015 - 100GB quality HDD storage $102 copies $20

2015 – 100GB in the Cloud $1/mo.Google or Glacier, and falls over time

It's not a TDR but neither are your hard drives on a shelf!

Understand Storage Costs

High Performance Storage

• 2007 GBAVF cost for 2TB $100,000, 2 FTE

• 2015 GVAVF cost for 100TB, $100,000, 2 FTE

managing 250TB of storage, 10TB/day, etc.

David Rosenthal – “Cost of storage is rising because we're storing more.”

http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/mow/VC_Rosenthal_et_al_27_B_1330.pdf

Understand Storage Costs

1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs

Less Bad Practice

If we're able to digitize it, though compressed, and it’s saved and accessible, isn’t it preserved?

If it's worth retaining access to it, but we accept there's some loss because the quality is poor to start with - that is, image quality isn't what we're concerned about saving, it's the intellectual content - if we don't call it preservation, what do we call it? Janet Gertz

Recast the question:

1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs5. Understand your actions - preserving the

intellectual essence

Less Bad Practice

• Are we there yet?• How will I know my institution is ready?• Isn’t my preservation space already “special”?• Can I afford it?• How often will it be a problem?

JPEG2000/MXF, AS-07

10-bit uncompressed 100 GBDV25 14DV50 27ProRes (HQ) 33JPEG2000 38

File Sizes

1. Use as High a data rate as possible2. Use Non-proprietary codec3. Understand what you’re throwing away4. Understand how much storage costs5. Understand your actions - preserving the

intellectual essence

Less Bad Practice

6. Exert yourself

"advocate for archives”-make good choices-Who was ever fired for doing a better job?

- Advocate for uncompressed

- Adopt JPEG2000/MXF

Less Bad Practice

On Choosing A Preservation File Format for Video: “TIFFs are too big to store”, or “We Used JPEGs and Nobody Died”

George Blood

Digital Preservation Interest GroupAmerican Library Association, Preservation and Reformatting SectionSan Francisco 2015