Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Computational Paradigmsand Audio Archives

Gianfranco BilardiDept. Information Engineering, University of Padova

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

ComputerScience- Not justtools ...

2

...,but paradigms that canenrich other disciplines• Mathematics:thenotion ofcomputability,applied toverification ofproofs,has enabled therigorous investigation ofproperties such asconsistency,completeness,anddecidability ofmathematicaltheories (Hilbert,Gödel,Turing,...)

• Linguistics:theview oflanguages as setsofstrings generated bygrammars orrecognized bycomputational machines has provided anewfoundation tolinguistics (Chomskyhierarchy,...)

• Physics:theview that the"substance"oftheuniverse isinformation("it frombit"- J.A.Wheeler)andthat theuniverse is a(quantum)computerhave opened newperspectives infundamental physics

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Re-mediation schemeFromperformancetorestoration

3

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Representation ofinformation

4

• ComputingentailsthetransformationofinformationThetransformationmayjustchangetheformat/encoding(reversible,orlosslesstransformation;e.g.,Fouriertransform)Thetransformationmayalsodistillasummary(nonreversible,orlossytransformation;e.g.,sumoftwonumbers)

• An“artwork”canbeviewedasasetofdocuments,typicallywithacommonarchetype(physicalorvirtual)

• Documentsencodeinformationwithintheconfigurationofthephysicalcarrier;primaryinformationandancillaryinformationarebothimportant

• Whenchangingrepresentationofadocument,itisdesirable:(a)todoitviareversibletransformations(notalwayspossible)(b)toincludeaclearspecificationoftheappliedtransformation

• Whennecessarytocompressinformation,modellingtherelativeimportanceofdifferentaspectscanguidethechoiceoftheencodingExample:whencuttingthenumberofsamplesinanaudiosignal,say,byafactoroftwo,dowedropeveryothersampleofthesignalorofalow-passfilteredversionofit?

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Algransolecaricod’amoreByLuigiNonoCourtesyofRAI(Milano)

YEntonces ComprendióByLuigiNonoCourtesyofRAI(Milano)

Ancillaryinformation writtenonthetape

5

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Reasons behind change ofrepresentation

6

• Compression:not enough bandwidth orstorage (intentional)• sourcecoding:eliminates redundant information• lossy compression:eliminates someinformation(lossy compression)Ex.:fromperformancetorecording

• Linecoding/decoding:adaptation tophysical medium(intentional)Ex.:equalization

• Noise management:reduction ofnoise after transmission (intentional)Ex.:DolbyA(analog),error correcting codes (digital)

• Undesirable causes (unintentional)Ex.:imperfections,misalignments,...

• Undoing undesirable transformations (intentional)Ex.:compensation forunintentional transformations

• Enable general-purpose processing:analog-to-digital conversionEx.:Rerecording

• Computational complexity reductionEx.:Fouriertransform forconvolution/filtering,applicable torestoration afteraging

• Faulttolerance,distributed storage;encryption,etc.

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Errorcorrecting codes for distributed archives

7

• Errorcorrectingcodesaugmenttheoriginalsignalwithredundantinformation,whichenablesreconstructionofthesignaleveninthepresenceofnoise

• Codingtheoryisawelldevelopedandsophisticateddisciplinetodesigncodeswiththeminimumredundancycompatiblewithagivenlevelofnoiseprotection

• Errorcorrectingcodescouldbeeffectivetorealizegeographicallydistributedarchives,usingthesameprincipleunderlyingRAIDtechnologyfordiskstorageEx.:DistributingdocumentsacrossmultiplearchiveswithRAID6wouldenabletorecoveranydocumentevenif(any)twoarchiveswerenotaccessible

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Homomorphic encryption for document privacy

8

• Afullyhomomorphicencryptionschemeisamethodtoencodeinformationinsuchawaythatanycomputationcanbecarriedoutontheencryptedinputandproducetheencryptedoutput

• Computationcanthusbecarriedoutbyathirdparty,whogainsnoaccesstotheunencryptedinformation

• Stateoftheart:existenceoffullyhomomorphicschemesestablishedcirca2008(C.Gentry).Sincethen,theapproachhasbeenrefinedandmademoreefficient.Notyetdeployableonalargescaleduetohighcomputationalcomplexity

• Inthefuture,HEcouldopenwaystoguaranteeprivacytoarchiveddocuments,awaytoregulateaccess,andawaytooutsourceserviceswithoutdisclosinginformation.

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Abstractdata types and models of interactionwith ArtWork

9

• Abstractdatatype=info+operationsEx.:dictionary=setwithinsertion,deletion,search

• Framingadocumentasaninstanceofanabstractdatatypemayenabledifferentwaystointeractwiththedocument,addingdimensionstoartisticexperienceandfruition

• Tosupport• soundpreservationofaudiohistory• soundpreservationofanartist• analysisofintentionalandunintentionalalterations,adatatypecouldencompass:hierarchical,modularrepresentationofmultiplesources,suchasarchivalcopy(preservationcopy),criticaledition,commercialedition

• Objectorientedanalogyabouthereditarity andmethods

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Machine Learningand models of interaction withArtWork

10

• MachineLearningdevelops/tunes complex models onthebasis ofknown examples,then applies such models tonewexamples

• Interesting operations applicable todocuments orcollections ofdocuments may beenabled byMachineLearningtechniquesThese operations could provide valuable support todocument philology,forexample in

• Determining which type ofequipment has been used forrecording• Determining whether two pieces have thesame author• Attribution toaperformer(fromaspecified set)• Classification within aSchool/style/period

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

IdentificationofancillaryinformationbycomputervisionandMachineLearningtechniques

11Courtesy ofCSC,University ofPadova

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Historicallyfaithfulfruition:Taperecorder

• Useractsasonarealtaperecorder• Faithfullexperienceoforiginalplayingatape• Accesstovisualinformationonthetapeandmetadata

12Courtesy ofCSC,University ofPadova

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Historicallyfaithfulfruition:grammophone

13Courtesy ofCSC,University ofPadova

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Automaticanalysisofmagnetictapes

• Detectionandclassificationofcuts,annotations,alterations

• Automaticanalysisbycomputervisionandmachinelearningtechniques

14Courtesy ofCSC,University ofPadova

Labo

rato

rio d

i Fon

dam

enti

di In

form

atic

a D

ept.

Info

rmat

ion

Engi

neer

ing

Inconclusion

15

• Wehopewehavegivensomeevidencethatframingsomeissuesarisinginthedomainofaudioarchiveswithinestablishedcomputingparadigmsmightshedsomelightandsuggestavenuestosolutions

AcknowledgementThespeakerisgreatlyindebtedwithSergioCanazza andGianniDePoli forprovidingsubstantialandvaluableinputtothistalk