jack wolf lecture

8/3/2019 Jack Wolf Lecture

http://slidepdf.com/reader/full/jack-wolf-lecture 1/120

CananInforma*onTheoristBeHappyinaCenterforInforma*on

Storage?

JackKeilWolf

CMRR,UCSD

PadovaniLecture

2010SchoolofInforma*onTheory

USC



This talk is dedicated to David Slepian

who taught me all that I know about

informa*ontheoryandalotmore.



Introduc*on

• Averyshortpictorialhistoryofmyrela*onshipwith

RobertoPadovaniandhowIendedupteachingatUCSD.



TheUniv.ofPennsylvania

1952-1956



PrincetonUniversity

1956-1959



USAF

1960-1963



NYU(Uptown)

1963-1965



BrooklynPoly

1965-1973



UMASS

1973-1984



RobertoPadovaniandMe

• RobertoPadovaniwasoneofmygraduatestudentsatthe

UniversityofMassachuse\s.

• HisM.S.thesiswasontheperformanceoferrordetec*ng

codesandhisPh.D.thesiswasonthedesignand

performanceoftrelliscodes.

• HejoinedLinkabitCorpora*onupongradua*onfromUMass.



RobertoPadovaniandMe

• HejoinedQualcommshortlya_eritwasfounded.

• OneofthefirstQualcommproductswasaTCMchipbaseduponapragma*ccodingschemeheco-developed.

• HewastheprinciplearchitectofQualcomm’shighspeedcellulardatasystem.

• HeispresentlyCTOofQualcomm.

• Heisagreatfriendandaterrificboss.



Adver*sementforTCMChip



RobertoPriortoGivinganInvited

LectureatUMassin2008



RobertoPresen*ngtheLecture



RobertoAnsweringQues*onsAbout

theFutureofCommumica*ons



UCSD

• In1983anewinterdisciplinaryresearchcenterwasbeingformedatUCSD.

• ItwascalledtheCenterforMagne*cRecordingResearch(CMRR)

andwasconcernedwitheduca*ngstudentsandpursuingresearchinmagne*crecording.

• Itsoundedinteres*ngtomebecause:

– Ourkidshadallle_Amherstandwewerelookingforsomethingnew.

– IhadworkedwithGoriedUngerboeckatIBMZurichoncodingforapar*alresponsechannelwhichIlearnedwasamodelforthemagne*crecordingchannel.

– UCSDwasinSanDiego.



Loca*on,Loca*on,Loca*on



AMinorProblem

• Iknewnothingaboutmagne*crecording.

• NotonlydidInotknowhowtospellcoercivitybutthefirst

*meImen*oneditinatalkImispronouncedit.

• ButUCSDreluctantlymademeanofferasthefirstfaculty

memberinCMRR.



AdvicefromOtherstoMe

• Berlekamphadwri\en: – "Communica*onlinkstransmitinforma*onfromheretothere.

Computermemoriestransmitinforma*onfromnowtothen.“

– Thatsoundedverygoodtome.

• Butmanyofmyverysmartfriendssaid: – Magne*crecordingisboring.

– Notonlyisitboringbutitisadeadend!Alltheadvanceshave

beenmade.Thefutureliesin…• Op*calrecording

• Holographicrecording

• Etc.,etc.,etc.



1956:IBMRAMACFirstMagne*cHard

Drive

TotalCapacity=5Mbyte



GigabyteDriveCirca1983

IBM3380



A20102TerabyteDrive

$119.00Amazon.com



Arealdensityhasbeenincreasedmorethan250million:meswithrespecttothe

firstRAMACin1956from0.002Mbit/in2to500Gbit/in2intoday

Weexpectmuchhigherarealdensityinthefuture,i.e.,1Tbit/in2and10Tbit/in2

HistoricalArealDensityIncrease

ofHardDiskDrives

25

JackWolfArrivesatCMRR

*

CAGR=Cumula*veAnnualGrowthRate



1980’s:IBM3380Drive



CoatofArmsofInforma*onTheorist

WorkinginDigitalRecordingError

Correction

Encoder

Modulation

Encoder

Write

Equalization

Equalization

and Detection

Modulation

Decoder

Error

Correction

Decoder

Channel

Channel Encoder

Channel Decoder

Notethatthechanneliscontrollable(butbyphysicists)



SomeItemsofInteresttoan

Informa*onTheorist• Thechannel(whichismadeupofthemagne*cmediaand

thewriteandreadheads)keepschanging.

– Bigimprovementsinrecordingdensityhavebeenachievedhere!!

• Theerrorcorrec*ngcodeusedforthelast25yearsisaReed

Solomoncodeinconjunc*onwithahardinputdecoder.

– ButLDPCcodesanditera*vedecodingareontheway!!

• Thepurposeofthemodula*oncodeistopreventcertain

badsequencesfrombeingwri\en.

– Toaninforma*ontheorist,thisiscodingforthenoiselesschannel.



Informa*onTheoristsLikeSimple

Models• Thewritesignalisplainvanilla+1/-1basebandbinarydata.

(NoQAM,M-PSK,etc.)

• AnAWGNchanneliso_enusedasafirstorder

approxima*onforthechannelmodel.Buttheactual

channelisreallymuchmorecomplex.

• Atlowrecordingdensi*esthereisessen*allynoISIso

matchedfilter(bitbybit)detec*onisop*mal(foranAWGN

channel).



SomeItemsofInteresttoan

Informa*onTheorist• However,athigherrecordingdensi*es,theISIcannotbe

ignored.

• Toachievehigherrecordingdensi*es,in1984theindustryabandonedbitbybitdetec*onandadoptedpar*alresponsesignalingwithViterbisequencedetec*ontocombatISI.

• IBMcalleditPRML(sincetheywantedtoavoidtheuseofViterbi’sname).

• EverydiskdrivetodayusessomeformofViterbidetec*on.



WhySuchAmazingProgress?

• Itdependsuponwhoyoutalkto.

– Physicistscreditadvancedmaterialsforheadsanddisks.

– Mechanicalengineerscreditadvancedmechanics.

– Informa*ontheoristscreditapplica*onsofShannontheory.

• Onees*mateisthatabout20%ofthe“progress”wasduetoadvancesinsignalprocessing.

• However,advancesinallfieldswererequiredtomakethesystemwork.



ProgressasSeenByaPhysicist

2007NobelPrize

inPhysicswasawardedtotheinventorsofthe

GMRhead



TheTrueSourcesofProgress

• Manydifferenttechnologicaladvancesledtothisamazingprogress.

• Newinven*onsweretheenablingtechnology.

• However,theconstantprogressbetweentheintroduc*onofthesenewinven*ons,wastheresultof scaling(i.e.,shrinkingthedimensionsofeverything).



Longitudinal magnetic recording (LMR)

technology

Perpendicular magnetic recording (PMR)

technology

Limit was around 150 Gbit/in2

It was achieved by 500 Gbit/in2 today

PMRtechnology- Highanisotropymaterial

- Ver:calalignmentofmagne:za:on- Muchsmallerbitispossible

41

Longitudinalvs.Perpendicular

Recording

Writing is due to flux leakingfrom the write head to the disk.

Reading is due from flux leakingfrom the disk to the read head.



ShingledWriteProcess

disk

100 nm

Gapis100nmbutbitsare25nm.Howcanthisbe??

100 nm



Modula*onCodes

• Thepurposeofmodula*oncodesisto prohibittheoccurrenceofcertaintroublesomesequencessuchassequenceswhichcauseexcessiveISIorwhichmake*mingrecoverydifficult.

• Themostwellknownexampleofamodula*oncodeistheso-called(d,k )code,wherenorunof0’slongerthank orlessthand ispermi\ed.d andk arenonnega*veintegersforwhichk>d .

• InearlyGbytedrives(circa1980),(2,7)and(1,7)codeswereused.Today,varia*onson(0,k )codesareused.

• Shannondiscussedsuchcodesintheverybeginningofhis1948paperinasec*oncalleden*tled“TheDiscreteNoiselessChannel”.



ShannonStatueatCMRR



ClaudeShannon

• Inhisclassic1948paper,Shannonshowedthatforlargen,thenumberoflengthnconstrainedsequences,N(n),isapproximately2Cn.Thequan*tyC iscalledthecapacityoftheconstrainedsystem.

• Saidinanotherway

• Therateofacode,R,isthe(average)ra*oofthenumberofunconstraineddigitstoconstraineddigits.ShannonshowedthatthereexistscodesatrateR,ifandonlyif

R<C .



Compu*ngtheCapacity

• Shannon(1948)gavetwoequivalentmethodsforcompu*ng

thecapacitywhichareapplicableto(d,k)codes.

Firstmethod:

• Forfinitek,N(n)sa*sfiesthelineardifferenceequa*on:

N(n)=N(n-(d+1))+N(n-(d+2))+…+N(n-(k+1)).



Compu*ngtheCapacity

• Bystandardmethodsofsolvinglineardifferenceequa*onsShannonshowedthatCisequaltothebase2logarithmofthelargestrealrootoftheequa*on:

xk+2-xk+1-xk-d+1+1=0.

SecondMethod:

• Shannonshowedthatthecapacityisequaltothebase2

logarithmofthelargesteigenvalueoftheadjacencymatrixofagraphwhichgeneratesthecodesymbols.

• Weillustratethesetwomethodsfora(1,2)code(i.e.,d=1andk=2).



Compu*ngtheShannonCapacityof

Binary(1,2)Codes Firstmethod

•

Ifd=1andk=2,theequa*on xk+2-xk+1-xk-d+1+1=0

becomes

x4-x3-x2+1=0.

• Thelargestrealrootofthisequa*onis1.3247andit’sbase2

logarithmis0.4057.



Compu*ngtheShannonCapacityof

Binary(1,2)Codes SecondMethod

• Aconstraintgraphthatgeneratescodewordsina(1,2)codeis:

• Theadjacencymatrixofthisgraphis:

010

101

100.

0 0

11



Example:ARate1/2(2,7)Code

• UsingeitherofShannon’smethods,thecapacity,C ,ofa(2,7)codeisfoundtobe0.5174.However,Shannondidnottellushowtoconstructcodesatratesnearoratcapacity.

• Avariablelength,fixedrate,R=½,(2,7)code:

informa*onphrases codewords

10 0100

11 1000

000 000100

010 100100 011 001000

0010 00100100

0011 0000100000100011

000

1011

010011

01.

Thecodewords

formaprefix-freecodesocanbe

decoded.



Example:(2,7)Codes

ThiscodewasusedtocombatISIinsystemsusingbitbybitdetec*on:

Nocoding:channelbitspacing=T

1011000

Withrate½(2,7)coding:channelbitspacing=T /2

01001000000100

Minimumsepara*on

between1’s=T

minimumsepara*on

between1’s=3T /2



ConstrainedCodes(2-Dimensions)

• Codingtheoristsarealsointerestedin2-dimensional

constrainedbinarycodes:i.e.,constrainedbinaryarrayswherethebinarydigitsarearrangedinanarrayofrowsand

columns.

• Suchcodesmighthaveapplica*onin2-dimensionalstorage.




• Wewilluseasanexample,a2-dimensionalarraywhere

everyrowandeverycolumnsa*sfiesa1-dimensional

(d,k)constraint.

• Otherinteres*ngconstraintsexist.



A2-Dimensional(1,2)Array

0100101001001... 1001010010010...

0010100100101...

0101001001010...

1010010010101... ...




• Letthearrayhavemrowsandncolumns.

• N(m,n):thenumberofarraysthatsa*sfythe2-

dimensionalconstraint.

• Thenthe2-dimensionalcapacity,C2,isdefinedas:



Capacityof(d,k)ConstrainedArrays

• For2-dimensional(d,k)constraints,C 2existsbutShannon

didn’ttellushowtocomputeit.

• Tothisday,forrectangularconstraints,theexactvalueofthe

capacityisunknownexceptforthetrivialcaseswhereC 2=0

orC2=1.



ECCCodes

• ReedSolomoncodesareusedintoday’sharddiskdrives.

• Weareonthevergeofseeingtheintroduc*onofLDPC

codeswithitera*vedecodinginHDD.



PressReleaseAugust4,2010

• …announcesitslow-densityparitycheck(LDPC)-based…deviceiscurrentlyshippinginmainstream2.5-inchmobileharddiskdriveproducts.…

• Today’sHDDdatarecoveryarchitecturesaremostlybasedonconcatenatedcodingschemeswhichuseReedSolomonerrorcorrec*oncodes,inventedalmost50yearsago.…

• Now,byusing…LDPC-basedsolu*ons,HDDvendorscancon*nuetodoublethestoragecapacityoftheirdrivesevery18months.…

• currentLDPC-baseddevicereducesthenumberoferrors

readfromadiskfrom1in100to1in100Millionbitsofdata,rela*vetothepreviously-usedconcatenatedcodingschemes.



TheFutureofHDD’s

• Itispossiblethatthearealdensitywillsaturateverysoonusingthepresenttechnology.

• Asthesizeofthestoredbitshrinks,thepresentmagne*cmaterialwillnotholdit’smagne*za*on.

• Thisiscalledthesuperparamagne*ceffect.

• Itisbelievedthataradicallynewsystemwillberequiredtoovercomethiseffect.



TheFutureofDiskDrives

• Twosolu*onsarebeingpursuedtoovercomethesuperparamagne*ceffect.

– Onesolu*onistouseamagne*cmaterialwithamuchhighercoercivity.Theproblemwiththissolu*onisthatyoucannotwriteonthematerialatroomtemperaturesoyouneedtoheatthemediatowrite.Thisisdonewithalaser

– Thesecondapproachiscalledpa\ernedmediawherebitsarestoredonphysicallyseparatedmagne*c“islands”separatedbyaseaofnon-magne*cmaterial.



FutureTechnology?

HAMR-HeatAssisted

Magne*cRecording

Pa\ernedMedia



Wri*ngonPa\ernedMedia

OrdinaryMediaPa\ernedMedia

Inordinarymedia,onecanwriteabitanywhereonthemagne*csurface.

Inpa\ernedmediaonemustwriteeachbitonamagne*cisland.Thisisa

difficulttasksinceonecannotreadandwritesimultaneously.



ShingledRecordingforPa\erned

MediaTime Data Islands / Recorded Bits

0 __ x x x x x x x x x

1 0 0 0 0 0 x x x x x

2 1 0 1 1 1 1 x x x x

3 1 0 1 1 1 1 1 x x x

4 0 (wrtten late) 0 1 1 1 0 0 0 0 x

5 0 (written late) 0 1 1 1 0 0 0 0 0

Note that if the data bit written late is the same as the previous bit, thereis no error in the recorded bit!!!



Mathema*calModelAt*mei=1,2,3,... X i databit{0,1}

Y i recordedbit {0,1}

Zistateofchannel {0,1}

Z i =0ifdatabitiswri\enoncorrectisland Z i=1ifdatabitiswri\enlate

Then: Y i = X i if Z i =0

Y i = X i-1if Z i=1.

Thus: Y i = X i ⊕( X i ⊕ X i-1) Z i



PreviousExample

X 0 1 1 0 0 1 0 ...

Z 0 0 0 1 1 0 1 ...

Y 0 1 1 1 0 1 1 ...



ASimpleRate½CodeandChannel

Capacity• Considerthetrivialbinaryrate½codewhereeachdatabitMi is

recordedtwice.

• Thatis,assumethat X 2i-1= X 2i =Mi ε{0,1}.Thensince

Y 2i = X 2i ⊕( X 2i ⊕ X 2i-1) Z 2i

Y 2i =Mi independentofthevalueof Z 2i .

• Adecodercandecodethisrate½codewithzeroerrorprobability

justbyobservingthevaluesofY withevenindicesandthusthezeroerrorcapacityofthischannelisatleast½.

• Asaresult,alowerboundtothecapacityofthechannelis½independentofthesta*s*calmodelassumedforthe Z process.



TwoDifferentModelsfor Z

• Random Z

– { Z i }isBernouliwithparameter p:B( p)

– Thatis,{ Z i }arei.i.d.,and p=Pr [ Z i =1]

• 2-stateGilbertmodel Z :G( p0,1 ,p1,0 )

Z i =0 Z i =1

p0,1

p1,0

1-p0,1

1-p1,0



ChannelCapacity

• Foranymodelforthe Z process,thechannelcapacityis

definedintheusualmanner.

• WecallthecapacityoftheBernoullistatemodelwith

parameter p,C B( p),andwecallthecapacityoftheGilbertstatemodelwithparameters( p0,1 ,p1,0 ),C G( p0,1 ,p1,0 ).

• FortheBernoulistatemodelonecanprovethat:

C B( p)=C B(1-p)andforGilbertstatemodel,onecanprovethat:

C G( p0,1 ,p1,0 )=C G( p1,0 ,p0,1).



ProofthatC B( p)=C B( p)=C B(1- p)

Details are given in the paper “Write Channel Model for Bit-Patterned Media Recording” which will appear in theIEEE Transactions of Magnetics.



BernoulliModel

• Theparameterspacecanthereforebereducedtothe

interval pε[0,1/2].

•

Furthermorethesamesymmetryargumentholdsfornot justtherate-maximizinginputdistribu*on,butforallinput

distribu*ons.

• ThecapacityoftheBernoullimodelisupperboundedby

theachievablerateforagenie-aideddecoder,i.e.,onewiththe{ Z }processrealiza*onknown.



UpperBoundonCapacityforthe

BernouliModel• Giventherealiza*onofthe Z process,whenever Z i−1=1and Z i =0 ,

thevalueof X i-1cannotbedeterminedfromtheY process.

• ThustheBernoullistatechannelisequivalenttoacorrelated

symmetricerasurechannelwithaverageerasureratePr { Z i−1=1, Z i =0 }= p(1-p).

• Theresul*ngerasurechanneliscorrelatedsince,erasuresbeing

dependenton1to0transi*onsinthe Z process,twoconsecu*ve

bitscannotbeerased.



UpperBoundonCapacityforthe

BernouliModel

• Thecapacityofacorrelatedsymmetricerasurechannelisthesameasthatofamemorylesssymmetricerasure

channelwiththesameerasureprobability.

• Therefore,C B( p)<[1−p(1-p)].



Example

Z 0 0 0 1 1 0 ...

Y 0 1 1 1 0 1 ...

X 0



Example

Z 0 0 0 1 1 0 ...

Y 0 1 1 1 0 1 ...

X 0 1



Example

Z 0 0 0 1 1 0 ...

Y 0 1 1 1 0 1 ...

X 0 1 1



Example

Z 0 0 0 1 1 0 ...

Y 0 1 1 1 0 1 ...

X 0 1 1 0



Example

Z 0 0 0 1 1 0 ...

Y 0 1 1 1 0 1 ...

X 0 1 1 0 ? 1



BernouliModel• Theinputdistribu*onthatmaximizesthemutual

informa*onisunknownsonoclosedformexpressionhas

beenfoundforthecapacityoftheBernoulimodel.

• Howeverlowerboundsonthecapacitycanbefoundby

assumingpar*cularformsfortheinputdistribu*on(e.g.,an

i.i.d.inputprocess).

• Very*ghtupperandlowerboundshavebeenfoundforthe

symmetricinforma*onratewhentheinputisuniformandi.i.d.Anaccuratees*mateofthesymmetricinforma*on

ratehasbeenobtainedusingtheBCJRalgorithm.



BernouliModel

SymmetricInforma*onRate



BernouliModel

SymmetricInforma*onRate

• Notethatonthepreviousslide,forsomevaluesof p,an

upperboundtothesymmetricinforma*onrateisstrictly

lessthan½.

• Butweknowthatthetruecapacityisgreaterthanorequal

to½forallvaluesof p.

• Thisshowsthatforthesevaluesof p,thecapacityachieving

inputprocessisnoti.u.d.



BernouliModel

• Toexplorethelossinachievablerateduetoani.i.d.input

weconsideredasymmetricfirstorderbinaryMarkovinput

wherePr { X i =1| X i-1=0 }=Pr { X i =0 | X i-1=1}=β.

• Upperandlowerboundswerefoundforthemutual

informa*onforaMarkovianinputasafunc*onofβand p.



BernouliModelwithMarkovianInput



ComparisonoftheSymmetric

Informa*onRateandtheInforma*onRateforaMarkovianSource



SummaryforBernouliStateModel

• ItwasfoundthatfortheBernoulimodelfor Z ,considerablegainsinthereliabletransferratearepossiblebyusingan

inputwithmemory.



GilbertModel

• Weknowlessaboutcompu*ngthecapacityforthismodel

thanfortheBernoulimodel

• ByusingagenietoinformthedecoderoftheZprocesswe

canobtainanupperboundtothecapacity.Againtheresult

isacorrelatederasurechannelwithaverageerasure

probabilityPr { Z i-1=1,Z i =0 }= p1,0 p0,1/( p1,0 +p0,1)resul*nginthe

upperboundforthecapacity:

C G( p0,1 ,p1,0 )<1–[ p1,0 p0,1/( p1,0 +p0,1)].



Inser*onsandDele*ons

• Foranymodelfor Z onecaninterprettheeffectsofthe

channelintermsofinser*onsanddele*ons.

• If Z i-1

=0and Z i =1,thenY

i-1=Y

i =X

i-1sothereisaninser*onof

X i-1intheY sequence.Ifontheotherhand Z j-1=1and Z j =0,

thenY j-1=X j-2andY j =X jsothereisadele*onof X j-1intheY

sequence.

• Notethatinthismodel,inser*onsanddele*onsalternateinoccurrenceandinser*onsarearepeatofthepreviousdata

digit.



Inser*onsandDele*ons

• Example

X 0 1 1 0 0 1 0 ...

Z 0 1 1 1 1 0 0

Y 0 0 1 1 0 1 0 ...

inser*ondele*on



SomeFinalRemarksONHDD’s

• Peoplehavebeenpredic*ngthedeathofmagne*charddisk

drivesformanyyears.

• Lackinga“prognos*scope”,itisdifficulttopredicthowlong

theHDDwillremainthestoragedeviceofchoice.

• However,magne*charddiskdrivesseemstobea“catwith

ninelives”havingbeatoutallcompe*torsinthepast.



91

CodingforFlashMemories



92

FlashMemory

• Flashisanon-vola*lememorywhichisfast,powerefficient

andhasnomovingparts.

• Electricallyprogrammedanderased.

• Usedin:

– Digitalcameras

– LowcapacityIPODS

– Mobilephones – Laptopcomputers

– Hybriddrives



FlashMemoriesStructure

Arrayofcellsmadefromfloa*nggatetransistors.

Cellsaresubdividedintoblocksandthenintopages.

Thecellsareprogrammedbypulsingelectronsviahot-electron

injec*on.



94





injec*on.



95


Eachcellcanhaveqlevels,

representedbydifferent

amountsofelectrons.

Intoday’sproducts,q=2,4,8or16.

Inordertoreduceacelllevel,

allthecellsinthatblockmustbe

resettolevel0beforerewri*ng.

–AVERYEXPENSIVEOPERATION




injec*on.



96

FlashMemoryStructure

• Thememoryconsistsofblocks

– Thesizeofeachblockis128(or256) KB.

– Eachblockconsistsof64 (or128)pages.

– Thesizeofeachpageis2KB.

• Wri*ng–Writesequen*allytothenextavailable

page.

• Erasing–Canonlyeraseanen*reblock!

Page 1

Page 2

Page 3

!

Page 63

Page 64

! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !



97

FlashMemoryConstraints

• Theenduranceofflashmemoriesisrelatedtotothenumber

of*mestheblocksareerased.

• Insinglelevelflashwithq=2,ablockcantolerate~104-105

erasuresbeforeitstartsproducingexcessiveerrors.

– SLC:SingleLevelCell

• Thelargerthevalueofq,thelesstheendurance.

– MLC:Mul*LevelCell

• TheGoal:Represen*ngthedataefficientlysuchthatblock

erasuresarepostponedasmuchaspossible.



98

ExperimentDescrip*on

• Foreachblockthefollowingstepswererepeated:

• Theblockwaserased.

• Pseudo-randomdatawaswri\entotheblock.

• Thedatawasreadandcomparedtofinderrors.

• Remark:

• Theexperimentwasdoneunderlabcondi*ons.Otherfactorssuchas

temperaturechange,intervalsbetweenerasuresandmul*plereadings

beforeerasureswerenotconsidered.



99

RawBERforSLCblock

!106

!10-4

Guaranteedlife*mebythemanufacturer



100

RawBERforMLCblock

!105

!10-3

Guaranteedlife*mebythemanufacturer



AnIntroduc*ontoWOM-codes

• WOM-codesallowustowriteseveral*mestothesameblockofmemorywithouterasing.

• Example:In1982,RivestandShamirfoundawaytowrite2

bitsofinforma*ontwiceusingonly3cells.

• WedenoteaWOMcodethatwritesk*mestoncellsasa<V 1,V 2,…,V k >/ncodewhereV i isthenumberofmessageswri\enontheithwrite.

• ThustheRivestShamircodeisa<4,4>/3codewithk =2.

101



TheRivest-ShamirCode

102

DataBits FirstWrite SecondWrite

00 000 111

01 001 110

10 010 101

11 100 011

Example1:FirstWrite:Wantto

storedata01:

Write001tomemory.

SecondWrite:Wanttostoredata

10:

Write101tomemory.

Ifwewanttowritethesamedataonthe

secondwrite,wedonotchangewhatiswri\enonthefirstwrite.

Example2:FirstWrite:Wantto

storedata01

Write001tomemory.

SecondWrite:Wanttostoredata

01:Leave001inmemory.

Notethatwhengoingfromfirstwrite

tosecondwrite,no1’sareerased.



RateofaWOM-code

• Therateoftheithwriteis:

Bitsofinforma*on

Totalnumberofbits

log2(V i)

n

• ThetotalrateofaWOM-codeisR=∑(Ri).

• TheRivestShamircodehasR1=R2=2/3andR=4/3.

Ri=

Ri=

103



WOMCapacityRegion

ThecapacityregionofabinaryWOMcodewithtwowritesis

C WOM={(R1,R2)|∃ p∈[0,0.5],R1≤h( p),R2≤1– p}

whereh( p)=- plog2( p)–(1- p)log2(1- p).

R=R1+R2<h( p)+1–p.

Therighthandsideismaximizedfor p=1/3yielding

Rmax=log2(3)=1.58…104

Our Construc*on for a 2 Write WOM



OurConstruc*onfora2-WriteWOM

Code• Choosealinearcode(n,k )withparitycheckmatrixH.Letr=n-k

sothatH isanr -rowbyn-columnmatrixofrankr.

• Foravectorv ϵ{0,1}n,letH (v )bethematrixH with0’sreplacing

thecolumnsthatcorrespondtotheposi*onsofthe1’sinv .

• Onfirstwrite,writeonlythosevectorsv suchthatrank(H (v ))=r .

LetV 1={v ϵ{0,1}n|rank(H (v ))=r }.

• ThenR1=log2|V 1| /n.

Our Construc*on for a 2 Write WOM



OurConstruc*onfora2-WriteWOM

Code• Assumethate1isvectorwri\enonthefirstwrite.

• Secondwrite: – Consideradatavectors2ofr bits.

–

Finde2suchthatH (e1)⋅e2=s1⊕s2,wheres1=H

⋅e1. – Asolu*one2existssincerank(H (e1))=r .

– Writee1⊕e2tomemory.

• Decodingonthesecondwrite: – Mul*plythestoredvectore1⊕e2byH :

H ⋅(e1⊕e2)=He1⊕He2=s1⊕(s1⊕s2)=s2

• Thus,R2=(n-k)/nandR=R1+R2=[log2|V 1|+(n-k)]/n.



WOM-CodeConstruc*on:AnExample

• LetH betheparitycheckmatrixofa(7,4)Hammingcode.

– For a vectorv ϵ{0,1}n

,letH (v )bethematrixH with0’sinthecolumnsthatcorrespondtotheposi*onsofthe1’sinv .

• Onthefirstwrite,weprogramonlyvectorsv suchthatrank(H (v ))=3,

V 1={v ϵ{0,1}n|rank(H (v ))=3}.

• ForHasshownabove,|V 1

|=1+7+21+35+(35-7)=92.Thus,wecanwriteoneof92messagesatthefirstwrite.

• Encodinganddecodingofthefirstwritearedonewithalookuptable.

• Saywewritee1=0101100.

1110100

1011010

1101001

H =n=#ofcolumns=7

r =#ofrows=3




e1=0101100• Notethats1=H ⋅e1=010.

• Insert0’sinthecolumnsofH thatcorrespondto1’sinthefirstwrite.ThisnewmatrixisH (e1):

H =

• Wecannowwriteamessageoflengthr =3bits.Saywewanttostores2=011onsecondwrite.

• Wanttofindavectore2suchthatH (e1)·e2=s1⊕s2.

• s1⊕s2=001

• Choosee2=0000001.

• Thenwewritee1⊕e2=0101101

1110100

1011010

1101001

1010000

1010010

1000001

1010000

10100101000001

=H(e1 )




• Notethatwecandecodebymul*plyingbyH :

– Wecanwrite92messagesatthefirstwritesoV 1=92and

R1=log2(92)/7=0.932,R2=3/7=0.429andR=1.361whichisbe\er

thantheRivest-Shamirconstruc*on.

– However,R1=R2fortheRivest-Shamirconstruc*on.

1110100

10110101101001

0

1

0

11

0

1

. =

0

11

= s2



SomeMoreResults

• ThebestvalueofRpreviouslyachievedfortwowriteswasbyWuwhoobtainedR=1.371.Weobtainedmanycodesthatbe\eredthisresult.

• FortheGolay(24,12)code,weobtainedR=1.4547.

• FortheGolay(23,11)code,weobtainedR=1.4632.

• ChoosingthecodeasthedualoftheReed-Muller(4,2)code,weobtainedR=1.4566.

• IfR1>R2,wecanlimitthenumberofmessagesusedonthefirstwritesothatR1=R2

andR=2R1.DoingthisforthedualoftheReed-Muller(4,2)code,weobtainedR=1.375.

110

Computer Search for Good 2-Write



ComputerSearchforGood2-Write

WOMCodes

• Constructarandommatrixofsizen×r andrankr.

• CyclethroughallvectorsoflengthnandHammingweightat

mostn-r .

• Foreachvectorv ,zerooutcolumnsofthematrixwhere1’s

existinv.

• Computetherankofthematrix.Ifitisthesameasthe

originalrank,addoneto|V 1|.

• Onceweknow|V1|,wecancomputetheratewith

R=(1/n)(log2|V 1|+r )

111



TimeSharing

• Ifweknowtwocodeswithrates(R1 ,R2)and(R3 ,R4),wecan

achieveanyratepair

(t*R1+(1-t )*R3 ,t*R2+(1-t )*R4)

fortara*onalnumberbetween0and1.

112



SomeAchievableRatePairsandCapacity

forWOMWithTwoWrites

113



114

MoreAchievableRatePairsandCapacityfor

WOMWithTwoWrites



CodesforMorethan2Writes

t -#ofWrites LowerBoundOur

Construc*onUpperBound

3 1.58 1.66 log4=2

4 1.75 1.95 log5=2.32

5 1.75 1.99 log6=2.58

6 1.75 2.14 log7=2.8

7 1.82 2.15 log8=3

8 1.88 2.23 log9=3.17

9 1.95 2.23 log10=3.32

10 2.01 2.27 log11=3.46



Ifyouhaven’tguessedalready,theanswerto

theques*on:

Can an Informa*on Theorist Be Happy in a

CenterforInforma*onStorage?

isaresoundingyes.



MyCollaboratorsonthisTalk

PaulSiegel AravindIyengar EitanYaakobi

withSco\Kayser



SomeoftheRestoftheCastatCMRR

AndaSpecialThankstoAllofMy



Ph.D.Students

Altekar,ShirishA.

Armstrong,AlanJ.

Aviran,SharonBaggen,ConstantP.M.J.

Barndt,RichardD.

Bender,Paul

Bernal,RobertW.

Bridwell,JohnD.Bunin,BarryJ.

Caroselli,JosephP.

Chiang,Chung-Yaw

Demirkan,Ismail

Dorfman,VladimirEddy,ThomasW.

Ergul,FarukR.

Fitzpatrick,James

Fredrickson,L.J.French,CatherineAnnFriedmann,ArnonA.

Goldberg,JasonS

Gupta,DevVart

Hartman,PaulD.

Ho,KelvinK.Y.Karakulak,Seyhan

Kerdock,AnthonyM.

Kim,ByungGuk

Klein,TheodoreJ.

Knudson,KellyJ.Kurkoski,BrianM.

Lee,Patrick

Levi,Karl

Liff,AllanI.

Lin,YinyiMa,HowardH.

Ma,JoongS.

MacDonald,CharlesE.

Mangano,DennisT.Marrow,MarcusMasnick,Burt

McEwen,PeterA.

Miller,John

Milstein,LaurenceB.

Padovani,RobertoPanwar,ShivendraS.

Pasternack,Gerald

Paerson,JohnD.

Philips,ThomasK.

Prohazka,CraigG.

Raghavan,SreenivasaA.

Ritz,Mordechai

Rodriguez,ManoelA.

Schiff,Leonard

Souvignier,ThomasV.Trismen,Robert

Wainberg,Stanley

Walvick,Edward

Weathers,AnthonyD.Zehavi,EphraimZhang,Wenlong



Thankyouforyourkinda\en*on.

jack wolf lecture

Documents