2016 s.1 markov chains and hidden markov models€¦ · chains and hidden markov models duality...
TRANSCRIPT
QuantitativeUnderstandinginBiologyConclusion:IntroductiontoMarkovChainsandHiddenMarkovModelsDualitybetweenKineticModelsandMarkovModelsWe’llbeginbyconsideringthecanonicalmodelofahypotheticalionchannelthatcanexistineitheranopenstateoraclosedstate.Wemightdescribethesystemintermsofchemicalspeciesandrateconstantsusingasimplerepresentationofthereactionlike…
𝑂𝑘!↔𝑘!!
𝐶
Weshouldbeonfamiliarground,andcanproceedtowritedifferentialequationsbasedontheprincipleofmassaction.
𝑑𝑥!𝑑𝑡 = −𝑘!𝑥! + 𝑘!𝑥!𝑑𝑥!𝑑𝑡
= 𝑘!𝑥! − 𝑘!!𝑥!
Herewehavewrittenthedifferentialsintermsofthefractionsofopenandclosedchannels,denotedasxOandxC.Youmightbeusedtowritingthesetypesofequationsintermsofconcentrations;however,itiseasytoshowthat,foraconstantvolumesystem,thepopulationfractionisproportionaltotheconcentration.
𝑂 =𝑛!𝑉=𝑛!𝑛!∙𝑛!𝑉= 𝑥! ∙ [𝑇]
Hereweseethattheconcentrationofopenchannelsisjustthenumberofopenchannels,nO,dividedbythevolumeofthesystem,V.Wecanintroducethetotalnumberofchannelsinthesystem,nt,andstillhaveavalidmathematicalrelations.Thisleadstothe(unsurprising)resultthattheconcentrationofopenchannelsisequaltothetotalconcentrationofchannelstimesthefactionofchannelsthatareopen.
Weshouldnotethatinexperimentsinvolvingchannels,theconcentrationofchannelsmaybemeasuredinchannelsperunitsurfaceareaofmembrane.Thisdoesnotchangetheresult,whichallowsustowritethedifferentialequationsabove,withtheunderstandingthattherateconstantshavetheappropriateunits.
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page2
Inthislecture,we’llbeconsideringonlydiscreettimeMarkovchains,soitwillbenaturalforustotransformourdifferentialequationsintoadiscretetimerepresentation.Theprocessshouldbefamiliar;representadiscreetchangeinthestatevariablesintermsoffinitebutsufficientlysmallΔt.
𝑥!,!!! − 𝑥!,! = −𝑘!𝑥!,!∆𝑡 + 𝑘!!𝑥!,!∆𝑡𝑥!,!!! − 𝑥!,! = 𝑘!𝑥!,!∆𝑡 − 𝑘!!𝑥!,!∆𝑡
Thiscanberewrittenas…
𝑥!,!!! = 1 − 𝑘!∆𝑡 𝑥!,! + 𝑘!!∆𝑡 𝑥!,!𝑥!,!!! = 𝑘!∆𝑡 𝑥!,! + 1 − 𝑘!!∆𝑡 𝑥!,!
…andexpressedinmatrixformas…
𝑥!𝑥! !!!
= 1 − 𝑘!∆𝑡 𝑘!!∆𝑡𝑘!∆𝑡 1 − 𝑘!!∆𝑡
𝑥!𝑥! !
Analternativeviewofthissystemistoconsiderasingleionchannel.Thisisasystemthatcanbeinoneoftwostates,labeledasOandC,asinthediagrambelow.Ifweimaginethechanneliscurrentlyintheclosedstate,thenwereasonthatinasmalltimeinterval,Δt,thesystemhasaprobability,α,oftransitioningintotheopenstate.Similarly,ifthechannelisintheopenstate,thereisaprobability,β,thatitwilltransitionintotheclosedstate.
Implicitinthisdiagramisthefactthattheprobabilityoftheionchannelremainingintheclosedstategiventhatitstartsinthatstateis1–α,andsimilarlythattheprobabilityofremainingintheopenstategiventhatasthestartingpointis1–β.Insomeofthemoreelementaryliterature,youwouldseethediagramfortheMarkovchaindrawnasshownbelow.Fortheremainderourdiscussion,we’llleaveoutthesecases,andunderstandthattheyareimplicitbecausealloftheprobabilitiesleavingastatemustsumtoone.
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page3
IfxOandxCrepresenttheprobabilityofthechannelbeingintheopenandclosedstate,thenwecanwriteamatrixequationthatderivesdirectlyfromthesediagrams.
𝑥!𝑥! !!!
= 1 − 𝛽 𝛼𝛽 1 − 𝛼
𝑥!𝑥! !
Thismatrixequationisverysimilartotheonewederivedfromthekineticviewofthesystem.Akeydifferenceisthatinthekineticview,wetookxOandxCtobethefractionofchannelsinapopulationthatwereopenorclosed,whereasinthesingle-channelview,wesaidxOandxCweretheprobabilitiesthataparticularchannelofinterestisopenorclosed.Itturnsoutthatwhenallofthestatesofasystemareaccessiblefromallotherstates,thesequantitiesareequal.Suchsystemsarecalledergodic,andtheequalityofthetimeaverageofasingleentityandthebulkaverageofanensembleofentitiesiscalledtheergodichypothesis.NotethatforaMarkovchaintobeegrodic,theremustbeawaytoreacheverystatefromeverystate,butnotnecessarilyinonestep.1
Aslightlymorecomplexmodelofanionchannelisonethatincorporatesaninactivestate…
𝐼𝑘!↔𝑘!!
𝑂𝑘!↔𝑘!!
𝐶
Thediscretizedversionsofthedifferentialequationsthatdescribethissystemare(youshouldbeabletoderivethisonyouown):
𝑥!𝑥!𝑥! !!!
=1 − 𝑘!Δ𝑡 𝑘!!Δ𝑡 0𝑘!Δ𝑡 1 − 𝑘!Δ𝑡 − 𝑘!!Δ𝑡 𝑘!Δ𝑡0 𝑘!Δ𝑡 1 − 𝑘!!Δ𝑡
𝑥!𝑥!𝑥! !
ItsMarkovchaindiagramwouldlooklikethis:
Thissystemisergodicbecausethereisapathfromeverystatetoeveryotherstate(e.g.,youcangetfromItoCthroughO.Thematrixequationthatdescribesthissystemis…
1Theconditionisactuallyalittlemorestrict.Afinite-stateMarkovchainisergodicifthereisafinitenumberNsuchthatanystatecanbereachedbyanyotherstateinexactlyNsteps.ThechaintotherightisnotergodicbecausethereisnosinglenumberofstepsfromwhichyoucangetfromAtoBandfromAtoC.
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page4
𝑥!𝑥!𝑥! !!!
=1 − 𝛿 𝛾 0𝛿 1 − 𝛽 − 𝛾 𝛼0 𝛽 1 − 𝛼
𝑥!𝑥!𝑥! !
Forboththetwoandthreestatesystems,youcanseethereisadirectcorrespondencebetweenthekineticrateconstantsandthetransitionprobabilities.IfyouknowthetransitionprobabilitiesforaspecificΔt,thenyouknowtherateconstants,andvice-versa.
AsignificantmotivationforusingMarkovchainsisthatitgivesustoolsforsystematicallydeterminingthetransitionprobabilitiesforcomplexsystems.Forthetwocanonicalionchannelsystemsabove,itisprettyeasytodesignvoltageclampexperimentsthatallowforthedeterminationoftherates.However,formorerealisticandcomplexmodels,thisbecomesincreasinglydifficult.Forexample,ClancyandRudy(1999)proposedamodelthatincludedthreeclosedstatesandbothafastandaslowinactivestate:
ThekeycharacteristicofMarkovprocessesisthattheprobabilityoftransitingtothenextstateisdependentONLYonthecurrentstate.Inotherwords,thesystemhasnomemory.ThisisknownastheMarkovproperty.
Weshouldalsopointoutthatformanyionchannelsofphysiologicalimportance,thetransitionprobabilitiesarevoltagedependent.Intheotherwords,α,β,γ,andδareallfunctionsofvoltage.Wewillonlyconsiderprocessweretheelementsofthetransitionmatrixareconstantovertime(asinanexperimentwiththevoltageclamped).
AbsorbingMarkovchainsNotallMarkovprocessesareergodic.Animportantclassofnon-ergodicMarkovchainsistheabsorbingMarkovchains.Theseareprocesseswherethereisatleastonestatethatcan’tbetransitionedoutof;youcanthinkifthisstateasatrap.Someprocesseshavemorethanonesuchabsorbingstate.
OneverycommonexampleofaMarkovchainisknownatthedrunkard’swalk.Inourvariationofthisclassictoyexample,weimagineadrunkpersonwanderingaone-dimensionalstreet.Therearefivelocationsalongthestreet,asshowninthediagrambelow.
LocationAistheman’shome,andlocationEisabar.Thedrunkwillstaggerfromonelocationtothenextwhileheisbetweenthebarandhishome,butoncehereachesoneofthetwolocations,hewill
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page5
staytherepermanently(oratleastforthenight).Thedrunkard’swalkisusuallypresentedwheretheprobabilityofmovingtotheleftorrightisequal,butinourscenario,we’llsaythattheprobabilityofmovingtotherightis2/3becausethepleasantodorsemanatingfromthekabobshopnexttothebarserveasanattractant.
Wecanwriteamatrixequationtomodelourdrunkard’swanderings:
𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !!! = 𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !
1 0 0 0 013
023
0 0
013
023
0
0 013
023
0 0 0 0 1
Notethatinthiscasewe’vewrittenthestatevariablesofoursystem(theprobabilitiesthatourdrunkardisatanyofthefivepositionsalongthestreet)asarowvector.ThisfollowstheconventionusedinmostoftheliteratureonMarkovmodels,sowe’veadoptedithere,andwe’lluseitfortherestofthislecture.Asaconsequence,ourequationstodescribethetimeevolutionmultiplythetransitionmatrixontheleft.Also,thematrixinthisrepresentationisthetransposeofthematrixwe’dhavewrittenifwewereusingcolumnvectors.Whenwrittenlikethis,alloftherowsofthetransitionmatrixmustsumtoone.
WecanuseresultsfromthetheoryofMarkovchainstoquicklyanswersomeinterestingquestionsaboutaparticularabsorbingchain.We’lllookatthreesuchquestions:howtocomputethenumberoftimeseachparticularstateislikelytobevisited,howlongasystemislikelytolastbeforebeingabsorbed,andwhattheprobabilityofbeingabsorbedintoeachoftheabsorbingstatesis.Ingeneral,theanswertoeachofthesequestionsisdependentonwhereyoustartonthechain.
Beforeweansweranyofthesequestions,we’llneedtore-arrangeourtransitionmatrixintoacanonicalform.Allweneedtodoisreorderthestatessothatthetransientonesarefirst,andtheabsorbingonescomelast.Forourexampleproblem,wehave…
𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !!! = 𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !
𝐵 𝐶 𝐷 𝐴 𝐸0 !
! 0 !! 0
!! 0 !
! 0 0
0 !! 0 0 !
!
0 0 0 1 0
0 0 0 0 1
𝐵
𝐶
𝐷
𝐴
𝐸
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page6
Herewehaveannotatedtherowsandcolumnswiththenamesofthestatethateachcorrespondsto.Thiscanonicalmatrix,T,canbesubdivideintofourpartsandwrittenas…
𝑇 = 𝑄 𝑅0 𝐼
…where…
𝑄 =
0 !!
0!!
0 !!
0 !!
0
𝑅 =
!!
00 00 !
!
…and0representsamatrixofallzeros,andIrepresentsanidentitymatrix(onesonthediagonal,zeroselsewhere).
Exercise:Showthat𝑻𝒏 = 𝑸𝒏 ∗𝟎 𝑰 ,wherethe*representsacomplicatedmatrix.
Itisusefultodefineonemorematrix…
𝑁 = 𝐼 − 𝑄 !!
Thematrixwe’vedenotedasNisknownasthefundamentalmatrixoftheoriginalMarkovchain.ItturnsoutthatseveralpracticalquestionscanbereadilyansweredaboutanabsorbingMarkovchainonceNandRandknown.Infact,itcanbeshownthatifyoustartinstatei,theexpectednumberoftimesthatyou’llvisitstatejisgivenbetheelementNi,jofthefundamentalmatrix(wewon’tprovethishere).
TocomputethisinMATLABforourdrunkardswalk…
>> Q = [ 0, 2/3, 0; 1/3, 0, 2/3; 0, 1/3, 0] Q = 0 0.6667 0 0.3333 0 0.6667 0 0.3333 0 >> N = (eye(size(Q)) - Q)^-1 N = 1.4000 1.2000 0.8000 0.6000 1.8000 1.2000 0.2000 0.6000 1.4000
Thistellus,forexample,thatifourdrunkardstartsoutinthemiddleoftheroad(stateC),onaveragehewillvisitstateB0.6times,stateC1.8times,andstateD1.2times(rememberthatthefirstrowofthis
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page7
matrixcorrespondstostateB,thesecondtostateC,andthelasttostateD).NotethatthequantityforCincludesthestartingvisit.
Wecanalsocomputetheaverageabsorptiontimeforeachstartingstate.ThisistrivialifweknowN.Forexample,considerstartinginstateC.WejustsawthatthequantitiesinthesecondrowofNgivetheaveragenumberoftimeseachtransientstatewillbevisited,sothesumofalloftheseentriesmustbetheaveragenumberofvisitstoalltransientstates.Butthenumberoftransientstatesonevisitsbeforebeingabsorbedistheabsorptiontime.
Acolumnvector,t,thatrepresentsthesumofrowentriesinthefundamentalmatrixcanbewrittenas..
𝑡 = 𝑁 ∙ 𝑐
…wherecisacolumnvectorofones.ContinuingtheexampleaboveinMATLAB…
>>t=N*ones(size(N,1),1)t=3.40003.60002.2000
ThisshowsthatifourdrunkardstartsatstateB,onaveragethedrunkardwillstaggeraroundfor3.4stepsbeforefinishinguptheeveningathomeoratthebar.
Wecanalsocomputetheabsorptionprobabilitymatrix,B,simplybymultiplyingNontherightbyR.TheelementsofBgivetheprobabilityofultimatelybeingabsorbedintoeachoftheabsorbingstatesforeachstartingstate.Inourexample…
𝐵 =
𝐵 𝐶 𝐷 𝐴 𝐸𝐵
𝐶
𝐷
1.4 1.2 0.8
0.6 1.8 1.2
0.2 0.6 1.4
!! 0
0 0
0 !!
=
𝐴 𝐸𝐵
𝐶
𝐷
0.4667 0.5333
0.2000 0.8000
0.0667 0.9333
TheseresultstellusthatifwestartatstateB,forexample,thereisa46.7%chancethatwe’lleventuallyabsorbintostateA,anda53.3%changethatwe’llabsorbintostateE.
Thedrunkard’swalkisclearlyahypotheticalproblem,butexamplesjustlikeitappearinmanyothercontexts.Forexample,consideratennisgameatdeuce(e.g.,tiedat40-40).Therulesoftennisrequirethatagamebewonbytwoserves.IfplayerIwinsthenextserve,thescoreissaidtobe“AdvantageI”,
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page8
andthatplayermustwinthenextservetowinthegame.Ifshedoesnot,thescoregoesbacktodeuce.IfweimagingthatplayerIhasaprobabilityαofwinninganyserve,thentherelevantMarkovchainis…
ComparingthiswiththeMarkovchainforthedrunkard’swalkshouldmakeitapparentthatthesediagramsinfacthavethesamestructure.Wecanthereforconclude,forexample,thatifoneplayerhastwicethechanceofwinningaserveasanother(α=2/3;1-α=1/3),thentheprobabilityofthelesserplayerwinningagamethathasreacheddeuceis20%(seecomputationoftheBmatrix,above).
HiddenMarkovModelsSofar,we’veconsideredMarkovmodelswhereweknowthevaluesofthetransitionmatrix,andwentontocomputepropertiesofthesystem.We’llnowinverttheproblem,andaddressthecasewherewehaveobservationsofasystem,andwanttoestimatethevaluesinthetransitionmatrix.Inthiscase,we’llassumethatweknowhowmanystatesthereare,andneedtoestimatetheprobabilitiesoftransitioningfromeachstatetoeveryotherstate.
Asaworkingexample,we’llconsiderthisinthecontextofionchannels,continuingthediscussionatthebeginningofthislecture.Asshownpreviously,theproblemistantamounttodeterminingtherateconstantsforthestatetransitions.
We’llbeginbygeneratingsomerepresentativedatafromamodelwithknownparameters,andthenpretendwedon’tknowtheparametervalues,andseehowwecanestimatethemfromthedata.Westartwiththesimple,two-statechannel:
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page9
Beforewebegin,we’llneedtosayafewwordsabouthiddenMarkovmodels(orHMMs).InahiddenMarkovmodel,thestatesofthesystemarenotdirectlyobservable;instead,weareonlyabletoobserveandrecordsomequantitythatcorrelateswiththeinternalstateofthesystem.Inthecaseofoursimpleionchannel,theobservedquantityisthemeasuredcurrent.Thereisnothingreallyhiddenaboutthismodelbecausethesystemhastwostates,andthestateofthesystemisobviousfromthecurrent–ifyouseeahighcurrent,thechannelisintheopenstate,andalow(nearzero)currentimpliesthatthechannelisintheclosedstate.Fornow,we’llchalkthisuptosemantics,butinashortwhilewe’llseeamodelwheresomeofthestatesarebetterhidden.
TheobservablequantitiesinanHMMarecalledemissions(sinceeachstate‘emits’suchaquantity),andanHMMalwayshasanemissionmatrixalongwithitstransitionmatrix.Forourfirstsystem,thetransitionmatrix(usingnotationconsistentwitharowvectorofstatevariables)is
𝑇 =𝐶 𝑂
1 − 𝛼 𝛼
𝛽 1 − 𝛽𝐶
𝑂
…andtheemissionmatrixis…
𝐸 =𝐿 𝐻1 0
0 1
𝐶
𝑂
Herewehaveannotatedtheemissionmatrixtoclarifywhichstateeachrowcorrespondsto,andwhichemissioneachcolumncorrespondsto.Theelementsoftheemissionmatrixareprobabilitiesandgenerallydon’thavetobezerosorones;theyjusthappentobeinthiscase.
WearenowreadytodescribethissysteminMATLAB,andgeneratesomerepresentativedataforthissystem.
>> alpha = 0.01; beta = 0.06; >> T = [1 - alpha, alpha; beta, 1 - beta]; >> E = [ 1 0; 0 1]; >> [seq, states] = hmmgenerate(100000, T, E);
HerewashaveaskedMATLABtogenerate100,000iterationsofourmodel.Wearegiventheemissionsintheseqvector,andthe‘unobservable’internalstatesofthesysteminthestatesvector.
Plottingthedatarequiresthatwespecifythescaleoftheaxes,sinceallthedatacan’tmeaningfullybeshown.
>> plot(seq); >> axis([0 1000 0 3]);
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page10
Youcanpantotherighttoseemoreofthetrace.Asasidenotehere,youshouldbeawarethatwhenmakeaplotthathasmoredatapointsthanyourscreenhaspixels,yourplottingprogrammayentirelymissartifactsinyourdata(thisdependsonhowyoursoftwaresamplesthedataforplotting).
Notethatintheplotproduced,therearesomelongperiodsoflowcurrentandsomeshortperiodsoflowcurrent,andsimilarlyfortheperiodsofhighcurrent.Wesaysthatthedwelltimesforeachstatefollowsomedistribution.Itshouldbeclearfromtheplotthattheclosedstatestendtolastlongerthantheopenstates.Thisimpliesthatβ>α,whichisinfactthecase.
NotethatMATLABnumbersthestatesandemissionsfrom1,soanemissionvalueof1correspondstothelowcurrentemission,andanemissionvalueof2representsahighcurrentemission.
Formally,anHMMconsistsofatransitionmatrix,anemissionsmatrix,andastartingstate.WhenusingMATLAB’sHMMfunctions,itisassumedthatthesystemalwaysstartsinstate1.Youcanalwaysrearrangethestatestoaccommodate(orseetheMATLABhelpforothertricks).
Nowthatwehavegeneratedagoodamountoftracedata(representativeofasinglechannelrecording),outgoalistorecovertheαandβparametersfromthetracedataalone.Forthissimplemodel,thereareseveralapproaches.Onewaytogoaboutthisistoanalyzethedistributionsofthedwelltimesintheopenandclosedstates.Tothisend,wecanwriteafunctionwhichreportsthedwelltimesforaparticularemission.Thesourcecodeiskeptinafilenameddwell_times.m:
function [ times ] = dwell_times(seq,s) %DWELL_TIMES Given a sequence of states (seq), % compute the dwell times state s in the sequence. imax = length(seq); times = []; previous_state = seq(1); counter = 0; for idx = 2:imax if(previous_state == seq(idx)) counter = counter + 1; else if (previous_state == s) times = [times ; counter]; end counter = 1; previous_state = seq(idx); end end end
Wecannowcomputethedwelltimesforthelowcurrentstate.
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page11
>> dt_closed = dwell_times(seq, 1); >> mean(dt_closed) ans = 96.6472
Thisistellingusthattheaveragedwelltimeinthelowcurrentstateis96.6.Thereciprocalofthisisourestimateofthetimeconstant,ortransitionprobability,toleavetheclosedtotheopenstate.Wethereforeestimatethatα=1/96.6472=0.0103.Similarly,inthisexample,weestimatetheβ=0.0642fromthedistributionofdwelltimesintheopenstate.
Inthisexample,wewereabletomaketheseestimatesbecausewecantriviallyinferthestateofthechannelfromitsemissionmeasurement.We’llnowconsideraslightlymorecomplicatedchannelmodelwhere(someof)thestatesarehidden.
We’llreturntoourthreestatechannelmode,assigningvaluestoallofthetransitionprobabilities.
Ourtransitionmatrixis…
𝑇 =
𝐶 𝐼 𝑂1 − 𝛼 0 𝛼
0 1 − 𝛿 𝛿
𝛽 𝛾 1 − 𝛽 − 𝛾
𝐶
𝐼
𝑂
…andouremissionmatrixis…
𝐸 =
𝐿 𝐻1 0
1 0
0 1
𝐶
𝐼
𝑂
Wecannowproceedtosimulatethissystemandmakeaplotofthesimulatedcurrent:
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page12
>> alpha = 0.1; beta = 0.01; gamma = 0.005; delta = 0.005; >> T = [1 - alpha, 0, alpha; 0, 1 - delta, delta; beta, gamma, 1 - beta - gamma]; >> E = [1, 0; 1, 0; 0, 1]; >> [seq, states] = hmmgenerate(500000, T, E); >> plot(seq); >> axis([0 1000 0 3]);
Ifyoupanthroughthecurrenttrace,you’llshouldobservethattherearealargenumberofshort-livedlow-currentstates,andasmallernumberoflong-livedlow-currentstates.Lookingatourparameters,weseethattheclosedstatesareexpectedtobeshortlivedrelativetotheinactivestates(becauseα>δ),andtheweshouldhavemoreoccurrencesoftheclosedstatethantheopenstate(becauseβ>γ).
NowwecanstarttoseewhathiddenMarkovmodelsareallabout.Ifyouobservealowcurrentemission,youdon’tknowifthechannelisintheclosedorinactivestate.However,youcanmakeguessbasedonthedwelltimeofthestate.Short-livedintervalsofzero-currentimplythatthechannelisintheclosedstate,andlongeronesimplyitisintheinactivestate.Butsincestatetransitionsarestochastic,youcanneverbesureifyourassignmentiscorrect.
OneoftheverypowertoolsintheworldofHMMsistheVitirbialgorithm.Thisalgorithmtakesasequenceofemissions(i.e.,yourobservations)andthespecificationofyourHMM(i.e.,yourTandEmatrices),andcomputesthemostlikelyhistoryofstatesthatwouldgiverisetotheemissionsthatyousupplied.Herewecomputethepredictedstates,andcomparethemgraphicallytotheactualstates(whichthealgorithmdidn’tknow).
>> predstates = hmmviterbi(seq,T,E); >> plot(states, 'color', 'blue', 'linewidth', 3); >> hold on >> plot(predstates, 'color', 'red', 'linewidth', 1); >> hold off >> axis([1 1000 0 4]);
Ifyouscrollthroughtheplot,you’llseethealgorithmdoesaprettygoodjob,butonceinawhilewillmiss-classifyastate.You’lllikelyneedtopanquiteabittofindthemistakes.YoucanquicklysummarizetheperformanceoftheVitirbialgorithmforthiscase:
>> sum(states == predstates)/length(states) ans = 0.9866
Hereweseethatforour(notsohiddenafterall)model,wecanassignthecorrectstate98.7%ofthetime.Ofcourse,ifthemeandwelltimesoftheinactiveandclosedstateswerecloser,itwouldbehardertotellthemapart,andtheerrorratewouldincrease.
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page13
OurlastapplicationofHMMswillbetodemonstrateatrainingalgorithm.Thejobofthetrainingalgorithmistodeterminethevaluesinthestatetransitionmatrixandtheemissionmatrixgivenatraceofsequences.Thisisaniterativeprocedure,soitwillrequireaninitialguessforthesetwomatrices.Inourexample,theemissionmatrixissimpleandtheguessisobvious.However,guessingthetransitionprobabilitiesfromonlythesimulateddataisabittrickier.
We’llbeginbycomputingthedwell-timesasbefore.
>> dt_low = dwell_times(seq, 1); >> dt_high = dwell_times(seq,2);
Nowthestructureofourmodeltellsusthatweshouldexpecttwoclassesofnon-conductingstateswithdifferentdwell-timedistributions.Wemightsuspectthatahistogramofthedwelltimeswouldhavetwopeaks,buttryingoutafewplotsdoesn’tyieldanythingobvious.However,ahistogramofthelogarithmofthedwelltimeisinformative.
>> hist(log(dt_low), 0:0.1:10)
Youshouldseeapeakveryroughlyaround2.5andapeakaround5.Thisimpliesthattherearetwo(somewhathidden)peaksate2.5=12.2ande5=148.Soourveryroughguessesofthetransitionprobabilitiesareα=1/12.2=0.08andδ=1/148=0.007.Wealsoneedtoguessβandγ;sincethetrainingalgorithmwilldotheheavylifting,we’llassumethatβ=γ.Nowtheaveragedwelltimefortheconductingstateis…
>> mean(dt_high) ans = 67.3133
…sowehavethatβ+γ(thetwowayswecanleavetheopenstate)=1/67.3=0.0149soweguessβ=γ=0.0149/2=0.0074.Wearenowreadytorunthetrainingalgorithm:
>> alphag = 0.08; betag = 0.0074; gammag = 0.0074; deltag = 0.007; >> Tg = [ 1 - alphag, 0, alphag; 0, 1 - deltag, deltag; betag, gammag, 1 - betag - gammag]; >> Eg = E; >> [T_estimate, E_estimate] = hmmtrain(seq, Tg, Eg);
Thetrainingalgorithmmaytakealittlewhiletofinish,butwhenitdoes,wecancomparetheestimatedtransitionmatrixwiththeonefromwhichthesampledatawasderived:
MarkovChainsandHiddenMarkovModels
©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page14
>> T_estimate T_estimate = 0.9001 0 0.0999 0 0.9950 0.0050 0.0098 0.0050 0.9852 >> T T = 0.9000 0 0.1000 0 0.9950 0.0050 0.0100 0.0050 0.9850
HerewecanseethattheHMMhasdoneanearlyperfectjobofextractingtheunderlyingparametersfromthedata.
Weshouldnoteherethatforbothcases,itispossibletodesignexperimentsthatestimatethetransitionprobabilities(orrateconstants)byothermeansthatsomemightconsidermoredirect.However,asmodelcomplexityincreases,thedifficultyofdesigningandexecutingsuchexperimentsbecomesincreasinglydifficult;considerhowonemightestimatetherateconstantsintheClancyandRudymodeldiscussedabove.OneofthegreatutilitiesofHMMsisthattheygiveaconsistentframeworkforthinkingaboutandsolvingproblemsofarbitrarycomplexity.