september 2007 iw-smi2007, kyoto 1 a quantum-statistical-mechanical extension of gaussian mixture...
TRANSCRIPT
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 11
A Quantum-Statistical-Mechanical A Quantum-Statistical-Mechanical Extension of Gaussian Mixture ModelExtension of Gaussian Mixture Model
Kazuyuki TanakaKazuyuki TanakaGraduate School of Information Sciences,Graduate School of Information Sciences,
Tohoku University, Sendai, JapanTohoku University, Sendai, Japanhttp://www.smapip.is.tohoku.ac.jp/~kazu/http://www.smapip.is.tohoku.ac.jp/~kazu/
In collaboration withIn collaboration withKoji TsudaKoji TsudaMax Planck Institute for Biological Cybernetics,Max Planck Institute for Biological Cybernetics, GermanyGermany
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 22
ContentsContents
1.1. IntroductionIntroduction
2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model
3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model
4.4. Quantum Belief PropagationQuantum Belief Propagation
5.5. Concluding RemarksConcluding Remarks
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 33
Information Processing Information Processing by using Quantum Statistical-Mechanicsby using Quantum Statistical-Mechanics
Quantum Annealing in OptimizationsQuantum Annealing in Optimizations
Quantum Error Correcting CodesQuantum Error Correcting Codes
etc...etc...
Massive Information Processing Massive Information Processing by means of Density Matrixby means of Density Matrix
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 44
MotivationsMotivations
How can we construct the quantum Gaussian mixture model?How can we construct a data-classification algorithm by using the quantum Gaussian mixture model?
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 55
ContentsContents
1.1. IntroductionIntroduction
2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model
3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model
4.4. Quantum Belief PropagationQuantum Belief Propagation
5.5. Concluding RemarksConcluding Remarks
September 2007 IW-SMI2007, Kyoto 6
Prior of Gauss Mixture Model
1)1( iXP
2)2( iXP
3)3( iXP
N
iii xXPxXP
1
)()(
1 32
1
23
Histogram
Label xi is generated randomly and independently of each node.
3 labels
xi =1 xi =2 xi =3
One of three labels 1,2 and 3 is assigned to each node.
September 2007 IW-SMI2007, Kyoto 7
Date Generating Process
Data yi are generated randomly and independently of each node.
22
)(2
1exp
2
1
)|(
kikk
iii
y
kXyYP
10,60 11
30,150 11
20,200 33
xi =1
xi =2
xi =3
September 2007 IW-SMI2007, Kyoto 8
Gauss Mixture Models
ixii xXP )(
N
i kikk
N
i kiiii
yg
kXPkXyYPyYP
1
3
1
1
3
1
)(
)()|(),,|(
2
2)(
2
1exp
2
1)()|( ki
kkikiii yygkXyYP
Prior Probability
Data Generating Process
),,|(maxarg)ˆ,ˆ,ˆ(),,(
yP
Marginal Likelihoodfor Hyperparameters, and
September 2007 IW-SMI2007, Kyoto 9
Conventional Gauss Mixture Models
1
0
1
0
)(
)(
N
iik
N
iiki
k
y
yy
k kk
kki μkg
μkgy
),,|(
),,|()(
ρ
1
0
1
0
2k
2
)(
)()-(
N
iik
N
iiki
k
y
yy
N
iikk y
N 1
)(1
, ,
(yi)
Data :2
1
Ny
y
y
y
Parameters :2
1
Nx
x
x
x
),,|(maxarg)ˆ,ˆ,ˆ(),,(
yP
Labels
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 1010
ContentsContents
1.1. IntroductionIntroduction
2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model
3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model
4.4. Quantum Belief PropagationQuantum Belief Propagation
5.5. Concluding RemarksConcluding Remarks
September 2007 IW-SMI2007, Kyoto 11
Quantum Gauss Mixture Models
N
i kikk ygyYP
1
3
1
)(),,|(
)(ln00
0)(ln0
00)(ln
ln00
0ln0
00ln
expTr
)(ln00
0)(ln0
00)(ln
expTr
)(ln00
0)(ln0
00)(
Tr
)(
3
2
1
3
2
1
33
22
11
33
22
11
3
1
i
i
i
i
i
i
i
i
i
kikk
yg
yg
yg
yg
yg
yg
yg
yg
yg
yg
2
2)(
2
1exp
2
1)( ki
kkik yyg
September 2007 IW-SMI2007, Kyoto 12
Quantum Gauss Mixture Models
3
2
1
ln00
0ln0
00ln
F
)(ln00
0)(ln0
00)(ln
)(
3
2
1
i
i
i
i
yg
yg
yg
yG
N
i
iyyP1 )exp(Tr
))(exp(Tr),,|(
F
GF
3
2
1
ln
ln
ln
F
N
i kikk ygyYP
1
3
1
)(),,|(
Quantum Representation
September 2007 IW-SMI2007, Kyoto 13
Quantum Gauss Mixture Models
3
1
3
1
)(
33
22
11
)(ln
)(ln
)(ln
)(
k lkl
ikl
i
i
i
i
B
yg
yg
yg
y
X
H
N
i
iyyP1 )exp(Tr
))(exp(Tr),,|(
F
H
)(
)()(ln)(
lk
lkygB ikkikl
100
000
000
000
000
010
000
000
001
331211 XXX
September 2007 IW-SMI2007, Kyoto 14
),,|(maxarg)ˆ,ˆ,ˆ(),,(
yP
)(
)(
)(
1
0)()()1(
)(
Tr
Tr
eTr
Tr)eTrln(
i
i
i
ii
i
yH
yHkl
y
yHkl
yHy
kl
e
eX
deXe
B
HH
and,forConditionExtremun
Linear Response Formulas
N
i
iyyP1 )exp(Tr
))(exp(Tr),,|(
F
H
Maxmum Likelihood Estimation in Quantum Gauss Mixture Model
September 2007 IW-SMI2007, Kyoto 15
Quantum Gauss Mixture Models
N
iikk
N
iikki
k
y
yy
1
1
)(Tr
)(Tr
ρX
ρX
)(
)(
Tr)(
i
i
y
y
ie
ey
H
Hρ
N
iikk
N
iikki
k
y
yy
1
1
2k
2
)(Tr
)(Tr)-(
ρX
ρX
N
iikkk y
N 1
)(1
lnTrexp ρX
, ,
(yi)
Data :2
1
Ny
y
y
y
Parameters :2
1
Nx
x
x
x
),,|(maxarg)ˆ,ˆ,ˆ(),,(
yP
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 1616
Image SegmentationImage Segmentation
Original Image
Histogram
ConventionalGauss Mixture
Model Quantum Gauss Mixture Model
= 0.2 = 0.4
0 2550 255 0 255
)exp(Tr
))(exp(Tr
),,|(
F
GF
i
i
y
yP
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 1717
Image SegmentationImage Segmentation
Original Image
Histogram
ConventionalGauss Mixture
Model Quantum Gauss Mixture Model
= 0.5 = 1.0
0 255 0 255 0 255)exp(Tr
))(exp(Tr
),,|(
F
GF
i
i
y
yP
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 1818
ContentsContents
1.1. IntroductionIntroduction
2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model
3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model
4.4. Quantum Belief PropagationQuantum Belief Propagation
5.5. Concluding RemarksConcluding Remarks
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 1919
Image Segmentation by Combining Image Segmentation by Combining Gauss Mixture Model with Potts Model Gauss Mixture Model with Potts Model
104321
,1924 ,1923
,1272 ,641
Belief PropagationBelief Propagation
NeighbourNearest :,
1
0
exp
)(
ijxx
N
ix
ji
i
J
xXP
== >
Potts Model
4 labels
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 2020
Image SegmentationImage Segmentation
Original Image Histogram Gauss Mixture
Model
Gauss Mixture
Model and
Potts Model
Belief Belief
PropagationPropagation
September 2007 IW-SMI2007, Kyoto 21
Density Matrix and Reduced Density Matrix
Bij
ijHH ˆ Hρ exp
1
Z
ρρ ii \tr ρρ ijij \tr
ijii ρρ \tr
Reduced Density Matrix
Reducibility Condition
ji
September 2007 IW-SMI2007, Kyoto 22
Reduced Density Matrix and Effective Fields
iBiki λρ
kiZexp
1
i\Bljl
j\Bkikijij
ji
λIIλHρ exp1
ijZ
i
jiAll effective field
are matrices
All effective field
are matrices
iB
jBi \iB j \
September 2007 IW-SMI2007, Kyoto 23
Belief Propagation for Quantum Statistical Systems
i\Bljl
j\Bkikij
j\Bkikij
ji
i
λIIλH
λλ
exptrlog \iij
i
Z
Z
Propagating Rule of Effective Fields
ijii ρρ \trji
Output
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 2424
ContentsContents
1.1. IntroductionIntroduction
2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model
3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model
4.4. Quantum Belief PropagationQuantum Belief Propagation
5.5. Concluding RemarksConcluding Remarks
September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 2525
SummarySummaryAn Extension to Quantum Statistical Mechanical An Extension to Quantum Statistical Mechanical Gaussian Mixture ModelGaussian Mixture ModelPractical Algorithm Practical Algorithm Linear Response Formula Linear Response Formula
Application of Potts Model and Application of Potts Model and Quantum Belief PropagationQuantum Belief Propagation
Applications to Data MiningApplications to Data MiningExtension to Quantum Deterministic Annealing
Future ProblemFuture Problem