duality in a maximum generalized entropy modeldjafari.free.fr/maxent2014/slides/42_slides.pdf ·...
Post on 30-Apr-2020
12 Views
Preview:
TRANSCRIPT
Duality in a maximum
Shinto Eguchi
MaxEnt, 201421-26 September 2014Chateau Clos Luce, Amboise
generalized entropy model
Osamu KomoriAtsumi Ohara
MaxEnt in ecology
Presence data obtained by an ecological study
Log-entropy (Boltzmann-Gibbs-Shannon)
Equal mean space
2
Maximum entropy distribution
Problem : find a distribution f to maximize H( f under the constraints:
Solution is Gibbs distribution
3
MAP estimator
Loss function
Cf. adaptive Lasso, Zou (2006, JASA)
4
Sequential L1-regularization (Phillip, Anderson, Schapire 2006)
GIS (Geographic Information System)
GBIF (Global Biodiversity Inventory Facility) in R
5http://www.seascapemodeling.org/
Habitat map of right whales
Area Under ROC Curve
6
Problems in MaxEnt
7
The duality between the model and estimation is not robust(model misspecification, over-fitting to data, cf. Bayes robust)
MaxEnt jointly suggests the statistical model and estimation,(exponential model and maximum likelihood)
MaxEnt is useful in presence-only-data in ecological studies
U-entropy leads to extension from MaxEnt to MaxU-Ent(The duality is extended in functional degree of freedom)
Model selection and model validation have simple forms
Hill’s diversity numbers(effective number of species) (1973)
8
richness Rare species are highly weighted
exp (log-entropy) All species are fairly weighted
Simpson index Dominant species are highly weighted
Several measures of entropy
Tsallis entropy (1988)
Hill’s diversity (1973)
Simpson (1949)
9
U-entropy (Eguchi, 2006)
Generalized entropy
10
Generator
U-entropy
Example
Convex conjugate
log entropy
Renyi-Tsallis entropy
Generalized divergence
11
U-cross entropy
U-divergence
MaxU-Ent
Problem : find a distribution f to maximize HU ( f under the constraints:
12
13
Minimum divergence geometry
Riemannian metric
Linear connections
14
Dually flat
Riemann metric
Linear connections
Conjugate convexity:
U-model
15
Pythagorean theorem for U-divergence
Theorem
m-geodesicU-geodesic
Max-Entropy model
MaxEnt and MinDiv
Duality of totally geodesic
17
MaxU-Ent= MaxU-model + MinU-divergence
18
Project
19
Which -MaxEnt is the best ?
Can we select the best based on presence data?
Test AUC comparison may suggest better Max -Ent.
Hill’s diversity numbers associate with
one parameter family of MaxEnt methods, { Max -Ent : in R }
Can the family improve the classical Maxent?
Folivora( Three-toed sloth )
20
Max-Ent
21MaxEnt)
22MaxEnt)
Max-Ent
U-MaxEnt
U-entropy
U-Gibbs distribution
U-estimator
U-divergence
U-MaxEnt
U-entropy
U-Gibbs distribution
U-estimator
U-divergence
A class of MaxU-Ent
23
MaxEnt
Log-entropy
Gibbs distribution
KL-divergence
MLE
U-MaxEnt
U-entropy
U-Gibbs distribution
U-estimator
U-divergence
MaxU-Ent
U-entropy
U-Gibbs distribution
U-divergence
MinU-estimator
Thank you!
24
Model validation
25
A brief tutorial on Maxent(Phillips, 2006)
model
loss function
U-loss
exp-model U-model
( U-estimator , U-model)
- log-likelihoodMLE
Robust estimator
Robust model
26
U-estimate
MaxU-Ent
Problem : find a distribution f to maximize HU ( f under the constraints:
27
Related problems
28
Maxent is equivalent to Poisson point process model
( Renner-Warton, 2013, Loyle et all. 2012,
Aarts et al. 2012, Fithian-Hastie, 2012)
-Maxent is equivalent to -Poisson point process model?
-Poisson distribution
5 10 15 20
0.05
0.10
0.15
0.20
0.25
0.30
0.35
5 10 15 20
0.05
0.10
0.15
0.20
0.25
0.30
0.35
5 10 15 20
0.05
0.10
0.15
0.20
0.25
0.30
0.35
5 10 15 20
0.05
0.10
0.15
0.20
0.25
0.30
0.35
29
Habitat map
30
= 0 = 0.5
top related