nonparametric estimation of - department of...
TRANSCRIPT
![Page 1: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/1.jpg)
Nonparametric estimation of
s−concave and log-concave densities:
alternatives to maximum likelihood
Jon A. Wellner
University of Washington, Seattle
Cowles Foundation Seminar, Yale University
November 9, 2016
![Page 2: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/2.jpg)
Cowles Foundation Seminar, Yale University
Based on joint work with:
• Qiyang (Roy) Han
• Charles Doss
• Fadoua Balabdaoui
• Kaspar Rufibach
• Arseni Seregin
![Page 3: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/3.jpg)
Outline
• A: Log-concave and s−concave densities on R and Rd
• B: s−concave densities on R and Rd
• C: Maximum Likelihood for log-concave and s−concave
densities
B 1: Basics
B 2: On the model
B 3: Off the model
• D. An alternative to ML: Renyi divergence estimators
B 1. Basics
B 2. On the model
B 3. Off the model
• E. Summary: problems and open questions
Cowles Foundation Seminar, Yale, November 9, 2016 1.2
![Page 4: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/4.jpg)
A. Log-concave densities on R and Rd
If a density f on Rd is of the form
f(x) ≡ fϕ(x) = exp(ϕ(x)) = exp (−(−ϕ(x)))
where ϕ is concave (so −ϕ is convex), then f is log-concave.The class of all densities f on Rd of this form is called the classof log-concave densities, Plog−concave ≡ P0.
Properties of log-concave densities:
• Every log-concave density f is unimodal (quasi concave).
• P0 is closed under convolution.
• P0 is closed under marginalization.
• P0 is closed under weak limits.
• A density f on R is log-concave if and only if its convolutionwith any unimodal density is again unimodal (Ibragimov,1956).
Cowles Foundation Seminar, Yale, November 9, 2016 1.3
![Page 5: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/5.jpg)
• Many parametric families are log-concave, for example:
B Normal (µ, σ2)
B Uniform(a, b)
B Gamma(r, λ) for r ≥ 1
B Beta(a, b) for a, b ≥ 1
• tr densities with r > 0 are not log-concave.
• Tails of log-concave densities are necessarily sub-exponential.
• Plog−concave = the class of “Polya frequency functions of
order 2”, PF2, in the terminology of Schoenberg (1951) and
Karlin (1968). See Marshall and Olkin (1979), chapter 18,
and Dharmadhikari and Joag-Dev (1988), page 150. for nice
introductions.
Cowles Foundation Seminar, Yale, November 9, 2016 1.4
![Page 6: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/6.jpg)
B. s− concave densities on R and Rd
If a density f on Rd is of the form
f(x) ≡ fϕ(x) =
(ϕ(x))1/s, ϕ convex, if s < 0exp(−ϕ(x)), ϕ convex, if s = 0(ϕ(x))1/s, ϕ concave, if s > 0,
then f is s-concave.
The classes of all densities f on Rd of these forms are called
the classes of s−concave densities, Ps. The following inclusions
hold: if −∞ < s < 0 < r <∞, then
Pr ⊂ P0 ⊂ Ps ⊂ P−∞
Cowles Foundation Seminar, Yale, November 9, 2016 1.5
![Page 7: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/7.jpg)
Properties of s-concave densities:
• Every s−concave density f is quasi-concave.
• The Student tν density, tν ∈ Ps for s ≤ −1/(1 + ν). Thus theCauchy density (= t1) is in P−1/2 ⊂ Ps for s ≤ −1/2.
• The classes Ps have interesting closure properties underconvolution and marginalization which follow from theBorell-Brascamp-Lieb inequality: let 0 < λ < 1, −1/d ≤ s ≤∞, and let f, g, h : Rd → [0,∞) be integrable functions suchthat
h(λx+ (1− λ)y) ≥Ms(f(x), g(y);λ) for all x, y ∈ Rd
where
Ms(a, b;λ) = (λas + (1− λ)bs)1/s, M0(a, b;λ) = aλb1−λ.
Then∫Rdh(x)dx ≥Ms/(sd+1)
(∫Rdf(x)dx,
∫Rdg(x)dx;λ
).
Cowles Foundation Seminar, Yale, November 9, 2016 1.6
![Page 8: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/8.jpg)
Special Case: s = 0: Prekopa-Leindler inequality.
Let 0 < λ < 1 and let f, g, h : Rd → [0,∞). if
h(λx+ λy) ≥ f(x)λg(y)1−λ for all x, y ∈ Rd,
then ∫Rdh(x)dx ≥
(∫Rdf(x)dx
)λ (∫Rdg(x)dx
)1−λ.
Proposition. (a) Marginalization preserves log-concavity.
(b) Convolution preserves log-concavity.
Proof. (a) Suppose that h(x, y) is a log-concave density on
Rm × Rn. Thus
h(λ(x, y) + (1− λ)(x′, y′)) ≥ h(x, y)λh(x′, y′)1−λ.
Cowles Foundation Seminar, Yale, November 9, 2016 1.7
![Page 9: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/9.jpg)
We want to show that
f(y) ≡∫Rm
h(x, y)dx is log-concave.
With h(x) ≡ h(x, λy + λy′), f(x) ≡ h(x, y), g(x) ≡ h(x, y′),
h(λx+ λx′) = h(λx+ λx′, λy + λy′)
≥ h(x, y)λh(x′, y′)λ = f(x)λg(x′)λ.
Thus Prekopa-Leindler yields
f(λy + (1− λ)y′) ≥(∫
Rmh(x, y)dx
)λ (∫Rm
h(x, y′)dx)1−λ
= f(y)λf(y′)1−λ.
(b) If X ∼ f and Y ∼ g are independent, f, g log-concave, then
(X,Y ) ∼ f ·g ≡ h is log-concave. Since log-concavity is preserved
under affine transforms (X−Y,X+Y ) has a log-concave density.
By (a), X + Y is log-concave. �
Cowles Foundation Seminar, Yale, November 9, 2016 1.8
![Page 10: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/10.jpg)
• If f ∈ Ps and s > −1/(d+ 1), then Ef‖X‖ <∞.
• If f ∈ Ps and s > −1/d, then ‖f‖∞ <∞.
• If f ∈ P0, then for some a > 0 and b ∈ R
f(x) ≤ exp(−a‖x‖+ b) for all x ∈ R.
• If f ∈ Ps and s > −1/d, then with r ≡ −1/s > d, then for
some a, b > 0
f(x) ≤ (b+ a‖x‖)−r for all x ∈ R.
• If s < −1/d then there exists f ∈ Ps with ‖f‖∞ =∞.
• If −1/d < s ≤ −1/(d + 1), then there exists f ∈ Ps with
Ef‖X‖ =∞.
Cowles Foundation Seminar, Yale, November 9, 2016 1.9
![Page 11: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/11.jpg)
-4 -2 2 4
0.02
0.04
0.06
0.08
0.10
f(x) =1
√rBeta(r/2,1/2)
(r
r + x2
)(1+r)/2,
with r = .05, so f ∈ P−1/(1+.05)
Cowles Foundation Seminar, Yale, November 9, 2016 1.10
![Page 12: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/12.jpg)
-4 -2 2 4
0.05
0.10
0.15
0.20
0.25
0.30
tr densities, r ∈ {0.1,0.2, . . . ,1.0}
Cowles Foundation Seminar, Yale, November 9, 2016 1.11
![Page 13: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/13.jpg)
-3 -2 -1 0 1 2 3
1
2
3
4
f(x) =1
2Γ(1/2)|x|−1/2exp(−|x|), with f ∈ P−2.
Cowles Foundation Seminar, Yale, November 9, 2016 1.12
![Page 14: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/14.jpg)
C. Maximum Likelihood:
0-concave and s-concave densities
MLE of f and ϕ: Let C denote the class of all concave function
ϕ : R→ [−∞,∞). The estimator ϕn based on X1, . . . , Xn i.i.d. as
f0 is the maximizer of the “adjusted criterion function”
`n(ϕ) =∫
logfϕ(x)dFn(x)−∫fϕ(x)dx
=
∫ϕ(x)dFn(x)−
∫eϕ(x)dx, s = 0,∫
(1/s)log(−ϕ(x))+dFn(x)−∫
(−ϕ(x))1/s+ dx, s < 0,
over ϕ ∈ C.
Cowles Foundation Seminar, Yale, November 9, 2016 1.13
![Page 15: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/15.jpg)
1. Basics
• The MLE’s for P0 exist and are unique when n ≥ d+ 1.
• The MLE’s for Ps exist for s ∈ (−1/d,0) when
n ≥ d(
r
r − d
)where r = −1/s. Thus n→∞ as −1/s = r ↘ d.
• Uniqueness of MLE’s for Ps?
• MLE ϕn is piecewise affine for −1/d < s ≤ 0.
• The MLE for Ps does not exist if s < −1/d. (Well known for
s = −∞ and d = 1.)
Cowles Foundation Seminar, Yale, November 9, 2016 1.14
![Page 16: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/16.jpg)
2. On the model
• The MLE’s are Hellinger and L1− consistent.
• The log-concave MLE’s fn,0 satisfy∫ea|x||fn,0(x)− f0(x)|dx→a.s. 0.
for a < a0 where f0(x) ≤ exp(−a0|x|+ b0).
• The s−concave MLE’s are computationally awkward; log is“too aggressive” a transform for an s−concave density. [Notethat ML has difficulties even for location t− families: multipleroots of the likelihood equations.]
• Pointwise distribution theory for fn,0 when d = 1;no pointwise distribution theory for fn,s when d = 1;no pointwise distribution theory for fn,0 or fn,s when d > 1.
• Global rates? H(fn,s, f0) = Op(n−2/5) for −1 < s ≤ 0, d = 1.
Cowles Foundation Seminar, Yale, November 9, 2016 1.15
![Page 17: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/17.jpg)
3. Off the modelNow suppose that Q is an arbitrary probability measure on Rdwith density q and X1, . . . , Xn are i.i.d. q.
• The MLE fn for P0 satisfies:∫Rd|fn(x)− f∗(x)|dx→a.s. 0
where, for the Kullback-Leibler divergence
K(q, f) =∫qlog(q/f)dλ,
f∗ = argminf∈P0(Rd)K(q, f)
is the “pseudo-true” density in P0(Rd) corresponding to q.In fact: ∫
Rdea‖x‖|fn(x)− f∗(x)|dx→a.s. 0
for any a < a0 where f∗(x) ≤ exp(−a0‖x‖+ b0).
Cowles Foundation Seminar, Yale, November 9, 2016 1.16
![Page 18: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/18.jpg)
• The MLE fn for Ps does not behave well off the model.
Retracing the basic arguments of Cule and Samworth (2010)
leads to negative conclusions. (How negative remains to be
pinned down!)
Conclusion: Investigate alternative methods for estimation in
the larger classes Ps with s < 0! This leads to the proposals by
Koenker and Mizera (2010).
Cowles Foundation Seminar, Yale, November 9, 2016 1.17
![Page 19: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/19.jpg)
D. An alternative to ML:
Renyi divergence estimators
0. Notation and Definitions
• β = 1 + 1/s < 0,
α−1 + β−1 = 1.
• C(X) = all continuous functions on conv(X).
• C∗(X) = all signed Radon measures on C(X) = dual space of C(X).
• G(X) = all closed convex (lower s.c.) functions on conv(X).
• G(X)◦ = {G ∈ C∗(X) :∫gdG ≤ 0 for all g ∈ G(X}, the polar
(or dual) cone of G(X).
Cowles Foundation Seminar, Yale, November 9, 2016 1.18
![Page 20: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/20.jpg)
Primal problems: P0 and Ps:
• P0: ming∈G(X)L0(g,Pn) where
L0(g,Pn) = Png +∫Rd
exp(−g(x))dx.
• Ps: ming∈G(X)Ls(g,Pn) where
Ls(g,Pn) = Png +1
|β|
∫Rdg(x)βdx.
Cowles Foundation Seminar, Yale, November 9, 2016 1.19
![Page 21: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/21.jpg)
Dual problems: P0 and Ps:
• D0: maxf {−∫f(y)logf(y)dy} subject to
f(y) =d(Pn −G)
dyfor some G ∈ G(X)◦.
• Ds: maxf∫ f(y)α
α dy subject to
f(y) =d(Pn −G)
dyfor some G ∈ G(X)◦.
Cowles Foundation Seminar, Yale, November 9, 2016 1.20
![Page 22: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/22.jpg)
Why do these make sense?
• Population version of P0: ming∈G L0(g, f0) where
L0(g, f0) =∫{g(x)f0(x) + e−g(x)}dx.
Minimizing the integrand pointwise in g = g(x) for fixed f0(x)
yields f0(x)− e−g(x) = 0 if e−g(x) = f0(x).
• Population version of Ps: ming∈G Ls(g, f0) where
Ls(g, f0) =∫{g(x)f0(x) +
1
|β|gβ(x)}dx.
Minimizing the integrand pointwise in g = g(x) for fixed f0(x)
yields f0(x)+(β/|β|)gβ−1(x) = f0(x)−gβ−1(x) = 0, and hence
g1/s = g1/s(x) = f0(x).
Cowles Foundation Seminar, Yale, November 9, 2016 1.21
![Page 23: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/23.jpg)
1. Basics for the Renyi divergence estimators:
• (Koenker and Mizera, 2010) If conv(X) has non-empty
interior, then strong duality between Ps and Ds holds. The
dual optimal optimal solution exists, is unique, and fn = g1/sn .
• (Koenker and Mizera, 2010) The solution f = g1/s in the
population version of the problem when Q = P0 has density
p0 ∈ Ps is Fisher-consistent; i.e. f = p0.
Cowles Foundation Seminar, Yale, November 9, 2016 1.22
![Page 24: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/24.jpg)
2. Off the model: Han & W (2015)
Let
Q1 ≡ {Q on (Rd,Bd) :∫‖x‖dQ(x) <∞},
Q0 ≡ {Q on (Rd,Bd) : int(csupp(Q)) 6= ∅}.
• Theorem (Han & W, 2015): If −1/(d + 1) < s < 0 and
Q ∈ Q0 ∩ Q1, then the primal problem Ps(Q) has a unique
solution g ∈ G which satisfies f = g1/s where g is bounded
away from 0 and f is a bounded density.
• Theorem (Han & W, 2015): Let d = 1. If fn,s denotes
the solution to the primal problem Ps and fn,0 denotes the
solution to the primal problem P0, then for any κ > 0, p ≥ 1,∫(1 + |x|)κ|fn,s(x)− fn,0(x)|pdx→ 0 as s↗ 0.
Cowles Foundation Seminar, Yale, November 9, 2016 1.23
![Page 25: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/25.jpg)
• Theorem (Han & W, 2015): Suppose that:
(i) d ≥ 1,
(ii) −1/(d+ 1) < s < 0, and
(iii) Q ∈ Q0 ∩Q1.
If fQ,s denotes the (pseudo-true) solution to the primal
problem Ps(Q), then for any κ < r − d = (−1/s)− d,∫(1 + |x|)κ|fn,s(x)− fQ,s(x)|dx→a.s. 0 as n→∞.
Cowles Foundation Seminar, Yale, November 9, 2016 1.24
![Page 26: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/26.jpg)
3. On the model: Q has density f ∈ Ps′; f = g1/s′ for some g
convex.
• Consistency: Suppose that: (i) d ≥ 1 and −1/d < s < 0 and
s′ > s if s ≤ −1/(d + 1), s′ = s if s > −1/(d + 1). Then for
any κ < r − d = (−1/s)− d,∫(1 + |x|)κ|fn,s(x)− f(x)|dx→a.s. 0 as n→∞.
Thus H(fn,s, f)→a.s. 0 as well.
• Pointwise limit theory: (paralleling the results of Balabdaoui,
Rufibach, and W (2009) for s = 0)
Cowles Foundation Seminar, Yale, November 9, 2016 1.25
![Page 27: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/27.jpg)
Assumptions:
• (A1) g0 ∈ G and f0 ∈ Ps(R) with −1/2 < s < 0.
• (A2) f0(x0) > 0.
• (A3) g0 is locally C2 in a neighborhood of x0 with
g0′′(x0) > 0.
Theorem 1. (Pointwise limit theorem; Han & W (2016))
Under assumptions (A1)-(A3), we have
n25(gn(x0)− g0(x0)
)n
15(g′n(x0)− g′0(x0)
)→d
−(g4
0(x0)g(2)0 (x0)
r4f0(x0)2(4)!
)1/5
H(2)2 (0)
−(g2
0(x0)[g
(2)0 (x0)
]3r2f0(x0)3
[(4)!
]3)1/5
H(3)2 (0)
,
and . . .
Cowles Foundation Seminar, Yale, November 9, 2016 1.26
![Page 28: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/28.jpg)
. . . furthermore
n25(fn(x0)− f0(x0)
)n
15(f ′n(x0)− f ′0(x0)
)→d
(rf0(x0)3g
(2)0 (x0)
g0(x0)(4)!
)1/5
H(2)2 (0)(
r3f0(x0)4(g
(2)0 (x0)
)3
g0(x0)3[(4)!
]3)1/5
H(3)2 (0)
,
where H2 is the unique lower envelope of the process Y2 satisfying
1. H2(t) ≤ Y2(t) for all t ∈ R;
2. H(2)k is concave;
3. H2(t) = Y2(t) if the slope of H(2)2 decreases strictly at t.
4. Y2(t) =∫ t0W (s)ds− t4, t ∈ R where W is two-sided Brownian
motion started at 0.
Cowles Foundation Seminar, Yale, November 9, 2016 1.27
![Page 29: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/29.jpg)
• Estimation of the mode for d = 1.
Theorem 2. (Estimation of the mode) Assume (A1)-(A4)
hold. Then
n1/5(mn −m0
)→d
(g0(m0)2(4)!2
r2f0(m0)g(2)0 (m0)2
)1/5
M(H(2)2 ), (1)
where mn = M(fn),m0 = M(f0).
• What is the price of assuming s < 0 when the truth f ∈ P0?
Assume −1/2 < s < 0 and k = 2. Let f0 = exp(ϕ0) be a log-
concave density where ϕ0 : R → R is the underlying concave
function. Then f0 is also s-concave.
Cowles Foundation Seminar, Yale, November 9, 2016 1.28
![Page 30: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/30.jpg)
Let gs := f−1/r0 = exp(−ϕ0/r) be the underlying convex function
when f0 is viewed as an s-concave density. Calculation yields
g(2)s (x0) =
1
r2gs(x0)
(ϕ′0(x0)2 − rϕ′′0(x0)
).
Hence the constant before H(2)2 (0) appearing in the limit
distribution for fn becomes(f0(x0)3ϕ′0(x0)2
4!r+f0(x0)3|ϕ′′0(x0)|
4!
)1/5
.
The second term is the constant involved in the limiting
distribution when f0(x0) is estimated via the log-concave MLE:
(2.2), page 1305 in Balabdaoui, Rufibach, & W (2009). The
ratio of the two constants (or asymptotic relative efficiency) is
shown for f0 standard normal (blue) and logistic (magenta) in
the figure:
Cowles Foundation Seminar, Yale, November 9, 2016 1.29
![Page 31: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/31.jpg)
-4 -2 0 2 4
0.2
0.4
0.6
0.8
1.0
Efficiency ratio: f0 = N(0,1), blue; f0 =logistic, magenta
Cowles Foundation Seminar, Yale, November 9, 2016 1.30
![Page 32: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/32.jpg)
• The first term is non-negative and is the price we pay by es-
timating a true log-concave density via the Renyi divergence
estimator over a larger class of s-concave densities.
• Note that the first term vanishes as r →∞ (or s↗ 0).
• Note that the ratio is 1 at the mode of f0.
• For estimation of the mode, the ratio of constants is always
1: nothing is lost by enlarging the class from s = 0 to s < 0!
Cowles Foundation Seminar, Yale, November 9, 2016 1.31
![Page 33: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/33.jpg)
E. Summary: problems and open questions
• Global rates of convergence?
• Limiting distribution(s) for d > 1? (nr with r = 2/(4 + d)?)
• MLE (rate-) inefficient for d ≥ 4 (or perhaps d ≥ 3)? How to
penalize to get efficient rates?
• Can we go below s = −1/(d+ 1) with other methods?
• Multivariate classes with nice preservation/closure properties
and smoother than log-concave?
• Algorithms for computing fn ∈ Ps? (Non-smooth and
convex; or non-smooth and non-convex?)
• Related results for convex regression on Rd: Seijo and Sen,
Ann. Statist. (2011).
Cowles Foundation Seminar, Yale, November 9, 2016 1.32
![Page 34: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/34.jpg)
Guvenen et al (2014)
have estimated models of income dynamics using very large (10percent) samples of U.S. Social Security records linked to W2data The density is not log-concave, but an s−concave densitywith s = −1/2 fits well:
−4 −2 0 2 4
−8
−6
−4
−2
0
x ~ log income annual increments
log
f(x)
−4 −2 0 2 4
−12
0−
80−
60−
40−
200
x ~ log income annual increments
−1
f(x)
−2 0 2
01
23
x ~ log income annual increments
f(x)
Courtesy Roger Koenker
Cowles Foundation Seminar, Yale, November 9, 2016 1.33
![Page 35: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/35.jpg)
F. Selected references
• Dumbgen and Rufibach (2009).
• Cule, Samworth, and Stewart (2010)
• Cule and Samworth (2010).
• Dumbgen, Samworth, and Schuhmacher (2011).
• Balabdaoui, Rufibach, and W (2009)
• Seregin & W (2010), Ann. Statist.
• Koenker and Mizera (2010), Ann. Statist.
• Han & W (2016): Ann. Statist., & arXiv:1505.00379v3.
• Doss & W (2016): Ann. Statist., & arXiv:1306.1438v2.
• Guntuboyina & Sen (2015): Prob. Theory & Rel. Fields.
• Chatterjee, Guntuboyina, & Sen (2015): Ann. Statist.
Cowles Foundation Seminar, Yale, November 9, 2016 1.34
![Page 36: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/36.jpg)
Many thanks!
Cowles Foundation Seminar, Yale, November 9, 2016 1.35
![Page 37: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/37.jpg)
Cowles Foundation Seminar, Yale, November 9, 2016 1.36
![Page 38: Nonparametric estimation of - Department of …economics.yale.edu/sites/default/files/wellner_metrics...Nonparametric estimation of s concave and log-concave densities: alternatives](https://reader036.vdocuments.us/reader036/viewer/2022062605/5fcb56bc3ca631338b4b5314/html5/thumbnails/38.jpg)
Cowles Foundation Seminar, Yale, November 9, 2016 1.37