robust estimation in parameter learning
TRANSCRIPT
![Page 1: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/1.jpg)
RobustEstimationinParameterLearning
SimonsInstituteBootcamp Tutorial,Part2
AnkurMoitra(MIT)
![Page 2: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/2.jpg)
CLASSICPARAMETERLEARNINGGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters?
![Page 3: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/3.jpg)
CLASSICPARAMETERLEARNINGGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters? Yes!
![Page 4: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/4.jpg)
CLASSICPARAMETERLEARNINGGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters?
empiricalmean: empiricalvariance:
Yes!
![Page 5: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/5.jpg)
Themaximumlikelihoodestimatorisasymptoticallyefficient(1910-1920)
R.A.Fisher
![Page 6: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/6.jpg)
Themaximumlikelihoodestimatorisasymptoticallyefficient(1910-1920)
R.A.Fisher J.W.Tukey
Whatabouterrors inthemodelitself?(1960)
![Page 7: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/7.jpg)
ROBUSTPARAMETERLEARNINGGivencorrupted samplesfroma1-DGaussian:
canweaccuratelyestimateitsparameters?
=+idealmodel noise observedmodel
![Page 8: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/8.jpg)
Howdoweconstrainthenoise?
![Page 9: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/9.jpg)
Howdoweconstrainthenoise?
Equivalently:
L1-normofnoiseatmostO(ε)
![Page 10: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/10.jpg)
Howdoweconstrainthenoise?
Equivalently:
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
![Page 11: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/11.jpg)
Howdoweconstrainthenoise?
Equivalently:
ThisgeneralizesHuber’sContaminationModel:Anadversarycanadd anε-fractionofsamples
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
![Page 12: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/12.jpg)
Howdoweconstrainthenoise?
Equivalently:
ThisgeneralizesHuber’sContaminationModel:Anadversarycanadd anε-fractionofsamples
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
Outliers:Pointsadversaryhascorrupted,Inliers:Pointshehasn’t
![Page 13: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/13.jpg)
Inwhatnormdowewanttheparameterstobeclose?
![Page 14: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/14.jpg)
Inwhatnormdowewanttheparameterstobeclose?
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
![Page 15: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/15.jpg)
Inwhatnormdowewanttheparameterstobeclose?
FromtheboundontheL1-normofthenoise,wehave:
observedideal
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
![Page 16: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/16.jpg)
Inwhatnormdowewanttheparameterstobeclose?
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
estimate ideal
Goal:Finda1-DGaussianthatsatisfies
![Page 17: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/17.jpg)
Inwhatnormdowewanttheparameterstobeclose?
estimate observed
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
Equivalently,finda1-DGaussianthatsatisfies
![Page 18: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/18.jpg)
Dotheempiricalmeanandempiricalvariancework?
![Page 19: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/19.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
![Page 20: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/20.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
![Page 21: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/21.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
Butthemedian andmedianabsolutedeviationdowork
![Page 22: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/22.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
![Page 23: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/23.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
Alsocalled(properly)agnosticallylearninga1-DGaussian
![Page 24: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/24.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
Whataboutrobustestimationinhigh-dimensions?
![Page 25: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/25.jpg)
Whataboutrobustestimationinhigh-dimensions?
e.g.microarrayswith10kgenes
Fact[Folklore]:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
![Page 26: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/26.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 27: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/27.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 28: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/28.jpg)
MainProblem:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussian
giveanefficientalgorithmtofindparametersthatsatisfy
![Page 29: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/29.jpg)
MainProblem:Givensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussian
giveanefficientalgorithmtofindparametersthatsatisfy
SpecialCases:
(1)Unknownmean
(2)Unknowncovariance
![Page 30: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/30.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
UnknownMean
![Page 31: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/31.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
![Page 32: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/32.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε)
![Page 33: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/33.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
![Page 34: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/34.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian
![Page 35: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/35.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)
![Page 36: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/36.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
![Page 37: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/37.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
Tournament O(ε) NO(d)
![Page 38: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/38.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
Tournament O(ε) NO(d)
O(ε√d)Pruning O(dN)
![Page 39: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/39.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian O(ε) NP-Hard
GeometricMedian O(ε√d) poly(d,N)
Tournament O(ε) NO(d)
O(ε√d)Pruning O(dN)
UnknownMean
…
![Page 40: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/40.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
![Page 41: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/41.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
![Page 42: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/42.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
![Page 43: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/43.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
Isrobustestimationalgorithmicallypossibleinhigh-dimensions?
![Page 44: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/44.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 45: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/45.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 46: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/46.jpg)
RECENTRESULTS
Theorem[Diakonikolas,Li,Kamath,Kane,Moitra,Stewart‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Robustestimationishigh-dimensionsisalgorithmicallypossible!
Moreoverthealgorithmrunsintimepoly(N,d)
![Page 47: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/47.jpg)
RECENTRESULTS
Theorem[Diakonikolas,Li,Kamath,Kane,Moitra,Stewart‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Robustestimationishigh-dimensionsisalgorithmicallypossible!
Moreoverthealgorithmrunsintimepoly(N,d)
Extensions:Canweakenassumptionstosub-Gaussianorboundedsecondmoments(withweakerguarantees)forthemean
![Page 48: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/48.jpg)
Independentlyandconcurrently:
Theorem[Lai,Rao,Vempala ‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotal
variationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Moreoverthealgorithmrunsintimepoly(N,d)
![Page 49: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/49.jpg)
Independentlyandconcurrently:
Theorem[Lai,Rao,Vempala ‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotal
variationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Moreoverthealgorithmrunsintimepoly(N,d)
Whenthecovarianceisbounded,thistranslatesto:
![Page 50: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/50.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
![Page 51: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/51.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
Let’sseehowthisworksforunknownmean…
![Page 52: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/52.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 53: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/53.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 54: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/54.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
![Page 55: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/55.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
![Page 56: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/56.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
ThiscanbeprovenusingPinsker’s Inequality
andthewell-knownformulaforKL-divergencebetweenGaussians
![Page 57: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/57.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
![Page 58: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/58.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
Corollary:Ifourestimate(intheunknownmeancase)satisfies
then
![Page 59: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/59.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
Corollary:Ifourestimate(intheunknownmeancase)satisfies
then
OurnewgoalistobecloseinEuclideandistance
![Page 60: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/60.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 61: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/61.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 62: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/62.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
![Page 63: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/63.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
=uncorrupted=corrupted
![Page 64: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/64.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
=uncorrupted=corrupted
Thereisadirectionoflarge(>1)variance
![Page 65: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/65.jpg)
KeyLemma:IfX1,X2,…XN comefromadistributionthatisε-closetoandthenfor
(1) (2)
withprobabilityatleast1-δ
![Page 66: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/66.jpg)
KeyLemma:IfX1,X2,…XN comefromadistributionthatisε-closetoandthenfor
(1) (2)
withprobabilityatleast1-δ
Take-away:Anadversaryneedstomessupthesecondmomentinordertocorruptthefirstmoment
![Page 67: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/67.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 68: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/68.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 69: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/69.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
![Page 70: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/70.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
![Page 71: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/71.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
wherevisthedirectionoflargestvariance
![Page 72: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/72.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
wherevisthedirectionoflargestvariance,andThasaformula
![Page 73: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/73.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
T
wherevisthedirectionoflargestvariance,andThasaformula
![Page 74: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/74.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
![Page 75: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/75.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
![Page 76: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/76.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
![Page 77: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/77.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
RunningTime: SampleComplexity:
![Page 78: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/78.jpg)
AWIN-WINALGORITHM
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
RunningTime: SampleComplexity:ConcentrationofLTFs
![Page 79: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/79.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 80: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/80.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 81: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/81.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
![Page 82: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/82.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
Howaboutforunknowncovariance?
![Page 83: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/83.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
![Page 84: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/84.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
(2)
![Page 85: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/85.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
![Page 86: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/86.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
Ournewgoalistofindanestimatethatsatisfies:
![Page 87: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/87.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
Ournewgoalistofindanestimatethatsatisfies:
Distanceseemsstrange,butit’stherightonetousetoboundTV
![Page 88: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/88.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
![Page 89: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/89.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
![Page 90: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/90.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
![Page 91: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/91.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
ProofusesIsserlis’s Theorem
![Page 92: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/92.jpg)
UNKNOWNCOVARIANCE
needtoprojectout
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
![Page 93: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/93.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
![Page 94: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/94.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
![Page 95: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/95.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers
![Page 96: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/96.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers,inwhichcase:
wouldhavesmallrestrictedeigenvalues
![Page 97: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/97.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers,inwhichcase:
wouldhavesmallrestrictedeigenvalues
Take-away:Anadversaryneedstomessupthe(restricted)fourthmomentinordertocorruptthesecondmoment
![Page 98: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/98.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
![Page 99: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/99.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
![Page 100: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/100.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
![Page 101: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/101.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
![Page 102: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/102.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
rightdistance,ingeneralcase
![Page 103: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/103.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
Nowusealgorithmforunknownmeanrightdistance,ingeneralcase
![Page 104: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/104.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 105: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/105.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� RecentResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� AWin-WinAlgorithm� UnknownCovariance
OUTLINE
PartIII:FurtherResults
![Page 106: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/106.jpg)
LIMITSTOROBUSTESTIMATION
Theorem[Diakonikolas,Kane,Stewart‘16]:Anystatisticalquerylearning* algorithminthestrongcorruptionmodel
thatmakeserrormustmakeatleastqueries
insertionsanddeletions
![Page 107: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/107.jpg)
LIMITSTOROBUSTESTIMATION
Theorem[Diakonikolas,Kane,Stewart‘16]:Anystatisticalquerylearning* algorithminthestrongcorruptionmodel
thatmakeserrormustmakeatleastqueries
*Insteadofseeingsamplesdirectly,analgorithmqueriesafnctn
andgetsexpectation,uptosamplingnoise
insertionsanddeletions
![Page 108: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/108.jpg)
LISTDECODING
Whatifanadversarycancorruptthemajority ofsamples?
![Page 109: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/109.jpg)
LISTDECODING
Whatifanadversarycancorruptthemajority ofsamples?
Thisextendstomixturesstraightforwardly
Theorem[Charikar,Steinhardt,Valiant‘17]:Givensamplesfromadistributionwithmeanandcovariancewherehavebeencorrupted,thereisanalgorithmthatoutputs
with thatsatisfies
![Page 110: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/110.jpg)
LISTDECODING
Whatifanadversarycancorruptthemajority ofsamples?
Thisextendstomixturesstraightforwardly
Theorem[Charikar,Steinhardt,Valiant‘17]:Givensamplesfromadistributionwithmeanandcovariancewherehavebeencorrupted,thereisanalgorithmthatoutputs
with thatsatisfies
[Kothari,Steinhardt‘18],[Diakonikolas etal’18] gaveimprovedguarantees,butunderGaussianity
![Page 111: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/111.jpg)
BEYONDGAUSSIANS
Canwerelaxthedistributionalassumptions?
![Page 112: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/112.jpg)
BEYONDGAUSSIANS
Theorem[Kothari,Steurer ‘18] [Hopkins,Li’18]:Givenε-corruptedsamplesfromak-certifiablysubgaussian distributionthereisanalgorithmthatoutputs
Canwerelaxthedistributionalassumptions?
![Page 113: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/113.jpg)
BEYONDGAUSSIANS
Theorem[Kothari,Steurer ‘18] [Hopkins,Li’18]:Givenε-corruptedsamplesfromak-certifiablysubgaussian distributionthereisanalgorithmthatoutputs
Canwerelaxthedistributionalassumptions?
Whenyouonlyknowboundsonthemoments,theseguaranteesareoptimal
![Page 114: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/114.jpg)
SUBGAUSSIAN CONFIDENCEINTERVALS
Estimatingthemeanaccuratelywithheavytaileddistributions?
![Page 115: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/115.jpg)
SUBGAUSSIAN CONFIDENCEINTERVALS
Estimatingthemeanaccuratelywithheavytaileddistributions?
Theorem[Hopkins‘18]:Givenniid samplesfromadistributionwithmeanandcovarianceandtargetconfidence,thereisapolynomialtimealgorithmthatoutputssatisfying
![Page 116: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/116.jpg)
SUBGAUSSIAN CONFIDENCEINTERVALS
Estimatingthemeanaccuratelywithheavytaileddistributions?
Theorem[Hopkins‘18]:Givenniid samplesfromadistributionwithmeanandcovarianceandtargetconfidence,thereisapolynomialtimealgorithmthatoutputssatisfying
Theempiricalmeandoesn’twork,andmedian-of-meansestimatordueto [Lugosi,Mendelson ‘18]ishardtocompute
![Page 117: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/117.jpg)
SUBGAUSSIAN CONFIDENCEINTERVALS
Estimatingthemeanaccuratelywithheavytaileddistributions?
Theorem[Hopkins‘18]:Givenniid samplesfromadistributionwithmeanandcovarianceandtargetconfidence,thereisapolynomialtimealgorithmthatoutputssatisfying
Theempiricalmeandoesn’twork,andmedian-of-meansestimatordueto [Lugosi,Mendelson ‘18]ishardtocompute
[Cherapanamjeri,Flammarion,Bartlett‘19]gavefasteralgorithmsbasedongradientdescent
![Page 118: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/118.jpg)
Summary:� Nearlyoptimalalgorithmforagnosticallylearningahigh-dimensionalGaussian
� Generalrecipeusingrestrictedeigenvalueproblems� Furtherapplicationstoothermixturemodels�What’snextforalgorithmicrobuststatistics?
![Page 119: Robust Estimation in Parameter Learning](https://reader030.vdocuments.us/reader030/viewer/2022011918/61d821bf9d7e1655d162ba99/html5/thumbnails/119.jpg)
Thanks!AnyQuestions?
Summary:� Nearlyoptimalalgorithmforagnosticallylearningahigh-dimensionalGaussian
� Generalrecipeusingrestrictedeigenvalueproblems� Furtherapplicationstoothermixturemodels�What’snextforalgorithmicrobuststatistics?