Download - Houman Owhadi - SIAM
![Page 1: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/1.jpg)
Houman OwhadiThe worst case approach to UQ
“The gods to-day stand friendly, that we may,Lovers of peace, lead on our days to age!But, since the affairs of men rests still uncertain,Let’s reason with the worst that may befall.”
Julius Caesar, Act 5, Scene 1William Shakespeare (1564 –1616)
![Page 2: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/2.jpg)
You want to certify that
Problem
and
![Page 3: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/3.jpg)
You want to certify that
Problem
and
You only know
![Page 4: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/4.jpg)
Worst and best caseoptimal bounds P[G(X) ≥ a]given available information.
Compute
![Page 5: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/5.jpg)
I. Elishakoff and M. Ohsaki. Optimization and Anti-Optimization of Structures Under Uncertainty. World Scientific, London, 2010.
Saltelli, A.; Ratto, M.; Andres, T.; Campolongo, F.; Cariboni, J.; Gatelli, D.; Saisana, M.; Tarantola, S. (2008). Global Sensitivity Analysis: The Primer. John Wiley & Sons.
Global Sensitivity Analysis
A. Ben-Tal, L. El Ghaoui, and A.Nemirovski. Robust optimization. PrincetonSeries in Applied Mathematics. Princeton University Press, Princeton, NJ, 2009
D. Bertsimas, D. B. Brown, and C. Caramanis. Theory and applications of robustoptimization. SIAM Rev., 53(3):464–501, 2011
Robust Optimization
A. Ben-Tal and A. Nemirovski. Robust convex optimization. Math. Oper. Res.,23(4):769–805, 1998
Optimal Uncertainty QuantificationH. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
![Page 6: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/6.jpg)
Set based design in the aerospace industry
Bernstein JI (1998) Design methods in the aerospace industry: looking for evidence of set-based practices. Master’s thesis. Massachusetts Institute of Technology, Cambridge, MA.
Set based design/analysis
B. Rustem and Howe M. Algorithms for Worst-Case Design and Applications toRisk Management. Princeton University Press, Princeton, 2002.
David J. Singer, PhD., Captain Norbert Doerry, PhD., and Michael E. Buckley,” What is Set-Based Design? ,” Presented at ASNE DAY 2009, National Harbor, MD., April 8-9, 2009. Also published in ASNE Naval Engineers Journal, 2009 Vol 121 No 4, pp. 31-43.
![Page 7: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/7.jpg)
![Page 8: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/8.jpg)
P. L. Chebyshev1821-1894
M. G. Krein1907-1989
A. A. Markov1856-1922
![Page 9: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/9.jpg)
Answer
![Page 10: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/10.jpg)
Answer
Markov’s inequality
![Page 11: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/11.jpg)
History of classical inequalitiesS. Karlin and W. J. Studden. Tchebycheff Systems: With Applications in Analysisand Statistics. Pure and Applied Mathematics, Vol. XV. Interscience PublishersJohn Wiley & Sons, New York-London-Sydney, 1966.
Theory of majorization
A. W. Marshall and I. Olkin. Inequalities: Theory of Majorization and its Appli-cations, volume 143 of Mathematics in Science and Engineering. Academic PressInc. [Harcourt Brace Jovanovich Publishers], New York, 1979
M. G. Krein. The ideas of P. L. Cebysev and A. A. Markov in the theory of limitingvalues of integrals and their further development. In E. B. Dynkin, editor, Elevenpapers on Analysis, Probability, and Topology, American Mathematical SocietyTranslations, Series 2, Volume 12, pages 1–122. American Mathematical Society,New York, 1959.
Classical Markov-Krein theorem and classical works of Krein, Markov and Chebyshev
![Page 12: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/12.jpg)
Connections between Chebyshev inequalities and optimization theoryH. J. Godwin. On generalizations of Tchebychef’s inequality. J. Amer. Statist.Assoc., 50(271):923–945, 1955.
K. Isii. On a method for generalizations of Tchebycheff’s inequality. Ann. Inst.Statist. Math. Tokyo, 10(2):65–88, 1959.
K. Isii. On sharpness of Tchebycheff-type inequalities. Ann. Inst. Statist. Math.,14(1):185–197, 1962/1963.
A. W. Marshall and I. Olkin. Multivariate Chebyshev inequalities. Ann. Math.Statist., 31(4):1001–1014, 1960.
A. W. Marshall and I. Olkin. Inequalities: Theory of Majorization and its Appli-cations, volume 143 of Mathematics in Science and Engineering. Academic PressInc. [Harcourt Brace Jovanovich Publishers], New York, 1979
I. Olkin and J. W. Pratt. A multivariate Tchebycheff inequality. Ann. Math.Statist, 29(1):226–234, 1958.
K. Isii. The extrema of probability determined by generalized moments. I. Boundedrandom variables. Ann. Inst. Statist. Math., 12(2):119–134; errata, 280, 1960.
![Page 13: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/13.jpg)
L. Vandenberghe, S. Boyd, and K. Comanor. Generalized Chebyshev bounds viasemidefinite programming. SIAM Rev., 49(1):52–64 (electronic), 2007
Connection between Chebyshev inequalities and optimization theory
H. Joe. Majorization, randomness and dependence for multivariate distributions.Ann. Probab., 15(3):1217–1225, 1987.
J. E. Smith. Generalized Chebychev inequalities: theory and applications in decisionanalysis. Oper. Res., 43(5):807–825, 1995.
D. Bertsimas and I. Popescu. Optimal inequalities in probability theory: a convexoptimization approach. SIAM J. Optim., 15(3):780–804 (electronic), 2005.
E. B. Dynkin. Sufficient statistics and extreme points. Ann. Prob., 6(5):705–730,1978.
A. F. Karr. Extreme points of certain sets of probability measures, with applications.Math. Oper. Res., 8(1):74–85, 1983.
![Page 14: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/14.jpg)
Stochastic linear programming and Stochastic Optimization
P. Kall. Stochastric programming with recourse: upper bounds and moment problems:a review. Mathematical research, 45:86–103, 1988
A. Madansky. Bounds on the expectation of a convex function of a multivariaterandom variable. The Annals of Mathematical Statistics, pages 743–746, 1959
A. Madansky. Inequalities for stochastic linear programming problems. Manage-ment science, 6(2):197–204, 1960.
G. B. Dantzig. Linear programming under uncertainty. Management Sci., 1:197–206, 1955.
J. R. Birge and R. J.-B. Wets. Designing approximation schemes for stochasticoptimization problems, in particular for stochastic programs with recourse. Math.Prog. Stud., 27:54–102, 1986
Y. Ermoliev, A. Gaivoronski, and C. Nedeva. Stochastic optimization problemswith incomplete information on distribution functions. SIAM Journal on Controland Optimization, 23(5):697–716, 1985
C. C. Huang, W. T. Ziemba, and A. Ben-Tal. Bounds on the expectation of a convexfunction of a random variable: With applications to stochastic programming.Operations Research, 25(2):315–325, 1977.
J. Zjackovja. On minimax solutions of stochastic linear programming problems.Casopis Pest. Mat., 91:423–430, 1966.
![Page 15: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/15.jpg)
Chance constrained/distributionally robust optimization
J. Goh and M. Sim. Distributionally robust optimization and its tractable approximations. Oper. Res., 58(4, part 1):902–917, 2010
R. I. Bot N. Lorenz, and G. Wanka. Duality for linear chance-constrained optimizationproblems. J. Korean Math. Soc., 47(1):17–28, 2010.
W. Wiesemann, D. Kuhn, and M. Sim. Distributionally robust convex optimization.Oper. Res., 62(6):1358–1376, 2014.
L. Xu, B. Yu, and W. Liu. The distributionally robust optimization reformulationfor stochastic complementarity problems. Abstr. Appl. Anal., pages 7, 2014.
S. Zymler, D. Kuhn, and B. Rustem. Distributionally robust joint chance constraintswith second-order moment information. Math. Program., 137(1-2, Ser.A):167–198, 2013.
A. A. Gaivoronski. A numerical method for solving stochastic programming problemswith moment constraints on a distribution function. Annals of OperationsResearch, 31(1):347–369, 1991.
G. A. Hanasusanto, V. Roitch, D. Kuhn, and W. Wiesemann. A distributionallyrobust perspective on uncertainty quantification and chance constrained programming.Mathematical Programming, 151(1):35–62, 2015
Value at Risk
Artzner, P.; Delbaen, F.; Eber, J. M.; Heath, D. (1999). Coherent Measures of Risk. Mathematical Finance 9 (3): 203.
W. Chen, M. Sim, J. Sun, and C.-P. Teo. From CVaR to uncertainty set: implicationsin joint chance-constrained optimization. Oper. Res., 58(2):470–485, 2010.
![Page 16: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/16.jpg)
Optimal Uncertainty Quantification
S. Han, M. Tao, U. Topcu, H. Owhadi, and R. M. Murray. Convex optimal uncertaintyquantification. SIAM Journal on Optimization, 25(23):1368–1387, 2015.
S. Han, U. Topcu, M. Tao, H. Owhadi, and R. Murray. Convex optimal uncertaintyquantification: Algorithms and a case study in energy storage placement for powergrids. In American Control Conference (ACC), 2013, pages 1130–1137. IEEE,2013
H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
T. J. Sullivan, M. McKerns, D. Meyer, F. Theil, H. Owhadi, and M. Ortiz. Optimaluncertainty quantification for legacy data observations of Lipschitz functions.ESAIM Math. Model. Numer. Anal., 47(6):1657–1689, 2013.
J Chen, MD Flood, R Sowers. Measuring the Unmeasurable: An Application of Uncertainty Quantification to Financial Portfolios, OFR WP, 2015
L Ming, W Chenglin. An improved algorithm for convex optimal uncertainty quantification with polytopic canonical form. Control Conference (CCC), 2015
H. Owhadi and Clint Scovel. Extreme points of a ball about a measure with finite support (2015). arXiv:1504.06745
H. Owhadi, C. Scovel and T. Sullivan. Brittleness of Bayesian Inference under Finite Information in a Continuous World. Electronic Journal of Statistics, vol 9, pp 1-79, 2015. arXiv:1304.6772
![Page 17: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/17.jpg)
Our proof relies on• Winkler (1988, Extreme points of moment sets)• Follows from an extension of Choquet theory (Phelps 2001, lectures on Choquet’s
theorem) by Von Weizsacker & Winkler (1979, Integral representation in the set ofsolutions of a generalized moment problem)
• Kendall (1962, Simplexes & Vector lattices)
G. Winkler. On the integral representation in convex noncompact sets of tightmeasures. Mathematische Zeitschrift, 158(1):71–77, 1978
G. Winkler. Extreme points of moment sets. Math. Oper. Res., 13(4):581–587,1988.
H. von Weizsacker and G. Winkler. Integral representation in the set of solutionsof a generalized moment problem. Math. Ann., 246(1):23–32, 1979/80.
D. G. Kendall. Simplexes and vector lattices. J. London Math. Soc., 37(1):365–371,1962.
![Page 18: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/18.jpg)
Theorem
H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
![Page 19: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/19.jpg)
Further Reduction of optimization variables
![Page 20: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/20.jpg)
McDiarmid inequality’s
Another example: Optimal concentration inequality H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
![Page 21: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/21.jpg)
Reduction of optimization variables
Theorem
![Page 22: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/22.jpg)
Theorem m = 2
C = (1, 1)hC(s) = a− (1− s1)D1 − (1− s2)D2
Explicit Solution m=2
![Page 23: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/23.jpg)
Theorem m = 2
Corollary
Explicit Solution m=2
![Page 24: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/24.jpg)
μ
f
A
Each piece of information is a constrainton an optimization problem.
Optimization concepts (binding, active) transfer to UQ concepts
Non bindingconstraint
Binding but nonactive constraint
Active constraint
Extremizer/Worst case scenario
![Page 25: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/25.jpg)
Optimal Hoeffding= Optimal McDiarmid for m=2
![Page 26: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/26.jpg)
Theorem m = 3Explicit Solution m=3
![Page 27: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/27.jpg)
F min( Yield Strain - Axial Strain )
Ground Acceleration
We want to certify that
Seismic Safety Assessment of a Truss Structure
![Page 28: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/28.jpg)
Power Spectrum
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0.1 0.6 1.1 1.6 2.1 2.6 3.1 3.6 4.1 4.6 5.1 5.6 6.1 6.6 7.1 7.6 8.1 8.6 9.1 9.6
Frequency
Am
plitu
de
Mean PSObserved PS
![Page 29: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/29.jpg)
Power Spectrum
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0.1 0.6 1.1 1.6 2.1 2.6 3.1 3.6 4.1 4.6 5.1 5.6 6.1 6.6 7.1 7.6 8.1 8.6 9.1 9.6
Frequency
Am
plitu
de
Mean PSObserved PS
![Page 30: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/30.jpg)
Power Spectrum
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0.1 0.6 1.1 1.6 2.1 2.6 3.1 3.6 4.1 4.6 5.1 5.6 6.1 6.6 7.1 7.6 8.1 8.6 9.1 9.6
Frequency
Am
plitu
de
Mean PSObserved PS
![Page 31: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/31.jpg)
Filtered White Noise Model
White noise
Ground acceleration
Filter0
0.2
0.4
0.6
0.8
1
1.2
1.4
0.1 0.6 1.1 1.6 2.1 2.6 3.1 3.6 4.1 4.6 5.1 5.6 6.1 6.6 7.1 7.6 8.1 8.6 9.1 9.6
Frequency
Am
plitu
de
Mean PSObserved PS
N. Lama, J. Wilsona, and G. Hutchinsona. Generation of synthetic earthquake accelogramsusing seismological modeling: a review. Journal of Earthquake Engineering, 4(3):321–354, 2000.
![Page 32: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/32.jpg)
Vulnerability Curves (vs earthquake magnitude)
![Page 33: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/33.jpg)
Identification of the weakest elements
H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
![Page 34: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/34.jpg)
Caltech Small Particle Hypervelocity Impact Range
G
Projectile velocity
Plate thickness
Plate Obliquity
Perforation area
We want to certify that
Problem
![Page 35: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/35.jpg)
What do we know?
Projectile velocity
Plate thickness
Plate Obliquity
Thickness, obliquity, velocity: independent random variables
Mean perforation area: in between 5.5 and 7.5 mm^2
Bounds on the sensitivity of the response function w.r. to each variable
![Page 36: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/36.jpg)
We only know
Worst case bound
Reduction calculus
![Page 37: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/37.jpg)
What if we know the response function?
Deterministic surrogate model for the perforation area (in mm^2)
![Page 38: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/38.jpg)
Optimal bound on the probability of non perforation
The measure of probability can be reduced to the tensorization of2 Dirac masses on thickness, obliquity and velocity
Application of the reduction calculus
![Page 39: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/39.jpg)
The optimization variables can be reduced to the tensorizationof 2 Dirac masses on thickness, obliquity and velocity
Support Points at iteration 0
![Page 40: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/40.jpg)
Numerical optimization
Support Points at iteration 150
![Page 41: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/41.jpg)
Numerical optimization
Support Points at iteration 200
![Page 42: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/42.jpg)
Velocity and obliquity marginals each collapse to a single Dirac mass. The plate thickness marginal collapses to have support on the extremes of its range.
Iteration1000
Probability non-perforation maximized by distribution supported on minimal, not maximal, impact obliquity. Dirac on velocity at a non extreme value.
![Page 43: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/43.jpg)
Important observations
Extremizers are singular
They identify key playersi.e. vulnerabilities of the physical system
Extremizers are attractors
![Page 44: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/44.jpg)
Initialization with 3 support points per marginal
Support Points at iteration 0
![Page 45: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/45.jpg)
Initialization with 3 support points per marginal
Support Points at iteration 500
![Page 46: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/46.jpg)
Initialization with 3 support points per marginal
Support Points at iteration 1000
![Page 47: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/47.jpg)
Initialization with 3 support points per marginal
Support Points at iteration 2155
![Page 48: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/48.jpg)
Initialization with 5 support points per marginal
Support Points at iteration 0
![Page 49: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/49.jpg)
Initialization with 5 support points per marginal
Support Points at iteration 1000
![Page 50: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/50.jpg)
Initialization with 5 support points per marginal
Support Points at iteration 3000
![Page 51: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/51.jpg)
Initialization with 5 support points per marginal
Support Points at iteration 7100
![Page 52: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/52.jpg)
Unknown response function G + Legacy data
Constraint on the mean perf. area
Modified Lipschitz continuity constraints on response function
Objective
Constraints on input variables
![Page 53: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/53.jpg)
Legacy Data
32 data points(steel-on-aluminium shots A48–A81) from summer 2010 at Caltech’s SPHIR facility:
These constrain the value of G at 32 points
T. J. Sullivan, M. McKerns, D. Meyer, F. Theil, H. Owhadi, and M. Ortiz. Optimal uncertainty quantification for legacy data observations of Lipschitz functions. ESAIM Math. Model. Numer. Anal., 47(6):1657–1689, 2013.
![Page 54: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/54.jpg)
The numerical results demonstrate agreement with the Markov bound
Only 2 data points out of 32 carry information about the optimal bound
![Page 55: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/55.jpg)
Legacy Data
32 data points(steel-on-aluminium shots A48–A81) from summer 2010 at Caltech’s SPHIR facility:
Only A54 and A67 carry information
The other 30 data points carry noinformation about least upper boundand could have be ignored.
T. J. Sullivan, M. McKerns, D. Meyer, F. Theil, H. Owhadi, and M. Ortiz. Optimal uncertainty quantification for legacy data observations of Lipschitz functions. ESAIM Math. Model. Numer. Anal., 47(6):1657–1689, 2013.
![Page 56: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/56.jpg)
What if we have model uncertainty?
What do we want?
What do we know?
![Page 57: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/57.jpg)
PSAAP numerical model
F
Plate thickness Perforation areaProjectile velocity
Plate Obliquity=0
v
F
4.5 km/s 7 km/s0mm2
30mm2
![Page 58: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/58.jpg)
59 data/experimental points
G
Plate thickness Perforation areaProjectile velocity
Plate Obliquity=0
v
F
4.5 km/s 7 km/s0mm2
30mm2
![Page 59: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/59.jpg)
Confidence sausage around the model
(h, v)
PerforationArea
F
Cy
![Page 60: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/60.jpg)
Admissible set
What we compute
Confidence sausage
![Page 61: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/61.jpg)
![Page 62: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/62.jpg)
![Page 63: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/63.jpg)
v
F
4.5 km/s 7 km/s0mm2
30mm2
![Page 64: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/64.jpg)
The extremizers led to the identification of a bug in an old model
![Page 65: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/65.jpg)
Caltech PSAAP Center UQ analysis
P.-H. T. Kamga, B. Li, M. McKerns, L. H. Nguyen, M. Ortiz, H. Owhadi, andT. J. Sullivan. Optimal uncertainty quantification with model uncertainty andlegacy data. Journal of the Mechanics and Physics of Solids, 72:1–19, 2014
A. A. Kidane, A. Lashgari, B. Li, M. McKerns, M. Ortiz, H. Owhadi, G. Ravichan-dran, M. Stalzer, and T. J. Sullivan. Rigorous model-based uncertainty quantificationwith application to terminal ballistics. Part I: Systems with controllable inputs andsmall scatter. Journal of the Mechanics and Physics of Solids, 60(5):983–1001,2012.
M. M. McKerns, L. Strand, T. J. Sullivan, A. Fang, and M. A. G. Aivazis. Buildinga framework for predictive science. In Proceedings of the 10th Python in ScienceConference (SciPy 2011), 2011.
L. J. Lucas, H. Owhadi, and M. Ortiz. Rigorous verification, validation, uncertaintyquantification and certification through concentration-of-measure inequalities.Comput. Methods Appl. Mech. Engrg., 197(51-52):4591–4609, 2008.
H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal Uncertainty Quantification. SIAM Review, 55(2):271–345, 2013.
T. J. Sullivan, M. McKerns, D. Meyer, F. Theil, H. Owhadi, and M. Ortiz. Optimal uncertainty quantification for legacy data observations of Lipschitz functions. ESAIM Math. Model. Numer. Anal., 47(6):1657–1689, 2013.
![Page 66: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/66.jpg)
Reduced numerical optimization problems solved using
• mystic: http://trac.mystic.cacr.caltech.edu/project/mystic– a highly-configurable optimization framework
• pathos: http://trac.mystic.cacr.caltech.edu/project/pathos– a distributed parallel graph execution framework providing a high-
level programmatic interface to heterogeneous computing
Mike McKerns
![Page 67: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/67.jpg)
Important observations
In presence of incomplete information on the distribution of input variables the dependence of the least upper bound on the accuracy of the model is very weak
We need to extract as much information as possible from the sample/experimental data on the underlying distributions
How do we reason with the worst in presence of data sampled from an unknown distribution?
![Page 68: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/68.jpg)
Quantity of Interest
You observe
You know μ† ∈ A
Problem:
θ(d)
![Page 69: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/69.jpg)
Player I Player II
Chooses θ
Mean squared error
Confidence error
Max Min
![Page 70: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/70.jpg)
Game theory and statistical decision theory
John Von Neumann Abraham Wald
J. Von Neumann. Zur Theorie der Gesellschaftsspiele. Math. Ann., 100(1):295–320,1928
J. Von Neumann and O. Morgenstern. Theory of Games and Economic Behavior.Princeton University Press, Princeton, New Jersey, 1944.
A. Wald. Contributions to the theory of statistical estimation and testing hypotheses.Ann. Math. Statist., 10(4):299–326, 1939.
A. Wald. Statistical decision functions which minimize the maximum risk. Ann.of Math. (2), 46:265–280, 1945.
A. Wald. An essentially complete class of admissible decision functions. Ann.Math. Statistics, 18:549–555, 1947.
A. Wald. Statistical decision functions. Ann. Math. Statistics, 20:165–205, 1949.
![Page 71: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/71.jpg)
Player I
Player II
3
1
-2
-2
Deterministic zero sum game
Player I’s payoff
Player I & II both have a blue and a red marbleAt the same time, they show each other a marble
How should I & II play the game?
![Page 72: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/72.jpg)
Pure strategy solution
3
1
-2
-2II should play blue and loose 1 in the worst case
I should play red and loose 2 in the worst case
![Page 73: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/73.jpg)
Mixed strategy (repeated game) solution
3
1
-2
-2II should play red with probability 3/8 and win 1/8 on average
I should play red with probability 3/8 and loose 1/8 on average
![Page 74: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/74.jpg)
3
1
-2
-2 Player I’s payoff
J. Von Neumann
![Page 75: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/75.jpg)
Max Min
Optimal bound on the statistical errormaxμ∈A
E(μ, θ)
Optimal statistical estimatorsminθ
maxμ∈A
E(μ, θ)
Pure strategy solution for Player II
![Page 76: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/76.jpg)
Max Min
Mixed strategy (repeated game) solution for Player II
Mixed strategy (repeated game) solution for Player I
![Page 77: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/77.jpg)
Theorem
Can we have equality?
![Page 78: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/78.jpg)
Theorem
The best mixed strategy for I and II = worst prior for II
A. Dvoretzky, A. Wald, and J. Wolfowitz. Elimination of randomization in certainstatistical decision procedures and zero-sum two-person games. Ann. Math.Statist., 22(1):1–21, 1951.
The best estimator is not random if the loss function is strictly convex
![Page 79: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/79.jpg)
Non Bayesian
Bayesian
Complete class theorem
Risk
Prior
EstimatorNon cooperative Minmax loss/error
cooperative Bayesian loss/error
Over-estimate risk
Under-estimate risk
![Page 80: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/80.jpg)
L. Le Cam. An extension of Wald’s theory of statistical decision functions. Ann.Math. Statist., 26:69–81, 1955
L. J. Savage. The theory of statistical decision. Journal of the American StatisticalAssociation, 46:55–67, 1951.
Further generalization of Statistical decision theory
A. Shapiro and A. Kleywegt. Minimax analysis of stochastic problems. Optim.Methods Softw., 17(3):523–542, 2002.
M. Sniedovich. The art and science of modeling decision-making under severeuncertainty. Decis. Mak. Manuf. Serv., 1(1-2):111–136, 2007
M. Sniedovich. A classical decision theoretic perspective on worst-case analysis.Appl. Math., 56(5):499–509, 2011.
L. D. Brown. Minimaxity, more or less. In Statistical Decision Theory and RelatedTopics V, pages 1–18. Springer, 1994.
L. D. Brown. An essay on statistical decision theory. Journal of the AmericanStatistical Association, 95(452):1277–1281, 2000.
I. Gilboa and D. Schmeidler. Maxmin expected utility with non-unique prior.Journal of Mathematical Economics, 18(2):141–153, 1989
![Page 81: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/81.jpg)
H. Owhadi and C. Scovel. Towards Machine Wald. Handbook for UncertaintyQuantication, 2016. arXiv:1508.02449.
If we want to make decision theory practical for UQ we need to introduce computational complexity constraints
Impact in econometrics and social sciences
R. Leonard. Von Neumann, Morgenstern, and the Creation of Game Theory: FromChess to Social Science, 1900–1960. Cambridge University Press, 2010.
O. Morgenstern. Abraham Wald, 1902-1950. Econometrica: Journal of the Econo-metric Society, pages 361–367, 1951.
G. Tintner. Abraham Wald’s contributions to econometrics. Ann. Math. Statistics,23:21–28, 1952.
How do we do that?
Is there a natural relation between game theory, computational complexity and numerical approximations?
![Page 82: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/82.jpg)
Ax = b
Φx = y
A: Known n× n symmetricpositive definite matrix
b: Unknown element of Rn
Approximate solution x of
Based on the information that
Φ: Known m× nrank m matrix (m < n)
y: Known element of Rm
A simple approximation problem
![Page 83: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/83.jpg)
![Page 84: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/84.jpg)
AMax Min
A
![Page 85: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/85.jpg)
![Page 86: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/86.jpg)
Deterministic zero sum game
3
1
-2
-2Player I’s payoff
How should I and II play the game?
![Page 87: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/87.jpg)
Pure strategy (classical numerical analysis) solution
3
1
-2
-2II should play blue and loose 1 in the worst case
I should play red and loose 2 in the worst case
![Page 88: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/88.jpg)
Mixed strategy (repeated game) solution
3
1
-2
-2II should play red with probability 3/8 and win 1/8 on average
I should play red with probability 3/8 and loose 1/8 on average
![Page 89: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/89.jpg)
3
1
-2
-2 Player I’s payoff
J. Von Neumann
![Page 90: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/90.jpg)
Game theoretic formulationAx = b
Max Min
Abraham Wald
Continuous game but as in decision theory under compactness it can be approximated by a finite game
![Page 91: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/91.jpg)
Best strategy: lift minimax to measuresAx = b
Max Min
The best strategy for I is to play at randomPlayer II’s best strategy live
in the Bayesian class of estimators
![Page 92: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/92.jpg)
Player II’s mixed strategy
Ax = b AX = ξξ ∼ N (0, Q)
Player II’s bet
![Page 93: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/93.jpg)
Player II’s mixed strategy
Ax = b AX = ξξ ∼ N (0, Q)
Theorem
Owhadi 2015, Multi-grid with rough coefficients and Multiresolution PDE decomposition from Hierarchical Information Games, arXiv:1503.03467, SIAM Review (to appear)
![Page 94: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/94.jpg)
Main Question
Can we turn the process of discovery of a scalable numerical method into a UQ problem and, to some degree, solve it as such in an automated fashion?
Can we use a computer, not only to implement a numerical method but also to find the method itself?
![Page 95: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/95.jpg)
− div(a∇u) = g, x ∈ Ω,u = 0, x ∈ ∂Ω,
(1)
Ω ⊂ Rd ∂Ω is piec. Lip.
a unif. ell.ai,j ∈ L∞(Ω)
Example: Find a method for solving (1) as fast as possible to a given accuracy
log10(a)
![Page 96: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/96.jpg)
Multigrid Methods
Multiresolution/Wavelet based methods[Brewster and Beylkin, 1995, Beylkin and Coult, 1998, Averbuch et al., 1998]
Multigrid: [Fedorenko, 1961, Brandt, 1973, Hackbusch, 1978]
• Linear complexity with smooth coefficients
Severely affected by lack of smoothnessProblem
![Page 97: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/97.jpg)
[Mandel et al., 1999,Wan-Chan-Smith, 1999,Xu and Zikatanov, 2004, Xu and Zhu, 2008], [Ruge-Stuben, 1987]
Robust/Algebraic multigrid
• Some degree of robustness but problem remains open with rough coefficients
Why?Don’t know how to bridge scales with rough coefficients!
Interpolation operators are unknown
[Vassilevski - Wang, 1997, 1998]
Stabilized Hierarchical bases, Multilevel preconditioners
[Panayot - Vassilevski, 1997]
[Chow - Vassilevski, 2003]
[Panayot - 2010]
[Aksoylu- Holst, 2010]
![Page 98: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/98.jpg)
Low Rank Matrix Decomposition methodsFast Multipole Method: [Greengard and Rokhlin, 1987]
Hierarchical Matrix Method: [Hackbusch et al., 2002]
[Bebendorf, 2008]:
N ln2d+8N complexityTo achieve grid-size accuracy in L2-norm
![Page 99: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/99.jpg)
Their process of discovery is based on intuition, brilliant insight, and guesswork
Common theme between these methods
Can we turn this process of discovery into an algorithm?
![Page 100: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/100.jpg)
YESAnswer:
Identify gameFind optimal strategy
N ln3dN complexityResulting method:
Compute fast
This is a theorem
Compute with partial information
Play adversarial Information game
To achieve grid-size accuracy in H1-normSubsequent solves: N lnd+1N complexity
Owhadi 2015, Multi-grid with rough coefficients and Multiresolution PDE decomposition from Hierarchical Information Games, arXiv:1503.03467, SIAM Review (to appear)
![Page 101: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/101.jpg)
Resulting method:
H10 (Ω) = W(1) ⊕aW(2) ⊕a · · ·⊕aW(k) ⊕a · · ·
(− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
For v ∈W(k)
C12k≤ kvka
k div(a∇v)kL2(Ω)≤ C2
2k
Theorem
< ψ,χ >a:=RΩ(∇ψ)Ta∇χ = 0 for (ψ,χ) ∈W(i) ×W(j), i 6= j
kvk2a :=< v, v >a=RΩ(∇v)T a∇v
Looks like an eigenspace decomposition
![Page 102: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/102.jpg)
w(k) = F.E. sol. of PDE in W(k)
Can be computed independently
u = w(1) + w(2) + · · ·+ w(k) + · · ·
u=
w(1) w(2) w(3)
w(4) w(5) w(6)
8× 10−3
1.5× 10−3 4× 10−4 4× 10−5
0.030.14
+
+
+
+
Multiresolution decomposition of solution space
![Page 103: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/103.jpg)
Quacks like an eigenspace decomposition
w(k) = F.E. sol. of PDE in W(k)
Can be computed independently
B(k): Stiffness matrix of PDE in W(k)
Theorem λmax(B(k))
λmin(B(k))≤ C
Just relax in W(k) to find w(k)
u = w(1) + w(2) + · · ·+ w(k) + · · ·
![Page 104: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/104.jpg)
Swims like an eigenspace decomposition
μ(x)∂2t u− div(a∇u) = g(x, t)
Application to time dependent problems
μ(x)∂tu− div(a∇u) = g(x, t)
[Owhadi-Zhang 2016, From gamblets to near FFT-complexitysolvers for wave and parabolic PDEs with rough coefficients]
Hyperbolic and parabolic PDEs with rough coefficientscan be solved in O(N ln3dN) (near FFT) complexity
u=
w(1) w(2) w(3)
w(4) w(5) w(6)
8× 10−3
1.5× 10−3 4× 10−4 4× 10−5
0.030.14
+
+
+
+
![Page 105: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/105.jpg)
Doesn’t have the complexity of an eigenspace decomposition
Theorem
Can be performed and stored in
V: F.E. space of H10 (Ω) of dim. N
V = W(1) ⊕aW(2) ⊕a · · ·⊕aW(k)
The decomposition
O(N ln3dN) operations
![Page 106: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/106.jpg)
ψ(1)i χ
(2)i χ
(3)i
χ(4)i χ
(5)i
χ(6)i
Basis functions look like and behave like wavelets:Localized and can be used to compress the operator
and locally analyze the solution space
![Page 107: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/107.jpg)
u
H−1(Ω)H10 (Ω)
um gm
div(a∇·)
g
Inverse Problem
Reduced operator
∈ RmRmNumerical implementation requirescomputation with partial information.
um ∈ Rm u ∈ H10 (Ω)Missing information
φ1, . . . ,φm ∈ L2(Ω)
um = (Ωφ1u, . . . , Ω φmu)
![Page 108: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/108.jpg)
Discovery process (− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
φ1, . . . ,φm ∈ L2(Ω)
u− u∗a
Player I Player IIChoosesg ∈ L2(Ω) Sees
Ωuφ1, . . . , Ω uφm
Chooses u∗ ∈ L2(Ω)kgkL2(Ω) ≤ 1
Max Min
Identify underlying information game
Measurement functions:
kfk2a :=RΩ(∇f)T a∇f
![Page 109: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/109.jpg)
Player I
Player II
3
1
-2
-2
Deterministic zero sum game
Player I’s payoff
Player I & II both have a blue and a red marbleAt the same time, they show each other a marble
How should I & II play the (repeated) game?
![Page 110: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/110.jpg)
Game theory
John Von Neumann
John Nash
Player I
Player II
3
1
-2
-2
= 3pq + (1− p)(1− q)− 2p(1− q)− 2q(1− p)=1− 3q + p(8q − 3) =− 1
8 for q = 38
q 1− q
p
1− p
Optimal strategies are mixed strategies
Optimal way toplay is at random
![Page 111: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/111.jpg)
Abraham Wald
The best strategy for A is to play at randomPlayer B’s best strategy live
in the Bayesian class of estimators
Player A Player BChoosesg ∈ L2(Ω) Sees
RΩuφ1, . . . ,
RΩuφm
Chooses u∗ ∈ L2(Ω)kgkL2(Ω) ≤ 1
Continuous game but as in decision theoryunder compactness it can be approximatedby a finite game
°°u− u∗°°a
![Page 112: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/112.jpg)
Player II’s class of mixed strategies
(− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
(− div(a∇v) = ξ in Ω,
v = 0 on ∂Ω,
ξ: Random field
u∗(x) := E£v(x)
¯ RΩv(y)φi(y) dy =
RΩu(y)φi(y) dy,∀i
¤Player II’s bet
g ∈ L2(Ω)
Pretend that player I is choosing g at random
min max problemover distribution of ξ
Player II’s optimal strategy?
![Page 113: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/113.jpg)
Theorem
ψi(x) := Eξ∼N (0,Γ)hv(x)
¯ RΩv(y)φj(y) dy = δi,j , j ∈ 1, . . . ,m
iψi: Elementary gambles/bets
ψiGamblets
Elementary gambles form deterministic basis functions for player B’s bet
u∗(x) =Pm
i=1 ψi(x)RΩu(y)φi(y) dy
ξ ∼ N (0,Γ)Computational efficiency
![Page 114: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/114.jpg)
Depend onWhat are these gamblets?
Example
Γ(x, y) = δ(x− y)φi(x) = δ(x− xi)
Ω
xi
x1
xm
a = Id ψi: Polyharmonic splines[Harder-Desmarais, 1972][Duchon 1976, 1977,1978]
ai,j ∈ L∞(Ω) ψi: Rough Polyharmonic splines[Owhadi-Zhang-Berlyand 2013]
• Γ: Covariance function of ξ (Player B’s decision)
• (φi)mi=1: Measurements functions (rules of the game)
![Page 115: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/115.jpg)
What is Player II’s best choice for
Γ(x, y) = E£ξ(x)ξ(y)
¤What is Player II’s best strategy?
Γ = LL = − div(a∇·)
RΩξ(x)f(x) dx ∼ N (0, kfk2a)
kfk2a :=RΩ(∇f)Ta∇f
Why? See algebraic generalization
?
![Page 116: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/116.jpg)
Theorem
u∗(x) is the F.E. solution of (1) in spanL−1φi|i = 1, . . . ,mku− u∗ka = infψ∈spanL−1φi:i∈1,...,m ku− ψka
If Γ = L then
The recovery is optimal (Galerkin projection)
L = − div(a∇·)(− div(a∇u) = g, x ∈ Ω,
u = 0, x ∈ ∂Ω,(1)
![Page 117: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/117.jpg)
Theorem ψi: Unique minimizer of(Minimize kψkaSubject to ψ ∈ H1
0 (Ω) andRΩφjψ = δi,j , j = 1, . . . ,m
Pmi=1 wiψi minimizes kψka
over all ψ such thatRΩφjψ = wj for j = 1, . . . ,m
TheoremOptimal variational properties
Variational characterization
![Page 118: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/118.jpg)
Selection of measurement functions
Theorem ku− u∗ka ≤ Hλmin(a)
kgkL2(Ω)
φi = 1τi τi
Ω
diam(τi) ≤ H
τj
Indicator functions of aExample
Partition of Ω of resolution H
![Page 119: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/119.jpg)
Elementary gamble
ψi
(− div(a∇u) = g, x ∈ Ω,
u = 0, x ∈ ∂Ω,(1)
Your best bet on the value of u
τi
Ωτj
10000
0000
0000
0
00
given the information thatRτiu = 1 and
Rτju = 0 for j 6= i
![Page 120: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/120.jpg)
Exponential decay of gamblets
TheoremRΩ∩(B(τi,r))c(∇ψi)
Ta∇ψi ≤ e−rlH kψik2a
x-axis slice
ψi
ψi
log10¡10−10 + |ψi||
¢x-axis slice
4
−10
r
Ω
τi
![Page 121: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/121.jpg)
r
Ω
τiSr
Theorem
ku− u∗,locka ≤ 1√λmin(a)
HkgkL2(Ω)
u∗,loc(x) =Pm
i=1 ψloc,ri (x)
RΩu(y)φi(y) dy
If r ≥ CH ln 1H
No loss of accuracy iflocalization ∼ H ln 1
H
ψloc,ri : Minimizer of(Minimize kψkaSubject to ψ ∈ H1
0 (Sr) andRSr
φjψ = δi,j
for τj ∈ Sr
Localization of the computation of gamblets
![Page 122: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/122.jpg)
Formulation of the hierarchical game
![Page 123: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/123.jpg)
Hierarchy of nested Measurement functions
φ(1)2 = 1
τ(1)2
φ(2)2,3 = 1
τ(2)2,3
φ(k)i1,...,ik
with k ∈ 1, . . . , qφ(k)i =
Pj ci,jφ
(k+1)i,j
τ(1)2 τ
(2)2,3
τ(3)2,3,1
φ(3)2,3,1 = 1
τ(3)2,3,1
Example
φ(1)i1
φ(2)i1,j1
φ(2)i1,j2
φ(2)i1,j3
φ(2)i1,j4
φ(3)i1,j2,k1
φ(3)i1,j2,k2
φ(3)i1,j2,k3
φ(3)i1,j2,k4
φ(k)i : Indicator functions of a
hierarchical nested partition of Ω of resolution Hk = 2−k
![Page 124: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/124.jpg)
iΠ1,2i
I1 I2 I3
j ∈ Π1,2i ⊂ Π2I Π2,3j
Π1,3i
τ(1)i τ
(2)j
φ(1)i φ
(2)i φ
(3)i
φ(4)i φ
(5)i φ
(6)i
In the discrete setting simply aggregate elements(as in algebraic multigrid)
![Page 125: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/125.jpg)
Player I Player IIChoosesg ∈ L2(Ω)
Formulation of the hierarchy of games
kgkL2(Ω) ≤ 1
(− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
Must predict
Sees Ωuφ
(k)i , i ∈ Ik
u and Ωuφ
(k+1)j , j ∈ Ik+1
![Page 126: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/126.jpg)
Player II’s best strategy(− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
(− div(a∇v) = ξ in Ω,
v = 0 on ∂Ω,
ξ ∼ N (0,L)
u(k)(x) := E£v(x)
¯ RΩv(y)φ
(k)i (y) dy =
RΩu(y)φ
(k)i (y) dy, i ∈ Ik
¤Player II’s bets
Fk = σ(RΩvφ
(k)i , i ∈ Ik) v(k)(x) := E
£v(x)
¯Fk¤
Theorem Fk ⊂ Fk+1v(k)(x) := E
£v(k+1)(x)
¯Fk¤
The sequence of approximations forms a martingale under the mixed strategy emerging from the game
![Page 127: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/127.jpg)
Player II’s best strategy(− div(a∇u) = g in Ω,
u = 0 on ∂Ω,
(− div(a∇v) = ξ in Ω,
v = 0 on ∂Ω,
ξ ∼ N (0,L)
u(k)(x) := E£v(x)
¯ RΩv(y)φ
(k)i (y) dy =
RΩu(y)φ
(k)i (y) dy, i ∈ Ik
¤Player II’s bets
u(1) u(2) u(3)
u(4) u(5) u(6)
![Page 128: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/128.jpg)
Gamblets Elementary gambles form a hierarchy of deterministic basis functions for player II’s hierarchy of bets
Theorem u(k)(x) =P
i ψ(k)i (x)
RΩu(y)φ
(k)i (y) dy
ψ(k)i : Elementary gambles/bets at resolution Hk = 2−k
ψ(k)i (x) := E
hv(x)
¯ RΩv(y)φ
(k)j (y) dy = δi,j , j ∈ Ik
iψ(1)i ψ
(2)i ψ
(3)i
ψ(4)i ψ
(5)i ψ
(6)i
![Page 129: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/129.jpg)
Theorem
V(k) ⊂ V(k+1)
V(k) := spanψ(k)i , i ∈ Ik ψ(1)i1
ψ(2)i1,j1
ψ(2)i1,j2
ψ(2)i1,j3
ψ(2)i1,j4
ψ(3)i1,j2,k1
ψ(3)i1,j2,k2
ψ(3)i1,j2,k3
ψ(3)i1,j2,k4
Gamblets are nested
ψ(k)i (x) =
Pj∈Ik+1 R
(k)i,j ψ
(k+1)j (x)
![Page 130: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/130.jpg)
R(k)i,j = E
£ RΩv(y)φ
(k+1)j (y) dy
¯ RΩv(y)φ
(k)l (y) dy = δi,l, l ∈ Ik
¤Interpolation/Prolongation operator
10 0
0
R(k)i,j
Your best bet on the value ofRτ(k+1)j
u
given the information thatRτ(k)iu = 1 and
Rτlu = 0 for l 6= i
τ(k)i R
(k)i,j
τ(k+1)j
![Page 131: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/131.jpg)
At this stage you can finish withclassical multigrid
But we want multiresolution decomposition
![Page 132: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/132.jpg)
Elementary gamble
Ω
00
0
χ(k)i
000
00τ
(k)i τ
(k)j
00
00
Your best bet on the value of u
given the information thatRτ(k)i
u = 1,Rτ(k)
i−u = −1 and
Rτ(k)j
u = 0 for j 6= i
100
-1
τ(k)i−
![Page 133: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/133.jpg)
+1−1
χ(k)i = ψ
(k)i − ψ
(k)i−
+1
−1
+1−1
i = (i1, . . . , ik−1, ik)
i− = (i1, . . . , ik−1, ik − 1)
ψ(1)i1
ψ(2)i1,j1
ψ(2)i1,j2
ψ(2)i1,j3
ψ(2)i1,j4
+1−1
![Page 134: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/134.jpg)
ψ(1)i χ
(2)i χ
(3)i
χ(4)i χ
(5)i
χ(6)i
χ(k)i = ψ
(k)i − ψ
(k)i−
![Page 135: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/135.jpg)
Theorem
W(k+1): Orthogonal complement of V(k) in V(k+1)
with respect to < ψ,χ >a:=RΩ(∇ψ)T a∇χ
H10 (Ω) = V(1) ⊕aW(2) ⊕a · · ·⊕aW(k) ⊕a · · ·
Multiresolution decomposition of the solution space
V(k) := spanψ(k)i , i ∈ IkW(k) := spanχ(k)i , i
![Page 136: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/136.jpg)
Theorem
u(k+1) − u(k) = F.E. sol. of PDE in W(k+1)
u=
u(1) u(2) − u(1) u(3) − u(2)
u(4) − u(3) u(5) − u(4) u(6) − u(5)
8× 10−3
1.5× 10−3 4× 10−4 4× 10−5
0.030.14
+
+
+
+
Multiresolution decomposition of the solution
Subband solutions u(k+1) − u(k)can be computed independently
![Page 137: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/137.jpg)
Uniformly bounded condition numbers
A(k)i,j :=
ψ(k)i ,ψ
(k)j
®a
B(k)i,j :=
χ(k)i ,χ
(k)j
®a
4.5log10(
λmax(A(k))
λmin(A(k)))
log10(λmax(B
(k))λmin(B(k))
)
Theoremλmax(B
(k))
λmin(B(k))≤ C
Just relax!In v ∈W(k)
to getu(k) − u(k−1)
![Page 138: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/138.jpg)
V = W(1) ⊕aW(2) ⊕a · · ·⊕aW(k)
Ranges of eigenvalues in V andW(k) (k = 1, . . . , 5) in log scale
£infψ∈V
kψk2akψk2
L2, supψ∈V
kψk2akψk2
L2
¤
£infψ∈W(k)
kψk2akψk2
L2, supψ∈W(k)
kψk2akψk2
L2
¤
![Page 139: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/139.jpg)
c(1)i
c(2)j
c(3)j
c(4)j
0 200 400 600 800-6
-4
-2
0
2
4x 10
-4
c(5)j c
(6)j
u =P
i c(1)i
ψ(1)i
kψ(1)i ka+Pq
k=2
Pj c(k)j
χ(k)j
kχ(k)j ka
0 1000 2000 3000 4000-4
-2
0
2
4x 10
-5
Coefficients of the solution in the gamblet basis
c(6)j
![Page 140: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/140.jpg)
Operator Compression
Throw 99% of the coefficients
u
Gamblets behave like wavelets but they are adapted to the PDE and can compress its solution space
Gamblet compression
Compression ratio = 105Energy norm relative error = 0.07
![Page 141: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/141.jpg)
Fast gamblet transform
Nesting A(k) = (R(k,k+1))TA(k+1)R(k,k+1)
Level(k) gamblets and stiffness matrices can be computedfrom level(k+1) gamblets and stiffness matrices
Well conditioned linear systems
ψ(k)i = ψ
(k+1)(i,1) +
Pj C
(k+1),χi,j χ
(k+1)j C(k+1),χ = (B(k+1))−1Z(k+1)
LocalizationZ(k+1)j,i := −(e
(k+1)j − e(k+1)j− )TA(k+1)e
(k+1)(i,1)
Underlying linear systems have uniformly bounded condition numbers
The nested computation can be localized without compromising accuracy or condition numbers
O(N ln3dN) complexity
![Page 142: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/142.jpg)
Theorem
Localizing (ψ(k)i )i∈Ik and (χ
(k)i )i to subdomains of size
≥ CHk ln 1Hk
≥ CHk(ln 1Hk
+ ln 1² )
Cond. No (B(k),loc) ≤ C
°°u− u(1),loc −Pq−1k=1(u
(k+1),loc − u(k),loc)°°a≤ ²
The number of operations to compute gamblets andachieve accuracy ² is O
¡N ln3d
¡max( 1
², N1/d)
¢¢(and O
¡N lnd(N1/d) ln 1
²
¢for subsequent solves)
Theorem
Complexity
O(N ln3dN) O(N lnd+1N)GambletTransform
LinearSolve
![Page 143: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/143.jpg)
u(3) − u(2)8× 10−3
u(2) − u(1)0.03
u(1)0.14
...
u(1)
ϕi, Ah,Mh χ
(q)i , B
(q)ψ(q)i , A(q) u(q) − u(q−1)
ψ(q−1)i , A(q−1) χ
(q−1)i , B(q−1) u(q−1) − u(q−2).
..
...
χ(3)i , B(3)ψ
(3)i , A(3) u(3) − u(2)
ψ(2)i , A(2) χ
(2)i , B(2) u(2) − u(1)
ψ(1)i , A(1)
ψ(1)i
χ(2)i
χ(3)i
ψ(1)i
ψ(2)i
ψ(3)i
Parallel operatingdiagram
both in space and in
frequency
![Page 144: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/144.jpg)
Numerical Homogenization
HMM
Harmonic Coordinates
[Gloria 2010]
Babuska, Caloz, Osborn, 1994Allaire Brizzi 2005; Owhadi, Zhang 2005
Engquist, E, Abdulle, Runborg, Schwab, et Al. 2003-...
MsFEM [Hou, Wu: 1997]; [Efendiev, Hou, Wu: 1999]
Nolen, Papanicolaou, Pironneau, 2008
[Chu-Graham-Hou-2010] (limited inclusions)
[Babuska-Lipton 2010] (local boundary eigenvectors)[Efendiev-Galvis-Wu-2010] (limited inclusions or mask)
[Malqvist-Peterseim 2012] Volume averaged interpolation
Flux norm Berlyand, Owhadi 2010; Symes 2012
Arbogast, 2011: Mixed MsFEM
Kozlov, 1979
Localization
[Fish - Wagiman, 1993]
Projection based method
[Owhadi-Zhang 2011] (localized transfer property)
![Page 145: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/145.jpg)
Statistical approach to numerical approximation
![Page 146: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/146.jpg)
Statistical approach to numerical approximation
![Page 147: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/147.jpg)
Some high level remarks
![Page 148: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/148.jpg)
What is the worst?
Au†: Unknown element of A
Robust Optimization worst case
Φ : A −→ Ru −→ Φ(u) Quantity of Interest
What is Φ(u†)?
infu∈AΦ(u) ≤ Φ(u†) ≤ supu∈A Φ(u)
![Page 149: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/149.jpg)
A
Game theoretic worst case
AMax Min
![Page 150: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/150.jpg)
Confidence error
Robust Optimization worst caseFailure is not an option. You want to always be right.
Game theoretic worst case
Interpretation depends on the choice of loss function.
You want to be right with high probability.
Quadratic errorYou want to be right on average.Well suited for numerical computation where you need to keep computing with partial information (e.g. invert a 1,000,000 by 1,000,000 matrix)
![Page 151: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/151.jpg)
Non Bayesian
Bayesian
Complete class theorem
Risk
Prior
EstimatorNon cooperative Minmax loss/error
cooperative Bayesian loss/error
Over-estimate risk
Under-estimate risk
Can we approximate the optimal prior?
![Page 152: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/152.jpg)
Numerical robustness of Bayesian inference
Can we numerically approximate the prior when closed form expressions are not available for posterior values?
![Page 153: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/153.jpg)
![Page 154: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/154.jpg)
• Brittleness of Bayesian Inference under Finite Information in a Continuous World. H. Owhadi, C. Scovel and T. Sullivan. Electronic Journal of Statistics, vol 9, pp 1-79, 2015. arXiv:1304.6772
• Brittleness of Bayesian inference and new Selberg formulas. H. Owhadi and C. Scovel. Communications in Mathematical Sciences (2015). arXiv:1304.7046
• On the Brittleness of Bayesian Inference. H. Owhadi, C. Scovel and T. Sullivan. SIAM Review, 57(4), 566-582, 2015, arXiv:1308.6306
• Qualitative Robustness in Bayesian Inference (2015). H. Owhadi and C. Scovel. arXiv:1411.3984
Positive Negative• Classical Bernstein Von Mises• Wasserman, Lavine, Wolpert (1993)• P Gustafson & L Wasserman (1995)• Castillo and Rousseau (2013)• Castillo and Nickl (2013)• Stuart & Al (2010+). • ….
• Freedman (1963, 1965)• P Gustafson & L Wasserman (1995)• Diaconis & Freedman 1998• Johnstone 2010• Leahu 2011• Belot 2013
Robustness of Bayesian conditioning in continuous spaces
![Page 155: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/155.jpg)
10.000 children are given one pound of play-doh. On average, how much mass can they put above awhile, on average, keeping the seesaw balanced around m?
Paul is given one pound of play-doh. What can you say about how much mass he isputting above a if all you have is the belief thathe is keeping the seesaw balanced around m?
Brittleness of Bayesian Inference under Finite Information in a Continuous World. H. Owhadi, C. Scovel and T. Sullivan. Electronic Journal of Statistics, vol 9, pp 1-79, 2015. arXiv:1304.6772
![Page 156: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/156.jpg)
What is the least upper bound on
If all you know is ?
Answer
![Page 157: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/157.jpg)
Theorem
![Page 158: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/158.jpg)
![Page 159: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/159.jpg)
![Page 160: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/160.jpg)
Reduction calculus with measures over measures
QΨ
QΨ−1 ⊂M(Q)M(A) ⊃Theorem
supπ∈Ψ−1Q
Eμ∼π Φ(μ)
supQ∈Q
hEq∼Q
£sup
μ∈Ψ−1(q)Φ(μ)
¤i=
M(X ) ⊃ Polishspace
Brittleness of Bayesian Inference under Finite Information in a Continuous World. H. Owhadi, C. Scovel and T. Sullivan. Electronic Journal of Statistics, vol 9, pp 1-79, 2015. arXiv:1304.6772
![Page 161: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/161.jpg)
What is the worst with random data? A
![Page 162: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/162.jpg)
A
Frequentist/Concentration of measure worst case
![Page 163: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/163.jpg)
Theorem
N. Fournier and A. Guillin, On the rate of convergence in Wasserstein distance of the empirical measure, Probability Theory and Related Fields, (2014), pp. 1-32.
A
![Page 164: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/164.jpg)
The extreme points of the Prokhorov, Monge-Wasserstein and Kantorovich metric balls about a measure whose support has at most n points, consist of measures whose supports have at most n+2 points.
• D. Wozabal. A framework for optimization under ambiguity. Annals of Operations Research, 193(1):21—47, 2012.
• P. M. Esfahani and D. Kuhn. Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. arXiv:1505.05116, 2015.
• Extreme points of a ball about a measure with finite support (2015). H. Owhadi and Clint Scovel. arXiv:1504.06745
Reduction calculus of the ball about the empirical distribution
![Page 165: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/165.jpg)
QuestionGame/Decision Theory + Information Based Complexity
Turn the process of discovery of scalable numerical solversinto an algorithm
Worst case calculus
?
![Page 166: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/166.jpg)
P. L. Chebyshev1821-1894
M. G. Krein1907-1989
A. A. Markov1856-1922
The truncated moment problem
³EX∼μ[X],EX∼μ[X2], . . . ,EX∼μ[Xk]
´Ψ
Study of the geometry of Mk := Ψ¡M([0, 1])
¢
![Page 167: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/167.jpg)
Finite dim.Infinite dim.
³EX∼μ[X],EX∼μ[X2], . . . ,EX∼μ[Xk]
´Ψ Mk := Ψ¡M([0, 1])
¢
Finite dim.
![Page 168: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/168.jpg)
Finite dim.Infinite dim.
Ψ
t1 t2 tj tN
![Page 169: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/169.jpg)
Index
Theorem
t1 t2 tj tN
t1 t2 tj tN
Upper
Lower
![Page 170: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/170.jpg)
Sn(α,β, γ) =Qn−1j=0
Γ(α+jγ)Γ(β+jγ)Γ(1+(j+1)γ)Γ(α+β+(n+j−1)γ)Γ(1+γ)
Selberg Identities
Sn(α,β, γ) :=R[0,1]n
Qnj=1 t
α−1j (1− tj)β−1|∆(t)|2γdt .
∆(t) :=Qj<k (tk − tj)
Brittleness of Bayesian inference and new Selberg formulas. H. Owhadi and C. Scovel. Communications in Mathematical Sciences (2015). arXiv:1304.7046
![Page 171: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/171.jpg)
Forrester and Warnaar 2008
The importance of the Selberg integral
Used to prove outstanding conjectures inRandom matrix theory and cases of the Macdonald conjectures
Central role in random matrix theory, Calogero-Sutherland quantum many-body systems, Knizhnik-Zamolodchikov equations, and multivariable orthogonal polynomial theory
![Page 172: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/172.jpg)
Index
Theorem
t1 t2 tj tNt∗
![Page 173: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/173.jpg)
New Reproducing Kernel Hilbert Spaces and Selberg Integral formulas related to the Markov-Krein representations of moment spaces.
Ψ ³EX∼μ[X],EX∼μ[X2], . . . ,EX∼μ[Xk]
´RImΣt−1 ·Qm
j=1 t2j (1− tj)2∆4m(t)dt = Sm(5,1,2)−Sm(3,3,2)
2RImΣt−1 ·Qm
j=1 t2j ·∆4m(t)dt = m
2 Sm−1(5, 3, 2)
Sn(α,β, γ) =Qn−1j=0
Γ(α+jγ)Γ(β+jγ)Γ(1+(j+1)γ)Γ(α+β+(n+j−1)γ)Γ(1+γ)
∆m(t) :=Qj<k (tk − tj)¡
Σφ¢(t) :=
Pmj=1 φ(tj), t ∈ Im
I := [0, 1]
![Page 174: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/174.jpg)
ZIm−1
hk(t)ΣQj(t)
m−1Yj0=1
t2j0 ·∆4m−1(t)dt = V ol(M2m−1)(2m−1)!(m−1)!(k + 2)!
(8k + 4)(k − 2)!δjk .
Theorem
ej(t) :=X
i1<···<ijti1 · · · tij
Πn0 : n-th degree polynomials which vanish on the boundary of [0, 1]Mn ⊂ Rn: set of q = (q1, . . . , qn) ∈ Rn such that there exists a probabilitymeasure μ on [0, 1] with Eμ[Xi] = qi with i ∈ 1, . . . , n.
Consider the basis of Π2m−10 consisting of the associated Legendre polyno-mials Qj , j = 2, .., 2m − 1 of order 2 translated to the unit interval I. Fork = 2, .., 2m− 1 define
ajk :=(j + k + k2)Γ(j + 2)Γ(j)
Γ(j + k + 2)Γ(j − k + 1), k ≤ j ≤ 2m− 1
hk(t) :=2m−1Xj=k
(−1)j+1ajke2m−1−j(t, t) .
Then for j = k mod 2, j, k = 2, .., 2m− 1, we have
Bi-orthogonal systems of Selberg Integral formulas
![Page 175: Houman Owhadi - SIAM](https://reader031.vdocuments.us/reader031/viewer/2022012105/61dc16a5e2a0c110b13aeb1f/html5/thumbnails/175.jpg)
Collaborators
Research supported by Air Force Office of Scientific Research
U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research, through the Exascale Co-Design Center for Materials in Extreme Environments
National Nuclear Security Administration
Clint Scovel (Caltech), Tim Sullivan (Warwick), Mike McKerns (Caltech), Michael Ortiz (Caltech), Lei Zhang (Jiaotong), Leonid Berlyand (PSU), Lan HuongNguyen, Paul Herve Tamogoue Kamga.
DARPA EQUiPS Program (Enabling Quantification of Uncertainty in Physical Systems)