bayesian model robust and model discrimination designs william li operations and management science...
TRANSCRIPT
![Page 1: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/1.jpg)
Bayesian Model Robust and Model Discrimination Designs
William Li
Operations and Management Science Department
University of Minnesota
(joint work with Chris Nachtsheim and Vincent Agtobo)
![Page 2: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/2.jpg)
Outline
• Introduction
• Bayesian model-robust designs
• Bayesian model discrimination designs
![Page 3: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/3.jpg)
Part I: Introduction• Main objective: find designs that are efficient over a
class of models– Model estimation: Are all models estimable?– Model discrimination: Can estimable models be
discriminated?
• Brief literature review– Early work: Lauter (1974), Srivastava (1975), Cook and
Nachtsheim (1982)– Cheng, Steinberg, and Sun (1999)– Li and Nachtsheim (2000)– Jones, Li, Nachtsheim, Ye (2006)
![Page 4: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/4.jpg)
More literature review
• Bingham and Chipman (2002): Bayesian Hellinger distance
• Miller and Sitter (2005): the probability that the true model is identified
• Montgomery et al. (2005): application of new tools in model-robust designs
• Loeppky, Sitter, and Tang (2005): projection model space
• Jones, Li, Nachtsheim, Ye (2006): model-robust supersaturated designs
![Page 5: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/5.jpg)
A general framework
• Li (2006): a review on model-robust designs• Framework: three main elements
– Model space: F={f1, f2, …, fu}
– Criterion (e.g., EC, D-, EPD)
– Candidates designs (e.g., orthogonal designs)
• Objective (rephrase): select an optimal design from candidate designs, such that it is optimal over all models in F, with respect to a criterion
![Page 6: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/6.jpg)
Model spaces
• Srivastava (1975): search designs– F = {all effects of type (ii) + up to g effects of type
(iii)}
• Sun (1993), Li and Nachtsheim (2000)– Fg = {all main effects + up to g 2f interactions}
• Supersaturated designs– F = {any g out of m main effects}
• Loeppky, Sitter, and Tang (2005)– Fg = {g out of m main effects + all 2f interactions}
![Page 7: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/7.jpg)
Criteria and candidate designs
• Criteria– Bayesian model-robust criterion (related to EC
and IC of LN)– Bayesian model discrimination criteria (related
to EPD of Jones et al.)
• Candidate designs– Orthogonal designs– Optimal designs
![Page 8: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/8.jpg)
Bayesian optimal designs
• Main elements– Prior distribution: p()– Distribution of data: p(y | )– Utility function: U(d, y)– Design space
![Page 9: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/9.jpg)
Selected literature
• “Bayesian Experimental Design: A Review” – Chaloner and Verdinelli (1995)
• DuMouchel and Jones (1994): Bayesian D-optimal designs
• Jones, Lin, and Nachtsheim (2006): Bayesian supersaturated designs
![Page 10: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/10.jpg)
Part II: Bayesian model-robust designs
• Focus: estimability of designs– Estimation capacity (EC): percentage of estimable
models• Model-robust designs: EC=100%
– Information capacity (IC): average D-criterion value over all models
• Model space– LN (2000): main effects + g 2fi’s– Loeppky et al. (2006): g main effects + all 2fi’s among g factors
![Page 11: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/11.jpg)
Bayesian criterion
![Page 12: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/12.jpg)
Bayesian criterion for model-robust designs
![Page 13: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/13.jpg)
Bayesian model-robust design
• Prior probabilities– Uniform prior– Hierarchical prior
• Chipman, Hamada, and Wu (1997)
• Bayesian model-robust (BMR) criterion
• Bayesian model-robust design (BMRD)
![Page 14: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/14.jpg)
Design evaluations
• Evaluating existing orthogonal designs – 12-, 16-, and 20-run designs (Sun, Li, and Ye,
2002)– Two model spaces– Compute BMR values and rank designs– Compare BMR ranks with generalized WLP
ranks • Generalized WLP: Deng and Tang (1999)• Ranks for GWLP: given in Li, Lin, and Ye (2003)
![Page 15: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/15.jpg)
Rank comparison plot (for 16*7 designs)
![Page 16: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/16.jpg)
Design constructions
• Optimal designs– Balanced (equal # of +’s and –’s)
• CP algorithm of Li and Wu (1997)
– General (unbalanced) optimal designs• Coordinate-exchange algorithm of Meyer and
Nachtsheim (1995)
![Page 17: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/17.jpg)
Part III: Bayesian model discrimination designs
• Issues beyond model estimation– How well can estimable models be distinguished from
each other?
– If true model is known, is it fully aliased with other models through the design?
![Page 18: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/18.jpg)
Criteria
• Atkinson and Fedorov (1975)
• EPD (expected prediction differnce) criterion (Jones et al. 2006)
![Page 19: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/19.jpg)
Expected non-centrality parameter (ENCP) criterion
![Page 20: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/20.jpg)
Bayesian EPD criterion
![Page 21: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/21.jpg)
Design results
• Evaluating orthogonal designs– A comprehensive study of designs– Candidate designs: 12-, 16-, 20-run designs – Model space: both LN and the projected space of
Loeppky et al. (2006)– Criteria: all model discrimination criteria (Bayesian and
non-Bayesian)
• Constructing optimal designs– CP: balanced– Coordinate-exchange: general (unbalanced)
![Page 22: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/22.jpg)
An example
mEPD aEPD mAF aAF mENCP aENCP
----------------------------------------
(n=16, m=5, g=2)
1-3 EC < 100%
4 0.125 0.205 0.347 0.567 16.000 26.182
5 0.063 0.187 0.173 0.520 4.000 22.109
6 EC < 100%
7 0.000 0.184 -9.999 -9.999 0.000 19.806
8 0.094 0.198 0.173 0.495 4.000 19.673
9 EC < 100%
10 0.094 0.198 -9.999 -9.999 8.000 17.673
11 0.058 0.177 -9.999 -9.999 4.667 15.702
![Page 23: Bayesian Model Robust and Model Discrimination Designs William Li Operations and Management Science Department University of Minnesota (joint work with](https://reader036.vdocuments.us/reader036/viewer/2022062519/5697bfc01a28abf838ca3985/html5/thumbnails/23.jpg)
THANK YOU!
More information: www.csom.umn.edu/~wli