![Page 1: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/1.jpg)
Final Exam Time and Place:Saturday, Dec 8,
9:00am - 12:00pmEN 1054
![Page 2: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/2.jpg)
Chapter 19.1 Exploratory Data Analysis
![Page 3: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/3.jpg)
What is Exploratory Data Analysis?
• An approach to analyze data sets to:– Discover patterns– Find a better model
• It’s an iterative process– Refine to uncover patterns
![Page 4: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/4.jpg)
Confirmatory vs. Exploratory
Confirmatory analysis• What decision can be made?• How certain can we be?• What are values of parameters?• Sample• ONE use of a sample (data-
grinding, otherwise)• Single analysis• p-value = ?• Yes/no decision• Residuals acceptable?• Experimental design
Exploratory analysis • What is the appropriate model?• What is data telling us?• What is structure of model?• Batch of data• Repeated use of a
batch .• Iterative search for pattern• Explained variance = ?• Best model• Residuals show pattern?• Factor analysis
![Page 5: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/5.jpg)
ExploratoryWhat is the appropriate model?
But remember, pattern ≠ cause
![Page 6: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/6.jpg)
ConfirmatoryWhat decision can be made?
![Page 7: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/7.jpg)
Inference
• Confirmatory– Narrow form of inference– Relate one Q to another Q
(e.g. βreg)
• Exploratory– Broader form of inference– Trying to discover a pattern
worth running through a confirmatory analysis
P corm P soilN corn ~ N soil C corn C soil ⁞ ⁞
![Page 8: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/8.jpg)
Don’t confuse confirmatory and exploratory analyses
• Refining models using p-values ≠ exploratory analysis
• Repeated analysis of the same data set is data dredging (aka: data grinding, data mining, data fishing, data snooping…)
• Any data set has a degree of randomness, so multiple comparisons may be bound to find a false association
![Page 9: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/9.jpg)
![Page 10: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/10.jpg)
![Page 11: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/11.jpg)
![Page 12: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/12.jpg)
Characteristics of Exploratory Analyses
• Relies strongly on graphical analyses
http://gallery.r-enthusiasts.com/thumbs.php
![Page 13: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/13.jpg)
Characteristics of Exploratory Analyses
• Simplify – determine best model for pattern
![Page 14: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/14.jpg)
Execution
1. Define all quantities that are used – Procedure statement
– Name and Symbol
– Values with Units
2. Identify response and explanatory variables
3. Decide whether to undertake exploratory or confirmatory analysis, stating reasons for choice
4. State screening criterion to distinguish exploratory from confirmatory analysis– Visual screening
– P-value based (e.g. keep if <0.1)
![Page 15: Final Exam Time and Place: Saturday, Dec 8, 9:00am - 12:00pm EN 1054](https://reader035.vdocuments.us/reader035/viewer/2022062805/5697c0031a28abf838cc3984/html5/thumbnails/15.jpg)
Box and Arrow Diagrams Logic
• Gordon Riley is interested in aquatic productivity of Georges Bank
LightNutrients (nitrates,
phophates)
Phytoplankton Zooplankton