vistas task4cd finalreport appendicesa-e only

8/10/2019 VISTAS Task4cd FinalReport AppendicesA-E Only

1/144


2/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixA_Jan02.doc

Table A-1. CMAQ Performance for the 1-21 January 2002 Episode: IMPROVE Network.

12 Km Grid

Performance Measure SO4 NO3 NH4 OC EC SOIL

Paired_Peak -0.11 0.58 -0.09 0.24 0.89 19.98

Coef_Determ 0.47 0.30 0.25 0.49 0.05 0.14

Norm_Mean_Err 34.93 119.70 42.23 38.17 56.74 1337.62

Rt_Mean_Sqr_Err 1.04 1.70 0.60 1.31 0.44 3.22

Frac_Gross_Err 43.18 95.43 51.24 39.70 48.34 167.32

Mean_Abs_G_Err 0.80 1.21 0.48 0.92 0.23 2.59

Mean_Norm_G_Err 34.69 211.89 54.25 43.41 66.95 2373.26

Mean_Bias -0.64 0.86 -0.06 0.01 0.07 2.59 Mean_Norm_Bias -22.13 170.41 11.13 14.30 36.50 2373.26

Mean_Fract_Bias -33.56 29.47 -10.98 0.67 4.49 167.32

Norm_Mean_Bias -27.89 84.78 -5.07 0.55 16.75 1337.62

Observed_Mean 2.30 1.01 1.13 2.42 0.41 0.19

Predicted_Mean 1.66 1.87 1.07 2.43 0.48 2.78

SD_Observation 1.10 1.10 0.57 1.64 0.26 0.14

SD_Prediction 0.95 1.75 0.63 1.73 0.41 1.97

Corr_Variance 1.08 2.87 0.36 1.70 0.20 10.35

Bias Factor 0.78 2.70 1.11 1.14 1.37 24.73


3/144


36 Km Grid


Paired_Peak 0.38 0.82 0.05 0.32 0.18 12.25

Coef_Determ 0.41 0.22 0.20 0.44 0.09 0.25

Norm_Mean_Err 38.10 168.00 52.13 43.42 64.99 1558.45

Rt_Mean_Sqr_Err 1.10 2.46 0.75 1.41 0.40 3.44

Frac_Gross_Err 46.00 100.28 54.78 43.10 53.24 172.88

Mean_Abs_G_Err 0.87 1.70 0.59 1.05 0.27 3.02

Mean_Norm_G_Err 38.74 318.48 74.79 52.06 74.62 2755.75

Mean_Bias -0.51 1.46 0.16 0.17 0.12 3.02

Mean_Norm_Bias -16.55 286.35 39.70 25.49 49.43 2755.75

Mean_Fract_Bias -29.59 48.86 3.46 7.85 17.12 172.88

Norm_Mean_Bias -22.04 143.97 13.90 6.97 29.22 1558.45

Observed_Mean 2.30 1.01 1.13 2.42 0.41 0.19

Predicted_Mean 1.79 2.47 1.29 2.59 0.53 3.21

SD_Observation 1.10 1.10 0.57 1.64 0.26 0.14

SD_Prediction 1.22 2.26 0.78 1.78 0.37 1.72

Corr_Variance 1.22 6.07 0.56 2.00 0.16 11.81

Bias Factor 0.83 3.86 1.40 1.25 1.49 28.56


4/144


Table A-2. CMAQ Performance for the 1-21 January 2002 Episode: CASTNet Network.

12 Km Grid

Performance Measure HNO3 SO2 NH4 SO4 NO3 TNO3 NH4-FN N

Paired_Peak -0.26 0.62 -0.12 -0.16 0.03 -0.08 -0.17

Coef_Determ 0.07 0.68 0.64 0.01 0.38 0.37 0.43

Norm_Mean_Err 61.57 65.88 33.37 32.77 72.49 30.28 24.96

Rt_Mean_Sqr_Err 1.00 6.43 0.42 0.83 1.55 1.19 0.37

Frac_Gross_Err 75.51 51.60 31.45 37.80 67.06 31.04 26.30

Mean_Abs_G_Err 0.88 5.25 0.34 0.69 1.25 0.95 0.32

Mean_Norm_G_Err 70.41 84.69 41.68 30.97 171.61 44.65 30.04

Mean_Bias -0.46 5.13 0.30 -0.59 1.08 0.63 0.09

Mean_Norm_Bias -9.51 83.65 38.48 -23.93 160.99 37.17 15.10

Mean_Fract_Bias -42.38 50.44 28.04 -31.68 49.43 22.80 8.97

Norm_Mean_Bias -32.33 64.37 29.76 -27.93 62.77 20.12 7.12

Observed_Mean 1.42 7.98 1.02 2.12 1.72 3.12 1.29

Predicted_Mean 0.96 13.11 1.33 1.52 2.80 3.75 1.39

SD_Observation 0.59 4.18 0.46 0.51 1.22 1.26 0.48

SD_Prediction 0.55 6.55 0.45 0.36 1.34 0.95 0.39

Corr_Variance 1.01 41.36 0.17 0.68 2.40 1.41 0.14

Bias Factor 0.90 1.84 1.38 0.76 2.61 1.37 1.15


5/144


36 Km Grid

Performance Measure HNO3 SO2 NH4 SO4 NO3 TNO3 NH4-FN N

Paired_Peak -0.02 0.47 -0.03 -0.22 0.22 0.12 -0.08

Coef_Determ 0.10 0.67 0.54 0.00 0.31 0.35 0.34 Norm_Mean_Err 64.96 63.57 48.00 32.94 104.26 44.54 31.61

Rt_Mean_Sqr_Err 1.08 6.11 0.59 0.85 2.14 1.68 0.49

Frac_Gross_Err 74.76 51.05 40.55 38.10 79.70 40.08 31.37

Mean_Abs_G_Err 0.92 5.07 0.49 0.70 1.80 1.39 0.41

Mean_Norm_G_Err 75.32 82.84 59.01 30.69 228.52 62.22 39.86

Mean_Bias -0.38 4.97 0.47 -0.61 1.65 1.28 0.25

Mean_Norm_Bias -0.43 82.00 57.45 -24.54 217.76 59.70 29.12

Mean_Fract_Bias -36.81 50.13 38.93 -32.62 62.50 37.40 18.89

Norm_Mean_Bias -26.59 62.35 46.14 -28.88 95.73 40.88 19.28

Observed_Mean 1.42 7.98 1.02 2.12 1.72 3.12 1.29

Predicted_Mean 1.05 12.95 1.49 1.50 3.37 4.40 1.54

SD_Observation 0.59 4.18 0.46 0.51 1.22 1.26 0.48

SD_Prediction 0.68 6.12 0.51 0.33 1.61 1.18 0.45

Corr_Variance 1.17 37.34 0.35 0.72 4.58 2.82 0.24

Bias Factor 1.00 1.82 1.57 0.75 3.18 1.60 1.29


6/144


Table A-3. CMAQ Performance for the 1-21 January 2002 Episode: STN Network.

12 Km Grid

Performance Measure SO4 NO3 NH4 OC EC SOIL TCM

Paired_Peak 0.01 0.77 0.29 -0.47 0.38 9.10 -0.42

Coef_Determ 0.36 0.24 0.25 0.61 0.06 0.45 0.50

Norm_Mean_Err 35.88 89.64 56.78 41.60 107.78 1339.27 38.70

Rt_Mean_Sqr_Err 1.36 2.24 0.93 3.48 1.58 8.06 4.02

Frac_Gross_Err 48.12 78.13 57.26 56.69 84.58 168.58 51.27

Mean_Abs_G_Err 1.11 1.62 0.73 2.45 1.19 6.39 2.70

Mean_Norm_G_Err 39.54 116.15 72.36 52.93 157.44 1873.55 55.55

Mean_Bias -0.63 1.07 0.37 -1.61 0.66 6.34 -0.94

Mean_Norm_Bias -18.07 72.56 41.08 -8.37 132.53 1861.80 11.23

Mean_Fract_Bias -32.87 5.95 8.74 -33.43 40.51 147.49 -16.71

Norm_Mean_Bias -20.53 59.13 28.76 -27.34 60.04 1327.93 -13.53

Observed_Mean 3.08 1.81 1.29 5.88 1.10 0.48 6.98

Predicted_Mean 2.45 2.88 1.66 4.27 1.77 6.82 6.04

SD_Observation 1.16 1.12 0.61 4.85 1.01 0.44 5.61

SD_Prediction 1.50 2.29 0.99 3.00 1.33 5.33 3.99

Corr_Variance 1.86 5.02 0.87 12.10 2.50 65.04 16.13

Bias Factor 0.82 1.73 1.41 0.92 2.33 19.62 1.11


7/144


36 Km Grid


Paired_Peak 0.39 1.09 0.52 -0.59 -0.29 6.25 -0.58

Coef_Determ 0.28 0.27 0.23 0.51 0.07 0.23 0.46Norm_Mean_Err 41.76 94.90 58.30 47.34 72.91 1289.72 42.91

Rt_Mean_Sqr_Err 1.49 2.43 0.96 4.14 1.12 7.61 4.60

Frac_Gross_Err 52.45 87.72 59.68 62.38 68.28 159.70 54.62

Mean_Abs_G_Err 1.29 1.72 0.75 2.78 0.80 6.16 3.00

Mean_Norm_G_Err 45.23 120.59 76.90 51.07 95.87 1875.78 48.40

Mean_Bias -0.62 1.13 0.36 -2.20 0.21 6.12 -1.99

Mean_Norm_Bias -16.17 69.25 41.45 -21.36 68.25 1868.83 -9.64

Mean_Fract_Bias -31.95 -2.60 5.74 -44.57 21.30 149.91 -31.79

Norm_Mean_Bias -20.03 62.38 27.80 -37.48 19.13 1282.37 -28.53

Observed_Mean 3.08 1.81 1.29 5.88 1.10 0.48 6.98

Predicted_Mean 2.46 2.94 1.64 3.67 1.31 6.60 4.99

SD_Observation 1.16 1.12 0.61 4.85 1.01 0.44 5.61

SD_Prediction 1.57 2.55 1.03 2.42 0.83 4.76 3.10

Corr_Variance 2.23 5.93 0.93 17.10 1.26 57.88 21.15

Bias Factor 0.84 1.69 1.41 0.79 1.68 19.69 0.90


8/144


Table A-4. CMAQ Performance for the 1-21 January 2002 Episode: AQS Network

with UCR Tools.

12 Km Grid

Performance Measure O3 CO NO2 SO2

Paired_Peak -0.26 0.33 -0.04 -0.08

Coef_Determ 0.31 0.09 0.36 0.03

Norm_Mean_Err 48.94 63.56 50.80 121.65

Rt_Mean_Sqr_Err 0.01 0.73 0.01 0.01

Frac_Gross_Err 78.66 68.48 53.15 92.08

Mean_Abs_G_Err 0.01 0.46 0.01 0.01

Mean_Norm_G_Err 109.79 76.91 94.10 219.67

Mean_Bias -0.01 -0.15 0.00 0.00

Mean_Norm_Bias 35.40 17.20 74.62 189.59

Mean_Fract_Bias -33.65 -13.62 27.49 52.57

Norm_Mean_Bias -25.05 -21.14 24.70 58.12

Observed_Mean 0.02 0.72 0.01 0.01

Predicted_Mean 0.02 0.57 0.02 0.01

SD_Observation 0.01 0.64 0.01 0.01

SD_Prediction 0.01 0.56 0.01 0.01

Corr_Variance 0.00 0.53 0.00 0.00

Bias Factor 1.30 1.12 1.73 2.58


9/144


36 Km Grid


Paired_Peak -0.23 -0.31 -0.21 -0.69

Coef_Determ 0.24 0.07 0.34 0.06

Norm_Mean_Err 49.66 64.15 50.90 92.25

Rt_Mean_Sqr_Err 0.01 0.72 0.01 0.01

Frac_Gross_Err 76.48 71.39 54.38 88.15

Mean_Abs_G_Err 0.01 0.46 0.01 0.01

Mean_Norm_G_Err 132.08 73.92 95.23 162.29

Mean_Bias 0.00 -0.22 0.00 0.00

Mean_Norm_Bias 66.35 6.81 73.42 130.99

Mean_Fract_Bias -22.60 -22.10 24.97 46.93

Norm_Mean_Bias -16.99 -29.81 21.44 22.23

Observed_Mean 0.02 0.72 0.01 0.01

Predicted_Mean 0.02 0.51 0.02 0.01

SD_Observation 0.01 0.64 0.01 0.01

SD_Prediction 0.01 0.48 0.01 0.01

Corr_Variance 0.00 0.52 0.00 0.00

Bias Factor 1.60 1.02 1.72 2.06


10/144


Table A-5. CMAQ Performance for the 1-21 January 2002 Episode: NADP Network.

12 Km Grid

Performance Measure SO4_wdep NO3_wdep NH4_wdep

Paired_Peak -0.41 1.81 2.79

Coef_Determ 0.12 0.10 0.02

Norm_Mean_Err 70.72 76.21 151.91

Rt_Mean_Sqr_Err 0.21 0.23 0.08

Frac_Gross_Err 118.68 92.68 98.51

Mean_Abs_G_Err 0.16 0.13 0.04

Mean_Norm_G_Err 76.62 105.85 304.47

Mean_Bias -0.12 0.00 0.03

Mean_Norm_Bias -47.95 22.30 241.11

Mean_Fract_Bias -102.03 -48.11 -9.19

Norm_Mean_Bias -54.05 -1.87 85.72

Observed_Mean 0.23 0.17 0.03

Predicted_Mean 0.10 0.17 0.05

SD_Observation 0.16 0.12 0.02

SD_Prediction 0.13 0.23 0.08

Corr_Variance 0.04 0.05 0.01

Bias Factor 0.52 1.22 3.41


11/144


36 Km Grid


Paired_Peak -0.62 -0.20 0.30

Coef_Determ 0.17 0.15 0.06

Norm_Mean_Err 64.72 64.98 77.98


Frac_Gross_Err 108.77 93.36 81.07

Mean_Abs_G_Err 0.15 0.11 0.02

Mean_Norm_G_Err 87.57 103.25 109.92

Mean_Bias -0.11 -0.03 0.00

Mean_Norm_Bias -22.90 22.88 42.43

Mean_Fract_Bias -84.95 -46.75 -27.48

Norm_Mean_Bias -48.49 -18.10 0.90






Bias Factor 0.77 1.23 1.42


12/144


Table A-6. CMAQ Performance for the 1-21 January 2002 Episode: SEARCH Daily

Network.

12 Km Grid

Performance Measure SO4 NO3 NH4 EC OCM SOIL PM25

Paired_Peak 0.34 1.15 0.43 -0.12 0.04 17.38 0.38

Coef_Determ 0.16 0.47 0.44 0.32 0.29 0.38 0.49

Norm_Mean_Err 39.92 101.20 40.15 52.97 42.64 2658.70 58.38

Rt_Mean_Sqr_Err 1.02 1.67 0.58 0.75 2.35 8.80 9.40

Frac_Gross_Err 41.99 76.57 37.38 47.09 44.32 182.50 44.23

Mean_Abs_G_Err 0.80 1.24 0.44 0.48 1.75 6.50 6.73

Mean_Norm_G_Err 42.52 165.81 44.03 64.42 49.83 3073.41 65.73

Mean_Bias -0.12 1.07 0.22 0.05 0.04 6.50 5.73

Mean_Norm_Bias 0.76 143.38 24.49 35.79 15.83 3073.41 55.48

Mean_Fract_Bias -12.30 42.74 12.13 9.19 -1.39 182.50 31.82

Norm_Mean_Bias -6.16 87.15 20.57 5.52 0.90 2658.70 49.72

Observed_Mean 2.00 1.22 1.09 0.90 4.11 0.24 11.52

Predicted_Mean 1.87 2.29 1.32 0.95 4.15 6.75 17.25

SD_Observation 0.77 1.02 0.50 0.82 2.48 0.24 6.03

SD_Prediction 1.03 1.75 0.72 0.78 2.55 6.10 10.31

Corr_Variance 1.03 2.80 0.34 0.56 5.54 77.46 88.34

Bias Factor 1.01 2.43 1.24 1.36 1.16 31.73 1.55


13/144


14/144


Table A-7. CMAQ Performance for the 10-21 July 1999 Episode: SEARCH Hourly Network.

12 Km Grid

Performance Measure SO4 NO3 NH4 TCM PM25_1 PM25_2

Paired_Peak -0.15 0.53 -0.43 -0.47 0.18 0.17

Coef_Determ 0.03 0.24 0.16 0.21 0.21 0.15

Norm_Mean_Err 62.33 119.55 83.79 85.01 106.01 80.52

Rt_Mean_Sqr_Err 1.70 2.46 0.99 4.68 15.49 14.81

Frac_Gross_Err 73.28 101.47 75.46 79.93 76.71 66.21

Mean_Abs_G_Err 1.32 1.64 0.72 3.25 10.58 9.88

Mean_Norm_G_Err 98.44 316.97 177.72 164.11 261.26 97.11

Mean_Bias -0.34 1.22 0.44 1.46 7.76 5.40

Mean_Norm_Bias 34.88 271.33 150.90 131.13 236.43 61.69

Mean_Fract_Bias -23.29 20.52 31.01 26.39 36.63 10.75

Norm_Mean_Bias -15.83 88.60 51.41 38.17 77.77 44.01Observed_Mean 2.12 1.37 0.86 3.82 9.98 12.28

Predicted_Mean 1.79 2.59 1.30 5.28 17.74 17.68

SD_Observation 1.29 1.41 0.70 4.18 8.20 7.52

SD_Prediction 1.30 2.44 0.89 4.33 14.97 14.89

Corr_Variance 2.90 6.04 0.98 21.87 239.92 219.38

Bias Factor 1.35 3.71 2.51 2.31 3.36 1.62


15/144


16/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixA_Jan02.doc A-16

Figure A-1. CMAQ Model Performance Evaluation Results for IMPROVE Sites Over the VISTAS

Domain for the 1-21 January 2002 Episode (all sites/all days): Particulate Sulfate (top

left), Particulate Nitrate (top right), Particulate Organic Carbon (OC) (bottom left) and

Particulate Elemental Carbon (EC) (bottom right).


17/144


Figure A-1. Concluded. Particulate Ammonia (top left), Soil (top right), Crustal Material

(bottom left) and PM2.5 (bottom right).


18/144


Figure A-2. CMAQ Model Performance Evaluation Results for CASTNet Sites Over the VISTAS

Domain for the 1-21 January 2002 Episode (all sites/all days): Gaseous HNO3 (top

left), Gaseous SO2 (top right), Particulate Ammonia (NH4) (bottom left) and

Particulate Nitrate (NO3) (bottom right).


19/144


20/144


21/144


Figure A-3. Concluded. Particulate Ammonia (top left), Soil (top right), Total Carbon Mass



22/144


23/144


24/144


Figure A-6. CMAQ Model Performance Evaluation Results for Daily SEARCH Sites Over the

VISTAS Domain for the 1-21 January 2002 Episode (all sites/all days): Elemental

Carbon (top left), Particulate Ammonia (top right), Particulate Nitrate (bottom left) and

Organic Carbon Mass (bottom right).


25/144


26/144


Figure A-7. CMAQ Model Performance Evaluation Results for Hourly SEARCH Sites Over the

VISTAS Domain for the 1-21 January 2002 Episode (all sites/all days): Particulate

Ammonium (top left), Particulate Nitrate (top right), PM2.5 Mass (bottom left) and

(bottom right).


27/144


Figure A-8. 24-Hr Particulate Sulfate (top) and Particulate Nitrate (bottom) Time Series at

Shenandoah National Park IMPROVE Site observed (red) and CMAQ estimated

using 36 km (blue) and 12 km (purple) grids.


28/144


29/144


30/144


31/144



James River Face Area IMPROVE Site observed (red) and CMAQ estimated using

36 km (blue) and 12 km (purple) grids.


32/144



Okefenokee Wildlife Area IMPROVE Site observed (red) and CMAQ estimated



33/144



Everglades National Park IMPROVE Site observed (red) and CMAQ estimated



34/144

APPENDIX B:

RESULTS OF THE 10-21 JULY 1999 CMAQOPERATIONAL PERFORMANCE EVALUATION


35/144


36/144


37/144


38/144


39/144


40/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixB (Jul '99 Tables and Figs).doc

Mean_Abs_G_Err 0.01 0.41 0.01 0.01

Mean_Norm_G_Err 103.92 76.67 110.12 199.34

Mean_Bias 0.00 -0.29 0.00 0.00

Mean_Norm_Bias 71.37 -15.67 62.48 155.86

Mean_Fract_Bias 3.08 -59.88 -2.18 17.18

Norm_Mean_Bias 3.88 -46.75 25.00 19.16

Observed_Mean 0.04 0.62 0.01 0.01

Predicted_Mean 0.04 0.33 0.02 0.01

SD_Observation 0.02 0.46 0.01 0.01

SD_Prediction 0.02 0.43 0.02 0.01

Corr_Variance 0.00 0.38 0.00 0.00

Bias Factor 1.71 0.84 1.62 2.56


41/144


Table B-3b. CMAQ Performance for the 10-21 July 1999 Episode: AQS Network with MAPS Tools.12 Km Grid

Date Day ATS AS AT AU A-MEAN N. Bias Bias N. Error Error Var M

10-Jul 191 -44.9 -44.9 -32.7 56.6 18.4 -10.3 -6.1 21.6 11.2 206.6

11-Jul 192 -72.1 -59.4 -61.8 41.8 15.7 -17.7 -9.2 21.6 10.9 153.012-Jul 193 -36.4 -35.0 -16.2 15.5 16.0 -18.3 -10.3 23.6 12.8 195.7

13-Jul 194 -27.2 -24.4 -22.5 27.4 20.7 -13.3 -7.8 24.4 13.2 223.9

14-Jul 195 -7.5 -7.4 -7.1 48.3 17.8 -2.2 -2.2 22.2 12.4 241.6

15-Jul 196 -24.3 -24.3 -16.2 19.3 15.2 -4.5 -3.9 19.5 11.5 217.1

16-Jul 197 -54.0 -53.7 -30.8 34.0 17.8 -10.2 -7.2 20.6 13.0 228.0

17-Jul 198 -32.2 -26.4 -18.6 98.3 15.1 -8.4 -5.7 20.7 12.3 236.2

18-Jul 199 -25.1 -25.1 8.7 72.3 16.9 -8.0 -6.0 20.5 12.2 215.7

19-Jul 200 -26.2 -25.5 -6.3 57.3 18.8 -10.7 -7.2 22.7 13.3 234.0

20-Jul 201 -38.9 -37.2 -27.2 37.3 20.5 -12.3 -8.4 24.4 14.4 272.7

21-Jul 202 -1.5 -0.9 -0.1 44.6 19.2 -8.1 -5.7 22.6 13.1 258.5

Average -32.5 -30.3 -19.2 46.1 17.7 -10.3 -6.6 22.0 12.5 223.6

36 Km GridDate Day ATS AS AT AU A-MEAN N. Bias Bias N. Error Error Var M

10-Jul 191 -34.2 -31.4 -12.0 146.9 15.2 -4.2 -2.9 22.6 11.6 218.3

11-Jul 192 -32.8 -25.2 1.5 41.2 16.6 -7.3 -4.3 18.1 9.2 162.1

12-Jul 193 -26.9 -26.9 -17.7 95.0 15.9 -10.9 -6.5 21.4 11.7 239.9

13-Jul 194 -19.3 -17.0 11.8 95.5 18.9 -3.9 -2.9 23.4 12.5 259.2

14-Jul 195 -30.5 -28.8 -26.5 63.1 18.4 5.7 2.0 21.8 12.0 213.8

15-Jul 196 -21.1 -21.1 -7.5 32.2 14.7 2.6 0.1 18.0 10.3 176.6

16-Jul 197 -41.0 -41.0 -19.6 41.7 15.0 -1.7 -2.2 18.1 11.1 198.2

17-Jul 198 -44.7 -25.2 -0.5 94.0 14.7 1.3 -0.4 21.6 12.7 278.1

18-Jul 199 -19.3 -19.3 31.3 98.8 15.8 2.3 -0.1 21.2 12.0 233.3

19-Jul 200 -22.9 -22.9 29.2 105.5 16.7 -0.5 -1.3 20.7 11.9 228.6

20-Jul 201 -36.3 -34.1 -34.7 34.8 18.4 -6.6 -5.3 23.1 13.6 273.921-Jul 202 -3.7 -3.1 2.2 17.7 20.2 -2.4 -2.7 22.8 13.2 280.5

Average -27.7 -24.7 -3.5 72.2 16.7 -2.1 -2.2 21.1 11.8 230.2


42/144


Table B-4. CMAQ Performance for the 10-21 July 1999 Episode: NADP Network.

12 Km Grid


Paired_Peak -0.41 -0.11 -0.51

Coef_Determ 0.11 0.03 0.11

Norm_Mean_Err 87.54 80.56 77.00


Frac_Gross_Err 98.73 94.40 89.71

Mean_Abs_G_Err 0.38 0.22 0.05

Mean_Norm_G_Err 335.11 243.48 255.62

Mean_Bias -0.06 -0.03 -0.02

Mean_Norm_Bias 270.43 175.47 192.20

Mean_Fract_Bias -6.60 -15.90 -14.20

Norm_Mean_Bias -13.36 -12.22 -30.03Observed_Mean 0.43 0.28 0.07





Bias Factor 3.70 2.75 2.92

36 Km Grid


Paired_Peak -0.17 0.60 -0.29

Coef_Determ 0.06 0.00 0.04

Norm_Mean_Err 100.97 95.90 86.13


Frac_Gross_Err 95.21 88.37 89.82


43/144


44/144


45/144


Mean_Abs_G_Err 2.69 0.29 0.76 0.49 1.42 2.55 5.84

Mean_Norm_G_Err 87.88 81.50 43.88 49.40 36.85 925.72 51.47

Mean_Bias 2.17 -0.24 -0.47 -0.37 -0.92 2.47 1.89

Mean_Norm_Bias 74.81 -74.69 -13.28 -26.35 -24.22 917.33 18.57

Mean_Fract_Bias 34.61 -137.87 -29.73 -48.74 -40.26 126.68 -0.14

Norm_Mean_Bias 40.33 -62.15 -21.70 -42.04 -29.28 633.68 12.59

Observed_Mean 5.38 0.38 2.15 0.88 3.14 0.39 14.98

Predicted_Mean 7.54 0.14 1.69 0.51 2.22 2.86 16.86

SD_Observation 4.10 0.36 1.50 0.79 1.51 0.29 9.70

SD_Prediction 4.98 0.32 1.31 0.52 1.75 2.48 11.94

Corr_Variance 14.10 0.17 1.10 0.62 3.91 12.11 55.80

Bias Factor 1.75 0.25 0.87 0.74 0.76 10.17 1.19


46/144


Table B-6. CMAQ Performance for the 10-21 July 1999 Episode: SEARCH Hourly Network.

12 Km Grid

Performance Measure O3 CO SO2 NO HNO3 NOy TCM

Paired_Peak -0.19 0.59 -0.83 -0.30 -0.13 -0.07 0.25

Coef_Determ 0.31 0.19 0.06 0.19 0.48 0.45 0.28

Norm_Mean_Err 39.64 61.09 121.23 112.60 62.05 62.28 47.48

Rt_Mean_Sqr_Err 0.02 0.20 0.01 0.01 0.00 0.01 3.26

Frac_Gross_Err 50.18 51.40 109.50 125.08 79.97 54.01 47.56

Mean_Abs_G_Err 0.01 0.11 0.00 0.00 0.00 0.01 2.42

Mean_Norm_G_Err 117.59 56.02 597.40 155.00 121.95 82.92 65.97

Mean_Bias 0.00 -0.01 0.00 0.00 0.00 0.00 0.79

Mean_Norm_Bias 79.83 -3.46 537.58 39.60 59.20 57.25 39.35

Mean_Fract_Bias -3.67 -26.77 10.26 -82.86 -17.75 18.69 13.02

Norm_Mean_Bias -6.56 -4.18 2.11 10.45 -22.50 17.57 15.49Observed_Mean 0.03 0.19 0.00 0.00 0.00 0.01 5.10

Predicted_Mean 0.03 0.18 0.00 0.00 0.00 0.01 5.89

SD_Observation 0.02 0.10 0.01 0.01 0.00 0.02 2.72

SD_Prediction 0.02 0.22 0.00 0.01 0.00 0.02 3.58

Corr_Variance 0.00 0.04 0.00 0.00 0.00 0.00 10.61

Bias Factor 1.80 0.97 6.38 1.40 1.59 1.57 1.39

36 Km Grid

Performance Measure O3 CO SO2 NO HNO3 NOy TCM

Paired_Peak 0.22 0.72 -0.95 -0.62 -0.19 -0.01 0.76

Coef_Determ 0.33 0.22 0.05 0.16 0.53 0.44 0.28

Norm_Mean_Err 39.90 61.64 103.21 87.62 58.14 59.69 80.02

Rt_Mean_Sqr_Err 0.02 0.21 0.01 0.01 0.00 0.01 5.66

Frac_Gross_Err 48.16 49.99 104.18 122.12 74.20 56.26 60.19


47/144


Mean_Abs_G_Err 0.01 0.11 0.00 0.00 0.00 0.01 4.08

Mean_Norm_G_Err 157.82 55.59 594.93 117.69 153.90 95.74 96.39

Mean_Bias 0.00 0.00 0.00 0.00 0.00 0.00 2.97

Mean_Norm_Bias 129.18 -1.43 555.82 1.85 100.99 72.92 75.99

Mean_Fract_Bias 7.49 -24.90 42.33 -87.88 -5.26 25.29 30.94

Norm_Mean_Bias 3.78 -1.05 -14.50 -43.82 -11.49 12.44 58.13

Observed_Mean 0.03 0.19 0.00 0.00 0.00 0.01 5.10

Predicted_Mean 0.04 0.18 0.00 0.00 0.00 0.01 8.07

SD_Observation 0.02 0.10 0.01 0.01 0.00 0.02 2.72

SD_Prediction 0.02 0.24 0.00 0.00 0.00 0.02 5.68

Corr_Variance 0.00 0.04 0.00 0.00 0.00 0.00 32.05

Bias Factor 2.29 0.99 6.56 1.02 2.01 1.73 1.76


48/144


49/144


50/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixB (Jul '99 Tables and Figs).doc B-16

AQS Mean Fractional Bias

-70.0

-60.0

-50.0

-40.0

-30.0

-20.0

-10.0

0.0

10.0

20.0

30.0

O3 CO NO2 SO2

FractionalBia

s

CMAQ 36 km

CMAQ 12 km

AQS Fractional Gross Error

0.0

20.0

40.0

60.0

80.0

100.0

O3 CO NO2

SO2

FractionalGrossError

CMAQ 36 km

CMAQ 12 km

AQS Normalized Mean Bias, %

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

80.0

O3 CO NO2

SO2

NormalizedMeanBias,%

CMAQ 36 km

CMAQ 12 km

Figure B-3a. CMAQ Model Performance Evaluation Results for AQS Sites Over the VISTAS Domain for the 10-21 July

1999 Episode (all sites/all days): Fractional Bias (top left), Fractional Error (top right), and Mean

Normalized Bias (bottom left).


51/144


AQS Mean Normalized Bias in 1-Hr Ozone, (%).

-20.0

-15.0

-10.0

-5.0

0.0

5.0

10.0

10

-Jul

11-Jul

12

-Jul

13

-Jul

14-Jul

15

-Jul

16-Jul

17

-Jul

18

-Jul

19

-Jul

20

-Jul

21-Jul

Average

Bias,%

CMAQ 36 Km

CMAQ 12 Km

AQS Mean Normalized Gross Error in 1Hr Ozone,

(%).

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

10-Jul

12-Jul

14-Jul

16-Jul

18-Jul

20-Jul

Avera

ge

GrossE

rror,%

CMAQ 36 Km

CMAQ 12 Km

AQS Average Ozone Accuracy over All Stations, (%).

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

25.0

30.0

10-Jul

11-Jul

12-Jul

13-Jul

14-Jul

15-Jul

16-Jul

17-Jul

18-Jul

19-Jul

20-Jul

21-Jul

AverageA

veragePeakPredict

ion

Accuracy,%

CMAQ 36 Km

CMAQ 12 Km

Figure B-3b. CMAQ Model Performance Evaluation Results for AQS Sites Over the VISTAS Domain for the 10-21 July

1999 Episode (all sites/all days): Mean Normalized Ozone Bias (top left), Mean Normalized Ozone Gross

Error (top right), and Average Ozone Prediction Accuracy over All Stations (bottom left).


52/144


53/144


SEARCH Daily Mean Fractional Bias

-150.0

-125.0

-100.0

-75.0

-50.0

-25.0

0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4 EC

OCM

SOIL

PM25

MeanFractionalBia

s

CMAQ 36 km

CMAQ 12 km

SEARCH Daily Fractional Gross Error

0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4 EC

OCM

SOIL

PM25

FractionalGross

Error

CMAQ 36 km

CMAQ 12 km

SEARCH Daily Normalized Mean Bias, %

-100.0

-50.0

0.0

50.0

100.0

150.0

SO4

NO3

NH4 EC

OCM

SOIL

PM25


CMAQ 36 km

CMAQ 12 km

Figure B-5. CMAQ Model Performance Evaluation Results for Daily SEARCH Sites Over the VISTAS Domain

for the 10-21 July 1999 Episode (all sites/all days): Fractional Bias (top left), Fractional Error (top

right), and Mean Normalized Bias (bottom left).


54/144


SEARCH Hourly Mean Fractional Bias

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

O3 CO SO2

NOHN

O3 NOy

TCM

PM2.5

MeanFractionalB

ias

CMAQ 36 km

CMAQ 12 km

SEARCH Hourly Fractional Gross Error

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

O3 CO SO2

NOHN

O3 NOy

TCM

PM2.5

FractionalGros

sError

CMAQ 36 km

CMAQ 12 km

SEARCH Hourly Normalized Mean Bias, %

-60.0

-50.0

-40.0

-30.0

-20.0

-10.0

0.0

10.0

20.0

30.0

40.0

50.0

60.0

O3 CO SO2

NOHN

O3 NOy

TCM

PM2.5


CMAQ 36 km

CMAQ 12 km

Figure B-6. CMAQ Model Performance Evaluation Results for Hourly SEARCH Sites Over the VISTAS Domain for

the 10-21 July 1999 Episode (all sites/all days): Fractional Bias (top left), Fractional Error (top right), and

Mean Normalized Bias (bottom left).


55/144


Figure B-7. CMAQ Model Performance Evaluation Results for IMPROVE Sites Over the VISTAS Domain for the 10-

21 July 1999 Episode (all sites/all days): Particulate Sulfate (top left), Particulate Nitrate (top right),

Particulate Organic Carbon (OC) (bottom left) and Particulate Elemental Carbon (EC) (bottom right).


56/144


Figure B-7. Concluded. Particulate Ammonia (top left), Soil (top right), Coarse Mass (bottom left) and PM2.5 (bottom

right).


57/144


58/144


59/144


60/144


61/144


Figure B-11. CMAQ Model Performance Evaluation Results for Daily SEARCH Sites Over the VISTAS Domain for the

10-21 July 1999 Episode (all sites/all days): Elemental Carbon (top left), Particulate Ammonia (top right),

Particulate Nitrate (bottom left) and Organic Carbon Mass (bottom right).


62/144


Figure B-11. Concluded. PM2.5 (top left), Sulfate (top right), and Soil (bottom left).


63/144


Figure B-12. CMAQ Model Performance Evaluation Results for Hourly SEARCH Sites Over the VISTAS Domain for

the 10-21 July 1999 Episode (all sites/all days): Carbon Monoxide (top left), Nitric Acid (top right), Nitric

Oxide (bottom left) and NOy (bottom right).


64/144


Figure B-13. Mean Normalized Bias in 1-Hr Ozone Concentrations Across the VISTAS Domain: 12 Km grid (top)

and 36 km Grid (bottom).


65/144


Figure B-14. Mean Normalized Gross Error in 1-Hr Ozone Concentrations Across the VISTAS Domain: 12 Km grid

(top) and 36 km Grid (bottom).


66/144


67/144


68/144


Figure B-17. 1-Hr Ozone Time Series at Two Monitors within the VISTAS 12 Km Domain: Knoxville, TN (top) and

Forsyth, NC (bottom).


69/144


Figure B-18. Daily Maximum 1-Hr Ozone Fields (ppb) for the 10-21 July 1999 Episode on the 36 Km Grid: 14 July

(top left), 16 July (top right), 18 July (bottom left) and 20 July (bottom right).


70/144

APPENDIX C:

RESULTS OF THE 13-27 JULY 2001 CMAQOPERATIONAL PERFORMANCE EVALUATION


71/144


72/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixC_Jul01.doc

36 Km Grid


Paired_Peak 0.73 -0.41 -0.03 0.43 -0.36 -0.45

Coef_Determ 0.71 0.01 0.71 0.41 0.29 0.12

Norm_Mean_Err 30.10 89.17 39.06 47.63 39.66 130.43

Rt_Mean_Sqr_Err 3.05 0.43 1.25 1.60 0.19 2.05

Frac_Gross_Err 35.91 127.79 60.39 62.98 49.60 104.15

Mean_Abs_G_Err 1.93 0.28 0.97 1.27 0.13 1.51

Mean_Norm_G_Err 43.39 93.66 44.58 51.05 61.61 222.10

Mean_Bias 0.52 -0.15 -0.83 -0.41 -0.06 0.26

Mean_Norm_Bias 17.09 -31.14 -32.48 -16.79 16.15 186.57

Mean_Fract_Bias 2.37 -100.01 -51.40 -39.52 -12.73 41.51

Norm_Mean_Bias 8.13 -49.33 -33.21 -15.25 -17.83 22.71

Observed_Mean 6.42 0.31 2.50 2.67 0.33 1.16

Predicted_Mean 6.94 0.16 1.67 2.26 0.27 1.42

SD_Observation 4.64 0.33 1.73 1.30 0.21 1.50

SD_Prediction 5.54 0.27 1.39 2.01 0.16 0.93

Corr_Variance 9.27 0.18 1.55 2.55 0.04 4.19

Bias Factor 1.17 0.69 0.68 0.83 1.16 2.87


73/144


74/144


75/144


Table C-3. CMAQ Performance for the 13-27 July 2001 Episode: STN Network.

12 Km Grid


Paired_Peak 0.21 1.73 0.39 0.06 -0.04 1.86 0.03

Coef_Determ 0.80 0.00 0.72 0.68 0.18 0.00 0.70

Norm_Mean_Err 36.71 78.62 48.50 30.08 100.43 412.89 25.64

Rt_Mean_Sqr_Err 3.17 0.68 1.11 2.38 0.71 6.17 2.30

Frac_Gross_Err 33.80 100.29 58.29 43.32 70.36 136.46 34.09

Mean_Abs_G_Err 2.45 0.41 0.85 1.57 0.56 4.53 1.48

Mean_Norm_G_Err 42.39 93.52 163.23 38.36 143.27 705.19 34.46

Mean_Bias 1.63 -0.15 0.67 -1.06 0.47 3.87 -0.59

Mean_Norm_Bias 28.04 -15.16 158.01 -10.76 138.13 694.64 3.34

Mean_Fract_Bias 16.97 -79.46 52.03 -24.42 64.55 118.33 -8.23

Norm_Mean_Bias 24.35 -29.32 38.13 -20.40 84.23 353.06 -10.27

Observed_Mean 6.68 0.52 1.76 5.21 0.56 1.10 5.77

Predicted_Mean 8.30 0.36 2.43 4.15 1.03 4.97 5.18

SD_Observation 4.54 0.23 1.49 3.58 0.49 1.45 3.93

SD_Prediction 5.89 0.63 1.65 3.60 0.50 4.49 3.78

Corr_Variance 10.02 0.46 1.22 5.66 0.51 38.03 5.27

Bias Factor 1.28 0.85 2.58 0.89 2.38 7.95 1.03


76/144


36 Km Grid


Paired_Peak 0.08 0.46 0.18 0.16 -0.37 0.97 0.11

Coef_Determ 0.77 0.12 0.79 0.70 0.43 0.02 0.75Norm_Mean_Err 28.91 76.19 36.42 34.23 58.97 329.46 29.91

Rt_Mean_Sqr_Err 2.56 0.47 0.83 2.53 0.40 4.68 2.43

Frac_Gross_Err 30.45 106.57 49.43 50.38 55.31 128.42 42.70

Mean_Abs_G_Err 1.93 0.39 0.64 1.78 0.33 3.61 1.73

Mean_Norm_G_Err 35.90 82.23 143.38 37.42 83.79 592.56 33.44

Mean_Bias 0.98 -0.15 0.45 -1.17 0.15 2.90 -1.03

Mean_Norm_Bias 22.50 -25.67 138.25 -28.02 64.73 581.07 -21.39

Mean_Fract_Bias 14.11 -77.48 43.68 -42.39 30.67 107.50 -32.40

Norm_Mean_Bias 14.68 -29.64 25.47 -22.48 26.34 264.13 -17.76

Observed_Mean 6.68 0.52 1.76 5.21 0.56 1.10 5.77

Predicted_Mean 7.66 0.36 2.21 4.04 0.71 3.99 4.75

SD_Observation 4.54 0.23 1.49 3.58 0.49 1.45 3.93

SD_Prediction 4.90 0.46 1.45 4.11 0.33 3.19 4.38

Corr_Variance 6.55 0.22 0.69 6.39 0.16 21.90 5.88

Bias Factor 1.23 0.74 2.38 0.72 1.65 6.81 0.79


77/144


Table C-4. CMAQ Performance for the 13-27 July 2001 Episode: AQS Network with

UCR Tools.

12 Km Grid


Paired_Peak -0.22 -0.37 0.85 -0.54

Coef_Determ 0.28 0.05 0.29 0.06

Norm_Mean_Err 45.50 62.92 77.49 113.21

Rt_Mean_Sqr_Err 0.02 0.46 0.01 0.01

Frac_Gross_Err 61.21 80.98 75.96 89.12

Mean_Abs_G_Err 0.02 0.34 0.01 0.01

Mean_Norm_G_Err 112.13 64.48 133.92 230.75

Mean_Bias 0.00 -0.27 0.00 0.00

Mean_Norm_Bias 68.86 -30.44 85.94 182.87

Mean_Fract_Bias -4.79 -64.40 1.27 12.72

Norm_Mean_Bias -6.76 -50.75 25.79 29.18

Observed_Mean 0.03 0.53 0.01 0.00

Predicted_Mean 0.03 0.26 0.01 0.01

SD_Observation 0.02 0.35 0.01 0.01

SD_Prediction 0.02 0.22 0.01 0.01

Corr_Variance 0.00 0.21 0.00 0.00

Bias Factor 1.69 0.70 1.86 2.83


78/144


79/144


Table C-5. CMAQ Performance for the 13-27 July 2001 Episode: NADP Network.

12 Km Grid


Paired_Peak 0.20 -0.55 0.47

Coef_Determ 0.00 0.00 0.00

Norm_Mean_Err 88.55 70.34 78.38


Frac_Gross_Err 85.32 94.51 82.47

Mean_Abs_G_Err 0.49 0.23 0.06

Mean_Norm_G_Err 332.32 91.11 173.15

Mean_Bias 0.13 -0.18 0.01

Mean_Norm_Bias 290.56 -3.88 122.39

Mean_Fract_Bias 21.48 -57.20 4.74

Norm_Mean_Bias 22.93 -54.93 9.30






Bias Factor 3.91 0.96 2.22


80/144


81/144


Table C-6. CMAQ Performance for the 13-27 July 2001 Episode: SEARCH Daily Network.

12 Km Grid


Paired_Peak 0.67 1.18 0.11 -0.19 0.37 7.74 0.57

Coef_Determ 0.67 0.08 0.53 0.38 0.91 0.02 0.73

Norm_Mean_Err 45.94 93.56 31.62 46.77 39.42 957.85 29.74

Rt_Mean_Sqr_Err 4.21 0.52 1.23 0.64 2.94 6.40 8.25

Frac_Gross_Err 42.14 131.35 35.77 51.66 34.04 152.69 32.00

Mean_Abs_G_Err 3.12 0.27 0.79 0.41 2.44 4.50 5.79

Mean_Norm_G_Err 59.62 96.15 32.99 46.63 45.24 1610.36 34.96

Mean_Bias 2.56 -0.07 -0.18 -0.21 2.44 4.38 2.91

Mean_Norm_Bias 46.80 -39.86 -4.26 -14.93 45.24 1602.88 14.17

Mean_Fract_Bias 25.40 -112.13 -14.06 -31.70 34.04 140.69 4.35

Norm_Mean_Bias 37.68 -24.35 -7.19 -24.08 39.42 932.44 14.94Observed_Mean 6.80 0.29 2.51 0.89 6.19 0.47 19.49

Predicted_Mean 9.36 0.22 2.33 0.67 8.64 4.85 22.40

SD_Observation 3.79 0.27 1.61 0.75 2.95 0.45 9.55

SD_Prediction 5.65 0.52 1.67 0.61 4.19 4.71 14.04

Corr_Variance 17.73 0.27 1.51 0.41 8.64 40.94 68.07

Bias Factor 1.47 0.60 0.96 0.85 1.45 17.03 1.14


82/144


36 Km Grid


Paired_Peak 0.36 2.48 -0.22 -0.20 0.37 4.25 0.35

Coef_Determ 0.75 0.12 0.61 0.36 0.98 0.01 0.80

Norm_Mean_Err 35.90 123.17 29.42 49.79 50.89 731.03 24.20

Rt_Mean_Sqr_Err 3.11 0.81 1.09 0.67 3.36 4.46 6.27

Frac_Gross_Err 36.88 126.61 36.38 58.59 43.40 148.32 29.16

Mean_Abs_G_Err 2.44 0.36 0.74 0.44 3.15 3.43 4.72

Mean_Norm_G_Err 50.06 110.77 31.71 50.51 56.08 1372.94 28.98

Mean_Bias 1.89 0.07 -0.27 -0.24 3.15 3.34 1.33

Mean_Norm_Bias 38.19 -14.21 -7.35 -17.98 56.08 1365.46 6.03

Mean_Fract_Bias 21.71 -97.49 -17.81 -38.72 43.40 135.95 -1.97

Norm_Mean_Bias 27.76 23.96 -10.57 -26.55 50.89 709.83 6.85

Observed_Mean 6.80 0.29 2.51 0.89 6.19 0.47 19.49

Predicted_Mean 8.69 0.36 2.24 0.65 9.35 3.80 20.82

SD_Observation 3.79 0.27 1.61 0.75 2.95 0.45 9.55

SD_Prediction 4.85 0.86 1.57 0.63 3.99 2.96 12.92

Corr_Variance 9.69 0.66 1.19 0.45 11.31 19.85 39.32

Bias Factor 1.38 0.86 0.93 0.82 1.56 14.65 1.06


83/144


Table C-7. CMAQ Performance for the 10-21 July 1999 Episode: SEARCH Hourly Network.

12 Km Grid


Paired_Peak 0.52 0.58 1.13 -0.85 0.27 0.25

Coef_Determ 0.44 0.00 0.37 0.07 0.41 0.42

Norm_Mean_Err 48.05 147.60 71.57 84.34 48.86 46.83

Rt_Mean_Sqr_Err 5.81 1.38 2.04 7.81 12.41 14.54

Frac_Gross_Err 49.91 149.36 71.64 80.36 55.85 57.86

Mean_Abs_G_Err 4.08 0.61 1.39 4.03 8.54 10.55

Mean_Norm_G_Err 63.42 208.90 156.38 106.38 81.85 54.60

Mean_Bias 1.62 0.03 0.72 0.71 1.98 -0.09

Mean_Norm_Bias 34.37 77.14 121.37 39.34 39.02 -0.78

Mean_Fract_Bias 7.48 -99.13 17.93 -23.05 -7.91 -24.78

Norm_Mean_Bias 19.00 8.39 37.13 14.87 11.30 -0.38Observed_Mean 8.50 0.42 1.94 4.78 17.48 22.53

Predicted_Mean 10.12 0.45 2.66 5.49 19.46 22.45

SD_Observation 5.17 0.64 1.44 6.05 10.75 12.39

SD_Prediction 7.45 1.24 2.40 6.75 16.01 19.06

Corr_Variance 33.76 1.89 4.18 60.96 154.10 211.44

Bias Factor 1.34 1.77 2.21 1.39 1.39 0.99


84/144


36 Km Grid


Paired_Peak 0.13 1.28 0.62 -0.83 0.57 0.31

Coef_Determ 0.46 0.02 0.42 0.07 0.44 0.44

Norm_Mean_Err 40.42 207.15 68.87 91.58 47.04 43.36

Rt_Mean_Sqr_Err 4.82 2.13 1.81 8.15 11.77 13.07

Frac_Gross_Err 45.36 144.60 68.75 86.77 55.99 56.71

Mean_Abs_G_Err 3.44 0.86 1.33 4.38 8.23 9.77

Mean_Norm_G_Err 58.86 272.02 166.53 111.52 82.09 50.40

Mean_Bias 0.97 0.35 0.75 0.81 1.22 -1.62

Mean_Norm_Bias 32.39 152.25 138.37 40.17 37.70 -6.55

Mean_Fract_Bias 7.16 -79.58 25.57 -26.47 -10.24 -28.37

Norm_Mean_Bias 11.44 83.18 38.73 17.02 6.99 -7.20

Observed_Mean 8.50 0.42 1.94 4.78 17.48 22.53

Predicted_Mean 9.47 0.76 2.69 5.60 18.71 20.91

SD_Observation 5.17 0.64 1.44 6.05 10.75 12.39

SD_Prediction 6.28 2.09 2.16 7.17 15.58 17.31

Corr_Variance 23.20 4.55 3.29 66.39 138.48 170.77

Bias Factor 1.32 2.52 2.38 1.40 1.38 0.93


85/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixC_Jul01.doc C-15

Figure C-1. CMAQ Model Performance Evaluation Results for IMPROVE Sites Over the VISTAS

Domain for the 13-27 July 2001 Episode (all sites/all days): Particulate Sulfate (top left),

Particulate Nitrate (top right), Particulate Organic Carbon (OC) (bottom left) and

Particulate Elemental Carbon (EC) (bottom right).


86/144


Figure C-1. Concluded. Particulate Ammonia (top left), Soil (top right), Coarse Mass (bottom

left) and PM2.5 (bottom right).


87/144


Figure C-2. CMAQ Model Performance Evaluation Results for CASTNet Sites Over the VISTAS

Domain for the 13-27 July 2001 Episode (all sites/all days): Gaseous HNO3 (top left),

Gaseous SO2 (top right), Particulate Ammonia (NH4) (bottom left) and Particulate

Nitrate (NO3) (bottom right).


88/144


Figure C-2. Concluded. Particulate Ammonia Ratio (top left), Particulate Nitrate Ratio (top right),

Particulate Sulfate (bottom left) and Total Nitrate (bottom right).


89/144


90/144


Figure C-3. Concluded. Particulate Ammonia (top left), Soil (top right), Total Carbon Mass



91/144


Figure C-4. CMAQ Model Performance Evaluation Results for AQS Sites Over the VISTAS

Domain for the 13-27 July 1999 Episode (all sites/all days): Carbon Monoxide (top left),

Nitrogen Dioxide (top right), Ozone (bottom left) and Sulfur Dioxide (bottom right).


92/144


Figure C-5. CMAQ Model Performance Evaluation Results for NADP Sites Over the VISTAS

Domain for the 13-27 July 2001 Episode (all sites/all days): Ammonia Wet Deposition

(top left), Nitrate Wet Deposition (top right), and Sulfate Wet Deposition (bottom left).


93/144


Figure C-6. CMAQ Model Performance Evaluation Results for Daily SEARCH Sites Over the

VISTAS Domain for the 13-27 July 2001 Episode (all sites/all days): Elemental Carbon

(top left), Particulate Ammonia (top right), Particulate Nitrate (bottom left) and Organic

Carbon Mass (bottom right).


94/144


95/144


Figure C-7. CMAQ Model Performance Evaluation Results for Hourly SEARCH Sites Over the

VISTAS Domain for the 13-27 July 2001 Episode (all sites/all days): Particulate

Ammonium (top left), Particulate Nitrate (top right), PM2.5 Mass (bottom left) and

(bottom right).


96/144


Figure C-8. 24-Hr Particulate Sulfate (top) and Particulate Nitrate (bottom) Time Series at

Shenandoah National Park IMPROVE Site observed (red) and CMAQ estimated



97/144



Mammoth Cave National Park IMPROVE Site observed (red) and CMAQ estimated



98/144


99/144



Cape Romain Wildlife Area IMPROVE Site observed (red) and CMAQ estimated



100/144



Sipsey Wilderness Area IMPROVE Site observed (red) and CMAQ estimated using

36 km (blue) and 12 km (purple) grids.


101/144



Okefenokee Wildlife Area IMPROVE Site observed (red) and CMAQ estimated



102/144



Everglades National Park IMPROVE Site observed (red) and CMAQ estimated



103/144

APPENDIX D:

RESULTS OF THE 10-21 JULY 1999 CMAQ ALOFT

MODEL EVALUATION WITH NOAA AIRCRAFT DATA


104/144


105/144


106/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixD (Jul '99 Aloft MPE).doc D-4

Figure D-1. Comparison of NOAA Aircraft Ozone Measurements (ppb) Aloft on 11 July 1999 with CMAQ Masscons12

Estimates: 1700 UCT Flight (top left), 1800 UTC Flight (top right), 1900 UTC Flight (bottom left), and 2000

UTC Flight (bottom right).


107/144


108/144






109/144



Estimates: 2200 UCT Flight (left) and 2300 UTC Flight (right).


110/144






111/144



Estimates: 2100 UCT Flight (top left), 2200 UTC Flight (top right), and 2300 UTC Flight (bottom left).


112/144


113/144


114/144


115/144




UTC (bottom right).


116/144


117/144


Figure D-6. Comparison of NOAA Aircraft SO2 Measurements (ppb) Aloft on 11 July 1999 with CMAQ Masscons12



118/144


119/144





120/144


121/144





122/144


123/144





124/144


125/144




UTC (bottom right).


126/144

APPENDIX E:

RESULTS OF THE CMAQ/CAMx HORIZONTAL GRID

SENSITIVITY ANALYSIS WITH THREE VISTAS EPISODES


127/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixE (Grid Resolution Analysis).doc

Table E-1. CMAQ Performance for the 1-21 January '02 Episode: Mass Patch 12 vs. 36 Grids

IMPROVE Network: Mass Patch 12 km

Performance Measure Notation SO4 NO3 NH4 OC EC SOIL

Bias Factor BF 0.76 2.11 1.03 1.10 1.52 23.16

Coeff. of Determination CD 0.49 0.37 0.43 0.39 0.20 0.23

Coeff. of Variation CV 1.03 2.82 0.32 2.06 0.53 18.77

Fractional Gross Error FE 45.46 76.18 43.43 39.50 49.93 169.66

Mean Abs. Gross Error MAGE 0.79 1.26 0.45 0.98 0.33 3.23

Mean Bias MB -0.60 0.72 -0.09 -0.02 0.22 3.23

Mean Fractional Bias MFB -36.21 20.26 -12.72 -2.81 16.72 169.66

Mean Normalized Bias MNB -24.40 110.90 3.15 10.09 51.73 2215.63

Mean Norm. Gross Error MNGE 36.58 147.20 43.90 42.34 75.34 2215.63

Normalized Mean Bias NMB -27.51 42.76 -7.13 -0.71 50.41 1396.03

Normalized Mean Error NME 36.48 74.41 34.01 40.54 77.62 1396.03

Mean Observed OM 2.17 1.69 1.31 2.42 0.43 0.23

Observed Std. Dev. OSD 1.09 1.43 0.64 1.51 0.25 0.17

Mean Predicted PM 1.58 2.41 1.22 2.41 0.64 3.46

Accuracy of Paired Peak PP 0.14 0.20 -0.19 0.39 2.50 31.20

Predicted Std. Dev. PSD 1.04 1.88 0.70 1.77 0.77 2.97

Root Mean Square Error RMSE 1.02 1.68 0.56 1.44 0.73 4.33

IMPROVE Network: Mass Patch 36 km

Performance Measure Notation SO4 NO3 NH4 OC EC SOIL Bias Factor BF 0.79 2.78 1.21 1.20 1.63 25.83






128/144


Mean Bias MB -0.53 1.14 0.05 0.11 0.24 3.52

Mean Fractional Bias MFB -33.79 33.69 -2.74 3.24 24.23 173.02

Mean Normalized Bias MNB -20.98 177.77 20.83 19.70 62.67 2482.57


Normalized Mean Bias NMB -24.35 67.51 3.81 4.59 55.32 1522.81

Normalized Mean Error NME 38.66 92.80 39.22 43.54 82.27 1522.81Mean Observed OM 2.17 1.69 1.31 2.42 0.43 0.23



Accuracy of Paired Peak PP 0.25 0.39 -0.11 0.32 1.52 23.31




129/144


Table E-2. CMAQ Performance for the 10-21 July '99 Episode: MassCons 12 vs. 36 Grids

IMPROVE Network: MassCons 12 km


Bias Factor BF 1.77 0.50 0.86 0.75 0.95 4.31





Mean Bias MB 2.11 -0.17 -0.75 -1.26 0.00 0.89

Mean Fractional Bias MFB 33.15 -118.44 -28.86 -50.78 -22.88 46.73

Mean Normalized Bias MNB 77.05 -50.28 -13.67 -24.93 -4.71 330.94


Normalized Mean Bias NMB 31.76 -56.97 -28.93 -37.46 0.51 82.33


Mean Observed OM 6.65 0.29 2.58 3.36 0.46 1.08



Accuracy of Paired Peak PP 0.11 0.00 -0.38 -0.19 1.33 0.32



IMPROVE Network: MassCons 36 km







130/144


131/144


132/144


Mean Fractional Bias MFB 2.37 -100.01 -51.40 -39.52 -12.73 41.51

Mean Normalized Bias MNB 17.09 -31.14 -32.48 -16.79 16.15 186.57


Normalized Mean Bias NMB 8.13 -49.33 -33.21 -15.25 -17.83 22.71


Mean Observed OM 6.42 0.31 2.50 2.67 0.33 1.16Observed Std. Dev. OSD 4.64 0.33 1.73 1.30 0.21 1.50


Accuracy of Paired Peak PP 0.73 -0.41 -0.03 0.43 -0.36 -0.45




133/144


Table E-4. CMAQ Performance for the 1-21 January '02 Episode: SAPRC 12 vs. 36 Grids

IMPROVE Network: SAPRC 12 km


Bias Factor BF 0.88 0.95 0.79 1.03 1.49 23.89





Mean Bias MB -0.32 -0.64 -0.39 -0.22 0.20 3.33

Mean Fractional Bias MFB -21.34 -57.18 -38.68 -10.16 14.03 170.78

Mean Normalized Bias MNB -12.21 -4.81 -21.39 2.57 48.57 2288.69


Normalized Mean Bias NMB -14.67 -38.16 -29.80 -9.14 47.50 1437.62


Mean Observed OM 2.17 1.69 1.31 2.42 0.43 0.23



Accuracy of Paired Peak PP 0.28 -0.25 -0.37 0.36 2.49 31.39



IMPROVE Network: SAPRC99 36 km







134/144


Mean Bias MB -0.25 -0.47 -0.32 -0.06 0.24 3.52

Mean Fractional Bias MFB -18.65 -40.48 -31.73 -1.99 24.21 172.99

Mean Normalized Bias MNB -8.63 23.17 -12.44 13.15 62.67 2481.79


Normalized Mean Bias NMB -11.47 -27.96 -24.14 -2.39 55.33 1522.35




Accuracy of Paired Peak PP 0.51 -0.23 -0.18 0.08 1.52 23.34




135/144


Table E-5. CMAQ Performance for the 10-21 July '99 Episode: SAPRC99 12 vs. 36 Grids



Bias Factor BF 1.26 0.39 0.69 0.44 0.74 3.76

Coeff. of Determination CD 0.51 0.06 0.27 0.13 0.46 0.02Coeff. of Variation CV 15.15 0.09 3.65 6.84 0.10 4.10



Mean Bias MB 0.07 -0.18 -1.09 -2.08 -0.10 0.59

Mean Fractional Bias MFB -0.49 -128.25 -51.87 -94.03 -47.36 35.36



Normalized Mean Bias NMB 1.10 -62.68 -42.21 -61.90 -21.69 54.46


Mean Observed OM 6.65 0.29 2.58 3.36 0.46 1.08



Accuracy of Paired Peak PP 0.04 0.01 -0.39 -0.38 0.83 0.03





Bias Factor BF 1.22 0.50 0.69 0.41 0.67 3.15





Mean Bias MB -0.08 -0.17 -1.06 -2.18 -0.16 0.33


136/144





Normalized Mean Bias NMB -1.13 -58.04 -41.27 -64.81 -35.49 30.44


Mean Observed OM 6.65 0.29 2.58 3.36 0.46 1.08Observed Std. Dev. OSD 5.00 0.18 1.86 1.66 0.20 1.17


Accuracy of Paired Peak PP -0.17 -0.12 -0.46 -0.48 0.14 -0.37




137/144


Table E-6. CMAQ Performance for the 7-28 July '01 Episode: SAPRC99 12 vs. 36 Grids



Bias Factor BF 0.94 0.49 0.54 0.61 0.91 2.72





Mean Bias MB -0.11 -0.20 -1.06 -1.01 -0.09 0.25


Mean Normalized Bias MNB -6.41 -50.79 -46.12 -38.90 -9.49 172.19




Mean Observed OM 6.42 0.31 2.50 2.67 0.33 1.16













138/144


Mean Bias MB -0.37 -0.17 -0.91 -0.81 -0.06 0.35


Mean Normalized Bias MNB -5.57 -37.43 -37.66 -30.42 12.88 201.26










139/144

G:\VISTAS Modeling\Task4cd Eval-Sens\Final\AppendixE (Grid Resolution Analysis).doc E-13

IMPROVE Mean Fractional Bias

-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

MeanFractionalBias

MassCons 36 km

MassCons 12 km


-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

MeanFractionalBias

Base 36 km

Base 12 km


-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

M

eanFractionalBias

Mass Patch 36 km

Mass Patch 12 km

Figure E-1. Mean Fractional Bias in Episode Average Aerosol Concentrations for the Three VISTAS Phase I Base

Cases: Jul 99 (top left), Jul 01 (top right), and Jan 02 (bottom left).


140/144


IMPROVE Fractional Gross Error

0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

FractionalGr

ossError

MassCons 36 km

MassCons 12 km


0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10


Base 36 km

Base 12 km


0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10


Mass Patch 36 km

Mass Patch 12 km

Figure E-2. Mean Fractional Gross Error in Episode Average Aerosol Concentrations for the Three VISTAS Phase I

Base Cases: Jul 99 (top left), Jul 01 (top right), and Jan 02 (bottom left).


141/144


142/144



-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

MeanFractionalBias

MassCons 36 km

MassCons 12 km


-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

MeanFractionalBias

Base 36 km

Base 12 km


-120.0

-100.0

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

M

eanFractionalBias

Mass Patch 36 km

Mass Patch 12 km

Figure E-4. Mean Fractional Bias in Episode Average Aerosol Concentrations for the Three SAPRC99 Sensitivity

Experiments: Jul 99 (top left), Jul 01 (top right), and Jan 02 (bottom left).


143/144



0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10

FractionalGr

ossError

MassCons 36 km

MassCons 12 km


0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10


Base 36 km

Base 12 km


0.0

25.0

50.0

75.0

100.0

125.0

150.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10


Mass Patch 36 km

Mass Patch 12 km

Figure E-5. Mean Fractional Gross Error in Episode Average Aerosol Concentrations for the Three SAPRC99

Sensitivity Experiments: Jul 99 (top left), Jul 01 (top right), and Jan 02 (bottom left).


144/144

IMPROVE Normalized Mean Bias,%

-75.0

-50.0

-25.0

0.0

25.0

50.0

75.0

100.0

SO4

NO3

NH4

OC ECSOIL CM

PM2.5

RCFM

PM10


MassCons 36 km

MassCons 12 km

IMPROVE Normalized Mean Bias,%

-75.0

-50.0

-25.0

0.0

25.0

50.0

75.0

100.0

SO4

NO3

NH4

OC EC

SOIL CM

PM2.5

RCFM

PM10

NormalizedMeanBia

s,%

Base 36 km

Base 12 km

vistas task4cd finalreport appendicesa-e only

Documents