framework for a comparison of eea and epa indicators
DESCRIPTION
Framework for a Comparison of EEA and EPA Indicators. Heather Case and Jay Messer U.S. EPA Ispra, Italy January, 2006. Purposes. Propose a framework for a cooperative effort to conduct an in-depth comparison of EEA and EPA environmental indicators. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/1.jpg)
Heather Case and Jay Messer U.S. EPA
Ispra, ItalyJanuary, 2006
Framework for a Comparison of EEA and
EPA Indicators
![Page 2: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/2.jpg)
• Propose a framework for a cooperative effort to conduct an in-depth comparison of EEA and EPA environmental indicators.
• Present examples of EPA indicators from the upcoming EPA Report on the Environment that illustrate the key comparison issues.
• Set the stage for discussing additional issues that involve electronic augmentation and updating of indicators going forward (next presentation).
Purposes
![Page 3: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/3.jpg)
EPA’s Report on the EPA’s Report on the Environment Environment
Recent events and future Recent events and future
directionsdirections
![Page 4: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/4.jpg)
Recent events & future directionsRecent events & future directionsSince we met in May 2005• July 2005: Peer review of proposed indicators• Oct. 2005: Second peer review of indicators
newly proposed or significantly revised
Looking Ahead• January 2006: Posting of “final” indicators for ROE 2007 on the internet• September 2006: Scientific peer review of full ROE Technical Document• Spring 2007: Final Release of Technical Document
![Page 5: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/5.jpg)
Background
• At Washington, DC meeting in May, 2005, we decided to pursue a comparison of EEA and EPA indicators (including scaling).
• EPA Post-Doc, Ellen Natesan, developed a white-paper comparing EEA core indicators with indicators from EPA’s 2003 Draft Report on the Environment.
• Since then, many indicators have been updated and regionalized, and new indicators have been added for the 2007 ROE.
![Page 6: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/6.jpg)
Propositions
• Indicators may fundamentally differ because of purpose, criteria, etc.
• Indicators may fundamentally differ because of monitoring design, methods, averaging period, scale, and reference points
• To the extent that the indicators are transparent and reproducible, and the date well-documented and accessible, if two indicators are not fundamentally different, an opportunity exists to calibrate one indicator against the other.
![Page 7: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/7.jpg)
Overview of Proposed Criteria for Comparisons
• Purpose of indicators
• Indicator definition, criteria and “ground rules”
• Monitoring design and data comparability
• Quality assurance
• Scaling
• Data management and accessibility
![Page 8: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/8.jpg)
Purpose of Indicators
• ROE indicators answer questions about the state of the environment over time (e.g., are ozone levels decreasing over time?)
• Accountability indicators track the effectiveness of particular programs (e.g., are controls on mobile sources reducing ozone?) – Must be responsive to early actions– Must differentiate among causes– May involve cost-effectiveness
![Page 9: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/9.jpg)
Purpose of Indicators
• Examples– What are the trends in outdoor air quality and
their effects on human health and the environment?
• Sulfur Dioxide Emissions• Ozone Injury to Forest Plants
– What are the trends in extent and condition of fresh surface waters and their effects on human health and the environment?
• Nitrogen and Phosphorus in Streams in Agricultural Watersheds
• Benthic Invertebrates in Wadeable Streams
![Page 10: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/10.jpg)
Indicator Definition and Criteria
• ROE Indicator – a numerical value derived from actual measurements of a pressure, ambient condition, exposure, or human health or ecological condition over a specified geographic domain, whose trends over time represent or draw attention to underlying trends in the condition of the environment.
![Page 11: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/11.jpg)
Indicator Types
![Page 12: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/12.jpg)
What is currently NOT included
• Administrative indicators (government actions and responses to them)
• Resource use
• Economic and “sustainability” indicators
![Page 13: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/13.jpg)
ROE Indicator Criteria
• The indicator is useful. It answers (or makes an important contribution to answering) a question in the ROE.
• The indicator is objective. It is developed and presented in an accurate, clear, complete, and unbiased manner.
• The indicator is transparent and reproducible. The specific data used and the specific assumptions, analytic methods, and statistical procedures employed are clearly stated
![Page 14: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/14.jpg)
ROE Indicator Criteria (cont.)
• The underlying data are characterized by sound collection methodologies, data management systems to protect its integrity, and quality assurance procedures.
• Data are available to describe changes or trends and the latest available data are timely.
• The data are comparable across time and space, and representative of the target population. Trends depicted in the indicator accurately represent the underlying trends in the target population.
![Page 15: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/15.jpg)
ROE Indicator Modeling “Ground Rule”
• A model may be used to calculate and indicator value based on a physical measurement that is not itself the indicator, as long as the physical value and the indicator are at the same hierarchical level.
– Permissable: NOX emissions based on fuel consumption and an emissions factor
– Not permissable: acid deposition based on SO2 emissions
![Page 16: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/16.jpg)
Monitoring design & data comparability
• What is being measured? Are the methods equivalent? Is guidance available and being followed?
• Where are the monitoring sites located? How were the locations chosen (e.g., purposive vs probability designs)
• When are samples collected? What is the averaging period?
• What are the reference points?
![Page 17: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/17.jpg)
Monitoring design & data comparability
• What is being measured? Are the methods equivalent? Is guidance available and being followed?
![Page 18: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/18.jpg)
Examples
• SO2 and VOC Emissions
• Fuel Combustion: Power Generators - emissions from coal, gas, and oil-fired power plants required to use continuous emissions monitors (SO2 only)
• Fuel Combustion: Other Sources -industrial, commercial, institutional and residential heaters and boilers not required to use CEMs – emissions factors and DOE Fuel use data
• Other Industrial Processes – e.g., chemical production and petroleum refining – emissions factors, production data
![Page 19: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/19.jpg)
SO2 and VOC Emissions
• On-road Vehicles – e.g. cars, trucks, buses, and motorcycles – FHWA mileage estimates and EPA’s MOBILE6 model
• Non-road Vehicles and Engines – e.g., farm and construction equipment, lawnmowers, chainsaws, boats/ships, aircraft – EPA’s NONROAD model
![Page 20: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/20.jpg)
National Emissions Inventory
• Conducted every three years
• EPA develops some data (electricity generators)
• States develop other data with guidance from EPA
• EPA performs consistency checks
• Methods evolve - only 1990 inventory fully reconciled to latest inventory year
![Page 21: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/21.jpg)
![Page 22: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/22.jpg)
![Page 23: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/23.jpg)
Monitoring design & data comparability
• Where are the monitoring sites located? How were the locations chosen (e.g., purposive vs probability designs)
• When are samples collected? What is the averaging period?
• What is the reference point?
![Page 24: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/24.jpg)
Examples
• Nitrogen and Phosphorus in streams
– Nitrate in streams in agricultural watersheds
– Nutrient Concentrations in wadeable streams
![Page 25: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/25.jpg)
Three possibilities
• Section 305(b) of Clean Water Act - States
• National Water Quality Assessment (NAWQA) – U.S. Geological Survey
• Wadeable Streams Assessment (SWA) – EPA and States
![Page 26: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/26.jpg)
Section 305(b) of Clean Water Act
• States determine (attainable) designated uses for each water body
• Monitor against water quality standards appropriate for the designated use
• Report to EPA every two years on percentage of water bodies that meet standards (possible indicator)
![Page 27: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/27.jpg)
Section 305(b) of Clean Water Act
• Only a small fraction of water bodies assessed
• Biases in designation of use and water bodies monitored
• Standards and methods vary from state to state
• Rejected as indicator in FY03 Draft ROE for failure to meet indicator criteria
![Page 28: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/28.jpg)
Nutrients in Streams
• NAWQA– Purposive design (50
watersheds)
– Sampled at many points in the watershed
– Sampled 12-13 times/year
– No reference levels
• WSA – Probability design
(1392 reaches)
– Sampled at one point on the reach
– Sampled once every 4 years (summer)
– Reference levels based on statistics from regional reference sites
![Page 29: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/29.jpg)
![Page 30: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/30.jpg)
![Page 31: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/31.jpg)
Nitrate in streams in agricultural watersheds, NAWQA (1992-1998)
52.3%
29.9%
8.4%9.3%
0%
20%
40%
60%
80%
100%
1992-1998
less than 2 ppm
2 to 6 ppm
6 to 10 ppm
10 ppm or more
Pe
rce
nt o
f Str
eam
Site
s
EPA's drinking water standard is 10 ppm (Maximum Contaminant Level).
![Page 32: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/32.jpg)
Nutrient Concentrations in Wadeable Streams
WSA (1999-2003)
51%
45%
17%
22%
32%
33%
0% 20% 40% 60% 80% 100%
TotalPhosphorus
Total Nitrogen
Percentage of stream miles
Low: Below 75th percentile of reference range
Moderate: Between 75th and 95th percentiles of reference range
High: Above 95th percentile of reference range
Source: U.S. EPA, Wadeable Streams Assessment
![Page 33: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/33.jpg)
Nutrients in Streams
• NAWQA – Better characterization
of sampled streams and watersheds
But
– Expensive
– Can’t be extrapolated to unsampled streams
– No confidence bounds for national estimates
• WSA– Unbiased estimates of
all wadeable streams, with known confidence
– Comparatively inexpensive
But
– Poor characterization of individual reaches
– No data for extreme events or other seasons
![Page 34: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/34.jpg)
Quality assurance
• Are controls in place to insure that the data are of adequate and know quality?
• Are the metadata available?
• Links to QA Plans and metadata for ROE indicators in Indicator QA forms (Heather Case’s presentation)
![Page 35: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/35.jpg)
Scaling
• What is the most disaggregated level at which the indicator is meaningful?
• Is the reference level appropriate for the extent and grain size of the indicator? How important are episodes?
• How sensitive is the indicator to the effects of a few very large entities?
![Page 36: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/36.jpg)
Scaling
• What is the most disaggregated level at which the indicator is meaningful?
– SO2 and VOC emissions
• ROE07 - 10 EPA Regions
• 3100 US counties (theoretically)
– N&P in streams
• ROE07 - national only
• NAWQA – 50 predominantly agricultural watersheds
• WSA – 10 EPA Regions (theoretically) or 9 ecoregions
![Page 37: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/37.jpg)
Scaling
• What is the most disaggregated level at which the indicator is meaningful?
![Page 38: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/38.jpg)
![Page 39: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/39.jpg)
![Page 40: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/40.jpg)
Scaling
• What is the most disaggregated level at which the indicator is meaningful?
![Page 41: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/41.jpg)
Scaling
• Is the reference level appropriate for the extent and grain size of the indicator? How important are episodes?
– Mean levels of toxic chemicals in a stream may not mean much if storm events do the damage
• How sensitive is the indicator to the effects of a few very large entities?
– A very small percentage of emitters may be responsible for a large fraction of total emissions – to the extent that they are concentrated in a few states or regions, they may skew national statistics
![Page 42: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/42.jpg)
Data management and accessibility
• The key to transparency and reproducibility
• All ROE indicators have
– Data underlying the figures available in excel spreadsheets online
– Links to parent databases
• Some ROE indicators have
– Links to datasets (or data in excel spreadsheets) that underlie the data supporting the figures.
![Page 43: Framework for a Comparison of EEA and EPA Indicators](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813c76550346895da60c7f/html5/thumbnails/43.jpg)
Conclusions
• Indicators may fundamentally differ because of purpose, criteria, etc.
• Indicators may fundamentally differ because of monitoring design, methods, averaging period, scale, and reference points
• To the extent that the indicators are transparent and reproducible, and the date well-documented and accessible, if two indicators are not fundamentally different, an opportunity exists to calibrate one indicator against the other.