how can triad tools strengthen site assessment data ......managing decisionuncertaintyis thekeystone...

How Can Triad Tools StrengthenSite Assessment Data & Decisions?

Managing Decision Uncertainty Affordably and Transparently

Office of Superfund Remediation and Technology InnovationTechnology Integration and Field services Division

U.S. Environmental Protection AgencyNational Site Assessment Symposium, Denver, June 21, 2007

2

“Triad Approach” is a Framework of Best Practices that Ensure Good Projects

Uncertaintymgt

Dynamic Work

Strategies

Systematic Project

Planning

Real-time Measurement Technologies

Managing Decision Uncertainty is the keystone conceptthat integrates the 3 general “how to” Triad elements

311Jun07 2007 NSAS Presentation

Key Triad Features for Proactively Managing Decision Uncertainty

CSM is your hypothesis

Test & refine CSM in real-timeCSM is a tool for consensus,communication & trust

CSM summarizes wt of evidence that supports decisions

Heterogeneouscontamination

Sample supportMultidisciplinary teamsDMAs optimize method & tool performance

Collaborative data sets: partner analytical techniques


Triad is NOT……written in all caps …just about field analytical (ALL data uncertainties: sampling & fixed lab)…a way to justify using field analysis without using proper QC (known & documented quality)…just about doing work dynamically (all sources of decision uncertainty) …a license to write vague work plans or escape regulatory oversight or accountability (negotiated & pre-approveddecision logic)…a regulatory program (Triad is a science-driven framework for best practices & a training initiative)

511Jun07 2007 NSAS Presentationwww.triadcentral.org

Get

Mor

e In

fo!!


A Well-Planned & Documented CSM Improves Data Quality

Develop preliminary conceptual site model (CSM) during upfront planning to help…

Identify decision inputs, including data

Predict nature & extent, receptors & exposure pathways AND contaminant DISTRIBUTIONS

Predict how good data must be (i.e., how sure that your data interpretations are correct?)

Predict sampling uncertainty

Predict how to control sampling uncertainty thru approp. sampling density & sample support


“Data Representativeness” Concept (Assume the Analysis Itself is Perfect)

Q: What does a data result (say, 75 ppm Pb in soil) represent?A: Certain result represents analyte mass measured ÷ mass of soil

extracted from: instr. measures 0.075 mg of Pb ÷ 1 g soil extracted = 75 mg/kg = 75 ppm Pb

Q: Confidently extrapolate back to parent matrix (say, 24 metric tons)? Am I sure conc’n of 1-gram sample accurately represents conc’n of 6-inch soil slab from 100-sq. yd. area?

A: It depends…

SAM“Not helpful!!”


Soil Core SamplePopulation

Analytical Sample Prep

Analytical Sample Unit

Why is Sample Representativeness Such a Challenge?

Field Subsample

23.4567 ppmGC

Lab Subsamples (Duplicates)


.. .

Scale of the Analytical Sample vs.

the Scale of Decision-makingDecision: Is the 100-yd2

grid block “dirty”?

~26 tons


Population

Analytical Sample Prep

Do Field & Lab Procedures Control for Within-Sample Variability?

Field Subsample

23.4567 ppmGC

sample

sub-sample sub-sub-

samplesub-sub-sub-sample

This result represents

???...

Soil Core Sample: Will the subsample be extrapolated to represent the entire core?

Lab: Do portions either represent the whole jar or a selected soil fraction as designated by the SAM?


Within-Sample Variability: Interaction between Contaminant & Matrix Materials

Firing Range Soil Grain Size (Std Sieve Mesh Size)

Pb Concentration in fraction by AA (mg/kg)

Greater than 3/8” (0.375”) 10

Between 3/8 and 4-mesh” 50

Between 4- and 10-mesh 108



Less than 200-mesh 1,970

Bulk Total 927 (wt-averaged)A

dapt

ed fr

om IT

RC

(200

3)

The decision determines representativeness


Micro-scale Heterogeneity Causes Highly Variable Results

Adapted from

DO

E (1978 )

Number of subsamples req’d to estimate true sample mean within a

range of…Subsample

Support(dried, ball-

milled, sieved to

<10-mesh)

Range of Results

[for 20 individual

subsamples(ppm)]

Coeffof

Var.

(CV)…± 25%

[ex: 100 ± 25% = 75 - 125 ppm]

…± 10%[ex: 100 ± 10% =

90 - 110 ppm]

1 g 1.01 - 8.00 0.79 39 24010 g 1.36 - 3.43 0.27 5 2850 g 1.55 - 2.46 0.12 1 6

100 g 1.70 - 2.30 0.09 1 4

How much confidence should be placed in any single result?

True mean known to be 1.92 ppm

Larger Sample Supports Produce More Consistent, Representative Data

Figu

re a

dapt

ed fr

om J

eff M

yers

, 200

1

Sample Volumes/Supports

Largest

Mid-sizedSmallest

Low ConcentrationHigh Concentration

Low contaminant concentrations & too small sample supports contribute to lognormal populations!!

14

Regulatory & field practices have long assumed that

sample size/volume has no effect on analytical results

Now we know the assumption is inaccurate because of micro-

scale (within-sample) heterogeneity.

Sample volume affects the analytical result!

Concentrated Particles within Less Concentrated Matrix = “Nugget Effect”

The Nugget Effect

Soil Subsample

Sample Prep

2 g 5 g

164 On-site136 Lab

1

27

6 3

45

331 On-site286 Lab

2 ft

39,800 On-site41,400 Lab

1,280 On-site1,220 Lab

27,800 On-site42,800 Lab

24,400 On-site27,700 Lab

500 On-site416 Lab

Figure adapted from Jenkins (CRREL), 1996

> 95% of variability due to sample

location

<5% due to diff. methods

Analytical Methods May Differ Somewhat, but Heterogeneity Effects Are Much Larger

http://www.crrel.usace.army.mil/techpub/CRREL_Reports/reports/SR96_15.pdf


Range of Results vs Sample Support

1

10

100

1000

10000

1 10 100 1000 10000 100000 1000000 10000000

Sample Support (gr)

(pp

)

True Average Site Concentration

Max Result

Min ResultXRF

StandardSample

Multi-IncrementSample

In Situ NaIReading

In SituHPGe

10 kg 100 kg 1000 kg 10 metric tons

Dramatic Variability Across Range of Sample Supports

Conc

(pp

m)

in lo

g sc

ale

Same Sample Support Issue Exists with Groundwater WellsLog from direct-push deployed membrane-interface probe w/ ECD detector (DP-MIP-ECD)

Variables to control:

Where is the well screened?

How long is the screen?

How much mixing between distinct stratigraphic zones occurs when the well is purged & sampled?

Graphic adapted from Columbia Technologies

1811Jun07 2007 NSAS PresentationSlide adapted from Columbia Technologies, Inc., 2003

The Real World is Messy

subsurface CSM from high density DP-MIP sensing data


The #1 Reason Why SAMs Need the Triad Framework


“Representativeness” As Reconciling Different Spatial Scales

Representative sampling designs will match the scale of sampling to both……the scale(s) of variability…the scale(s) of decision making

Scale of 2-g samplingScale of micro-heterogeneityScale of macro-heterogeneityScale of decision making

(For illustrative purposes only)//


Triad’s Dynamic Work Strategies Benefit Site Assessors

Pre-planned dynamic decision tree Adapts to the maturing CSMFills data gaps immediately as they are identifiedIncludes “stop work” (“we know we are done”) milestones

If it is found that extent of contamination actually quite small, could quick removal be sufficient? If so, stop SA activities & switch to removal program. Can potential switch to removal be anticipated & built into work planning, so that project seamlessly glides into delineation & removal w/o a demob interruption?


Dynamic Work Strategies Improve Data & Decision Quality

Continual re-evaluation of COCs, contaminant patterns, release/fate mechanisms, exposure unit dimensions & exposure pathwaysModify field/lab procedures & QC to ensure defensible data if matrix properties or concentrations changeInvestigate & resolve outlier resultsReport solid information to public much fasterAdjust collaborative S&A strategy as needed to get most information out of each analysis

Simple substitution of field for lab analyses WILL NOT WORK!!


Real-time Analysis Improves Data & Decision Quality

Cost-effectively measure & control sampling variability by increasing sample numbers in the right places

Understand contaminant distributions (build the CSM)Increase reps to measure variability & its sourcesAdapt sampling & decision-making scales as needed per CSMAdaptively optimize analytical performance before $$$ spent

Real-time = able to seamlessly affect current work if using the right tools

Is the detection of contamination “real”? Does detection mean a lot or a little contamination? Maybe we just missed it?

A Second-Generation Data Quality Model (for Heterogeneous Matrices)

Costlier/rigorous analytical methods (ex: lab, field, std, non-std)

Cheaper/rapid analytical methods (ex: lab, field, std, non-std)

Targeted high density sampling Low DL + analyte specificity

Manages CSM & sampling uncertainty

Manages analytical uncertainty

Collaborative Data SetsCollaborative data sets complement each other so all sources of data

uncertainty are managed. Using either kind of data alone will not produce reliable information.


Collaborative Data Sets Use Special Techniques & Wt of Evidence

A single result w/o any knowledge of variability offers no decision confidence (flip a coin)

Hi-density non-specific analyses can establish the representativeness of analyte-specific analyses

Simply substituting field for fixed lab analysis inadequate

Regression often not usefulMerging data sets sometimes possible…but don’t count on itNon-parametric techniques often best


How Triad Uses Data Collaboratively

Non-quantitative comparability will be most commonMerger of data sets often not feasibleHi-density non-specific analyses used to establish the representativeness of relatively few analyte-specificEstablish data/decision comparability using non-parametric statistics

Quantitative comparabilityr2 often used to assess, but correlation coefficients can be very misleading If data sets are to be merged but slope (method sensitivity) & bias (y-intercept) are too different, will need to be transformed (if possible) Does the calculated confidence interval narrow around estimates of the mean if data sets are merged?


We’ve Seen the Science, But Is This Approach Legally Defensible?

YES!! Courts use two standards:1. The “Frye Test”

Scientific data must be “relevant” and “reliable”Methods must be “generally accepted”

The “Daubert” Standard“General acceptance” may not be required IF sufficient “weight of evidence” can be demonstrated

Regardless of method, admissibility depends on following documented SOPs & QA/QC


Triad Consistent with EPA Policy

EPA Order 5360.1 A2 Policy and Program Requirements for the Mandatory Agency-Wide Quality System

Use of a systematic planning approach to develop acceptance or performance criteria for work covered by this order. (See Section 3.3.8 EPA Quality Manual for Environmental Programs)


Triad and EPA Policy Continued

OSWER Memorandum 9200.1-55 “Advancing Best Management Practices: Applying the Triad Approach in the Superfund Program”Unified Federal Policy Quality Assurance Project Plan (UFP-QAPP) “Definitive data can be generated by a field method, an on-site laboratory, or an off-site laboratory.” EPA QA/G5-S Sampling design, compositing, decision support tools, and other issues associated with evaluating “representativeness”


Sampling in Support of HRS

Identify targets exposed to a hazardous substanceIdentify hazardous substances present at the siteSample to demonstrate a release

1. Significantly above background (3 times background)

2. Demonstrate that release is at least partially attributable to site

Discriminate among alternative sourcesDetermine representative background concentrationsVerify field and laboratory practices


Site Inspection Guidance Section 3.4

“The SI investigator must plan which analytical methods and services to use. Although laboratory methods are routinely used, field analyses may often provide the type and quality of data needed to support site assessment decisions and satisfy data quality objectives. SI investigator should consider”…

See Handouts


Triad in Support of HRS

SI, focused SI, expanded SI. Real-time analytical results and dynamic work strategies can condense this process, limit mobilizations, and focus sample collection for CLP analyses.

Did they sample the right locations?What do these samples represent?


Listing Decisions

The decision to place a site on the NPL is serious businessMany sites listed with small data sets (<10 samples)CLP analyses are the preferred method to avoid litigation issues (Remember Daubert and Frye!/ Plan for appropriate QA/QC)Triad principles are often used to augment the NPL data set but are not mentioned in the listing package. You may be using some aspects of Triad and not even know it!

Systematic planning, dynamic work strategies, real-time measurementsConceptual site modelField analytics


Superfund Triad Support Team (STST)

Partners

EPA Emergency Response Team

US Army Corps of Engineers

Argonne National Laboratory

Mission Support Contracts


Superfund Triad Support Team (STST)

Technical support services & trainingFacilitate systematic planningBuild/refine conceptual site modelSampling design, work scope & sequencing Real-time measurements, field analytics/collaborative data setsStatistical data analysisDecision support toolsWork-plan development/reviewRemedy optimization

http://www.superfundtriadsupport.org/

how can triad tools strengthen site assessment data ......managing decisionuncertaintyis thekeystone...

Documents