step 6: plan selection

47
Step 6: Plan Selection Step 6: Plan Selection Leigh Skaggs, CECW-PC, and Erin Wilson, CEIWR Leigh Skaggs, CECW-PC, and Erin Wilson, CEIWR Planning for Ecosystem Restoration Planning for Ecosystem Restoration PROSPECT 2010 PROSPECT 2010

Upload: jed

Post on 21-Feb-2016

44 views

Category:

Documents


0 download

DESCRIPTION

US Army Corps of Engineers Institute for Water Resources. Step 6: Plan Selection. Leigh Skaggs, CECW-PC, and Erin Wilson, CEIWR Planning for Ecosystem Restoration PROSPECT 2010. Specify Problems & Opportunities. Corps Planning Process: Six Steps . Inventory & Forecast - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Step 6:  Plan Selection

Step 6: Plan SelectionStep 6: Plan Selection

Leigh Skaggs, CECW-PC, and Erin Wilson, CEIWRLeigh Skaggs, CECW-PC, and Erin Wilson, CEIWRPlanning for Ecosystem Restoration Planning for Ecosystem Restoration

PROSPECT 2010PROSPECT 2010

Page 2: Step 6:  Plan Selection

Specify Problems Specify Problems & Opportunities& Opportunities

Inventory & Forecast Inventory & Forecast ConditionsConditions

Formulate Formulate Alternative PlansAlternative Plans

Evaluate Effects of Evaluate Effects of Alternative PlansAlternative Plans

Compare Compare Alternative PlansAlternative Plans

Select Select Recommended PlanRecommended Plan

Corps Corps Planning Planning Process:Process:Six Steps Six Steps

Page 3: Step 6:  Plan Selection

Learning ObjectivesLearning Objectives To describe possible plans that may To describe possible plans that may

be recommendedbe recommended To explain what is meant by the To explain what is meant by the

National Ecosystem Restoration National Ecosystem Restoration (NER) Plan (NER) Plan

To explain the criteria and To explain the criteria and considerations used to designate considerations used to designate the NER Plan and Recommended the NER Plan and Recommended PlanPlan

Page 4: Step 6:  Plan Selection

ReferencesReferences Planning Guidance Notebook Planning Guidance Notebook

(ER 1105-2-100) - April 2000(ER 1105-2-100) - April 2000– Chapter 2, Planning PrinciplesChapter 2, Planning Principles– Appendix E, Civil Works Missions & Appendix E, Civil Works Missions &

Evaluation ProceduresEvaluation Procedures Planning Manual Planning Manual

(IWR Report 96-R-21)(IWR Report 96-R-21)– Chapter 11 Chapter 11

Collaborative Planning (EC 1105-2-Collaborative Planning (EC 1105-2-409) 409)

Page 5: Step 6:  Plan Selection

SelectionSelection Screening is an iterative activity Screening is an iterative activity

based on criteria based on criteria Selection of a recommended plan is Selection of a recommended plan is

the final screening activity the final screening activity Different selection criteria will give Different selection criteria will give

you different recommendationsyou different recommendations Plans don’t go away; they’re just not Plans don’t go away; they’re just not

selectedselected

Page 6: Step 6:  Plan Selection

GeneralGeneral Single alternative selected & Single alternative selected &

recommended from all those considered recommended from all those considered ““No Action” is the default No Action” is the default

recommendation recommendation Why is recommended plan preferable to Why is recommended plan preferable to

No Action or any other alternative?No Action or any other alternative?– ““Telling your story”Telling your story”

P&G: display sufficient number of P&G: display sufficient number of alternatives; include mitigation; identify alternatives; include mitigation; identify R&UR&U

Page 7: Step 6:  Plan Selection

Possible Plans to Possible Plans to Recommend Recommend

No ActionNo Action National Economic Development (NED) National Economic Development (NED) National Ecosystem Restoration (NER) National Ecosystem Restoration (NER) Multipurpose Plan formerly “Combined Multipurpose Plan formerly “Combined

NED/NER Plan”NED/NER Plan” National Interest Plan – reflect full National Interest Plan – reflect full

range of Federal Interest – NED, RED, range of Federal Interest – NED, RED, EQ, and OSEEQ, and OSE

Locally Preferred PlanLocally Preferred Plan

Page 8: Step 6:  Plan Selection

NED Plan NED Plan For all project purposes other than For all project purposes other than

ecosystem restorationecosystem restoration Reasonably maximizes net national Reasonably maximizes net national

economic benefits (consistent w/ economic benefits (consistent w/ protecting environment)protecting environment)

Recommend NED, unless ASA(CW) Recommend NED, unless ASA(CW) grants exceptiongrants exception– locally preferred plan smaller than NEDlocally preferred plan smaller than NED– LPP larger but sponsor pays differenceLPP larger but sponsor pays difference

Page 9: Step 6:  Plan Selection

NER Plan NER Plan For ecosystem restoration projectsFor ecosystem restoration projects ReasonablyReasonably maximizes net maximizes net

ecosystem benefits compared to ecosystem benefits compared to costs costs

Must be cost effectiveMust be cost effective Desired level of incremental Desired level of incremental

output must be justifiedoutput must be justified– Is it “worth” it?Is it “worth” it?

Page 10: Step 6:  Plan Selection

Plan Selection – NER Benefits Plan Selection – NER Benefits OnlyOnly

RULE: Reasonably maximize ecosystem RULE: Reasonably maximize ecosystem restoration benefits compared to costsrestoration benefits compared to costs

Plan Cost HU’s Incr. Cost

Incr. HU’s

Incr. $/HU

NA $0 0 $0 0 $0

A $105 50 $105 50 $2.1

B $135 60 $30 10 $3.0

C $180 65 $45 5 $9.0

Page 11: Step 6:  Plan Selection

NER Plan – Incremental Cost NER Plan – Incremental Cost DisplayDisplay

AA BB

CCNER Plan = NER Plan = Is it Worth it?Is it Worth it?

Page 12: Step 6:  Plan Selection

Decision MakingDecision MakingGuidelinesGuidelines

CEA/ ICA ResultsCEA/ ICA Results

Is it worth it?Is it worth it?

Page 13: Step 6:  Plan Selection

Is it worth it?Is it worth it?

Decision making guidelines:Decision making guidelines:• output targetoutput target• output thresholdsoutput thresholds• cost limitcost limit• breakpointsbreakpoints• unintended effectsunintended effects• does it make sense?does it make sense?

Page 14: Step 6:  Plan Selection

NER Plan – NER Plan – Decision-Making Guidelines - Decision-Making Guidelines -

TargetsTargets

AA BB

CC

Out

put

targ

et ?

Out

put

targ

et ?

Page 15: Step 6:  Plan Selection

NER Plan – NER Plan – Decision-Making Guidelines - Decision-Making Guidelines -

ThresholdsThresholds

AA BB

CCMin

imum

Min

imum

Max

imum

Max

imum

Page 16: Step 6:  Plan Selection

NER Plan – NER Plan – Decision-Making Guidelines – Cost Decision-Making Guidelines – Cost

LimitsLimits

AA BB

CCCost limitCost limit

Page 17: Step 6:  Plan Selection

AA BB

CC

NER Plan – NER Plan – Decision-Making Guidelines - Decision-Making Guidelines -

BreakpointsBreakpoints

BreakpointBreakpoint

Page 18: Step 6:  Plan Selection

Intended and Unintended Effects

PLAN LAND OWNERSHIP IMPACT ON OTHER SPECIES RESIDENTIAL RELOCATION

NO ACTION NOT APPLICABLE NO IMPACT NONE

A STATE NO IMPACT NONE

B STATE -5 NONE

C STATE AND PRIVATE -20 5 HOMES

AA BB

CC

Page 19: Step 6:  Plan Selection

Does it make sense?Does it make sense?

Red faceRed face testtest

testtest

““Idiot”Idiot” test test

Page 20: Step 6:  Plan Selection

NER Plan - Additional NER Plan - Additional Considerations Considerations

Meets planning objectives & constraintsMeets planning objectives & constraints Passes criteria:Passes criteria:

– significance, acceptability, completeness, significance, acceptability, completeness, effectiveness, efficiencyeffectiveness, efficiency

Ecosystem context Ecosystem context – Restores structure, function, dynamic processesRestores structure, function, dynamic processes

Reasonableness of costsReasonableness of costs In most cases, should be “best buy” planIn most cases, should be “best buy” plan

– ER 1105-2-100 (E-41 c.): Rarely will the NER plan ER 1105-2-100 (E-41 c.): Rarely will the NER plan not be among the “best buys”not be among the “best buys”

Page 21: Step 6:  Plan Selection

NER Plan - Additional Considerations: NER Plan - Additional Considerations: Risk and UncertaintyRisk and Uncertainty

Required analysisRequired analysis Often poorly done or missingOften poorly done or missing Report should address differences Report should address differences

in:in:– Risk and uncertainty of the Risk and uncertainty of the

alternatives (strive to minimize R&U)alternatives (strive to minimize R&U)– Potential for failurePotential for failure– Certainty of outcomeCertainty of outcome– Potential for Adaptive ManagementPotential for Adaptive Management

Page 22: Step 6:  Plan Selection

NER Plan - Additional NER Plan - Additional Considerations Considerations

Partnership contextPartnership context– Cooperative projects have higher priorityCooperative projects have higher priority– Regional or national interagency programsRegional or national interagency programs

Policy Issues Policy Issues – Terrestrial vs AquaticTerrestrial vs Aquatic– Real Estate proportion (< 25% costs)Real Estate proportion (< 25% costs)– Should not require mitigationShould not require mitigation– Recreation may not diminish ecosystem Recreation may not diminish ecosystem

output (cannot increase costs >10%)output (cannot increase costs >10%)

Page 23: Step 6:  Plan Selection

Budget ECBudget ECConsiderationsConsiderations

While not direct role in selection, affects eventual While not direct role in selection, affects eventual ability to advance project; these criteria change ability to advance project; these criteria change over timeover time– Scarcity Scarcity – Connectivity Connectivity – Special Status Species (provides significant Special Status Species (provides significant

contribution to key life requisite of special contribution to key life requisite of special status species)status species)

– Hydrologic character (restoration of natural Hydrologic character (restoration of natural hydrology) hydrology)

– Geomorphic condition (restoration of natural Geomorphic condition (restoration of natural geomorphic processes: erosion, sediment geomorphic processes: erosion, sediment transport, deposition)transport, deposition)

– Plan Recognition (contributes to watershed or Plan Recognition (contributes to watershed or basin plans as emphasized in “CW Strategic basin plans as emphasized in “CW Strategic Plan”)Plan”)

– Self-Sustaining / SustainabilitySelf-Sustaining / Sustainability– Cost per AcreCost per Acre

Page 24: Step 6:  Plan Selection

SustainabilitySustainability What is the sustainability of the plan?What is the sustainability of the plan? Does it work with natural river / Does it work with natural river /

aquatic processes?aquatic processes? Can it be sustained in current setting?Can it be sustained in current setting? Is the project working to address key Is the project working to address key

issues associated with sustainability issues associated with sustainability (dredging & sediment reduction)?(dredging & sediment reduction)?

What are O&M requirements?What are O&M requirements?

Page 25: Step 6:  Plan Selection

Multipurpose PlanMultipurpose Plan“Combined NED/NER Plan”“Combined NED/NER Plan”

For projects with NED & ecosystem For projects with NED & ecosystem restoration benefitsrestoration benefits

No alternative has higher excess NED No alternative has higher excess NED benefits plus NER benefits over total benefits plus NER benefits over total project costsproject costs

Maximize sum of net NED & NER benefitsMaximize sum of net NED & NER benefits– ““Best” balance between objectivesBest” balance between objectives– Based on B/C analysis, CE/ICA, & trade-off Based on B/C analysis, CE/ICA, & trade-off

analysisanalysis

Page 26: Step 6:  Plan Selection

National Interest (Balanced) Plan – National Interest (Balanced) Plan – reflect full range of Federal Interest – reflect full range of Federal Interest –

NED, RED, EQ, and OSENED, RED, EQ, and OSE NED – National Economic DevelopmentNED – National Economic Development

– (FDR, Water Supply, Recreation, etc.)(FDR, Water Supply, Recreation, etc.) RED - Regional Economic DevelopmentRED - Regional Economic Development

– (construction, employment, etc.)(construction, employment, etc.) OSE – Other Social Effects OSE – Other Social Effects

– (effects on tax base, etc.)(effects on tax base, etc.) EQ – Environmental Quality EQ – Environmental Quality

– (ecosystem, water quality, cultural (ecosystem, water quality, cultural resources, etc.)resources, etc.)

Page 27: Step 6:  Plan Selection

Locally Preferred Plan Locally Preferred Plan – May deviate from NED & NER if requested by May deviate from NED & NER if requested by

non-Federal sponsor & approved by ASA(CW)non-Federal sponsor & approved by ASA(CW)– When LPP smaller, usually approvedWhen LPP smaller, usually approved

Assist sponsor in identifying others willing & able to Assist sponsor in identifying others willing & able to participateparticipate

Must have > net benefits than smaller plansMust have > net benefits than smaller plans Sufficient number of alternatives analyzedSufficient number of alternatives analyzed ID tradeoffs & opportunities foregoneID tradeoffs & opportunities foregone Complies w/ laws & policiesComplies w/ laws & policies

– When LPP larger, may be approvedWhen LPP larger, may be approved Sponsor pays differenceSponsor pays difference NED/NER does not meet local objectivesNED/NER does not meet local objectives Outputs similar in kind & = or > than Fed planOutputs similar in kind & = or > than Fed plan Complies w/ laws & policiesComplies w/ laws & policies

Page 28: Step 6:  Plan Selection

Systematic Formulation and Systematic Formulation and Plan Selection OptionsPlan Selection Options

Formulate small plan that makes senseFormulate small plan that makes sense Add justified incrementsAdd justified increments If Sponsor constraint: Stop.If Sponsor constraint: Stop.

Select LPP Select LPP NED / NER / Balanced PlanNED / NER / Balanced Plan If no Sponsor constraint: Maximize net If no Sponsor constraint: Maximize net

benefits.benefits. Select NED / NER / Balanced PlanSelect NED / NER / Balanced Plan

If NED / NER / Balanced Plan does not meet If NED / NER / Balanced Plan does not meet objectives: Add Unjustified Increments.objectives: Add Unjustified Increments.

Select LPP > NED / NER / Balanced PlanSelect LPP > NED / NER / Balanced Plan

Page 29: Step 6:  Plan Selection

NER Example #1: Elizabeth River Ecosystem Restoration

SouthernBranch

Eastern BranchNWestern Branch

Page 30: Step 6:  Plan Selection

Elizabeth River Elizabeth River Ecosystem RestorationEcosystem Restoration

Planning objectives: Planning objectives: – Overall, restoration of the Overall, restoration of the

Elizabeth River’s aquatic & Elizabeth River’s aquatic & wetlands ecosystems wetlands ecosystems

– Specifically:Specifically: Wetlands restorationWetlands restoration Sediment quality restorationSediment quality restoration

Page 31: Step 6:  Plan Selection

Wetlands & Sediment Sites

Somme Avenue

Sugar Hill

Crawford Bay

Page 32: Step 6:  Plan Selection

Wetlands Loss: > 50% since 1944

Page 33: Step 6:  Plan Selection

Sediments Clean-Up OutputsSediments Clean-Up Outputs

• Reduced Sediment Toxicity• Improved Bottom Community Health and Diversity• Reduced Fish Cancers• Improved Sediment Quality

Page 34: Step 6:  Plan Selection

A = Sugar Hill A = Sugar Hill G = Woodstock PkG = Woodstock PkB = Carolanne FarmsB = Carolanne Farms H = Lancelot DrH = Lancelot DrC = Somme Ave C = Somme Ave I = Grandy VillageI = Grandy VillageD = ScuffletownD = Scuffletown J = ODU DrainageJ = ODU DrainageE = NW Jordan BrE = NW Jordan Br K = Prtsmth City PkK = Prtsmth City PkF = Crawford BayF = Crawford Bay

EE +I+I +B+B +F+F+J+J+D+D+G+G +H+H

+K+K+A+A+C+C

CE/ICA Results for Elizabeth River

BreakpointBreakpoint

Page 35: Step 6:  Plan Selection

CE/ICA Results for Elizabeth River

First Best Buy Plan: 0.6 Mean ERM QuotientTotal Cost: $413,800 Total Score: 7.84Incr. Cost: $413,800 Incr. Score: 7.84Incr. Cost/ Unit: $52,781

Second Best Buy Plan: 0.4 Mean ERM QuotientTotal Cost: $890,000 Total Score: 10.29Incr. Cost: $476,200 Incr. Score: 2.45Incr. Cost/ Unit: $194,367

BreakpointBreakpoint

Page 36: Step 6:  Plan Selection

National Ecosystem Restoration National Ecosystem Restoration PlanPlan

Wetlands:Wetlands: – 9 of 11 candidate restoration sites (ranked sites up to & 9 of 11 candidate restoration sites (ranked sites up to &

including Portsmouth City Park) - 19.5 acresincluding Portsmouth City Park) - 19.5 acres– Cost effectiveCost effective, 9th , 9th best buybest buy plan plan– On functional score, On functional score, sharp breakpointsharp breakpoint after P. City Park after P. City Park– On HEP score, On HEP score, breakpointbreakpoint before P. City Park before P. City Park– Include P. City Park: only site on Western Branch Include P. City Park: only site on Western Branch

(completeness)(completeness), complements city’s plan for site, public access , complements city’s plan for site, public access & educational value & educational value (acceptability)(acceptability)

Sediment Restoration:Sediment Restoration:– Medium level clean-up (0.6 SQV)Medium level clean-up (0.6 SQV)– Cost effectiveCost effective, 1st , 1st best buybest buy plan - lowest cost per unit of clean- plan - lowest cost per unit of clean-

up benefit of any alternativeup benefit of any alternative– Sharp Sharp breakpointbreakpoint after medium (0.6 SQV) level after medium (0.6 SQV) level– Substantial benefits include reduced toxicity & contamination, Substantial benefits include reduced toxicity & contamination,

improved benthos & aquatic resourcesimproved benthos & aquatic resources

Page 37: Step 6:  Plan Selection

NER Plan - Additional Decision NER Plan - Additional Decision CriteriaCriteria

SignificanceSignificance - Ches. Bay Agreement - Region of - Ches. Bay Agreement - Region of Concern, priority urban area; LOC’s Local Legacies Concern, priority urban area; LOC’s Local Legacies program; Eliz. River Project - Watershed Action Plan program; Eliz. River Project - Watershed Action Plan to restore riverto restore river

ScarcityScarcity - historic - historic wetlandswetlands loss, few “available” loss, few “available” sites; sites; toxic sedimentstoxic sediments - scarcity of aquatic life: low - scarcity of aquatic life: low diversity, biomass, high cancer rates diversity, biomass, high cancer rates

AcceptabilityAcceptability - ERP, Watershed Action Team: clean- - ERP, Watershed Action Team: clean-up & wetlands #1 & #2 critical areasup & wetlands #1 & #2 critical areas

Non-Federal sponsors - all 4 juris., VA, ERPNon-Federal sponsors - all 4 juris., VA, ERP EffectivenessEffectiveness - addresses 2 greatest problems, large - addresses 2 greatest problems, large

geographic area, interconnected to natural systemgeographic area, interconnected to natural system EfficiencyEfficiency - passes tests of CE/ICA - passes tests of CE/ICA

Page 38: Step 6:  Plan Selection

NER Example #2: Indian River Lagoon – South NER Example #2: Indian River Lagoon – South Comprehensive Everglades Restoration Plan ProjectComprehensive Everglades Restoration Plan Project

Page 39: Step 6:  Plan Selection

Problems: Water Problems: Water QualityQuality

Page 40: Step 6:  Plan Selection

Problems: Water Problems: Water QuantityQuantity

…too much

…too little

Page 41: Step 6:  Plan Selection

Problems: Timing & Problems: Timing & HydroperiodHydroperiod

Wrong timing & distribution of flows

Ditched and drained wetland systems

Page 42: Step 6:  Plan Selection

IRL-S Objectives & IRL-S Objectives & ConstraintsConstraints

Restore Ecological Values:Restore Ecological Values: Re-establish a natural pattern of freshwater flows to the St Re-establish a natural pattern of freshwater flows to the St

Lucie Estuary (SLE) & Indian River Lagoon (IRL)Lucie Estuary (SLE) & Indian River Lagoon (IRL) Improve water quality in the SLE and IRLImprove water quality in the SLE and IRL Improve habitat for estuarine biotaImprove habitat for estuarine biota Increase spatial extent & functional quality of watershed Increase spatial extent & functional quality of watershed

wetlands & native upland/wetland mosaic wetlands & native upland/wetland mosaic Increase diversity & abundance of native plant & animal Increase diversity & abundance of native plant & animal

species, including threatened & endangered species species, including threatened & endangered species Improve Economic Values & Social Well-Being:Improve Economic Values & Social Well-Being: Increase water supplyIncrease water supply Maintain existing flood protectionMaintain existing flood protection Improve opportunities for tourism, recreation, & Improve opportunities for tourism, recreation, &

environmental educationenvironmental education Improve commercial & recreational fisheries Improve commercial & recreational fisheries

Page 43: Step 6:  Plan Selection

Incremental Cost Analysis Results: Incremental Cost Analysis Results: Combined Watershed & Estuary Combined Watershed & Estuary

IndexIndex

Alt 6 w/ artificial SAV habitat

Alt 4 w/ artificial SAV habitat

BreakpointBreakpoint

Page 44: Step 6:  Plan Selection

Telling the Story: Rationale for Telling the Story: Rationale for IRL-S Alt 6IRL-S Alt 6

Best meets planning objectives:Best meets planning objectives:– Restoration of Restoration of estuarineestuarine aquatic ecosystem ( aquatic ecosystem (>> all other alts) all other alts)– Increased spatial extent of Increased spatial extent of watershedwatershed wetlands & uplands (secondary wetlands & uplands (secondary

objective)objective) Reasonably maximizesReasonably maximizes ecosystem output while passing tests of: ecosystem output while passing tests of:

– Cost effectivenessCost effectiveness– (Best Buy) Incremental Cost Analysis (Alt 6 w/ artificial SAV)(Best Buy) Incremental Cost Analysis (Alt 6 w/ artificial SAV)

Provides Provides 95% outputs95% outputs of largest alternative (Alt 4), yet costs $53.4 of largest alternative (Alt 4), yet costs $53.4 million less than Alt 4million less than Alt 4

Lowest per unit costsLowest per unit costs of all alts in production of all outputs (Alt 6 w/ of all alts in production of all outputs (Alt 6 w/ artificial SAV)artificial SAV)

Why include artificial habitat?Why include artificial habitat?– Low total costLow total cost of artificial habitat increment ($630k aaec) of artificial habitat increment ($630k aaec)– ““Jump-start” in benefitsJump-start” in benefits provides immediate results provides immediate results– Builds Builds public supportpublic support by demonstrating “restoration” quickly by demonstrating “restoration” quickly– Strong Strong inter-agency/ stakeholderinter-agency/ stakeholder support support

Page 45: Step 6:  Plan Selection

Who Selects the Plan? Who Selects the Plan? ““Bottom-up” process – project delivery team Bottom-up” process – project delivery team

selects with input from partnersselects with input from partners Chain-of-command decision-makers (vertical Chain-of-command decision-makers (vertical

PDT) review & agree or disagreePDT) review & agree or disagree For continuing authorities, review & approval For continuing authorities, review & approval

by Divisionby Division For congressionally authorized projects, For congressionally authorized projects,

ultimate decision makers are ASA(CW), OMB, ultimate decision makers are ASA(CW), OMB, CongressCongress

Bottom lineBottom line: planners advise; decision-makers : planners advise; decision-makers decide; good internal and external decide; good internal and external communication keycommunication key

Page 46: Step 6:  Plan Selection

Why Plans Don’t Why Plans Don’t Succeed Succeed

Plan is flawed Plan is flawed – wrong objectives; incomplete; bad assumptionswrong objectives; incomplete; bad assumptions

Circumstances change Circumstances change – priorities; policies; people; valuespriorities; policies; people; values

Never funded Never funded – lack of $; prioritieslack of $; priorities

Implementation is blocked Implementation is blocked – decision-makers; interest groups; legal actiondecision-makers; interest groups; legal action

Watch for the signs & take time to reevaluate!Watch for the signs & take time to reevaluate!

Page 47: Step 6:  Plan Selection

Summary Summary No “NED-like” rule to select No “NED-like” rule to select

single NER plansingle NER plan Rather, NER plan is Rather, NER plan is designateddesignated

as the plan that:as the plan that:– Best meets planning objectives & Best meets planning objectives &

constraintsconstraints– Reasonably maximizes ecosystem Reasonably maximizes ecosystem

restoration benefits while passing restoration benefits while passing tests of CE/ICA (“worth it?”)tests of CE/ICA (“worth it?”)

– Meets significance, acceptability, Meets significance, acceptability, completeness, effectiveness, & completeness, effectiveness, & efficiency criteria + R&Uefficiency criteria + R&U