U.S. Department of Energy’sU.S. Department of Energy’sConsolidated Laboratory Consolidated Laboratory

AuditingAuditing and Proficiency Testing and Proficiency Testing

ProgramsProgramsGeorge E. DetsisGeorge E. Detsis

Manager, Analytical Services ProgramManager, Analytical Services ProgramOffice of Corporate Safety Analysis, HS-30Office of Corporate Safety Analysis, HS-30

Presented before the 2009 Department of Presented before the 2009 Department of DefenseDefense

Environmental Monitoring and Data Quality Environmental Monitoring and Data Quality WorkshopWorkshop

March 30 – April 3, 2009March 30 – April 3, 2009San Antonio, TexasSan Antonio, Texas

Presentation OutlinePresentation Outline

•Mission StatementMission Statement

•Auditing BackgroundAuditing Background

•Audit ChecklistsAudit Checklists

•Laboratory Audit FindingsLaboratory Audit Findings

•Laboratory Audit Schedule FY09Laboratory Audit Schedule FY09

•Proficiency Testing BackgroundProficiency Testing Background

•Proficiency Testing Proficiency Testing AccomplishmentsAccomplishments

Program MissionProgram Mission

To foster development of To foster development of quality environmental data of quality environmental data of known and documented known and documented value; thereby, improving the value; thereby, improving the confidence level of decision-confidence level of decision-makers, enhancing public makers, enhancing public trust, and reducing inherit trust, and reducing inherit agency liabilities.agency liabilities.

DOECAP BackgroundDOECAP BackgroundProgram Purpose:Program Purpose:

• Elimination of audit redundanciesElimination of audit redundancies

• Standardization of audit methodology, policies and Standardization of audit methodology, policies and proceduresprocedures

• Communication of Lessons LearnedCommunication of Lessons Learned

• Reduction of overall Department risks and liabilitiesReduction of overall Department risks and liabilities

Program Implementation:Program Implementation:• Administrative funding by DOE-HSS Headquarters through Administrative funding by DOE-HSS Headquarters through

the Analytical Services Programthe Analytical Services Program

• Linked to Site and Contractor laboratory contractual Linked to Site and Contractor laboratory contractual agreementsagreements

• Involves participation by all DOE line and field organizationsInvolves participation by all DOE line and field organizations

• Auditors are primarily DOE contractorsAuditors are primarily DOE contractors

• Auditors are trained and qualifiedAuditors are trained and qualified

• Federal DOE staff supplement laboratory audit staffFederal DOE staff supplement laboratory audit staff

Audit FindingsAudit Findings

Priority I Finding:Priority I Finding:• Key programmatic/systematic failureKey programmatic/systematic failure

• Imminent threat to worker or public safety/healthImminent threat to worker or public safety/health

• Documented unlawful or unethical practicesDocumented unlawful or unethical practices

• Significant data quality failuresSignificant data quality failures

• Failure to implement corrective actions/repeat Failure to implement corrective actions/repeat deficienciesdeficiencies

Priority II Finding:Priority II Finding:• Nonconformance with internal Nonconformance with internal

requirements/procedures requirements/procedures

• Undocumented procedures or processesUndocumented procedures or processes

• Nonconformance with DOE requirements Nonconformance with DOE requirements

Observation:Observation:• Isolated instance or minor deficiencyIsolated instance or minor deficiency

• Opportunity for improvementOpportunity for improvement

Audit ChecklistsAudit Checklists

DOECAP Laboratory ChecklistsDOECAP Laboratory Checklists

Module 1Module 1 - - Quality Assurance Management Quality Assurance Management Systems and Systems and General Laboratory PracticesGeneral Laboratory PracticesModule 2Module 2 - - Data Quality for Organic AnalysesData Quality for Organic AnalysesModule 3Module 3 - - Data Quality for Inorganic and Wet Data Quality for Inorganic and Wet Chemistry Chemistry AnalysesAnalysesModule 4Module 4 - - Data Quality for Radiochemistry Data Quality for Radiochemistry AnalysesAnalysesModule 5Module 5 - - Laboratory Information Management Laboratory Information Management Systems Systems and Electronic Data Managementand Electronic Data ManagementModule 6Module 6 - - Hazardous and Radioactive Materials Hazardous and Radioactive Materials

ManagementManagement

Laboratory Findings byLaboratory Findings byAudit DisciplineAudit Discipline

Findings and Observations/ LaboratoryOver Past Nine Fiscal Years

Laboratory Audit Performance Statistics

0

10

20

30

40

50

60

2000 2001 2002 2003 2004 2005 2006 2007 2008

Fiscal Year Audit Cycle

Find

ings

& O

bser

vatio

ns /

Lab

orat

ory

Findings

Observations

Totals

Laboratory and Commercial Waste

Vendor Contractual Agreements Projected FY09 DOECAP Audited Facilities Utilized by Each DOE Site

05

1015

2025

Oak Ridge Office

Richland Office

Idaho Office

Los Alamos Nat'l Lab

Lawrence Berkley Nat'l Lab

Lawrence Livermore Nat'l Lab

Legacy Management

Nevada Office

Paducah Operations

Pantex Operations

Sandia Nat'l Lab

Savannah River Office

# Audited Labs Used

# Audited TSDFs Used

Common LaboraCommon Laboratortory Findings y Findings

QA Management Systems and General Laboratory PracticesQA Management Systems and General Laboratory Practices

• A lack of SOPs or inadequate SOP content, particularly in sub-A lack of SOPs or inadequate SOP content, particularly in sub-sampling and support equipment maintenance.sampling and support equipment maintenance.

• QAP deficiencies including a lack of direction for internal QAP deficiencies including a lack of direction for internal assessments, certification documentation, major equipment assessments, certification documentation, major equipment documentation, etc.documentation, etc.

• Inadequate or untimely closure of corrective actions.Inadequate or untimely closure of corrective actions.• A lack of logbook review, with incomplete or inconsistent A lack of logbook review, with incomplete or inconsistent

implementation of corrections.implementation of corrections.• Lack of back-up systems to prevent catastrophic loss of samples (i.e., Lack of back-up systems to prevent catastrophic loss of samples (i.e.,

monitoring refrigerators on a calendar day basis).monitoring refrigerators on a calendar day basis).• Inadequate control of log books, notebooks, or operational aids.Inadequate control of log books, notebooks, or operational aids.• Failure to notify clients of accreditation loss or Performance Testing Failure to notify clients of accreditation loss or Performance Testing

failures.failures.

Common LaboraCommon Laboratortory Findings y Findings

Data Quality for Organic AnalysesData Quality for Organic Analyses

• All target analytes are not included in matrix spikes or All target analytes are not included in matrix spikes or laboratory control samples over a two year period.laboratory control samples over a two year period.

• Calibration deficiencies.Calibration deficiencies.

Data Quality for Inorganic and Wet Chemistry Data Quality for Inorganic and Wet Chemistry AnalysesAnalyses

• SOPs do not reflect actual practices being implemented in SOPs do not reflect actual practices being implemented in the lab.the lab.

• Failure to utilize a solid matrix for laboratory control samples Failure to utilize a solid matrix for laboratory control samples during analysis of soil and other solid matrices.during analysis of soil and other solid matrices.

• Consecutive Performance Test failures in soil matrices.Consecutive Performance Test failures in soil matrices.

Common LaboraCommon Laboratortory Findings y Findings Data Quality for Radiochemistry AnalysesData Quality for Radiochemistry Analyses

• Consecutive Performance Test failures in soil matrices.Consecutive Performance Test failures in soil matrices.• Inadequate documentation and references for gamma and alpha Inadequate documentation and references for gamma and alpha

spectroscopy instrument libraries.spectroscopy instrument libraries.• Methods and/or practices do not include all required Methods and/or practices do not include all required

Programmatic QC elements.Programmatic QC elements.

Laboratory Information Management Systems and Laboratory Information Management Systems and Electronic Data ManagementElectronic Data Management

• Lack of computer hardware/software security measures (e.g., Lack of computer hardware/software security measures (e.g., controlled server access, unprotected spreadsheet formulas for controlled server access, unprotected spreadsheet formulas for calculating data).calculating data).

• Inadequate inspections and testing of software and electronic Inadequate inspections and testing of software and electronic data.data.

• An absence of SOPs and system documentation.An absence of SOPs and system documentation.

Common LaboraCommon Laboratortory Findings y Findings

Hazardous and Radioactive Materials ManagementHazardous and Radioactive Materials Management

• Deficiencies in sample receipt of potentially radiological Deficiencies in sample receipt of potentially radiological materials.materials.

• Inadequate tracking and documentation of unused sample Inadequate tracking and documentation of unused sample portions to disposal or return to client. portions to disposal or return to client.

FY09 Laboratory Audit ScheduleFY09 Laboratory Audit Schedule

LaboratoryLaboratory Proposed MonthProposed Month LaboratoryLaboratory Proposed Proposed MonthMonth

S&ME, IncS&ME, Inc October 2008 October 2008 Paragon Paragon March 2009 March 2009AssaigaiAssaigai October 2008 October 2008 SwRI SwRI April 2009 April 2009Test America, Knoxville November 2008Test America, Knoxville November 2008 GEL GEL April 2009 April 2009GPLGPL , Maryland , Maryland December 2008 December 2008 EMAX EMAX

April 2009April 2009USEC PaducahUSEC Paducah December 2008 December 2008 BC Laboratories BC Laboratories May 2009 May 2009ORISE, Oak RidgeORISE, Oak Ridge January 2009 January 2009 USEC Portsmouth USEC Portsmouth May 2009 May 2009Eberline Oak RidgeEberline Oak Ridge January 2009 January 2009 Shaw Shaw May 2009 May 2009BWXT LynchburgBWXT Lynchburg February 2009 February 2009 Test America, Richland Test America, Richland

June 2009June 2009UTB RMALUTB RMAL February 2009 February 2009 Davis and Floyd Davis and Floyd June 2009 June 2009Eberline RichmondEberline Richmond February 2009 February 2009 Caltest Caltest June 2009 June 2009XencoXenco February 2009 February 2009 CEBAM CEBAM June 2009 June 2009MCLMCL March 2009 March 2009 Test America, Arvada Test America, Arvada July 2009 July 2009LionvilleLionville March 2009 March 2009 Shealy Environmental Shealy Environmental July 2009 July 2009GPLGPL , Alabama , Alabama March 2009 March 2009 DataChem SLC DataChem SLC July 2009 July 2009Test America, St. LouisTest America, St. Louis March 2009 March 2009 ARS ARS July 2009 July 2009

* Highlighted Laboratories are both DOE/DOD Audited Laboratories* Highlighted Laboratories are both DOE/DOD Audited Laboratories

MAPEP BackgroundMAPEP Background

Program Purpose:Program Purpose:• To test and evaluate environmental analytical laboratory To test and evaluate environmental analytical laboratory

performance.performance.

• To provide defensible data that instills confidence in DOE To provide defensible data that instills confidence in DOE decision-making.decision-making.

Program Focus:Program Focus:• ISO 17025 Accredited PT Provider with traceability to ISO 17025 Accredited PT Provider with traceability to

NISTNIST

• Radiological, stable inorganic, and organic analyses.Radiological, stable inorganic, and organic analyses.

• Soil, water, air filter, and vegetation media.Soil, water, air filter, and vegetation media.

• Interaction with DOECAP and QSAS.Interaction with DOECAP and QSAS.

- - MAPEP required for all laboratories performing MAPEP required for all laboratories performing analysis for DOEanalysis for DOE

MAPEP LaboratoriesMAPEP Laboratories

MAPEP Laboratory Participants:MAPEP Laboratory Participants:• 109109 Domestic LaboratoriesDomestic Laboratories

• 17 17 International LaboratoriesInternational Laboratories

- Nuclear Test Ban Treaty Laboratories- Nuclear Test Ban Treaty Laboratories

New Zealand (1); Brazil (1)New Zealand (1); Brazil (1)

- Cooperative Monitoring- Cooperative Monitoring

Canada (2); United Kingdom (2)Canada (2); United Kingdom (2)

- Middle Eastern Nations- Middle Eastern Nations

Jordan (3); Irag (1); Oman (1); Jordan (3); Irag (1); Oman (1); Kuwait (1);Kuwait (1);

Qatar (1); Turkey(1); Morocco (1); Qatar (1); Turkey(1); Morocco (1); Israel (1)Israel (1)

- International Atomic Energy Agency (1)- International Atomic Energy Agency (1)

MAPEP AccomplishmentsMAPEP Accomplishments

DOE’s - Radiological and Environmental Sciences DOE’s - Radiological and Environmental Sciences Laboratory (RESL) on the Idaho National Laboratory (RESL) on the Idaho National Laboratory, Idaho Falls, IdahoLaboratory, Idaho Falls, Idaho

• ISO 17025 Accredited Proficiency Test Provider ISO 17025 Accredited Proficiency Test Provider (American Association for Laboratory Accreditation (American Association for Laboratory Accreditation (A2LA) approval on a 2 year cycle)(A2LA) approval on a 2 year cycle)

• RESL won A-76 Study competition for outsourcingRESL won A-76 Study competition for outsourcing

• Letter of Obligation – formalizes work commitments Letter of Obligation – formalizes work commitments

(3 yrs./2 option years)(3 yrs./2 option years)

• MOU – NE/HSS/RESL defines roles/responsibilitiesMOU – NE/HSS/RESL defines roles/responsibilities

MAPEP InitiativesMAPEP Initiatives

• Test session 20 distribution (March 2009)Test session 20 distribution (March 2009)

• Test session 21 distribution (September 2009)Test session 21 distribution (September 2009)

• 60 day timeframe for laboratory reporting test 60 day timeframe for laboratory reporting test results to RESLresults to RESL

• New remedial PT sample policy – one remedial New remedial PT sample policy – one remedial attempt, then must also pass subsequent routine attempt, then must also pass subsequent routine PT sessionPT session

• HSS Letters of Concern for consecutive PT failuresHSS Letters of Concern for consecutive PT failures

• Track and trend reasons for Letters of ConcernTrack and trend reasons for Letters of Concern

Proficiency Testing Analyses

Fiscal Year SeriesNumber of

MAPEP Samples# of Analyses by

Laboratories

FY07 15 & 16 1098 13628

FY08 17 & 18 1136 13605

FY09 19 & 20 1200 14000

MAPEP Matrix Series 19 Matrix Id. Total Samples Foreign Labs

Mixed-Analyte Soil MaS 113 16

Mixed-Analyte Water MaW 133 17

Semi-volatile Organic Water OrW 38 0

Radiological Vegetation RdV 70 15

Radiological Air Filters RdF 87 13

Gross alpha/beta Water GrW 72 11

Gross alpha/beta Filter GrF 75 8

Samples Distributed to Participating Laboratories, MAPEP Series 19 (2008)

Increase in Samples Distributed and Analyses by Laboratories

Laboratory Auditing and Proficiency Test Preparation

Radiological Analysis Liquid Scintillation Counting

2009 ASP Workshop2009 ASP Workshop

Date:Date: September 13-17, 2009 September 13-17, 2009

Location:Location: Snow King LodgeSnow King Lodge Jackson Hole, Jackson Hole,

Wyoming Wyoming

ASP Program Contact InformationASP Program Contact Information

George E. Detsis, George E. Detsis, Analytical Services Program (ASP) Analytical Services Program (ASP) ManagerManagerPhone: (301) 903-1488Phone: (301) 903-1488E-mail:  E-mail:  George.Detsis@hq.doe.gov

Jorge Ferrer - Jorge Ferrer - DOECAP Manager, DOECAP Manager, DOE Oak Ridge DOE Oak Ridge Phone: (865) 576-6638Phone: (865) 576-6638E-mail:  E-mail:  ferrerjaferrerja@oro.doe.gov

Guy MarletteGuy Marlette, , MAPEP Coordinator, MAPEP Coordinator, Idaho National LaboratoryIdaho National LaboratoryPhone: (208) 526-2532Phone: (208) 526-2532E-mail: E-mail: marletgm@id.doe.govmarletgm@id.doe.gov

Brent PulsiferBrent Pulsifer, , VSP Coordinator, VSP Coordinator, Pacific Northwest National Pacific Northwest National LaboratoryLaboratoryPhone: (509) 375-3989Phone: (509) 375-3989E-mail: E-mail: Brent.Pulsipher@pnl.govBrent.Pulsipher@pnl.gov