tier 3 webinar
TRANSCRIPT
1
2
HORIBA Webinar November 19, 2015
Tier 3 – Performance Based Approach for Laboratory Regulatory Measurements
•Control of Air Pollution From Motor Vehicles: Tier 3 Motor Vehicle Emission and Fuel Standards •United States Code of Federal Regulations (CFR), Title 40 Part 80, subpart D, section 80.47
3
HORIBA Webinar November 19, 2015
Disclaimer: The information represented and presented here is not formal USEPA, HORIBA Scientific or ASI Standards
documentation. It is provided solely for the convenience of interested parties. Users / Webinar attendees assume all risk and liability for any use of any or all information presented.
HORIBA Scientific and ASI Standards makes no representations or warranties concerning this information and
expressly disclaims any implied warranties, including warranties of merchantability for a particular purpose.
Caution should be taken as changes in applicable regulations could occur at any time.
U.S. EPA Tier 3 Goals and Status• Motor Vehicles and Fuels as an Integrated System: • Tier 3 Motor Vehicle Emissions
– More stringent vehicle emissions standards• Fuel Standards
– Reduce sulfur in gasoline beginning in 2017• Now averaging in the 30 mg/kg S range• Future 8-11 mg/kg S, or less
– Ethanol content • Newly defined Specification Fuel has 10 volume %• Future – 15 volume % ?
• California, Europe and Japan currently have gasoline specifications similar to Tier 3
• Final Rule Published April 28, 2014• Revision Published February 19, 2015
4
Tier 3 Influences
• Tier 3 Impacts are far reaching and relate to:– Motor Vehicle Emission and Fuel Standards– Non-road Engine and Equipment Programs– MARPOL Annex VI Implementation
• The performance-based measurement system (PBMS) approach enacted for Tier 2 diesel fuels is made mandatory.– Performance Testing– Quality Control
• Self-Qualification for Reporting Labs 5
Tier 3 Impact – 1)• EPA recognized approving methods through rulemaking
is a barrier for innovative analytical test methods.– EPA decided to incorporate the PBMS approach
• Consistent with the Agency’s PBMS, EPA proposes not to require the use of specific, prescribed analytical methods.
• The PBMS approach is intended to be more flexible and cost-effective for the regulated community; it is also intended to encourage innovation in analytical technology and improved data quality.
• EPA is not precluding the use of any method, whether it constitutes a voluntary consensus standard or not, as long as it meets the performance criteria specified.
6
Tier 3 Impact – 2)• Once fully implemented, this rule will:
– (1) Requires individual laboratories to demonstrate adequate measurement quality
– (2) allow laboratories to choose methods that meet their own needs, provided they can meet prescribed criteria rather than specific methods
– (3) requires all laboratories making regulatory measurements to establish and maintain a statistical quality control program
• Qualification process intent - demonstrate the facility’s capability with the test method they choose.– Every facility operator is not required to qualify– Labs may have multiple qualified instruments using one or
more methods 7
Tier 3 Impact – 3)• PBMS is based upon an established performance criteria
for accuracy and precision.• Laboratories demonstrate that a particular analytical test
method is acceptable for demonstrating compliance.– Precision - The degree of agreement in a set of measurements
performed on the same property of identical test material.– Accuracy – The closeness between an observed value from a
test measurement and an accepted reference value (ARV).– QC Program - require all labs making regulatory measurements
to establish / maintain a statistical quality control (SQC) program.• Beginning January 1, 2016, sulfur in gasoline must
be determined by a test method approved under § 80.47 - Performance-based Test Method Approach.
• EPA predicts -750 labs subject to info collection.
8
Tier 3 Definitions (1• Absolute fuel parameter - a fuel parameter for which a
gravimetric standard is practical to construct and use. Sulfur content of gasoline or diesel fuel are examples.
• Method-defined fuel parameter - a fuel parameter for which an EPA-prescribed primary test method or designated method defines the regulatory standard. Examples include Reid vapor pressure (RVP) and distillation parameters.
• Correlation equation (For method-defined fuel parameters) - a correction equation as determined by ASTM standard practice D6708. This standard practice determines whether the comparison between the alternative test method and the designated test method is a null result (No Bias). If the comparison is not null (Bias Found), then the standard practice provides for a correlation equation (Correction Factor) that predicts designated test method results.
9
Tier 3 Definitions (2• Voluntary consensus-based standards body (VCSB)
means a domestic or international organization that plans, develops, establishes, or coordinates voluntary consensus standards using agreed-upon procedures and which possesses the attributes of openness, balance of interest, due process, and consensus, as explained in OMB Circular A-119 and the National Technology Transfer and Advancement Act of 1995, P.L. 104-113, sec. 12(d).
• Non-voluntary consensus-based standards body (non-VCSB) means a domestic or international regulated party that has developed a proprietary analytical test method that has not been adopted by a VCSB organization.
10
Tier 3 Definitions (3• Non-voluntary consensus-based standards body (non-
VCSB) Test Method Hurdles:• The test facility must provide:• Full test method documentation, including a description of
the technology and/or instrumentation that makes the method functional.
• Accuracy Test and Precision Test• Bias examination per D6708• Viable SQC Program• An independent third-party scientific review and written
report and verification of the information provided • Submit Information to EPA Director, No self-qualification
11
Tier 3 Sulfur PBMS Tests and SQC (1Absolute fuel parameter-VCSB
• Sulfur repeatability - (r)• Maximum allowable standard deviation (STDEV) computed
from a minimum 20 results made over at least 20 days – Minimum 5 Batches (group of results) – Maximum 4 Tests per Batch
• Calculation = Maximum STDEV ≤ 1.5 * (r / 2.77) – ‘‘r’’ equals the repeatability of D7039–13
• Chosen because it had biggest r of alternate methods • Example: A 10 mg/kg sulfur gasoline sample:
– maximum allowable standard deviation of 20 gasoline tests
≤ 1.5*(1.76 mg/kg / 2.77) = 0.94 mg/kg• Compare 20 Diesel (5-15 mg/kg S) – Max STDEV = 0.72
12
Tier 3 Sulfur PBMS Tests and SQC (2Absolute fuel parameter - VSCB
• Sulfur accuracy• Average of at least 10 continuous tests performed on a
commercially available gravimetric sulfur standard; testing within two ranges is required.– In the range of 1–10 mg/kg shall not differ from the ARV of
the standard by more than 0.75 * Maximum STDEV for the level being tested
– In the range of 10–20 mg/kg shall not differ from the ARV of the standard by more than 0.75 * Maximum STDEV for the level being tested
• In applying the accuracy tests individual test results shall be compensated for any known chemical interferences.
13
MESA 7220 Tier 3 Testing Results – repeatability (r) @ 10.0 mg/kg S
Tier 3 - r Testmg/kg S – 20 tests
STDEV ≤ 1.5 * (r / 2.77)
Test # Result
Day 1
1 9.12 10.73 9.9
Day 5
4 9.15 9.96 10.4
Day 9
7 9.78 9.79 9.3
10 9.414
Test # Result
Day 15
11 10.912 9.413 9.7
Day 19
14 9.015 10.616 9.317 10.5
Day2218 9.919 9.720 10.4
Mean 9.8STDEV 0.58
Max STDEV for 10 mg/kg = 0.94
PASS 0.58 is less than 0.94 mg/g S
PBMS Precisionmg/kg
S r Max STDEV
3 0.9 0.494 1.1 0.575 1.2 0.656 1.3 0.717 1.4 0.778 1.5 0.839 1.6 0.8910 1.7 0.9411 1.8 0.9912 1.9 1.0413 2.0 1.0814 2.1 1.1315 2.2 1.17
MESA 7220 Tier 3 Testing Results – repeatability (r) @ 6.0 mg/kg S
Tier 3 - r Testmg/kg S – 20 tests
STDEV ≤ 1.5 * (r / 2.77)
Test # Result
Day 1
1 5.82 6.73 6.64 6.4
Day 5
5 6.56 4.87 5.9
Day 9
8 69 6.3
10 6.515
Test # ResultDay 14
11 6.112 6.313 5.9
Day 17
14 515 5.916 5.317 5.9
Day 21
18 6.619 5.920 5.8
Mean 6.0STDEV 0.52
Max STDEV for 6 mg/kg S = 0.71
PASS 0.52 is less than 0.71 mg/g S
PBMS Precisionmg/kg
S r Max STDEV
3 0.9 0.494 1.1 0.575 1.2 0.656 1.3 0.717 1.4 0.778 1.5 0.839 1.6 0.8910 1.7 0.9411 1.8 0.9912 1.9 1.0413 2.0 1.0814 2.1 1.1315 2.2 1.17
MESA 7220 Tier 3 Testing Results - AccuracyTier 3 Low Sulfur accuracy - Gasoline
Average of 10 continuous tests commercially available sulfur standard range of 1–10 ppm within
Accuracy Calculation = (MAX STDEV x 0.75)
16
Low 1-10 mg/kg STest # Result
1 5.12 5.33 5.74 5.45 5.46 5.27 4.98 4.89 4.610 4.8
Mean 5.12mg/kg S 5.0
∆ COA 0.12
∆ of Mean did not exceed 0.48 mg/kg SorPASS - 0.1 is less than 0.48 mg/g S
PBMS Accuracymg/kg
S r Max STDEV Max ∆
3 0.9 0.49 0.374 1.1 0.57 0.435 1.2 0.65 0.486 1.3 0.71 0.537 1.4 0.77 0.588 1.5 0.83 0.629 1.6 0.89 0.66
10 1.7 0.94 0.7011 1.8 0.99 0.7412 1.9 1.04 0.7813 2.0 1.08 0.8114 2.1 1.13 0.8415 2.2 1.17 0.88
Example Tier 3 Testing Results - AccuracyTier 3 High Sulfur accuracy - Gasoline
Average of 10 continuous tests commercially available sulfur standard range of 1–10 ppm within
Accuracy Calculation = (MAX STDEV x 0.75)
17
High 10-20 mg/kg STest # Result
1 15.52 15.73 15.94 15.85 16.16 15.97 15.58 15.89 15.910 16.0
Mean 15.81mg/kg S 15.0
∆ COA 0.81
∆ of Mean did not exceed 0.88 mg/kg SorPASS - 0.81 is less than 0.88 mg/g S
PBMS Accuracymg/kg
S r Max STDEV Max ∆
3 0.9 0.49 0.374 1.1 0.57 0.435 1.2 0.65 0.486 1.3 0.71 0.537 1.4 0.77 0.588 1.5 0.83 0.629 1.6 0.89 0.66
10 1.7 0.94 0.7011 1.8 0.99 0.7412 1.9 1.04 0.7813 2.0 1.08 0.8114 2.1 1.13 0.8415 2.2 1.17 0.88
Record Keeping
• Documentation - Records of accuracy and precision testing and any investigations into any exceedance of these criteria must be kept by the facility for a period of five years.
• Once precision and accuracy testing requirements are established measurement quality is maintained through mandatory Tier 3 statistical quality control (SQC) program requirements per cited sections of ASTM D-6299 – Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
18
Statistical Quality Control (SQC)Code of Federal Regulations (CFR), Title 40, Part 80, subpart D, section 80.47, subsection (n) Accuracy and
Precision Statistical Quality Control (SQC) Requirements for the Absolute Fuel Parameters.
• Beginning January 1, 2016, a test shall not be considered a test using an approved test method unless the following quality control procedures are performed separately for each instrument used to make measurements:– Subsection (1)(i) Accuracy SQC– Subsection (2)(i) Precision SQC– References to ASTM D6299 - Standard Practice for
Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance 19
Accuracy SQC • Mandatory at-least 3 times a year testing each instrument with
gravimetric reference material or check standard as defined in ASTM D6299
• 3.2.3 check standard, n—in QC testing, a material having anaccepted reference value used to determine the accuracy of ameasurement system.• 3.2.3.1 Discussion—A check standard is preferably a materialthat is either a certified reference material with traceability to a nationally recognized body or a material that has an accepted reference value established through interlaboratory testing. For some measurement systems, a pure, single component material having known value or a simple gravimetric or volumetric mixture of pure components having calculable value may serve as a check standard.
20
Precision SQC • Every facility shall conduct tests on every instrument with
a quality control material as defined in paragraph 3.2.8 in ASTM D6299 either once per week or once per every 20 production tests, whichever is more frequent.
• D6299 - 3.2.8 quality control (QC) sample, n—for use in quality assurance programs to determine and monitor the precision and stability of a measurement system, a stable and homogeneous material having physical or chemical properties, or both, similar to those of typical samples tested by the analytical measurement system. The material is properly stored to ensure sample integrity, and is available in sufficient quantity for repeated, long term testing.
21
Example Precision SQC
22
Example Accuracy SQC
23
Other SQC • Validation of New QC Material. When making a transition from
one batch of QC material to the next batch, the facility will either:
• Construct an “I” chart as described in section 8.7 and section A1.5.1 of ASTM D6299, or
• Follow the “Q-Procedure” in Annex 1.9 of ASTM D6299. – In following the Q-Procedure, if the plot from the “old” and “new” QC
materials on its respective chart shows no special-cause signals, then the result of the “new” QC material will be considered valid.
– For test facilities opting to use the Q-procedure, the first run on the new QC batch should be validated by either an overlap in-control result of the old batch, or by a single execution of an accompanying standard reference material.
– The new QC material result would be considered validated if the single result of the standard reference material is within the established site precision (R') of the Accepted Reference Value of the standard reference material
24
25
Tier 3 TimelineReview
• USEPA Control of Air Pollution From Motor Vehicles: Tier 3 Motor Vehicle Emission and Fuel Standards – DONE, – Final rule Published April 28, 2014– PBMS testing mandatory by January 1, 2016– Tier 3 Vehicles mandatory by 2017 model year
• Marpol 73 / 78, Annex VI Testing– MARPOL 73/78 is the International Convention (Treaty) for
the Prevention of Pollution from Ships. Originally adopted by the International Maritime Organization (IMO) in 1973 and revised in 1978. Its goal is to reduce pollution from all aspects of marine vessel operation.
– Annex VI – Max S levels of 0.1000 mass % effective January 1, 2015
26
The Way Forward
27
The Way Forward
Tier 3 Qualification Offerings from ASI Standards
Chris Carolan – Product Marketing Manager
Materials for Accuracy Testing• SEPA-T3-L-500 - Sulfur in Gasoline Check Standard
Used for Tier 3 Low Accuracy Test, 500 ml each, Concentration 1-10 mg/kg (ASI Recommends 9 mg/kg)
• SEPA-T3-H-500 - Sulfur in Gasoline Check Standard Used for Tier 3 High Accuracy Test, 500 ml each, Concentration 10-20 mg/kg (ASI Recommends 19 mg/kg)
• SEPA-T3-SQCA-1GAL - Sulfur in Gasoline Quality Control Standard Used for Tier 3 Accuracy SQC, 1 Gallon each, Concentration 1-20 mg/kg (ASI Recommends 7-10 mg/kg)
Materials for Precision Testing• SEPA-T3-P-1000 - Sulfur in Gasoline Quality
Control Standard Used for Tier 3 Precision Test, 1000 ml each, Concentration 1-20 mg/kg (ASI Recommends 6-10 mg/kg)
• SEPA-T3-SQCP-1GAL - Sulfur in Gasoline Quality Control Standard Used for Tier 3 Precision SQC, 1 Gallon each, Concentration 1-20 mg/kg (ASI Recommends 8 -10 mg/kg)
Thank you for your time!
34
35
36