certification: assisting the ir measurement community randy j. jost utah state university roger w....
Post on 22-Dec-2015
215 views
TRANSCRIPT
Certification: Assisting the IR Measurement Community
Randy J. JostUtah State University
Roger W. DavisEG&G Technical Services, Inc.
6th Annual RCS Measurement Facilities Certification Conference9-11 July 2002
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
2
"Measurement science today is a race between engineers and physicists trying to develop better, and more detailed, idiot-proof calibration and characterization procedures, and the universe trying to build bigger and better idiots.
So far, the universe is winning."
R. Jost
Motivation #1
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
3
"Nothing is fool-proof
to a talented fool”- Unknown
“Good documentation helps find
the proper fool to counsel”- R. Jost
Motivation #2
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
4
This briefing will cover:
• What is ANSI Z540 and what are its origins?
• Why and how has ANSI Z540 been applied in signature measurements?
• Issues for the IR measurement communities to address in order to employ ANSI Z540.
• How do we get there from here?
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
5
ANSI Z540 and its origins?
• ANSI/NCSL Z540-1-1994 (Formal name)
• American National Standard for Calibration – Calibration Laboratories and Measuring and Test Equipment – General Requirements (Formal title)
– Approved 27 July 1994
“The intent of this Standard is that laboratories meeting its requirements comply, for calibration activities, with the relevant requirements of the ISO 9000 series (ANSI/ASQC Q 90 series) of standards, … when they are acting as suppliers producing calibration results.”– From the Foreword, emphasis added
• American implementation of ISO Guide 25
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
6
Some Background
• Radar Cross Section (RCS) measurement community joined forces to collectively raise the quality bar
• Primarily DoD and its contractors, with important contributions made by NIST, NASA & academia
• National Institute of Standards & Technology (NIST) surveyed existing standards
– Found ISO Guide 25 met many of the needs of the RCS community
• ANSI/NCSL Z540-1-1994 US implementation
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
7
RCS Community Adopted ANSI Z540
• ISO 9000-like approach to improve quality of RCS measurements– Primarily a documentation standard
– Provides focus on key issues common to RCS measurement facilities of all kinds
– Enables uniform concepts & language for addressing & discussing quality issues
– Can be used in a certification program, but not necessarily so
• the term certification caused some friction within the RCS community
• Lessons learned can be applied elsewhere
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
8
History of RCS Certification Program• Initiated 1995 in DoD by USAF & Navy
– Major proponents: Air Force Research Laboratory & Naval Air Warfare Center Aircraft Division
– Other DoD participants: 46th Test Wing (Holloman & Eglin), Army Research Lab & Aberdeen Test Center, Air Force Information Warfare Center
• NIST playing key role– Objectivity and degree of rigor not commonly found in DoD
• Industry on board– Lockheed Martin, Boeing, Northrup-Grumman, Raytheon, Pratt &
Whitney, GE Aircraft Engines, BAE
• First two DoD facilities certified June 2000 as compliant with published evaluation criteria based on Z540– First Industry facility certified in June 2001; Six facilities now
certified
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
9
Certification: What it is & is not• Certification, in context of National RCS
Certification Program, means:– Reviewed by independent reviewers using published
evaluation criteria and found to be substantially compliant with the documentation provisions of Z-540
• Certification does NOT mean:– A facility can make measurements to some degree of
accuracy or other quantitative metric
• Why the distinction?– Quantitative assessments readily and often abused,
especially by political hacks
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
10
Possible Benefits to Other Communities
• Rigorous & methodical approach helps focus on common issues impacting quality of data
• Document what you do, who does what, how it is done, why, what training is required, what processes & procedures are followed, & when
• Establishes regular reviews (internal / external)
• Provides excellent visibility into your operations by customers and potential customers
– Assures customers they are getting more than lip service when it comes to quality data products
Value to training new personnel, alone, is worth the effort!
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
11
Possible Benefits to Other Communities
• Note that these benefits are available to any entity, group or community that is required to perform calibrated measurements as part of their operations.
• For the remainder of this briefing, we will focus on the various IR measurement communities.
• The IR community is a much more segmented group than the RCS community, and thus faces additional challenges and advantages.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
12
IR community-specific issues
• Challenges
– More types of measurements => more types of calibration approaches.
• reflection, emission, transmission and absorption processes.
• Incoherent, partially coherent, coherent radiation
• Transmission path a major player
– Spectrum of interest is much more “diverse” than that of the RCS community.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
13
IR community-specific issues
• Advantages
– NIST is already involved, especially the Gaithersburg office.
– “Basic science” of calibration has been examined for a longer period of time, and is potentially better understood.
– More people are looking at the problem => more approaches to consider (note that this could be construed by some people to be a challenge).
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
14
IR community-specific issues
• Benefits
– Many of the same benefits that the RCS community has seen
– Will help develop a priority list of “best defined” characterization/calibration methods
– Facilitate filling in the calibration and methodology “gaps” that may be identified.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
15
IR community-specific issues
• How do we get there?
– Realize that as a community it won’t happen overnight, but as an individual entity, benefits are available in the short run
– Need to identify 1-2 prime movers with an ownership mentality
– Find a means to achieve a clearly defined and early success
– Take advantage of the RCS community’s expertise, but tailor to IR community’s needs
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
16
IR community-specific issues
• How do we get there?
– Establish contacts in participating organizations.
• 2001 RCS Conference: ~ 34 organizations
• 2002 SDL IR Conference: ~ 40 organizations
• 12 of these organizations were common to both conferences, Government, Industry and Academia
– NIST, AF Metrology, NASA-Langley, MIT/LL, AEDC
– Ball Aerospace, Lockheed-Martin, Northrop-Grumman, Raytheon, SAIC, Sverdrup/AEDC
– USU
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
17
Range Books = One Approach1 Introduction and Endorsement2 References3 Glossary4 Organization and Management5 Quality System, Audit, and Review6 Personnel7 Accommodation and Environment8 Equipment and Reference Materials9 Measurement Traceability and Calibration10 Calibration Methods
11 Handling of Calibration Items12 Records13 Certificates and Reports14 Subcontracting of Calibration15 Outside Support Services and Suppliers16 Complaints17 Inter-range Comparison Programs18 Data Processing Procedures19 Range-Specific Uncertainty Analysis20 Ongoing Research, Planned Improvements
Sections 9, 10, 17 & 19 deserve and receive considerable attention and yield tremendous benefit through rigorous examination; however, the holistic approach is necessary to ensure consistent quality across the board.
If applied in a certification program, sections such as these best applied with a priority and weighting scheme.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
18
ANSI Z540 / ISO Guide 25 / ISO 17025?
• At the outset, the application of ISO Guide 25 & ANSI Z540 ran into some difficulties relating to ISO 9000
• Re-write to address those issues yielded ISO 17025– published in late 1999
• 5-year migration plan in US
• If starting a completely new quality or certification program in a signature measurement discipline, consider 17025
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
19
Summary• Across-the-board data quality improvement was
the original impetus; actual, measurable benefits to date far exceed that goal
• What makes sense for your facility is key element of National RCS Certification Program– flexibility, ingenuity, applicability
• A well-structured, phased approach built on the approach that the RCS community used can be also be used by the IR community– Identify a prime mover and a champion (with
resources)
– Locate and use all (experienced) company resources.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
20
Summary• We all face tremendous personnel issues
– training = critical component to ensuring adequate skill sets
– documentation is key to any training program related to complex systems
• Z-540 offers an approach if you need to “eat the elephant”: – do it one orderly bite at a time
– documenting how you hold the fork and knife as well as the number of times you chew and swallow.
• This approach is based on a process, thus, develop a long term vision.
IR Certification, Jost & Davis @ 6th Annual RCS Measurement Facilities Certification Conference - 9 July 02
21
Concluding Thought
"If you can't describe what you
are doing as a process, you
don't know what you're doing."
- W. Edwards Deming
“If you can’t document your process,
no one else will believe you
know what you are doing.”
- R. Jost
Questions / Comments ?Randy J. Jost
Assistant Professor of Electrical & Computer Engineering, Utah State University(435) 797-0789
Roger W. DavisConsultant
EG&G Technical Services, Inc.(301) 863-0443
For a copy of this presentation, go to:
http://www.engineering.usu.edu/ece/faculty/rjost/Research/Research.html