asqc quality assurance guidelines document.asqc quality assurance guidelines document draft 1...

ASQC R&D COMMITTEE

ASQC QUALITY ASSURANCE GUIDELINES DOCUMENT

DRAFT 1

3/31/88

AND

DRAFT 0

4/27/87

6"040701 2o80WS 1 WASTE 07

PDR

ASQC Quality Assurance Guidelines DocumentDraft 1 3/:31/88

INTRODUCTION

This Guidelines Manual is intended to provide guidancein the application of Quality Assurance practices to researchand development activities. It is intended for use by -

researchers and research management. Research projects-varyfrom those that take months or years and many millions ofdollars to complete to those that take just a few days orhours and are relatively inexpensive. Not all the guidelinesdescribed here will apply to all projects. The researchershould pick and choose the use of the guidelines given inthis document as they apply to a particular project.

Researchers in all scientific disciplines have inheritedgood laboratory practices" from the scientist and

researchers that have gone before them. These good laboratorypractices resulted from the lessons learned in the quest forvalid, accurate, reproducible data. The practices providecontrol over the various sources of loss and failure that canoccur at any time in any experiment. Good laboratorypractices are used by the researcher to help assure that allresearch activities are properly planned and executedresulting in reliable data which is necessary to supportpatentable processes, material, systems or hardware.

A research environment is quite distinctive.Researchers tend to be highly educated and highly selfmotivated people. Creativity and individuality are not onlyencouraged but are requirements of the job. Mistakes, nottolerated in the manufacturing environment, are commonplace.Research is trial and error. Failures may occur until thecorrect solution is arrived at. It would appear then, thatquality assurance practices that are aimed toward doingthings right the first time so as to preclude failures wouldbe incongruous with the concept of trial and error.Actually, a comparison of quality assurance procedures togood engineering practices shows that the two are notmutually exclusive and that researchers have in reality beenusing some quality assurance procedures all along.

As a necessary foundation for any Quality Program theremust be solid support from management. With the properculture, employees know that management truly andenthusiastically believes in the Quality Program. This beliefis demonstrated when management helps to set the goals of thequality program, authorizes and gives full support to qualityimprovement projects, and then periodically evaluatesadherence to and effectiveness of the quality program. Whenupper management shows this much concern and commitment,others in the organization will realize that quality is ofgreat importance.

tr A ,- r I I

ASQC Quality Assurance Guidelines DocumentDraft 1 3/31/88

The end product of most research efforts is data. In alarge percentage of cases, this data becomes design inputsand its accuracy is critical to design and other processes.The goal of the quality program should be to ensure that thework performed achieves the intended objectives of theresearch project and that the work can be understood and ifnecessary, reproduced successfully by others. The guidelinescontained in the following sections of this manual shouldhelp to ensure the integrity, adequacy, accuracy, precision,reproducibility, and retrievability of the data.

PAClF "


I. Responsibility

The responsibility of ensuring that proper qualitycontrols and checking activities are carried out should beplaced upon the individual project leader. A project leaderis the person responsible for the planning, tracking andreporting on a project. The individual project leader is bestable to decide which technical elements of the project affectthe quality or validity of the resulting data. Based onthese decisions the project leader should then determine whatelements of the QA program are applicable to the project. Theproject leader may call on the quality department to helphim/her make these decisions. It should also be theresponsibility of the project leader to ensure that allspecial customer requirements, such as materialcertifications, inspections, special tests or instrumentcalibrations are performed and appropriate documentation onfile. Further, it should be the responsibility of the projectleader to identify appropriate resources, technical andmanagerial, that are needed to meet the customer'srequirements. It is suggested that the project leader providean organizational chart and/or a list of responsibilities sothat each project member is aware of his/herresponsibilities. And finally the project leader isaccountable to their management for the successful completionof the project.

II. Project Planning

It has been shown that the successful completion of aresearch project is directly proportional to the quality offront end planning which is performed. The project leadershould in all cases discuss with the customer and come to anagreement on the workscope and milestones of the proposedproject.

A detailed planning document should be developed todescribe and control the project. The plan can be a separatestand alone document, a contract statement of work, aresearch proposal or some other project authorizationdocument as long as it meets certain criteria. An adequateplan is detailed enough that a technically competent personcan understand and perform the required work without recourseto project personnel for clarification. When writing theplan, the project leader should consider defining projectgoals and objectives, the design of experiment and theanalysis required, test planning, pre-test reviews,limitations of the approach and probability of success,parameters to be investigated, verification processes to beutilized, test apparatus required, acceptable measurementuncertainty, whether formal inspections are required, whethercustomer approval of material substitutions is required, dataevaluation methods required, reports required and any otherdeliverables. Preparation and submission of this work plan to

lDArV 9


the customer should be the first milestone of the project.The purpose of submitting the plan to the customer is toprovide the customer with the details of the project asunderstood by the project leader. If the customer disagreeswith any item of the test plan this allows him ampleopportunity to clarify the work required. Copies of the planshould be provided to project team members as appropriate.

Planning should include facilities requirements andavailability. Such items as buildings, utilities, space andtemporary project labor may be needed to fulfill projectrequirements. The project leader should inform supportpersonnel of the project's requirements at the outset ofproject planning in order to avoid costly delays later.

Included in this planning should be safetyconsiderations of the project. The project leader isresponsible for assuring that all work activities of theproject are performed according to applicable safety rules,approved procedures, and government safety standards. Theproject leader should know the hazards and the approvedprocedures related to the work which will be performed.

The project leader should be aware of such safetyconcerns as personal protective equipment, hazardous chemicaluse, storage and disposal, and radiation hazards. Safetyconcerns should be discussed with the Laboratory's SafetyOfficer at the outset of project planning to avoid hazardousor life threatening situations latter in the project.

III. Statistical Design of Experiment

Statistically designed experiments are recommended bothfor projects investigating parameters and their interactiveeffects, and for experiments seeking optimum conditions.Design of experiment techniques should also be used when theoutcome of an experiment is to be defined by an equation.

The documented design should include a statement of thetest objectives and a design matrix. The statement of thetest objectives may be in the form of questions to beanswered, the hypotheses to be tested, or the parametereffects to be estimated. The design matrix should consist ofthe parameter combinations to be tested to meet the testobjectives.

PA(1T'P A


IV. ProJect Control

Design Reviews

The purpose of a design review program is to assurehigh quality, reliable, cost effective products or processesbeing developed or remodeled by the Laboratory. Designreviews can also be extended to new or redesignedexperimental facilities within the Laboratory. Need for adesign review may be dictated by the customer. In those caseswhere the customer has no requirement the need should beconsidered on a case-by-case basis by the appropriatemanager. Design reviews dealing with experimental facilitiesmay not need to meet all the suggested requirements whichfollow and should be conducted at a level appropriate to meetthe requirements of the particular situation.

It is recommended that at least two reviews beperformed. The first should be during the initial stages ofplanning to concur with design specifications.

Additional reviews should be performed after the designhas been finalized and before transfer of the product orprocess to the customer or construction of the facility takesplace. These reviews should be performed in formal documentedmeetings having formal agendas. Agendas should encompassesuch considerations as design objectives, design alternates(including associated costs) and comments from specialgovernment or industry review committees.

All reviews should be performed by technicallyexperienced and skilled personnel from within the laboratory.Selection of these participants should be the responsibilityof the appropriate manager.

Recommendations, hopefully based on a consensus of thereview participants, should be documented in formal designreview reports. All participants should sign this documentdenoting understanding of the recommendations. Opinions ofparticipants disagreeing with the recommendations should beattached to the report.

Drawing Control

Engineering drawings are defined as representations ofmechanical or electrical equipment, test facilities, etc.that are produced on drawing media with formatted borders andtitle blocks. Sketches are defined as representations ofequipment, test facilities, etc. that are used to communicatetemporary, preliminary, or intermediate engineeringinformation pertaining to a project. Sketches are usually notto scale. Formal engineering drawings are recommended for:

gA-V V

ASQC Quality Assurance Guidelines DocumentDraft 1 :3/31/88

a. Documentation or long-term retention of designand/or construction of test articles necessary forreproduction of the experiment.

b. All applications such as construction of equipment,test facilities, etc. requiring any type of Codeconformance.

c. Design or construction of equipment, test facilitiesincluding buildings, utility systems (i.e. air,water, steam, gases, electrical, etc.), or otherreasonable permanent installations, or that requiredocumentation for insurance purposes.

d. Documentation of electrical circuits forconstruction or modifications to (reasonably)permanent equipment or test facilities.

All drawings should be checked by an individual otherthan the drafter and approved by the project leader prior torelease for use. Revisions to drawings should be reviewed andapproved in the same manner as the original.

Independent Technical Review

Independent technical reviews should be performed onprojects which involve the taking and/or analyzing of data.The reviews should be performed by a person or personsknowledgeable in the subject of the research, but who had noresponsibility in the project being reviewed. The projectleader may select the reviewer or reviewers and inform themof the purpose of the project and of the funding limitations.

Prior to the fabrication of a test article or the set-up of a complicated test structure, critical characteristicsspecified in the detailed work plan or by the project leadershould undergo an independent review. These characteristicsmay include calculations for scaling, test performance,dimensional stability and Code and design interfaces.

During the course of the project or during the finalreview at the completion of the project, the followingquestions concerning the project should be considered by thereviewer commensurate with their effect on the quality or thevalidity of the data:

a. Project performance with customer specifications andrequirements;

b. Results of analysis; that is, do the resultsobtained meet the objectives established in thedetailed test plan; and


c. Were specified quality standards, calculations,and/or computer programs checked for adequacy andaccuracy.

Independent reviews should be legibly documented.Reviewers should indicate the approach used for performingthe review and the results of the review (i.e. acceptabilityof the data reviewed). Questions concerning the data shouldbe reviewed with the originator and answered to thesatisfaction of the reviewer. Unresolved questions should beelevated to a higher manager. The reviewer should sign anddate the review documentation and present it to the projectleader for inclusion in the project records.

V. Change Control

Work Plans, calculations, independent reviews, designreviews, etc. should be baselined and assigned revisionnumbers for configuration control of changes. When changesare made to one of these documents, affected projectdocuments should be reviewed for required changes.

Drawings and drawing revisions and specificationsshould be controlled to ensure that incorrect or obsoletedrawings and specifications are not inadvertently used. Ifspecifications or drawings are revised, all associateddocumentation should also be revised.

VI. Calculations

Calculations which should be documented are thoserequired to substantiate the design of a test article andthose made during test data reduction and analysis.Calculations can be documented on a designated form,laboratory notebooks, computer output sheets or magneticmedia. Calculations should be legible and sufficientlydetailed such that a person qualified in the subject canunderstand the calculation without contacting the originator.

To make calculations suitable for filing, retrievingand reproduction, the first page of the calculation shouldcontain the project title, project identification number,page number, name of the person making the calculation, datethe calculation was made, and a cross reference to thehardware identification or the system or test phase to whichthe calculations apply. Project identification number andpage numbers should be included on each page of thecalculation.

Calculation detail should typically includeidentification of the objective or purpose of thecalculation, design inputs and their sources, methods usedand assumptions made, including identification of thoseassumptions requiring verification, parameters to be

ID Ars 7


investigated, any references used, units, results of theanalysis and conclusions, if any.

The project leader in conjunction with a superiorshould determine if calculations for the project require anindependent review. The decision for this review should bebased on the risk inherent in the use of the calculations.Calculations which may require review include calculationsbased on customer inputs, design engineer's calculations fortest article fabrication, calculations used for analysis oftest data, and calculations required to support the finalreport. After the reviewer has completed the review thecalculation should be dated and signed by the reviewer toindicate the review was performed. The reviewer shoulddocument the results of the review and note anydiscrepancies. Unresolved discrepancies should be elevated tohigher management.

VII. Software

Whether using an existing program, commerciallyacquired software or developing software internally, a riskassessment associated with the use of the software should bemade. The following are suggested definitions of risk.

a. High Risk - Errors in the software could result insignificant cost, product liability or impairedcustomer relations.

b. Medium Risk - Software which effects the R&D productbut not to the extent that a significant cost orproduct liability could be incurred.

c. Low Risk - Software which does not affect the R&Dproduct or results produced by the software will beverified in developing the product

The risk assessment should be used in developing aquality plan which specifies the documentation, reviews andtesting required to ensure that the software is an accurateimplementation of the required method and that the correctresults are being obtained. A risk assessment should beperformed as early in the project planning as possible toidentify funding requirements.

Additional controls should be performed as risk levelof using the software and required effort to produce thesoftware increases. Documentation should also increaseproportionately. A minimum level of document control includesthe assignment of software identification and revision levelor date to be included on all software output.

PA(C4F P


Documentation should be prepared during each phase of adevelopment program as appropriate. As a minimum,documentation should include the program identification, abrief description of significant limitations, capabilities,intended use, and a permanent record of the source codelisting.

The project leader should be held responsible for alldata and other information produced with computers. Prior tousing existing software, whether it has been purchased ordeveloped internally, the project leader should ensure thatthe software is reviewed in sufficient detail to assure thatthe methods used by the software and the results obtainedfrom the software are correct for the intended application.

VIII. Procurement

The laboratory should conduct its purchasing practicesto reflect experience with supplier performance. The QualityAssurance department should maintain experience records forselected vendors meeting prescribed quality requirements.When specifying equipment purchases, the project leadershould consider the experience records maintained by qualityassurance and the importance of the equipment to the qualityand validity of the data in specifying the extent of suppliercertification and qualification testing needed. Allresearchers should assist in gathering experience informationby reporting significant performance failures as well asoutstanding performances to Quality Assurance for inclusionin its supplier experience records.

The project leader should identify those procurementsaffecting the quality or validity of data, code materials,and instrumentation. Quality assurance requirements should bedocumented on the purchase requisition. Purchase requisitionsfor code materials, instrumentation and any other itemsaffecting the quality or validity of data should be reviewedand approved by the Quality Assurance department forappropriate quality assurance requirements. Purchaserequisitions for instrumentation should also be approved bythe laboratory's Instruments Department.

Purchase requisitions should contain a clear statementof what is required, date required, approximate costestimate, necessary department approvals, qualityassurance/code requirements, charge numbers and the name ofthe requisitioner.

Records for procurements for a project should bemaintained by the project leader in the project files.

PACTMF 9

ASOC Quality Assurance Guidelines DocumentDraft 1 3/31/68

IX. Inspections

Receiving Inspection

Once a purchased item is received, it should beimmediately inspected to verify that the material does, infact, meet the purchase order requirements. This inspectionmay be performed by the Quality Control personnel or by theproject leader or his designee. The inspection can bedocumented on a copy of the purchase order, in the projectlogbook, instrumentation records for an instrument or aformal receiving inspection report if one exists.

Inspection documentation should include the name andother identification of the item inspected, purchase ordernumber, item number, characteristics inspected,certifications received, deficiencies noted, whether the itemis accepted or rejected, the inspector's signature and thedate the inspection was performed. This information should beinputed to the supplier experience record system.

Immediate corrective action should be taken if thematerial is discrepant. Appropriate identification should beused to identify material which is being held for inspection,that which has been inspected and accepted, and that which isdiscrepant.

Material furnished by the customer should also receivea documented receiving inspection to assure that the materialis complete, undamaged, and meets all specificationspreviously agreed to by the Laboratory and the customer.

In-Process

Inspections are usually performed for those physicalcharacteristics identified by the project leader as criticalto the test results. The need for inspections can beidentified on receiving copies of Purchase Orders, RouteSheets, Test Procedures, Inspection Checklists. The projectleader should be held responsible for all inspectionsperformed on his project. Inspections can be performed byshop personnel, quality control personnel, or technicalsection personnel.

Critical dimensions should have a correspondingchecklist or route sheet prepared by the project leader orhis designee to direct the inspection of critical dimensions.If there are construction codes involved with test apparatusfabrication, the inspection of the attributes required byCode can be handled in a similar manner. The followingguidelines may be helpful.

PAPR I n


a. Inspection checklists should be used if theinspection is fairly simple and if all thecharacteristics can be inspected at one time.

b. Inspections of manufacturing operations that aremore complicated (many-inspections required atdifferent times) should be controlled by a routesheet.

c. All inspections performed by the technical sectionregarding the condition of the test article duringand after test should be documented in the projectlogbook.

d. Occasionally the customer may want to witnessinspection or testing operations. These witness orhold points can be included on Route Sheets or TestProcedures. The customer's representative should beasked to sign and date the above records verifyinghis witness of the test or to provide writtenauthority to proceed if the hold point is waived.

Inspections can be performed by any person familiarwith the operation. Inspections performed during test areusually done by the project leader or his designee. Shopinspections may be performed by the operator responsible formaking the part or, if required, by an independent inspector.The project leader should determine if an independentinspector is required or if the person responsible forperforming the operation can perform his own inspection. TheLaboratory may wish to maintain a list of inspectors who maybe called upon to perform special inspections. This listshould be maintained by the Quality Department

The person performing the inspection should beresponsible for the inspection documentation. Inspectiondocumentation should include the type of inspection performed(i.e., visual, dimensional, or type of NDE used), results ofthe inspection, deficiencies, whether the item was acceptedor rejected, reference to corrective action documentation,the inspector's signature and the date the inspection wasperformed.

Nonconformances detected during fabrication of the testarticle should be identified and documented. The articleitself should be identified as discrepant and segregateduntil the project leader has reviewed the nonconformance andany adverse effects it may have on the test results.Nonconformances discovered during test should be documentedin the project logbook. A nonconformance report may also beused. The project leader should be responsible fordetermining the necessary corrective action. This mayrequire customer approval.

PAGE 11

Specific Comments to ASQC Quality Assurance Guidelines DocumentDraft 0 Dated 4/27/87

Nonconformances

Nonconformance costs are an important part of overall qualitycosts. The quality assurance people in an R&D organizationhave a tremendous opportunity to increase the "value formoney" aspects of--a research contract. By helping keep down-the costs of research, we can help make our laboratories moreefficient and more competitive. I suggest therefore, thatnonconformances and their reporting should be brought fullywithin the orbit of any quality assurance program which is inplace in the laboratories. Care must be taken to avoidantagonizing the researcher by pointing out things that gowrong! (W.J. Langford)

Paragraphs 1 and 3: I'm not sure if a traditional NCR systemis appropriate or necessary for a research effort- this ismore appropriate in production. (R. Geoffrion)

Paragraph 2, last sentence, delete "Again," so that it reads:"The nonconformance may require customer approval." (D.Maxwell)

Paragraph 3, last sentence, add "for the project" so that itreads: "Allowing QA to be involved in the corrective actionprocess brings their particular skills into the problemsolving aspects of the quality program for the pro.iect." (D.Maxwell)

Test Control

A section on test control does not seem appropriate to aresearch effort to me. I would think that all of thisinformation could be adequately covered in a general qualityassurance program plan or in detailed procedures. (R.Geoffrion)

Paragraph 1, first sentence, add "project" so that it reads:"Tests should be specified in the detailed project work plandeveloped at the start of the project." (D. Maxwell)


X. Material Identification and Control

The project leader should be held accountable forassuring that all project materials affecting test resultsare identified and controlled. These controls can beaccomplished by-maintaining identification tags and markings-on the materials so that they cannot be mixed in or confusedwith unidentified stock or materials from another project.Standard quality control tags such as "Hold for Inspection"tags, "Material/Part Identification" tags, green "Accepted"tags and red "Discrepancy" tags are available and can be usedby the project leader to segregate and identify his material.

If materials will be needed by a support group such asa machine shop or weld shop, the project leader may want toset-up some type of central storage area so that when thesupport group needs to withdraw materials they are availableand appropriately protected.

The project leader may choose to delegate theresponsibility for material control and identification toanother section within the Laboratory. The project leadershould transmit all material control and identificationrequirements to the delegated section. The delegated section,in turn, has a responsibility to the project leader to ensurethat all material control and identification requirements(e.g. heat numbers, lot numbers, etc.) are adhered to for aslong as the material is in their possession.

The project leader should maintain a list of materialsor parts, and components that affect the validity of the testresults. The list should indicate if certified test reportsor certificates of conformance are required. Any additionaltests to be performed by the Laboratory to verify materialidentity should be specified. The project leader shouldmaintain any test reports or material certifications in theproject files. Copies of material test reports should be onfile in the Quality Department for those materials used on aproject or test facility which may become a "permanent" partof the Laboratory. State and local codes may influence ordetermine what substitutions if any are allowed. The p-ojectleader may have to obtain customer approval of any proposedmaterial substitutions prior to utilizing the substitutematerial. This requirement should be made clear duringinitial project planning.

PAGE 12


XI. Nonconformances

Nonconformances or conditions adverse to quality mayoccur in several areas. Nonconformances may be documented inproject log books and should be documented on an appropriatenonconformance report. If the nonconformance is on a specimenor fixture the item should be tagged with a discrepancy tagand segregated to prevent inadvertent use whenever possible.The project leader should review the nonconformance anddetermine what if any adverse effects the nonconformance willhave on test data. The project leader should determine whatcorrective action to take whether it be "use as is","repair", or "scrap". If a "use as is" or "repair" decisionis reached the project leader must determine whether thedecision will cause a deviation to any of the customerspecifications or commitments agreed to during the up-frontplanning or detailed test plan. The project leader may haveto obtain customer approval of proposed corrective actions.Once a nonconformance is satisfactorily resolved thediscrepancy tag may be removed.

Nonconformances to a quality system, such as adeviation to a test plan or procedure, should also bedocumented on some type of nonconformance report. At the veryleast the nonconformance should be documented in the projectlogbook or somewhere in the project records. Again, thenonconformance may require customer approval.

QA should receive copies of nonconformance reports.This allows QA to gather noncompliance data and perform longterm collective analysis of various research projects so thatundesirable trends or chronic violations can be detected andcorrected. Allowing QA to be involved in the correctiveaction process brings their particular skills into theproblem solving aspects of the quality program.

XII. Test Control

Tests should be specified in the detailed work plandeveloped at the start of the project. Suggested Work plancontents have b'en discussed in Section II. After the workplan has been agreed to the test program itself may begin.Control of specific tests may be achieved by individual testprocedures. Some of these may be standardized procedures suchas those published by the American Society of MechanicalEngineers (ASME) or the American Society of Testing andMaterials (ASTM). These can be identified by number andrevision in the work plan or somewhere in the project files,such as in the project logbook. Some procedures may have tobe written specifically for the test. Details of performingthe test may not be known at the start of the test and willbe developed as the testing proceeds. These procedures can bewritten very open ended so that they can be developed as thetest proceeds. The procedure would contain only those steps

PAGE 13

Specific Comments to ASQC Quality Assurance-Guidelines DocumentDraft 0 Dated 4/27/87

Test Control

Paragraph 4, first sentence, add "and signed and dated" sothat it reads: "Test results should be reviewed by theproject leader and sioned and dated." (D. Maxwell)

Instrumentation

It might be worth stating more strongly that calibratedinstrumentation should be independently verified withinexisting quality programs. I would not want to see severaldifferent calibration groups being formed, depending ondifferent research groupings. (W.J. Langford)


and acceptance criteria known at the start of the test. Astesting continues and details of the test become moresolidified the test procedure would be amended to reflect theactual testing perform. In the case where a phenomena isbeing characterized there is no acceptance criteria.Therefore, the test procedure shoul-d only state how the testdata is to be recorded. In some cases the detailed work planmay contain enough detail that a separate test procedure isnot required.

Detail contained in test procedures should include testprerequisites, such as special operator training orcertification requirements and test conditions thatshould be verified prior to beginning the test. If tests aresequenced or require specified inspections or independentverification of test the procedure should requireverification of the requirement prior to proceeding to thenext step of the test. The procedure should contain a listingof typical test equipment to be used. Reference to specificinstrumentation or equipment by serial number should beavoided since if substitute equipment is used the procedurewould have to be revised. Actual equipment set-ups,including serial numbers should be documented in the projectrecords.

Test data not processed directly through a computer ordocumented on computer output sheets should be recorded onformal data sheets or logbooks. All recorded data shouldreference the project title, date, time, data taker andfacility used. This data should reference, if not includedelsewhere, other documentation that describes how the testwas performed, a description of the test facility, and testarticle, including sketches or drawings utilized,instrumentation used, test procedure and revision number orstandard method used, actual parameters used and deviationsfrom the test procedure.

Test results should be reviewed by the project leader.Calculations and/or computer programs used in the analysisand interpretation of data should be independently reviewehThe review should determine not only the mathematicalcorrectness of the calculation but also whether theexperimental objectives were met and whether the effect orinteractions were interpreted correctly.

XIII. Instrumentation

Researchers have learned that an accurately calibratedinstrument traceable to NBS does not always give ameasurement of the desired accuracy for the project. It is,however, an important component. Environmental conditions oroperator error can have a devastating effect on the accuracyof an measurement. Total measurement uncertainty, randomerror as well as systematic error, determines whether or not

PAGE 14

Specific Comments to ASQC Quality Assurance-Guidelines DocumentDraft 0 Dated 4/27/87

Instrumentation

Paragraph 3, first sentence: This is good. (R. Geoffrion)

Paragraph 3, forth sentence, replace "can reject or acceptthe data" with "shall inform the operating group if theinstrumentation is out of calibration" so that it reads:"After evaluating calibration data the centralized controlgroup shall inform the operating group if the instrumentationis out of calibration." (R. Geoffrion)

Paragraph 4- Wordy and redundant. (R. Geoffrion)


a measurement is accurate. A documented calibration programclearly defining the responsibilities of the technical andsupport sections using calibrated equipment is the first stepa Laboratory can take in the effort of determining the totalmeasurement uncertainty associated with its variousinstrumentation. And, is the first step to proyvidingmeasurement quality assurance.

The program should include all measuring devices in theLaboratory. Some type of physical identification should begiven to each instrument identifying it immediately as to itscalibration status. That is, is it within its calibrationinterval or is it in need of calibration or perhaps itdoesn't require calibration as it is used for roughindications only. This identification should also identifythe history files for the instrument, which indicate itsaccuracy.

It is recommended that a centralized calibration groupbe established. This group would have the authority to"certify" the calibration status of an instrument and thetechnical expertise to oversee the calibration function. Thisgroup should have the power to perform the initial evaluationand assignment of an instrument status or category, approvecalibration procedures, calibration set-ups, calibrationpersonnel qualifications, and calibration data (whethergenerated by its own people, other technical sections of theLaboratory, or calibration suppliers). After evaluatingcalibration data the centralized control group can reject oraccept the data. If the data is rejected the appropriateproject personnel should be contacted so that the impact ofthe out-of-calibration condition can be evaluated for theproject data. Activities of this centralized calibrationcontrol group should be evaluated by Quality Assurance.

Even with all these controls in place it is still theproject leader's responsibility to assure that the properinstrumentation is used to meet the accuracy requirements ofthe project. The project leader should document the actualinstrumentation used on the project in the project records.If the facility being used to perform the test has its ownlist of instrumentation this list may be referenced in theproject records. However, the project leader should be awareof instrument substitutions made on the facility during thetest and update his records appropriately. All instrumentsused for taking data during the test should be calibrated.Instruments which have been designated as "indication onlyinstruments" should be used for indication purposes only andnot for acquiring data on the project. Instruments notcalibrated should not be used for taking data. The projectleader should also ensure that the instrumentation used iswithin its calibration interval throughout the time frame ofthe test.

PAGE 15


Reports

This section does not say much- would probably be better toemphasize peer review. (R. Geoffrion)

Paragraph 1, second sentence, add "or contract" so that itreads: "Latitude should be given the project leader inchoosing the particular style of report, whether it be aletter report or a more formal report, to suit thecircumstances and the customer requirements agreed to duringthe up-front planning or contract." (D.. Maxwell)

Project Records

Paragraph 1, third sentence, delete "drawing" and replacewith "specifications" and add "test" so that it reads"Suggested records which should be maintainedinclude... design reviews, sketches, drawing orspecifications list ..... and test article, test proceduresused and all test data." (D. Maxwell)

Paragraph 1. third sentence, add "and peer reviews" so thatit reads "Suggested records which should bemaintained... procedures used, all test data, and peerreview." (R. Geoffrion)

Paragraph 2, first sentence, add "authenticated and" so thatit reads: "All project records should be authenticated andidentified in enough detail that they can be easilyreproduced, filed and retrieved at a latter date." (D.Maxwell)

Paragraph 2, first sentence, delete "enough detail that theycan" and "reproduced, filed and" and add "and filed in orderto" so that it reads: "All project records should beidentified and filed in order to be easily retrieved at alater date." (R. Geoffrion)


XIV. Reports

Reports will vary with regard to type, size,complexity, timing and format. Latitude should be given theproject leader in choosing the particular style of report,whether it be a letter report or a more formal report, tosuit the circumstances and the customer requirements agreedto during the up-front planning. The project leader should beaware that a report published by the Laboratory is theprinciple evidence of the quality of its performance andability. Personnel of the Laboratory will be judged by thecontents of their reports, therefore their writing abilitymust match their research ability. The production ofeffective reports is a major part of their professionalperformance.

XV. Project Records

Complete documentation of the work performed by theLaboratory is quite important. Any information, memos,meeting minutes or special references that have a bearing onthe direction the project has taken, the data required, orthe interpretation of results should be included in theproject records. Suggested records which should be maintainedinclude work authorizations and any revisions to workauthorizations, all customer communications, purchaserequisitions, purchase orders, receiving inspectiondocumentation of purchased items and customer furnishedmaterials, inspection documentation of inspections performedin the shop on "critical dimensions," design reviews,sketches, drawing or drawing list, instrumentation list,calculations, calculation reviews, reference to computerprograms used, computer program reviews, computer printouts,characteristics of the test facility and test article,procedures used and all test data. Results such aspreliminary plots or data should be subject to the sametraceability requirements as calculations. The project leadershould be held responsible for maintaining all projectrecords.

All project records should be identified in enoughdetail that they can be easily reproduced, filed andretrieved at a later date. The Laboratory as a whole or theindividual technical sections should devise and maintain astandard system by which all project records within theindividual section of the Laboratory are retrievable.

XVI. Audits

Quality Assurance should perform general surveillanceover all projects being conducted to the general qualityguidelines of the Laboratory. A more detailed review or auditshould be performed periodically. The purpose of auditingprojects performed under standard laboratory Practice is to

PAGE 16


Audits

Paragraph 1, forth sentence (A statistical sample giving thedesired level of confidence...): SPC does not seem justifiedfor R&D projects, too many variables. (D. Maxwell)

Paragraph 2, second sentence (The audit report should bewritten to summarize the results of the entire lab.): Why isthis needed, delete in its entirety. (R. Geoffrion)

Paragraph 2, last sentence (Audit data should be summarizedstatistically... ): Again, not justified for R&D.(D. Maxwell)

Paragraph 2, last sentence (Audit data should be summarizedstatistically...): I don't understand the need for this.(R. Geoff rion)

Paragraph 3: A pareto analysis would be better if applicable.(D. Maxwell)

ASOC Quality Assurance Guidelines Document* Draft 0 4/27/87

characterize how projects are being performed with respect togeneral Laboratory guidelines. A statistical sample givingthe desired level of confidence in'the data should beselected from the total number of projects being conducted atthe time of the audit. Checklists for these audits should betaken from the guidelines and requirements for the variousadministrative systems utilized during the course of aproject. A checklist of standard questions applicable to anyproject audited should be designed. For. support systems whichmay or may not be used by the projects, such as thepurchasing function, a separate checklist should be developedand used when applicable to the project being audited. Theauditor should feel free to deviate from the checklist andpursue an independent line of questioning when answers givenappear to warrant this type of questioning.

The audit report should be written to the uppermanagement of the Laboratory and should include anintroduction, summary of significant results, conclusions,recommendations and detailed analysis of the audit results.The audit report should be written to summarize the resultsof the entire lab. However, supplemental reports can bewritten to summarize the results of each individual technicalsection if the section manager should request it. Audit datashould be summarized statistically so that the reader of theaudit report can make up his own mind as to the importance ofthe statistic to the laboratory or to his individualtechnical section and whether corrective action is required.

Statistical sampling can also be used when conducting ageneric systems audit. Major systems, such as purchasing anddrawing control, have an enormous data base to audit. Toaudit 100% would be time consuming and impractical. Because ageneric system usually involves only a handful of controllingprocedures, a checklist which covers all aspects of theprocedures is more easily designed and a very good degreestatistical inference can be made as to the degree of actualcompliance. Again a statistical report showing significanttrends using percentages and graphics, allowing the numbersto speak for themselves is best Variations and the approachto corrective action are dependent upon the seriousness ofthe noncompliance found. Serious noncompliances should not gouncorrected just because there is a lack of sufficientstatistical data to prove a trend exists. The auditor'sjudgement and experience should take over from the unbiasedobjectivity of the pure numbers presented.

Audits of suppliers are similar to audits of any otherLaboratory process. Audits of instrumentation and calibrationsuppliers should be of particular importance to theLaboratory. Like other audits, supplier audits should beperformed periodically, and like other audits, a checklist isdesigned to meet the requirements of the activity beingaudited. It may be beneficial to the Laboratory for its audit

PAGE 17

Specific Comments to ASQC Quality Assurance Guidelines-DocumentDraft 0 Dated 4/27/87

Audits

Paragraph 4, seventh sentence: How was two weeks selected asthe time period in which responses should be received by thelaboratory from a supplier for reported nonconformances?(T. Baer)

Training

I can only support the idea of training. My experience isthat senior research managers, having come through the normalprogression route from direct research involvement, do notlook kindly.upon quality assurance and do not wish to burdentheir staff with it. There are difficulties in having thequality department presenting training seminars if they havecome from the manufacturing background, as so many have. Thetwo sides tend to speak different languages. Referring againto the introduction to Roberts' book, I feel that a directapproach based on value for money, and the need forreliability of data, is more likely to get a response fromresearch groups, because it reminds them that the peoplepaying for the work want to know that their money is wellspent. (W.J. Langford)

A statement is needed about qualified by position..(T. Baer)

What about qualification of-technical/peer reviewers?(T. Baer)

Quality Cost

Does not apply to R&D. As stated earlier there may be-toomany false starts and to many paths to evaluate. The only.true evaluation that can be made is did the cost justify theend result or vice versa. (D. Maxwell)

I'm not sure if this would be useful in a researchenvironment, but it would be very unrealistic. (R. Geoffrion)


teams to include not only a QA auditor but also a technicalexpert in the area being audited. An audit team composed of atechnical expert and QA Auditor can perform a more throughaudit thus giving the audit more credibility. An audit reportshould be written and transmitted to the supplier within oneweek of performing the audit. Noncompliances should beaddressed by the supplier and a response returned to theLaboratory within two weeks. Responses received from thesupplier should be evaluated. A decision should be made as tothe acceptability of the response and whether the supplierwill be approved. Final verification of implementation of thecommitted action items may be held off until the nextregularly scheduled audit if the findings are relativelyminor, or they may be the subject of a special follow-upaction or reaudit if they are major.

XVII. Training

Introduction of a Quality Program to a ResearchLaboratory will require some employee indoctrination. One waythis can be accomplished is by the Quality Departmentpresenting mandatory training seminars. However, a better waywould be for upper managers to train lower managers in thequality practices to be followed and the lower mangers inturn then train the researchers. This approach again showsmanagement ownership of the Quality Program and gives moremeaning to the training received by the researchers. Manylaboratories are divided into various disciplines. Thisapproach allows the management of the various disciplines toemphasize the quality procedures it feels most important tothe type of research it performs thus allowing a tailor fitquality program.

XVIII. Quality Cost

There are many ready references detailing how tocollect quality costs in a manufacturing environment.However, no such references exist for the researchlaboratory. Collecting accurate, reliable quality costs canbe difficult. Unlike a manufacturing facility where the sameprocess is performed repeatedly, research projects tend to beunique and one of a kind. It is suggested that thetraditional quality costs, prevention, appraisal and failure,be evaluated on a Section basis. This can be done by havingthe Section perform an evaluation to determine whatpercentage of their project dollars as well as overheadbudget, is spent on these costs. Actual dollars spent by theSection can be obtained from accounting and used to produce aquality cost report for the Section on a monthly or aquarterly basis. The Section reports can then be combined toproduce a report of quality cost for the Laboratory. Overtime the percentages stated by the Sections will become morerefined and a very accurate report will be produced. Quality

PAGE 18


costs for such functions as accounting, purchasing andpersonnel can also be obtained and added to the Laboratory'squality cost report for a complete picture of quality relatedcosts.

PAGE 19

asqc quality assurance guidelines document.asqc quality assurance guidelines document draft 1...

Documents