oiml bulletin october 1999

8/12/2019 Oiml Bulletin October 1999

1/37

to enable specific regional issues to be resolved. Full reportsof all three meetings will of course be published in the

January 2000 issue of the Bulletin.As we go to press, a number of key events are taking

place at around the time of the CIML Meeting. The Softwarein Measuring Instruments Seminar will take place in Paris:the implications for legal metrology of changing technologyare multiple, and we must keep abreast of new develop-ments if we are to adequately adapt our activities to evolvingmeasuring instruments. The APLMF and WELMECCommittee Meetings and the 13th IAF Plenary Meeting arescheduled for September, followed in October by the 21st

Confrence Gnrale des Poids et Mesures, the ILAC 99General Assembly and Mtrologie 99. Three major events arealready scheduled for the year 2000: Metrology 2000 in Cuba(March 2000), theInternational Conference on Metrology inJerusalem in May and of course the 11th OIML InternationalConference of Legal Metrology, to be held in London inOctober 2000. Reports will be published on all these eventsin next years volume.

So the end of 1999 promises to be as active, if not moreso, than the first nine months of the year; the BIML is atcruising altitude and will continue to offer coordination,advice and assistance to its new and established Membersworldwide, to seek new ways to initiate and respond toprogress, develop the Organizations long term strategiesand react to our ever-active environment with a commit-ment to advance.

T

he publication of this, the fourth and last Bulletin for1999 (some would say of the century!) does not mean

that we can just slacken off and idly await the Year2000. On the contrary, legal metrology activity has neverbeen so intense, as is proved by the extensive calendar ofevents we published in the July Bulletin and from the con-tinuing stream of information that emanates from theBureau (several hundred kg of mail per month!).

The BIML is moving into the fast lane and playing anever-increasing role in coordinating events, bringing spe-cialists together from around the world and providing in-formation to Members and manufacturers, as well as adviceand technical support on a daily basis.

The culmination of another years activity is, of course,

the CIML Meeting. This October our host will be Tunisia,which is especially appropriate since in addition to theregular CIML Meeting an extended Development Councilmeeting will be held: Tunisia took over the Chair of theCouncil a year ago. Things are set to move fast and a num-ber of concrete proposals will be tabled.

In addition, a Round Table on Euro-Mediterranean Co-operation in Legal Metrology will take place in Tunis withthe aim of assisting countries in the Mediterranean regionto organize a cooperation in legal metrology which wouldbe specific to the region and which would extend anddeepen OIML work at regional level. In this era of lightning-fast information transfer via the Internet, computer tech-nology and scientific progress, it may be that certain funda-mental issues still have to be addressed before it is possible

E d i t o r i a l

Before the year is out...

Chris PulhamEditor


2/37

Abstract

Whenever standards are used for testing measuring instru-ments, they must be traceable to national or international

standards. When the instruments have been calibrated, themeasurement uncertainty is normally given on the certifi-

cate. If, however, the measuring instruments have beenverified, the measurement uncertainty is not always quoted.This may be due to the maximum permissible errors (mpe)both on initial verification and in service. Generally, the

requirements for calibration and testing are met by legalmetrology, however some measures may have to be takento ensure transparency and documentation.

Introduction

Due to the ever-increasing significance of qualitymanagement, a growing number of companies through-out the world have had their quality systems certified tothe ISO 9000 series of standards. Both certification

bodies as independent bodies for conformity assessmentof products and calibration and testing laboratoriesneed quality systems; in Europe these must meet therequirements found in the EN 45000 series of standards.These standards require measuring and test equipmentto be traceable to national or international standards. Asa rule, the quantities to be measured are traceable to SIunits in an unbroken chain of comparison measure-ments carried out by competent bodies.

The concept of traceability not only requires an un-broken chain of comparison measurements, but also astatement and documentation of the measurement un-

certainties. The statement of measurement uncertaintieswith reference to the standards used is an essential part

of every calibration. Competent bodies will thereforenormally accept calibrated instruments as test equipment

within a quality management framework. The use oflegally verified instruments for this purpose sometimespresents problems, since although the mpes for theinstruments are known, no measurement uncertaintiesare explicitly given. These problems are due to the differ-ent tasks and objectives of verification and calibration aswell as to a lack of understanding between the two sys-tems.

The authors hope to clearly identify the differences,but, at the same time, must point out that the same prin-ciples apply to the identical metrological aspects of bothactivities. No matter whether the metrological activities

are performed in the regulatory or the non-regulatoryareas, they must not deviate by more than is justified bythe given objectives.

1 Objectives of verification

1.1 Historical development

The units of mass, volume and length are importantsince in commercial transactions their measurementdetermines the price. In the past, various interests as wellas regional and historical differences led to differingunits and systems. As cross-border trade increased insignificance, pressure grew for harmonization; thisresulted in the introduction of the SI system which notonly became the legal basis for official dealings andcommercial transactions, but also gained in importancein the non-regulatory field of industrial metrology. Anefficient metrological infrastructure is the basis of allmodern industrial societies and from this point of view,

legal metrology was the pioneer of uniform measure-ment.

5O I M L B U L L E T I N V O L U M E X L N U M B E R 4 O C T O B E R 1 9 9 9

t e c hn ique

UNCERTAINTY

Uncertainty of measurement and error limitsin legal metrology

WILFRIED SCHULZ, Physikalisch-Technische Bundesanstalt (PTB), Braunschweig, GermanyKLAUS-DIETERSOMMER, Landesamt fr Mess- und Eichwesen, Thringen, Germany


3/37

1.2 Legal requirements

The main objective of legal metrology is to protectcitizens against the consequences of false measurementsin official dealings and commercial transactions as wellas in the labor, health and environment areas. As theinterests of the parties concerned by measurements inthese areas differ, the characteristics of the instrumentsused cannot be satisfactorily controlled by market forces.Legislation therefore lays down requirements not onlyfor measuring instruments, but also for measuring andtesting methods.

In Germany, these regulations are controlled byEuropean Directives and by the Verification law. Forindividual categories of instruments, the regulationscover:

the mpes both on verification and in service;

nominal conditions of use;

susceptibility to external interference;

electromagnetic compatibility (EMC);

labeling;

durability;

tamper resistance; and

reverification periods.

Everyone concerned with measurements should haveinstruments which give correct results within specified

mpes under the local environmental conditions. As theparties concerned by the measurements are not normallymetrological experts, and do not have the capability ofchecking the results they are given, the State thereforetakes responsibility for the validity of measurementswithin the framework of legal metrology.

1.3 Measures and procedures

In order to reach the objectives of legal metrology, bothpreventive and repressive measures are needed. Prevent-ive measures are taken before the instruments areplaced on the market or put into use and include patternapproval and verification. Market surveillance is anexample of a repressive measure, and involves inspec-tion of the instrument at the suppliers, owners or userspremises. Here misuse of the instruments will bedetected, and the offence may be punished by a fine.

The manufacturer has to file an application forpattern approval with the competent body. In Germany,

this is the PTB; other European bodies as well as PTBare also responsible for European pattern approvals.

At least one sample of the instrument is examined toensure compliance with the legal requirements. Appro-val tests and calibrations are carried out, and the resultsshow whether the given requirements are met. It isparticularly important to determine whether the mpes

at rated or foreseeable in situ operating conditions arelikely to be met. The sample instrument is also subjectedto quality tests which should guarantee its reliability inuse.

For reasons of efficiency, verification usually onlyrequires a single measurement (observation) to be car-ried out. It is therefore important that the spread ordispersion of measured values is determined during thetype approval tests. This determination of so-called a-

priori characteristic values forms the justification for theevaluation of the uncertainty of measurement on thesubsequent verifications.

Upon successful type approval testing, a manufact-urer has in principle proven his technical competence tomanufacture an instrument that meets the legal require-ments.

As pattern approval is a test of the pattern, it is fol-lowed by verification testing on each instrument. Thisensures that every single instrument conforms with thepattern. After the initial verification validity period hasexpired, reverification will be done by a verificationbody. When a single owner (particularly an energy orwater utility) has a large number of instruments, reveri-fications may be carried out on samples. The reverifica-

tion requirements, in particular the mpes, are the sameas those at initial verification, which means that themeasurement uncertainties have to be handled in thesame way.

European harmonization allows the manufacturer tocarry out conformity assessment on new instruments asan alternative to verification by a verification body. Thisleads to the need to harmonize the measuring andtesting methods, including determination of the meas-urement uncertainties and accounting for them inconformity assessments. Some relevant terms and defi-nitions are given below.

2 Metrological terms and definitions

2.1 Uncertainty of measurement

According to the VIM (3.9) [1] measurement uncertaintyis a parameter, associated with the results of a meas-

urement, that characterizes the dispersion of the valuesthat could reasonably be attributed to the measurand.

6 O I M L B U L L E T I N V O L U M E X L N U M B E R 4 O C T O B E R 1 9 9 9

t e c hn ique


4/37

Measurement uncertainty is usually made up of manycomponents, some of which may be determined fromthe statistical distribution of the results of series ofmeasurements and which can be characterized byexperimental standard deviations. The other components,

which can also be characterized by standard deviations,are evaluated from assumed probability distributionsbased on experience or other information.

Contributions to the measurement uncertainty are:

the standards used;

the measuring and test equipment used;

the measuring methods;

the environmental conditions;

susceptibility to interference;

the state of the object to be measured or calibrated;and

the person performing the measurement or calibra-tion.

The Guide to the Expression of Uncertainty in Meas-urement (GUM) [2] and document EA-4/02 [3] givedetailed information on the determination of measure-ment uncertainties and a summary of the contributions(cf. Section 3).

2.2 Calibration

The VIM (6.11) [1] defines a calibration as a set ofoperations that establish, under specified conditions,the relationship between values of quantities indicatedby a measuring instrument or measuring system, orvalues represented by a material measure or a referencematerial, and the corresponding values realized bystandards. This means that the calibration shows howthe measured value or the nominal value indicated by aninstrument relates to the true or conventional true valuesof the measurand. It is assumed that the conventionaltrue value is realized by a reference standard traceable

to national or international standards.Not only the measurement uncertainty but also the

environmental conditions during the calibration aresignificant. The calibration is often carried out in a placewith well-known environmental conditions, which leadsto low measurement uncertainties. When the calibratedinstrument is used in a different environment themeasurement uncertainty determined by the calibrationlaboratory will often be exceeded if the instrument issusceptible to its environment. There can also be aproblem if instrument performance deteriorates afterprolonged use. The user of the calibrated instrument

must therefore consider any environmental or secularstability problems.

2.3 Testing

According to ISO 8402 [4] testing implies the statementthat conformity for each of the characteristics was

achieved. EN 45001 [5], however, states that a test is atechnical process in the sense of an examination todetermine the characteristic values of a product, pro-cedure or service.

The quantitative requirements stipulated for instru-ments refer to the measurement errors, the values ofwhich must not exceed the mpes. The measurementerror itself is in practice recognized to be the result of ameasurement minus a conventional true value [1]. Cali-bration of the instrument over the given measuringrange at given environmental conditions is the pre-requisite for an assessment of conformity with regard toerror limit requirements being met.

Whereas a measurement result implies an uncert-ainty of measurement, a complete testing result impliesan uncertainty of testing. This leads to an uncertainty of

decision with regard to conformity assessment. A dis-tinction must be drawn between quantitative andqualitative tests, and as a rule a measurement uncert-ainty can be assigned in a quantitative test. An assess-ment of any qualitative characteristics of the objectunder test, e.g. of a measuring instrument, also requiresuncertainty statements. This means that the measure-ment uncertainty determined during the calibration isonly a contribution to the total uncertainty.

2.4 Verification

The verification regulations lay down the tests andmarking of an instrument. The initial elements of verifi-cation are:

a qualitative test, which is effectively an inspection;and

a quantitative test, which is almost the same as a cali-bration.

These two elements of verification are tests in thesense of the EN 45000 series of standards. Once theyhave been performed, the matter of certification can beconsidered.

Here the test results are evaluated to ensure that thelegal requirements are being met. During this evaluationit is particularly important to establish that the calibra-tion results demonstrate that the mpe requirements aresatisfied.

Assuming that the evaluation leads to the instrumentbeing accepted, a verification mark or label must be

fixed to it, and, where relevant, tamper evident seals. Averification or evaluation certificate may be issued.


t e c hn ique


5/37

3 Calculation of the measurementuncertainty

Basically, the determination of the measurement un-

certainty refers to the calibration inherent in conformityverification (cf. 2.4). Therefore the procedures given inthe GUM [2] and in EA-4/02 [3] are applicable:

(a) Defining the objective

As a rule, the basic objective in legal metrology is thedetermination of the expanded measurement uncer-tainty (k = 2), for the difference between the measur-ing instrument under test and the standard.

(b) Drawing up a model function

The model function expresses in mathematical termsthe dependence of the measurand (output quantity)Yon the input quantitiesXi according to the follow-ing equation:

Y=f(X1,X2, ...,XN) (1)

In most cases it will be a group of analytical ex-pressions which include corrections and correctionfactors for systematic effects [3].

Where a direct comparison is being madebetween the indications shown by the instrument

under test and the standard, the basic equation maybe simple:

Y=X1 X2 (2)

(c) Type A evaluation of uncertainty contributions

This is done by statistical analysis of a series of ob-servations, normally by calculation of the arithmeticmean value and its experimental standard deviation.

The estimatesxi of the input quantitiesXi have to bedetermined, and the standard uncertainties ui are thestandard deviations mentioned above [2], [3].

(d) Type B evaluation of standard uncertaintyof input quantities

Method A normally assumes that the measurementvalues are normally distributed and that the stand-ard uncertainty is indicated in terms of the empiricalstandard deviation of the mean. When using methodB however, the probability distribution to be appliedmust be considered in more detail.

If the distribution is unknown, and no data fromwhich an uncertainty could be deduced are available,values have to be based on scientific experience. Ifmaximum or minimum tolerances can be assumed(even by approximation), the standard uncertaintyhas to be calculated on the basis of a rectangulardistribution [2], [3]. This is also applicable to meas-urements with working standards in legal metrology.


t e c hn ique

Example 1: Testing of a filling station fueldispenser by means of a standardmeasuring container

Measurand Y: Deviation of the indicated fuel

volume from that actuallydelivered

Input quantitiesXK

Fuel dispenser indication,related to the instrument measuring system temperature,to be verified: liquid temperature, etc.

Input quantitiesXl Level indication, deviation ofrelated to the container from horizontal, fuelstandard used: environment temperatures,

foam layer thickness, etc.

Other input quantitiesXm: Loss of fuel during the measuringprocess due to evaporation oradhesion, incorrect operation, etc.

Example 2: Measurement with a 50 L standardmeasurement container

The uncertainty contribution for a 50 L standard measure-ment container where only the nominal volume and mpes aregiven has to be determined by applying the rectangular prob-ability distribution.

mpe (VN/VN)max: 0.1 %

Resulting standarduncertainty u(VN): VN/

3 29 cm3

(e) Calculation of the sensitivity coefficients

The sensitive coefficient can be found from the modelfunction by:

partial differentiation of the model function by theindividual input quantities at all relevant values oftheir estimates:

ci (Y/Xi) | xi

(3)

and/or:

(computerized) numerical variation of the inputquantities according to their quantification andtaking into account the change in the output.

Experimental determination of the relationshipbetween output and input quantities is also possible.


6/37

(f) Compilation of an uncertainty budget

Sources of uncertainty must be listed in tabular form,together with their respective input estimates xi,standard uncertainties uk(xi), and contributions

ui(y) to the uncertainty associated with the outputestimatey.

(g) Calculation of the output estimate and of theassociated standard uncertainty

The standard uncertainty of the output estimate isdetermined by adding the contributions ui(y) inquadrature. This gives the square of the standarduncertainty u(y) of the measurand. It is essential toconsider the possibility that some of the contribu-tions may be correlated, and so not truly inde-pendent [2], [3].

(h) Statement of the complete measurement result

The complete measurement result includes theoutput estimate y and the expanded uncertainty ofmeasurement U(y). This identifies the range withinwhich the output will be found with a probability ofapproximately P = 95 %.

When the measurement uncertainty of a verificationis to be determined, it should be remembered thatnormally only individual measurements are made. Thismeans that evaluation method A may only be applied if

relevanta-priori data, e.g. for the standard deviation of acertain type of instrument, exist. Logically, the standarduncertainty of the individual measurement, i.e. thestandard deviation of a series of observations, will thenbe included in the output rather than the standarduncertainty of the mean.

As a rule, a-priori data are determined in typeapproval tests. Moreover, for many instrument cat-egories, e.g. fuel dispensers, comprehensive experienceor statistical values are available.

Formal application of the above scheme is notsufficient for the determination of the measurement

uncertainties. The chief prerequisite for a realistic resultis a complete model which is close to reality. Critical andhonest evaluation of the estimated values of the inputquantities can only be based on sound experience.

4 The significance of measurementuncertainty in practice

4.1 Calibration

A calibration gives a systematic measurement error to-gether with a statement of the measurement uncertainty.

This not only relates to the correct value derived fromthe reference standard, but also takes account of theenvironment during calibration. The temperature is ofparticular importance here but humidity, air pressureand electromagnetic fields may also make a consider-

able contribution to the measurement uncertainty.As a rule, instruments to be used as reference stand-

ards will be calibrated under controlled environmentalconditions. If these newly calibrated instruments arethen used in the same environmental conditions, it canbe assumed that they will have the same measurementuncertainty. When an instrument is being calibratedagainst such standards, its uncertainty us enters into thetotal uncertainty of measurement umeas as an (uncor-related) contribution:

u2meas = u2s + u

2i

(4)

where ui are contributions to the measurement un-certainty related to the calibration procedure and to thenature of the object under test.

If, on the other hand, a calibrated standard is used indifferent environmental conditions and after prolongeduse, higher uncertainties must normally be assigned.

Calibration therefore makes a statement about aninstruments behavior only at the moment it is carriedout. The user must assess on the basis of his technicalknowledge whether the calibrated instrument is suitableor not. If a calibrated instrument is to be used to evalu-ate the uncertainties of measurements and tests underother environmental conditions, particularly strenuousrequirements will have to be met. Calibration certifi-cates do not normally contain any statements about thelong-term behavior of the object.

4.2 Testing

While the term measurement uncertainty is clearlydefined and used [1], the term uncertainty of testing,which means uncertainty as to the properties of theobject under test, is not yet harmonized. Proposals for

harmonization have been put forward by the EuropeanCooperation for Accreditation of Laboratories (EA) [7].

No matter whether an application is covered by regu-lations or not, a quantitative test on a measuring instru-ment should state whether the values determined liewithin the mpe. For this reason, a calibration (includinga measurement uncertainty statement) is required.

Figure 1 shows possible interrelations between theintrinsic error of a measuring instrument [1], the mpeand the uncertainty of measurement.

In cases a, b and c, the instrument is within the mpe.In case d, non-compliance with the requirements is

proven and in cases e and f no unequivocal statement ofconformity can be made. Here, the parties concerned


t e c hn ique


7/37

must agree on acceptance or rejection of the instrument.This kind of assessment is, inter alia, required for thetesting through measurement of manufactured itemsand instruments by ISO 14253 [8].

4.3 Verification

4.3.1 Maximum permissible errors on verificationand in service

Verification is a special method of testing covered byregulations laid down by legislation. In OIML Recom-mendations and in many economies with developedlegal metrology systems, two kinds of error limits aredefined:

the mpe on verification; and the mpe in service, which in most cases is twice the

mpe on verification.

The mpe on verification equals an mpe on testingwhich only applies at the time of the verification. Thempe in service is the one that is legally relevant for theuser of the instrument.

Figure 2 explains this approach to the effect thatduring the time of use of a measuring instrument withinthe period of the validity of the verification, the indi-cated measured value will drift to some extent and theuncertainty of measurement will in most cases clearly

rise due to the realistic operation conditions and exter-nal interference. In particular the following influencesmust be taken into consideration:

measurement uncertainty from the metrological testduring verification;

normal operating conditions;

external interference during normal operation; and

long-term behavior, drifting, aging and durability.

The mpe on verification may be exceeded here, how-ever requirements regarding the mpe in service must ingeneral be met. As a result, verification implies a highprobability that under normal conditions of use themeasuring instrument will furnish measurement resultswithin the given mpes in service during the entire valid-ity period of the verification.

In practice, measuring instruments are considered tobe in compliance with the legal regulations:


t e c hn ique

Fig. 1 Influence of the (expanded) uncertainty of measurement of various measurement results j on conformity assessment in testing

error limit (mpe)

error limit (mpe)

a b c d e f

y

y2

yN

y1+ U

U

Note: N = the value realized or indicated by a standard


8/37

if the indicated value is smaller than or equal to thempe on verification when the test is performed by averification body under unified test conditions; and

if the uncertainty of measurement at the 95 % prob-ability level is small compared with the prescribed

error limit.

In legal metrology at present, the uncertainty ofmeasurement is usually considered to be small enough ifthe so-called one-third uncertainty budget is notexceeded:

U(k = 2) 1/3 MPEV (5)

where MPEVis the mpe on verification.

The criteria for the assessment of compliance areillustrated in Fig. 3. Compliance with the requirementsof the verification regulations is given in casesa, b, c and

d. Casese andfwill result in rejection, although all thevalues including the uncertainty of measurement liewithin the tolerances fixed by the mpes in service.

As regards the mpe on verification, the describedapproach above is called the shared risk concept: pro-vided that inequation (5) applies, the (systematic) error

of measurement determined is not extended by the un-certainty of measurement when one checks whether itexceeds the error limits on verification. In this way thereis an approximately shared risk that a test result lying onthe extreme edge of the tolerance band may be inside or

outside the permissible limit.Therefore, the mpe on verification of a newly verified

measuring instrument will in the worst case be exceededby 33 %. However, as the legally prescribed mpes in ser-vice apply for the user of the measuring instrument,there is no shared risk in the sense that no measuredvalue - even if the measurement uncertainty is taken intoaccount - will be outside this tolerance band.

So far, the mpes on verification can be seen as a sup-porting guide for the conformity assessment of mpes inservice being met in order to take the above-mentionedinfluences into consideration.

To a far-reaching extent the influence of the oper-ating conditions at the place of use, the effect of inter-ferences and the long-term behavior must be ascer-tained during pattern testing by the type approval body;here, experience gained with the same category of meas-uring instruments will be included in the assessment.


t e c hn ique

Fig. 2 Consideration of long-term drift and external influences by definition of two kinds of error limits: mpe on verification and in service

increaseduncertainty

increaseduncertainty

drift

drift

y(t3)

+ U

U

yN

meas. value y

y(t1)

MPEV

MPES

MPEV: Maximum permissible error on verification

MPES: Maximum permissible error in service

MPES

MPEV

Period of validity of the verification

Measurement of the samequantity y at instants t i(moments of verification)

time tt3t2t1


9/37

4.3.2 Standards and testing methods

When carrying out the tests required in legal metrology,the uncertainty of measurement predominantly depends

on the reference standards. To ensure that the test istraceable to national standards, the reference standardsof the conformity verification bodies are calibrated bythe relevant national institute of metrology (in Germany,the PTB). The systematic errors and the measurementuncertainties associated with these reference standardsare given on test or calibration certificates.

The verification bodies derive the traceability of theirworking standards from these reference standards. Inmost countries with a highly developed legal metrologysystem, the working standards can deviate from theconventional true value indicated or realized by the

reference standard used by no more than one third ofthe mpe on verification. Here the expanded measure-ment uncertainty (k = 2) of the measured quantityshould be taken into consideration. As the comparisonof the standards is performed under laboratory condi-tions, the measurement uncertainty may be minimized.As a rule, systematic components will predominate the

error budget. If the working standard meets the one-third uncertainty requirement, its systematic error andthe measurement uncertainty will not be consideredduring verification in order to make the metrologicaltests cost-efficient. When the one-third uncertainty

requirement (cf. 4.3.1) does not apply, systematic errorshave to be individually accounted for.

ISO/IEC DIS 17025 [6], which is a more recent draftstandard, also recommends the 1:3 ratio between themeasurement error or measurement uncertainty, andthe prescribed tolerance. This is a practice which hasbeen applied in legal metrology for many years. The test-ing periods for standards are also laid down by law.

In verification, metrological testing methods areapplied which were optimized and harmonized by theresponsible bodies based on the experience of verifyingmillions of measuring instruments. In Germany, there

are about 25 million instrument verifications per year.As long as the prescribed conditions at the place oftesting are met, additional external influences and sub-jective factors will not cause measurement uncertaintiesto exceed the error limits of the working standards. As arule these uncertainty contributions are thereforeneglected. However, this practice is only acceptable if:


t e c hn ique

Fig. 3 Influence of the uncertainty of measurement (k = 2) of various measurement results i on conformity assessment in verification

MPEV(mpe on verification)

MPES(mpe in service)

MPES

MPEV

a b c d e fy

1/3 MPEV

y2

yN

y1+ U

U

Note: N = the value realized or indicated by a standard


10/37

the uncertainty attached to the working standard isclearly within the one-third uncertainty budget;

the additional contribution of uncertainties does notcontain serious systematic error components; and

all contributions other than the measurement un-

certainty of the standard used total less than 20 % ofthe mpe on verification.

what is the proportion of newly verified measuringinstruments to be expected which actually exceeds thempe on verification?

what is the proportion of newly verified measuringinstruments to be expected which actually exceeds1.33 times the mpe on verification?


t e c hn ique

Fig. 4 Consideration of uncertainties of measurement in conformity decisions

Example 3: Verification of a fuel dispenser

mpe on verification (MPEV) 0.500 %

One-third uncertainty budget (0.33 MPEV) 0.166 %

Relative expanded measurement uncertainty(k = 2) of the standard measuring container(58 cm3; cf. Example 2) 0.116 %

Contribution of measurement uncertainties(k = 2) arising from procedure andexternal influences (20 % of 0.500 %) 0.100 %

Total measurement uncertainty(added in quadrature) 0.153 %

The above strategy may also be based on the normallyrather high error limits in legal metrology, the one-thirduncertainty budget being on the safe side as far as themeasurement uncertainties are concerned.

However, the effect of ignoring the measurement un-

certainties arising from the test procedure and fromexternal influences must be considered critically. If thespecified ambient conditions are exceeded in the test,the respective contribution of these uncertainties canincrease to more than 20 % of the mpe on verification.

If all the requirements are met, the mpe on verifi-cation (MPEV) will in the worst case be exceeded by33 % (cased in Fig. 3).

For reasons of consumer protection and efficientmanufacturing of measuring instruments, it is import-ant for competent authorities and manufacturers tohave a quantitative estimate of the consequences of the

measurement uncertainty in conformity verification onthe quality of the instruments to be placed on themarket. The following two questions are of particularimportance:

U(k = 2) = 1/3 MPEV

pu(u)

2

1

P = p() d

=x

pA(A)

MPEVMPEV

Assumptions:

95 % within the MPEV Normal distribution

2

2.61 +2.61

Note: Figure not to scale

01.96 +1.96

+2

95 %Uncertaintyof the testprocedure

Distribution of the errors ofmeasuring instruments to be verified

The measurement uncertainty contribution from the procedure andthe external influences is unusually large, amounting to 20 % of thempe on verification. Despite this, due to addition in quadrature, thetotal uncertainty in the above example is not much greater than thatof the standard, and the one-third budget will be met.

Example 3 illustrates these relations:


11/37

The significance of statement (3) made in Example 4has to be emphasized.

To reach the mpe in service an extra (reserve) toler-ance is therefore available. It equals 66 % of the mpe onverification at minimum. Thus the effects of temporaldrift and additional external influences on the

measurement result may safely be compensated (cf.4.3.1). This means even after prolonged use and withvarying external influences, the risk for the user that thempe in service is exceeded is practically zero.

The fact that, even in the worst case, the value of thempe on verification is not exceeded by a factor of morethan 1.33 facilitates conversion into other systems. Anexample is the situation where it is required that thesum of the measurement error and of the expandedmeasurement uncertainty must lie within given assess-ment limits.

into consideration so that there is no shared risk. If theenvironmental conditions during use are the same asthose prevailing during verification, the values can beexpected to be even better than the mpes on verification(probability 95 %). But in the worst case, for a small

proportion of the instruments, they may be 1.33 timesthe mpe on verification. This value is significant for theuser of an instrument which is outside the regulatorysphere, but who still has to prove traceability to nationalstandards.

Verification comprises calibration and certification.These should comply with the international technicalregulations, e.g. with ISO/IEC DIS 17025 [6]. For calcu-lation or estimation of measurement uncertainties, theGUM [2] and EA-4/02 [3] are equally applicable.However, calibration is only part of the verification, andadaptations are required to take this into account. Some

generalizations are also required for consumer protec-tion reasons.

5.2 Further development

It should be remembered that the GUM guidelinesconcerning measurement uncertainties in legal metrol-ogy are incomplete. New instructions for calibration andtesting including measurement uncertainty calculationshave to be established and a future concept shouldinclude at least the following measures:

integration in the uncertainty budget (one-thirduncertainty budget, presently being confined to theworking standard) due to the testing procedure andthe instrument under test during verification;

definition of the mpe in service as the limit whichmust not be exceeded in the metrological evaluationfor consumer protection reasons. The deviation ex-tended by the measurement uncertainty (k = 2) shouldbe less than the mpe.

if someone is going to use a verified measuring instru-ment as a standard in accordance with the ISO 9000series, they should be informed of the relationshipsbetween:

measurement uncertainty;

the mpe on verification; and

the mpe in service.

In the past, government bodies were not required toprove such transparency. Due to the increasing transferof regulatory tasks to private institutions and manu-facturers within the framework of international har-monization, government authorities are also required tomeet uniform regulations.


t e c hn ique

Example 4: Consequences of the measurementuncertainty in conformity assessment

on a batch of instruments

The calculation will be based on the typical case of a unit-tested batch of instruments where the intrinsic measurementerrors due to manufacturing variation more or less follow anormal distribution. This implies that 5 % of the instrumentswill in fact lie outside the error limits. In addition, it isassumed that the spread of values resulting from the un-certainties in the metrological test are normally distributed,and that the measurement uncertainty amounts to themaximum permissible value of U(k = 2) = 0.33 MPEV.

Figure 4 illustrates these conditions. If a measurementerror is based on the (lower) error limit, combination of bothdistributions will result in the following:

(a) The expected proportion of faulty instruments which areassessed as indicating correctly will be less than 2 %;

(b) The probability that the mpe on verification will beexceeded by a factor of more than 1.33 is practically zero.(cf. Fig. 4).

5 Conclusions

5.1 Present situation

Due to the legal regulations to which verification issubjected, the user of a verified instrument may assumethat during the validity period the instrument willindicate values within the mpes in service. This is thecase even if the measurement uncertainties are taken

The answers to these questions are illustrated byExample 4.


12/37

References

[1] International Vocabulary of Basic and General Termsin Metrology, BIPM, IEC, IFCC, ISO, IUPAC, IUPAP,OIML, 1993

[2] Guide to the Expression of Uncertainty in Measure-ment, BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML,Corrected and reprinted, 1995

[3] EA-4/02, Expression of the Uncertainty of Measure-ment in Calibration, Ed. 1, European Cooperationfor Accreditation of Laboratories, April 1997,(previously EAL-R2)

[4] ISO 8402, Quality - Terms and Definitions

[5] EN 45001, General criteria for the operation of testinglaboratories, 1990

[6] ISO/IEC DIS 17025:1998 General requirements forthe technical competence of testing and calibrationlaboratories

[7] EA-3/02, The Expression of Uncertainty in Quantitat-ive Testing, Ed. 1, European Cooperation forAccreditation of Laboratories, August 1996,(previously EAL-G 23)

[8] ISO 14253-1: 1998, Geometrical Product Specifica-tion (GPS) - Inspection by measurement of work-

pieces and measuring equipment, Part 1: Decisionrules for proving conformance or nonconformancewith specification


t e c hn ique

Dr. Wilfried SchulzDirector and Professor

Physikalisch-Technische Bundesanstalt (PTB)Bundesallee 100

D-38116 BraunschweigGermany

Tel.: +49 531 592 8300Fax: +49 531 592 8015

E-mail: [email protected]

Dr. Klaus-Dieter SommerDirector

Landesamt fr Me- und EichwesenThringen

Unterprlitzer Str. 2D-98693 Ilmenau, Germany

Tel.: +49 3677 850 100Fax: +49 3677 850 400

E-mail: [email protected]

Note from the BIML: A Secondary Guide to the expression of measurement uncertainty in legal metrology is being developed as anapplication of the GUM. A working document prepared by Grard Lagauterie, Sous-Direction de la Mtrologie,France, was distributed at the TC 3 meeting held in June 1999, a report of which is published in this Bulletin.Several attending persons, including Dr. Sommer, participated in discussions. The continuation of this workproject has been allocated to the recently established TC 3/SC 5 Conformity Assessment, under the jointresponsibility of the USA and the BIML.


13/37

Abstract

In some cases, mathematical statistical checking is theonly way to adequately test gambling devices and the twomost frequently used methods (i.e. the 3method and the

2 method) are discussed in this paper. Because the 2

method cannot be used in its classical form for testinglottery-type games, it was necessary to develop it so that it

could also be applicable for this purpose. For both rouletteand lottery-type games, the second and the third membersin the series of the distribution function of the variable2

are also determined in order to form an improved testmethod.

1 Introduction

In Hungary, as in some other European countries, thetechnical control of gambling devices is legally the taskof the national metrological institute; mathematicalstatistical checking of the random (and therefore fair)operation of such devices also falls under the institutesresponsibility. In some cases huge prizes are drawn bysimple devices made of wood, whose structure cannot

be tested by classical metrological means; mathematicalchecking of these devices is therefore the only possibleofficial control method. Considering the fact that in thisfield written standards are not available, it is vital thatthe authoritys decision as to whether a device is ac-cepted or rejected be well-founded and indisputable.

This paper shows the mathematical bases of the twomost important statistical methods mentioned above,which are most frequently used for type approval and

verification. These mathematical methods can be usednot only for gambling devices but also for other pur-poses, for example to check a hypothesis about a prob-

ability distribution or to check a computers randomnumber generator.

In some cases during the authors research it wasnecessary to further expand on existing methods, or

even to elaborate new ones; these can be found inparagraphs 4, 5 and 6 below.

2 The 3 test

The method is given for the case of roulette wheels butcan, of course, be used in other fields too.

Roulette wheels used in Hungary have numbers 1 to36 plus a single zero, so the number of possible spin

outcomes equals 37, usually designated by . One of thesteps in testing the randomness is to check thehypothesis that the probability of spinning any numberout of the 37 is the same:

pj =1 (j = 0, 1, ..., 36).

When a roulette wheel is submitted for testing theclient must provide a list of the lastNspins, in sequen-tial order. Such lists used to be written by hand, thoughnowadays they are made electronically using a data

collector connected to the corresponding roulette tableelectronic display device. The ith number on the list, i.e.one of the numbers 0 to 36, is designated by

xi (i = 1, 2, ...,N) andN is prescribed not to be less than100= 3700 in order to ensure that all 37 numbers occurat least 100 times on average. The prescribed number

N= 100 can really be judged as being a compromise:sometimes clients consider it as being too big, but toascertain the smaller deviations of a roulette wheel moredata would be necessary.

On the basis of the list the frequencies kj of theoccurrence of the numbers j (j = 0, 1, ..., 36) aredetermined, where the frequency k

j

shows how manytimes the numberj occurred among the valuesxi.


t e c hn ique

STATISTICAL CHECKING

Mathematical control of the randomnessof gambling devices

ERNO GTI, Head of Software Testing Section, OMH, Budapest, Hungary


14/37

In the case of a roulette wheel operating regularly,i.e. randomly, every frequency kj is a random variablefollowing a Bernoulli distribution with an expectation:

and with a theoretical standard deviation:

So for every frequencykj the condition:

j 3j kj j + 3j

has to be fulfilled with a high probability, the value of

which is approximately 99.73 %. The factor 3 that pre-cedes j gave this test its name, which - according to theauthors experience - can be judged as being optimal.The choice of a smaller factor would result in a higherrisk of thefirst order error, when roulette wheels operat-ing correctly would be rejected more often, since thedata would be outside the interval more often. Thechoice of a higher factor would increase the risk of the

second order error, when unacceptable roulette wheelswould be accepted.

The above method is used by most roulette wheelmanufacturers to check their product, with the modifi-cation that the frequencies of odd/even, red/black, etc.numbers are also examined. However, this test onlycontrols the uniformity of the frequencies of the num-bers spun, and says nothing about their randomness. Aroulette wheel spinning consecutively increasing num-bers (e.g. 15, 16, 17 or 35, 36, 0, etc.) would pass theabove test properly; every frequencykj would fall in the

centerof the prescribed interval, despite the fact that thedecisions of the given roulette wheel would beextremely

predictable and not random at all. The operation of aroulette wheel is deemed to be random if no regularitycan be found in adjacent numbers.Hence the randomness

of the sequence of the numbers spun must also be checked.

For this purpose one can employ several methods,for instance the correlation method. According to theauthors experience one of the simplest, most efficientand most demonstrative methods is to take the differ-ence between the numbers in sequence:

yi,1 xi+1 xi (i = 1, 2, ...,N1).

Since the previous differences can be negative, thenext non-negative quantitieszi,1 are formed:

The so-called modulated differences zi,1 can have valuesbetween 0 and 36 too, and if the roulette wheel operatesregularly the distribution of the frequencies kj, calcu-lated on the basis of the valueszi,1, must also obviouslybe approximately uniform, similarly to the frequencieskjcalculated from the original data in the list. Hence forthe frequencies kj calculated on the basis of the values

zi,1, the foregoing 3test is also performed, taking (N 1)instead of N. A wheel spinning numbers incrementallywould fail this test; the frequency k1 belonging to thechannel number 1 (i.e. belonging tozi,1 = 1) would bemuch bigger than the upper limit of the given intervaland all other frequencieskj would equal zero.

Considering that the 3test is used not only for rou-lette wheels, but to control computer programs drawingprizes, for example, the above test is performed not onlyfor the differences zi,1, calculated from the adjacent

numbers on the original list, but for the modulated dif-ferenceszi,k, that are calculated from thek

th neighbors.Thezi,k modulated differences are defined as follows:

yi,k xi+k xi ; herek = 1, 2, 3, ... and i = 1, 2, ...,Nk.

Using this method the gross errors, the short period-icity of the random generator of the computer or of thedrawing program can be found out.

Performing the test for the 4th neighbors too, thereare 5 37 = 185 frequencies on the data page of a rou-lette wheel. Since only P = 0.9973 probability belongsto 3, among these data sometimes (but regularly) somefrequencies do not fulfil the 3condition, however theroulette wheel under control operates well. In thesecases the decision to accept or reject it is partly sub-

jective; the quantity, the place (the column and the line

in the page) and the magnitude of the overstepping areexamined.The great advantage of the 3 test is that it can

generally be used for most gambling games such as slotmachines or for lottery-type games, and in the case ofrejection it gives an indication as to the possible reasonfor the deviation. However the 3 test does have adisadvantage: unambiguous mechanical decisionscannot be made in the case of overstepping, i.e. whenone or more frequencies are outside the given interval.Partly for this reason and also to render the decisionbetter founded, as an addition to the 3method some-

times the2

test is also performed on the same data. Inother cases only the2 method is used.


t e c hn ique

j kj =Npj =N

j +(kj j)

2= + Npj(1 pj) = +

N

1 1

zi,1 yi,1 ifyi,1 0

yi,1 + ifyi,1 < 0

zi,k

yi,k ifyi,k 0

yi,k + ifyi,k < 0

(1)


15/37

3 The2 test for roulette-type games

Games are considered as being of the roulette-typewhere, at least in principle, it is possible to witness the

same phenomenon of successive incremental spinsdescribed above. Roulette wheels (and most slot ma-chines) belong to this category for instance. In the caseof roulette wheels, repetitions of numbers must occurregularly. From the point of view of mathematics,roulette-type games differ from lottery-type games to asignificant degree: in the latter, the numbers drawnduring one game are always different. Such games are,for instance, the 90/5 lottery, where five different num-bers are always drawn out of 90, the 45/6 lottery, the80/20 keno, and all types of bingo games.

The2 test is based on the very important statistical

theorem described below, which is widely used forchecking hypotheses on probability distributions.

Let the random eventsA1,A2, ,Aj, ,A mutuallyexclude each other and let them constitute a completesystem of events. In this case for the probabilities

p1,p2, ,pj, ,p of the events the condition:

p1 +p2 + +pj + +p = 1

is true, i.e. during one experiment one (and only one)event out of different possible events will occur. Letkjdesignate the frequency of the eventAj occurring out of

Nexperiments where, of course,kj = 0, 1, 2, ...,N and

(it is important to note that the greatest possible value ofeach frequencykj equalsN, which is the total of all thenumbers spun in the case of roulette wheels). The set offrequencies kj follows a -variable Bernoulli distribu-tion:

P(k1,k2, ...,kv) = pk

1

1p

k2

2...p

k

and according to the theorem the distribution limit ofthe next random variable constituted from the frequen-cieskj:

is a 2

distribution withr= 1 degrees of freedom, ifN . Using the formulae, ifN :

Here the index R of the variable 2 indicates theroulette-type game, and F1(x) designates the distribu-tion function of the2 distribution withr= 1 degreesof freedom. Developed over the course of time, the factthat a random variable and a probability distribution aretraditionally designated by the same symbol 2 may beinconvenient, though the random variable2R appearingin (2) above is of an2 distribution only in the case when

N , and not in general.As a reminder, the density function fr, the distribu-

tion function Fr and the first two moments of the 2

distribution withrdegrees of freedom are given, whichhave the greatest importance during practical use:

Here the function is traditionally defined as:

Ifx is an integer, then (x) = (x 1)! The expectation

of the2 distribution withr degrees of freedom

For practical purposes it may also be important thatthe2 distribution withrdegrees of freedom can be wellapproximated by a normal distribution, whose expecta-tion equalsrand whose standard deviation equals 2r, ifthe degree of freedomris big enough.

When performing the mathematical test on roulettewheels, for instance, the 2 method can be used as fol-lows:

a) on the basis of the data in the list sent in by the client,the frequencieskj are determined;

b) supposing that thepj =1 =

137 condition is true, i.e. the

roulette wheel under control operates regularly, eachnumber is spun with the same probability, on thebasis of formula (2), the value of the variable 2R iscalculated;

c) for an appropriate probability P of acceptance (the

value of which usually equals 0.9973 belonging to the3), supposing that the number Nof the data is big


t e c hn ique

kj =N

N !k1!k2! ...,kv!

(kj Npj)2

Npj

j=1

2R

j=1

Pr(2R


16/37

enough for formula (3) to be used, the critical value2crit is determined from the condition

P = 2R

(x =2crit);

d) the values of the variable2R

and the critical values2crit are also determined for the modulated differ-enceszi,k, if necessary.

From a mathematical point of view the roulette wheelcan be accepted if the condition(s):

2R 2crit

are fulfilled for the base data and for the modulateddifferenceszi,k as well.

The great benefit of the2 test is that it always givesthe possibility to make an unambiguous decision, but itsdisadvantage is that it does not give any information

about the possible reasons in the case of rejection.

4 Improvement of the2 test forroulette-type games

The2 test is used not only for checking roulette wheels,where the condition N >> 1, necessary for the use offormula (3), is fulfilled in practice, but its applicationwould sometimes be useful when the number Nof theexperiments is not big enough or if it cannot be ascer-

tained whether it is big enough. In these cases the deci-sion of the authority based on mathematical tests wouldnot be well enough founded and sufficiently indisput-able. For this reason the distribution function 2R(x) ofthe random variable

defined by formula (2), was examined in detail, and itwas established that the relation 2R(x) = F-1(x), that istrue in the case of N , is the first member, inde-pendent ofN, of a series according to the powers of

In order to make the distribution function moreexact the second and the third members of the serieswere determined too. Since the second member, whichis proportional to

equals zero, the distribution function of the variable2R is:

The previous formulae contain the expression

which is small for a fixed value of when all the prob-abilities pj are almost equal, i.e. when the probabilitydistribution of the events Aj is approximately uniform.This fact confirms the rule, as can be seen from the liter-ature, that prescribes almost uniform distribution forthe successful use of the first approximation, i.e. of theclassical2 test. According to that rule it is also necessaryfor every eventAj to occur at least 10 times. In the case

of slot machines neither the first nor the secondconditions can be fulfilled: the probability distributionof the different winning combinations occurring differsfrom the uniform distribution to a significant degree,and it cannot be ensured either during all the tests thatthe very infrequent jackpot will occur 10 times at least.

Hence considering the second and the third memberin the series makes it possible toprove the correctness ofusing the first approximation or to make well-foundeddecisions even in the cases when the conditions neces-sary for the use of the first approximation cannot befulfilled.

5 The2 test for lottery-type games

If out of the numbers 1, 2, ...,j, ..., during one draw ndifferent numbers are drawn, for instance in the case ofa 90/5 lottery n = 5 different numbers out of = 90 andevery number has the same probability of being drawn,then the probability of being drawn is obviously:

pj =p = n for all the numbers.


t e c hn ique

j=1

1

N

1

N

(kj Npj)2

Npj

2x

1

1

pj

1

pj

1

pj

2 + 2 2

32 6 + 4

8

24

5

Bx2

+ 3 1

N3/2

x(A + 2B)

2R

2

R

(x) = F-1(x) +

where:

andf-1(x) and F-1(x) are the density function and distri-

bution function of the2 distribution with 1 degreesof freedom, respectively.

A =

B =

f-1(x) + O A B + + 1

j=1

j=1

j=1


17/37

If the draw is repeatedNtimes, the frequencieskj ofthe numbersj occurring among the numbers drawn canbe determined from the data of theNdrawings, similarlyto roulette-type games. However, here for the frequen-cies kj obviously the following conditions have to be

fulfilled:

kj = 0, 1, 2, ...,N and

Every frequencykj follows the same Bernoulli distri-bution:

with the expectation kj =Np and with the variance(kj )2

2 =Np(1 p).

It is quite logical on the basis of the foregoing todefine the next random variable:

(4)

similarly to the case of roulette, and to hope that itsdistribution extends to a known distribution limit, forexample to the evident 2 distribution, if N . Thedistribution limit of this variable 2L (whereL refers tothe lottery) cannot be derived on the basis of the theo-

rem shown in paragraph 3 above, since the conditions ofuse of that theorem are not fulfilled here; the set of fre-quencies kj does not follow a multivariable Bernoulli

distribution, however, every kj in itself is of (simple)Bernoulli distribution. Even in principle it is impossiblethat every number occurs Nn times (as in the case ofroulette), though the number of all the drawn numbersequalsNn. Every number j can occurNtimes at most.However, it is a lucky circumstance, that for n = 1 the

variable2L is the same as the variable2R, if the relation

pj = 1/is true. Therefore their distributions (and conse-quently their distribution limits) must be the same as

well. This fact makes it easier to check the correctness ofthe relations obtained.In order to guess the distribution limit required, let

us determine the expectation of the variable2L.

Is the distribution limit that is sought after a 2

distribution with a degree of freedom n? Let us also

determine the variance of2L. As a result of calculationsthat are more complicated than the foregoing,

Because forN

guessed that the distribution of the next variable Y

(5)

goes to a 2 distribution with 1 degrees of freedom,since the expectation of the variable Yequals Y =v 1and its variance is:

Indeed, using the method of the characteristic func-tions it was proved that for the case ofN

Y = F-1(x) (6)

The theorem can also be proved for the more generalcase, when the probabilities pj belonging to the event,that the number j is occurring among the n numbersdrawn, are not equal, but this proof is quite complex andis not necessary for our purposes.

Applying the above theorem the 2 test can also be

used for lottery-type games, if the numberNof the drawresults, in the list available for the tests, is great enough.

6 Improvement of the2 test forlottery-type games

Sometimes huge prizes are drawn in lotteries, which iswhy it is of special importance that the mathematical

test methods of the randomness of the games must beindisputable. For this purpose the second and thirdmembers of the distribution function of the variable Y,given by formula (5), were defined as well. Hence thedistribution function:


t e c hn ique

j=1

j=1

j=1

j=1

j=1

kj =Nn.

(kj Np)2

Np

( n)2

1

( n)2

1

1

n

1

1

N

1N

(kj Np)

2

Np

(kj Np)2

Np(1 p)

Np(1 p)

Np

2

Np

2

2L

(2L 2L)2

= 2

2L Y

2Y

= 2( 1)

2

2L 2

2( 1)

Y(x) = F-1(x)

+

x

6N

x2

+ 3

f-1(x)

+ 1

2

Np

2L =

2L

=

= =

=

= =(1 p) = n

Nkj

pkjj

pkj

Nkj

1

1

can be given.

, it can be

P(kj) = (1 pj)

N-kj

= (1 p)N-k

j

3(+ 1 x) +

+ 1 2x +

1

N3/2

+ O


18/37

where:

It can be seen that n appears only in the formula of.

The values of for n = 1 and for n = 1 are the same,therefore the distributions of Ymust also be the same inthese two cases. This fact is not unexpected at all, sincethe task of choosing one number out of is equivalent tothe task of choosing 1 different numbers out of . Atthe same time, for n = 1 the distribution function of the

variable Y is equal to the distribution function of thevariable2R, if

is substituted therein, as it must be, since fulfilling con-dition (7) - i.e. in the case of uniformly distributedprobabilitiespj - the variable

2R forms a special case of

the variable Y.Because in the case of lottery-type games the order

in which the numbers are drawn is not of any signifi-cance, here it is not reasonable to perform the2 test forthe modulated differences zi,k too. The sequence of theresults of lottery drawings has not been recorded instatistics for several decades; the numbers drawn arereported only in increasing order.

7 Conclusions

One means (and in the given cases the only possiblemeans) of testing gambling devices is the mathematicalstatistical checking of draw or spin results. This papershows the 3 and the2 tests, which are most frequentlyused. The advantages and disadvantages of both methodsare given, on the basis of which it can be established thatthe two methods are complementary to each other,therefore if possible both should be used. If a 2 testresults in a rejection, it is useful to perform the 3 testtoo in order to ascertain the reasons for this rejection.

The conventional 2 method cannot be used fortesting lottery-type games, which is why the author hasput forward additional calculations that allow the2 testto be used for this purpose as well.

Considering that the 2 test is used not only forchecking scientific hypotheses but also for establishingan official decision about acceptance or rejection, theimprovement to the 2 test was made for both kinds ofgames, i.e. for roulette and lottery-type games. Thisgives the possibility toprove the correctness of use of thefirst approximation or to use the 2 test in cases whenthe conditions for using the first approximation cannotbe fulfilled, for instance when the number N of theexperiments is not big enough, or when the probabilitydistribution of the eventsAj is not uniform.


t e c hn ique

j=1

j=1

= 2 (7)pj = , i.e. =1

1pj

( 1)( 2n)2

n( 2)( n)


19/37

e vo lu t i on s


The Aragonese were faced with a complex situationin the field of weights and measures in so far as

the ancient traditional measures used in tradewere very disparate due to the presence of a number ofindependent measurement standards, to the lack of anyeffective control on the part of the authorities and even,in some cases, to outright abuse by these authorities inimplementing checking procedures.

To remedy this situation and hence ensure uniformityof measurements, King Ferdinand I of Aragon issued alegislative act in April 1480. He even had a monumentcarved to ensure not only that word got round about thisact but also that it was actually put into practice, andadditionally sent details of the act to the appointed head

magistracy. Even though the monument became damagedover time, it was nevertheless preserved up until the lastcentury in the Castelcapuano courtyard in Naples, whichwas the office of the Royal Court of the Weights andMeasures Mint. The marble stump bore Aragonese insig-nia and the Latin inscription:

FERDINANDUS.REX.INUTILITAT

EM.REI.P.HAS.MENSURAS.PER.MAGIST

ROS.RATIONALES.FIERI.MANDAVIT

The monument was engraved with the linear andcapacity measures in force at that time. Some samples

were also conserved by the church, in line with anancient law of Giustiniano that was, by then, becominga tradition since in a Christian society the church wasconsidered as being the best place to confer absoluteand indisputable values to measures that were derivedfrom the human body, such as the palm of a hand or thefoot.

Ferdinands legislative act is not conserved in pub-lished collections, but was recorded thanks to the trans-cription made by the humanist Melchiorre Delfico in1787 [5].

Ferdinands edict was of note because it led to order

being restored in the field of weights and measures thatremained untouched for the next four centuries.

Unfortunately, there is no historical record of howthe weights and measures that he imposed as being

legally binding under his reign were actually deter-mined.

At the beginning of the 19th century a Committee ofLearners was charged by Gioacchino Murat (King ofNaples from 1808 to 1815) to compare the Aragonesemeasures with the Parisian metre, to inquire into theorigins of the system of measures being used and to estab-lish if a scientific principle existed that had inspired theinventors of the Neapolitan system of weights and mea-sures [2].

HISTORY OF ITALIAN METROLOGY

Legal metrology in the city of Naples duringthe Aragonese domination (14421503)

SILVANA IOVIENO, W&M Officer, Ufficio Provinciale Metrico e del Saggiodi Metalli Preziosi di Napoli, Italy

Castelcapuano, which was the site of theRoyal Court of the Weights and Measures Mint


20/37


e vo lu t i on s

This comparison showed that the Parisian meter wasin fact bigger that the Aragonese four palm measureby 1/200 m, but the sample examined was not in a goodstate and was marred by the intensive use that had beenmade of it over the four centuries during which it

constituted the sole legal basis.The committee found no reference as to the origins

of the system of measurement used in Naples, despitehaving searched right through the Royal Archives.

Moreover, it is curious to note that since theAragonese era the mile (the unit of the customary surveymeasure used in Naples) was defined to be 1/60 of adegree of the earths meridian and the palm (the unit oflinear measure used in Naples) as 1/7 000 of the mile; itcan therefore be affirmed that since the time of Alfonso Iof Aragon, linear measurement was strictly related tothe length of the earths meridian.

The fact that in 1811 the Committee found such asmall and wonderful [5] difference between theAragonese measure and the French metre led the scien-tific establishment in the last century to consider thatthe palm originated as an aliquot part of the earthsmeridian almost four centuries before the invention ofthe metre. It would have been a major achievement forthe illustrious (and unfortunately ignored) Committee ofNaples at the Aragonese court to have proved this!

It can be noted from the analysis of the instructionsKing Ferdinand gave to his treasurers that an ad hocstructure for regular control of every weight, measureand instrument that the sellers used to weigh and mea-sure was created for the first time in the Reign of Naples.

During Ferdinands reign the law also established ahierarchy of primary standards, to render it easier tocompare these with the standards used by instrumentsin trade.

It provided for the creation of a register containingdetails of anyone using instruments to weigh and

King Ferdinand I of Aragon

View of the Castel Nuovo (also called Maschio Angioino),site of the Aragon Government

Measure conversion table from the Monitore del Regno delledue Sicilie (Official journal of the Kingdom of Naples) published

in 1813 under King Gioacchino Murat


21/37


e vo lu t i on s

Bibliography

[1] Melchiorre Delfico: Memoria sulla necessit di rendere uniformi i pesi e le misure del regno (Naples, 1787)

[2] Saverio Scrofani: Memoria delle misure e pesi dItalia (Naples, 1812)

[3] Cavalier Cagnazzi: Memoria sui valori delle misure e dei pesi degli antichi Romani desunti dagli originali esistenti nelReal Museo Borbonico di Napoli (Naples, 1825)

[4] De Ritis: Il nostro sistema metrico sanzionato con la Legge del di 6 di aprile 1840 (from the 1841 Annals)

[5] Afan de Riveira: Sulla restituzione del nostro sistema di misure, pesi e monete alla sua antica perfezione (Naples, 1838)

[6] Ceva Grimaldi: Sulla riforma de pesi e delle misure ne Reali Domini al di qua del Faro, considerazioni di Ceva Grimaldi (Naples)

[7] F. de Luca: Esame critico di alcuni opuscoli pubblicati intorno al sistema metrico decimale della citt di Napoli (Naples, 1839)

[8] Col. V. Ferdinando: Del sistema metrico uniforme che meglio si conviene ai domini al di qua del Faro del Regno delle due Sicilie (Naples, 1832)

[9] Vladimiro Valerio: Societ uomini ed istituzioni cartografiche nel mezzogiorno dItalia (Istituto Geografico Militare Eds.)

The Latin text of King Ferdinands Weights and Measures Actdated 4th April 1480, as reported by Melchiorre Delfico

measure for trade purposes (in the Middle-Age Italianlanguage: quinterni lucidi et clari); this register enabledthe Central Administration to examine the behavior ofthe appointed head magistracy for instrument control.

Ferdinand also laid down that every instrument had

to be marked with the Royal insignia to certify its accu-racy; the owners of instruments had to re-mark them inthe presence of a Royal official after any repairs werecarried out.

Many dispositions that Ferdinand laid down are stillpresent in the Italian regulations currently in force, withsome modifications made of course.

Lastly, the punishment for offenders who did notobserve the law was a fine of 1 000 golden ducats pay-able to the Crown - an amount so high that breachingthe law was effectively discouraged [1].

King Ferdinand I of Aragon set up the basic legal

metrology system so well that his directives were leftuntouched over four centuries until Gioacchino Muratintroduced the French decimal metric system inNaples.

Cover page of the book referred to as [1] in the Bibliography


22/37

Secretariat: United States of America

Chairman: Sam Chappell

Participation: Forty delegates representingnineteen OIML Member States,one Corresponding Member,the OIML Development Council,one liaison organization andthe BIML, as detailed below

P-members: Australia, Belgium, Bulgaria,P.R. of China, Czech Republic,Denmark, France, Germany, Japan,

Netherlands, Norway, Poland, Russia,Sweden, United Kingdom, USA

O-members: Finland, Slovakia, Switzerland

OIML Corresponding Member: Albania

Liaison institution: CECIP

OIML Development Council: Tunisia

Discussion topics reported on:

1 TC 3/SC 5 Conformity Assessment

2 Fourth Draft OIML DocumentMutual Acceptance Agreementon OIML Pattern Evaluation

3 Expression of Uncertainty in Measurement

4 Reports on the Status of Work

5 Resolutions of the Meeting

Objective:

To discuss the fourth draft OIML Document MutualAcceptance Agreement on OIML Pattern Evaluation and

to provide a status report on the work of OIML TC 3Metrological Control

25

upda t e

O I M L B U L L E T I N V O L U M E X L N U M B E R 4 O C T O B E R 1 9 9 9

International Working Group: OIML TC 3 Metrological Control

Maison de la Chimie, Paris, 13 June 1999

Forty delegates attended the OIML TC 3 meeting in Paris, chaired by Sam Chappell (CIML Vice-President)


23/37

1 TC 3/SC 5 Conformity Assessment

Prior to the meeting, the Secretariat distributed a ballotto P-members for comment and vote on the establish-

ment of a new subcommittee TC 3/SC 5 ConformityAssessment, whose objective would be to establish rulesand procedures for fostering mutual confidence in theresults of testing measuring instruments under legalmetrology control among OIML Member States. TheUSA and the BIML were proposed to hold the jointSecretariat for TC 3/SC 5.

The results of the ballot were as follows: 22 out of the24 ballots were returned (missing: Austria and France)with 20 Yes votes and 2 abstentions (Germany andNorway). Written comments on the ballot were receivedfrom Australia, Germany and the United Kingdom. Ofthose who responded, 18 Member States registered tobecome P-members and 7 as O-members. It was de-clared that TC 3/SC 5 could be established subject toCIML approval.

During the meeting, the following projects wereidentified as to be maintained:

Document on the OIML Certificate System for Measur-ing Instruments;

Draft Document on Mutual Acceptance Agreement onOIML Pattern Evaluation; and

Working Draft Document on the Expression ofUncertainty in Measurement in Legal Metrology Appli-

cations.

2 Fourth Draft OIML DocumentMutualAcceptance Agreement on OIMLPattern Evaluation

The fourth draft Document was reviewed clause byclause and the following principal changes were agreedupon:

the requirements for participation should be clearlyexpressed in the Scope and elsewhere where appropri-ate;

provision should be made for OIML Member States toindicate that they would accept certificates of con-formance issued under the Agreement;

both the evaluation body and the certification bodyshould be assessed for competence;

accreditation or peer review could be used to assesscompetence for the purpose of establishing mutualconfidence;

requirements for establishing competence should beequivalent for either process;

assessment teams should be made up of experts fortesting the category of instruments addressed and atleast one quality systems expert; and

the Questionnaire on National Capabilities, Annex C inthe Draft Document, should be made generic.

It was agreed that supplementary Documents neededto be developed for assessing the competence of parti-cipants. Such Documents should be based on existing ordraft ISO/IEC Guides and Standards in which the rele-

vant legal metrology applications would be addressed.The following were identified:

ISO DIS 17025 General Requirements for the Compe-tence of Testing and Calibration Laboratories.G. Lagauterie provided a complementary text forinterpreting the requirements as applied to patternevaluation in legal metrology;

ISO/IEC Guide 65 General Requirements for BodiesOperating Product Certification Systems. John Birch,assisted by the secretariat, will investigate the statusof this Guide and recommend how an interpretationdocument should be developed as applied to certifyingbodies issuing certificates under the agreement; and

ISO/IEC Guide 68 Considerations on Entering intoMutual Recognition Agreements and EA-2/02EA Policy

and Procedures for the Multilateral Agreement(November 1998). G. Engler agreed to seek permis-sion from his management to develop criteria forassessment teams that would evaluate participants in

the Agreement.

3 Expression of Uncertainty in Measurement

A second Working Draft Document on the Guide forConsidering Measurement Uncertainty in Legal Metrologyby G. Lagauterie had been distributed together withcomments by the USA prior to the meeting. The topic ofuncertainty was discussed in presentations as follows:

K. D. Sommer gave a presentation on Uncertainty inMeasurement in which he addressed the approach forexpressing uncertainty separately for calibrated andverified measuring instruments (see technique);

S. Chappell gave a presentation on Traceability inMeasurement and its importance in establishing con-fidence in measurement at international level and itsinfluence on the definition of uncertainty for legalmetrology applications; and

G. Lagauterie presented a paper with examples forfluid measuring instruments to supplement theapproach taken in the working draft. The title of the

paper was Origins of measurement uncertainties whencalibrating or verifying a measuring instrument.

26

upda t e



24/37

It was agreed to continue the work in developing anOIML Document on the Expression of MeasurementUncertainty in Legal Metrology Applications. A task groupwith representatives from France, Germany and theUSA was identified to continue the effort. Others having

an interest were invited to participate and shouldidentify themselves.

4 Reports on the Status of Work

Written reports were provided by the Secretariats on thestatus of the work of the following subcommittees:

TC 3/SC 1 Pattern Approval and Verification (USA);

TC 3/SC 2 Metrological Supervision (Czech Republic);

TC 3/SC 3 Reference Materials (Russian Federation);and

TC 3/SC 4 Statistical Methods (Germany)*.

Time did not permit a discussion of these reports,but copies may be obtained from the BIML on request.

5 Resolutions of the Meeting

1. According to the response to the ballot of P-members,

TC 3/SC 5 Conformity Assessment was approved byTC 3 to be established with the co-Secretariat of theUSA and the BIML.

2. The minutes of the meeting to record the majorpoints of the discussions will be prepared by theSecretariat and distributed to all participants withinone month.

3. Written comments by participants on the 4th draftOIML Document Mutual Acceptance Agreement onOIML Pattern Evaluation should be submitted to theSecretariat by no later than July 15, 1999.

4. On the basis of the discussions held at the meeting

and the written comments received, the Secretariatwill prepare a 5th draft OIML Document for distribu-

tion to collaborators in the work of OIML TC 3/SC 5by August 15, 1999.

5. At its October meeting, the CIML will be providedwith a report on the proposal to establish OIMLTC 3/SC 5 and its objectives, scope and work pro-

gram. The CIML will be requested to endorse thework.

6. An interpretation Document shall be developed by atask group on the application of the ISO DIS 17025 tolaboratories performing pattern approval tests inlegal metrology. Comments on the initial draft on thissubject should be submitted to Mr. Lagauterie andthe Secretariat by no later than July 15, 1999.

7. An interpretation Document shall be developed by atask group on the application of the ISO/IECGuide 65: 1996 General Requirements of Bodies Oper-

ating Quality Product Certification Systems to nationalresponsible bodies performing pattern evaluationand/or issuing certificates of pattern approval inlegal metrology.

8. A task group consisting of representatives fromFrance, Germany and the USA was requested todevelop a new draft on the expression of uncer-tainty as applicable in legal metrology based on thediscussions held at the meeting.

27

upda t e


* Since TC 3/SC 4 is to restart its activities under theresponsibility of Germany, a full report is printed overleaf

Sam Chappell


25/37

Status Report: TC 3/SC 4 Application of statistical methods for measuring instruments

1 Preliminary remarks

At the 33rd CIML Meeting in Seoul Prof. Kochsiekagreed that Germany should take on the chairmanshipof TC 3/SC 4Application of statistical methods.

Statistical control methods in legal metrology areurgently needed (and indeed already widely discussed)in connection with a number of International Recom-mendations; below are some suggestions as to how thissubject could be dealt with in the future.

2 Statistical control methodsin legal metrology

Statistical sampling methods are a compromise betweenthe (reduced) accuracy of an estimation and the wholeentity of a test as would be necessary for a complete orindividual test. Statistical control methods may also beconsidered as quality assurance measures taken indiffering cases, i.e.:

preventive assessment with a view to future use;

follow-up assessment on whether the given charac-

teristics were actually met.

A different case is the assessment of measuringdevices already in use, for example electricity meters.In this case it is possible to examine by sampling inspec-tion whether, after several years of operation, the elec-tricity meters still give measurement results which areso good that the meters may remain part of the elec-tricity supply system for a further time period. In itsmodified form the sampling plan may provide infor-mation about the state of a measuring instrumentbatch already in use.

As there are a large number of differing appli-cations of statistical methods, a uniform control levelshould not be assumed. Experience gained from a large

variety of measuring instruments has shown that it ismore expedient to define individual problems withtheir own statistical conditions.

For example, where measuring instruments aremanufactured in highly automated processes signifi-cant statements on measurement parameters mayalready be made on the basis of internal quality checks.However, where they are manufactured in manualprocesses other marginal conditions apply which willalso have to be taken into consideration by the statis-

tical control methods.

3 Level of protection (essential for

the sampling plan)

Similar to a modern production line that is managedusing a quality system, quality objectives also have tobe initially defined in the legal metrology field.

The concept of a sampling plan aimed at eitheracceptance or rejection will always be oriented to suchquality objectives, i.e. the level of requirements. Forexample, where measuring instruments subject to legalcontrol do not meet the relevant requirements, eco-nomic disadvantages for the supplier or the consumer,health risks or safety problems may arise.

These shortcomings have to be weighted accordingto their significance and are to be taken intoconsideration in the testing procedure. Wherever thehighest level is to be achieved the individual control ofeach measuring instrument with the correspondingworkload involved will be necessary. However, in manycases it will be expedient to specify the control level inaccordance with the application of the unit under testin order to optimize the cost-benefit ratio - meaning toadapt the scope of control to the metrological needs.Hence, statistical tests at a statistically calculated pro-tection level will generally be possible and will make

sense. On the other hand this will mean that the meas-uring instrument manufacturer will have to orientatehis quality system to the protection level required bylegal metrology.

4 Fundamental assessment situations

Statistical tests at a corresponding protection level areconceivable as follows:

A batch of new measuring instruments is to be usedfor the first time in the legal sphere.

Statistical assessment considers the new state of theinstruments, which have to comply with a given pat-tern and which are assessed according to thecharacteristics of the pattern. The sampling plan willtake into account which batch qualities will implydefinite acceptance and which will imply definiterejection. The acceptance and rejection characteris-tics have to be clearly and basically defined; rejectionof the batch will (in the worst case) lead to a market-ing prohibition which may, however, be repealed if

the instruments are repaired.

28

upda t e



26/37

29

upda t e


A batch of instruments has been in use for a longerperiod of time. Within the framework of a marketsurveillance it is to be assessed whether the batchdoes in fact fulfil the requirements to be met for the

application, or the batch condition is to be analyzed.In such a case it will be necessary to apply a satis-factory separation method on the ba