jcgm survey (gum) collated responses - bipm · this document contains the collated responses to the...

JCGM Survey (GUM)

Collated responses

2012-07-19

Willem KOOL BIML Assistant Director [email protected]

JCGM-WG1-SC5-N12-15

This document contains the collated responses to the JCGM Survey on the GUM for which the closing date was 15 June 2012. Personal information about the respondents has been omitted.

This document has three parts:

Part 1: The survey questionnaire 5

Part 2: Summary information about the responses to each of the questions 13

Part 3: Collated responses to each of the open questions 23

JCGM Survey (GUM) Collated responses

2012-07-19 of 91

PART 1

The survey questionnaire


2012-07-19 of 91


2012-07-19 of 91

PART 2

Summary information about the responses to each of the questions


2012-07-19 of 91


2012-07-19 of 91

1 of 95

JCGM_GUM

1. Which of the following best describes your occupation?

Response

Percent

Response

Count

Scientist 23.4% 45

Engineer 22.4% 43

Metrologist 35.4% 68

Teacher 4.7% 9

Technician 3.6% 7

Linguist 0.0% 0

Other (please specify)

10.4% 20

answered question 192

skipped question 8


2012-07-19 of 91

2 of 95

2. Which of the following best describes your primary field of expertise ?

Response

Percent

Response

Count

physics 20.3% 39

chemistry 11.5% 22

technology 18.2% 35

metrology 34.9% 67

clinical laboratory sciences 3.6% 7

legislation 0.5% 1

psychology 0.0% 0

Other (please specify)

10.9% 21


skipped question 8

3. How would you describe your degree of knowledge of the GUM ?

Response

Percent

Response

Count

[a] none, or almost none 7.8% 15

[b] not enough to take full

advantage of its content11.5% 22

[c] sufficient for my needs 31.8% 61

[d] good 35.9% 69

[e] excellent 13.0% 25

Comment (optional):

9


skipped question 8


2012-07-19 of 91

3 of 95

4. In which field do you use the GUM ?

Not in my

field of

expertise

Not at all

(0)

Occasionally

(1)

Regularly

(2)

Intense

(3)

Response

Count

Metrology 7.1% (13) 9.2% (17) 20.1% (37) 35.3% (65) 28.3% (52) 184

Research 14.2% (21) 18.2% (27) 26.4% (39) 27.7% (41) 13.5% (20) 148

Teaching / education 18.5% (29) 13.4% (21) 26.8% (42) 28.7% (45) 12.7% (20) 157

Industrial practice 22.2% (32) 18.1% (26) 28.5% (41) 20.8% (30) 10.4% (15) 144

Comment (optional):

11


skipped question 8

5. a. Internal inconsistency: Conflicting (frequentist and Bayesian) views of probability,

which cause a number of problems, especially in the evaluation of a coverage interval.

Response

Count

63


skipped question 137

6. b. Internal inconsistency: Conflicting use of terminology, such as for the term “value”

Response

Count

63




2012-07-19 of 91

4 of 95

7. c. External inconsistency: With respect to Supplements 1 and 2 and the other

documents being developed according to a consistent conceptual framework.

Response

Count

51



8. d. External inconsistency: With respect to the VIM 3.

Response

Count

57



9. e. Inadequacy: Since its publication, the need to evaluate measurement uncertainty

has been recognised in an increasing number of scientific disciplines, for which the

present GUM does not provide sufficient guidance.

Response

Count

73




2012-07-19 of 91

5 of 95

10. f. Inadequacy: The present GUM is focused mainly on the situation of a physical

quantity which “…can be characterized by an essentially unique value”. This has

probably inhibited the wide use of the GUM in some scientific disciplines, such as

chemistry and biology, in which a definition of the measurand according to the

requirements of the present GUM can be impossible.

Response

Count

65



11. g. Ambiguities: Notational and terminological.

Response

Count

47



12. a. Clarity of presentation.

Response

Count

61



13. b. Structure as close as possible to that of the present GUM.

Response

Count

64


skipped question 136JCGM Survey (GUM) Collated responses

2012-07-19 of 91

6 of 95

14. c. Level of presentation comparable to that of the present GUM.

Response

Count

54



15. d. Better specification of the conditions of applicability.

Response

Count

58



16. a. Increased guidance in the evaluation of standard uncertainties associated with

input estimates.

Response

Count

63



17. b. Bayesian approach extended to Type A evaluations of uncertainty.

Response

Count

57




2012-07-19 of 91

7 of 95

18. c. Increased number of examples, with applications taken from biology, chemistry

etc.

Response

Count

75



19. d. Links to GUM Supplements where appropriate.

Response

Count

65



20. What is your view on this timescale?

Response

Count

74



21. This is the end of the survey. If you have any additional remarks, please provide them

here:

Response

Count

45




2012-07-19 of 91


2012-07-19 of 91

PART 3

Collated responses to each of the open questions

Note:

The numbering of the responses to individual questions in this part does not imply a relationship between the answers to different questions, i.e. answer no. 7 to one question is not necessarily from the same person as answer no. 7 to another question.


2012-07-19 of 91


2012-07-19 of 91

9 of 95

Page 2, Q1. Which of the following best describes your occupation?

1 Accreditor Jun 15, 2012 7:46 PM

2 Consultant & Laboratory Accreditation Assessor Jun 7, 2012 3:25 PM

3 x Jun 6, 2012 11:29 AM

4 Consultant Jun 1, 2012 6:30 AM

5 state officer - metrology May 30, 2012 12:49 PM

6 medical physicist, clinical and in a secondary standard dosimetry lab May 20, 2012 11:42 AM

7 Statistician May 16, 2012 10:30 PM

8 Engineering Student Apr 26, 2012 8:06 PM

9 Hydrometerist Apr 3, 2012 11:28 AM

10 Mathematician Mar 22, 2012 5:56 PM

11 student Mar 21, 2012 5:47 AM

12 consulting engineer Mar 20, 2012 2:51 PM

13 Student Mar 14, 2012 2:50 PM

14 Accreditation assessor Feb 27, 2012 6:04 AM

15 QA Director Feb 26, 2012 7:34 AM

16 employee of an accreditation body Feb 24, 2012 2:28 PM

17 lab Feb 24, 2012 7:08 AM

18 Medical practitioner Feb 23, 2012 9:32 AM

19 Laboratory Physician Feb 14, 2012 5:03 PM

20 ISO/IEC17025 assessor Feb 14, 2012 6:53 AM


2012-07-19 of 91

11 of 95

Page 2, Q2. Which of the following best describes your primary field of expertise ?

1 Mechanical engineering Jun 6, 2012 2:22 PM

2 x Jun 6, 2012 11:29 AM

3 Statistics May 21, 2012 12:57 PM

4 medical physics, radiation dosimetry May 20, 2012 11:42 AM



7 Statistics Apr 11, 2012 1:27 PM

8 Environmental microbiology Apr 6, 2012 3:52 PM

9 Hydrometry Apr 3, 2012 11:28 AM

10 mathematics Mar 22, 2012 5:56 PM

11 manufacturing Mar 14, 2012 2:55 PM

12 streamflow Mar 12, 2012 9:46 AM

13 flow measurement Mar 9, 2012 4:19 PM

14 Laboratoire d'essais ferroviaires Mar 9, 2012 12:25 PM

15 Materials science Feb 29, 2012 7:10 PM

16 flow measurement Feb 29, 2012 4:54 PM

17 Metallurgy Feb 28, 2012 5:56 PM

18 accreditatin of calibration labs Feb 24, 2012 2:28 PM

19 quality control Feb 20, 2012 11:34 PM

20 statistics Feb 15, 2012 8:12 AM

21 - Feb 14, 2012 5:35 AM


2012-07-19 of 91

13 of 95

Page 2, Q3. How would you describe your degree of knowledge of the GUM ?

1 I served as co-director of the project on adoption of GUM and associatedGuidelines as national standards of Russian Federation (GOST R 54500-series), published some articles and reports on basic issues of GUM andtook part in discussions of such issues at international seminars(conferences)

Jun 13, 2012 1:02 PM

2 My knowledge of the content of the GUM is sufficient for me to use thedocument, but my statistical background is not sufficient to understand someof the concepts presented in the document.

Jun 8, 2012 3:25 PM

3 Comprehension is difficult for people who have learned their skills by on-the-job training and hard work and do not have the benefit of universityeducation. The guide should be understandable by a skilled technician.

Jun 7, 2012 3:25 PM

4 Proposed to be included in a new GUM -- Generalized Gaussian ErrorCalculus Michael Grabe, Germany, 38104 Braunschweig, Am Hasselteich 5,[email protected] Abstract – Uncertainty assessments include thedifferences between the true values and the expected values of estimatorsas caused by unknown systematic errors which, on there part, are constantin time. To this end, the common practice to randomize systematic errorsshould be abolished. This proceeding and a new treatment of random errors,resolving a misinterpretation of old regarding the alleged interchangeability oftheoretical and empirical moments of second order, forges ahead what maybe addressed as "Generalized Gaussian Error Calculus". The formalismissues reliable uncertainties being up to localize the true values of physicalquantities. Index Terms – Generalized Gaussian error calculus, localizationof true values, measurement uncertainties. I. INTRODUCTION In view ofmodern metrology, the classical Gaussian error calculus is obsolete. Firstly,Gauss himself deliberately ignored so-called unknown systematic errors,these being errors constant in time and unknown with respect to magnitudeand sign. Without doubt, metrological traceability is a must. Hence, in myview, experimenters are asked to treat unknown systematic errors asbiases. At the same time, this way of proceeding would cause the classicalGaussian fomalism to break down – entailing, incidentally, the end of theanalysis of variance as well -- may we like it or not. Apart from theomission of biases in the past, error calculus still suffers from a sore pointscarcely ever addressed. To evaluate measurements, experimenters areused to adopting, without further ado, a formalism created bymathematicians. However, while mathematicians are in a position to freelypreset theoretical and empirical moments of second order (i.e. variances andcovariances), metrologists have nothing but empirical moments at theirdisposal and the common metrological practice to substitute empiricalmoments for theoretical ones is prone to spoil their formalism. Thus, thereis an abyss of old separating mathematical and metrological applications. Aswill be indicated, an appropriate metrological processing of empiricalmoments of second order presupposes the ostensibly trivial concept to referto equal numbers of repeated measurements. II. MAIN BODY As much aswe assume the laws of physics to be true, we expect the constants ofphysics to possess true values. In this sense, it should be the mission ofexperimenters to localize the true values of measurands via properlyspecified uncertainty intervals. Unfortunately, neither the GUM's “1standard uncertainties” nor its “expanded uncer-tainties” are up to localizethe true values of measurands. An unknown systematic errors shifts thebulk of repeated measurements as a whole, thus introducing a permanent,time-constant difference between “the center of gravity” of the repeatedmeasurements and the true value of the quantity aimed at. Repeatedmeasurements take place not until this fixed difference has been established

Apr 27, 2012 3:01 PM


2012-07-19 of 91

14 of 95


experimentally, i.e. over the course of the setting up of the components ofthe measuring device. Obviously, this being done, there is no longeranything random about the aforesaid difference. But as traceability asks usto reckon with any possible value, I propose to resort to biassed estimatorsand to subsequently submit suchlike biases to worst case assessments.Indeed, this proceeding would map the basic working principle of stationarymeasuring devices. Textbooks on the statistical treatment of empirical dataare used to allocating ample space to Student’s distribution density, to theStudent-factor and, eventually, to confidence intervals. Remarkably enough,these discussions are regularly confined to just one measured quantity andno hint is given as to proceed in case of two or even more variables or howto handle possible dependences between measured quantities. Given themeasured data are to be considered normally distributed, or at leastapproximatetely so, the distribution density of the empirical moments ofsecond order reveals, amazingly enough, a depence between the empiricalvariances and the empircal covariances, be the variables themselvesdependent or not. Obviously, just this tells us not to ignore empiricalcovariances, even if the considered variables happen to be independent. Atthe same time we should strive for equal numbers of repeated easurements,as otherwise empircal covariances turn out to be undefined. After all, thecrucial point is that considering consistently empirical covariances, be theimplied variables dependent or not, we may generalize the idea ofconfidence intervalls according to Student. And this, indeed, will be wellworth the effort However, at present, in regard to error propagation, to askfor equal numbers of repeated measurements, seems to be outside ofcommon practice. If we are willing to comply with the properties of themultidimensional model of normally distributed variables, we are put in aposition to define confidence intervals in error propagation. This covers, inprinciple at least, arbitrary many variables up to the mechanisms of leastsquares.This perspective appears exciting in regard to the attempt to localizethe true values of measurands. The idea to ask for equal numbers ofrepeated measurements put experimenters in a position to shape theirformalim outside the common practice, namely to start with theoreticalmoments and to afterwards substitute empirical moments for the theoreticalones. As an example, let us consider a set of independent variables. Here,the theoretical covariances are set to zero, leaving over the theoreticalvariances. In hindsight, however, when metrologists insert their empiricalvariances, as they have nothing else at their disposal, the empiricalcovariances are tacitly left behind which, in fact, spoils the formalism. Afterall, equal numbers of repeated measurements, allowing to introducecomplete sets of empirical variances and covariances, establish a well-structured methodology of data evaluation. This proceeding wouldstipulate measurement uncertainties in terms of linear sums of confidenceintervals and appropriatley designed worst-case estimations. Should theunderlying error model apply, the associated uncertainty intervals wouldlocalize the true values of the measurands. With respect to the method ofleast squares, non-randomized systematic would errors abrogate the Gauss-Markoff theorem, which, as is known, specifies the weighting matrix. Asweighting factors shift the adjusted parameters and shrink their uncertaintiesthe fundamental property of the error model discussed here reads:Regardless of the choice of weighting factors, the sums of confidenceintervals and appropriately designed worst-case estimations yielduncertainties being up to localize the true values of the LS-estimators [4,5].III. CONCLUSION Taking recourse to the legendary panel discussion of1971 by P. L. Bender et al.: Should least squares adjustments of thefundamental constants be abolished?, [3], it seems reasonable to suppose:For the tangly contradictions within the then bulk of measuring results not the


2012-07-19 of 91

15 of 95


method of least squares was to be blamed but rather the error modelcolleagues referred to. Starting from [1] – [2], I would like to propose anessentially new, self-contained draft of error calculus proposed to be termedGeneralized Gaussian Error Calculus, its intrinsic properties being to localizethe true values of measurands and thus to safeguard traceability. Thisshould be the core of the new GUM. REFERENCES [1] Eisenhart, C., “TheReliability of Measured Values – Part I Fundamental Concepts”,Photo¬grammetric Engi-neering, vol. 18, pp.543-561, 1952. [2] Grabe, M.,„Über die Fortpflanzung zufälliger und systematischer Fehler“, Seminar überdie Angabe der Messunsicherheit, 20. Und 21. Februar 1978; Physikalisch-Technische Bundesanstalt Braunschweig. [3] Bender, P.L. et al., „Shouldleast squares adjustments of the fundamental constants be abolished?”NBS Special Publications 343, 1971, United States Department ofCommerce, Washington D.C. [4] Grabe, M., “Measurement uncertainties inScience and Technology”, Springer-Verlag Berlin, Heidelberg 2005. [5]Grabe, M., “Generalized Gaussian error calculus“, Springer-Verlag Berlin,Heidel¬berg 2010.

5 For example, spending additional time to learn how to rigorously computeeffective degrees of freedom, if actually possible, does not seem to be worththe effort at this point.

Mar 22, 2012 5:56 PM

6 BS degree ME Mar 14, 2012 2:55 PM

7 Very useful, but might occasionally allow or include other approaches Mar 12, 2012 6:11 PM

8 entre B et C Mar 9, 2012 12:25 PM

9 Sorry, but I am not at all happy with the GUM. In my view it is terrible.Please, abolish the GUM.

Mar 6, 2012 7:19 PM


2012-07-19 of 91

16 of 95

Page 2, Q4. In which field do you use the GUM ?

1 As the most experience user of GUM within my laboratory organization (50scientiest), I have the supplemental responsibility for the training of othermeasurement scientist on the application of GUM. The AnalyticalLaboratories (within the Savannah River National Laboratory) uses GUMWorkbenchTM for measurement uncertainty calculations.

Jun 15, 2012 9:07 PM

2 Planning to conduct measurements, in general, in accordance with theconcepts of the GUM.

Jun 15, 2012 7:21 AM

3 1) accreditation of testing laboratories 2) development of standards 3)training courses for metrologists

Jun 13, 2012 1:02 PM

4 The GUM is used as an essential reference in development of documents inISO TC212 regarding metrological traceability.

Jun 12, 2012 2:58 PM

5 The teaching is on metrology and related matters. Jun 6, 2012 4:55 PM

6 I am having the course of Experimental statistics and mechanicalmeasurement and I am not able to understand completely what is going onabout the Lab. experiment of the course.

Apr 26, 2012 8:06 PM

7 Concisely written, excellent implementation Mar 12, 2012 6:11 PM

8 The GUM is an extremely important document to me; I deal with uncertaintyon a daily basis.

Mar 5, 2012 7:23 PM

9 Our laboratory is accredited following ISO 17025 for the calibration ofpressure and temperature devices and also for the analysis of referencegases.

Mar 5, 2012 11:47 AM

10 Industrial taken to include clinical lab services. Feb 23, 2012 9:32 AM

11 Our company has prepared an in-house procedure for estimation ofmeasurement uncertainty based on GUM

Feb 16, 2012 9:19 AM


2012-07-19 of 91

18 of 95

Page 3, Q1. a. Internal inconsistency: Conflicting (frequentist and Bayesian) views of probability, which causea number of problems, especially in the evaluation of a coverage interval.

1 Not a significant concern for my application of GUM. Jun 15, 2012 9:28 PM

2 There are inconsistencies when the contributions are not from normalprobabilitydistributions and which are assigned a high number of degrees offreedom. It must be said that both methods provide the tools important tocarry out the aplicattion that are to be carried or used

Jun 15, 2012 8:15 PM

3 True. It would be nice to see an explicit treatment of the Bayesian approach. Jun 15, 2012 1:27 PM

4 The current situation mixes frequentist techniques (in type A evaluation ofuncertainty) and Bayesian techniques (in type B evaluation of uncertainty).Hence, uncertainties determined by type A and type B evaluations havedifferent meanings, and, strictly speaking, cannot be combined. This requireschanges. One important example is the construction of expandeduncertainties, for which currently the Welch-Satterthwaite formula incombination with effective degrees of freedom is recommended. However, itis not appropriate to mix frequentist and Bayesian results. As aconsequence, for instance, it may even happen that expanded uncertaintiesbecome smaller if an additional, independent and additive uncertaintycomponent was added*. *M. Ballico, Limitations of the Welch-Satterthwaiteapproximation for measurement uncertainty calculations, Metrologia 2000,37, 61-64.

Jun 13, 2012 4:11 PM

5 I don't agree. The key idea enforced strictly throughout the GUM is that boththe measurement result (evaluation of the measurand) and evaluation ofinput quantities shall be only interpreted as a subjective judgment of theresearcher concerning their possible values. This excludes frequentistviewpoint fundamentally. While dealing with type-A evaluation, it is onlymentioned that data of observationes should be treated statistically. Thismay not be considered as a direct reference to the frequentist inferencebecause the Bayesian inference is also based on statistical analysis ofseries. The only arguable point in Gum is Clause G.3 of Annex G whosecontent coupled with the key GUM's idea mentioned above becomes a forceto suspect a fiducial inference (fiducial, not frequentist!).

Jun 13, 2012 3:06 PM

6 From a theoretical point of view it might be satisfactory to establishconsistency. On the practical level measurement uncertainty evaluation ismuch more than just using the “correct” statistical method. The mostimportant part is recognizing all essential influences and coming up with amodel of the measurement. Statistics is just a small part and in manysituations frequentist and Baysian techniques will lead to the same resultanyway. Mixing different statistical methods is not per se bad. The emphasisshould be on applicability, feasibility and simplicity. Consistency.should bemore of a second order goal.

Jun 12, 2012 4:44 PM

7 Not a major concern for the field of laboratory medicine. Jun 12, 2012 3:07 PM

8 As a user with only moderate statistical training, this distiction eludes me.The GUM fundamentally breaks uncertainty components down intoestimates of standard deviation and introduces this approach on the basis offrequentist statistics. Overall, one of the GUM's weaknesses is that itrequires the reader to have a very strong statistics background, so manyreaders find it a difficult document to understand and use. Thisinconsistency in fundamental probablistic viewpoints only adds to thedifficulty in using the standard.

Jun 8, 2012 3:25 PM

9 I am not so sure Jun 8, 2012 12:44 PM


2012-07-19 of 91

19 of 95


10 Agree - pick one method and stay with it. Or make two publication (or onepublication with two parts). One part of the publication should be suitable fora journeyman bench technician to use on the job. The other part can bee foracademic and engineering use. Methods and terminology suitable foracademic use and study will almost certainly be too complex for the benchtechnician to handle on a daily basis. They are looking for clarity, simplicityand speed.

Jun 7, 2012 3:31 PM

11 Interesting topics for rather "academic" discussions but no problems found inthe practice.

Jun 6, 2012 5:34 PM

12 The main conflict is between Type A and Type B where Type A evaluation isoften seen as frequentist since it is evaluated statistically. although this hassubsequently been addressed satisfactorily. With coverage interval the mainneed is NOT to consider it in terms of the probability that it contains an(unknowable) true value. It is an interval that contains a given percentage ofthe values ATTRIBUTABLE to the measurand. This interval can readily beobtained given the probability distribution of the values attributable to themeasurand as obtained from the evaluation of the uncertainty on the result.The lack of knowledge of this probability distribution is reflected in the lack ofknowledge about the coverage interval not its definition.

Jun 6, 2012 2:48 PM

13 Methodology to approach uncertainty or error (old fashsioned but intuitive)cannot be an obstacle to operative and quantitative evaluation.

Jun 6, 2012 2:33 PM

14 Although there is a philosophical inconsistency in the GUM most of the usersof the GUM are unaware of this and even if aware unconcerned. What theyrequire is a practical and pragmatic uniform guide to determine theuncertainty of measurement - the current version of the GUM provides thissolution an attempt to make the GUM more consistent by moving to for eg aBayesian approach will make the GUM less accessable to most users andtherefore diminish the importance and use of the GUM leading tofragmentation of uncertainty analysis.

May 28, 2012 8:39 PM

15 Though this is an issue, I would support keeping the document in thefrequentist perspective. If needed, a Bayesian approach could be generatedas a separate document.

May 16, 2012 10:33 PM

16 In most practical engineering applications such conflicts do not have adecisive impact

Apr 24, 2012 7:33 PM

17 I completely agree. The GUM's approach to the evaluation of theuncertainties is essentially frequentist and it should incorporate more of theBayesian methods.

Apr 24, 2012 2:42 PM

18 It seems both point Apr 18, 2012 8:44 AM

19 The evaluation of the coverage interval is not problematic for needs ofindustrial purposes (large uncertainties). The problem is in the interpretationof it.

Apr 11, 2012 10:01 PM

20 I think that Bayesian view is more consistent with metrological application,especially for definition of Type B uncertainty. Interpretation of credibilityinterval (instead of coverage interval) is easier and more natural for non-statistician scientist. However, prior elicitation is complicated and bayesiancalculations have some programming challenges.

Apr 11, 2012 1:57 PM


2012-07-19 of 91

20 of 95


21 An important element in fields of environmental analyses. Apr 6, 2012 3:58 PM

22 coverage interval idea is not part of mainstream GUM and it should berelegates as a not essential part of uncertainty in measurement

Apr 3, 2012 4:08 PM

23 no opinion Apr 2, 2012 2:31 AM

24 This is a very important issue, in my opinion. Bayes' theorem provides arigorous way of combining indications and expert opinion (the Type A vs.Type B distinction), yet it is hardly mentioned in the GUM, if at all. A soundBayesian methodology (such as the Observation Equation described byPossolo and Toman, Metrologia, 2007) is needed to take aleatoryuncertainty (or variability) in the measurement process of gatheringindications, and combine them rigorously with epistemic uncertainty fromexpert opinion to give a result of measurement as a probability distributionthat can be rigorously interpreted as an epistemic uncertainty in the value ofsome presumed "true parameter". This doesn't necessarily mean getting ridof measurement equations for some measurement problems, but one mustensure that all inputs to the measurement equation have a consistent(epistemic) interpretation before propagating this uncertainty. A Bayesianmethodology can be used to convert indications with aleatory uncertaintyfrom a measurement process into parameter values with purely epistemicuncertainty that may be propagated further in a measurement equation toproduce the desired measurand. Can a new GUM successfully address theassociated challenges with a Bayesian methodology?

Mar 22, 2012 6:31 PM

25 No opinion Mar 21, 2012 2:13 PM

26 Agree. Mar 20, 2012 2:55 PM

27 The uncertainty is a combination of real measurements and statisticalcalculation. but the coverage factor is based in the field of the theory. I thinkthat the Bayesian probability calculations are better in this field.

Mar 20, 2012 2:26 PM

28 I do not take a stand, because I do not analyse this problem in my works. Mar 14, 2012 7:09 PM

29 Probably Mar 14, 2012 12:45 PM

30 I partially agree Mar 13, 2012 5:49 PM

31 This is becoming more and more of an issue -- thank you for addressing this. Mar 13, 2012 12:08 AM

32 waste of time Mar 12, 2012 6:27 PM

33 might be improved Mar 12, 2012 6:11 PM

34 The document provides a number of views not on occasion there is no cleardirection for the user leaving them confused.

Mar 9, 2012 4:21 PM

35 consent Mar 9, 2012 4:19 PM

36 It is important to address this issue, and use a consistent Bayesianapproach.

Mar 8, 2012 7:01 PM

37 Consistency is fundamental. I would be in favor of an approach that showsthat the current calculation steps in the GUM form a good approximation of aconsistent theory, rather than changing the entire procedure to make it

Mar 8, 2012 9:45 AM


2012-07-19 of 91

21 of 95


consistent in itself.

38 Yes, this is true. Mar 6, 2012 7:21 PM

39 I am not a statistician. It does seem that the uncertainty using eitherfrequentist or bayesian statistics is roughly the same. GIven that NMIs havedifferences in uncertainty on nearly the same process (gage blocks, forexample) of 10% or more, the type of statistics doesn's seem important.

Mar 5, 2012 7:34 PM

40 Should be solved Mar 3, 2012 12:22 PM

41 Probability distribution shape assumption (rectangular, triangle, normal...) Mar 1, 2012 4:45 AM

42 Nice to know experts are looking at this sort of thing, but I don't think it willchange how I deal with it practically.

Feb 29, 2012 4:29 PM

43 Agree Feb 29, 2012 4:26 PM

44 I think the Bayesian view should be added on this revision. As the MonteCarlo method comes from a Bayesian concept, therefore the Monte Carlomethod should be emphasized in this new version.

Feb 28, 2012 6:14 PM

45 We need to be practical here. There was considerable resistance to thepublication of the original TAG4 document because it convolved actual andapproximated PDFs. Uncertainty is only good to 10 to 20 % of the numberquoted and we should keep that in mind while accepting that it is inconsistentand there are other ways to do things. For all that it has been remarkablysuccessful-just look at all accredited labs scope or at the KCDB.

Feb 27, 2012 6:20 AM

46 Including Bayesian probability points of view would be very valuable. Alsowhat to do if the Welch–Satterthwaite equation is not valid

Feb 23, 2012 11:54 PM

47 I agree that there are conflicting processes (i.e. frequentist and Bayesian) inthe present version. There were considered in detail when the GUM was firstbeing formulated and the rationale to the approach was described in theREPORT of the BIPM WORKING GROUP ON THE STATEMENT OFUNCERTAINTIES (1st meeting, October 1980); particularly in Sections 3and 4. In practice, the outcome (i.e., expressing both as standarduncertainties) generally works well, at least in reasonably straightforwardcases. It does add complications, such as the necessity to evaluate degreesof freedom in some cases and not in others. So a review of this aspect of theGUM is probably beneficial.

Feb 23, 2012 2:39 PM

48 Insufficient knowledge. Feb 23, 2012 9:39 AM

49 One challenge to bringing these two paradigms together is that in practicalapplications of the frequentist approach, effective degrees of freedom needto be estimated for B-type evaluations of uncertainty. This is rarely done.Instead, coverage factors of k=1 or k=2 are often used, bypassing anyprobabilistic interpretation of the results. Thus, the practice of the frequentistapproach would have to change.

Feb 22, 2012 3:19 PM

50 I have no such problem with the exisiting GUM. Feb 21, 2012 5:07 PM

51 I am not concerned (don't use coverage intervals) Feb 21, 2012 3:29 PM

52 Agreed. The GUM is not consistent with its founding document (the CIPM Feb 20, 2012 9:49 PM


2012-07-19 of 91

22 of 95


report) which was entirely frequentist.

53 yes for a basic use bayesian statistic are meaningless, people just need toget a idea of the uncertainty with a knowledge of a posible under of overestimate

Feb 16, 2012 2:32 PM

54 can't judge Feb 15, 2012 2:37 PM

55 This issue should be addressed indeed Feb 15, 2012 12:00 PM

56 I agree. Firsst of all internal inconsistency causes misleading in Type Auncertainty evaluation

Feb 15, 2012 8:04 AM

57 I think that the major point concerns the implicit reference to normaldistributions. I would prefer a document focusing on standard uncertainties,with a supplement for those interested in coverage intervals. To be morespecific, there is no need to suppose a statistical distribution of themeasurand, if the concept of uncertainty is expressed as the standarddeviation of an unknown distribution.

Feb 14, 2012 3:41 PM

58 agreed Feb 13, 2012 3:39 PM

59 gb Feb 13, 2012 2:56 PM

60 Not a principal difficulty, user must be aware of the difference between thesetreatments. Much more important is to test assumptions underlying ameasurement process, otherwise in both cases the estimates obtained maybe invalid. Present GUM is not dealing with this issue sufficiently(assumptions underlying Type A estimation for example).

Feb 7, 2012 2:42 PM

61 - Feb 7, 2012 2:08 PM

62 OK. Feb 6, 2012 10:15 PM

63 Agree Feb 6, 2012 5:31 PM


2012-07-19 of 91

24 of 95

Page 3, Q2. b. Internal inconsistency: Conflicting use of terminology, such as for the term “value”

1 This has not generated any practical problems in my application of GUM. Jun 15, 2012 9:28 PM

2 We think that the term "value" is not a cause of conflict Jun 15, 2012 8:15 PM

3 Not sure. I haven't noted any particular problems - maybe this comes fromtrying to apply the over-rigorous approach found in the VIM3?

Jun 15, 2012 1:27 PM

4 Terminolgy should be stated accurately. VIM3 can be osed for terminolgy Jun 15, 2012 1:17 PM

5 The conflict using different terminologies in all JCGM documents shall beavoided. Wherever possible, notation might be better aligned with standardstatistical terminology to facilitate potential dissemination and acceptance ofthe GUM beyond its current fields of application.

Jun 13, 2012 4:11 PM

6 I agree. Bearing in mind that the measurement result should be consideredas synonym of "posterior for random variable associated with themeasurand" such concepts as "measured value" and "best estimate of thevalue of the measurand" loose their meaning.

Jun 13, 2012 3:06 PM

7 The terminology should be clear and consistent. Jun 12, 2012 4:44 PM

8 This is an occasional concern for the the field of laboratory medicine. Jun 12, 2012 3:07 PM

9 I find that the standard is internally inconsistent in its treatment of systematicerrors. In paragraph 3.2.4, the expectation that all systematic effects havebeen corrected for limits the usefulness of the standard. While the idealsituation is that every estimate of measurement uncertainty will be taskspecific, and the systematic error for that task has been quantified, in realitysome estimates of uncertainty will apply to measurements in which thesystematic error can only be estimated by a maximum expected value, or bya range of possible values. In addition, the treatment of systematic effects inthe GUM is confusing and inconsistent. In 3.2.4, the expectation is thatsystematic effects have been quantified and corrected for, but Annex Epresents a discussion on the treatment of systematic versus random effects,and concludes that they are “treated in the same way in E.3.6.c. ParagraphE.4.4 and its associated example are also difficult to interpret. Appendix Fpresents yet another treatment of systematic effects in section F.2.4.5. Thistreatment in appendix F appears to have practical value, but it is hidden sodeeply within the standard that many users will never see it.

Jun 8, 2012 3:25 PM

10 I agree Jun 8, 2012 12:44 PM

11 Refer to and use the terms as defined in the VIM. Jun 7, 2012 3:31 PM

12 No problems found in the practice after the concept is clear. Jun 6, 2012 5:34 PM

13 Over zealous attempts to achieve internal consistency lead to multitudinousdefinitions of terms which conflict with the terms currently in use. The use ofthe term value does not cause a problem. .

Jun 6, 2012 2:48 PM

14 Ok. The effort in consistency for precise definitions make them sometimeslittle operative

Jun 6, 2012 2:33 PM

15 This should be made consistent with a frequentist view May 28, 2012 8:39 PM

16 Terminology must be cleaned up. May 16, 2012 10:33 PM

17 Agree May 16, 2012 1:34 PM


2012-07-19 of 91

25 of 95


18 Important, we all to use the same language May 16, 2012 9:20 AM

19 In most practical engineering applications the meaning of the terminology iscompletely understood by its context.

Apr 24, 2012 7:33 PM

20 Not even JCGM 100 (the Guide) and JCGM 104 (Introduction to the Guide)are consistent when using these terms.

Apr 24, 2012 2:42 PM

21 None. Apr 11, 2012 10:01 PM

22 Important because of the various uses of terminologies by different trainingthat staff received before acquiring work with the City.

Apr 6, 2012 3:58 PM


24 not a problem Mar 26, 2012 8:55 PM

25 What is the "best estimate" of a measurand, especially in nonlinearsituations? First, the mean of the output of a nonlinear measurementequation is generally not the value of the measurement equation at the meanof the inputs. The GUM should be rewritten so that this situation is notmisperceived when moving from linear(ized) measurement equations tononlinear ones. Second, why should the mean be chosen over the median ormode (if output is uni-modal)? In fact, is stating a single "best estimate" evenwell-advised if an x%–(100-x)% coverage interval is available (assuming thisinterval can be computed reliably)?

Mar 22, 2012 6:31 PM

26 No opinion Mar 21, 2012 2:13 PM

27 Agree. Mar 20, 2012 2:55 PM

28 No problems with this term Mar 20, 2012 2:26 PM

29 Yes, I agree with it. Mar 14, 2012 7:09 PM

30 True Mar 14, 2012 12:45 PM

31 agree Mar 13, 2012 5:49 PM

32 Yes, this needs to be cleaned up. As does the word "precision." Mar 13, 2012 12:08 AM


34 might be improved Mar 12, 2012 6:11 PM

35 This is a problem as different users will have their own vocabulary, in myindustry error is often deemed as an allowance under the overall uncertaintyi ahve to explain that error (bias) and uncertainty are not the same.

Mar 9, 2012 4:21 PM


37 Important Mar 8, 2012 7:01 PM

38 I'm still using the concept of true value and error as well. I would rather givethis concepts clear definitions than not to use them at all. The vocabularycan indeed be clarified.

Mar 8, 2012 9:45 AM

39 This seems to be unimportant on the practical level of estimating uncertainty. Mar 5, 2012 7:34 PM


2012-07-19 of 91

26 of 95


Uncertainty is not statistics.

40 More philosophical importance Mar 3, 2012 12:22 PM

41 Stnadard value, nominal value, and reading value Mar 1, 2012 4:45 AM

42 The GUM can be hard to read, and if this sort of thing can be sorted out, thiswould be good.

Feb 29, 2012 4:29 PM

43 Agree Feb 29, 2012 4:26 PM

44 Some of the internal problems are no doubt due to the compromises thatwere made to get the document published. Some of the philosophicalassumptions are critical if the document is to be useful at all.

Feb 27, 2012 6:20 AM

45 Make consistent with VIM3. Feb 23, 2012 11:54 PM

46 Yes, there are internal inconsistencies - another one is the ambiguity ofupper and lower case symbols relating to the measurand Y and its value y,which are sometimes used interchangeably.

Feb 23, 2012 2:39 PM

47 Insufficient knowledge - but consistency desirable. Feb 23, 2012 9:39 AM

48 Ok, maybe I am not such a GUM expert. I don't remember this problem. Feb 22, 2012 3:19 PM

49 I have no problem with the use of this term in the exisiting GUM. Feb 21, 2012 5:07 PM

50 Not so important in practice for my work Feb 21, 2012 3:29 PM

51 this has not been an obstacle Feb 20, 2012 9:49 PM

52 conflicting uses of terminology must in any case be removed, as this reallymakes it hard for "beginners" to understand, or even to look expressions up

Feb 16, 2012 3:40 PM

53 no Feb 16, 2012 2:32 PM

54 hardly confusing Feb 15, 2012 2:37 PM

55 Should be consistent with the VIM Feb 15, 2012 12:00 PM

56 I don't see it explicitly. Feb 15, 2012 8:04 AM

57 Not a problem for me. Feb 14, 2012 3:41 PM

58 did never realize Feb 13, 2012 3:39 PM

59 gfb Feb 13, 2012 2:56 PM

60 The particular use of terms must be clarified more flexibly according to theparticular situation and context. Very strict definitions of generally used termsmay cause additional difficulties and inconsistencies with the manystandards, recommendations and textbooks already in use and newlyappearing in parallel.

Feb 7, 2012 2:42 PM

61 - Feb 7, 2012 2:08 PM

62 What about VIM3? Feb 6, 2012 10:15 PM


2012-07-19 of 91

27 of 95


63 Agree Feb 6, 2012 5:31 PM


2012-07-19 of 91

29 of 95

Page 3, Q3. c. External inconsistency: With respect to Supplements 1 and 2 and the other documents beingdeveloped according to a consistent conceptual framework.

1 It is important that the collection of GUM standards be self-consistent.However, the current documents have not generated any practical problemsin my application of GUM to measurements performend at my laboratory.

Jun 15, 2012 9:28 PM

2 We do not have enough knowledge of the supplements 1 and 2 Jun 15, 2012 8:15 PM

3 The main GUM document does seem slightly outdated, and is not alwaysconsistent with the newer supplements

Jun 15, 2012 1:27 PM

4 Requires improvement. For instance, Supplement 1 to the GUM actuallygives different results for a type A uncertainty evaluation than the GUM. Thisdifference is not due to a violation of assumptions in the GUM (i.e. linearity ofmodel) but due to the fact that the GUM proposes a frequentist type Aevaluation of uncertainty while Supplement 1 carries out a Bayesian type Aevaluation of uncertainty. The current (frequentist) type A evaluation ofuncertainty yields smaller standard uncertainties than the Bayesianapproach. However, as the Bayesian approach better reflects practicalexperience, particularly in case of small sample sizes, this inconsistencyrequires a change in the Guide.

Jun 13, 2012 4:11 PM

5 I don't agree. GUM and Supplements 1 and 2 and other relevant guidelinesexist within the same conceptual framework based on the Bayesianapproach to measurements. (Even if the words "Bayes" and "Bayesian" arenot encountered in GUM, Bayesian approach is assumed there.)

Jun 13, 2012 3:06 PM

6 If possible the conceptual framework should be the same in the GUM in allits supplement documents.

Jun 12, 2012 4:44 PM

7 Not a concern in laboratory medicine. Jun 12, 2012 3:07 PM

8 The GUM makes a strong argument for avoiding the terminology "true value"of a measurand, but in JCGM 104:2009 Evaluation of Measurement Data –An Introduction to the “Guide to the Expression of Uncertainty ofMeasurement” and Related Documents, the term "true value" is introducedearly in the document, a statement is made that this conflicts with the GUM,but a further statement is made that the concept of a "true value" will be usedanyway. This is highly inconsistent.If a suplement can simply choose toignore guidelines provided in the GUM, can I simply choose to ignore otherparts ofthe GUM that I don't like?

Jun 8, 2012 3:25 PM

9 The consistency could be significantly improved Jun 8, 2012 12:44 PM

10 True, but this is hard to manage as each document is on a different timeline. Jun 7, 2012 3:31 PM

11 A minor one on the so called law of propagation of uncertainties that it is anapproximation to combine standard uncertainties in an easy way rather thana LAW. In my opinion the apporach should be highlighted as anapproximation, and hopefully let the denomination of LAW down.

Jun 6, 2012 5:34 PM

12 See comments in b Jun 6, 2012 2:48 PM

13 I am not sure who uses the supplements - I have not found any need toresort to them in my work in Temperature. But if asked to give an opinion thesupplements should be revised to make them frequentist in approach as thatis the approach that most metrologists understand and reflects physicalreality.

May 28, 2012 8:39 PM


2012-07-19 of 91

30 of 95


14 In my field (personal protective clothing standards, where we discuss aboutresutls from test on tensile strength or chemical resistance or heatresistance), results are given without any information on uncertainity. This isalso not a normative requirement from the standards and it is not somethingtest houses would give to the customers.

May 16, 2012 9:20 AM

15 I prefer to use the approach of the Supplement 1 in each of the applicationsto avoid such problems. Additionally, the traditional GUM can not be appliedin most measurement systems that tend to use due to the high nonlinearitythat characterices them.

Apr 24, 2012 7:33 PM

16 Yes, there is inconsistency. Supplement 1 is closer related to bayesianstatistics than the GUM. But it call my attention that in examples proposed inSupplement 1 input quantities have known parameters; there is notexplanation about how to evaluate the posterior pdfs of each input quantity(of a measurement model) taking on account the observations and a priorpdf and and how to combine all posteriors pdfs to get the posterior pdf fromthe measurement model (seehttp://www.cenam.mx/sm2010/info/pviernes/sm2010-vp03d.pdf)

Apr 11, 2012 10:01 PM

17 The concept is laudable. Apr 6, 2012 3:58 PM


19 Can the confusing and ad hoc effective degrees of freedom be gotten rid ofacross GUM and all supplements? Not familiar enough with S2 yet tocomment about this supplement.

Mar 22, 2012 6:31 PM

20 Do not know enough to express an opinion Mar 21, 2012 2:13 PM

21 No comments. Mar 20, 2012 2:55 PM

22 No problems with this Mar 20, 2012 2:26 PM


24 Not applicable to me Mar 14, 2012 12:45 PM

25 Till now I did not need to use the supplements Mar 13, 2012 5:49 PM



28 Of course. At the present moment it is difficult to explain to people that useuncertainty during their testing activities that GUM and the supplements havesome inconsistencies.

Mar 8, 2012 7:01 PM

29 Same as in a: try to show that simple, intuitive calculations (which may beinconsistent conceptually) form a good approximation of a consistent theory.

Mar 8, 2012 9:45 AM

30 Bayesian statistics seems to be intrinsically more complicated, even abtruse.If the different frameworks results in very different answers I would pay moreattention. In dimensional metrology the differences do not seem important.

Mar 5, 2012 7:34 PM

31 Sould be done Mar 3, 2012 12:22 PM


2012-07-19 of 91

31 of 95


32 Again, this is worth doing. Feb 29, 2012 4:29 PM

33 Do not know Feb 29, 2012 4:26 PM

34 Yes, I agree. All the documents and also Eurachem (for chemical area) haveto be harmonized.

Feb 28, 2012 6:14 PM

35 Yes Feb 27, 2012 6:20 AM

36 Good idea! Feb 23, 2012 11:54 PM


38 Important. Feb 22, 2012 3:19 PM

39 I encounter no problems with respect to the Supplements. Feb 21, 2012 5:07 PM

40 I can live with this inconsistency and apply the GUM or its supplementsdepending on the type of problem.

Feb 21, 2012 3:29 PM

41 The new supplements are absolutely shocking. They are less consistent thatthe GUM, give rise to serious definitional problems about uncertainties andprobability, and it is difficyult to see how they can work in practice.

Feb 20, 2012 9:49 PM

42 same remark as before complex concept are of interest but it must be warnabout their use

Feb 16, 2012 2:32 PM

43 can't judge Feb 15, 2012 2:37 PM

44 Agree Feb 15, 2012 12:00 PM

45 I share this opinion. Feb 15, 2012 8:04 AM


47 agreed Feb 13, 2012 3:39 PM

48 fgb Feb 13, 2012 2:56 PM

49 Conceptual contradictions inside the GUM and its supplements should beavoided.

Feb 7, 2012 2:42 PM

50 - Feb 7, 2012 2:08 PM

51 No opinion Feb 6, 2012 10:15 PM


2012-07-19 of 91

33 of 95

Page 3, Q4. d. External inconsistency: With respect to the VIM 3.

1 I think that it is important for GUM and VIM to mesh well and not haveconflict.

Jun 15, 2012 9:28 PM

2 There are not inconsistency that can affect the standard Jun 15, 2012 8:15 PM

3 I would hesitate to revise the GUM to be consistent with the VIM3, as theVIM3 seems to be a confusing and complex evolution of the VIM2

Jun 15, 2012 1:27 PM

4 VIM3 is the specific document for general metrology vocabulary. Therefore,GUM can be revised using VIM3 terminology.

Jun 15, 2012 1:17 PM

5 Agreement with all fundamental terms and concepts deemed as veryimportant.

Jun 15, 2012 7:31 AM

6 Yes Jun 14, 2012 4:20 PM

7 To avoid confusion caused by different terminologies in both JCGMdocuments according harmonization is required. Examples are the differentmeaning of "measurement result" or "bias" in both documents.

Jun 13, 2012 4:11 PM

8 The question seems confusing. GUM shall not be consistent with VIM. It isVIM that shall be consistent with GUM.

Jun 13, 2012 3:06 PM

9 VIM and GUM have to be consistent, especially when regarding suchimportant definitions as uncertainty and value. For example: The concept“true value” should be identically defined and interpreted in both documents.

Jun 12, 2012 4:44 PM

10 Inconsistency with VIM3 is problematic on occasion. Also, the term'measurand' can be especially challenging when considering complex orheterogeneous analytes in the field of laboratory medicine.

Jun 12, 2012 3:07 PM

11 None noted, but I did not actively look for inconsistencies. Jun 8, 2012 3:25 PM

12 The consistency could be significantly improved Jun 8, 2012 12:44 PM

13 Refer to and use the terms as defined in the VIM. Jun 7, 2012 3:31 PM

14 Please avoid non relevant changes in the VIM. Please allow flexibility toaccept clarifying and updating notes as needed, when, for example, changesin the VIM are required.

Jun 6, 2012 5:34 PM

15 The above comments apply even more to VIM 3. Also the comments madein (a) above where VIM jumps without any justification, from a reasonabledefinition of uncertainty, to one of confidence interval in terms of theprobability that it contains an unknowable true value..

Jun 6, 2012 2:48 PM

16 The VIM should be revised to be consistent. May 28, 2012 8:39 PM

17 Agree May 16, 2012 1:34 PM

18 I never heard of GUM and VIM in my life I am afraid. I did statistics at theuniversity and I know some basics. Nevertheless, it is extreely importateverybody uses the same terms, possible th same symbols.

May 16, 2012 9:20 AM

19 I believe that it is important to unify the terminology. Nevertheless it hasnever been a barrier to understand the concepts.

Apr 24, 2012 7:33 PM

20 None Apr 11, 2012 10:01 PM


2012-07-19 of 91

34 of 95


21 The definition of many metrological terms are not consistent with VIM3. Thisshould be updated as soon as possible.

Apr 2, 2012 2:31 AM

22 Cannot comment meaningfully on this. Mar 22, 2012 6:31 PM

23 OK Mar 21, 2012 2:13 PM


25 No problems with this Mar 20, 2012 2:26 PM

26 Yes, a lot of definitions were changed in VIM. Mar 14, 2012 7:09 PM

27 N/A Mar 14, 2012 12:45 PM

28 I did not notice Mar 13, 2012 5:49 PM



31 VIM 3 and GUM should have identical definitions (and so, should undergosimultaneous revisions)

Mar 8, 2012 7:01 PM

32 I hardly use the VIM. Mar 8, 2012 9:45 AM

33 It would be nice to be consistent with the VIM, but VIM 3 is considerablydifferent from VIM 2 so it is difficult to require consistency with an "living"document.

Mar 5, 2012 7:34 PM

34 Agree Mar 5, 2012 5:44 PM

35 Is a continuous process, would be nice Mar 3, 2012 12:22 PM

36 These should agree. Feb 29, 2012 4:29 PM


38 Yes but the GUM was not developed for non-physical measurements. ISO5725 was referred to for those. See my comments on the VIM.

Feb 27, 2012 6:20 AM

39 YES. Feb 23, 2012 11:54 PM

40 It is not necessary to express, in Section 2: Definitions- General metrologicalterms, all terms defined in VIM3, it is sufficient to refer to VIM3. Moreexamples in Annexes

Feb 23, 2012 11:12 PM

41 The document needs to be aligned to VIM3 Feb 23, 2012 5:00 PM

42 There do need to be some updates to address VIM3 terminology. Feb 23, 2012 2:39 PM


44 Good idea. The word "measurand" is of interest. It refers to the reportedquantity, rather than the directly-measured quantity. You and I couldcompare measurements of a measurand, say, the length of a rod. You maybe measuring distance using a meter stick, I may be measuring time using aclock, then converting the time to distance using a known speed, and

Feb 22, 2012 3:19 PM


2012-07-19 of 91

35 of 95


reporting that distance. We intend to measure the same quantity - the lengthof the rod. I find it useful to note that we were actually measuring things ofdifferent quality (you distance, me time). Is there a word for that quality thateach of us measures?

45 I have great difficulty with VIM3. It would be a massive step backwards forthe GUM to assimilate some of the flaws of VIM3.

Feb 21, 2012 5:07 PM

46 Don't know: did not read VIM3 yet. Feb 21, 2012 3:29 PM

47 agreed - but conflicts are not at all serious. Feb 20, 2012 9:49 PM

48 problems occur when looking at the definition of calibration in the VIM whichcannot bu used in any cases

Feb 16, 2012 2:32 PM

49 hardly confusing Feb 15, 2012 2:37 PM

50 Agree Feb 15, 2012 12:00 PM

51 Yes. But it requires few changes. Feb 15, 2012 8:04 AM


53 did not realize so far Feb 13, 2012 3:39 PM

54 fgb Feb 13, 2012 2:56 PM

55 VIM 3 is treating a significant part of general terms presently used in differentfields related to metrology (from pure mathematics like a quantity calculus tosimple practical measurements), and often these terms are helpful and usedwith a very good reason. Therefore one should be rather careful advisingthat the use of some terms in future preferably should be avoided.

Feb 7, 2012 2:42 PM

56 - Feb 7, 2012 2:08 PM

57 No opinion Feb 6, 2012 10:15 PM


2012-07-19 of 91

37 of 95

Page 3, Q5. e. Inadequacy: Since its publication, the need to evaluate measurement uncertainty has beenrecognised in an increasing number of scientific disciplines, for which the present GUM does not providesufficient guidance.

1 Not a serious issue for the destructive analytical methods used at my nuclearlaboratory. It is more challenging to apply GUM to nondestructive analysismethods, mostly because of calibration challenges.

Jun 15, 2012 9:28 PM

2 In our opinion the GUM provide the sufficient guidance (for our needs asaccreditation body)

Jun 15, 2012 8:15 PM

3 True. Certainly some wider examples would be a good idea. Jun 15, 2012 1:27 PM

4 Number of examples should be increased for new disciplines. For example,optics, acoustic, acceloremeter,mass calibration and chemistry, biologyeamples can be added to annexH

Jun 15, 2012 1:17 PM

5 Agree to this. A new version is needed. Jun 14, 2012 4:20 PM

6 Even within metrology the Guide has severe limitations. For instance, priorknowledge or physical constraints cannot be taken into account at present.Furthermore, the assumptions required by the Guide (linear models,Gaussian distributions) restrict its application significantly. For example, theknowledge that a physical quantity is positive can often reduce theuncertainty; however, the Guide is not capable of using such information. Arevision of the Guide should enable metrologists to apply accordingtechniques, which are readily available, for adequate uncertainty evaluation.

Jun 13, 2012 4:11 PM

7 I don't agree. The only strict limitation imposed by GUM is need for a model.Provided with a model, researcher can make good use of principlesdescribed in GUM and its Supplements for specific tasks and applications.

Jun 13, 2012 3:06 PM

8 This is true. The GUM should promote also simplified approaches and give(simple) guidance to treat multi-dimensional quantities, functional quantities(e.g. frequency dependent quantities), complex parameters (such asroughness, resulting from peak or average values from a filtered profile),chemistry etc.

Jun 12, 2012 4:44 PM

9 This is a major issue for the field of laboratory medicine. There is a need forspecific worked examples in the laboratory medicine field, but this could behandled as an Annex or Supplement to the VIM, since this should nottranslate to need for any refinements in the fundamental theory, concepts orcalculations of measurement uncertainties.

Jun 12, 2012 3:07 PM

10 the document needs to have varieties of practical examples applicable tovarious disciplines

Jun 12, 2012 2:20 PM

11 I find that the primary inadequacy of the GUM is that it is written at atechnical level that is too high for many users to understand it and use iteffectively. Overall, the standard is thorough in its documentation of aconsistent approach for estimating measurement uncertainty. However, aconsequence of its thoroughness is that it is cumbersome and not easilyunderstood by the typical user, which limits the standard’s adoption. Itmakes the topic of uncertainty estimation appear so complex, that manyusers will be discouraged from even attempting the process. A revision thatcan help address this is: in the Introduction, immediately direct readers toJCGM 104:2009 Evaluation of Measurement Data – An Introduction to the“Guide to the Expression of Uncertainty in Measurement and RelatedDocuments” and strongly encourage reading that document first; or, considermaking JCGM 104 the primary standard relating to measurement

Jun 8, 2012 3:25 PM


2012-07-19 of 91

38 of 95


uncertainty, and make the present GUM a supplement, like the existingdocuments JCGM 101, 102, etc. However, even JCGM 104:2009 will bedaunting for many readers – creating a new, even more basic “Overview”document would probably be of greatest benefit. In its present thorough butcumbersome form, users will rely on other documents that claim to follow theprincipals standardized in the GUM. This can easily result in variations inusers understanding of uncertainty principals and methods for uncertaintyevaluation – the very goal of a single standard.

12 I totally agree Jun 8, 2012 12:44 PM

13 The application of the GUM does require the knowledge of the VIM, and sothis should be stressed in the GUM with enough visibility. On the other hand,the hypothesis and assumptions to aplly the GUM should be the most explicitand guidance to other Supplements should be included as well when thesehypothesis are not fulfilled.

Jun 6, 2012 5:34 PM

14 Suitable guidance has been written for some areas. There is no problem ifone sticks to the basic principles of GUM. Problems can arise if criticsassume that the only ways to evaluate uncertainty is in accordance withthese principles are those illustrated in GUM

Jun 6, 2012 2:48 PM

15 The use of probabilistic distributions is a wide umbrella for many physicalphenomena, but eventually cannot cover those events that are not ordinarymodellized in statistical terms.

Jun 6, 2012 2:33 PM

16 Frequentist supplements should be developed to address these without awholesale revision of the GUM.

May 28, 2012 8:39 PM

17 The present GUM will have to provide additional detailed information onIonizing Radiation Metrology, more specifically to the evaluation of standardand expanded uncertainties. In the field of medical physics, measurement isat the heart of everything. Uncertainty evaluation is therefore a fudamentalskill for those working in this field.

May 20, 2012 11:49 AM

18 Agreed. A number of supporting publications have been required in manyeconomies.

May 17, 2012 6:22 AM

19 GUM should be generalized and not developed for a specific discipline. May 16, 2012 10:33 PM

20 Agree. Include non-physical measurements as well such as responses tosurveys

May 16, 2012 1:34 PM

21 I wish I would have known of GUM before. I am pretty sure somebody else inthe standardisation comittee know it and it did not bring it to the committeediscusion because he/she thought it would not help. I just had a brief look atthe document. It would be extremely helpful to give more examples and writea short smmary on which method to use according to the different type ofresults achieved. Additionaly, I would support the GUM comittee to presenttheir work to CEN/ISO technical committee, to enhance the use of GUM andVIM in the standardisation work.

May 16, 2012 9:20 AM

22 The application of the guide is complicated in many scientific measurements(uncertainty type B). The guide is more suitable for laboratory calibration.

May 3, 2012 11:15 AM

23 It is true, especially in the area of engineering and compliance testing. Apr 24, 2012 7:33 PM


2012-07-19 of 91

39 of 95


24 I use the GUM for the industrial instrumentation metrology and it works justfine.

Apr 24, 2012 2:42 PM

25 It's right. Analytical methods used in Chemistry need to apply a differentapproach, based on "design of experiments" and "variance components"methodologies. The GUM is based on "deterministic" models that aresuitable for physics but in chemistry there are much more variables that arenot easy to explain using a deterministic equation like F = m a. In chemistrystatistical models are much more useful.

Apr 11, 2012 10:01 PM

26 In chemistry is very common to measure two or more true replicate analysis(i.e subsampling from the same test portion). However, GUM does notprovide some guidance to handle repeated measurements. In someinstances we have applied other statistical tools to address this situation (i.e,variance components).

Apr 11, 2012 1:57 PM


28 Because of notations and concepts, lot of people I've met think that the Gumis not based on correct statistical rules. In the other hand, some statisticiansthink that such a document is quite useless, as the statistics rules aredefined for a long time in so many books ! Could the guidance given in theGUM focus on: "Applying GUM, it's just applying correct statistical rules, toestimate uncertainty" ? Examples should be taken in different fields, for basicmeasurements (not for "national calibration devices")

Mar 26, 2012 8:55 PM

29 There are engineering models that need to be calibrated and validated usingmeasurements, and it does not seem like the GUM is fully up to this task,mostly for reasons related to part a of this question.

Mar 22, 2012 6:31 PM


31 Fully agree, especially regarding the fields of biological measurements andsocial sciences.

Mar 20, 2012 2:55 PM

32 The true is the GUM is very oriented in the field of mechanical metrology andelectrical metrology, but the another areas lack of information about the howGUM his the true tool to use for the uncertainty calculations.

Mar 20, 2012 2:26 PM

33 agree Mar 20, 2012 5:18 AM

34 Yes, the scope of GUM (in context of different scientific disciplines) is toonarrow.

Mar 14, 2012 7:09 PM

35 Do not agree Mar 14, 2012 12:45 PM

36 The GUM is quite complicate. If I found the uncertainties rules elsewhere Idid not use GUM

Mar 13, 2012 5:49 PM

37 For the most part, I have always found it to be relevant and appropriate. Inthe context of using audits/repeatability to do retroactive quantification ofmeasurement error, I think things could be better standardized.

Mar 13, 2012 12:08 AM

38 thank God Mar 12, 2012 6:27 PM

39 no opinion Mar 9, 2012 4:19 PM


2012-07-19 of 91

40 of 95


40 Yes. I work with chemists, and they appreciate EURACHEM much more thanGUM, and they need to consider many other approaches. It is similar to thesituation that we had with VIM 2

Mar 8, 2012 7:01 PM

41 Procedures dealing with measurement uncertainty evaluation ofmicrobiologal enumeration methods should also be covered.

Mar 8, 2012 10:23 AM

42 Not so much in my experience. Mar 8, 2012 9:45 AM

43 Yes, this is true. Mar 6, 2012 7:21 PM

44 I do not deal with any fields that this is a problem. Mar 5, 2012 7:34 PM

45 Agree Mar 5, 2012 5:44 PM

46 adaption is useful Mar 3, 2012 12:22 PM

47 UM estimation example for different fields and applications. Mar 1, 2012 4:45 AM

48 There is a risk here. All disciplines really want a version of the GUM writtenfor themselves with all the examples from their own field. For instance inphotometry and radiometry (a long-term standard metrological discipline witha CC of its own) our uncertainties tend to be expressed as relativeuncertainties, our measurement equations are multiplicative and correlationswith wavelength are very important - it can be hard to "interpret" the GUM forthis situation. I suspect that for some other disciplines the step is evenharder. But the effort to write the GUM for each community separately ishuge. It probably needs collaboration between the GUM people andsomeone interested in that community.

Feb 29, 2012 4:29 PM

49 Agree Feb 29, 2012 4:26 PM

50 I agree. Feb 28, 2012 6:14 PM

51 True Feb 27, 2012 1:20 PM

52 It has but it remains to be seen whether the chemical and life sciences willcontinue to use it. There are increasing number of disciplines using the topdown approach to uncertainty rather than GUM and these have beenaccepted by accreditation bodies. In addition there are papers appearing injournals questioning the relevance of GUM to chemical testing in particularand emphasising the usefulness of PT and collaborative testing.

Feb 27, 2012 6:20 AM

53 More examples would be valuable. Feb 23, 2012 11:54 PM

54 I support this opinion and evaluation of measurement uncertainty for somespecific field of measurement and scientific disciplines

Feb 23, 2012 11:12 PM

55 True statement wrt increased profile. Feb 23, 2012 9:39 AM

56 Good point. Many of us are asked to teach or comment about GUM toscientists, students, engineers, executives. Much of that teaching involvesformulas. Most of those formulas are from the late 1800's. These factssupport your assertion that GUM is inadequate. The diverse readership isnot understanding the important concepts and practical points from theGUM. This may be due to their lack of mathematical felicity, which disrupts

Feb 22, 2012 3:19 PM


2012-07-19 of 91

41 of 95


their understanding of the practical points. That is, the reader is sooverwhelmed by partial differential equations, that he or she does not botherconsidering all the possible sources of uncertainty or error in ameasurement. That's a problem.

57 There is some truth in thois statement. My belief is that the GUM shouldremain unchanged, and additional Guidance developed for such situations9eg where there is insufficient knowledge of the system to build a reasonableuncertainty model).

Feb 21, 2012 5:07 PM

58 Good idea to enlarge the GUM readership Feb 21, 2012 3:29 PM

59 agreed. However, the main benefit of the GUM has been the commonlanguage. The classical methods underlying the GUM are obvious enough,and their extensions of multivariate and non-linear systems are obviousenough. Only serious weakeness is errors with asymetric distributions.

Feb 20, 2012 9:49 PM

60 I fully agree Feb 16, 2012 3:40 PM

61 basic knoledge of the calculation of uncertainty is of high interest but in somecases laboratories do not understand that there is no race for getting thelower uncertainty. this race in some case lead to large undertestimate of theuncertainty.

Feb 16, 2012 2:32 PM

62 As a representative from the industry I can say that we use the basicdirections from GUM. However, to make the procedures applicable toeveryday life we have set our own requirements for how to handle correlatedinput quantities and how to handle systematic effects. In a productionenvironment we can not always correct for identified systematic effect, butwe have to handle them in relation to the uncertainty statement for themeasurement result.

Feb 16, 2012 9:35 AM

63 currently GUM is not widly used in scientific aplications in Belarus thus poseslittle problem.

Feb 15, 2012 2:37 PM

64 This lack of guidance is what I regard as the biggest issue with respect to theapplicability of the GUM. As well as the lack of documented examples howthe GUM is applied, for example, to various analytical methods

Feb 15, 2012 12:00 PM

65 I don't think that it's a cause for revision of the GUM. This "inadequacy" canbe resolved by working out the supplements for uncertainty evaluation indisciplines which are far away from the conventional ones. At present GUMhas wide area of application.

Feb 15, 2012 8:04 AM

66 As the GUM gives a general framework, I do not see any problem. User inspecific fields can write their own application note. I do not think that theGUM can cover every fields.

Feb 14, 2012 3:41 PM

67 more practical examples in different scientific disciplines would be helpful Feb 13, 2012 3:39 PM

68 fgnb Feb 13, 2012 2:56 PM

69 Fully agree with this, particularly for testing and in forensics. Feb 8, 2012 7:58 PM

70 The present GUM is already quite complicated, its extension to cover allscientific disciplines is hardly possible, and not justified if no significant

Feb 7, 2012 2:42 PM


2012-07-19 of 91

42 of 95


changes of the present principal concepts are needed. Additionalsupplements likely are preferable.

71 - Feb 7, 2012 2:08 PM

72 yes, indeed the next item Feb 6, 2012 10:15 PM

73 Agree particularly the use of measurment performance data Feb 6, 2012 5:31 PM


2012-07-19 of 91

44 of 95

Page 3, Q6. f. Inadequacy: The present GUM is focused mainly on the situation of a physical quantity which“…can be characterized by an essentially unique value”. This has probably inhibited the wide use of the GUMin some scientific disciplines, such as chemistry and biology, in which a definition of the me...

1 My laboratory deals mainly with samples for which an essentially uniquievalue is desired.

Jun 15, 2012 9:28 PM

2 We think that in disciplines as chemistry and biology must have specificguidance and must be developed with EURACHEM or another organization

Jun 15, 2012 8:15 PM

3 Certainly merits consideration. The GUM should be as widely applicable aspossible, to encourage the adoption of sound metrological principles in allfields. However, the understandability and core framework should not becompromised by an attempt to make the document too general

Jun 15, 2012 1:27 PM

4 Eurachem/Citac, EDQM/OMCL guides can be used for these disciplines. Jun 15, 2012 1:17 PM

5 Agree. In these "new" fields the concept of present GUM in some cases isnot ok. .

Jun 14, 2012 4:20 PM

6 Not really a limitation. The actual limitation often rather is lack of aquantitative model relating the measurand to the relevant input quantities.

Jun 13, 2012 4:11 PM

7 I believe, chemists and biologists deal with physical quantities which "can becharacterized by essential unique values" too. The term "physical" ought tobe interpreted as characterizing a matter property rather than referring to aspecific scientific discipline (physics).

Jun 13, 2012 3:06 PM

8 This critique is true and efforts should be made to improve the GUM in thisrespect.

Jun 12, 2012 4:44 PM

9 The GUM is too complex for routine use by most chemists and biologists inthe field of laboratory medicine. Development of an Annex or Supplementwith real-world, fully worked examples would be a useful improvement thatwould aid in the adaptation of the GUM by practitioners in the LaboratoryMedicine field.

Jun 12, 2012 3:07 PM

10 None noted. Jun 8, 2012 3:25 PM

11 I would add the measurment of quantities which are function of time Jun 8, 2012 12:44 PM

12 Please see my comment e. Jun 6, 2012 5:34 PM

13 The claim of inadequacy is often an excuse for not wanting to evaluateuncertainty. However it can be justified if critics maintain that the only way toevaluate uncertainty is in accordance with the examples in GUM not with itspriciples. Uncertainty evaluation is now carried out widely in analyticalchemistry

Jun 6, 2012 2:48 PM

14 The increasing share of many disciplines of the complexity or chaosdynamics (deterministic but little predictable) can collision to the probabilisticuncertainty approach. In the other extreme industrial applications requiressimple straitforward quantifications.

Jun 6, 2012 2:33 PM

15 See answer to e) May 28, 2012 8:39 PM

16 Agreed. Application to qualitative testing requires more support. May 17, 2012 6:22 AM

17 Potential to generalize the definition of value. May 16, 2012 10:33 PM


2012-07-19 of 91

45 of 95


18 Agree, see above May 16, 2012 1:34 PM

19 For testing pourposes, I believe we often do not have the possibility toperform so many test to have a reasonable confidence on our data. How todeal with it without increasing cost?

May 16, 2012 9:20 AM

20 I use the GUM for the industrial instrumentation metrology and it works justfine.

Apr 24, 2012 2:42 PM

21 None. Apr 11, 2012 10:01 PM

22 The GUM is very high level and theoretical - it almost takes a laboratory viewon uncertainty. In real life in river flow measurement there can be manycontributors to uncertainty that the GUM does not address.

Apr 3, 2012 11:32 AM

23 This should be also solved by upgrading the GUM. Apr 2, 2012 2:31 AM

24 this point is very important ! Mar 26, 2012 8:55 PM

25 An interesting question: How does one use the GUM to characterize adistribution representing an aleatory phenomenon in science/engineering? Ifthe distribution is parametrized, then one can use GUM S2 to state the(possibly joint) uncertainty in these parameters. What if the distribution isnon-parametric, however? Does one try to measure a range of percentiles ofthe unknown distribution's CDF?

Mar 22, 2012 6:31 PM



28 agree Mar 20, 2012 5:18 AM


30 Possibly Mar 14, 2012 12:45 PM

31 I agree Mar 13, 2012 5:49 PM

32 Agreed, this is a challenge to encourage traditional "organismal ecologists"to adopt many of the practices contained in the GUM. Sample size is oftena challenge as the Central Limit Theorem can not often be invoked to forceGaussian Statistics.

Mar 13, 2012 12:08 AM

33 it is still useless Mar 12, 2012 6:27 PM


35 Yes. Mar 8, 2012 7:01 PM

36 This item causes some problems with respect to the expression of e.g."CFU or colony forming units", which is well known and widely-used inmicrobiology to express bacterial contamination degrees or "viable counts"

Mar 8, 2012 10:23 AM

37 No experience. Mar 8, 2012 9:45 AM

38 Unclear, in my view. Mar 6, 2012 7:21 PM


2012-07-19 of 91

46 of 95


39 I do not know of any measurement that is not related to a measurand. Mar 5, 2012 7:34 PM

40 Agree Mar 5, 2012 5:44 PM

41 Would increase practical usage Mar 3, 2012 12:22 PM

42 Yes, this is important. Also to account for examples where you can't write themeasurand as a function of the different effects but the solution is done, e.g.numerically.

Feb 29, 2012 4:29 PM

43 Agree, but care has to be taken that definitons and procedures do ntobacome to complicated, if the cases are basically simple. Perhaps a newGUM should be written for the cases mentioned above.

Feb 29, 2012 4:26 PM

44 I agree. In chemical area, the Eurachem Guide is more used. Feb 28, 2012 6:14 PM

45 At the moment, this is one of the largest shortcomings of the GUM Feb 27, 2012 1:20 PM

46 Even in physics the essentially unique value may be stretched. We aretalking about an estimate of a the variability in a measurement. Thatvariability in a well defined physical measurement may be ppm or less. Formany empirical measurements the variability can be 100 % even using thesame method. Definitions are not that relevant.

Feb 27, 2012 6:20 AM

47 Work with EUROCHEM and CITAM and IUPAC for their input. Feb 23, 2012 11:54 PM

48 Another possible area is in qualitative testing, where the result cannot beexpressed numerically but the uncertainties associated with the underlyingconditions still have to be evaluated.

Feb 23, 2012 2:39 PM

49 True, rarely primarily used in clinical sciences and I doubt most of myprofessional colleagues know [or care] of its existence. I do.

Feb 23, 2012 9:39 AM

50 Good point. Feb 22, 2012 3:19 PM

51 As mentioned above, I propose that the GUM should be left unchanged andadditional guidance (possibly in the form of Supplements) developed forthose areas where it is difficult to apply.

Feb 21, 2012 5:07 PM

52 Additional guidance for these applications would be welcomed. Problem ofinhomogeneity of the sample can also occur in physics.

Feb 21, 2012 3:29 PM

53 No problem. Feb 20, 2012 9:49 PM

54 yes, is it alweys necessary to give uncertainty when it is far more than 50%in such a case it is better to give a maximum deviation. the calculus of solarge uncertainty is meaningless

Feb 16, 2012 2:32 PM

55 The present GUM and the wording in several sections are unfamiliar for aseries of disciplines. A document with more examples from e.g. chemistryand biochemical analyses (where the exact definition of the measurand maybe a problem) will improve the use in these disciplines. At present, severalinterpretating guidance documents has been prepared to cope with this.

Feb 16, 2012 9:35 AM

56 not a problem. Why not to use EURACHEM Guide instead? Feb 15, 2012 2:37 PM


2012-07-19 of 91

47 of 95


57 "GUM can be impossible" not because of an "inadequate" definition of wordsbut because of a lack of guidance on how to do it in a case by case leadingby example(s) - pragmatic versus theoretical!

Feb 15, 2012 12:00 PM

58 Propably such disciplines as biology needs partiqular documents, e.g.concerning uncertainty evaluation for ordinal quantites.

Feb 15, 2012 8:04 AM

59 Not concerned by this point. Feb 14, 2012 3:41 PM

60 does not apply for my background Feb 13, 2012 3:39 PM

61 gfd Feb 13, 2012 2:56 PM

62 Fully agree. Clearly defining the measurand in these fields is difficult. Feb 8, 2012 7:58 PM

63 This quite specific problem obviously needs a special treatment for anyparticular case. It seems not to be only the problem of term "measurementuncertainty", the term "value" also needs to be clarified at the same time.

Feb 7, 2012 2:42 PM

64 - Feb 7, 2012 2:08 PM

65 OK Feb 6, 2012 10:15 PM


2012-07-19 of 91

49 of 95

Page 3, Q7. g. Ambiguities: Notational and terminological.

1 "Ambiguities: Notational and terminological." is too open ended aquestion/comment. I have no comment.

Jun 15, 2012 9:28 PM

2 We do not found Jun 15, 2012 8:15 PM

3 I'm sure there are some, but I've never noticed any practical problems. Jun 15, 2012 1:27 PM

4 see b) and d) above. Jun 13, 2012 4:11 PM

5 I do not see significant notational and/or terminological ambiguities. Jun 13, 2012 3:06 PM

6 no comment Jun 12, 2012 4:44 PM

7 Not a major issue for Laboratory Medicine. Jun 12, 2012 3:07 PM

8 None noted. Jun 8, 2012 3:25 PM

9 What is useful to the practical end user - which is different from the desiresof the academic.

Jun 7, 2012 3:31 PM

10 Please keep the notation as simple as possible, keeping in mind the out-of-NMI-metrologists.

Jun 6, 2012 5:34 PM

11 These will always be present and over zealous attempts to overcome themcan lead to an unreadable document

Jun 6, 2012 2:48 PM

12 For spread and general use of GUM: Error continues beeing intuitive, whileuncertainty in terms of standard deviation is harder to understand.

Jun 6, 2012 2:33 PM

13 Of course ambiguity should be avoided where possible. May 28, 2012 8:39 PM

14 Agree May 16, 2012 1:34 PM

15 As I do not know the VIM, I cannot make any comment May 16, 2012 9:20 AM

16 Different terms and concepts are messed up by the guide. May 3, 2012 11:15 AM

17 I recommend that the notation must be kept similar in order to made it easierto understand for the ones that already uses the GUM.

Apr 24, 2012 7:33 PM

18 None. Apr 11, 2012 10:01 PM


20 this point is important (see comments in e.) Mar 26, 2012 8:55 PM

21 Coverage vs. credible vs. confidence interval. Be very clear about alllinearity, normality, and other key assumptions.

Mar 22, 2012 6:31 PM


23 Agree. Mar 20, 2012 2:55 PM

24 Yes, I agree. Mar 14, 2012 7:09 PM

25 Not sure any are present Mar 14, 2012 12:45 PM


2012-07-19 of 91

50 of 95

Page 3, Q7. g. Ambiguities: Notational and terminological.

26 I agree Mar 13, 2012 5:49 PM



29 Yes Mar 8, 2012 7:01 PM

30 Not aware of. Mar 8, 2012 9:45 AM

31 Always worth getting right Feb 29, 2012 4:29 PM


33 ? Feb 23, 2012 11:54 PM

34 Insufficient knowledge - but clarity desirable. Feb 23, 2012 9:39 AM

35 This should be cleared up. Regarding this whole effort: We should considerthe audience and the mode of communication here. If this document is goingto be used on paper, by metrologists, then the current format may be fine.But if the new GUM will be interactive and electronic, and read by non-metrologists, then there may be better formats. For instance, on a website,mouse-over tips could define technical words or link to simple examples.Redundancy could be included. There is some redundancy in the originalGUM, and parenthetical notes. That informality is very helpful as it diversifiesthe modes of communication.

Feb 22, 2012 3:19 PM

36 I observe no problems here. Feb 21, 2012 5:07 PM

37 No comment Feb 21, 2012 3:29 PM

38 none. Feb 20, 2012 9:49 PM

39 no Feb 16, 2012 2:32 PM

40 not noticed. Some ambigueities due to translation doesn't influence generalunderstanding.

Feb 15, 2012 2:37 PM

41 ? Feb 15, 2012 12:00 PM

42 I don't see. Feb 15, 2012 8:04 AM

43 The type A and type B evaluation methods are frequently understood as typeA and type B uncertainties. I think that the GUM would be clearer withoutdistinction between the evaluation methods, which is mostly useless.

Feb 14, 2012 3:41 PM

44 as experimental physicists could not find such examples Feb 13, 2012 3:39 PM

45 ghnb Feb 13, 2012 2:56 PM

46 - Feb 7, 2012 2:08 PM

47 No OK Feb 6, 2012 10:15 PM


2012-07-19 of 91

52 of 95

Page 4, Q1. a. Clarity of presentation.

1 Good. Jun 15, 2012 9:34 PM

2 Very clear Jun 15, 2012 8:17 PM

3 Important. The document is pretty good at present, and clarity should not beallowed to suffer in the course of making the concepts more generallyapplicable (easy to say, but not always easy to achieve...)

Jun 15, 2012 1:31 PM

4 GUM should be understandable for all reader. Jun 15, 2012 1:19 PM

5 Good Jun 14, 2012 4:23 PM

6 Definitely important. The current Guide is not written in a didactical way, andits revision shall be improved in this respect. In addition, understanding ofideas would be highly facilitated by presenting the modern and coherentBayesian approach for uncertainty evaluation which already forms the basisof the Supplements 1 and 2 to the Guide.

Jun 13, 2012 4:11 PM

7 I agree, the essense of GUM's approch can be illustrated quite simply (see,for example, I. Szajniak, Decision Rule in a Conformity AssessmentProcedure under Measurement Uncertainty / International Seminar"Mathematics, statistics and computation to support measurement quality",St. Petersburg, June, 2012).

Jun 13, 2012 3:07 PM

8 Very important Jun 12, 2012 4:44 PM

9 Clarity of presentation could be improved by inclusion of relevant examplesin the field of laboratory medicine. Current presentation details are oftenobscured by lack of familiarity or prior knowledge of some of the technicalareas that form the basis for the examples or concepts being presented.

Jun 12, 2012 3:12 PM

10 The present GUM seems disorganized in places, and some concepts thatwould help a reader only become clearer in the appendices. Any changes toimprove clarity would be a welcome improvement.

Jun 8, 2012 3:25 PM

11 Good, some formal expressions about probability theory maybe difficult tohandle

Jun 8, 2012 12:48 PM

12 An absolute necessity, along with simplicity. Anything that aids ease ofunderstanding and use.

Jun 7, 2012 3:32 PM

13 The presentation is generally correct and clear for NMIs metrologists, but anumber of documehts have had to be produced to take the GUM applicationto the out-of-NMI-metrologists. The decision of who the GUM is aimed to hasto be taken, to keep the format as it is in a paper-like one, or changing it to aone that facilitates its understanding,

Jun 6, 2012 5:34 PM

14 Present version suffiiently clear Jun 6, 2012 2:48 PM

15 Not necessary, but the appendixes for more theoretycal explanations. Jun 6, 2012 2:36 PM

16 As not familar with the document can not comment Jun 1, 2012 11:29 AM

17 The GUM is already clear in its presentation. It is very suitable for teachingpurposes and can be grasped by laboratory technicians - this would not bethe case if a wholesale Bayesian revision was undertaken. This would putuncertainty analysis into the hands of mathematical experts which should beresisted - even if this means that some inconsistency has to be tolerated.

May 28, 2012 8:42 PM


2012-07-19 of 91

53 of 95


18 The presentation is more-or-less well structured. May 20, 2012 11:50 AM

19 Agree May 16, 2012 1:35 PM

20 I find the document clear. May 16, 2012 9:26 AM

21 The guide is complicated to apply with too Annex. The examples arenecessary to understand the text and application of the guide.

May 3, 2012 11:18 AM

22 This always has been a strong point in the GUM Apr 24, 2012 7:34 PM

23 The GUM is clear, maybe it could contain more examples along the text (notjust in the appendices).

Apr 24, 2012 2:44 PM

24 The clarity will be improved if the inconsistency between bayesian andfrequentist statistics is eliminated.

Apr 11, 2012 10:03 PM

25 GUM provides a clear description of uncertainty concept and calculations forpeople with at least a basic background in statistics and metrology. But, thesupplement 1 is more technical and for some collegues with anotherbackground (i.e, chemistry, biology) maybe it can be hard to implement(requieres some programming proficiency)

Apr 11, 2012 2:04 PM


27 Yes ! Mar 26, 2012 8:55 PM

28 Hopefully the presentation can be more succinct if the current hybridBayesian/frequentist version of the GUM is abandoned.

Mar 22, 2012 6:37 PM


30 Irrelevant. Mar 20, 2012 2:57 PM

31 as clear as possible Mar 16, 2012 5:33 PM

32 Present GUM is clear, I think. Mar 14, 2012 7:12 PM

33 Fairly good as is Mar 14, 2012 12:47 PM

34 It would be a good improvement Mar 13, 2012 5:52 PM

35 I think some of the "chapters" and appendices/annexes could be combinedso as to enhance readability.

Mar 13, 2012 12:09 AM

36 it is unnecesssary Mar 12, 2012 6:27 PM

37 very good Mar 12, 2012 6:11 PM

38 Need more examples for specific industries for example ISO 5167 for lfowmeasurement has an exmaple within it on uncertainty calculations.

Mar 9, 2012 4:22 PM


40 Yes. It is very important that, although derived documents for the differentsectors are provided by different organizations, GUM, as the commonsource, should be as clear as possible, while being comprehensive,unambiguous and deep enough.

Mar 8, 2012 7:05 PM


2012-07-19 of 91

54 of 95


41 Important. Maybe a simple cook-book part, and a more theoretical, formal,consistent part. Many people find the GUM difficult to read.

Mar 8, 2012 9:50 AM

42 I think the GUM is prettly well written, and is more practical than I expected itto be.

Mar 5, 2012 7:41 PM

43 Pay attention to basic knowledge necessary to understand concepts. Follw aprogressive logical construction on the concept development.

Mar 5, 2012 7:42 AM

44 Could be optimized, but is not bad in the actual form. The examples are veryhelpful. The first part is very theoretically. Difficult to transfer to other people,needs additional documentation.

Mar 3, 2012 12:34 PM

45 clarity is ok, but sometimes becomes too scientific for the common reader Feb 29, 2012 4:59 PM

46 This is the most important step. People need to feel they can use it as a tool,rather than admire it as a theoretical document for experts! The GUM isactually easier to read than its reputation suggests and I suspect some minorchanges would make a big difference.

Feb 29, 2012 4:32 PM

47 Good. Feb 29, 2012 4:27 PM

48 Like any aspect of physics or maths the concepts are not that simple. Ibelieve the existing document is clear enough.

Feb 27, 2012 6:23 AM

49 OK now. Feb 23, 2012 11:56 PM

50 reasonable clarity Feb 23, 2012 11:19 PM

51 Excellent Feb 23, 2012 9:39 AM

52 Necessary. The GUM should look familiar to 2015 audience, whoever thatintended audience is.

Feb 22, 2012 3:21 PM

53 The GUM is presented in the style of a textbook, rather than a users guide. Ihave no difficulty wth this.

Feb 21, 2012 5:08 PM

54 Essential Feb 21, 2012 3:51 PM

55 important Feb 16, 2012 3:41 PM

56 it is good enough in the present version Feb 16, 2012 2:34 PM

57 Goes without saying Feb 15, 2012 12:02 PM

58 I think that it would be achieved by avoiding comparison with "error concept". Feb 15, 2012 8:05 AM

59 presently o.k. Feb 13, 2012 3:45 PM

60 hg Feb 13, 2012 2:58 PM

61 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

56 of 95

Page 4, Q2. b. Structure as close as possible to that of the present GUM.

1 Good. Jun 15, 2012 9:34 PM

2 GUM has been well structured Jun 15, 2012 8:17 PM

3 Good Idea. The GUM has a large estabiliehd user base, and a revisedversion should try to be as compatible with the exisiting version as practical.

Jun 15, 2012 1:31 PM

4 There is no need to big alteration from present GUM Jun 15, 2012 1:19 PM

5 Structure should be changed if required for clarity or consistency. Jun 15, 2012 7:37 AM

6 In general the structure of the GUM is ok. Jun 14, 2012 4:23 PM

7 Desirable, but not mandatory. Jun 13, 2012 4:11 PM

8 No! No! No! This is in evident contradiction with the declared above principleof "clarity of presentation" (if "presentation" as presentation of a subject is inmind). A new document should combine GUM and its supplements into oneguidelines and be based on statistical ideas presented in the Supplement 1.

Jun 13, 2012 3:07 PM

9 Yes, this is very desirable. Jun 12, 2012 4:44 PM

10 Current structure is fine in principle. Sector specific examples could beincluded as a series of Annexes or Supplements.

Jun 12, 2012 3:12 PM

11 It is recommendable to have similar structure too the present version in orderto maintain user friendiliness to current users.

Jun 12, 2012 2:21 PM

12 Improved clarity should be the priority. I see no reason to strive to maintainthe structure.

Jun 8, 2012 3:25 PM

13 I agree Jun 8, 2012 12:48 PM

14 Not necessary - be flexible in the design. Maybe three sections: the basicprinciples and practices in plain language; then a set of worked examples foras many fields as practical; and then the in-depth mathematical andacademic discussion. The first two will make the guide much easier to use bythe people doing the daily work.

Jun 7, 2012 3:32 PM

15 Agree. Jun 6, 2012 5:34 PM

16 Yes Jun 6, 2012 2:48 PM


18 Yes of course May 28, 2012 8:42 PM

19 YES. The structure shoud be close to the present GUM. May 20, 2012 11:50 AM

20 Agree May 16, 2012 1:35 PM

21 The structure seem very good May 16, 2012 9:26 AM

22 The guide is complicated to apply with too Annex. The examples arenecessary to understand the text and application of the guide.

May 3, 2012 11:18 AM

23 Yes, I believe that it is important to keep it familiar to the readers. Apr 24, 2012 7:34 PM


2012-07-19 of 91

57 of 95


24 I agree. Apr 24, 2012 2:44 PM

25 Ok. Apr 11, 2012 10:03 PM


27 The structure should change, if it can be made more succinct and put on asound mathematical/statistical footing. Include a section on constructinguncertainty budgets for better understanding of sources of uncertainty andguidance on improving measurements.

Mar 22, 2012 6:37 PM


29 Very relevant, to ensure continuity of use. Mar 20, 2012 2:57 PM

30 similar to the present GUM Mar 16, 2012 5:33 PM

31 Yes, ok. Mar 14, 2012 7:12 PM

32 Suits me as it is Mar 14, 2012 12:47 PM

33 A continuity would be desirable Mar 13, 2012 5:52 PM

34 Would like to see the appendices folded into the chapters Mar 13, 2012 12:09 AM

35 waste of effort Mar 12, 2012 6:27 PM


37 Yes. A lot of effort has been invested in GUM diffusion, and the new editionshould be seen as a non-traumatic improvement.

Mar 8, 2012 7:05 PM

38 Actually, I apply more frequently the EURACHEM/CITAC Guide onQuantifying Uncertainty in Analytical Measurement than the GUM-Guide -The reason is that for my opinion GUM isn`t easy to understand for scientistsothers than mathematicans or statisticans. A more user friendly structure andpresentation made for practicioners would be useful.

Mar 8, 2012 10:23 AM

39 Not so important. Quality of the document is more important. Calculationsteps and final uncertainty estimates should be close to that of the presentGUM.

Mar 8, 2012 9:50 AM

40 Please no! Mar 6, 2012 7:22 PM

41 For my use I think it is fine. Mar 5, 2012 7:41 PM

42 Agree Mar 5, 2012 5:45 PM

43 Structure is ok. Mar 3, 2012 12:34 PM

44 current structure is good Feb 29, 2012 4:59 PM

45 It helps for older cross-references, but not at the expense of an improvementin clarity.

Feb 29, 2012 4:32 PM

46 Good Feb 29, 2012 4:27 PM

47 Yes Feb 27, 2012 6:23 AM


2012-07-19 of 91

58 of 95


48 Yes. Indicate changes in updaed GUM. Feb 23, 2012 11:56 PM

49 present structure is good Feb 23, 2012 11:19 PM

50 yes Feb 23, 2012 8:55 PM

51 Stay with a structure similar to the present version of the GUM Feb 23, 2012 5:01 PM

52 No concern here as long as a logical structure is used. Feb 23, 2012 2:40 PM

53 OK Feb 23, 2012 9:39 AM

54 Not necessary. Feb 22, 2012 3:21 PM

55 Would be useful for many readers Feb 21, 2012 3:51 PM

56 does not seem so important to me. Feb 16, 2012 3:41 PM


58 Not necessarily Feb 16, 2012 9:46 AM

59 Unless an evolution becomes a revolution? Feb 15, 2012 12:02 PM

60 It's important. Feb 15, 2012 8:05 AM

61 yes, but include e.g. examples in the main text Feb 13, 2012 3:45 PM

62 hgn Feb 13, 2012 2:58 PM

63 - Feb 7, 2012 2:09 PM

64 Agree Feb 6, 2012 5:32 PM


2012-07-19 of 91

60 of 95

Page 4, Q3. c. Level of presentation comparable to that of the present GUM.

1 Good, however more is not necessarily better. GUM is sometimes moredetailed on minor topics then typical and target users need. Concise is oftenmore helpful than addressing every possible detail.

Jun 15, 2012 9:34 PM

2 We think that the level of presentation has been well developed Jun 15, 2012 8:17 PM

3 The present level is probably about right. It needs to be supplemented bysome simple introductory guides for inexperienced users, or those new tometrology, but that is not a problem.

Jun 15, 2012 1:31 PM

4 Revised GUM shouldn't be too complicated to be able to comprehensible. Jun 15, 2012 1:19 PM

5 More expaining notes, examples and pictures could be used to give betterunderstanding to the contant.

Jun 14, 2012 4:23 PM

6 It is important that the whole framework is presented in a way that is easilyunderstandable. Technical issues should be referred to technicalSupplements as much as possible.

Jun 13, 2012 4:11 PM

7 Level of presentation in the general part should be similar. Details should beplaced in the appendices.

Jun 12, 2012 4:44 PM

8 Sector-specific examples, annexes or supplements should strive toincorporate terminology and concepts that are generally familiar to thepractitioners in the specific fields of interest.

Jun 12, 2012 3:12 PM

9 In some way, the GUM needs to be made more understandable by theaverage user. Within my company, the most common comment I havereceived about the GUM is that it is difficult to understand, contains too muchmath, and is too confusing and daunting overall.

Jun 8, 2012 3:25 PM

10 Ok but formal expressions must be explained, or simplified (or avoided?),also in order to allow a better dissemination to a wider audience

Jun 8, 2012 12:48 PM

11 Needs to be much easier to understand and use. Otherwise, it will not beused in daily work.

Jun 7, 2012 3:32 PM

12 Please see my comment a. Jun 6, 2012 5:34 PM

13 Yes Jun 6, 2012 2:48 PM


15 Yes May 28, 2012 8:42 PM

16 YES. May 20, 2012 11:50 AM

17 Agree May 16, 2012 1:35 PM

18 Fine May 16, 2012 9:26 AM

19 Yes, I believe that it is important to keep it familiar to the readers. Apr 24, 2012 7:34 PM

20 I agree. Apr 24, 2012 2:44 PM

21 Ok. Apr 11, 2012 10:03 PM



2012-07-19 of 91

61 of 95


23 Don't be afraid to discuss aleatory variability in a measurement process thatgenerates indications, vs. the epistemic uncertainty that results in ourestimates of underlying parameters we are trying to measure.

Mar 22, 2012 6:37 PM


25 Relevant. Mar 20, 2012 2:57 PM

26 possibly easier in the text, referring to the appendices for more details Mar 16, 2012 5:33 PM

27 Yes. Mar 14, 2012 7:12 PM

28 Yes, agree Mar 14, 2012 12:47 PM

29 Improve the presentation Mar 13, 2012 5:52 PM

30 waste of money Mar 12, 2012 6:27 PM


32 Yes. See answer to a) Mar 8, 2012 7:05 PM

33 see also comment under b, Mar 8, 2012 10:23 AM

34 No. See a: add a very simple, down to earth, possibly inconsistent partreadable for anybody.

Mar 8, 2012 9:50 AM

35 The current GUM can be usefull read ignoring the equations, and thus is avery usable document. I believe that there are adequate presentations of a"simple" GUM (M3003 for example) and a more technical document wouldbe much less useful to me.

Mar 5, 2012 7:41 PM

36 More examples Mar 5, 2012 7:42 AM

37 I would expect a more modern form of the theoretical part, with someexamples for better support of easy learning and transfer to praxis

Mar 3, 2012 12:34 PM

38 additional examples on presentation of uncertainty tables would be useful Feb 29, 2012 4:59 PM

39 I think this is fine, but that more examples are required. Feb 29, 2012 4:32 PM

40 Good Feb 29, 2012 4:27 PM

41 Yes Feb 27, 2012 6:23 AM

42 Yes. Feb 23, 2012 11:56 PM

43 More examples for different fields of measurement: chemistry, biology,medicine,...

Feb 23, 2012 11:19 PM

44 yes Feb 23, 2012 8:55 PM

45 OK Feb 23, 2012 9:39 AM

46 More diversity. Feb 22, 2012 3:21 PM

47 Agree Feb 21, 2012 3:51 PM


2012-07-19 of 91

62 of 95


48 yes Feb 16, 2012 3:41 PM


50 ? Feb 15, 2012 12:02 PM

51 yes Feb 15, 2012 8:05 AM

52 for further widespread use level could be lowered Feb 13, 2012 3:45 PM

53 hgn Feb 13, 2012 2:58 PM

54 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

64 of 95

Page 4, Q4. d. Better specification of the conditions of applicability.

1 Good. Jun 15, 2012 9:34 PM

2 We think that would be good having the largest number of examples Jun 15, 2012 8:17 PM

3 Not quite sure what this is driving at. Jun 15, 2012 1:31 PM

4 I hope this also include examples, picutes that can be used for the readers tobetter understand the purpose of the text..

Jun 14, 2012 4:23 PM

5 Definitely important. Jun 13, 2012 4:11 PM

6 The best specification should be GUM to be applied at all times and in allplaces.

Jun 13, 2012 3:07 PM

7 no comment Jun 12, 2012 4:44 PM

8 This should be addressed in the same fashion as discussed in Item C,above.

Jun 12, 2012 3:12 PM

9 The methodology in applying in various fields of science need to beelaborated accompanied with examples.

Jun 12, 2012 2:21 PM

10 I recommend reinforcing the fact that the GUM presents a statisticalapproach for evaluating measurement uncertainy, with the primary goal ofproviding a consistent evaluation approach that allows for a simple methodof combining uncertainties(e.g., adding variances). I would reinforce to theuser that the mathematical descriptions used, namely normal or normalizeddistributions of sources of variation, are a convenient way to apply math tothe physical phemonena, but do not necessarily represent the underlyinglaws by which the physical phenomena actually operate.

Jun 8, 2012 3:25 PM

11 I agree. Examples taken from industrial practice would be very appreciated.The industrial field should be generic and as wide possible, not primarilyrelated with metrology.

Jun 8, 2012 12:48 PM

12 Yes, but be careful not to make a situation where unscrupulous people canpoint at it and claim an exemption.

Jun 7, 2012 3:32 PM

13 Please see my comment e in the previous section. Jun 6, 2012 5:34 PM

14 Present specification adequate Jun 6, 2012 2:48 PM

15 Always. The aim of creating a standard of general application for alldisciplines can make it tasteless in many fields. The converse approach ofan operative document with application in many disciplines (anexes, forinstance) can spread it use.

Jun 6, 2012 2:36 PM


17 Yes - but only modest additions are required. May 28, 2012 8:42 PM

18 I would recommend sub-volumes or 'collateral standards' type of volumes toacoompany the basic GUM for the different scientific disciplines which usemetrology.

May 20, 2012 11:50 AM

19 Agree May 16, 2012 1:35 PM

20 Yes, this might help. May 16, 2012 9:26 AM


2012-07-19 of 91

65 of 95


21 The guide is difficult to apply to all measures, more designed for use incalibration and testing.

May 3, 2012 11:18 AM

22 Yes, in order to make it easy to select the right methodology to estimateuncertainty.

Apr 24, 2012 7:34 PM

23 Nice improvement. Apr 24, 2012 2:44 PM

24 This would be excellent for accreditation purposes. Accreditation bodiesshould be aware about the difficulties of this subject.

Apr 11, 2012 10:03 PM


26 Yes ! Mar 26, 2012 8:55 PM

27 Be clear about all assumptions. Mar 22, 2012 6:37 PM


29 Very relevant. Strongly reccommended to cover new knowledge fields. Mar 20, 2012 2:57 PM

30 agree Mar 20, 2012 5:19 AM

31 This part requires the maximum effort to extend the understanding andapplication of the standard

Mar 16, 2012 5:33 PM

32 Yes, should be better than on the present GUM. Mar 14, 2012 7:12 PM

33 Only where shorfalls are already identitifed Mar 14, 2012 12:47 PM

34 desirable Mar 13, 2012 5:52 PM

35 never used. Mar 12, 2012 6:27 PM


37 Yes. It needs to have a clear disctinction betwwen GUM and the "GUMframework", and the different conditions of applicability

Mar 8, 2012 7:05 PM

38 Yes, and make a summary page of the GUM and a flow scheme showingwhen to use which document.

Mar 8, 2012 9:50 AM

39 I think Chapter 3 which points out the usefulness of Measurement Assuranceand check standards in place of mathematical methods is good and shouldbe strengthened. ISO 17025, section 5.9.2 requires data that can be used inthis manner, and the data based uncertainty could use some formalizationthat might help accredited laboratories.

Mar 5, 2012 7:41 PM

40 Agree Mar 5, 2012 5:45 PM

41 Examples are quite formal, should include more practical relation Mar 3, 2012 12:34 PM

42 ?? Feb 29, 2012 4:32 PM

43 Good Feb 29, 2012 4:27 PM

44 Would be useful particularly if it is intended for other disciplines. There is nota clear understanding that the GUM requires systematic effects to be

Feb 27, 2012 6:23 AM


2012-07-19 of 91

66 of 95


removed wherever possible. That means traceability must be establishedand trueness effects removed.

45 Yes. Especially how to apply on the "shop floor" or in routine calibrations(like using a Multifunction Calibrator to measure a digital multimeter).

Feb 23, 2012 11:56 PM

46 Yes, there is a need for better specificatio of the conditions of applicability Feb 23, 2012 11:19 PM

47 yes Feb 23, 2012 8:55 PM

48 Good Feb 23, 2012 9:39 AM

49 More diversity of examples and notes. Stress the concepts and practicalnotes.

Feb 22, 2012 3:21 PM

50 Essential Feb 21, 2012 3:51 PM

51 I consider that useful Feb 16, 2012 3:41 PM

52 yes in some case see previous remarks Feb 16, 2012 2:34 PM

53 As a user of the document we can not always fulfill all preconditions for anuncertainty estimation. Therefore, it is preferred that the revised documentalso gives guidances for situations where e.g. it is not possible to correct fora systematic effect due to the design of the measurement process.

Feb 16, 2012 9:46 AM

54 Won't hurt, but when it comes to "applicability" what is needed is to lead byexample(s)

Feb 15, 2012 12:02 PM

55 I don't see what exactly can be improved here. Feb 15, 2012 8:05 AM

56 presently o.k. Feb 13, 2012 3:45 PM

57 hn Feb 13, 2012 2:58 PM

58 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

68 of 95

Page 5, Q1. a. Increased guidance in the evaluation of standard uncertainties associated with input estimates.

1 Lack expertise to comment. Jun 15, 2012 9:39 PM

2 Will be the only way to clarify all doubts Jun 15, 2012 8:18 PM

3 Good idea Jun 15, 2012 1:32 PM

4 Recommending to associate guidance with one or more examples. Jun 15, 2012 7:43 AM

5 Very good. Look at developed guidelines like the EA 4/02, M3003 and thecalibration guidelines from Euramet.

Jun 14, 2012 4:28 PM

6 Definitely important. For example, current Type A evaluation addresses onlythe Gaussian sampling distribution. But also available techniques for theelicitation of prior knowledge should be considered. Application of suchtechniques could yield smaller uncertainties by better utilizing the availableknowledge. Employment of Bayes theorem could allow to make coherentuse of new data in combination with available prior knowledge.

Jun 13, 2012 4:11 PM

7 I can't agree with the idea that law of propagation of uncertainty should bethe central concept of the revised GUM because that law is only a specialcase of propagation of distributions as stated in the Supplement 1.

Jun 13, 2012 3:38 PM

8 This would be helpeful Jun 12, 2012 4:45 PM

9 This may need to be addressed, again in Sector-specific documents, whereappropriate. For the field of Laboratory Medicine, additional guidanceregarding the estimation of standard uncertainties associated with inputquantities would be helpful.

Jun 12, 2012 3:15 PM

10 Such guidance will be helpful so as to avoid subjectivity in determining inputuncertainties

Jun 12, 2012 2:23 PM

11 If I understand the previous paragraph, it states that the GUM will switchfrom a frequentist to a Bayesian approach.This is a major and fundamentalchange, and will require a large amount of increased guidance. Thefrequentist approach, which should be familiar to a larger number of readers,is already difficult to understand. The new approach will only add to thisdifficulty. This sounds like it may be going in the wrong direction by makingthe GUM even more confusingto the average user. I understand that theBayerist approach may provide a better description of the physical world,butthe GUM has to focus on useability, not just mathematical rigor.

Jun 8, 2012 3:25 PM

12 I agree Jun 8, 2012 12:51 PM

13 Most practical end users don't know what a Bayesian approach is, and don'tcare. They just want to do their job well and quickly. Most users are lookingfor more of a "cookbook" presentation.

Jun 7, 2012 3:33 PM

14 Could help but perhaps this is best done in sector specific guides Jun 6, 2012 2:48 PM


16 no comment May 28, 2012 8:47 PM

17 YES May 20, 2012 11:52 AM

18 Agreed May 17, 2012 6:24 AM


2012-07-19 of 91

69 of 95


19 Agree May 16, 2012 1:36 PM

20 Yes, please May 16, 2012 9:27 AM

21 Ok. Apr 24, 2012 7:37 PM

22 It is necessary to explain how to evaluate the posterior pdf for each quantityand how to use that information fo the evaluation of the pdf of the outputquantity.

Apr 11, 2012 10:10 PM

23 Especially for repeated measurements. Apr 11, 2012 2:12 PM


25 Yes, with basic examples from various fields. Mar 26, 2012 8:59 PM

26 Be clear about the trade-off's between computational simplicity resulting fromlinearity/linearization and the potential loss of accuracy. How is one to knowwhether to linearize or to use Monte Carlo, because each has potential forloss of accuracy?

Mar 22, 2012 6:42 PM


28 Agree. Mar 20, 2012 2:58 PM

29 Maybe is good idea to develop a example for each magnitude and disciplineof metrology to be used has a guide.

Mar 20, 2012 2:28 PM

30 racommended Mar 16, 2012 5:36 PM

31 Yes Mar 14, 2012 7:15 PM

32 Do not try to simplify too much, will only allow unqualified people to use it inerror

Mar 14, 2012 12:48 PM

33 yes Mar 13, 2012 5:54 PM

34 This could be useful (but could also be confusing -- care must be taken withthe presentation).

Mar 13, 2012 12:10 AM


36 Yes definately Mar 9, 2012 4:23 PM


38 Good Mar 8, 2012 7:06 PM

39 Yes Mar 8, 2012 10:01 AM

40 Examples are absolutely the most important part of the GUM for many labsthat do not have high level scientists available for uncertainty analysis.

Mar 5, 2012 7:47 PM

41 important Mar 5, 2012 7:44 AM

42 This would be helpful Mar 3, 2012 12:39 PM

43 That's great. Mar 1, 2012 4:48 AM


2012-07-19 of 91

70 of 95


44 That will help - though often the "problem" for a beginner is a step earlier, inworking out what the input estimates are.

Feb 29, 2012 4:36 PM

45 Good Feb 29, 2012 4:27 PM

46 Good. Feb 28, 2012 6:20 PM

47 I would have thought there was enough given the guidance available fromaccreditation bodies and regional metrology organisations

Feb 27, 2012 6:26 AM

48 Not important. Feb 23, 2012 11:57 PM

49 Yes Feb 23, 2012 11:22 PM

50 Good idea Feb 23, 2012 8:56 PM

51 Useful. Feb 23, 2012 2:42 PM

52 good Feb 23, 2012 9:40 AM

53 Good idea. One caution is that following a method can lead tooverconfidence. For example, if a distribution is unknown and one makessome hand-waving guess that it is rectangular, and GUM gives officialguidance on how to propagate uncertainty from that distribution, one candevelop over-confidence in the result. On the other hand, if one asks "could itbe normal?" and answers "probably" and makes another estimate, one maybe surprised by the comparison of the final result. This overconfidence ismanifest in the many reports of 3-digit uncertainties in recent papers. Pre-GUM this did not happen. In fact, those uncertainties - by any interpretation -are not known to 3 digits. But the formality of following a procedure orworksheet with a bunch of formulas can give a false sense of confidence.Again, this comes from formula-grabbing, rather than following the guidanceon interpretation, intervals etc.

Feb 22, 2012 3:28 PM

54 This would be useful - but does not require revision of the whole GUM. Itwould be more convenient to develop additional supplements to address thisissue.

Feb 21, 2012 5:09 PM

55 Very useful Feb 21, 2012 3:51 PM

56 helpful Feb 16, 2012 3:42 PM

57 yes necessary Feb 16, 2012 2:35 PM

58 Great! Feb 15, 2012 12:03 PM

59 Just the question! Associated with input estimates or with input quantities? Feb 15, 2012 8:05 AM

60 This is of course a main issue but the actual GUM is for me very clear onconcepts. The evaluation of standard uncertainties on input estimates canonly be done by a person knowing the physics of the specific measurementprocess and the difficulties of the measurement. Subjective evaluation issometimes necessary but cannot be formalized in a guide.

Feb 14, 2012 3:50 PM

61 definitely required for the unexperienced reader Feb 13, 2012 3:48 PM

62 hgm Feb 13, 2012 3:00 PM


2012-07-19 of 91

73 of 95

Page 5, Q2. b. Bayesian approach extended to Type A evaluations of uncertainty.

1 Lack expertise to comment. Jun 15, 2012 9:39 PM

2 We think that will be hard to top as the Bayesian theory is based on thesubjetive interpretation of probability

Jun 15, 2012 8:18 PM


4 Required for coherent treatment of uncertainties, e.g. in order to combineuncertainties determined by type A and type B evaluations.

Jun 13, 2012 4:11 PM

5 O.K., even though this means to knock at an open door. Besides, I wouldrecommend eliminating the concepts of "type-A evaluation" and "type-Bevaluation" as confusing.

Jun 13, 2012 3:38 PM

6 Getting rid of degrees of freedom would make things simpler. However, themathematical formula is the smallest aspect of it. The most important is themotivation for this choice that may be understood by non-mathematicians.

Jun 12, 2012 4:45 PM

7 Not a major issues for Laboratory Medicine. Jun 12, 2012 3:15 PM

8 My undersdtanding of the application of Bayesian statistics is not strongenough to comment. However, a general comment on treatment of Type Auncertainties: In paragraph 4.2.1 – It is confusing to assume that Type Auncertainties are only associated with measurements for which the normalprocess consists of taking multiple observations and reporting the average.What about the case of using experimental data, such as from and R&Rstudy, and applying it to measurements in which the normal process consistsof taking a single observation as the measurement result? This is alluded tolater in the standard, for example, in paragraph 4.2.4 in which the pooledestimate of variance is introduced, but the treatment still focuses onprocesses that consist of taking multiple observations and reporting theaverage.

Jun 8, 2012 3:25 PM

9 I agree, with examples explaining the differences with a frequentisticapproach. I would avoid the presentation of these views as conflictive.

Jun 8, 2012 12:51 PM

10 If needed, please frame the discussions on the Bayessian approach todistinguish it in the GUM process to estimate uncertainties. Provideexamples where the differences between the frequentist and Bayessianapproaches lead to significantly different results.

Jun 6, 2012 5:34 PM

11 Yes Jun 6, 2012 2:48 PM


13 No Bayesian is too subjective and does not reflect classical measurementsituations - eg a normal distribution IS what you get when you do repeatedmeasurements. Rather type B evaluations should either be left as they are -or if possible made consistent with frequentist approach.

May 28, 2012 8:47 PM

14 Perhaps a draft for a Bayesian approach should be released and evaluatedby end-users,

May 20, 2012 11:52 AM

15 Agreed May 17, 2012 6:24 AM

16 Extension or exclusive approach? May 16, 2012 1:36 PM

17 ?? May 16, 2012 9:27 AM


2012-07-19 of 91

74 of 95


18 I'm not certain we should be focusing so heavily on Bayesian concepts. Arethey really accepted contractually between customers and suppliers?

May 15, 2012 10:24 PM

19 Ok. Apr 24, 2012 7:37 PM

20 Excellent! If the GUM contemplates this subject, it will become significantlybetter.

Apr 24, 2012 2:51 PM

21 It would be a great advance to provide some guidance about a soundBayesian approach to Type A evaluation of uncertainty. Maybe new GUMshould give some computational examples by using some pieces of opensource software (WinBugs, R o Phyton)

Apr 11, 2012 2:12 PM


23 Yes. Could we imagine to definitively avoid the distinction between "type A"and "Type B" ?

Mar 26, 2012 8:59 PM

24 This is a great idea. See my earlier comments about observation equationsvs. measurement equations.

Mar 22, 2012 6:42 PM


26 Agree. Mar 20, 2012 2:58 PM

27 Is good idea Mar 20, 2012 2:28 PM


29 Yes Mar 14, 2012 7:15 PM

30 Agree Mar 14, 2012 12:48 PM

31 yes Mar 13, 2012 5:54 PM

32 Will be excellent Mar 13, 2012 12:10 AM



35 Very important Mar 8, 2012 7:06 PM

36 This might risk to be to different from common practice in some cases andmost people don't change easily. Rather show that current practice is in mostcases good enough, i.e.< 20% off from consistent Bayesion type Aevaluation, and give guidance to Bayesian type A approach for informationand the cases where it is really needed.

Mar 8, 2012 10:01 AM

37 I don't see how this is overly helpful. Uncertainty is, in fact, uncertain. Forexample, the uncertainty in the standard deviation of 10 measurements isabout 25%. Most physical measurements do not have even 10measurements, so any change in uncertainty from the use of Bayesianmethods would need to be significantly larger than in the examples I haveseen.

Mar 5, 2012 7:47 PM

38 Good. Mar 1, 2012 4:48 AM


2012-07-19 of 91

75 of 95


39 I'm not completely sure what this means! It is useful to do lots ofmeasurements and take a standard deviation, is that not a Bayesianmethod? There needs to be guidance on the "degrees of freedom"associated with Type B evaluations - cf book by Kirkup and Frenkel, I'd neverrealised that they were anything other than infinity until recently.

Feb 29, 2012 4:36 PM

40 Good Feb 29, 2012 4:27 PM

41 Good. Feb 28, 2012 6:20 PM

42 Maybe. Are we going to add anything given my earlier comments? Feb 27, 2012 6:26 AM

43 Yes, this is the new area that needs to be covered. Feb 23, 2012 11:57 PM

44 Bayesian statistics is a difficult concept to understand. Its going to takesome explaining.

Feb 23, 2012 8:56 PM

45 Interesting; I'd like to see how this is to be done before commenting further. Feb 23, 2012 2:42 PM

46 OK Feb 23, 2012 9:40 AM

47 Yes. Feb 22, 2012 3:28 PM


Feb 21, 2012 5:09 PM

49 Detailed guidance and explanation of the concepts will be essential if youwant that users accept this change. In addition, it is important to know, in anuncertainty evaluation, whether the uncertainty of a given input quantity isbased on measurements, on prior knowledge or on an educated guess. Thisshould be part of the uncertainty report.

Feb 21, 2012 3:51 PM

50 Actually this is point of concern. The Type A approach adopted in S1 isobjective Bayesian whereas the Type B approach is subjective Bayesian.These two approaches are every bit as conflicted as Bayesian andfrequentis, indeed I would say the result is much worse.

Feb 20, 2012 9:51 PM

51 interesting but meaningless for the commom users Feb 16, 2012 2:35 PM

52 Great! Feb 15, 2012 12:03 PM

53 It 's the main point for revision of present GUM. Feb 15, 2012 8:05 AM

54 No need to complicate a generally simple task. Wo cares in the real life? Feb 14, 2012 3:50 PM

55 questionable Feb 13, 2012 3:48 PM

56 hgm Feb 13, 2012 3:00 PM

57 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

77 of 95

Page 5, Q3. c. Increased number of examples, with applications taken from biology, chemistry etc.

1 User friendly and practical examples are always good. Jun 15, 2012 9:39 PM

2 That will be expected. These exampels will be helpful Jun 15, 2012 8:18 PM


4 At the AnnexH, new examples from optics,acoustics,pressures, masscalibration and biology, chemistry should be added

Jun 15, 2012 1:23 PM

5 In favor. Jun 15, 2012 7:43 AM

6 This will make the use of measurement uncertainty in these areas muchmore easier and theefor better in use.

Jun 14, 2012 4:28 PM

7 Important and helpful. Fully worked out examples can be used as templatesolutions for related problems. It should, however, be considered to putthese examples into a supplementary document.

Jun 13, 2012 4:11 PM

8 Very good, these areas often still lack decent uncertainty evaluation. Jun 12, 2012 4:45 PM

9 Definitely a need in the field of Laboratory Medicine. As stated above, thiscan be addressed in section-specific annexes or supplements.

Jun 12, 2012 3:15 PM

10 More emphasis needs to be given in testing Jun 12, 2012 2:23 PM

11 Please do not limit the examples to biology and chemistry. Even the "easier"areas of dimensional metrology would benefit from increased examples.

Jun 8, 2012 3:25 PM

12 Ok. But above all an increased numebr of examples from engineering,specifically from mechanical engineering, biomedical engineering (sorry, butthis is my field of expertise).

Jun 8, 2012 12:51 PM

13 Yes, and also more examples from electronics, mechanics, dynamic, & soon. Especially valuable would be real-world guidance for practical MUcalculation for multi-function multi-range instruments.

Jun 7, 2012 3:33 PM

14 Agree Jun 6, 2012 5:34 PM

15 Could help but perhaps this is best done in sector specific guides Jun 6, 2012 2:48 PM

16 OK Jun 6, 2012 2:37 PM


18 examples are very important to understand the procedure how to count theuncertainty in different fields

May 30, 2012 12:52 PM

19 Not required. However maybe a GUM supplement should be written from afrequentist perspective that can address uncertainty analysis in these areas.The GUM isn't broken it works and it shouldn't be made over complicatedbecause of issues in particular disciplines - instead they should beaccomodated through supplements.

May 28, 2012 8:47 PM

20 Yes an increased number of examples would be good. Most ISO 17025laboratories will need to apply the GUM to their routine work.

May 20, 2012 11:52 AM

21 Fully supported May 17, 2012 6:24 AM


2012-07-19 of 91

78 of 95


22 Agree May 16, 2012 1:36 PM

23 Please, yes May 16, 2012 9:27 AM

24 Good! May 15, 2012 10:24 PM

25 In general, the examples are much needed to understand the guide. It wouldbe interesting to include examples of different disciplines.

May 3, 2012 11:23 AM

26 Examples from ionizing radiation metrology and dosimetry will may improvethe result of the revision

Apr 29, 2012 10:21 AM

27 And please, consider also to include engineering examples, such: antennameasurements

Apr 24, 2012 7:37 PM

28 As I have said, I use the GUM for industrial instrumentation application and itworks just fine. Since examples are the best way of learning, it would beinteresting to add some examples concerning the measurement offundamentals variables in industry (temperature, pressure, flow and level).

Apr 24, 2012 2:51 PM

29 In chemistry it woud be very useful to give some examples about calibration(least squares, bracketing, one-point calibration, etc.)

Apr 11, 2012 2:12 PM

30 It is always better to have more examples. Apr 2, 2012 2:35 AM

31 Yes. … and from "basics" measurements. Mar 26, 2012 8:59 PM

32 A standard set of examples, especially for verifying and benchmarkingcomputation software for GUM problems, would be very useful.

Mar 22, 2012 6:42 PM


34 Agree. Mar 20, 2012 2:58 PM

35 yes is good idea Mar 20, 2012 2:28 PM

36 agree Mar 20, 2012 5:19 AM


38 Yes Mar 14, 2012 7:15 PM

39 To help these topics, agree Mar 14, 2012 12:48 PM

40 it will be good Mar 13, 2012 5:54 PM

41 Would also be helpful. Mar 13, 2012 12:10 AM


43 other disciplines too Mar 9, 2012 4:23 PM


45 Important Mar 8, 2012 7:06 PM

46 That would be very useful and desirable. Mar 8, 2012 10:23 AM


2012-07-19 of 91

79 of 95


47 Yes Mar 8, 2012 10:01 AM

48 This is important. Laboratory accreditation is moving into new fieldsconstantly and the requirements for uncertainty cause significant anxiety andlead to costly consultants for uncertainty.

Mar 5, 2012 7:47 PM

49 Agree Mar 5, 2012 5:52 PM


51 Not only increased numbers but also more pratical content Mar 3, 2012 12:39 PM

52 That's a good idea. Mar 1, 2012 4:48 AM

53 It could be recommendable, because ilustrates the concepts given. Feb 29, 2012 4:49 PM

54 Yes. More examples the better. Ideally it would be written with examples inseparate documents - one from each of several disciplines, written incombination with experts from that field (for physics - experts from each CC,for biology/chemistry, the equivalent). A reader wants examples fromhis/her own field only!

Feb 29, 2012 4:36 PM

55 Perhaps Feb 29, 2012 4:27 PM

56 I agree. I work in chemical metrology and people needs examples relatedwith your area.

Feb 28, 2012 6:20 PM

57 Very important! Feb 27, 2012 1:21 PM

58 Unless there is a commitment from those disciplines to use it I feel this wouldbe wasted. The battle may be already lost.

Feb 27, 2012 6:26 AM

59 Yes. This is vey useful. Feb 23, 2012 11:57 PM

60 Yes, I support increasing of the number of examples Feb 23, 2012 11:22 PM

61 Yes Feb 23, 2012 8:56 PM

62 Good idea, although there are some good examples already in theEurachem guide.

Feb 23, 2012 2:42 PM

63 Excellent - and should include rationale of why applicable to clinical labtesting/measurement - which is poorly accepted in my circles.

Feb 23, 2012 9:40 AM

64 Yes. I find that having more, simple, examples is better than fewer,complicated ones. That way, the reader can understand the variouspossibilities and then combine them as needed.

Feb 22, 2012 3:28 PM


Feb 21, 2012 5:09 PM

66 In case of a large number of examples, a classification would be useful tofind out the example that we are looking for.

Feb 21, 2012 3:51 PM

67 OK. But lets see them done properly this time. Feb 20, 2012 9:51 PM

68 yes very interesting but see the previous remarks on the meaning of large Feb 16, 2012 2:35 PM


2012-07-19 of 91

80 of 95


uncertainty values

69 YES, please And with examples of practical use from the industry as well(not only in calibration but also in production process monitoringmeasurements.

Feb 16, 2012 9:48 AM

70 The more the better! Feb 15, 2012 12:03 PM

71 I am not sure that the GUM shoul be extended to include uncertaintyevaluation in biology.

Feb 15, 2012 8:05 AM

72 definitely yes, as stated already elsewhere above Feb 13, 2012 3:48 PM

73 hnhhn Feb 13, 2012 3:00 PM

74 Much needed examples of these types. Feb 8, 2012 7:59 PM

75 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

82 of 95

Page 5, Q4. d. Links to GUM Supplements where appropriate.

1 Good Jun 15, 2012 9:39 PM

2 Would be nice Jun 15, 2012 8:18 PM


4 It can be given links to 3 GUM Supplements to be able to provide integrity. Jun 15, 2012 1:23 PM

5 In favor. Jun 15, 2012 7:43 AM

6 The Guide should be much better aligned with the supplements. It should bethe central document, explaining the underlying framework and principles. Itssupplements should then address specific situations and technical issues.

Jun 13, 2012 4:11 PM

7 I don't agree. GUM should be revised together with its supplements. Jun 13, 2012 3:38 PM

8 yes Jun 12, 2012 4:45 PM

9 This would be a nice-to-have feature, but not essential. Jun 12, 2012 3:15 PM

10 it will be advantageous to have online expert forums and additionalresources on the subject matter.

Jun 12, 2012 2:23 PM

11 This would be useful. Jun 8, 2012 3:25 PM

12 Right Jun 8, 2012 12:51 PM

13 Yes. Jun 7, 2012 3:33 PM

14 Please take into account my comment e, section 3. Jun 6, 2012 5:34 PM

15 Yes Jun 6, 2012 2:48 PM

16 Always adecuated. Jun 6, 2012 2:37 PM


18 very useful May 30, 2012 12:52 PM

19 Not required. May 28, 2012 8:47 PM

20 YES May 20, 2012 11:52 AM

21 Agreed May 17, 2012 6:24 AM

22 Agree May 16, 2012 1:36 PM

23 Yes, it would be helpfull May 16, 2012 9:27 AM

24 These are necessary for the understanding of the guide. May 3, 2012 11:23 AM

25 Such as the Supplement 3 that is being prepared Apr 24, 2012 7:37 PM

26 Do you mean electronic links (as presented in the JCGM 104 supplement) ortextual links? Either way they are both great, but I have found the electroniclinks inserted in the 104 supplement really useful.

Apr 24, 2012 2:51 PM

27 Good idea. Apr 2, 2012 2:35 AM


2012-07-19 of 91

83 of 95


28 Yes. Mar 26, 2012 8:59 PM

29 This would be preferable, esp. if it can keep the respective documentsshorter.

Mar 22, 2012 6:42 PM


31 Agree. Mar 20, 2012 2:58 PM

32 yes Mar 20, 2012 2:28 PM


34 Yes Mar 14, 2012 7:15 PM

35 Agree Mar 14, 2012 12:48 PM


37 Always welcome Mar 13, 2012 12:10 AM


39 Yes Mar 9, 2012 4:23 PM


41 Very much needed Mar 8, 2012 7:06 PM

42 improves applicability Mar 8, 2012 10:23 AM

43 Yes. Mar 8, 2012 10:01 AM

44 Links to other documents related to the GUM would also be good. Mar 5, 2012 7:47 PM

45 Agree Mar 5, 2012 5:52 PM


47 Very nice Mar 3, 2012 12:39 PM

48 Good. Mar 1, 2012 4:48 AM

49 Yes Feb 29, 2012 4:36 PM

50 Good Feb 29, 2012 4:27 PM

51 Okay. This is an important subject. Feb 28, 2012 6:20 PM

52 Maybe although most people familiar with the GUM would know these exist. Feb 27, 2012 6:26 AM

53 Of course. Feb 23, 2012 11:57 PM

54 Yes Feb 23, 2012 11:22 PM

55 Good idea. Feb 23, 2012 2:42 PM

56 Good Feb 23, 2012 9:40 AM


2012-07-19 of 91

84 of 95


57 Past supplements? Let's make an entirely new guidance. It should becompletely open, it should be the first return from a google search of "GUM""uncertainty".

Feb 22, 2012 3:28 PM

58 Of course Feb 21, 2012 3:51 PM

59 very helpful Feb 16, 2012 3:42 PM

60 yes necessary Feb 16, 2012 2:35 PM

61 Great! Feb 15, 2012 12:03 PM

62 yes, it's useful Feb 15, 2012 8:05 AM

63 agreed Feb 13, 2012 3:48 PM

64 hn Feb 13, 2012 3:00 PM

65 - Feb 7, 2012 2:09 PM


2012-07-19 of 91

86 of 95

Page 6, Q1. What is your view on this timescale?

1 Acceptable, but I would also comment that at age 59 and 1/2, I am planningfor other activities if your schedule slide to much beyond the this window. :)

Jun 15, 2012 9:46 PM

2 We think that the timescale is prudent Jun 15, 2012 8:19 PM

3 Sounds reasonable. Jun 15, 2012 1:32 PM

4 This work needs to time. Therefore 2014 is logically date to final draft. Jun 15, 2012 1:24 PM

5 Realistic (although at first glance it seems distant). Jun 15, 2012 7:46 AM

6 To long but it is better that the new version is correct and approved by theusers before being published.

Jun 14, 2012 4:36 PM

7 A circulation of a final draft within 2013 is desirable. Jun 13, 2012 4:11 PM

8 Unrealistic. Jun 13, 2012 3:45 PM

9 Would be good, but sounds ambitious Jun 12, 2012 4:46 PM

10 While the stated timescale for revision to the GUM primary document isacceptable, the availability of worked examples in sector-specific fields suchas laboratory medicine would have the greatest impact on specializedsectors. For the field of laboratory medicine, it would also be very helpful if atimescale could be established for development of an appropriateSupplement, with availability scheduled in a similar timeframe (i.e. end2014).

Jun 12, 2012 3:19 PM

11 2014 seems long. Mid 2013 ias a reasonable project timescale. Jun 12, 2012 2:29 PM

12 This sounds aggressive and I commend the committee for setting thistimeline to have a draft available is a relatively short time.

Jun 8, 2012 3:26 PM

13 Good point Jun 8, 2012 12:55 PM

14 Aggressive but achievable Jun 7, 2012 3:34 PM

15 It seems fine. Jun 6, 2012 5:39 PM

16 Adequate Jun 6, 2012 2:49 PM

17 Probably adecuated if there are a medium consensus in the additions/modifsto the GUM

Jun 6, 2012 2:38 PM


19 I was hoping it would be earlier, like 2013. May 20, 2012 11:53 AM

20 Adequate May 17, 2012 6:24 AM

21 Very good!!! May 16, 2012 9:28 AM

22 The timescale is appropriate for the objectives. May 3, 2012 11:24 AM

23 This timescale is feasible Apr 29, 2012 10:22 AM

24 I believe it is fair, if the public dicussion of it starts at least 1 year after. Apr 24, 2012 7:38 PM


2012-07-19 of 91

87 of 95


25 It is a reasonable time to produce an excellent draft. Not that you haveasked, but I am really looking forward for to JCGM 103: Evaluation ofmeasurement data — Supplement 3 to the “Guide to the expression ofuncertainty in measurement” — Modelling.

Apr 24, 2012 2:55 PM

26 seems a bit long. Apr 23, 2012 8:04 AM

27 It´s fine. Apr 11, 2012 10:10 PM

28 It's a long timescale, but in my experience this is the usual timescale inmetrology.

Apr 11, 2012 2:19 PM

29 looks good Apr 5, 2012 6:08 PM


31 Ok for me. There is a lot of work to write the examples. Mar 26, 2012 9:01 PM

32 Does "everyone" agree to try settle on a Bayesian approach? Mar 22, 2012 6:44 PM


34 Reasonable timescale. Mar 20, 2012 3:00 PM

35 Is good to know that the first draw is near Mar 20, 2012 2:28 PM

36 agree Mar 20, 2012 5:20 AM

37 it is possible Mar 16, 2012 5:38 PM

38 I think is ok, but it is a long perspective. Mar 14, 2012 7:16 PM

39 Acceptable Mar 14, 2012 12:48 PM

40 it's a didfficult task Mar 13, 2012 5:55 PM

41 This sounds realistic. If an extra year is needed, I would find itunderstandable.

Mar 13, 2012 12:11 AM

42 waste of time and money Mar 12, 2012 6:28 PM


44 I am waiting for it Mar 8, 2012 7:06 PM

45 sounds reasonable Mar 8, 2012 10:03 AM

46 I have no problems with the date. Documents this basic to a field should notchange very much or very often.

Mar 5, 2012 7:50 PM

47 OK Mar 5, 2012 5:52 PM

48 it seems a fairly long time Mar 5, 2012 7:45 AM

49 ok Mar 3, 2012 12:39 PM

50 This timeline is too late, hope this may early to end of 2013. Mar 1, 2012 4:53 AM


2012-07-19 of 91

88 of 95


51 seems appropriate Feb 29, 2012 5:02 PM

52 About normal, but likely to be ambitious! Feb 29, 2012 4:36 PM

53 O.k. Feb 29, 2012 4:28 PM

54 I agree. Because discussions on the topics should be wellcome. Feb 28, 2012 6:24 PM

55 Acceptable Feb 27, 2012 3:12 PM

56 Realistic Feb 27, 2012 1:21 PM

57 No comment Feb 27, 2012 6:27 AM

58 2-3 years is OK. Feb 23, 2012 11:58 PM

59 This is acceptable Feb 23, 2012 11:24 PM

60 OK Feb 23, 2012 8:57 PM

61 Good goal for timescale Feb 23, 2012 5:02 PM

62 Seems reasonable. Feb 23, 2012 2:42 PM

63 Fine Feb 23, 2012 9:41 AM

64 That sounds ambitious, but I am not sure of the present state of affairs, or ofthe relevant politics.

Feb 22, 2012 3:30 PM

65 2014 seems extremely ambitious. This is further reason for my review thatthe focus should be additionla Supplements, not on a full revision at thisstage.

Feb 21, 2012 5:10 PM

66 There is no hurry. Better to take time and produce a better document Feb 21, 2012 3:51 PM

67 OK, probably tight given the serious issues you must address. Feb 20, 2012 9:59 PM

68 corect i can sork with the present version Feb 16, 2012 2:36 PM

69 OK Feb 16, 2012 9:49 AM

70 realistic Feb 15, 2012 2:43 PM

71 Takes time thus take your time Feb 15, 2012 12:04 PM

72 It's too optimistic, isn't it? Feb 15, 2012 8:06 AM

73 Too long. Feb 14, 2012 3:56 PM

74 ambitious timescale, proper preparation and discussion with interestedparties has priority

Feb 13, 2012 3:51 PM


2012-07-19 of 91

90 of 95

Page 6, Q2. This is the end of the survey. If you have any additional remarks, please provide them here:

1 This survey was very abstract compared to the VIM survey. I wish yousuccess in your efforts. I have great value for what has already beendeveloping in the current and past generations of GUM. My focus is more onimplementing at the working level and ensuring the customers understandwhat they are getting, including: regulators, legistators; managers,accountants, and other stakeholds. I believe that the ESARDA is workinghard to implement and educate the nuclear community on GUM and itsvalue.

Jun 15, 2012 9:46 PM

2 NO Jun 15, 2012 8:19 PM

3 A working group including people from the accreditation organizations. likeILAC, should also be involved in this updating work. It is very critical to makethis new version more understanding to commoncalibration/testing/chemistry laboratories around the world. If this work failsthe use of additional developed branch documents will continue.

Jun 14, 2012 4:36 PM

4 Filled out by Presidential Office of PTB after internal discussions. Statementsgiven here represent view of PTB.

Jun 13, 2012 4:11 PM

5 In my opinion, the way of revision aligned with the structure of GUM:1995looks erroneous.

Jun 13, 2012 3:45 PM

6 Please note that the responses in this survey have been provided by ametrology expert. Upon receipt of the ILAC call to members to participate inthe survey, SADCAS circulated the survey questionnaire to a number ofmetrologist and only one response was received. The information containedtherein represents the views of the expert and not necessarily SADCASviews.

Jun 12, 2012 2:29 PM

7 I commend the writers of the GUM for their efforts to create a document withtechnical rigor to provide a methematicalfoundation for evaluatingmeasurement uncertainty. I recommend that any revisions place the highestpriority on improving the understandability of the document, and place alower priority on further refining the technical aspects of the standard.

Jun 8, 2012 3:26 PM

8 Thank you very much for your efforts. Please remember that the key-note ofsuccess of all these documents is to be widespread all over the world, in allkind of human scientific and technical activities. As a university professor Iknow how is difficult to express a concept in easy-to-understand words, ittakes a long time to prepare a good lesson. Thank you again and keep upthis great work!

Jun 8, 2012 12:55 PM

9 A very common impression is that the GUM is written by academics andsenior professionals in various fields - but the people who are expected toactually understand the GUM are the much lower level technicians and juniorengineers who are actually doing the day to day work of meeting customerrequirements while complying with laws, regulations, quality andenvironmental management systems, measurement management systemsand so on. Is should always be written to be understandable to and usableby those end users.

Jun 7, 2012 3:34 PM

10 GUM has become crucial for metrology. It should be understood and appliedby ALL metrology users regardless of the metrological level of theorganization they are in, all of them are important. Please work on areference document for all of them without losing the correctness of itscontent. If that is not viable, take provision to produce interpretationdocuments of the reference document.

Jun 6, 2012 5:39 PM


2012-07-19 of 91

91 of 95


11 It is essential that the basic principles are not changed, the concept that theuncertainty gives information about the distribution of values attributable tothe measurand must be maintained. The use of the term "true value" mustbe avoided, it caused endless problems and much futile discussion beforethe clarification of the concept of uncertainty that was introduced by GUM

Jun 6, 2012 2:49 PM

12 Thank you Jun 6, 2012 2:38 PM

13 Standard such as this are not used in every day engineering practice- Jun 1, 2012 11:31 AM

14 I would stress again - the GUM as it is is a very successful tool. It hasfacilitated a uniform analysis of uncertainties around the world - preciselybecause it is relatively straightforward to use. This is a great gain and thisshould not be sacrificed to get increased consistency,

May 28, 2012 8:49 PM

15 Type B estimates are generally poorly developed and often do not matchwhat would have been found with Type A estimates for the samemeasurement. If the GUM focused primarily on Type A estimates it would bea much better (and more respected) guide.

May 21, 2012 1:01 PM

16 As already mentioned, I would support the application of GUM in ISO andCEN committee. Any movement in this direction, it will be much appreciated.

May 16, 2012 9:28 AM

17 Proposed to be included in a new GUM -- Generalized Gaussian ErrorCalculus Michael Grabe, Germany, 38104 Braunschweig, Am Hasselteich 5,[email protected] Abstract – Uncertainty assessments include thedifferences between the true values and the expected values of estimatorsas caused by unknown systematic errors which, on there part, are constantin time. To this end, the common practice to randomize systematic errorsshould be abolished. This proceeding and a new treatment of random errors,resolving a misinterpretation of old regarding the alleged interchangeability oftheoretical and empirical moments of second order, forges ahead what maybe addressed as "Generalized Gaussian Error Calculus". The formalismissues reliable uncertainties being up to localize the true values of physicalquantities. Index Terms – Generalized Gaussian error calculus, localizationof true values, measurement uncertainties. I. INTRODUCTION In view ofmodern metrology, the classical Gaussian error calculus is obsolete. Firstly,Gauss himself deliberately ignored so-called unknown systematic errors,these being errors constant in time and unknown with respect to magnitudeand sign. Without doubt, metrological traceability is a must. Hence, in myview, experimenters are asked to treat unknown systematic errors asbiases. At the same time, this way of proceeding would cause the classicalGaussian fomalism to break down – entailing, incidentally, the end of theanalysis of variance as well -- may we like it or not. Apart from theomission of biases in the past, error calculus still suffers from a sore pointscarcely ever addressed. To evaluate measurements, experimenters areused to adopting, without further ado, a formalism created bymathematicians. However, while mathematicians are in a position to freelypreset theoretical and empirical moments of second order (i.e. variances andcovariances), metrologists have nothing but empirical moments at theirdisposal and the common metrological practice to substitute empiricalmoments for theoretical ones is prone to spoil their formalism. Thus, thereis an abyss of old separating mathematical and metrological applications. Aswill be indicated, an appropriate metrological processing of empiricalmoments of second order presupposes the ostensibly trivial concept to referto equal numbers of repeated measurements. II. MAIN BODY As much aswe assume the laws of physics to be true, we expect the constants ofphysics to possess true values. In this sense, it should be the mission of

Apr 27, 2012 3:02 PM


2012-07-19 of 91

92 of 95


experimenters to localize the true values of measurands via properlyspecified uncertainty intervals. Unfortunately, neither the GUM's “1standard uncertainties” nor its “expanded uncer-tainties” are up to localizethe true values of measurands. An unknown systematic errors shifts thebulk of repeated measurements as a whole, thus introducing a permanent,time-constant difference between “the center of gravity” of the repeatedmeasurements and the true value of the quantity aimed at. Repeatedmeasurements take place not until this fixed difference has been establishedexperimentally, i.e. over the course of the setting up of the components ofthe measuring device. Obviously, this being done, there is no longeranything random about the aforesaid difference. But as traceability asks usto reckon with any possible value, I propose to resort to biassed estimatorsand to subsequently submit suchlike biases to worst case assessments.Indeed, this proceeding would map the basic working principle of stationarymeasuring devices. Textbooks on the statistical treatment of empirical dataare used to allocating ample space to Student’s distribution density, to theStudent-factor and, eventually, to confidence intervals. Remarkably enough,these discussions are regularly confined to just one measured quantity andno hint is given as to proceed in case of two or even more variables or howto handle possible dependences between measured quantities. Given themeasured data are to be considered normally distributed, or at leastapproximatetely so, the distribution density of the empirical moments ofsecond order reveals, amazingly enough, a depence between the empiricalvariances and the empircal covariances, be the variables themselvesdependent or not. Obviously, just this tells us not to ignore empiricalcovariances, even if the considered variables happen to be independent. Atthe same time we should strive for equal numbers of repeated easurements,as otherwise empircal covariances turn out to be undefined. After all, thecrucial point is that considering consistently empirical covariances, be theimplied variables dependent or not, we may generalize the idea ofconfidence intervalls according to Student. And this, indeed, will be wellworth the effort However, at present, in regard to error propagation, to askfor equal numbers of repeated measurements, seems to be outside ofcommon practice. If we are willing to comply with the properties of themultidimensional model of normally distributed variables, we are put in aposition to define confidence intervals in error propagation. This covers, inprinciple at least, arbitrary many variables up to the mechanisms of leastsquares.This perspective appears exciting in regard to the attempt to localizethe true values of measurands. The idea to ask for equal numbers ofrepeated measurements put experimenters in a position to shape theirformalim outside the common practice, namely to start with theoreticalmoments and to afterwards substitute empirical moments for the theoreticalones. As an example, let us consider a set of independent variables. Here,the theoretical covariances are set to zero, leaving over the theoreticalvariances. In hindsight, however, when metrologists insert their empiricalvariances, as they have nothing else at their disposal, the empiricalcovariances are tacitly left behind which, in fact, spoils the formalism. Afterall, equal numbers of repeated measurements, allowing to introducecomplete sets of empirical variances and covariances, establish a well-structured methodology of data evaluation. This proceeding wouldstipulate measurement uncertainties in terms of linear sums of confidenceintervals and appropriatley designed worst-case estimations. Should theunderlying error model apply, the associated uncertainty intervals wouldlocalize the true values of the measurands. With respect to the method ofleast squares, non-randomized systematic would errors abrogate the Gauss-Markoff theorem, which, as is known, specifies the weighting matrix. Asweighting factors shift the adjusted parameters and shrink their uncertainties


2012-07-19 of 91

93 of 95


the fundamental property of the error model discussed here reads:Regardless of the choice of weighting factors, the sums of confidenceintervals and appropriately designed worst-case estimations yielduncertainties being up to localize the true values of the LS-estimators [4,5].III. CONCLUSION Taking recourse to the legendary panel discussion of1971 by P. L. Bender et al.: Should least squares adjustments of thefundamental constants be abolished?, [3], it seems reasonable to suppose:For the tangly contradictions within the then bulk of measuring results not themethod of least squares was to be blamed but rather the error modelcolleagues referred to. Starting from [1] – [2], I would like to propose anessentially new, self-contained draft of error calculus proposed to be termedGeneralized Gaussian Error Calculus, its intrinsic properties being to localizethe true values of measurands and thus to safeguard traceability. Thisshould be the core of the new GUM. REFERENCES [1] Eisenhart, C., “TheReliability of Measured Values – Part I Fundamental Concepts”,Photo¬grammetric Engi-neering, vol. 18, pp.543-561, 1952. [2] Grabe, M.,„Über die Fortpflanzung zufälliger und systematischer Fehler“, Seminar überdie Angabe der Messunsicherheit, 20. Und 21. Februar 1978; Physikalisch-Technische Bundesanstalt Braunschweig. [3] Bender, P.L. et al., „Shouldleast squares adjustments of the fundamental constants be abolished?”NBS Special Publications 343, 1971, United States Department ofCommerce, Washington D.C. [4] Grabe, M., “Measurement uncertainties inScience and Technology”, Springer-Verlag Berlin, Heidelberg 2005. [5]Grabe, M., “Generalized Gaussian error calcu¬lus“, Springer-Verlag Berlin,Heidel¬berg 2010.

18 Thank you for your valuable work, Apr 24, 2012 7:38 PM

19 I don’t know how the process of commenting the JCGM’s draft works, but Iam at full disposal to comment on it as a simple user of the GUM and it’ssupplements (not as a metrology specialist).

Apr 24, 2012 2:55 PM

20 Annex H.1 has needs revision. In the corrent version: H.1.3.3 gives 2*10^-6/sqrt(3) for the uncertainty of the thermal expansion coefficient. H.1.3.5gives 1*10^-6/sqrt(3) for the uncertainty of the difference in thermalexpansion coefficients. However, these don't make sense because normallyit is expected that the uncertainty of difference of quantities haveingrectangular probility distribution will be a triangular distrubution with it halfwidth being 2 times teh half width of the rectangular distribution. Thus, itwould make sense if: H.1.3.3 gives 1*10^-6/sqrt(3) for the uncertainty of thethermal expansion coefficient, H.1.3.5 gives 2*10^-6/sqrt(6) for theuncertainty of the difference in thermal expansion coefficients.

Apr 2, 2012 2:45 AM

21 Thanks for asking my opinion! Mar 22, 2012 6:44 PM


23 Perhaps an electronic version (.html, with hyperlinks) could be produced.The same idea applies to the revised VIM.

Mar 20, 2012 3:00 PM

24 No comments Mar 20, 2012 2:28 PM

25 no. Mar 14, 2012 7:16 PM

26 Thank you for this survey Mar 14, 2012 12:48 PM

27 the whole process should be canned. it is a waste of time and money Mar 12, 2012 6:28 PM


2012-07-19 of 91

94 of 95


28 no remarks Mar 9, 2012 4:21 PM

29 Make sure that people will accept and use the document by extensive priorconsultation

Mar 8, 2012 10:03 AM

30 I think the GUM is a pretty good document. It is reasonably practical andcan be used by most metrologists at some level. ILAC has a number ofuncertainty examples, and perhaps the editors of the GUM might have awebsite that specialized in uncertainty examples that have been approved ascompetent.

Mar 5, 2012 7:50 PM

31 There is still some confusion/reluctance to follow the GUM and Type A andType B versus random and systematic. A supplement to the GUM that wouldbe useful would be a continuing discussion on why one should follow Type Aand Type B instead of random and systematic. If there were an exampleshowing the differences (pros and cons) it would be easier to explain toothers.

Feb 29, 2012 5:02 PM

32 Unfortunatly, my knowledge in the GUM is not as deep as needed in thissurvey.

Feb 29, 2012 4:49 PM

33 It is important not forget the goal: a practical GUM, accessible to all. Feb 28, 2012 6:24 PM

34 I am not convinced there is a need to change it. Feb 27, 2012 6:27 AM

35 Thanks for asking the metrology community for their input. It also keeps usinformed about the JCGM's activities and focus.

Feb 23, 2012 11:58 PM

36 No Feb 23, 2012 8:57 PM

37 Nope. Thanks for asking. Feb 23, 2012 9:41 AM

38 I look forward to the result. I'd be happy to be involved further, if that wouldhelp. This is an important project.

Feb 22, 2012 3:30 PM

39 Suggestion: produce a detailed document entitled "why to change the GUM"before the publication of the GUM revision, in order to prepare the public tothis event. Good luck

Feb 21, 2012 3:51 PM

40 Coinicdentally, as Chairman of CCT WG3 (Uncertainty in contactthermometry), I am preparing a working document summarising some of theissues associated with the change from Frequentist to Bayesian statistics -this should be of interest to you. I agree that there are some small problemswith the GUM, which are actually easily ironed out, but the Bayesianapproach represented by the S1 is a recipe for disaster. S1 has all the hallmarks of a forced religious conversion without any regard for theconsequences - and there are several serious consequences. If the GUM isto be redeveloped it is an absolute must that you have (i) expert practicingmetrologists represented on the committee, and (ii) statisicians that actuallyunderstand both philosophies fully. There is no place for do-good computerscientists who have no expert knowledge of either (as evident in S1).

Feb 20, 2012 9:59 PM

41 thank you for reading my remarks Feb 16, 2012 2:36 PM

42 How can we follow the revision? Will draft documents only be commented inthe authoring organisations?

Feb 16, 2012 9:49 AM

43 Thanks! Feb 15, 2012 12:04 PM


2012-07-19 of 91

95 of 95


44 good luck Feb 15, 2012 8:06 AM

45 I spent so much time to make people use the GUM that I would be very sorryif a big change is made to its philosophy. This is for me a reasonable andconsensual document, not perfect but very useful in practice, based on clearconcepts. Metrology and evaluation of uncertainties is too important to be leftto statisticians!

Feb 14, 2012 3:56 PM


2012-07-19 of 91