measurement with persons

MEASUREMENT WITH

PERSONS

Y116134.indb 1 10/10/11 2:11 PM

Y116134.indb 2 10/10/11 2:11 PM

Psychology Press

New York London

MEASUREMENT WITH

PERSONSTheory, Methods, and Implementation Areas

Edited by

BIRGITTA BERGLUNDGIOVANNI B. ROSSI

JAMES T. TOWNSENDLESLIE R. PENDRILL

Y116134.indb 3 10/10/11 2:11 PM

Psychology PressTaylor & Francis Group711 Third AvenueNew York, NY 10017

Psychology PressTaylor & Francis Group27 Church RoadHove, East Sussex BN3 2FA

© 2012 by Taylor and Francis Group, LLCPsychology Press is an imprint of Taylor & Francis Group, an Informa business

Printed in the United States of America on acid-free paper10 9 8 7 6 5 4 3 2 1

International Standard Book Number: 978-1-84872-939-1 (Hardback)

For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organiza-tion that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging‑in‑Publication Data

Measurements with persons : theory, methods, and implementation areas / editors, Birgitta Berglund ... [et al.].

p. cm. -- (Scientific psychology series)Includes bibliographical references and index.ISBN 978-1-84872-939-1 (hardback)1. Perception--Mathematical models. 2. Senses and sensation--Mathematical

models. 3. Human information processing--Mathematical models. I. Berglund, Birgitta. II. Title. III. Series.

BF311.M4325 2011153.7028’7--dc22 2011008495

Visit the Taylor & Francis Web site athttp://www.taylorandfrancis.com

and the Psychology Press Web site athttp://www.psypress.com

Y116134.indb 4 10/10/11 2:11 PM

on ne voit bien qu’avec le coeur

l’essentiel est invisible pour les yeux

Antoine de Saint-Exupéry, Le Petit Prince

Y116134.indb 5 10/10/11 2:11 PM

Y116134.indb 6 10/10/11 2:11 PM

vii

Contents

Preface ixContributors xi

1 Measurement across physical and behavioral sciences 1BirgittaBerglund,GiovanniBattistaRossi,andAndrewWallard

2 Measurement in psychology 27BirgittaBerglund

3 Measurements of physical parameters in sensory science 51TeresaGoodman

4 Meaningful and meaningless statements in epidemiology and public health 75FredS.Roberts

5 Toward a probabilistic theory of measurement 97GiovanniBattistaRossi

6 Multivariate measurements 125GerieW.A.M.vanderHeijdenandRagneEmardson

7 The prospects for measurement in infinite-dimensional psychological spaces: Modern notions for geometric person measurements in finite and infinite dimensional spaces 143James T. Townsend, Devin Burns, and Lei Pei

8 Psychophysical linguistics 175StephenLink

9 Mathematical foundations of Universal Fechnerian Scaling 185EhtibarN.Dzhafarov

Y116134.indb 7 10/10/11 2:11 PM

viii Contents

10 Neural networks and fuzzy systems 211ChristianEitzingerandWolfgangHeidl

11 Psychological measurement for sound description and evaluation 227PatrickSusini,GuillaumeLemaitre,andStephenMcAdams

12 Nociception and pain in thermal skin sensitivity 255DieterKleinböhl,RupertHölzl,andJörgTrojan

13 Measurement-related issues in the investigation of active vision 281BorisM.Velichkovsky,,FransCornelissen,Jan-MarkGeusebroek,Sven-ThomasGraupner,RiittaHari,JanBernardMarsman,SergeyA.Shevchik,andSebastianPannasch

14 Electrical and functional brain imaging 301PasqualeAnthonyDellaRosaandDanielaPerani

15 Body language: Embodied perception of emotion 335CharlotteB.A.Sinke,,MariskaE.Kret,andBeatricedeGelder,

16 Risk assessment and decision making 353LeslieR.Pendrill

Index 369

Y116134.indb 8 10/10/11 2:11 PM

ix

Preface

Measurementwithpersonsrefertomothersinwhichhumanperceptionandinter-pretation are used for measuring complex holistic quantities and qualities. Theseareperceivedorcreatedbythehumanbrainandmind.Providingmeansforrepro-ducible measurement of parameters such as pleasure and pain have importantimplicationsinevaluatingallkindsofproducts,services,andconditions.Progressin this area requires the interlinking of related developments across a variety ofdisciplines,embracingthephysical,biological,psychological,andsocialsciences.Moreover, itfacesanever-increasingdemandforvalidmeasurementsasthebasisfordecisionmaking.

Asignificanthurdletosurmountisthehistoricaldivisionthataroseinthemiddleof the twentiethcenturybetweenphysicistsandpsychologists.The twosidesdis-agreedstronglyonthemeaningofmeasurementandthepossibilityof“measuring”sensory events. This led to parallel developments in measurement science withinthetwoseparatecamps.Bothwentontogenerateremarkableresults,butthelackofcommunicationbetweenthempreventedcoherentandinteractiveprogress.

Thisbook inauguratesanewera for this subject.Herea largeboardof schol-arsandscientistsfromphysical,psychological,biological,andsocialscienceshaveacceptedthechallengeofworkingtogethertoreachacommonunderstandingofthetheoryofmeasurementandthemethods.Theaimistoprovide,seemingly,thefirstbookeverissuedcoveringthetopicofmeasurementwithpersonsbymulti-,inter-,and transdisciplinary approaches. That means that the complementary aspects ofgeneral theory,measurementmethods, instrumentation, andmodeling are treatedtogetheror integratedbyworld-renownedscientists in thefieldsofpsychophysicsandgeneralpsychology,measurementtheory,metrologyandinstrumentation,neu-rophysiology,engineering,biology,andchemistry.Suchacomprehensiveapproachisdevelopedingeneraltermsinthefirstpartofthebookandthendemonstratedinfrontierimplementationsinthesecondpart.

This, so far, unparalleled coordination effort has been possible thanks to theEuropeanFrameworkProgrammeSeven,CoordinationActionMINET:Measuringthe Impossible Network, chaired by Birgitta Berglund, which has provided theopportunity, and funding from the European Commission (Contract no. 043297).ThebookresultsfromanInternationalTrainingCourseforseniorscientistsheldinJune2008inGenova,Italy,andorganizedintheMINETenvironment.B.Berglund

Y116134.indb 9 10/10/11 2:11 PM

x Preface

andG.B.Rossichairedthecourse,anditwasattendedbyresearchersandadvancedstudentsfrom14differentcountries.

Thebookisorganizedintwoparts.Inthefirstpart,generictheoreticalandmeth-odologicalissuesaretreated,includingtheconceptualbasisofmeasurementinthevariousfieldsinvolved;thedevelopmentofformal,representational,andprobabilis-tictheories;theapproachtoexperimentation;andthetheories,models,andmethodsformultifacetedproblems.Inthesecondpart,severalimplementationareasarepre-sented,includingsound,visual,andskinperception;functionalbrainimaging;bodylanguageandemotions;and,finally,theuseofmeasurementindecisionmaking.

B. Berglund, G. B. Rossi, J. T. Townsend, and L. R. Pendrill

Y116134.indb 10 10/10/11 2:11 PM

xi

Contributors

Birgitta BerglundDepartmentofPsychology,Stockholm

UniversityandInstituteofEnvironmentalMedicine,

KarolinskaInstitutetStockholm,Sweden

Devin BurnsDepartmentofPsychologyIndianaUniversityBloomingtonBloomington,Indiana,USA

Frans CornelissenUniversityMedicalCentreGronigenGronigen,TheNetherlands

Beatrice de GelderFacultyofSocialandBehavioural

SciencesTilburgUniversityTilburg,TheNetherlands

Pasquale Anthony Della RosaDepartmentofNeuroscienceUniversitàVitaSaluteSanRaffaeleMilan,Italy

Ehtibar N. DzhafarovDepartmentofPsychologicalSciencesPurdueUniversityWestLafayette,Indiana,USA

Christian EitzingerProfactorGmbHSteyr-Gleink,Austria

Ragne EmardsonSPTechnicalResearchInstituteof

SwedenBorås,Sweden

Jan-Mark GeusebroekInstituteofInformaticsUniversityofAmsterdamAmsterdam,TheNetherlands

Teresa GoodmanNationalPhysicalLaboratoryTeddington,UnitedKingdom

Sven-Thomas GraupnerAppliedCognitiveResearchUnitDresdenUniversityofTechnologyDresden,Germany

Riitta HariBrainResearchUnit,LTLHelsinkiUniversityofTechnologyEspoo,Finland

Wolfgang HeidlProfactorGmbHSteyr-Gleink,Austria

Y116134.indb 11 10/10/11 2:11 PM

xii Contributors

Rupert HölzlOtto-SelzInstituteforApplied

PsychologyUniversityofMannheimMannheim,Germany

Dieter KleinböhlOtto-SelzInstituteforApplied

PsychologyUniversityofMannheimMannheim,Germany

Mariska E. KretFacultyofSocialandBehavioural


Guillaume LemaitreInstitutdeRechercheetdeCoordination

Acoustique/MusiqueParis,France

Stephen LinkDepartmentofPsychologyUniversityofCalifornia,SanDiegoLaJolla,California,USA

Jan Bernard MarsmanUniversityMedicalCentreGronigenGronigen,TheNetherlands

Stephen McAdamsCIRMMTSchulichSchoolofMusicMcGillUniversityMontréal,Québec,Canada

Sebastian PannaschDepartmentofPsychologyDresdenUniversityofTechnologyDresden,Germany

Lei PeiDepartmentofPsychologyIndianaUniversityBloomingtonBloomington,Indiana,USA

Leslie R. PendrillSPTechnicalResearchInstituteofSwedenBorås,Sweden

Daniela PeraniDepartmentofNeuroscienceUniversitàVitaSaluteSanRaffaeleMilan,Italy

Fred S. RobertsCenterforDiscreteMathematicsand

TheoreticalComputerScienceRutgersUniversityPiscataway,NewJersey,USA

Giovanni Battista RossiDIMECUniversitàdegliStudidiGenovaGenova,Italy

Sergey A. ShevchikInstituteofCognitiveStudiesKurchatovResearchCentreMoscow,Russia

Charlotte B.A. SinkeFacultyofSocialandBehavioural


Patrick SusiniInstitutdeRechercheetdeCoordination

Acoustique/MusiqueParis,France

James T. TownsendDepartmentofPsychologyIndianaUniversityBloomingtonBloomington,Indiana,USA

Y116134.indb 12 10/10/11 2:11 PM

Contributors xiii

Jörg TrojanCentralInstituteofMentalHealthDepartmentofCognitiveandClinical

NeuroscienceMannheim,Germany

Gerie van der HeijdenBiometrisWageningenUniversity&Research

CentreWageningen,TheNetherlands

Boris M. VelichkovskyDepartmentofPsychologyDresdenUniversityofTechnologyDresden,Germany

Andrew WallardBureauInternationaldesPoidset

MesuresSèvres,France

Y116134.indb 13 10/10/11 2:11 PM

Y116134.indb 14 10/10/11 2:11 PM

1

Measurement across physical and behavioral sciences

Birgitta Berglund,1 Giovanni Battista Rossi,2 and Andrew Wallard3

1DepartmentofPsychology,StockholmUniversityandInstituteofEnvironmentalMedicine,KarolinskaInstitutetStockholm,Sweden2DIMEC,UniversitàdegliStudidiGenovaGenova,Italy3BureauInternationaldesPoidsetMesuresSèvres,France

1.1 The origins

Althoughmeasurementhasbeenakeyfactorinthedevelopmentofmodernscience,studiesonitsfoundationsappearedrelativelylate,inthesecondhalfofthenineteenthcentury.Theyconcerned,at thesametime,bothphysicalandbehavioralsciencesandtheyparalleledtheconstitutionoftheinternationalsystemofmetrology,withthesigningoftheMetreConvention.Itisimportanttobeawareofsuchcommonrootsforunderstandingandputtingintherightperspectivewhathappenedlater,uptothepresentday.So,ahistoricaloverviewofthesecrucialnineteenthcenturydevelop-mentsispresentedinthefirstpartofthischapter,uptothedivisionthataroseamongthetwocommunities—physicistsandengineersontheoneside,psychologicalandbehavioralscientistson theother—in thefirstpartof the twentiethcentury.Withlackofcommunicationsuchdivisionledtoanessentiallyparalleldevelopmentonthetwosides.Nevertheless,noteworthydevelopmentsinmeasurementscienceandtechnology,aswellasrecentmeasurementneedsemerginginscienceandsociety,callforacommonefforttowardreachingacommonview,enablinginterdisciplinarycollaborationandensuringacommondevelopment.Thisisthesubjectofthesecondpartofthechapter.Lastly,inthethirdandfinalpart,newtrendsarepresentedanddiscussedandresearchneedsaddressed.

1

Y116134.indb 1 10/10/11 2:11 PM

2 Measurement with persons: Theory, methods, and implementation areas

1.2. In search of a theory for physical measurement

1.2.1 Helmholtz: The analogy between measuring and counting

Helmholtz, in a Memoire published in 1887 (Helmholtz, 1971), investigated “theobjectivemeaningofthefactthatweexpressasquantities,throughconcretenum-bers,situationsofrealobjects”andhewantedtodiscuss“underwhatcircumstancesweareallowed todoso.”“Concretenumbers,” inhis language,are thosearisingfromthecountingofrealobjects.Hefoundabrilliantsolution to theproblembyestablishingananalogybetweenmeasurementandcounting.

Thekeyideaisthat,inmanycases,thecharacteristicwewanttomeasureisaquantity,inthatitistheamountofsomething,andthusitmaybethoughtofasthesumofanumberofelementaryparts,orunits,ofthatsomething.Inthosecasesmea-surement is equivalent to the counting of such units.Fromthisanalogyitispossibletoderivetheconditionsthatmustbemetinorderformeasurementtomakesense,thatis,theconditionsformeasurability.Countingispossiblethankstothepropertiesofnaturalnumbers,whichundergoanorder,basedontherelation“greaterthanorequalto,”andmaybeaddedtoeachother.Similarly,measurementispossibleandwell foundedwhenever it ispossible to identify theempiricalcounterpartsof theorderrelationandoftheadditionoperationfortheobjectscarryingthecharacteristicofinterest.Forexample,inthecaseofmassmeasurement,ordermaybeestablishedbycomparingobjectsbyanequal-armsbalanceandadditionoftwoobjectsconsistsinputtingthemonthesamepanofthebalance.Thankstothesepropertiesitispos-sibletoconstructameasurementscale,whichsupportsthepracticeofmeasurement,aswesoonshow.Animportantquestionisnowwhetheritispossibletoestablishtheabovepropertiesforallkindsofmeasurement.Helmholtzadmitsitisnotandmen-tionsanindirectapproachasanapproach.ThisideawasdevelopedafterwardsbyCampbell,yieldingthedistinctionbetweenfundamentalandderivedquantities.

1.2.2 Campbell: The foundation of physical measurement

ThefirstorganicpresentationofatheoryforphysicalmeasurementwasbyCampbell,inthesecondpartofhisbook,Physics—The Elements,publishedin1920(Campbell,1957).LikeHelmholtz,heconsiderstheproblemof

… why can and do we measure some properties of bodies while we do notmeasureothers?…Ihavebeforemytable[hewrites]atraycontainingseveralsimilarcrystals.Thesecrystalspossessmanypropertiesamongwhichmaybeincludedthefollowing:Number,weight,density,hardness,colour,beauty.Thefirstthreeofthesequalitiesareundoubtedlycapableofmeasurement—unlessitbejudgedthenumberistobeexcludedasbeingmorefundamentalthananymeasurement;concerninghardnessitisdifficulttosaywhetherornotitcanbemeasured, for thoughvarioussystemsofmeasuringhardnessare incommonuse,itisgenerallyfeltthatnoneofthemarewhollysatisfactory.Colourcannotbemeasuredastheotherscan,thatistosayitisimpossibletodenotethecolour

Y116134.indb 2 10/10/11 2:11 PM

Measurement across physical and behavioral sciences 3

ofanobjectbyasinglenumberwhich can be determined with the same freedom from arbitrariness which characterises the assigning of a number to represent weight or density. The last property, beauty, can certainly not be measured,unlessweaccepttheviewwhichissowidelycurrentthatbeautyisdeterminedbythemarketvalue.What is the difference between the properties which deter-mine the possibility or impossibility of measuring them?(Campbell,1957)

Toanswerthisquestion,heconsiderstwokindsofquantities,

• Fundamental(e.g.,mass)• Derived(e.g.,density)

Bothoftheserequireanempiricalpropertyoforder,whichis(accordingtoHelmholtz)thebasicrequirementformeasurement.Butfundamentalquantitiesallowforaphys-ical-additionoperationalso.Whyisthisoperationsoimportant? Becauseitiskeyinpermittingthegeneralprocedureforfundamentalmeasurementtobeapplied.Suchaprocedureconsistsinconstructingameasurement scale,thatis,aseries of standardswithproperlyassignednumericalvalues,andthenincomparinganyunknownobjectr toit,inordertoselecttheelementintheseriesthatisequivalenttoit.Thenitwillbepossibletoassigntorthesamenumber(measure)astheselectedelement.

Let us see this in more detail, considering again the mass-measurement case.Fortheconstructionofthemeasurementscale,wefirstarbitrarilyselectoneobject,u,whichwillserveastheunitofthescale,andweassignthenumber1toit,thatis,m(u)=1,wheremisthemeasurefunction.Thenwelookforanotherelementu′,equivalenttou,suchthat,putintheoppositepanofthebalance,itwillbalanceit.Wenowsumthetwoelementsbyputtingthemonthesamepanofthebalanceandwelookforathirdelementthatbalanceswiththem.Clearly,wemayassignthenumber2.Sowehaveconstructedamultipleoftheunit,andwemayproceedsimilarlyfortheothermultiples.Submultiplesmayalsobeconstructedinasimilarway.Oncethescaleisavailable,massmeasurementmaybeperformedbycomparinganunknownobjectr,withtheelementsofthescale,withthebalance,uptofindingtheelementoftheseries,says,equivalenttoit:thenweassignm(r)=m(s).Theschemeofthedirect-measurementprocedurejustconsideredmaybedepictedasinFigure 1.1.

Noteworthy,intheprocessofconstructionofthescale,theonlyarbitrarychoiceconcernstheselectionoftheunitaryelement;afterwards,thevaluestobeassignedtotheotherelementsarefullyconstrainedbytheneedforconformitywiththeresults

Object

Quantity to bemeasured

Comparator

(directly)measuredquantity

Reference scale

Figure 1.1 Basicschemeforthedirectmeasurementofaquantity.

Y116134.indb 3 10/10/11 2:11 PM


ofthesummingoperation.Asaconsequenceofthis,themeasuremaybeinterpretedastheratiobetweenthevalueofthecharacteristicinobjectrandintheunitaryele-mentu.Inotherwords,m(r)=p/qimpliesthatthesumofq“copies”ofrbalanceswiththesumofpunitaryelements.Notethatqcopiesofr mayberealizedbyproperamplificationdevices,forexample,byusinganunequal-armbalance,witharmsinaratioq:1toeachother.

Now we may understand Campbell’s statement that only qualities “which canbedeterminedwith the same freedom fromarbitrarinesswhichcharacterises theassigningofanumbertorepresentweight”fullyqualifyasmeasurable,andwemayalsocomprehendtherationalebehindit.Whathasbeenconsideredsofarappliestofundamentalquantities,yetthereisanotherwayofmeasuringsomething,thewaythatappliestoderivedquantities.Considerthecaseofdensity,ρ.Forthisquantitywemayfindameaningfulcriterionfororder,becausewemaysaythataisdenserthanb, ifwemayfinda liquid inwhichbfloats,whereasa sinks,butwedonothaveanycriterionofempiricalsummation.Yetdensity“canbedeterminedwiththesamefreedomfromarbitrarinesswhichcharacterisestheassigningofanumbertorepresentweight,”becausewemayidentifydensityastheratioofmasstovolume:ρ=M/V,andwecanmeasuremassandvolume.So,givenanobjecta,assumingweareabletomeasureitsmass,obtainingmM(a),anditsvolume,obtainingmv(a),wemayassignameasuretoitsdensityasmρ(a)=mM(a)/mv(a).Theotherwaytofoundmeasurement—thewaythatappliestoderivedquantities—thusconsistsinfindingsomephysical law thatallowsexpressing themeasureof thecharacteristicofourinterest as a functionof themeasureofotherquantitieswhosemeasurabilityhasalreadybeenassessed.

Tosumup,Campbellholdsthatmeasurabilitymaybeestablishedfirstbyprovingthat thecharacteristicunderinvestigationinvolvesanempiricalorderrelationandtheneitherbyfindingaphysicaladditionoperationthatallowstheconstructionofareferencemeasurementscaleandtheperformingofmeasurementbycomparisonwithit,orbyfindingsomephysicallawthatallowsthemeasuretobeexpressedasafunctionofotherquantities.Thefirstprocedureappliestofundamentalquantities,thesecondtoderivedonesandisillustratedinFigure 1.2.

In the case of derived measurement the foundation of measurement is subjecttothephysical(moregenerallywecouldsay“natural”)lawthatisinvoked.Inthisregard,wemayconsidertwopossibilities:eitherit isalsopossibletomeasurethequantitydirectly,andsoweonlyhavetocheckwhetherthedirectandtheindirectapproachesproduceconsistentresults,orthelawhasanintrinsicvalidityand,ifthesamemagnitudeappearsintwoormorelawsacceptedinthesamescientificdomain,theconsistencyof thesystemmaybe invokedtosupport themeasurabilityof thequantityunderconsiderationandsoplaysafoundationalrole.

Object

Measuredquantity(ies)

Physical law

(indirectly)measuredquantity

Figure 1.2 Basicschemefortheindirectmeasurementofaquantity.

Y116134.indb 4 10/10/11 2:11 PM


In thefirstcase,derivedmeasurementsareessentiallyreduced tofundamentalonesanddonothaveanindependentfoundation.Inthesecond,quantitiesarecon-sideredasapartofasystemandthefocusisshiftedtowardtheperformanceoftheoverallsystemratherthanontheindividualproperties.Thisconceptofquantitiesasapartofasystemwasclearlyaffirmed,inaboutthesameperiod,withtheconstitu-tionoftheMetreConvention.

1.3 The constitution of the international system of metrology

Theneedforreferencestandardsformeasurementintrade,agriculture,andconstruc-tionhasbeen recognizedbymankind fromancient times.Metrological activitieshavebeendevelopedonaregionalbasis,followingtheevolutionofthegeopoliticalscenario,uptorelativelyrecenttimes.Onlyinthenineteenthcenturyhasinterna-tionalcoordinationbeenachieved,asaconsequenceofaprocessthatstartedneartheendofthepreviouscentury.AtthetimeoftheFrenchRevolution,thedecimal metric systemwasinstitutedandtwoplatinumstandardsrepresentingthemeterandthekilogramweredeposedintheArchives de la RépubliqueinParis(1799).Thisrationalizationofthesystemofunitsmayperhapshavebeenexplainedbythecon-currenceofafewfactors.Atthattimemodernsciencehadbeenfirmlyestablishedandtheneedofaccuratemeasurementsforitsdevelopmenthadbeenclearlyrecog-nized.PhilosophersoftheEnlightenmentwereinsearchofarationalfoundationofknowledge,whichhasanaturalcounterpart inscience in thesearchforuniversalreferencestandards,independentofplaceandtime.

ThisprocesscontinuedinthenineteenthcenturyandultimatelyledtotheMetreConvention,a treaty thatwassignedinParis in1875byrepresentativesofseven-teennations.TheConventioncreatedtheBureau International des Poids et Mesures(BIPM), the international referencebodyformetrology,andestablishedaperma-nentorganizationalstructureforcoordinatingmetrologyactivities.Theorganizationincludes the International Committee for Weights and Measures (CIPM), a bodysupported by a number (currently 10) of technical committees for providing rec-ommendationsforthedevelopmentofthedifferentfieldsofmetrology.TheCIPMreportstotheGeneralConferenceonWeightsandMeasures(CGPM),ameetingofnationaldelegationsfromthememberstatesoftheBIPMthathaveaccededtotheMetreConvention.TheCGPMmakesthenecessarydecisionsfortheoperationoftheworldmetrologysystemand,inparticulartheInternationalSystemofUnits(SI).TheworkoftheBIPMprovidesthebasisforasinglecoherentsystemofmeasure-ments throughout the world, acting in many forms, from direct dissemination ofunits to coordination through international comparisonsof national measurementstandards.TheConvention,modifiedslightlyin1921,remainsthebasisofinterna-tionalagreementonunitsofmeasurement.Therearenow54membersoftheBIPM,includingallthemajorindustrializedcountries.

Fromourstandpointitisusefultostresstwomajoradvantagesofsuchasystem.First, for each quantity a unique, stable, primary reference is maintained, recog-nized,andacceptedworldwide.Second,quantitiesinthesystemarelinkedbyaset

Y116134.indb 5 10/10/11 2:11 PM


ofrelations,thecurrentlyacceptedphysicallaws,andconsequentlyprogressinonequantityinfluencesotherquantitiesaswellandwemaysaythatthemeasurabilityofeachquantityisfoundednotonlyonthepropertiesofthatquantity,butalsoontheoverallsystem’scoherence,whichiscontinuallychecked,boththeoreticallyandexperimentally.TheinternationalsystemofmetrologyistreatedingreaterdetailinChapter3.Whentheinternationalsystemformetrologywasconstituted,anotewor-thydevelopmentofmeasurementinpsychologywastakingplace,aswenowshow.

1.4 The role of measurement in the birth of experimental psychology

“Asanexactsciencepsychophysics, likephysics,mustrestonexperienceandthemathematicalconnectionof thoseempirical facts thatdemandameasureofwhatis experienced or, when such a measure is not available, a search for it.” This isFechner, inhisElements of Psychophysics,published in1860(Fechner,1966); in1879WundtestablishedhisfamouslaboratoryinLeipzig,givingrisetoexperimen-talpsychologyasanautonomousdiscipline,stronglygroundedinmeasurement.Buthowwasmeasurementconceivedinpsychophysics?

WehaveseenthatHelmholtzfoundedmeasurementonananalogywithcount-ing.Campbelldevelopedthisapproachbyintroducing,inanembryonicform,thenotionofmeasurementscaleandexplicatingthedistinctionbetweenfundamentalandderivedquantities.TheproblemFechnerhadtofacewasevenmorechallenging,inasmuchas“unlikephysicalprocesses,whichareexternal,public,objective,andopentodirectmeasurement,mentalprocessesareinternal,private,subjective,andcannotbemeasureddirectly.Somehowan indirectmethodhad tobedeveloped,”Wozniak(1999)notes.

Fechner’sideawastomeasuretheincrementsofmentalactivitythroughthemea-surementof theincrementsof theenergyofthephysicalphenomenonthatcausesthem.Butwhatlawlinksthem?Weberhadalreadyshownthat,inseveralcircum-stances,theincrementofthephysicalstimulus,δφ,neededtocauseajustpercep-tiblevariationinthecorrespondingsensationisproportionaltotheintensityofthestimulus,φ;thatis,

δφ=kδ, (1.1)

where k is a constant that depends upon the sensation considered. This is calledWeber’slaw.

Butwhathappensonthesensationside,ontheinternalside?Interestinglyenough,toanswerthisquestionFechnerrefers,asHelmholtz,tothecounting paradigm. Helooksforazeroandaunitfortheevokedsensation,withanaturalandconvincingchoice: thezerois theperceptionthreshold, theunit is theincrementofsensationcorrespondingtoajustnoticeabledifferenceinstimulusquantitythatheassumestobeinvariantandindependentofthevalueofthestimulus:simpleandgenial!

Inthisperspective,theintensityofasensationisthesumofanumberofelemen-taryvariationsofsensationintensity,allequalandcorrespondingtojustnoticeable

Y116134.indb 6 10/10/11 2:11 PM


differencesinthestimulus(NowellJones,1974).If,inturn,justnoticeablediffer-encesinthestimulusfollowWeber’slaw(1.1),alogarithmiclawwouldfollow,

ψ=α lnφ+β, (1.2)

whereψistheintensityofthesensoryperceptionandαandβareconstantparam-etersthatcharacterizetheresponseofanaverageobservertothestimulus.Soitispossibletomeasuretheintensityofasensoryperceptionψindirectlybymeasuringtheintensityofthestimulusφandapplyingthepsychophysicallaw.ThisisdepictedinFigure 1.3.

Thesimilaritywiththeprocedureforindirectphysicalmeasurement,illustratedinFigure 1.2,maybenoted.Summarizing,

• Whatwemeasureisacharacteristicofaphysicalobject(orevent),thathasthecapacityofevokingasensoryperceptioninindividualsthathasdiffer-entpossibledegreesofintensity.

• Measurementmaybeperformedindirectly,bymeasuringtheintensityofthestimulusandapplyingthepsychophysicallaw.

• ThepsychophysicallawisbasedonWeber’slawandonFechner’shypothe-sisthatthejustnoticeabledifferencesinthestimulusintensityevokemutu-allyequaldifferencesintheintensityofthesensoryperception.

• Theparametersofthepsychophysicallawrefertoanaverageobserver.

Some comments are now in order. The assumption of Weber’s law, althoughmathematicallyconvenientwheneverapplicable,isnotamust.Whatisneededisarelationbetweenjustnoticeabledifferencesandtheintensityofthestimulus,thatis,thefunction

δφ=1(φ) (1.3)

whichmaybederivedexperimentally.Fechner’shypothesisinsteadiscrucialanditwascriticized,asweshowinthe

nextsection.Moreover,referringmeasurementtoanaverageobserverisagainnotamust,becausemethodsfordealingwithindividualdifferencesmaybedeveloped,aswewillshowatalaterstage.

Tosumup,thusfarwehaveshownhowinthelastpartofthenineteenthcenturyanoteworthyincreaseinawarenessofthefoundationsofthemeasurementprocessdevelopedinthedisciplinesandinthescientificcommunitieshavinganexperimen-talbasis.Thisultimatelyledtoacomparisonoftheirviews,whichtookplaceatthebeginningofthetwentiethcentury,with,unfortunately,aninauspiciousresult.

Object

Measuredstimulus

Psychophysicallaw

(indirectly)measured

characteristic

Figure 1.3 Indirectpsychophysicalmeasurement.

Y116134.indb 7 10/10/11 2:11 PM


1.5 The schism: The report of the British Association for the Advancement of Science

In1932theBritishAssociationfortheAdvancementofScience(BAAS)appointedacommitteeincludingphysicistsandpsychologiststoconsiderandreportuponthepossibilityofquantitativeestimatesofsensoryevents.Thecommitteeproducedafinalreport, in1939,someofwhoseconclusionsaresummarizedhere(Ferguson,Myers,&Bartlett,1940).Aswehavealreadyseen,itispossiblebypsychophysicalexperimentstoobtainacurvethatlinksjustnoticeabledifferencesinastimulustoitsintensity,formula(1.3).Oftenthisresultsinasmoothcurve,sothatitisfeasibletoassumethatsomeempiricalrelationholdsbetweenthem.Onthesepremises,thecommitteeagreed,butthequestionwasifsuchafunctionfullydescribesthefacts.

Thepsychologistsclaimedtheneedforsomequantitativedescriptionoftheinten-sityofthesensoryperceptionsinvolved.Aswehaveseen,onesuchdescriptionmaybeobtained,accordingtoFechner,byfurtherassumingthateachjustnoticeablestepintheintensityofthestimulusδφproducesastepinsensationδψ,andthatsuchstepsareallequal,whichleadstoformula(1.2).

Thephysicists argued that thepossibilityofmathematicallyobtaining formula(1.2)doesnot“prove”themeasurabilityoftheintensityofthesensationψ,inasmuchasthatformulaisbasedonanassumptionthatcannotbeverified,unlessitispos-sibletoindependentlymeasuretheintensityofperception.Butthisisnotpossible,intheirview,becausenoadditiveoperationisdefinableamongsensations.Themainpointagainstthemeasurabilityoftheintensityofasensationthuswasultimatelytheimpossibilityofsatisfactorilydefininganadditionoperationforit. Ontheotherhand,thepsychologistsclaimedthat,althoughFechner’spositioncannotbefullysupported,the result ofpsychophysical experiments cannot, ingeneral, be expressedonly intermsofstimulusintensities(Stevens,1936).Infact,theparticularintensitychangesδφthatarebeingplotted“areobtainedonlybyreferencetothesensationtheyevoke,andarebasedontheequalityorinequalityofthesesensationintensities.”

Thereportofthecommitteehadanenormousinfluenceinthefollowingyearsandwemaysaythat it ledtoanessentiallyparalleldevelopmentofmeasurementscienceinphysicalscienceononesideandinbehavioralsciencesontheother,withconsequencesuptothepresentday.Wereturnsoontotheconclusionsofthecom-mittee,afterbrieflyreviewingsomeadvancesinmeasurementthattookplaceinthetwentiethcentury.

1.6 Twentieth-century development

1.6.1 Development of measurement in psychology

1.6.1.1 Psychometrics

Inthebeginningofthetwentiethcenturythescopeofpsychologicalmeasurementbroadened,andwemayidentifytwomainschools,psychophysicsandpsychomet-rics.Aswehaveseen,psychophysicsorperceptualmeasurementfocusesonwithin-individual processes that can be assumed to vary very little between individuals

Y116134.indb 8 10/10/11 2:11 PM


(Gescheider,1997).Thusavariationinthestimulus(outsidetheindividual)wouldroughly be processed the same, or at least according to the same principles. Socharacteristicssuchasredorcoffeeodorareessentiallyperceivedinthesamewaybyindividuals.

Psychometricsinsteadfocusesonmeasuring“hostfactors,”thatis,conceptsthatareuniquetoanindividual,suchastrait,andwhereinterindividualdifferenceswouldtypicallyexist(Lord&Novick,1968).Examplesarepersonality,emotions,mood,and attitudes. Thus psychophysics aims at measuring attributes of stimuli, whichvarywith“invariantpersons,”whereaspsychometricsaimsatmeasuringattributesofpersons,whichvarywith“invariantstimuli.”Herethestimuliaretestitems.

AswehavealreadyillustratedinFigure 1.3,psychophysicsdealswiththemea-surementofcharacteristicsofobjects,asperceivedbyhumans.Inordertodothis,apsychophysicallawisused,referredtoastandardobserver:thesensationevokedbythecharacteristicunderinvestigationmaybeindirectlymeasuredbymeasuringthestimulusandapplying thepsychophysical law to it. Individualdifferencesareconsideredasaninherentuncertaintysource,ornoise,insuchmeasurements.

Analternativeapproachhasbeenpursuedinpsychometrics,wherethecharacter-isticsofindividualsareofinterest.Theyaremeasuredbytheresponsestheyprovidetomentaltests,whichplaytheroleofmeasuringinstruments.Onthebasisofsuchresponses,thepsychologicalvariablesofinterestmaybemeasured.ThisisdepictedinFigure 1.4.Similartotheseare,tosomeextent,measurementsinsociologyandeco-nomics,whereoftenpreferencesareassessedandtheuseofquestionnairesistypical.

1.6.1.2 Stanley S. Stevens: The direct measurement of sensory events

LetusnowreturntothereportoftheBritishAssociation.Aswehavenoted,twomainpointswereraisedagainstthemeasurabilityofsensoryevents,theimpossibil-ityofdirectlymeasuringperceivedintensityandtheneedforadditivityasaneces-sarypropertyformeasurementingeneral.AnanswertobothpointscamefromthecontributionofStevens.

Forthefirstpoint,thepossibilityofdirectlyevaluatingtheintensityofasensa-tion,heintroducesnewtypesofmeasurement(orscaling)methods,inwhichmagni-tudeorratioofsensationsisestimateddirectlybythesubjectsandcalledmagnitude estimation orproduction andratio estimationorproduction.

Presentalineofagivenlengthandtelltheobservertocallitsomenumber,say,10.Thenpresentalineofsomeotherlengthandsay:“Ifthefirstlinewas10,whatwouldyoucallthesecondline?Useanynumberthatseemsappropriate—fractional,decimal,wholenumber—buttrytomakethenumberproportional

Individual

Characteristicto be measured

Mental test

Responseto the test Evaluation

of theresponse

Measuredcharacteristic

Figure 1.4 Measurementbasedonapsychometrictest.

Y116134.indb 9 10/10/11 2:11 PM


totheapparentlengthasyouseeit.”Thenproceedtootherlengths,inirregularorder,andaskforsimilarestimatesofapparentlength.(Stevens,1959)

Thisismagnitude estimation;inmagnitude production,instead,theobserverisaskedtoalterthelengthinordertomatchnumbers.Inratio estimationtheobserverisaskedtoestimatetheapparentratiobetweenseverallengthsandastandardlengthwhichismaintainedconstantduringtheexperiment.Last,inratio productiontheobserver is asked tochangeavariable length inorder tomatchgiven ratioswithrespecttothestandardone.Notethatinparticularinmagnitudeestimationa person acts as a measuring system (Stevens,1956).Thisisanimportantpoint,andisillus-tratedinFigure 1.5.

Thankstothesenewmethods,StevenscouldcheckFechner’slawandactuallyherejectedit,proposingasasubstitutetheso-calledpower law,

ψ=ηφθ, (1.4)

where,asusual,φisthephysicalstimulus,ψisthecorrespondingperceivedquan-tity, andη andθ areconstantsthatdependuponthekindofsensationconsidered.AsinFechner’sapproach,theintensityofasensationmaystillbemeasuredindirectlybymeasuringastimulusquantityandapplyingapsychophysical law,butwithanimportantdifference:herethelawhasnotbeenassumedasahypothesisaboutthewayperception takesplace, but rather it hasbeendetermined experimentally, bydirectlymeasuringboththestimuliandtheresultingsensations,thankstothenewscalingmethods(Stevens,1956).

1.6.1.3 Stanley S. Stevens: The classification of measurement scales

Considering the crucial point of additivity, Stevens circumvents it by takinganotherway.

Inthemeantime,unawarethat theBritishcommitteeofBAASwastryingtosettletheissue,someofusatHarvardwerewrestlingwithsimilarproblems….WhatIgainedfromthesediscussionswasaconvictionthata more general the-ory of measurement was needed,andthatthedefinitionofmeasurementshouldnotbelimitedtoonerestrictedclassofempiricaloperations.

ObjectStimulus

Internalcomparison

Directlymeasuredsensation

Internalreference scalefor sensation

intensity

PersonSensation

Figure 1.5 Magnitudeestimation:apersonactsasameasuringsystem.

Y116134.indb 10 10/10/11 2:11 PM


The best way out seemed to approach the problem from another point ofview,namely,thatofinvariance,andtoclassifyscalesofmeasurementintermsofthegroupoftransformationsthatleavethescaleforminvariant….Afourfoldclassificationofscalesbasedonthisnotionwasworkedoutsometimearound1939andwaspresentedtotheInternationalCongressfortheUnityofSciencein1941.WorldWarIIthencamealong,andpublicationwasdelayeduntil1946.(Stevens,1959)

ThisfamousfourfoldclassificationofmeasurementscalesissummarizedinTable 1.1(Stevens,1946).

Nominalscalesarerelatedtoclassificationoperationsandnumbersserveonlytodistinguishoneclassofobjectsfromanother.Anyone-to-onesubstitutionispermis-sible,becauseidentificationisstillpossible.Ordinalscalespermitarankorderingofobjectsandremaininvariantundermonotonicincreasingtransformations.Intervalscalesentailaconstantunitofmeasurement;thatis,theyintroduceametric,andsopermitthecalculationofdifferencesbetweenanytwovalues.Theyremaininvariantunder linear transformations.Ratioscalesalso featureconstantunitsofmeasure-ment,but,inaddition,theyallowtheratiooftwovaluestobeevaluated,becauseanabsolutezeroexists.Theyareinvariantunderanysimplymultiplicativetransforma-tion.Notethatequalityofratioshereplaystheroleofanempiricalrelation,substitut-ingempiricalsuminmeasurementsonratioscales.

Table 1.1 Classificationofmeasurementscales,accordingtoStevens

ScaleBasic empirical

operationsMathematical group

structure Examples

Nominal Determinationofequality

Permutationgroup

m′(a)=f [m(a)]

wherefisanyone-to-onetransformation

PerceptionthresholdsDetectionofdefects(forexampleweldingdefectsinweldedstructures)

PatternrecognitiontechniquesOrdinal Determinationof

greaterorlessIsotonicgroup

m′(a)=f [m(a)]

wherefisanyincreasingmonotonicfunction

HardnessofmineralsEarthquakeintensityWindintensityGradesofleather,lumber,wool

Interval Determinationoftheequalityofintervalsorofdifferences

Linearoraffinegroup

m′(a)=αm(a)+β

withα>0

Temperature(FahrenheitorCelsius)

PositionTime(calendar)

Ratio Determinationoftheequalityofratios

Similaritygroup

m′(a)=αm(a)

withα>0

Length,mass,density,timeintervals

Temperature(kelvin)Loudness(sone)Brightness(bril)

Note: Exampleshavebeenpartiallychangedfromtheoriginal.

Y116134.indb 11 10/10/11 2:11 PM


Summarizing,Stevensshowswaysforovercomingtheseverelimitationinmea-surabilityappearingintheBAAScommitteereport,basedonincreasingthenumberofallowablemeasurementscales,throughaninnovativeapproachtotheirclassifica-tionbasedonaninvarianceprinciple,andonconsideringratioequalityasanempiri-calrelation.

WithStevens's contributionswehave reached the secondhalfof the twentiethcentury.Atthattimeaconsiderablebodyofresultshadbeenobtainedinmeasure-menttheoryandtherewasaneedforsystematization,whichwasachievedwiththerepresentationaltheoryofmeasurement.Butpriortothatwehavetotakealookattheprogressoftheinternationalsystemofmetrology.

1.7 Measuring the human response to physical stimuli: The candela in the SI

TheMetreConvention,establishedin1875,wasrevisedin1921,extendingthescopeandresponsibilitiesoftheBIPM,originallyconcernedwithmechanicalquantities(length,mass,andtime)tootherfieldsinphysics.ThentheConsultativeCommitteeforElectricity(CCE,nowCCEM)wascreated(1927)andafour-dimensionalsystembasedonthemeter,kilogram,second,andampere,theMKSAsystem,wasadopted(1946).Thenumberofbasequantitieswasthenextendedtosix,withtheintroduc-tionofthekelvinandthecandela,respectively,forthermodynamictemperatureandluminousintensity(1954).

ThenameInternationalSystemofUnits,withtheabbreviationSI,wasgiventothesystemin1960.Lastly,in1971,thecurrentversionoftheSIwascompletedbyaddingthemoleasthebaseunitforamountofsubstance,bringingthetotalnumberofbaseunitstoseven(BIPM,2006).

Forourpurposes,thecandeladeservesspecialinterest.Mattersconcerningpho-tometryare treated, inparticular,by theConsultativeCommittee forPhotometryandRadiometryoftheBIPM.InamonographonthePrinciples governing photom-etry(BIPM,1983),thespecialcharacterofphotometricquantitiesispointedoutbynotingthatthey

must take intoaccountboth thepurelyphysicalcharacteristicsof the radiantpowerstimulatingthevisualsystemandthespectralresponsivityofthelatter.Thesubjectivenatureofthesecondofthesetwofactorssetsphotometricquanti-tiesapartfrompurelyphysicalquantities.

Thisisaveryimportantstatement,becauseitmakesclearthatitmakessense,andactuallyitisveryuseful,tomeasurequantitiesthatarenot“purelyphysical,”butratherhavea“subjectivenature,”inthattheymeasurethehumanresponsetosomeexternalstimuli.Basically,themeasurementofluminousintensitymaybedonebyspectrophotometers, which are instruments that first detect the radiant power bysensorsandthenprocesstheresultingsignalbyapplyingaspectralweightingthat

Y116134.indb 12 10/10/11 2:11 PM


accountsforthespectralresponsivityofanaverageobserver.Suchweightingwasdeterminedbyexperimentsofintensitymatchingbypanelsofsubjects:

The matching of brightness is the fundamental operation of photometry. Indirectvisualphotometry,thevisualstimuliprovidedbytwojuxtaposeduniformlightpatchesofsimilarshapeandangularareaintheobserverfieldofviewareadjustedsothatthepatchesappearequallybright.(Wyszecki,1982)

Therefore the ultimate foundation for the measurement of luminous intensityincludestheresultsofpanel-testingexperiments.Thepathforintroducingquanti-tiesrelatedtoperceptionwasthusopenedthen,in1954,withtheintroductionofthecandela,althoughtheculturalconsequencesofthateventperhapshavenotyetbeenfully recognized.Luminous intensityhasbeen theoutrider forwhatwenowcall“physiologicalquantities,”towhichwereturnatalaterstage.Bynowitisimportanttonote further thatmeasurements in thefieldof biology, health, and safetyhavethusfarbecomemoreandmoreimportant.Interestinglyenough,inthe21stGeneralConferenceonWeightsandMeasures(GCPM)in1999,itisrecommendedthat

those responsible for studies of Earth resources, the environment, humanwell-beingandrelatedissuesensurethatmeasurementsmadewithintheirpro-grammesareintermsofwell-characterisedSIunitssothattheyarereliableinthelongterm,arecomparableworldwideandarelinkedtootherareasofsci-enceandtechnologythroughtheworld’smeasurementsystemestablishedandmaintainedundertheMetreConvention.(BIPM,1999)

Tosumup,inthemidstofthetwentiethcenturytheinternationalsystemofmetrol-ogytookamajorstepforwardwiththeconstitutionoftheSI,withthesubsequentinclusioninthefundamentalquantitiesofaunit,thecandela,thattakesaccountofthehumanperceptionofaphysicalstimulus.Increasingattentiontoquantitiesofthiskindhasemerged,andtheyhavebeenrecognizedastreatablewiththeparadigmofderivedmeasurementthatwehavediscussedintheprevioussection.Thetimewasrightforattemptingasystematizationofthetheoryofmeasurement.

1.7.1 An attempt at systematization: The representational theory of measurement

Aremarkablesystematizationof the formal theoryofmeasurementwasachievedby the representational theory (Krantz, Luce, Suppes, & Tversky, 1971; Roberts,1979/2009; Suppes, Krantz, Luce, & Tversky, 1989; Luce, Krantz, Suppes, &Tversky,1990),whichcombinestheviewpointsofCampbellandStevens,andareseenascomplementaryratherthanopposing.Themainidea,traceable,aswehaveseen, toHelmoltz, is that thenumbersweobtain throughmeasurement representempirical relations. This holds true for fundamental physical measurements, as

Y116134.indb 13 10/10/11 2:11 PM


intendedbyCampbell,herecalledextensive,butalsoforother,weaker,scales,asenvisagedbyStevens.ConsequentlytheclassificationofscalesproposedbyStevensmayberetainedandeachscaleisnowcharacterizedbyarepresentation theoremandauniqueness theorem.

Arepresentationtheorem,foragivenscale,statesinwhichway,forthatscale,empiricalrelationsaremappedintocorrespondingnumericalones.Forexample,inanordinalscale,itreads

a⋟b⇔m(a)≥m(b), (1.5)

thatis,anempiricalorderbetweentwoobjects,a⋟b,holdsifandonlyifthecor-respondingnumericalorderholdsbetweentheirmeasures,m(a)≥m(b).Considerthenotation:thesymbol⋟denotesanempiricalrelationbetweentwoobjects,whereasthesymbol≥denotesasimilarrelationforthecorrespondingmeasures,whicharenumbers.Thiscarefuldistinctionbetweenempiricalrelationsandnumericalonesistypicaloftherepresentationaltheoryandrequiresapropernotation.

Theuniquenesstheoremsayswhattransformationsmaybesafelyappliedtothescalewithoutalteringthemeaningofthemeasurement:forexample,intheordinalscalecasemonotonicincreasingtransformationsmaybeapplied,becausetheyareorderpreserving.Throughthistheoremthemeaningfulnessofstatementsconcern-ingmeasurementmaybeaddressed:wemaysay that a statementconcerning theresultsofmeasurementonagivenscaleismeaningfulifitstruthisunaffectedbyadmissibletransformationsonthatscale.

AsummaryoftherepresentationframeworkispresentedinTable 1.2,whichispro-videdherejusttogiveanoverviewofsomeofthemainresultsoftherepresentational

Table 1.2 Summaryofthemainscalesforfundamentalmeasurement,asconsideredintherepresentationaltheory

Empirical structure

Empirical relations

Scale type Representation

Admissible transformations

Nominal Equivalenceamongelementsineachclass

Nominal a~b⇔m(a)=m(b) Bi-univocal

Order Weakorderamongtheobjects

Ordinal a⋟b⇔m(a)≥m(b) Monotonicincreases

Difference Asaboveplusweakorderamongintervals

Interval Δab⋟Δcd⇔m(a)–m(b)≥m(c)–m(d)

Linearpositive m′=αm+b α>0

Extensive Asaboveplusaconcatenationoperation

Ratio a~b∘c⇔m(a)=m(b)+m(c)

Similarity m′=αm α>0

Note: Symbols~,≻,⋟,∘denoterelationsoroperationsamongobjects,whereas=,>,≥,+denotesimilarrelationsoroperationsamongnumbers. Inparticular,“~”means“equivalent to”,“≻”“greaterthan”,“⋟”“greaterthanorequivalentto,and“∘”denotesempiricaladdition.

Y116134.indb 14 10/10/11 2:11 PM


approach.Wedonotgointodetailsnow;therepresentationalapproachisprobedfur-therinChapter4,withaspecialfocusonuniqueness,andisalsodiscussedinChapter5,wheretheformalismisreviewedandaprobabilisticreformulationprovided.

Therepresentationaltheoryhasbeendevelopedmainlyinthefieldofbehavioralsciences(Iverson&Luce,1998)buthasbeenbroughttotheattentionofphysicistsandengineerssincethe1970s,mainlybyFinkelstein,whohassupporteditsfeasi-bilityforallkindsofmeasurements.Hisdefinitionofmeasurementas“aprocessofempirical,objectiveassignmentofsymbolstoattributeofobjectsandeventsofthe realworld, in suchawayas to represent themor todescribe them” (1982) isfamous.Thistheoryalsoreceived,afterwards,contributionsfromthatcommunity(Finkelstein,2005;Muravyov&Savolainen,2001;Rossi,2007)anditactuallycon-stitutesanexcellentstartingpointforaunifiedtheoryofmeasurement.

1.8 Preparing the future

Itisnowtimeforbrieflyreviewingthecurrentstateoftheartandforindicatingpos-sibledevelopmentlines.

1.8.1 Trends in metrology

In the last part of the twentieth century some very important issues emerged inmetrology.Theneedofaccompanyingmeasurementresultswithastatementontheiruncertaintywas recognizedasameansof fullyprovidingmeasurement informa-tion.Aworkinggroupwasappointedtosearchforaninternationallyagreedwayofexpressinguncertaintyandtoprepareguidelines.Afteralongdiscussion,anotewor-thyproposalwasproduced,theGuide for expression of uncertainty in measurement(GUM),publishedin1993(BIPM,2008a).Thiseventhaspushedparallelandsuc-cessiveresearchefforts,bothonthetheoreticalandontheapplicationside.

Atthesametimeanotherimportantinitiativewascarriedoutconcerningtherevi-sionofthelanguageinmetrology,whichyieldedthepublicationoftheinternationalvocabularyofbasicandgeneraltermsinmetrology(1984),nowinitsthirdedition(BIPM, 2008b). All the revisions have implied a profound reconsideration of thematter,whichdemonstratesthecriticalityofthisissue.

Third,thecooperationamongmembersoftheBIPMhasbeenreinforcedandmademoreeffectivewiththeinstitutionofaMutual Recognition Agreement(CIPMMRA)(BIPM,2008c),whichspecifiestheorganizationalandtechnicalrequirementsforthemutualrecognitionofmeasurementsperformedbynationalmetrologicalinstitutes(NMIs).Amajortoolforsuchrecognitioniskey comparisons.Insomecases,com-parisons are performed directly against an international reference facility at theBIPM.InothersastabletravelingstandardiscirculatedamongseveralNMIs,whichare asked to provide ameasurement value for it, accompanied by an uncertaintystatement. An international committee of NMI experts in the field evaluates theresulting data and provides practical information on the degree of comparabilityoftheindividualresults.Similarexercises,calledintercomparisons,areperformedamonglaboratoriesat lowerlevelsofthemetrologicalstructureandtheyarevery

Y116134.indb 15 10/10/11 2:11 PM


effectiveforguaranteeingtheperformanceoftheoverallsystemofmetrology.Theirusecouldperhapsbeextendedtomeasurementinbehavioralsciencesalso,aswementionatalaterstage.TheCIPMMRAagreementfurtherconfirmsthebenefitofmakingasystemofmeasurementactivitiesperformedintheworld(BIPM,2003).Such a systemmayperhapsundergo anoteworthy reorganization in a fewyears,inasmuchasthereisaproposaltorevisethedefinitionofthebasequantitiesthroughthenaturalconstantsthatlinkthem(Millsetal.,2006;Wallard,2009,2011).Thereisnoroomherefordiscussingthistopicindetail;wesimplynotethatthesystemofmetrologyhasanevolutionarynature,andwiththiscontinuousprocessofrevisionitcontributessignificantlytotheoverallprogressofscienceandtechnology.

In thecoreof this system,anewparadigmhas recentlyemerged,soft metrol-ogy, defined as “measurement techniques and models which enable the objectivequantificationofpropertieswhicharedeterminedbyhumanperception,”where“thehuman response may be in any of the five senses: sight, smell, sound, taste andtouch”(Pointer,2003).Auniqueopportunityforpursuingstudiesinthisareahasalsoemergedmeantime.

1.8.2 The “measuring the impossible” environment

Themeasurementofquantities related tohumanperceptionhas recently receivedimportantsupportbyaEuropeanCall,namedMeasuringtheImpossible(MtI),asapartoftheNewandEmergingScienceandTechnology(NEST)programme.NESTaimsatsupportingunconventionalandvisionaryresearchwiththepotentialtoopennew fields for science and technology, as well as research on potential problemsuncoveredbyscience.

MotivationsundertheMtICallincludescientific arguments,“manyphenomenaof significant interest to contemporary science are intrinsically multidimensionaland multidisciplinary, with strong cross-over between physical, biological, andsocial sciences,” economic aspects, “products and services appeal to consumersaccording to parameters of quality, beauty, comfort, etc., which are mediated byhumanperception,”andsocial reasons,“publicauthorities,andquasipublicbodiessuchashospitals,providecitizenswithsupportandserviceswhoseperformanceismeasuredaccordingtoparametersoflifequality,securityorwellbeing”(EuropeanCommission,2007).Thefocusison“measurementswhichareholisticandmulti-dimensional innature,andwhich involvecomplex issuesof interpretation,and/ormediationbyhumanperception.”

Severalprojectshavebeen launched in thisperspectiveandaneedforcoordi-natedactionhasbeenenvisaged,whichhasgivenrisetoanactioncalledMINET,MeasuringtheImpossibleNetwork.MINETistheresultofanewwayofapproach-ing the problems that we have so far discussed. It starts from the recognition oftheinterdisciplinarycharacterofmeasurementinvolvingpersonsandidentifiesout-comesfromtheconsiderationthattheinterdisciplinarynatureoftheMtIraisesnewchallenges.TheMINETactivityhasincludedtheorganizationofworkshops,thinktankevents,andaninternationalcourse,heldinGenova,Italy,inJune2008,whichwasattendedby70peoplefrom14differentcountries.Thiscoursehasgivenriseto

Y116134.indb 16 10/10/11 2:11 PM


thisbook.Thisiswherewearenow:itisthustimetobrieflyreviewsomecurrentareasofresearchandtoaddressfutureneeds.

1.9 Current research issues and future needs

Researchissuescentraltomeasurementwithpersonsmaybegroupedinthreemaincategories,

• Instrumentationandmethods• Foundationandtheory• Implementationareasandapplications

1.9.1 Instrumentation and methods

Instrumentation-orientedresearchconcernsboththemeasurementofphysicalevents(thestimuli)thatgiverisetoasensoryresponseandthephysiological(orbehavioral)responsestointernal/externalstimuli.Itwouldalsoincludeperceptionandinterpreta-tionprocessesandthedevelopmentofsensorsthatmimic,tosomeextent,humanper-ception.Sincethetimeofthe“schism”inthe1940s,progressintheseareashasbeenenormous;concerning,forexample,themeasurementofsound,wenowhavehighlyaccuratemeasurementmicrophonesandbinauralrecordingdevicesthatmakeitpos-sibletomeasuretheacousticstimuliastheyappearattheinputtotheauditorysystem.Wealsohavesophisticatedbinauralreproductiondeviceswithprocessorsandalgo-rithmsfortherequiredsignalprocessing.SoundperceptionistreatedinChapter11inthisbook.Inthecaseofsight,wecanmeasurenotonlyluminousintensityandcolor,butalsoparametersoftheinteractionbetweenlightandmatter,aswellaspropertiesofsurfaces,suchastexturethatalsoinvolvesophisticatedsignalprocessing.VisualperceptionistreatedinChapter13andtheskinsensesinChapter12.

Concerning the measurement of physiological processes, novel techniques areavailable,especiallyinthefieldofbrainimaging(Bronzino,1995).Thesetechniquesdevelopedrapidlybecauseof theirgreatvalueinneurophysiology/neuroscienceandaretreatedamplyinChapter14.Aswehavementioned,thereisincreasinginterestinexploringphysiologicalquantities,thatis,quantitiescausedbyphysiologicalresponsesinthehumanbody,andtheircompatibilitywiththeinternationalsystemofmetrology(Nelson&Ruby,1993).AworkshoponthisverytopicwasorganizedinNovember2009,bytheBIPM,dealingwithopticalradiation,radiowavesandmicrowaves,ion-izingradiation,acoustics,magneticfields,andotherinternationalstandardmeasuresandunitsapplied,forexample,indocumentsbytheWorldHealthOrganization.

Inabroadervisionofhumanresponse,wemayincludebehavioralaspectsalso.In this regard, image-basedmeasurementsplay akey role; they are essential, forexample,forstudyingemotionsorbodylanguage,whicharetreatedinChapter15.Emotionalresponsesmayalsobeassessedbyphysiologicallyorientedmeasurementswithinstruments.Galvanicskinresponse,forexample,measuresdetectablechangesin skinconductance, inducedby the sympatheticnervous systemdue to stressoremotions.Althoughoftenreplacedbybrainimaging, thisrelativelyoldmethodis

Y116134.indb 17 10/10/11 2:11 PM


still profitably used, because of its simplicity and low cost. Another approach toemotionistomeasuretheactivationofcriticalfacialmusclesbyelectromyography(Dimberg,1990).Moreover,highlyvaluableinformationonthecomplexmechanismofvisualperceptionmaybegainedbythetrackingofsaccadiceyemovements:cur-rentadvancesinthisareahaverevealedspace–timegapsthatseemtobeinterpre-tableviarelativistic-likeprinciples.

Sensorsthatmimichumanperceptioninclude,forexample,theelectronicnose.Incurrentresearchaimingatdirectlysensingthequalityofindoorair,suchadevicehasbeenbuiltaccordingtoaperceptualodorspacedeterminedempiricallyformaterialsemissions.Itismultidimensionalandtheinterdistancesbetweenodorqualitieswerecalibratedwiththeaidofasetofodorreferencesthatfulfillsreliabilityrequirements.Otherexamplesaretactilesensors,thathaveanincreasingabilityto“perceive”surfacetexture,orthevisualsensorscalledartificialretinas.Advancedroboticsandclinicalapplicationsareexpectedforthesesensors.Oneexampleistheartificialarmequippedwithtactilesensorswhichmakeitpossibletoappreciatetheaestheticsofsculptures.

Infact,aswehaveexplicitlynotedfortheelectronicnose,alltheseinstrumen-tation-relatedpossibilitiesshouldnotmakeusforgetthatthescreeningandtestingofparticipantsasmeasuringinstrumentsareabsolutelynecessaryforreliableandvalidpsychologicalmeasurement.Manyreliable methodsof measurement areavail-able,rangingfromthetraditionalmethodsofpsychophysics(themethodoflimits,ofaverageerror,andofconstantstimuli,whichweredevelopedbyG.T.Fechnerfordeterminingabsolute thresholdsofdetectionandjustnoticeabledifferencesor, ingeneral,equalitiesorinequalitiesamongsensations)throughthebasicmethodsofdirectscalingintroducedbyS.S.Stevens(magnitudeorratioestimationandpro-duction),tothemoreadvancedapproaches,suchasthemasterscaling(Berglund&Harju,2003).PsychophysicalmeasurementsaretreatedinChapter2.Psychometricsisanotherschoolofmeasurementinpsychology,aswehavebrieflymentioned,thatuses standardized tests for collecting data, that is, in the form of questionnaires,interviews,orperformancetasks.

Thurstonianscaling (Thurstone,1927) is somewhat intermediatebetweenpsy-chophysicsandpsychometricsandissometimescalled“indirectscaling”asopposedtoStevens’sdirectscaling.Fromapsychophysicalperspective,thisapproachisbasedontheassumptionthatasinglevalueofanexternalstimuluswillevokeperceptionsthataremappedonaninnerpsychologicalcontinuum,givingrisetoaprobabilitydistribution,usuallyassumedtobeGaussian.Soitispossibletoestablishapreciserelationbetweendistancesonsuchacontinuumandtheprobabilityoforderrela-tions.Accordingly,itispossibletoinfer,fromapair-comparisontest,ametricscal-ing.Actually,theseinnerrepresentationsarenotnecessarilyrelatedtoanexternalstimulusandthisiswhythisapproachmayalsobeusedinpsychometrics:indeed,itwasfirstproposedformeasuringattitude.Aremarkablemodelforpsychometrictestsisprovidedbytheitemresponsetheory,inwhichtheprobabilityofthecorrectresponsetoatestitemisafunctionofpersonanditemparameters.

Tosummarize,manymethodsareavailableforgatheringinformationfromtherealworld,whetherouterorinner,andwemustfocusonhowbesttousesuchinfor-mation.Thisiswheremeasurementtheorycomesintoplay.

Y116134.indb 18 10/10/11 2:11 PM


1.9.2 Foundations and theory

Severalfoundationalandtheoreticalissuesare,inouropinion,ofprimeinterestinthisarea,including

• Thelanguage• Theconceptsofmeasurementscaleandofmeasuringsystem(orinstrument)• Thelogic(deterministic,probabilistic,fuzzy,etc.)• Theissueofmeasurability• Multidimensionalityandmathematicalmodeling

Languagehasbeenamainissueintwentieth-centuryscientificandphilosophicaldebate.Inthemetrologycommunity,anoteworthyrevisionoflinguistictermshasbeen undertaken, starting with the publication of the international vocabulary ofbasicandgeneraltermsinmetrology(1984),nowinitsthirdedition(BIPM,2008b).Itisinterestingtonotetheevolutionofsucheditions,becausethetrendhasbeentoincludefurtherdisciplinaryareas.Itmaybeenvisagedthatinthenearfuture,mea-surementwithpersonsmayplayarolethere.Moreover,intheworkshopsandthinktankeventsorganizedbyMINET,theissueoflanguage/vocabularysoonemerged.Itisnaturalthatthiswillbeachallengeinanymultidisciplinaryenvironment.Yet,inanyrevisionofterms,arevisionofconceptsandtheoriesmustalsobeconsidered,thatindeedwillbebeneficialfortheentireworldofmeasurement(Rossi,2009a).

Anytheoryofmeasurementshoulddeal,atleast,withtwomaintopics,themea-surement scaleandthemeasurement process.Aswehaveseen,thenotionofscaleiscentral to representational theory,and it shouldbegivenmoreconsideration inphysicalmeasurementtoo.Althoughnowadaysweknowmuchaboutscales,furtherresearchisstillneededforensuringthatthisconceptisreallycommontomeasure-ment inphysical,psychological,andsocialsciences.This includes,forexample,aprobabilisticformulation(Schweizer&Skal,1983;Rossi,2006),abetterunderstand-ingofthenotionofscaleinthecaseofderivedquantities,andabettersystemati-zationof the foundations forvariouspsychologicalmethodsofmeasurement.TheprobabilisticapproachistreatedinChapter5andthefuzzyoneinChapter10.

Theconceptofmeasurement processhasastrangehistory.Itiscloselyrelatedtothenotionofmeasuring system,orinstrument.AlthoughscientificinstrumentshavebeenthekeytodevelopingmodernsciencesincethetimeofGalileo,andtheyareakeytopicinuniversitycoursesonmeasurement(Bentley,2005),itisinterestingtorealizethatinstrumentswereignoredintheresearchonthefoundationofmea-surement.Onlyrecentlywasthetheoreticalroleofthemeasuringsystemoutlined.Inparticular,itisimportanttoconsiderwhetherthisisimportantonlyforphysicalmeasurementorforpsychologicalmeasurement,too.

Thequestionofthelogicistransversaltoalltheabove.Ifuncertaintyisrecog-nizedasanessentialaspect inmeasurement,aprobabilisticora fuzzy logicmaybe best suited (De Coensel, Botteldooren, Debacq, Nilsson, & Berglund, 2008).Systematicstudiesintheselatterperspectivesarestillintheirinfancy.

Y116134.indb 19 10/10/11 2:11 PM


Measurabilityhasrecentlybeendiscussedandproposals,mainlybasedon therepresentational approach, have been presented (Finkelstein 2005, 2008; Rossi,2007).Thisisclearlyakeypointrequiringcarefuldiscussiontogetherwithmise en pratique issues.

AsoutlinedintheMtICall,multidimensionalityisofteninvolvedintheprocessesofhumanperceptionandinterpretation.Ashiftfromunidimensionaltomultidimen-sionalmeasurementswillresultinsignificantchanges.Inaunidimensionalscale,thekeypropertyisorder,whereasinamultidimensionalspace,thekeypropertyisdis-tance(orcontent).Moreover,inthelattercasetheproblemofdimensionalityreduc-tionbecomesmostimportant.Infuturework,itwouldbebeneficialtoproceedwithfoundationalsearch inparallelwithmathematicalandnumericaldevelopmentsofmodelsandmethods(Muravyov&Savolainen,2001).MultidimensionalityistreatedinChapters6–9,fromvariousstandpoints.

Humanperceptionandinterpretationof,say,thelivingorworkingenvironment,maybeunderstoodthroughthemediationofmodeling.Modelingofcomplexper-ceptualenvironmentsrequiresacombinationofphysicalandperceptualmeasures.AnexampleofmodelingispresentedinChapter8.

1.9.3 Implementation areas and applications

Measurementsrelatedtohumanperceptionandinterpretationhaveawiderangeofactualandpotentialapplications(EuropeanCommission,2007;Rossi,2009b;Rossi&Berglund,2009).Herewebrieflymentiontheareasofperceivedquality(ofprod-uctsandservices),environment,ergonomics,safety,security,andclinics.TheroleofmeasurementindecisionmakingisaddressedinChapter16.

Inthefirstpartofthelastcentury,theimpactofmassproductionwassohighthatqualitativeaspectsofgoodsweresomewhatneglected.Today,theshortageofenergysourcesandtheconcernforpollutionmaycauseanincreasingrequestfordurable,high-qualitygoods.Thus,perceived quality,whichresultsfromperceptionandinter-pretationofsensoryinput,mayplayakeyroleinindustrialcompetition.Examplesofproductsincludefood,materials,andsimpleandcomplexdevices.Agoodcupofcoffee,forexample,isappreciatedonthebasisofacombinationoftaste,smell,sight,andtouch.Commonmaterialsofdailyuseincludefabric,paper,wood,andstone.Forthese,thefeelingofnaturalnessisimportant:inthisregard,asalreadymentioned,researchisongoingforrelatingnaturalnesswithacombinationofvisualandtactileinputs(Goodmanetal.,2008).Domesticelectricappliancesareappreciatednotjuston thebasis of their performance, but also, perhapsmainly, becauseof the soundqualitytheyproduceaswellastheirvisualappearance.Colorphotocopiersseemtobeevaluatedmainlyonthebasisofpleasantnessinsteadoffidelity;inthecaseofsinglecolors,pleasantnessseemsmainlytodependonhue,lightness,andchroma,whereasfor combinations of colors, predictions are more difficult. The next generation oftouchscreensonmobiledevicesmayprovidesometouchfeedback,thatis,simulatingtexturebyvaryingfrictionfactorthroughcontrolledultrasonicoscillation.Last,formanyyears,carproducershavebeenawareofhowinteriorcarnoise,door-closuresound, and even interior smellwill affect a customer’sdecision tobuy anewcar.

Y116134.indb 20 10/10/11 2:11 PM


Thelastexampleisparticularsignificantbecauseinthelast,say,20years,perceivedqualityhasbeenthemain(oreventheonly)motivationforsupportingresearchintheproductdevelopmentarea,atleastattheEuropeanCommunitylevel.Yet,webelievethatevenifthisremainsanimportantapplicationarea,aswehaveclaimed,thereareotheremergingareas,perhapsevenmorevaluableinastrategicperspective.

Outdoorandindoorenvironmentsaregoingtobeofmajorconcernintheyearstocome.Outdoors,visual,olfactory,andauditoryperceptionprovidethebasisforqualityevaluation.Researchprojectsconcernedwiththecharacterizationofland-scapesandsoundscapes(i.e.,acombinationofsoundsthatresultsfromanimmersiveenvironment)maybementioned,aswellasmeasurementcampaignsforreducingloudnessorodorintensityoffertilizersinthesurroundingsofindustrialplants.Thisstudyarea,called“environmentalpsychophysics,”faceschallengesofcharacterizingexposures inamultisensoryway,varyingover time,andoftenobscuredbyback-groundconditions,thatrequirescarefullydesignedandcontrolledmeasurementpro-cedures(Berglund,1991).Indoorenvironmentisalsoofgreatimportance,becausepeople spend about 90% of their time indoors, either at work, at home, or whencommutingbetweenworkandhome.Thequalityoftheindoorenvironmentdependsuponthequalityofitssubsystems:airquality,soundscapes,visual–tactualsurfaces,andtheirintegration.Perceptualstudiesandmeasurementsmustthusbecombinedwithsophisticatedmodelingofcomplexsystems.

The indoor environment provides an immediate link toergonomics.Althoughoriginallyintendedtodealwithworksystemsonly,ergonomicshasnowanewdefi-nitionbytheInternationalErgonomicsAssociation:

Thescientificdisciplineconcernedwiththeunderstandingof theinteractionsamonghumanandotherelementsofasystem,andtheprofessionthatappliestheory,principles,data,andmethodstodesigninordertooptimizehumanwell-beingandoverallsystemperformance.

It is now concerned with human activities in general, including work, study,recreation,orrest.Therelationshipbetweenhumanbeingsandtheirenvironment,includingmachinesanddevices,isexperiencedthroughthesensesandperceptualmeasurementsarekeywaysforobtainingvaluablescientificandprofessionaldata.Atypicalergonomicconcernisthemeasurementofcomfort.Intransportationsystemsdiscomfortisoftenassociatedwithnoiseandvibrationexposuresinwhichcaseper-ceptionplaysacentralrole.Epidemiologicalorquasi-experimentalstudiesintheirvariousformsrelyonmeasurementwithpersonsastheirmaintool.

Ergonomicsaimsatensuringagoodqualityoflifeforoperatorsand,ontheotherhand,abestperformanceof thesystemconcerned.Consider thecaseofadriver:ensuringthatheorsheisworkinginoptimalconditionsispossiblythebestmeansofguaranteeingthesafetyofthepeoplecarried.Consideralsothecaseofawatchman:herehisorherperformanceaffectssecurity.

Securityisanotherimportantapplicationarea.Thecaseoffacerecognitionforthe identification of suspected persons may be briefly considered. So far, severalapproaches have been implemented for the automation of this task: comparing a

Y116134.indb 21 10/10/11 2:11 PM


pictureofasuspectwithadatabaseofcriminalsmaybetooheavyataskforahumanbeing.Atpresent,approachesrelatedtothepsychologyoffacerecognitionseemtobepromising (Wenger&Townsend, 2001).They are related tomultidimensionalmeasurementsandtoperceptualmodelsandaretreatedinChapter7.Forensicsci-enceisacloselyrelatedfield;amajorproblemthereisthereliabilityofeyewitnesstestimony,notbecauseofanywishtolie,buttofailuresinmemory.Thereareongo-ingstudiesinperceptionandmemoryformationthatmayresultinpracticalmethodsforassessingthereliabilityofeyewitnesses.

Clinicalapplicationsarealsoimportant.Themeasurementoftheintensityofpainmayhelp tooptimize treatments (Berglund&Harju, 2003); changes in sensorialsensitivitymaybeusedindiagnostics(e.g.,decreasedsmellsensitivityasanearlywarningsymptomforAlzheimer’sdisease)or for themonitoringofrehabilitationprocesses.Last,humanoidroboticsaimsatdevelopingmachinesthatresemble,tosomeextent,someaspectofhumanbehavior.Theymustbeequippedwithsophisti-catedsensorinterfacesthatmimiccertainaspectsofhumanperceptionandmaybeusedinrehabilitationandspecialassistanceprograms.

1.10 Final remarks

Wearenowattheendofthisshortjourneyalongthedevelopmentofmeasurementacrossthephysicalandbehavioralsciences.Wehavepaidspecialattentiontotheschismofthe1930sthatimpededcoordinatedprogressinthevariousdisciplinesinvolved.Wehavetriedandexplicatedthetermsofthedisputeinsomedetailandwehaveshownhowthesuccessivedevelopmentofmeasurementscience,initsvariousaspects,hasessentiallycontradicted theconclusionsof theBAASreporton the impossibilityoffindingacommongroundforinvestigation.Contactpointsbetweenmeasurementinphysicsandinpsychologyarenowmoresubstantialthandifferences,inouropinion,andtherearesignsofinitiativestorebuildthebridgebetweenthedisciplines.

Thecaseofphysiologicalquantitiesisemblematic:theultimatefoundationforthemeasurementofluminousintensity,forexample,issummarizedinasetofexper-imental curves that basically express the perceptual response of persons to someaspectsoflight,ascollected,checked,andfinallyaccepted,inathoroughexperi-mentcarriedoutatthebeginningofthelastcentury(BIPM,2009).

Wehavealsoseenthatsoundscientific,social,andeconomicreasonspushnowforaconcertedeffortbythescientificcommunitytofacethechallengeofmeasure-mentwithpersons.Wearenowatthebeginningofapaththatwillrequirepropersteps.Wetrytofigureoutthreeofthem(Rossi&Berglund,2011).

Thefirststepmaysomehowbethehardest;itrequiressomechangeinthecon-ventionalattitudesofbothparties.Physicistsandengineersshouldperhapsbemoreopen toaccept thatmeasurementmayalsobeperformed through thehumanper-ceptionandinterpretation,although,obviously,inproperlydesignedandconductedexperiments.Thismaycontrast somewhatwith the traditionaleducationofmanythatofteninterpretstherequisiteof“objectivity”asabsenceofhumanintervention,ratherthanascollective(i.e.,intersubjective)agreementonmethodsandprocedures.Ontheotherhand,psychologistsandbehavioralscientistsshouldperhapsdevelopa

Y116134.indb 22 10/10/11 2:11 PM


greatersensitivitytothebenefitofaninternationalorganizationsupportingthemea-surementof,atleast,somekeyperceptualquantities,suchas,forexample,loudnessorodorintensity.Someofthequality-controlproceduresoftheinternationalsystemofmetrology,suchasintercomparisons,couldperhapsbeapplied,afterproperadap-tation,tothequalificationofmethodsformeasuringperceptualquantities.

Anothermajorstepwouldconsistinmakingajointefforttodevelopacommontheoryformeasurement.Thiswouldalsogreatlyhelpinachievingacommonlan-guage:oncecommonideasaredeveloped,thewordsforphrasingthemwillbefoundaswell.

Last, the third step (logically, not necessarily temporally) would be to worktogetheroncommonprojects.Anexcellentopportunityinthisregardhasbeenpro-videdbytheMtICallandinparticularbytheassociatedMINETAction.Thisbookisaresultofsuchacollaborationanditishopedthatitwillinaugurateaneweraforthissubject.Alargebandofscholarsandscientistsfromphysical,psychologi-cal,biological,andsocialscienceshaveacceptedthechallengeofworkingtogetherto reachacommonunderstandingofmeasurement theoryandmethods.Theaimistoprovide,seemingly,thefirstbookeverissuedcoveringthetopicofmeasurementwithpersonsthroughamulti-andinterdisciplinaryapproach.Thecomplementaryaspects of general theory, measurement methods, instrumentation, and modelingaretreatedtogetherbyscientistsintherelatedfieldsofpsychophysicsandgeneralpsychology,measurementtheory,metrology,andinstrumentation,neurophysiology,engineering,biology,andchemistry.

Thebookisorganizedintwoparts.Inthefirst,generictheoreticalandmethod-ological issuesare treated, including theconceptualbasisofmeasurement in thevariousfieldsinvolved, thedevelopmentofformal,representational,andprobabi-listictheories,theapproachtoexperimentationandthetheories,models,andmeth-ods for multidimensional problems. In the second, several implementation areasarepresented,includingsound,visual,skin,andodorperception,functionalbrainimagining,bodylanguageandemotions,and,finally,theuseofmeasurementsindecisionmaking.

References

Bentley, J. P. (2005). Principles of measurement systems (4th ed.). Harlow, Essex, UK:PearsonEducation.

Berglund,B.,(1991).Qualityassuranceinenvironmentalpsychophysics.InS.J.Bolanowski,& G.A. Gescheider (Eds.), Ratio scaling of psychological magnitudes. Hillsdale, NJ:Erlbaum.

Berglund,B.,&Harju,E. (2003).Masterscalingofperceived intensityof touch,coldandwarmth.European Journal of Pain,7,323–334.

BIPM.(1983).Principles governing photometry.Luisant:ImprimerieDurand.BIPM. (2001). Proceedings of the 21st General Conference on Weights and Measures,

11–15October.BIPM.(2003).Guidelines to CIPM key comparisons.BIPM.(2006).The International System of Units(8thed.).Paris:STEDI.

Y116134.indb 23 10/10/11 2:11 PM


BIPM.(2008a).Evaluation of measurement data—Guide to the expression of uncertainty in measurement.(JCGM100:2008).

BIPM. (2008b). International vocabulary of metrology—Basic and general terms (VIM).(JCGM200:2008).

BIPM.(2008c).Mutual recognition.Paris:STEDI.BIPM.(2009).BIPMWorkshoponPhysiologicalQuantitiesandSIUnits.Online.Available

HTTP:http://www.bipm.org/en/events/physiological_quantities/(accessed14April2011)BIPM. (2010). The new SI. Online. Available HTTP: http://www.bipm.org/en/si/new_si/

(accessed14April2011)Bronzino,J.D.(Ed.).(1995).The biomedical engineering handbook.BocaRaton,FL:CRC

&IEEEPress.Campbell,N.R.(1957).Foundations of science.NewYork:Dover.(Originalwork,Physics—

The elements,published1920)De Coensel, B., Botteldooren, D., Debacq, K., Nilsson, M. E., & Berglund, B. (2008).

Clustering outdoor soundscapes using fuzzy ants. Paper presented at the 2008 IEEE World Congress on Computational Intelligence, WCCI 2008,HongKong.

Dimberg,U.(1990).Facialelectromyographyandemotionalreactions.Psychophysiology,27,481–494.

EuropeanCommission.(2007).Measuring the impossible.(EUR22424).EuropeanCommunities.Fechner, G. T. (1966). Elements of psychophysics (Vol. I). NewYork: Holt, Rinehart and

Winston.(Originalworkpublished1860)Ferguson, A., Myers, C. S., & Bartlett, R. J. (1940). Quantitative estimation of sensory

events—FinalBAASReport.Advances of Science,2,331–349.Finkelstein, L. (1982).Theory and philosophy of measurement. In P. H. Sydenham (Ed.),

Handbook of measurement science(Vol.1,pp.1–30).Chichester,UK:Wiley.Finkelstein,L.(2005).Problemsofmeasurementinsoftsystems.Measurement, 38,267–274.Finkelstein,L.(2008,September).Problemsofwidely-definedmeasurement.Paperpresented

at the 12th IMEKO TC1 and TC7 Symposium on Man, Science and Measurement,Annecy,France.

Gescheider,G.A.(1997).Psychophysics: the fundamentals(3rded.).London:Erlbaum.Goodman,T., Montgomery, R., Bialek,A., Forbes,A., Rides, M.,Whitaker,T.A., Overvliet,

K.,McGlone,F.,&vanderHeijden,G.(2008,September).Themeasurementofnatural-ness.Paperpresentedatthe12th IMEKO TC1 and TC7 Symposium on Man, Science and Measurement,Annecy,France.

Helmholtz,H.(1971).Anepistemologicalanalysisofcountingandmeasurement.InR.Karl(Ed.andTrans.),Selected writing of Hermann Helmholtz.Middletown,CT:WesleyanUniversityPress.(Originalworkpublished1887)

Iverson,G.,&Luce,R.D.(1998).Therepresentationalmeasurementapproachtopsychophys-icalandjudgmentalproblems.InM.H.Birnbaum(Ed.),Measurement, judgment, and decision making (pp.1–79).NewYork:AcademicPress.

Krantz,D.H.,Luce,R.D.,Suppes,P.,&Tversky,A.(1971).Foundations of measurement(Vol.1).NewYork:AcademicPress.

LordF.M.,&Novick,M.R. (1968).Statistical theories of mental test scores.Reading,MA:Addison-Wesley.

Luce,R.D.,Krantz,D.H.,Suppes,P.,&Tversky,A.(1990).Foundations of measurement—Representation, axiomatization and invariance.NewYork:AcademicPress.

Mills,J.M.,Mohr,P.J.,Quinn,T.J.,Taylor,B.N.,&Williams,E.R.(2006).Redefinitionofthekilogram,ampere,kelvinandmole:AproposedapproachtoimplementingCIPMrecommendation1(CI-2005).Metrologia, 43,227–246.

Y116134.indb 24 10/10/11 2:11 PM


MuravyovS.,&Savolainen,V.(2001).Specialinterpretationofformalmeasurementscalesforthecaseofmultipleheterogeneousproperties.Measurement, 29,209–224.

Nelson,R.A.,&Ruby,L.(1993).PhysiologicalunitsintheSI.Metrologia, 30,55–60.NowellJones,F.(1974).Historyofpsychophysicsandjudgement.InE.C.Carterette,&M.P.

Friedman(Eds.),Handbook of perception (Vol.2).NewYork:AcademicPress.Pointer,M.R.(2003).New directions—Soft metrology requirements for support from math-

ematics statistics and software(NPLReportCMSC20/03).Roberts,F.S.(1979).Measurement theory, with applications to decision-making, utility and the

social sciences.Reading,MA:Addison-Wesley.DigitalReprinting(2009).Cambridge,UK:CambridgeUniversityPress.

Rossi,G.B.(2006).Aprobabilistictheoryofmeasurement.Measurement, 39,34–50.Rossi,G.B.(2007).Measurability.Measurement,40,545–562.RossiG.B.(2009a).Cross-disciplinaryconceptsandtermsinmeasurement.Measurement,

42,1288–1296.Rossi,G.B. (2009b)Measurementofquantities related tohumanperception. InF.Pavese

etal.(Eds.),Advanced mathematical and computational tools in metrology and testing (AMCTM VIII) (pp.279–290).Singapore:WorldScientific.

Rossi,G.B.,&Berglund,B.(2009,September).Measurementrelatedtohumanperceptionandinterpretation—Stateoftheartandchallenges.PaperpresentedattheXIX IMEKO World Congress,Lisbon.

Rossi,G.B.,&Berglund,B.(2011).Measurementinvolvinghumanperceptionandinterpre-tation.Measurement,44,815–822.

Schweizer,B.,&Sklar,A.(1983).Probabilistic metric spaces.NewYork:NorthHolland.Stevens,S.S. (1936).A scale for themeasurement of psychologicalmagnitude, loudness.

Psychological Review, 43,405–416.Stevens,S.S.(1946).Onthetheoryofscalesandmeasurement.Science,103,677–680.Stevens, S. S. (1956). The direct estimation of sensory magnitudes: Loudness. American

Journal of Psychology,69,1–25.Stevens,S.S. (1959).Measurement,psychophysicsandutility. InC.W.Churchman,&P.

Ratoosh(Eds.),Basic concepts of measurements(pp.1–49).Cambridge,UK:CambridgeUniversityPress.

Suppes,P.,Krantz,D.H.,Luce,R.D.,&Tversky,A.(1989).Foundations of measurement—Geometrical, threshold and probabilistic representations.NewYork:AcademicPress.

Thurstone, L. L. (1927). A law of comparative judgements. Psychological Review, 34,273–286.

Wallard,A.(2009,September).Worldmetrology—Thenexttenyears.InvitedpaperpresentedattheXIX IMEKO World Congress,Lisbon.

Wallard,A.(2011).NewsfromtheBIPM—2010.Metrologia,48,59–67.Wenger,M.J.,&Townsend,J.T.(Eds.).(2001).Computational, geometric and process issues

in facial cognition: Progress and challenges.Hillsdale,NJ:Erlbaum.Wozniak,R.H.(1999).Classics in psychology, 1855–1914: Historical essays.Bristol,UK:

ThoemmesPress.Wyszecki,G.(1982).Color science.NewYork:JohnWiley&Sons.

Y116134.indb 25 10/10/11 2:11 PM

Y116134.indb 26 10/10/11 2:11 PM

27

Measurement in psychology

Birgitta BerglundDepartmentofPsychology,StockholmUniversityandInstituteofEnvironmentalMedicine,KarolinskaInstitutetStockholm,Sweden

2.1 Introduction

Inpsychology,therearetwomainschoolsofmetrology:psychophysicsandpsycho-metrics.Theformerisgroundedinphysicsandthelatterinstatistics,butbothpre-sumethatattributesofpsychologicalexperiencemaybeviewedasquantities,which,inturn,mayberepresentedbynumbers.Inpractice,psychophysicshasutilizedtheassumptionofa“standardobserver.”Thiswouldmeanthatsingleindividualsareinprincipleviewedasmeasuringinstruments(persons)thatmaybeusedformeasuringcertainperceptual/evaluativefeaturesoftheenvironmentorofotherpersons(e.g.,detection,discrimination,perceivedintensity,pleasantness,naturalness,friendliness,andsoon;“measurementwithperson(s)”).Inpractice,psychometricshasutilizedtheideaofa“standardstimulus.”Stimuliorsetsoftestitemsareinprincipleviewedasameasuringinstrument(psychologicaltest)thatmaybeusedformeasuringmainlymentalprocessesofsingleanduniqueindividuals(e.g.,emotions,personality,politi-calattitudes,psychomotorskills,etc.;“measurementofperson(s)”).

Whereasthemainfocusofpsychometricshasbeentomeasureinterindividualdif-ferencesinpsychologicalprocesses,psychophysicshasfocusedonutilizingsimilari-tiesinsensoryfunctionsformeasuringperception,interpretation,andevaluationofcomplexphenomena.Earlyapplicationsofpsychophysicswereprimarilyconcernedwithinterindividualdifferencesinsensoryfunctions,forexample,diagnosingspe-cificcolorblindnessandfittingeyeglassesordeterminingthedegreeofhearinglossandfittinghearingaidsordiagnosingchronicpainconditions.

Metrologyschoolsgroundedintheoriesofpsychophysicshavealsodevelopedinotherfieldsofscience,includingneuroscience,ergonomics,sensoryanalysis,robot-ics, acoustics, optics, and aesthetics. Similar to psychometrics, statistically basedmetrologyschoolsarefoundinbiometrics,chemometrics,politicalscience,sensoryanalysis,sociology,epidemiology,mathematicalpsychology,assessmentdesigners,marketing,andeducation,amongothers.Thischapterintroducesthemainpsycho-physicalmeasurementtechniques,whichtheauthorandmanyothershavefounduse-fulinapplicationareasinsideandoutsidepsychology.Althoughmeasurementwith

2

Y116134.indb 27 10/10/11 2:11 PM


personsandmeasurementofpersonsrequirewell-designedexperiments,theissuesofexperimentaldesignarenotdiscussedhere,nor is themethodologicalresearchonthemethodsthemselves(forintroductions,see,e.g.,Cook&Campbell,1979andGescheider,1997,respectively).

2.2 Perceptual, hedonic, and multisensory constructs

Psychophysicsprovidesmanymethodsandcomplementarytechniquesformeasur-inghumansensoryperceptionsuchasperceived intensityandperceived quality.Thedifferentmethodsandtechniquesarelinkedtocertainkindsofconstructstobemea-sured.It is important tokeepperceptual(orphysical)constructsseparatebecausetheyrefertodifferentkindsofabilitiesofoursensory–perceptualsystems.Hedonicandmultisensoryconstructsarealsomeasuredwiththesametechniques.

Basicperceptual constructsarelinkedtohowhumansensorysystemsarefunc-tioning or what they can do in terms of measurement. Because the methods formeasuringperceptiondeliverinformationspecifictoasensorymodality,theexam-plesshownhereareprimarilyfromhumanolfaction.Theseareodordetection(orabsoluteodorthreshold),odor-qualityidentification(orrecognition),odor-intensitydiscrimination,odor-qualitydiscrimination,odor-intensityquantification,andodor-qualityclassification,mapping,orlabeling(datacollectionconcerningcategoriza-tion, similarities/dissimilarities, andodor-qualityprofiling, respectively).Someoftheseconstructs requireone-pointmeasurements(absoluteodor threshold);othersmeasurequantity(odorintensity)orcategorizeandmapodors(odorquality).

Thehedonic constructsarealsobasicandincludehowhumanemotionsaffectand are integrated in sensory–perceptual processes. Examples are likes/dislikes,pleasantness/unpleasantness, preferences, and similar kinds of “emotional-evaluative”attributes(beautiful,tasty,joyful,etc.).Hedonicconstructsareofcoursegroundedinsensory–emotionalcoprocessingacquiredthroughperceptuallearning,andfeelingswouldbeasimmediateasthesensoryperceptions.Hedonicevaluationsinvolveperceptionandcognition,learning,memory,andlastbutnotleast,emotions.Onewayofunderstandingahedonicconstructistoviewemotionsasbeingattrib-utedtoobjectsoreventsperceivedwiththeaidofsensorysystems.Toexemplify,redisaperception,likeodor,whereasfavoringaredcolorofaparticularcarpetoranodorofaparticularperfumeisamatterofpersonalpreference(=hedonicevalua-tion).Wewouldallagreethatweseered,butallwillnotagreethatredisabeautifulcolor.Thesameistrueforperfumesorairdeodorantsorindeedindoorairqualities.Itfollowsthattherearelargeinterindividualdifferencesinmeasurementsofhedonicconstructsascomparedtosensory–perceptualconstructs.

Athirdsetofconstructsisthemultisensory–perceptualconstructs,suchasthe perceptionof indoorairquality.Obviously, this isamorecomplexconstruct thanodorquality,whichoriginatesinonesensorymodality.Perceivedairqualityinvolvesmanykindsofsensoryperceptionsapartfromodor[heregivenwithinbrackets]andalltheseareinterpretedandevaluated.Thephysicalorchemicalinputs[perceptions]are temperature [warmthorcold],airmovement [cooling,warming,active touch],humidity[dryness,wetness],andirritants[sensoryirritation,pain].Nearlyallodorous

Y116134.indb 28 10/10/11 2:11 PM

Measurement in psychology 29

substancesinindoorairhavethepotentialtoproducesensoryirritation(e.g.,Kobal&Hummel,1991),butatmuchhigherconcentrationsthaniscommoninindoororoutdoorair.Therehasbeen renewedscientific interest inmultisensoryperception(e.g.,Culvert,Spence,&Stein,2004). Multisensoryperceptual interpretationsandevaluationsshouldberecognizedasmeasuringabilitiesuniquetohumanbeings.

2.3 Measurement with persons require experiments or other study designs

Measurement of perceptual, hedonic, and multisensory constructs (each assumedtobeone-dimensional), is conductedwithpersonsas themainmeasuring instru-ment.Thismeansthatagroupofpersonshastobeselectedwhoindividuallyorasapanel constitute themeasuring instrumentof theperceptual, hedonic, ormulti-sensoryevaluations.Similarly,asampleofstimulihastobeprovided,whichspecificphysical–chemicalpropertieshavealsotobedefinedasrelevantandmeasuredsepa-rately.Dependingonwhatkindofstudyistobeconducted,eitherwell-controlledstimuli (odoroussubstances,materialemissions)areprovidedinapsychophysicalexperimentwithpersonsinalaboratoryornaturalstimuli(indoorair)areprovidedinaquasi-experimentwithvisitorsofbuildings(fieldstudy).Inthelattercase,theenvironment(location,timeoftheday,ventilationrate,temperature,humidity,etc.)iswellcontrolledandmeasured inparallelwith thevisitors’perceptual,hedonic,andmultisensorymeasurements(alternatively,residentsoroccupantsmayconstituteapanel).Ofteninpsychologicalresearch,theconceptofmeasurementisreplacedbymoreimprecisewordssuchasassessmentandevaluation.

AsillustratedinFigure 2.1,themethodselectedformeasuringperceptionswiththeaidofindividualsorpanelsisbutoneofthenecessarycomponentsforthepsy-chophysicalexperimentor thequasi-experiment infieldsituations.Thequalityoftheolfactometer,thesampleofstimuliandthepanel(sampleofpersons)areallveryimportantingredientsasaretheinstructions,experimentalprocedure,andtypesofdatatreatmentsplannedinordertoconstructthescalesfromthesetsofcollecteddata(e.g.,assumptionsontheformofpsychophysicalfunctionandtypeofdistribu-tionmodelsappliedforuncertainty).Thekindsofscaledeliveredbythepanel(nom-inal, ordinal, interval, or ratio) arevital to subsequent statisticalormathematicalmodelingofperceptual/hedonicorpsychophysicaldata(e.g.,S.S.Stevens,1946).

Measurement in psychology has developed in parallel with the developmentwithin thegeneralphilosophyofscienceandmeasurement inphysics. ImpressiveknowledgeisfoundinTorgerson(1958),LordandNovick(1968),BairdandNoma(1978),Roberts(1979),Falmange(1985),andGescheider(1997).

2.4 Psychophysical methods

2.4.1 What is measured?

The psychophysical experiment may be used for measuring various functions ofhumansensory perception.Basicaspectsare:

Y116134.indb 29 10/10/11 2:11 PM


1. Identification:Meredetectionorrecognition(presenceofphysicalquantity) 2. Discrimination: Minimal change in perceived intensity (certain physical

quantity) 3.Perceived Intensity: One-dimensional scaling of a quantity (perceptual

quantity) 4.Perceived Quality: Sorting, clustering, or multidimensional scaling

(mappingqualities)

Identification,discrimination,andperceivedintensityareallpossibletomeasurewithpsychophysicalmethods.Strictly,thiswouldmeanmeasuringperceptualcon-structs in termsofaphysicalquantity. Identification isaone-pointphysicalmea-surement(threshold),whereasdiscriminationisatwo-pointphysicalmeasurement(minimaldistance).Thepsychophysicalmethodformeasuringperceivedintensityistheequal-intensitymatchingoftwodifferentqualities(ofthesameordifferentsen-sorymodalities)atdifferentintensitylevels.Alsoperceivedqualityhasbeenmea-suredpsychophysically.Forexample,thecolortrianglewasoriginallyconstructedwiththepsychophysicalmethodofperceived-qualitydiscriminationcombinedwith

Type ofolfactometer

Selectionof panel

Stimulussample

Measuringexposure

Measuringpanel

performance

Measuringperformance

Measuring perceptions/hedonicskind of scale

(ordinal, interval, ratio)

Modelingpsychophysical

functions

Modelingperceptual/hedonic

processes

Statistical and/or neutralnetwork modeling of perceived

quality or odor quality

Method for measuringperception/hedonics

Experimental or Quasi-Experimental Design

Figure 2.1 Illustration of relevant components of experiments or quasi-experimentsdesignedforperceptual,hedonic,ormultisensorymeasurementswithpersonsasmeasuringinstruments.

Y116134.indb 30 10/10/11 2:11 PM


wavelengthsrepresentingperceivedqualities(=colors).Thepsychophysicalmethodsdescribedaboveallmeasureaperceptualconstructonaphysicalscale.Forexample,the method of equal-intensity matching would provide equal-loudness functionswithdifferentslopesfortonesofdifferentfrequencies.Evenifseveraldifferenttoneswerematchedtothesame1,000-Hztone,itwouldnotbeappropriatetoinferthatacertaindB-increaseofseveraltonesactuallycorrespondstoaninvariantincreaseinloudness(“perceivedintensity”ofsound).

Sections2.4.2,2.4.4,and2.4.6presentpsychophysicalmethodsofmeasurement,whichallmeasurevariousaspectsofperceptions in termsofaphysicalquantityandunit.

2.4.2 Detection or recognition

To measure perceptual detections is a psychophysical problem. A prerequisite isthatpeopleareputintosituations,typicallyexperiments(olfactometerandclean-airbooth)butalsofieldstudies,wheretheymaybestimulatedandbecomeawarethat,forexample,anolfactorystimulusispresent.Therearetwoprincipaldetectiontheories:the“traditional”absolutethresholdtheory(e.g.,Engen,1971)andthesignaldetectiontheory(SDT,e.g.,Green&Swets,1962).Notably,thethreebasicmethodsformeasur-ingtheabsolute detection thresholdwereoriginallycontributedbyGustavTheodorFechnerhimself.Signaldetectiontheorygrewoutofthedevelopmentofinformationtheoryinmathematicalstatisticsandfromworkonelectroniccommunications.

Odordetectionexperimentsrequireanadvancedprecisionolfactometerbywhichsmall steps of low concentrations can be produced repeatedly and presented tohumansinwell-controlledsamplesforinhalations(Berglund,Berglund,&Lindvall,1986).Ingeneral,ithasbeenfoundthattheabsolutedetectionthreshold,whichisexpressed as a concentration or dilution value, varies widely with chemical sub-stances (Devos,Patte,Rouault,Laffort,&vanGemert,1990). It alsovarieswiththe method selected for determining the threshold. The method of limits usuallygiveslowerthresholdsthanthemethodofconstantstimulus.Recognitionthresholds(includingidentificationofodorquality)aregenerallyhigherthanthemeredetectionthresholdsof“something”unidentified.

Theabsoluteodorthresholdisameasureofoneperson’ssensitivitytoanodoroussubstance.Asinothersenses,forinstance,visionandhearing,theinterindividualdifferencesinolfactorysensitivitymayalsobelarge.Typically,thresholddistribu-tionsfordifferentindividualsarepositivelyskewedevenifexpressedinlogscaleofconcentration(Figure 2.2).Itisnotknowniftheabsoluteodorthresholdsforalargersetofodoroussubstanceswouldrankthesamefordifferentpersonsbelievedtobenormosmics.Inepidemiology,themethodoflimitsissometimesusedfordetermin-ingtheeffectivedose50(ED50)forapopulationofpeopleoranimals,forinstance,fortestingtheimpactsofdrugsorodorousairpollutants(e.g.,WHO,1989,formal-dehyde).Thepersonsoranimalsarethenassumedtobeinterchangeable(invariantdetector)andaGaussiandistributionofindividualthresholdsistypicallypostulated.ED50isinappropriatetouseforsmallsamplesofpeoplebecauseitreliesonstatisti-callyrepresentativerandomsamplesofatargetpopulation.

Y116134.indb 31 10/10/11 2:11 PM


2.4.2.1 The method of limits

Themethodoflimitsmeasuresjustnoticeabledifference(JND)orminimalchangeandcanbecharacterizedasserialexploration.Itisthemostdirectmethodforlocat-ingthethreshold.Thetaskof theobserver is todetect thepresenceofastimulus(odoroussubstance)inaseriesofsmallconcentration(ordilution)steps(ca.18–20)inanascendinganddescendingpresentationorder.The“one-point”ofthemomen-taryabsolute thresholdcanbeestimateddirectly fromeachseriesaccording toapredefined rule, for example, in ascending series two detections in a row and indescendingseriestwonondetectionsinarow.Thestartingconcentrationshouldbeirregularlyvariedbetweentrials.Themethodgivesafairlyquickmeasurementoftheindividual absolute thresholdcalculatedasthemeanoftheresponsedistributionmeasuredinconcentration(ordilution)valuesfromeachtrial.Thestandarddevia-tionorstandarderroristypicallyusedastheuncertaintymeasure.

Figure 2.2showsadistributionofindividualabsolutethresholdsforformaldehydeodordeterminedbythemethodoflimits.Eveninalogarithmicscaleofconcentra-tion,theabsolutethresholddistributionforagroupofpersonsistypicallypositivelyskewed(Ahlström,Berglund,Berglund,Lindvall,1986).Figure 2.2 thereforealsoshowstheED50measurefordeterminingtheabsolute thresholdforagroupof64persons(effectivedosefor50%ofthe64persons).Thisisnotanoptimalpopulationestimateoftheformaldehydeabsolutethresholdbecausethedistributionisnotsym-metricalornormalandthesampleistoosmallandconsistsofvoluntaryobservers.

2.4.2.2 The methods of adjustment

Themethodofadjustmentmeasurestheaverageerrorandthe“reproduction,”anditcanbecharacterizedastheequationmethod.Theobserverhimself typicallyadjusts

Absolute Order reshold Concentrationof Formaldehyde in Log ppb

01.0 2.0 3.0

n = 64

2Num

ber o

f Sub

ject

s4

6

8

10

Figure 2.2 Absoluteodorthresholdsofformaldehydedeterminedbythemethodoflimitsforeachof64persons.Thedistributionof individual thresholds isgiven ina logarithmicscaleofconcentrationinppb.ReprintedfromAhlströmetal.(1986).

Y116134.indb 32 10/10/11 2:11 PM


theconcentrationvalueofastimulus(odoroussubstance),whichcanbevariedcontinu-ously,andrepeatedlysetsittoavaluethathejudgestobethelowestthathecandetect;theaverageofthesesettingsistakentobetheindividualabsoluteodorthreshold.Thedifferentstartingconcentrations,bothbelowandabovetheexpectedthreshold,arepre-sentedinirregularorderbytheexperimenter.Similartothemethodoflimits,thestimu-lusconcentrationissystematicallyincreasedordecreaseduntilachangeinresponseoccurs(odorornoodordetections).Thedifferenceisthattheobserveradjuststhecon-centrationcontinuouslyinthemethodofadjustment,whereastheexperimenterchangestheconcentrationinpredeterminedinvariantdistinctstepsinthemethodoflimits.

Themethodofadjustmentgivesameasurementoftheindividualabsolutethresh-oldcalculatedasthemeanoftheresponsedistributionmeasuredinconcentration(ordilution)valuesfromeachtrial.Thestandarddeviationorstandarderroristypicallyusedas theuncertaintymeasure.Themethodofadjustment isnotwellsuitedfordeterminingodorthresholdsbecauseofthedifficultiesofchangingodorconcentra-tionscontinuouslybyanolfactometer.

2.4.2.3 The method of constant stimuli

The method of constant stimuli is used to determine right and wrong cases andcanbecharacterizedasthefrequencymethod.Aseriesofconcentrationstepsarechosen,similarlytothemethodoflimits.However,insteadofpresentingthecon-centrationstepsinsystematicorder, theyarenowpresentedrepeatedlyinrandomorder.Theobserver’staskissimple.Sheshouldrespond“Yes”ifanodorispresentor“No”iftheodorisnotpresent.Aforcedchoiceprocedure(whichdoesnotallow“doubtful”) is usually used. Two kinds of recommended procedures are in com-monuse:(a)randompresentationoftheconcentrationstogetherwithblanks(cleanair),and(b)randompresentationoftheconcentrationsinoneoftwo,three,orfourmarkedtimeintervals;theotherintervalsarethenblanks(cleanair).

Inthefirstcase,chanceperformance(guessing)isdeterminedfortheblanks(ca.33%ofallpresentations)andusedforestimatingtheactualdetectionperformanceforeachconcentration.Inthesecondcase,chanceperformanceisdeterminedas50%correct,33%correct,or25%correctfortwo,three,orfourintervals,respectively.

In the first procedure, a psychometric curve is determined for each observerbyplottingtheproportionofcorrectdetections(Pc)foreachoftheconcentrations(10–18steps)afteradjustingthehitrate(PHits)forfalsealarms(PFA).Theabsolutethreshold(PcinEquation(2.1)),

PP P

Pc

Hits FA

FA

= −−1

, (2.1)

isdeterminedastheconcentrationcorrespondingtothe50%probabilityofcorrectdetection(P50).Thequartilerange(P75–P25)expressedinconcentrationistypicallyusedforestimatinguncertainty.Inthesecondprocedure,theprobabilityofdetectionabovechancelevelisdetermined.Theabsolutethresholdmaythenbedefinedastheconcentrationat75%probabilityofdetection,if50%ischanceperformance.

Y116134.indb 33 10/10/11 2:11 PM


2.4.3 Signal detection theory and choice theory

Thereisatheoreticalmathematicalsystemthatformallydealswithbothdecisionalandsensorycomponentsindetectionanddiscriminationtasks.Insignaldetectiontheory,thedistancebetweenthemeansoftheabsentsignal(noise)andthepresentsignal(signalplusnoise)distributions(typicallyassumedtobeGaussiandistribu-tions) is themeasureofdetection.There isalsoa likelihoodcriterionsetbyeachobserver.Bothpositiveandnegativefalseresponsescanbeestimated(seeTable 2.1).A basic assumption is that no absolute detection threshold exists. The detectionsdependonhowwellthesignalcanbeseparatedperceptuallyfromthenoise.Inaddi-tion, theperformanceof individualobservers in thisperceptual“separation” taskalsodependsontheirindividuallikelihoodcriterioninrespondingyesorno(forcedchoicedesignisused).

Typically, the measure of sensory sensitivity selected is d ′ of signal detectiontheory.ItrepresentsthedistanceonanexcitationcontinuumbetweenthemeansoftheGaussiandistributionsfortheabsentsignal(“sensorynoise”)andforthepres-entsignal(signal+“sensorynoise”;Green&Swets,1962).Analternativebutverysimilarprocedure fordetermining identification (simultaneousdetectionand rec-ognition)ofastimulusisLuce’schoicetheory(Luce,Bush,&Galanter,1963).Thesignaldetectiontheoryrequireshigh-qualityequipmentforrapidrepetitivestimula-tionofodoroussubstances/mixturesandcleanair(blanks).Berglund,Berglund,andLindvall(1974)determinedtherelationshipbetween(a)thed ′measureandsignalstrength(hydrogensulfideconcentrationinmg/L),and(b)thed ′measureandthequantityofperceivedodorintensityobtainedbythemethodofmagnitudeestimation(seeSection2.4.7.3).Theyfoundthatd′increasesasalinearfunctionoftheloga-rithmforstimulusconcentration,thatis,

d ′=k1+k2logS, (2.2)

whereSisthephysicalconcentration,d ′istheindexofdetectability,andk1andk2areconstants.Inaddition,theyshowedempiricallythatthelogarithmofperceivedintensity(logR)increasesasalinearfunctionofthed ′measure;thatis,

logR=c1+c2d ′, (2.3)

Table 2.1 Outcomefrequenciesinaforced-choicesignaldetectionexperiment

Signal

Response

Yes No

Present Hit MissAbsent Falsealarm Correctrejection

Y116134.indb 34 10/10/11 2:11 PM


whereRisperceivedodorintensity,d ′theindexofdetectabilityofodor,andc1andc2areconstants.ThecombinationofEquations(2.2)and(2.3)showsthattheperceivedodorintensity(R)increasesasapowerfunctionofstimulusconcentration(S);thatis,

logR=a+nlogS (2.4)

orinlinearscales,apowerfunctionwithamultiplicativeconstantandanexponentn (seeEquation(2.7)).

Thepracticaluseofthedetectiontechniqueislimitedbecausethemethodassumeserrorvariationinthesubject’sjudgments.Thesignalhastobesoweakthatitistosomeextentconfusedwiththenoise,withoutatthesametimecausingtheexperi-ment to become unreasonably large. The detection measure d ′ is well suited fordescriptionsofrapidconcentrationchangesatextremelylowodorlevels.Comparedtoabsolutethresholdmeasurements,d ′determinationsappeartobesuperiorfromatheoreticalpointofview,astheyareindependentofvariationsintheresponsecrite-rionofthesubject,duetoalteredguessingtendencies,carelessness,lackofmotiva-tion,ortheeffectofexpectations.

2.4.4 Intensity discrimination

In determining discrimination or the difference threshold for intensive continua,thesamegeneralpsychophysicalprocedureisfollowedasfortheabsolutedetectionthreshold.Theprincipaldifferenceisthatoneachtrial, twostimuliarepresentedforcomparison,ascontrastedtoonestimulusinthetypicalabsolutethresholdpro-cedure.Severalseriesofascendinganddescendingconcentrationsarerunandtheaverageconcentrationdifferencesofonestandardconcentrationarecalculated.Intheexperiment, the timeorderorpositionof thestandard is typicallyvariedran-domly,asisthestartingconcentrationofthecomparisonstimulusineachseries.Asfortheabsolutethreshold,versionsofthemethodoflimits,methodofadjustments,and method of constant stimulus are all also used for determining the minimal-intensitydiscriminationatdifferentconcentrationlevels(=standardstimulus)ofthesameodoroussubstance.

2.4.4.1 Method of limits

Observersarepresentedwithpairsofstimuli;oneistheinvariantstandardstimu-lusandtheotherthevariablecomparisonstimulus.Insteadofdetectionornot,theobserversarenowaskedtorespondwhichofthepairis“larger,”“smaller,”or“simi-lar.”Inthemostcommonversion, theforcedchoiceprocedure,onlytheresponsealternatives“larger”or“smaller”areused.Inthemethodoflimits,thecomparisonstimulusischangedsystematicallyinverysmallstepsinaseriesofincreasingordecreasingconcentrationsofanodoroussubstance(ordilutedemission).Thestan-dardstimulusconcentrationisselectedtobeinthemiddleoftheseseries.Thedataaretheconcentrationvaluesofthecomparisonstimuluswheretheobservershifts

Y116134.indb 35 10/10/11 2:11 PM


fromoneoftheresponsecategoriestoanother.Statistically,thepointofsubjectiveequality(PSE)andthevariabilityofthecomparisonconcentrationsareusedtocalcu-latethedifferencelimen(seeSection2.4.3.3)atdifferentstandardconcentrations.

2.4.4.2 Method of adjustment

Observersthemselvesadjustthevalueofacomparisonstimulus,whichcanbevariedcontinuouslyorinverysmallsteps,andthetaskistosetittothesameperceivedintensityas thestandardstimulusconcentration.This ismaderepeatedly,andthecentraltendencyandvariabilityofthesettingsarecomputed.Theaveragesettingisadirectindicationofthepointofsubjectiveequalityandthevariabilitycanbeusedtocalculatethedifferencelimen(ΔI).Thismethodisnotwellsuitedforinvestigatingthedifferencelimenforodorintensitybecauseoftheneedforcontinuousvariationofconcentrationbytheparticipant.

2.4.4.3 Method of constant stimuli

Forthedifferencelimen(ΔI),eachofseveralfixed,discretevaluesofacompari-son stimulus is compared with the standard stimulus many, many times, and therelativefrequencyofthedifferentresponses,forexample,“smaller”or“larger”arecountedforeachofthesefixedvalues(=thestandard).Ifonlytworesponsecatego-riesareused,theobserverwillberighthalfofthetimebyjustguessing,andhisorherdifferencelimenisthereforeusuallydefinedastheincrementordecrementthatwascorrectly judged75%of the time (=probability inbetween50%and100%);The50%standsforchanceperformance.Ifthreeresponsecategoriesareused(add-ing“equal”), themethodofconstant stimulibecomesverysimilar to themethodoflimits.

2.4.5 Weber’s law and Fechner’s JND scale

Thewell-knownWeber’slawdescribestherelationshipbetweenthedifferencelimenordifferencethreshold(ΔI)andthemagnitudeofthestandard(I):

∆I kI= . (2.5)

Weber’slaw,whichwasproposedin1834,iswellgroundedinpsychophysicalexper-iments,mainlyonweights.Fechner(1860),whobelieveditimpossibletomeasuresensationdirectly,acknowledgedWeber’slawandpostulatedthatiftheunits of sen-sationwereallofequalsize,thenitwouldbepossibletocountthemupandthusmeasuresensationindirectly.Hehimself,andothersbeforehim,hadnotedthatinorderforachangeinastimulustobecomejustnoticeable,afixedpercentagemustbeadded:thejust noticeable difference (abbreviatedasJND).

InaccordancewithWeber’slaw,Fechner’slawstatesthatthemagnitudeofsensa-tionorresponsevariesdirectlywiththelogarithmofthestimulusmagnitude(forthederivationpleaseseeBaird&Noma,1978;Falmange,1985),

Y116134.indb 36 10/10/11 2:11 PM


Sk

II

=

⋅

1

0

ln , (2.6)

whereS is themagnitudeof sensationa stimuluselicits (equalsnumberof JNDsabove0atabsolutethreshold),I/I0isthephysicalmagnitudeofthestimulus(inten-sity[I]relativetotheabsolutethresholdstimulusmagnitude[I0]),1/kistheinverseoftheWeberfraction(k=ΔI/I)andlnisthenaturallogarithm(logarithmtothebasee).Inthisequation,S=0forI/I0=1.Fechner’scontributionisrecognizedbecausehegavepsychologyawayofconstructingsensoryscales.AscalebasedontheJNDcanbeobtainedeitherbyaddingtheindividualJNDstepsthatareobtainedbyactualmeasurementorbyassumingthevalidityofthemoreconvenientlogarithmictypeofformula.AlthoughexperimentalresearchhasshownthatWeber’slawdoesnotholdfor very weak or very strong stimulus intensities, the logarithmic transformationmaystillbesatisfactoryforthemiddleranges,whichoftenareofgreatestinterest.However,withtheincreaseduseofcategoryscalingandmagnitudeestimationandthedevelopmentofS.S.Stevens’kindsofmeasurement levels (nominal,ordinal,interval, and ratio scales), much research has been devoted to criticize Fechner’stheoreticallyderivedlogarithmicfunction(e.g.,S.S.Stevens,1975).

2.4.6 Equal-intensity matching

2.4.6.1 Intramodality matching of perceived intensity

Thisisascalingprocedureinwhichtheobserveradjuststheintensityofastimu-lusqualityuntil itappears tobeequally intenseasanotherstimulusqualityfromthesamesensorymodality.Thus,intramodality matchingmeansthattheperceivedintensityofeachoftwo(odor)qualitiesofonemodality(olfaction)ismatchedtobeequallyintense.Theresultofeverymatchisregisteredasaconcentrationvalueforeach of the corresponding two odorous substances. Intramodality matching is inprinciplethesameasthemethodofadjustmentwhenusedfordeterminingodordis-criminationatdifferentconcentrationlevels(varianceisthen,however,ofintentnotlevel).Themostfamousexampleof intramodalitymatchingis theequal-loudnesscurvesdeterminedfordifferentfrequenciesofsoundinwhichthe1,000Hztonewasusedasthecomparisonandalltheotherfrequenciesasstandardtonesonebyone.ThesedataresultedintheA-filterbuiltintostandardsoundlevelmetersformeasur-ingtheA-weightedequivalentcontinuoussoundlevelindBLAeq.

Tospeedupdatacollectionwithsubjects,anodorintensitymatchingproceduregrounded in successive approximations was combined with a high-flow dynamicolfactometer (Svensson&Szczygiel,1974).Theobservers’ task isonly toanswerwhichismoreintense,areferenceoramatchingstimulus.Newmatchingstimulusconcentrationsaregivenaccordingtoanalgorithmthatbisectsintervalsupordownin successive trials.Thismethod iswell suited for testing consumerproducts forodorquality,forexample,biologicaltoiletsorkitchenfans.Thisisbutoneexampleofadevelopmentofthematchingprocedure.

Y116134.indb 37 10/10/11 2:11 PM


2.4.6.2 Cross-modality matching of perceived intensity

Thisisascalingprocedureinwhichtheobserveradjuststheintensityofastimulusuntilitappearstobeequallyintenseasanotherstimulusfromadifferentsensorymodality.Thus,incross-modalitymatching,observersareaskedtoadjustlevelsofonesensorymodalitytomatchvariouslevelsofanothermodality(e.g.,S.S.Stevens,1975).Forotheraspectsofthematchingprocedure,seeintramodalitymatchinginSection2.4.6.1.

2.4.6.3 Equal-intensity matching to a number scale of references

Kasche(2005)presentedanintramodalitymatchingmethodformeasuringtheodorintensityofmaterialsemissions.Theproposalisfirsttoconstructanequalintervalscaleofperceivedodorintensityofareferencesubstancegroundedinsevenconcen-trationsofacetone.Thuseachofthesevenperceivedodorintensitiesofacetoneisfirstassignedfixednumbers(thesameforallobservers)torepresentthepsychophysicalfunctionofodorintensityasafunctionofacetoneconcentration(basedonFechnerianintegration,alogarithmicfunctionisconsidered).Theodorintensitiesofthetargetmaterialemissionsarethenscaledbyassigninganumber:oneofthefixednumbersoranynumberinbetweenthesereferenceodorintensities.Asaresult,theperceivedintensityofthetargetodoristheneitherexpressedasascalevalueofthereferenceodorintensityorasareference(acetone)concentrationvalue.Inthisprocedure,itisassumedthatnointerindividualdifferencesexistinperceivedodorintensityoftheacetonereference.Awell-trained panelisthereforeusedtoachieveacommonuseofthepostulatednumberscaleofperceivedodorintensity.Anyscalevarianceisviewedasuncertaintyinestimatingtheperceivedintensityofthetargetemission.

2.4.7 Quantification: Direct scaling methods

Apartfromequalmatchingofperceivedintensities,measuredinphysicalquantities,therearetwokindsofquantificationmethods:thedirectscalingmethodspresentedinthissection(2.4.7),andtheindirectscalingmethodspresentedinSection2.4.8.Themostaccepteddirectscalingmethodisthemethod of ranking.Itproducesanordinal scale and therefore isnot further considered,because it isproblematic tocomparesuchscalevaluesobtainedindifferentexperimentsunlessasetofidenticalstimulusquantitiesisintroduced,whichwouldrepresentareferencerankorder.

2.4.7.1 Category scaling

Categoryscalingisamethodinwhichodorintensitiesare“sorted”inapredeter-minednumberofcategoriesonthebasisoftheirperceivedintensity.Ina5-categoryscale,typically,thelowestCategory1isusedfortheleastintenseofthestimuli(ortheblank), and thehighestCategory5 for themost intenseof the stimuli (whichatmostmaybe“overpowering”initsperceivedodorintensity,Figure 2.3,left).Intheoriginalformofcategoryscaling,theobserverwasinstructedtodistributethe

Y116134.indb 38 10/10/11 2:11 PM


remaining stimuli in the other categories (e.g., Categories 2–4 of the 5-categoryscale)insuchawaythattheintervalsbetweenthecategoryboundarieswouldbeper-ceptuallyequal.Thus,thedifferenceintheperceivedquantitybetweenthelowerandthehigherboundariesofCategory1shouldbethesameasthatforCategory2,andsoon.Inotherwords,allcategoriesshouldbethesame.Thus,itisutterlyimportanttoinstructtheobserverstokeeptheprincipleof“equal-interval”scaling.Iftheydonotadheretothisprinciple,rankorderingispotentiallyobtained.

Inmostcategoryscalesinuse,thecategoriesaregivenverballabelsinsteadofnumbers(e.g.,seeFigure 2.3,“no,”“slight,”“moderate,”“strong,”“verystrong,”and“overpowering”odorintensityorirritationintensity,asusedbyKnudsen,Nielsen,Clausen,Wilkins,&Wolkoff,2003).Thishasraisedthequestionofwhetherit ispossibletofindverballabelsorso-called“multipliers”thatwouldrepresentcategorydistances.Forexample,aretheperceivedinterdistancesbetween“no”and“slight”irritationin theeyesthesameasbetween”strong”and“verystrong”irritationintheeyesorwould“verystrong” irritation in theeyesrepresent thesamequantityasverystrongirritationinthethroat?JonesandThurstone(1955),Cliff(1959),andG.Borg(1998)haveshownthatthewayofphrasingthequestionandtheselectedverbalcategorylabelsmaybothbeinterpreteddifferentlybydifferentobservers.

Byonlydefiningtheendpointsverbally,a“2-pointcategoryscale”hasbeenusedforquantification:suchascaleiscalledavisual analogue scale(VAS).Theideaisthattheobservershouldputamarkonthelineofthisscale,wheretwo-endpointcat-egoriesdelimittheanaloguescale.Themarkwoulddirectlycorrespondtoaquantityinrelationtoeitherendpoint.Anexampleofsuchascaleisthebipolaracceptabilityandunacceptabilityscale(seeFigure 2.4),usedinindoorairqualityinvestigations(e.g.,Knudsen,Clausen,Wilkins,&Wolkoff,2007).

Theacceptabilityscaleisnormallyaccompaniedbythequestion,“Imaginethatyou, during your daily work, would be exposed to the air in this diffuser/room.How acceptable is the air quality?” The Danish indoor climate labeling system

Assess Odor Intensity Assess Irritation in

Eyes Nose roatNo odor No irritation

Slight odor Slight irritation

Moderate odor Moderate irritation

Strong odor Strong irritation

Very strong odor Very strong irritation

Overpowering odor Overpowering irritation

Figure 2.3 Categoryscalesforevaluatingperceivedodorintensity(left)andsensory-irrita-tionintensityintheeyes,nose,orthroat(right).ReprintedfromKnudsenetal.(2003).

Y116134.indb 39 10/10/11 2:11 PM


formaterialsemissionsusescategoryscalingtodistinguishbetweenacceptedandrejectedconstructionproducts.

2.4.7.2 Borg’s category-ratio scale

Borg’s category-ratio scalewasdevelopedwith thepurposeof adjusting interdis-tancesbetweencategoriesofthecategoryscalesuchthattheintervalsapproachedaratioscaleasobtainedbymagnitudeestimation(seeSection2.4.7.3).Anotherpur-posewastogivelabelstoquantitiesatappropriateinterdistancesonthecategory-ratio scale, thus answering questions such as how much “weaker” is “extremelyweak”inrelationto“veryweak.”Inbrief,theBorg’scategory-ratioscales(CR10©andCR100©;thenumberreferstothemaximumlevelofthescale;seeFigure 2.5)wereintendedtomimicscalesobtainedinabsolutemagnitudeestimation(“numer-ousmatching,”Section2.4.7.4)andalsotogivemeaningtolevel (G.Borg,1998).

Borg’scategory-ratioscalinghasbeenappliedindifferentareasforscalinginten-sivevariablesinperceivedexertion,pain,loudness,taste,odor,andcolorperception.TheCR10andCR100scalesarewidelyusedinvariousapplicationsaroundtheworld,suchasforperceivedexertioninhealth,sport,disease,andrehabilitation.Forsafetyreasonsinolfaction,themethodpresentsaprobleminthatmaximumexposuremaynotbeproducedformanyodorousandtoxicsubstancesorenvironmentalemissions.

Borgarguedthatallbiologicalsystemshavetheirboundaries,fromaminimaltoamaximalcapacity.AccordingtoBorg’srangemodel,thetotalnatural,perceptual,dynamicrangefromzero(orminimalintensity)toamaximalornearmaximalinten-sityshouldbeperceptuallyapproximatelythesameformostindividuals.Asacon-sequenceanyperceivedlevelofintensitycanbeevaluatedinrelationtoitspositionintheindividualrange,andtheresponseforanystimulusintensitycanbecomparedacrossindividualsevenwhenthephysicaldynamicrangevaries(seeG.Borg,1998).

How do you assess the air quality?Clearly acceptable

Pay attention to the dichotomy betweenacceptable and not acceptable

Just acceptableJust not acceptable

Clearly not acceptable

Figure 2.4 Pairofvisualanaloguescales(VAS)forassessingindoorairqualityonabipo-laracceptabilityscale,thatis,asregardsdegreeofacceptableanddegreeofnotacceptable.ReprintedfromKnudsenetal.(2007).

Y116134.indb 40 10/10/11 2:11 PM


Eisler(1965)supportsthisidea.Forperceivedexertion,theCR10andCR100scaleshavebeensuccessfullyvalidatedagainstphysiologicalcorrelatessuchasheartrateandbloodlactate.

2.4.7.3 Magnitude estimation and magnitude production

Magnitude estimationisadirectscalingmethod.Itisaprocedureinwhichobserversareaskedtoassessdirectlytheintensityofaperceptionorthequantityofanypsy-chologicalvariable(S.S.Stevens,1975).Intheearlyversions,astandardwasplacedinthemiddleofthestimulusrangeandcalled10(or100).Latersubjectsweresimplyinstructedtoassignnumbers toaseriesofperceivedmagnitudesof thepresentedstimulus(e.g.,intensityorpleasantness).Thislatterprocedureiscalledfree-number magnitude estimationorfree-numbermatching(cf.Zwislocki’sabsolutemagnitudeestimationinZwislockiandGoodman,1980;seeSection2.4.7.4).

Anearlyversionofmagnitudeestimationwasalsocalledthemethodofratioesti-mationinwhichthemoreconstrained“multiplying”and“dividing”wererequiredfromtheparticipantbeforeshereportedthemagnitudes.Acompleteratio scaling method was further developed by Ekman to involve full matrices of all stimulusmagnitudes,inwhichallpossiblepairsofstimulusmagnitudewerecomparedand

Nothing at all

Extremely weak

Very weak

Weak

Moderate

Strong

Very strong

Extremely strong

Absolute maximum

0

0,3

0,5

0,7

1

1,5

2

2,5

3

4

5

6

7

8

9

11

10

•

Just noticeable

Highest possible

"No I"

“Strongest I”

Light

Heavy

Absolute maximum

“Maximal”Extremely strong

Very strong

Strong

Moderate

Weak

Very weak

Extremely weak

“Minimum”

Nothing at all

Just noticeable

Light

Heavy

Max I

1.5

10090

80

70

60

50

40

30

20

15

25

10

5

32

1

35

45

110

55

65

75

85

4

68

13

17

0

120

95

Figure 2.5 TheBorgCR10(©G.Borg,1998)andCR100(©G.Borg&E.Borg,2001)scales.

Y116134.indb 41 10/10/11 2:11 PM


judged(e.g.,Ekman,1958;Ekman&Sjöberg,1965).Formanycontinua,perceivedintensity(Ψ)increaseswithstimulusmagnitude(Φ)asapowerfunction,

Ψ Φ= ⋅k n , (2.7)

wheretheconstantkdependsontheunitsofmeasurementandnisthevalueoftheexponent. The exponent has been found to differ depending on the sensory con-tinuumandperceivedqualities(S.S.Stevens&Galanter,1957;Berglund,Berglund,Ekman,&Engen,1971).S.S.Stevens(1975)claimsthatthevalueoftheexponentis an important property of a sensory continuum. For example, the exponent forpainintensityhasbeenreportedtobeashighas3.5,whereastheexponentforloud-nesstypicallyiscloseto0.3(soundintensity;0.6soundpressure).Comparedwithothersenses,odoroussubstancesexhibitalargevariationinexponentsoftheirpowerfunctions forperceivedodor intensity.Thereasonfor this isnotwellunderstood,althoughitisbelievedthatsimultaneoustrigeminalstimulation,resultinginsensoryirritationperception,maycausehigherexponentsthan“pure”odoroussubstances.

Magnitude productionistheoppositescalingproceduretomagnitudeestimation.Insteadofassigningnumberstosensationmagnitudes,subjectsinsteadadjustastimuluscontinuum(e.g.,soundpressurelevelofapinknoise)suchthattheperceivedintensityofthestimulusquantitymatchesthesizeofthenumbersgivenbytheexperimenter.

2.4.7.4 Absolute magnitude estimation

Zwislockiandcoworkers(Hellman&Zwislocki,1963;Zwislocki&Goodman,1980;seealsoGescheider,1997)suggestthatfree-numbermagnitudeestimation(Section2.3.7.3),withaspecialinstruction,maymeetwiththe“absolute”demandsnecessaryforinterindividualcomparisons.Henamedthemethodabsolute magnitude estima-tion(AME).Themainideaisthatthroughourexperienceofnumerousness,num-bersthemselvesacquireakindof“absolute”magnitudeandthereforeAME(orthenumbers)workssomewhatlikeasecondmodalitysimilarlytothemethodofcross-modalitymatching(seeSection2.4.6.2),whichhasbeenshowntoworkwell.

2.4.7.5 Magnitude matching

J.C.StevensandMarks(1980) introduced themethodofmagnitudematching. Itclassifiesasa joint scaling procedure (cf.Berglund,Berglund,&Lindvall,1978).Participantsmakefreenumbermagnitudeestimationsofsetsofstimuliona“com-mon” numerical scale, shifting between two (or more) modalities, for example,loudnessandbrightness,fromtrialtotrial.Therationaleisthatpersonalnumerical“idiosyncrasies”shouldbecancelledoutofthematchingfunction.Participantsserveastheirowncontrolsreducingvariabilityinslope,position,andshapeofthepsycho-physical functions. Under the strong assumption that thepsychophysical functionofoneofthemodalitiesis“thesame”inallindividuals,itispossibletoassessand“calibrate”forindividualdifferencesintheother,targetmodalityinwhichthesame

Y116134.indb 42 10/10/11 2:11 PM


individualsare“notthesame”(J.C.Stevens&Marks,1980;Marks&Algom,1998).Thismethodhas,forexample,beenusedbyBartoshuk(2000)tostudysensorydif-ferencesintasteperceptionbetweennontasters,tasters,andsupertasters.Berglundand Nordin (1992) found that the assumptionof finding amodality,which is thesameforall individuals,whereasanother targetmodality isnot, isnoteasilyful-filled.Theyshowedthatinsmokers,ascomparedtononsmokers,notonlytheodorperceptionwasaffectedbutalsotheloudnessperception,afindingthatrejectedtheinvarianceassumptionbutfoundsupportinthusfarunknownfindings.

2.4.7.6 Constrained scaling

Compared to the method of magnitude matching (Section 2.4.6.5) and of masterscaling(Section2.4.7.7),Ward(1992) takesonasomewhatoppositeviewonhowto treat individualdifferences, namely toview it asmeasurementbehavior (West&Ward, 1994; West, Ward,&Kohsla, 2000).He emphasizes thevalueof “con-strained”scalingbyusinganarbitraryscale.Accordingtothismethod,participantsaretaughtandtrainedintheuseofacertainstandardscale(cf.Section2.4.6.3)withaspecificexponentofthepsychophysicalpowerfunctionforsomechosenmodality,untiltheyareabletoreproducethechosenexponentofaparticularpowerfunctionwithhighaccuracy.Thesamepersonsthenusethesamescaleforscalingperceptualmagnitudesoftargetstimuli.

AsimilarapproachhasalsobeendevelopedbyFanger(1988)andbeenproposedforapplicationsinthefieldofindoorairquality.Theparticipantsaretrainedtoscaleperceivedodorintensityofacetoneandsubsequentlytoldtousethissamenumberscalealsowhenscalingotherquantities,suchasperceivedairqualityinrooms(visi-tors’condition).BothFanger(1988)andKasche(2005)trainobserverstouseaspecificpostulatedscale,thusgettingridofbetween-observervariancefortheodor-intensity set of concentrationsof a reference substance. Inprinciple, constrainedscaling isoftenthepsychophysicalmethodalsopreferredinsensoryanalysisoffoodproducts.

2.4.7.7 Master scaling

Inordertokeeptheexperimentalcontext invariant,Berglund(1991)recommendsjointscalingofreferencesandtargetstimuli.Hermasterscalingwasoriginallydevel-opedforthecalibratedmeasurementofonepointonacontinuumofperceivedinten-sity(theonetarget)withtheaidofoneindividual.Themethodhasbeenusedwithgreatsuccessinstudiesof,forexample,trafficnoise,odorousairpollution,andinpatientswithchronicpain(e.g.,Berglundetal.,1974;Berglund&Lindvall,1979;Berglund&Harju,2003;Berglund,Harju,Kosek,&Lindblom,2002).

Inmasterscaling,measurementofperceivedodorintensityisviewedasacon-textuallybaseddynamicprocess.Characteristically,sensorysystemsaredynamicandadaptable.Eachobserver’smagnitudeestimates(Section2.4.7.3)consistofcon-text-dependent relativevalues rather thanabsolute levelsor invariables.Between-observervariationisassumedtoreflect trueperceptualdifferences.This isnot tosaythatanobserverhasnomeasurementerror,onlytosaythattrueamong-observer

Y116134.indb 43 10/10/11 2:11 PM


differencesin(perceived)odorintensityexist.Masterscalingmeans,forexample,thatodorintensityofasetofwell-definedreferenceconcentrations(e.g.,pyridine)isscaledjointlywiththetargetodoremission(s);hereaftertheodorintensityofthetargetiscalibratedtotheagreed-uponmasterscaleofthereferences.Therearetwoprincipalgoalswithmasterscaling:(a)Theexperimentalprocedureofmasterscal-ingisusedformeasuringandevaluatingeachobserver’sscalingbehaviorwiththeaidofreferencestimuli(scalingcontext);and(b)themasterscaletransformationisusedtomeasureeachobserver’sperceivedintensityof targetodoremissionsonacalibrated(perceived)odorintensityscale.Itisconsideredespeciallyimportanttodifferentiatetheodor-intensityvarianceofthereferencesfromthevarianceofthetarget,aprobleminherentinanydirectequal-intensitymatchingprocedure.

Themasterscaletransformationisasfollows.Theperceivedodorintensity(R)ofthereferencesisapowerfunctionofconcentration(S),R=cSn,andcandnaremul-tiplicativeconstantandexponent,respectively(seeEquation(2.7)).Letthisequationofthemasterfunctionforthepyridinereferencesbesubscribedbymandthecorre-spondingequationforeachobserver’sempiricalodorintensitiesforthereferencesbesubscribedbyi.Becausetheconcentrationmeasuresofthereferencesarethesameinthesetwopower-functionequations(Sm=Si),equatingthetwoandrearrangingthetermsgivetheequationforthemasterscaletransformation,

R cRc

m mi

i

n

nm

i

=

, (2.8)

whereRi istheempiricalodorintensityofeachsubject’sreferencescale,andRmisthecorrespondingodorintensitytransformedtothemasterscale.Byinsertingtheindividualempirical(perceived)odorintensityvaluesofthetarget materials emis-sion (Ri) intoEquation (2.8), these canbe transformed to theunit and reference points of the master scale of odor intensity(Rm),definedforthesetofodorintensi-tiesofthepyridineconcentrations.

Theparticipantsarequalityassured in that theyare toproduceapower func-tionforthereferenceconcentrations.Participantsarescreenedadhoconthequalityofthesepowerfunctions,whichtheyproduceinthescalingexperiment.Theyarefreetousethenumberscaletheyfeelcomfortablewith,asisthecaseinmagnitudeestimation. Inprinciple,master scaling involvesa“calibration”of individualpar-ticipant’s perceived odor intensity scales. The uncertainty is determined for eachparticipantwiththeaidofthereferences.Acriterionofacceptableuncertaintymaythusbeusedbeforeaveragingtheempiricaldata.Theodor intensityof thetargetstimulicaneitherbeexpressedinmaster-scalenumbersorinequivalentconcentra-tionvaluesofthereference.

2.4.8 Quantification: Indirect scaling methods

Indirectscalingmethodsmaybeparticularlysuitedforscalingpsychologicalattri-butes of complex stimuli for which a relevant and common stimulus quantity is

Y116134.indb 44 10/10/11 2:11 PM


lacking.Theindirectscalingmethodsdifferfromdirectscalingmethodsinthatthecollecteddatawillnotcarryenoughinformationformeasurementatintervalorratioscales.The judgments areatordinal level: larger thanor smaller than.Toobtainmeasurement on a higher order scale than the ordinal scale, certain assumptionshavetobeintroducedandappliedtothecollectedempiricaldata,anditisalsoneces-sarytotesttheassumptionsintroduced(datatheory;Jacoby,1991).Apotentialcon-clusionofthemeasurementwithindirectscalingmethodsmaybethattheassumedmeasuringmodeldidnotfitthedata.

2.4.8.1 Thurstone’s judgment scaling model: Pair comparisons of stimuli

In Thurstonian scaling, the scaling of stimuli must always be done indirectly(Torgerson,1958).Eachstimulus,whenpresentedtoanobserver,givesrisetoadis-criminatoryprocess,whichhasavalueonthepsychologicalcontinuum(e.g.,degreeofannoyance).Becauseofmomentaryfluctuationsintheorganism,agivenstimulusdoesnotalwaysexcitethesamediscriminatoryprocess,butmayexciteonewithahigherorlowervalueonthecontinuum.Thediscriminatoryprocessreflectshowtheorganismidentifies,distinguishes,orreactstostimuli.Itispostulatedthatthevaluesofthediscriminatoryprocessformnormalfrequencydistributionsonthepsycho-logicalcontinuum.Themean(sjandsk)andthestandarddeviation(σjandσk)ofthedistributionsassociatedwithtwodifferentstimuli,jandk,aretheirscalevalueanddiscriminatorydispersion,respectively.

2.4.8.1.1 Law of Comparative Judgment. Thecompleteformofthelawofcom-parativejudgmentreads(Torgerson,1958,p.161):

sk – sj = zjk(σj2+σk

2–2rjkσjσk)1/2, (2.9)

wherezjkisthedistancefromthemeanexpressedwiththevariancemeasureasunit.Byassumingzerocorrelations (r=0) andequaldiscriminatorydispersions (σj=σk=c),thelawofcomparativejudgmentreducestoThurstone’sCaseV:

sk–sj=zik c(2)1/2. (2.10)

Ifc(2)1/2isusedasunit,theequationreducesto

sk – sj = zjk. (2.11)

Asanexample,agroupof100participantsconductspairwisecomparisonsofannoy-anceattributedtosevensoundrecordingsofdifferentkindsofaircraftoverflights.Each participant is to report which of two aircraft overflights is more annoyingamongthe21uniquepairs.Theresponseproportionsfor the100participantsarecalculatedandtransformedtoz-values,whichaccordingtoCaseVconstitutescaleintervals.Byaddingthez-valuesofeachcolumnofthesquarematrix,andbyselect-ingazero-pointofthescale,theannoyancescaleisestablishedasanintervalscale.

Y116134.indb 45 10/10/11 2:11 PM


2.4.8.1.2 Law of Categorical Judgment. In principle, this law states that theboundariesbetweenadjacentpointsonacategoryscalebehaveasdothestimuliintheexampleabove.Italsoassumesthereisapsychologicalcontinuumoftheobserverthatcanbedividedintoaspecificnumberoforderedcategories.Forthiscasediffer-entscalingmodelsmaybeuseddependingonthetypeofreplicationsincollectingthedata(severaltrialsperobserverorobserversviewedasreplicationsoramixofthesetwo).

Berglund,Berglund,andLindvall(1975)appliedThurstone’sCaseVtoannoy-ance questionnaire data (6-point category scale) in an epidemiological study of1,400–2,000 inhabitants ineachoffiveareasofaircraftnoiseexposure.The fre-quencyofresponseforeachofthreeenvironmentalagentswastabulatedrelativetothe6pointsofacategoryscale.Frequencieswerethentransformedtoproportionsofthetotalnumberofresponses,andtheproportionsfurthertransformedintoanormaldeviate.Itisfromthenormaldeviatesthatthecontinuumofannoyancewasderivedonwhichbothcategoryboundariesandtheparticipant’sdegreeofannoyancetotheenvironmentalnoisewerescaled;inthiscaseGulliksen’smethodofleastsquareswasused(Torgerson,1958).Oncetheannoyancescaleisestablished,locationsoftheboundariesfortheverbalcategoriesandtheenvironmentalagentsineachareaareeasilydetermined.

Theintervalpropertyofthescalemeansthattheannoyancescalesfordifferentlyexposedareascanbecalibratedtoacommonreferencepointandacommonunitofmeasurement.InFigure 2.6,thedegreeofannoyanceforthreeenvironmentalagentsisprovidedon thesamecalibratedscaleasobtained in theaircraftnoise investi-gation (all five areas). The specific assumptions applied are that the variances ofeachresponsecategoryboundaryareconstantandindependentofthespecificenvi-ronmentalagent.InthisspecificAreaI,thedegreeofannoyanceforeachagentisexpressedbythemeanofthedistributions.Themeansarelocatedonanintervalscaleofannoyanceandthereforequantitativecomparisonsoftheagentsarepossible.

2.4.9 Combining methods

Theresearchproblemhascertainrequirementsastowhatmethodtouse.Itmaybeimpossibletoknowbeforehandtherangeofstimulusintensitiestouse,forexample,

0

25

Degree of Annoyance

Area IAircraft noiseTraffic noiseStandard noise

50f

1 2 3 4 5 6

Figure 2.6 Empiricallyobtainedfrequencydistributionsofannoyancewithregardtothreedifferentkindsofnoises.Themeansofthedistributionsrepresentthedegreeofannoyanceonacalibratedintervalscale.ReprintedfromBerglund,Berglund,&Lindvall(1975).

Y116134.indb 46 10/10/11 2:11 PM


iftheodorsensitivityofsmokersandnonsmokers(Berglund&Nordin,1992)weretobetestedorofpatientswithAlzheimer’sdisease(Nordin,Almkvist,Berglund,& Wahlund, 1997). In such cases, it may be favorable to introduce blanks in anexperimentdesignedfor themethodofconstantstimulisuchthatsignaldetectiontheorymaybeappliedinthedatatreatment.Thereasonforthisprecautionisthattheexpectedabsolutethresholdmaydifferagreatdealamongpatientsandindividualsmokersornonsmokers.

Insomecases,itmayalsobefavorabletocombinethemethodofconstantstimuliwithmagnitudeestimation(Berglund&Nordin,1992).Forexample,firstaskifthesubjectcanperceiveasmellornot(blanksmaystillbepresented),and,ifthesub-jectsaysyes,thenaskhowintensetheodorofthesniffisandgetaquantitybythemethodofmagnitudeestimation.

2.5 Descriptor profiling and multidimensional scaling

In environmental field applications, it is not possible to present exactly the sameexposureseveraltimes.Forexample,theannoyanceofanaircraftoverflightcannotberepeatedwithashorttimeintervalortheodorofindoorairwillnotbestableforlong.Alternatively,itmaythenbepossibletoaskagroupofsubjectstojudgeseveralaspectsoftheirperceptionsbyaprofilingtechnique.Forexample,severalcategoryscalescouldbeusedformeasuringthesoundoftheoverflight(e.g.,loudnessand/orannoyance of the maximum, the perceived duration of the overflight, etc.) or thequality of the indoor air (e.g., warmth, stuffiness, odor intensity, etc. to create aquantityprofileofthequalities).

Ideally,descriptorprofileswouldbebuiltfrom8–10perceptual–emotionalattri-butes,alldescribing the“qualityof the indoorair.”Asheetwithvisualanaloguescales for eachattribute is preparedwith endpointsmarked0%match and100%match.Theselectionofattributeshastobepilotedbeforethedescriptorprofilemaybeconsideredpartofameasuringinstrumentforperceivedairquality(ormaterialsemissions).Alternatively,applicabilityofattributestoindoorairsamplesormaterialsemissionmaybeused(Baird,Berglund,&ShamsEsfandabad,1994).Itisimportantthatcarefulcross-translationsaremadebetweenlanguagesofdifferentcountries.

Thescalevaluesofattributematches(0–100%intervalscale)provideacharacter-isticprofileoverattributesforeachrespondent.Acorrelationmatrixmaybeformedfrom pairs of profiles, each profile characterizing the indoor air quality in partsof buildings or different buildings. Inasmuch as a profile represents one emission(oneindoorair),thecorrelationmatrixcanbeviewedasamatrixof“similarities”(r 2 = shared variance).DatatreatmentbyPCA(principlecomponentsanalysis,con-tentmodel)orMDS(multidimensionalscaling,distancemodel)woulddeliverarepre-sentationoftheinterrelationsamongthematerialsemissionsorthesamplesofindoorair.Therearemethodsbywhichinterindividualdifferencesmaybetreatedaccordingtospecifictheory(e.g.,INDSCAL,seeSchiffman,Reynolds,&Young,1981).Jointspaceanalysis(attributesandstimuli)withcorrespondenceanalysisisalsoapossibil-ity(Noma,Berglund,Berglund,Johansson,&Baird,1988;Greenacre,1984).

Y116134.indb 47 10/10/11 2:11 PM


References

Ahlström,R.,Berglund,B.,Berglund,U.,&Lindvall,T.(1986).Formaldehydeodoranditsinteractionwiththeairofasickbuilding.Environment International, 12, 289–295.

Baird, J. C., Berglund, B., & Shams Esfandabad, H. (1994). Longitudinal assessment ofsensory reactions in eyes andupper airwaysof staff in a sickbuilding.Environment International,20,141–160.

Baird,J.C.,&Noma,E.(1978).Fundamentals of scaling and psychophysics.NewYork:Wiley.Bartoshuk,L.(2000).Comparingsensoryexperiencesacrossindividuals:Recentpsychophys-

ical advances illuminate genetic variation in taste perception. Chemical Senses, 25, 447–460.

Berglund,B.(1991).Qualityassuranceinenvironmentalpsychophysics.InS.J.Bolanowski&G.A.Gescheider(Eds.), Ratio scaling of psychological magnitudes–In honor of the memory of S. S. Stevens(pp.140–162).Hillsdale,NJ:Erlbaum.

Berglund,B.,&Harju,E.-L.(2003).Masterscalingofperceivedintensityoftouch,coldandwarmth.European Journal of Pain, 7,323–334.

Berglund,B.,&Lindvall,T.(1979).Olfactoryevaluationofindoorairquality.InP.O.Fanger&O.Valbjørn(Eds.), Indoor climate. Effects on human comfort, performance, and health in residential, commercial, and light-industry buildings (pp. 141–157). Copenhagen:DanishBuildingResearchInstitute.

Berglund,B.,&Nordin,S.(1992).Detectabilityandperceivedintensityforformaldehydeinsmokersandnon-smokers.Chemical Senses,17,291–306.

Berglund,B.,Berglund,U.,&Lindvall,T.(1974),Apsychologicaldetectionmethodinenvi-ronmentalresearch.Environmental Research,7,342–352.

Berglund,B.,Berglund,U.,&Lindvall,T. (1975).Scalingof annoyance in epidemiologi-cal studies. In Proceeding of the International Symposium Recent Advances in the Assessment of the Health Effects of Environmental Pollution, Vol. I (pp. 119–137).Luxembourg:CommissionoftheEuropeanCommunities.

Berglund,B.,Berglund,U.,&Lindvall,T.(1978).Separateandjointscalingofperceivedodorintensityofn-butanolandhydrogensulfide.Perception & Psychophysics,23,313–320.

Berglund,B.,Berglund,U.,&Lindvall,T.(1986).Theoryandmethodsofodourevaluation.Experientia, 42,280–287.(OdourBiologyIssue)

Berglund,B.,Berglund,U.,Ekman,G.,&Engen,T.(1971).Individualpsychophysicalfunc-tionsfor28odorants. Perception & Psychophysics,9(3B),379–384.

Berglund,B.,Harju,E.-L.,Kosek,E.,&Lindblom,U.(2002).Quantitativeandqualitativeper-ceptualanalysisofcolddysesthesiaandhyperalgesiainfibromyalgia.Pain,96,177–187.

Borg,G.(1998).Borg’s perceived exertion and pain scales.Champaign,IL:HumanKinetics.Borg,G.,&Borg,E.(2001).Anewgenerationofscalingmethods:Level-anchoredratioscal-

ing.Psychologica,28,15–45.Cliff,N.(1959).Adverbsasmultipliers.Psychological Review, 66,27–34.Cook,T.S.,&Campbell,D.T.(1979).Quasi/experimentation. Design and analysis issues for

field settings.Boston:HoughtonMifflin.Culvert,G.,Spence,C.,&Stein,B.E.(Eds.).(2004).The handbook of multisensory processes.

London,Boston:BradfordBook(MIT).Devos,M.,Patte,F.,Rouault,J.,Laffort,P.,&vanGemert,L.J.(Eds.).(1990).Standardized

human olfactory thresholds.Oxford,UK:IRLPress.Eisler, H. (1965). The ceiling of psychophysical power functions. American Journal of

Psychology, 78,506–509.

Y116134.indb 48 10/10/11 2:11 PM


Ekman,G.(1958).Twogeneralizedratioscalingmethods.Journal of Psychology,45,287–294.Ekman,G.,&Sjöberg,L.(1965).Scaling.Annual Review of Psychology,16,451–474.Engen,T.(1971).Psychophysics.I.Discriminationanddetection.InJ.W.Kling&L.A.Riggs

(Eds.),Woodworth and Schlosberg’s experimental psychology(pp.11–46).NewYork:Holt,Rinehart&Winston.

Falmange,J.-C.(1985).Elements of psychophysical theory.NewYork:OxfordUniversityPress.Fanger,P.O.(1988).Instructionoftheolfandthedecipolunitstoquantifyairpollutionper-

ceivedbyhumansodorsandoutdoors.Energy and Buildings,12,1–6.Fechner, G. T. (1860). Elemente der psychophysik [Elements of psychophysics] (Vol. 2).

Leipzig:BreitkopfandHärtel.Gescheider,G.A.(1997).Psychophysics: The fundamentals(3rded.).London:Erlbaum.Green, D. M., & Swets, J.A. (1962). Signal detection theory and psychophysics. New

York:Wiley.Greenacre, M. J. (1984). Theory and applications of correspondence analysis. London:

AcademicPress.Hellman,R.,&Zwislocki,J.J.(1963),Monauralloudnessfunctionat1000cpsandinteraural

summation.Journal of the Acoustical Society of America,35, 856–865.Jacoby,W.G.(1991).Datatheoryanddimensionalanalysis.London:Sage.(No.07-078)Jones,L.V.,&Thurstone,L.L.(1955).Thepsychophysicsofsemantics:Anexperimental

investigation.Journal of Applied Psychology,39,31–36.Kasche,J.(2005).Thenewperceivedintensityscale.PresentationatTUBerlin(Unpublished).Knudsen,H.N.,Nielsen,P.A.,Clausen,P.A.,Wilkins,C.K.,&Wolkoff,P.(2003).Sensory

evaluationofemissionsfromselectedbuildingproductsexposedtoozone.Indoor Air, 13(3),223–231.

Knudsen,H.N.,Clausen,P.A.,Wilkins,C.K.,&Wolkoff,P.(2007).Sensoryandchemicalevaluationofodorousemissionsfrombuildingproductswithandwithoutlinseedoil.Building and Environment,42,4059–4067.

Kobal, G., & Hummel, T. (1991). Olfactory potentials in humans. In T. V. Getchell, R.Doty,L.M.Bartoshuk,&J.B.Snow,Jr.(Eds.),Smell and taste in health and disease(pp.255–275).NewYork:RavenPress.

Lord,F.M.,&Novick,M.R.(1968).Statistical theories of mental test scores.Reading,MA:Addison-Wesley.

Luce,R.D.,Bush,R.R.,&Galanter,E.(Eds.).(1963).Handbook of mathematical psychol-ogy.NewYork:Wiley.

Marks, L. E., & Algom, D. (1998). Psychophysical scaling. In M. H. Birnbaum (Ed.),Measurement, judgment, and decision making.Boston:AcademicPress(pp.81–178).

Noma,E.,Berglund,B.,Berglund,U.,Johansson,I.,&Baird,J.C.(1988).Jointrepresenta-tionofphysicallocationsandvolatileorganiccompoundsinindoorairfromahealthyandasickbuilding.Atmospheric Environment,22,451–460.

Nordin,S.,Almkvist,O.,Berglund,B.,&Wahlund,L.-O.(1997).OlfactorydysfunctionforpyridineanddementiaprogressioninAlzheimer’sdisease.Archives of Neurology,54,993–998.

Roberts,F.S.(1979).Measurementtheorywithapplicationstodecisionmaking,utility,andthesocialsciences.InC.-C.Rota(Ed.),Encyclopedia of mathematics and its applica-tions.Vol.7.Reading,MA:Addison-Wesley.

Schiffman,S.S.,Reynolds,M.L.,&Young,R.W.(1981).Introduction to multidimensional scaling. Theory, methods and applications.London:AcademicPress.

Y116134.indb 49 10/10/11 2:11 PM


Stevens,J.C.,&Marks,L.E.(1980).Cross-modalitymatchingfunctionsgeneratedbymag-nitudeestimation.Perception & Psychophysics,27,379–389.

Stevens,S.S.(1946).Onthetheoryofscalesofmeasurement.Science, 103,677–680.Stevens,S.S.(1975).Psychophysics. Introduction to its perceptual, neural and social pros-

pects.NewYork:Wiley.Stevens,S.S.,&Galanter,E.H.(1957).Ratioscalesandcategoryscalesforadozenpercep-

tualcontinua.Journal of Experimental Psychology, 54,377–411.Svensson,L.T.,&Szczygiel,K.(1974).Amatchingmethodofsuccessiveapproximations

anditsinstrumentation.Behavioral Research Methods & Instrumentation, 6(1),13–18.Torgerson,W.A.(1958).Theory and methods of scaling.NewYork:Wiley.Ward,L.M.(1992).Whoknows?InG.Borg&G.Neely(Eds.),Fechner Day 92(pp.217–

222).Stockholm:InternationalSocietyforPsychophysics.West,R.L.,&Ward,L.M.(1994).Constrainedscaling.InL.M.Ward(Ed.),Fechner Day

1994(pp.225–230).Vancouver:UniversityofBritishColumbia.West,R.L.,Ward,L.M.,&Kohsla,R. (2000).Constrainedscaling:Theeffectof learned

psychophysicalscalesonidiosyncraticresponsebias.Perception & Psychophysics, 62,137–151.

WHO.(1989).Formaldehyde. Environmental Health Criteria 89.Geneva,Switzerland:TheWorldHealthOrganization.

Zwislocki,J.J.,&Goodman,D.A.(1980).Absolutescalingofsensorymagnitude:Avalida-tion.Perception & Psychophysics,28,28–38.

Y116134.indb 50 10/10/11 2:11 PM

51

Measurements of physical parameters in sensory science

Teresa GoodmanNationalPhysicalLaboratoryTeddington,UnitedKingdom

3.1 The importance of measurement

Measurementistheprocessofquantifyingtheextentorquantityofsomethingbycomparisonwithafixedunitorareferenceobjectofknownsize.Itisthemeansbywhichwemakeobjectivecomparisonsbetween thingsonaconsistentand repro-duciblebasis,andprovidesacommonlanguagebywhichwecandescribeobjectsorbehaviors.Measurementthereforesitsattheheartofourmoderntechnologicalworld,supportingbusiness,trade,industry,andscience,andunderpinningtheregu-latory framework thathelpsmaintainand improveourqualityof life, inareasasdiverseaspublichealthandsafety,climatechange,andsportsandleisure.Itisanessentialpartofall scientificand technological researchanddevelopment,allow-ingus,forexample,tospecifythephysicalorchemicalattributesofasubstanceoranobject,toevaluatetheactivityofdifferentpartsofthebrain,toquantifyhumanperceptualresponsesandbehaviors,toassesssocietaltrends,oreven(throughthechoiceofanappropriatemeasurementmethodandscale)toenumeratesuchseem-inglyabstractqualitiesasintelligenceorhappiness.InthewordsofLordKelvin:

Whenyoucanmeasurewhatyouarespeakingabout,andexpressitinnumbers,youknowsomethingaboutit;butwhenyoucannotmeasureit,whenyoucannotexpressitinnumbers,yourknowledgeisofameagreandunsatisfactorykind:itmaybethebeginningofknowledge,butyouhavescarcely,inyourthoughts,advanced to the stageof science (asquoted inAmericanAssociation for theAdvancementinScience).

Theimportanceofphysicalmeasurementsforapplicationssuchasmanufactur-ing,construction,andcommercehasbeenrecognizedsincethebeginningsofhumancivilization.InthetimeoftheEgyptianpharaohs,forexample,thearchitectsrespon-sibleforbuildingthepyramidsfacedthedeathpenaltyiftheyfailedtocalibratetheirreferencemeasurementstandards(“cubitsticks”)ateachfullmoon.Byimplementingsucharigorousmeasurementregime,theywereabletoachieveconstructionaccu-racyof0.05%.Earlymeasuresoflengthweregenerallybasedonhuman-centered

3

Y116134.indb 51 10/10/11 2:11 PM


references.InthecaseoftheRoyalEgyptiancubit,forexample,thereferencewasthelengthoftheforearmofthepharaohrulingatthattime(measuredfromhisbentelbowtothetipofhisextendedmiddlefinger)plusthewidthofthepalmofhishand.Thiswasthentransferredtoacarvedgranitereferencestick,againstwhichallothermeasuringstickscouldbecompared.Similarly,thereferencesusedforotherbasicmeasures,suchasweight,werebasedoncommonlyavailablematerials;forexample,theEgyptiansandGreeksusedawheatseedasthesmallestunitofweight,astan-dardthatwasveryuniformandaccurateforthetimes.

However,despitetheirsuccess,thereweresignificantproblemsassociatedwithallsuchmeasures,duetothefactthattheywereimpossibletoreproducereliablyoveralongperiodoftime,orindifferentlocations.ThedeathofapharaohinEgypt,forexample,wouldresultinastepchangeinthesystemforlengthmeasurementusedinthatcountry,andayearofdroughtorexcessiveraincouldchangetheweightofawheatseed.Astheneedforconsistentandreliablemeasurementunitsincreased,particularlyforthepurposesoftradebetweenregions,sothesystemsofmeasure-mentmovedtobeingbasedoninternationallyagreedreferenceartifacts,whichwerekeptincarefullycontrolledconditionstoensurelong-termconsistencyandagainstwhichallotherscouldbecomparedtoensureregion-to-regionagreement.Eventhisapproachhaditslimitations,however,inasmuchasnotonlycouldthereferencearti-factschangewithtime(evenifcarefullymaintained)butalsobecausetheaccuracywithwhich theycouldbe transferredwas limitedby theaccuracyof themeasur-ing instrumentation used. Modern physical measurements are therefore based onasystemofsevenbase units,whicharealldefinedfromfirstprinciples(with theexceptionoftheunitofmass,thekilogram,whichisstillbasedonasinglereferenceartifact)andfromwhichallotherphysicalmeasurescanbederived.Thisistheso-calledInternationalSystemofUnits(SI),whichisdiscussedfurtherinSection3.2.

Ofcourse,asalreadymentioned,theconceptofmeasurementisnotlimitedtothequantificationofphysicalparameters,suchasmass,time,orelectriccurrent,butisalsousedinmanyotherareasofscience,business,andindustry.Oftenthesetypesofmeasurementaredirectly(andobviously)linkedtophysicalmeasurements.Forexample,measurementsofactivityindifferentregionsofthebraincarriedoutusinganelectroencephalography(EEG)systemarebaseddirectlyonmeasurementsoftheelectriccurrent(orvoltage)foreachofthesensorsandonknowledgeofthepositionofeachsensoronthehead.If theelectricalmeasurementsaremadeusingequip-ment thathasnotbeencalibrated (orwhoseperformancehasnotbeenotherwisevalidated),orifthesensorsareincorrectlypositioned,thenanyconclusionsdrawnfrom such EEG measurements may be invalid. In other cases, however, the linkwithphysicalmeasurementsislessclear,butneverthelessimportant.Forexample,measuresofhumansubjectiveresponses,suchasthelevelofenjoymentexperiencedwhenlisteningtodifferenttypesofmusic,maybederivedsolelyfromresponsestoquestionnaires,butincreasinglythisinformationislinkedwiththemeasurementofphysical responses (e.g., heart rate or blood pressure) to provide additional infor-mation.Thusphysicalmeasurementplaysanimportantroleinallareasrelatedtounderstandinghumanperceptionandinterpretationor“measuringtheimpossible.”

Y116134.indb 52 10/10/11 2:11 PM

Measurements of physical parameters in sensory science 53

3.2 The Metre Convention and the International System of Units

Thevalueofaquantityisgenerallyexpressedastheproductofanumberandaunit,wheretheunitisaparticularexampleofthequantityconcernedwhichisusedasareference,andthenumberistheratioofthevalueofthequantitytotheunit.Itispossibletouseanydefinedreferenceasthebasisforagivenmeasurement;allthatisnecessaryisforthechosenreferencetobeavailableatthetimethemeasurementismade.This,aswehaveseen,wastheapproachtakenfortheearliestmeasurementsanditissometimesstillusedincaseswhereconsistencybetweendifferentpeoplemakingthemeasurementsisnotneededandwherelong-termcontinuityofmeasure-mentisnotimportant.Whichofushasnot,forexample,atsometime“measured”lengthbycomparisonwithourown thumbor arm length, inorder toprovideanapproximateassessmentof“size.”Butsuchmeasurementsarebydefinitionimpre-ciseanddifficult tocommunicate reliably.Theyareentirelyunsuited formodernscience, technology, industry,andcommercewhere, forexample,componentsaresourcedfrommanydifferentsuppliers,goodsaretradedinternationally,safetyneedstobedemonstratedandassured,andperformancespecificationsneedtobemet.

Thisneedforconsistencyandreproducibilityinmeasurementhasbeenenshrinedinaninternational treaty,namedtheMetreConvention,whichwasfirstsignedinParisin1875byrepresentativesof17nationsandestablishedapermanentorgani-zationalstructureunderwhichmembergovernmentsactincommonaccordonallmattersrelatingtounitsofmeasurement.Thistreaty,whichwasmodifiedslightlyin 1921, remains the basis of international agreement on units of measurementand now has 53 member states, including all the major industrialized countries.TheMetreConventioncreatedtheInternationalBureauofWeightsandMeasures(BIPM),anintergovernmentalorganizationthatoperatesundertheauthorityoftheGeneralConferenceonWeights andMeasures (CGPM)and is supervisedby theInternationalCommitteeforWeightsandMeasures(CIPM).ThetaskoftheBIPMistoensureworldwideunificationofmeasurementsandactasacentralfocusforworldmetrology,particularlyconcerningthedemandformeasurementstandardsofever-increasingaccuracy,range,anddiversity,andtheneedtodemonstrateequivalencebetweennationalmeasurementstandards.

AkeyfeatureoftheMetreConventionistheestablishmentofapracticalsystemofunitsofmeasurement,knownastheSystèmeInternationald’Unités(InternationalSystemofUnits,internationalabbreviationSI;seeOrganisationIntergouvernementalede laConventionduMètre,2006),whichhasbeenusedaround theworldas thepreferredlanguageofscienceandtechnologysinceitsadoptionin1948throughaResolutionofthe9thCGPM.Underthissystemtherearesevenwell-definedbase units,whicharedefinedbyinternationalagreementbytheCGPMandwhichare,byconvention,regardedasdimensionallyindependent.Thesebaseunitsare:themetre,the kilogram, the second, the ampere, the kelvin, the candela, and the mole (seeTable 3.1).AllotherunitswithintheSIaretermedderived units,formedbycombin-ingbaseunitsaccordingtothealgebraicrelationslinkingthecorrespondingquanti-ties (which are accordingly described as base quantities and derived quantities).Thusalthoughforsimplicitysomederivedunitshavebeengivenspecialnames,such

Y116134.indb 53 10/10/11 2:11 PM


Table 3.1 BasequantitiesandunitsoftheSI

SI base quantity SI base unitUnit

symbolQuantity symbol Definition

Length Metre m l,x,etc. Themetreisthelengthofthepathtravelledbylightinavacuumduringatimeintervalof1/299,792,458ofasecond.

Mass Kilogram kg m Thekilogramistheunitofmass;itisequaltothemassoftheinternationalprototypeofthekilogram.

Time,duration Second s t Thesecondisthedurationof9,192,631,770periodsoftheradiationcorrespondingtothetransitionbetweenthetwohyperfinelevelsofthegroundstateofthecaesium133atom.

Electriccurrent Ampere A I,i Theampereisthatconstantcurrentwhich,ifmaintainedintwostraightparallelconductorsofinfinitelength,ofnegligiblecircularcross-section, andplaced1metreapartinavacuum,wouldproducebetweentheseconductorsaforceequalto2×10-7newtonpermetreoflength.

Thermodynamictemperature

Kelvin K T Thekelvin,unitofthermodynamictemperature,isthefraction1/273.16ofthethermodynamictemperatureofthetriplepointofwater.

Luminousintensity

Candela cd Iv Thecandelaistheluminousintensity,inagivendirection,ofasourcethatemitsmonochromaticradiationoffrequency540×1012hertzandthathasaradiantintensityinthatdirectionof1/683wattpersteradian.

Y116134.indb 54 10/10/11 2:11 PM


asthehertz,volt,watt,newton,andlumen,allderivedunitscanalsobeexpressedasproductsofpowersofthebaseunits.Theequationsthatexpressthederivedquanti-tiesintermsofthebasequantitiesalsodefinetheexpressionforthederivedunitsintermsofthebaseunits.Forexample,thederivedquantityvelocityisdefinedasthedistancemoveddividedbythetimetaken,thatis,aslengthdividedbytime,sothederivedunitforvelocityismetrepersecond(m·s–1).Similarlythederivedquantityforce,F,isrelatedtotheSIbasequantitiesthroughtherelationship:

F maml

t= = 2

wheremisthemass,aistheacceleration,listhedistancemoved,andtthetimetaken.ThustheSIunitforforceiskilogrammeterpersecondsquared(kg·m·s–2),whichisgiventhespecialnameofthenewton,symbolN.

ItisimportanttonotethattheSIisnotstatic,butcontinuestoevolvetomatchtheworld’sincreasinglydemandingrequirementsformeasurement.Historicallythefirstunitstobedefinedasbaseunitswerethemetre,thekilogram,andthesecond,withtheampere,thekelvin,andthecandelabeingaddedbyaresolutionofthe10thCGPM,in1954,andthemolebeingagreedasaseventhbaseunitatthe14thCGPMin1971,afterlengthydiscussionsbetweenphysicistsandchemists.ThedefinitionofeachbaseunitoftheSIiscarefullydrawnupsothatitisuniqueandprovidesasoundtheoreticalbasisuponwhichthemostaccurateandreproduciblemeasurementscanbemade.Therealizationofthedefinitionofaunitistheprocedurebywhichthis

Table 3.1 (continued) BasequantitiesandunitsoftheSI

SI base quantity SI base unitUnit

symbolQuantity symbol Definition

Amountofsubstance

Mole mol n 1. Themoleistheamountofsubstanceofasystemwhichcontainsasmanyelementaryentitiesasthereareatomsin0.012kilogramofcarbon12.

2. Whenthemoleisused,theelementaryentitiesmustbespecifiedandmaybeatoms,molecules,ions,electrons,otherparticles,orspecifiedgroupsofsuchparticles.

Inthisdefinition,itisunderstoodthatunboundatomsofcarbon12,atrestandintheirgroundstate,arereferredto.

Y116134.indb 55 10/10/11 2:11 PM


definition isused toestablish thevalueandassociateduncertaintyof theunit.Astechnologyhasadvanced,ithasbecomebothpossibleandnecessarytochangethedefinitionsofthebaseunits,toachieveloweruncertaintiesorgreaterconsistency.Forexample,theunitoftime,thesecond,wasatonetimeconsideredtobethefraction1/86,400ofthemeansolarday,withtheexactdefinitionof“meansolarday”beingleft to the astronomers. However, measurements showed that irregularities in therotationoftheEarthmadethisanunsatisfactorydefinition,anditwouldcertainlynotbesuitableforuseinmoderntechnologicalapplications,suchasglobalposition-ing systems. This need for more precise and reproducible measurements of time,coupledwithadvancesintechnologythatallowedmeasurementsoftimebasedonthefrequencyofatomicoscillations,hasledtotheadoptionofanatomicstandardoftime,basedonatransitionbetweentwoenergylevelsofthecesium133atom.ThemostrecentchangeinthedefinitionofoneoftheSIbaseunitswasin1983(atthe17thCGPM)andwasrelatedtothedefinitionoftheunitoflength,themetre.Manyofthechangesthathavebeenmadetothedefinitionoftheunits,particularlysincetheearliestdaysoftheSI,reflectadesiretoensurethatallthebaseunitscanbereal-izedindependentlyfromfirstprinciples,withoutreferencetoaphysicalartifact;anobjectivewhichisdrivingcurrentinternationalresearchaimedatestablishinganewdefinitionforthekilogram(thelastunittobelinkedtoaphysicalreferenceartifact).

Inthecontextofmeasurementsrelatingtohumanperceptionandcognition,whichisamajorfocusof“measuringtheimpossible,”thepositionoftheSIwithrespecttounitsforquantitiesthatdescribebiologicaleffectsisparticularlyimportant.TheseunitsareoftendifficulttorelatetoSIunitsbecausetheytypicallyinvolveweightingfactorsthatmaynotbepreciselyknownordefined,andwhichmaybebothenergyandfrequencydependent.Forexample,electromagneticradiationcancausechemicalchangesinliv-ingornonlivingmaterials,butthenatureandmagnitudeofthesechangesdependsonthefrequency(or,equivalently,thewavelength),power,andgeometricalpropertiesoftheradiation.AsaresultitisnotpossibletoestablishasimplerelationshipbetweentheSIbaseunitsandthebiologicalresponsethatappliesforallelectromagneticradiation,orindeedevenforallsituationsacrossspecificlimitedspectralregions,suchastheultraviolet.Uniqueinthisrespectisthecandela,whichhasbeenformallydefinedasoneoftheSIbaseunitsandappliestomeasurementsoftheinteractionoflightwiththehumaneyeinvision(so-calledphotometry).Severalotherphotometricquantitieswithunitsderivedfromthecandelahavealsobeendefined,suchasthelumenandthelux.ForallotherphotobiologicalandphotochemicalmeasurementstheSIrecommendsthattheopticalradiationinquestionbecharacterizedbyitsspectraldistribution.Thismaythenbeweightedbytheactionspectrum(i.e.,therelativespectraleffectivenessof optical radiation) for thephotochemical or photobiological effect considered, togiveanintegralvalue,butinthiscasetheunitusedfortheintegratedradiantquantityisthesameasthatfortheunderpinningspectralquantity;thusitisessentialalsotostatetheactionspectrumthathasbeenusedinordertounderstandthemeaningofthequantitativevalue.AnexampleisgiveninTable 3.2.

Manypsychophysicalresponsescanbecharacterizedintermsofdimensionlesslogarithmicratios,mostnotablytheresponsetochangesinsoundintensityatagivenfrequency.TherearenospecificSIunitsforsuchlogarithmicratios,buttheCIPM

Y116134.indb 56 10/10/11 2:11 PM


Table 3.2 Calculationofphotobiologicalquantitiesinradiometry

Wavelength, λnm

Lamp spectral flux, Φ(λ)W

UV hazard action spectrum, suv(λ)

Erythemal action spectrum, ser(λ)

<200 0.000000 — —200 0.000000 0.030000 1.000000205 0.051863 0.051000 1.000000210 0.112560 0.075000 1.000000215 0.236439 0.095000 1.000000220 0.303842 0.120000 1.000000225 0.330000 0.150000 1.000000230 0.360590 0.190000 1.000000235 0.441261 0.240000 1.000000240 0.471595 0.300000 1.000000245 0.503237 0.360000 1.000000250 0.526061 0.430000 1.000000255 0.555685 0.520000 1.000000260 0.630966 0.650000 1.000000265 0.622298 0.810000 1.000000270 0.771722 1.000000 1.000000275 0.835807 0.960000 1.000000280 0.813363 0.880000 1.000000285 0.767389 0.770000 1.000000290 0.724387 0.640000 1.000000295 0.729527 0.540000 1.000000300 0.757817 0.300000 0.648634305 0.732355 0.060000 0.219786310 0.709882 0.015000 0.074473315 0.713278 0.003000 0.025235320 0.706919 0.001000 0.008551325 0.649260 0.000500 0.002897330 0.640969 0.000410 0.001413335 0.630451 0.000340 0.001189340 0.625314 0.000280 0.001000345 0.677279 0.000240 0.000841350 1.271558 0.000200 0.000708355 0.707310 0.000160 0.000596360 0.640000 0.000130 0.000501365 0.541735 0.000110 0.000422370 0.455264 0.000093 0.000355375 0.300987 0.000077 0.000299380 0.200462 0.000064 0.000251385 0.100364 0.000053 0.000211

(Continued)

Y116134.indb 57 10/10/11 2:11 PM


hasofficiallyacceptedtheunitsoftheneper,bel,anddecibelforusewiththeSI.Theneper,Np,isusedtoexpressthevaluesofquantitieswhosenumericalvaluesarebasedontheuseoftheneperian(ornatural)logarithm,ln=loge.Thebelandthedecibel,BanddB,where1dB=(1/10)B,areusedtoexpressthevaluesofloga-rithmicratioquantitieswhosenumericalvaluesarebasedonthedecadiclogarithm,lg=log10.Measurementsofsoundintensity(or“loudness”)areusuallyexpressedindecibels,relativetoaspecified0dBreferencewhichistypicallysetatthethresholdofperceptionofanaveragehuman.Themainreasonforusingthedecibelforsoundperceptionisthattheeariscapableofdetectingaverylargerangeofsoundpres-sures,coveringaratioinexcessofamilliontoone.Becausethepowerinasoundwaveisproportionaltothesquareofthepressure,theratioofthemaximumpowertotheminimumpowerismorethan1012.Todealwithsucharange,logarithmicunitsareuseful:thelogarithmof1012is12,sotheratioofmaximumpowertominimumpowerforhumanauditoryperceptionrepresentsadifferenceof120dB.Whenmak-ingmeasurementsofsoundforhumanperception,accountalsohastobetakenofthefactthatthehumanearisnotequallysensitivetoallsoundfrequencies:thisprocessiscalledfrequencyweightingandissimilartotheprocessusedforspectralweight-ingofphotobiologicalquantities.

3.3 Relevant physical parameters for studies in sensory science

Weinteractwithourenvironment,andwithobjectswithinthatenvironment,throughourfivesenses;thereforethephysicalmeasurementsthataremostrelevantforsen-sorysciencearethoserelatingtotheparametersthataresensedthroughoursensorytransducers,suchaslightandsurfacereflectance(inthecaseofvision)orsurfaceroughnessandthermaleffusivity(inthecaseoftouch).ThesearelistedinTable 3.3.However, as mentioned previously, other types of physical measurement are alsoof interest where they are used in instrumentation to measure human behavioral

Table 3.2 (continued) Calculationofphotobiologicalquantitiesinradiometry

Wavelength, λnm

Lamp spectral flux, Φ(λ)W

UV hazard action spectrum, suv(λ)

Erythemal action spectrum, ser(λ)

390 0.054211 0.000044 0.000178395 0.000000 0.000036 0.000150400 0.000000 0.000030 0.000126

>400 0.000000 — —Integratedvalue(inwatts–theactionspectrumusedmustalsobequoted)

20.904 6.071 10.525

Note: Inthisexamplethetotallampfluxis20.904W,theUVhazardfluxis6.071W(weightedusingtheUVhazardactionspectrum)andtheerythemalfluxis10.525W(weightedusingtheerythemalactionspectrum).Whenquotingtheresult,theactionspectrummustbegiven,aswellastheunit(W).

Y116134.indb 58 10/10/11 2:11 PM


Table 3.3 Keyphysicalparametersforsensorytransduction

Sensory modality Physical parameter and SI units Sensory response

Vision Luminance(candela per metre squared, cd · m–2):lightreflectedfromoremittedbyasurface

Brightness

Gloss (dimensionless):lightreflectedinspecificdirectionsrelativetotheincidentdirection

Shininess

Chromaticity for non-self-luminous surfaces (dimensionless):spectralreflectanceofsurfacecombinedwithspectralirradianceofilluminatinglightsource

Color

Chromaticity for self-luminous surfaces (dimensionless):spectralradianceofsurface

Color

Dimensional characteristics (metre, m):length,volume,etc.

Sizeandshape

Chroma, saturation, hue and other color appearance measures (dimensionless): derivedfromspectralreflectanceandspectralradiance/irradiancemeasurements,definedintermsofvariouscolormeasurementsystems

Colorappearanceinthecontextofthevisualenvironment

Goniometric and spatial surface characteristics (dimensionless):spectralreflectanceasafunctionofpositionandanglecombinedwithspectralirradianceofilluminatinglightsourceasafunctionofpositionandangle

Visualtextureandpattern

Light scattering characteristics (dimensionless):spectraltransmittanceasafunctionofposition,angleandthickness

Transparency,clarity,haze,translucency

Touch Surface topography (metre, m):heightofsurfaceasafunctionofposition

Roughness/smoothness

Friction (Newton, N):forceexperiencedwhenmovingafingertipoverthesurface

Stickiness,slipperiness

Hardness (dimensionless):resistancetoindentation(measuredonvariousdefinedratioscales)

Hardness

Tensile strength, elasticity (pascal, Pa, or newton per metre squared, N · m–2):resistancetodeformation

Stretchiness,bendability,drape,compressibility

Thermal effusivity (joule per metre squared per kelvin per square root second, J · m–2 · K–1 · s–½ or watt square-root second per metre squared per kelvin, W · s½ · m–2 · K–1):abilityofamaterialtoexchangethermalenergywithitssurroundings

Coldness,wetness

(Continued)

Y116134.indb 59 10/10/11 2:11 PM


responses,suchasbrainactivity.ThosephysicalparametersthataremostrelevantinthiscontextarelistedinTable 3.4.

3.4 Calibration, traceability, and measurement uncertainty

Calibration is the act of checking or adjusting (by comparison with a standard)the values shown by a measuring instrument in order to confirm their validity:thatis,it istheprocessofassigning“correct”valuestothemeasurementscaleof

Table 3.3 (continued) Keyphysicalparametersforsensorytransduction

Sensory modality Physical parameter and SI units Sensory response

Sound Acoustic pressure (pascal, Pa, or newton per metre squared, N · m–2):soundwaveamplitude

Loudness

Acoustic intensity (watt per metre squared, W · m–2):soundpowerperunitarea

Loudness

Acoustic frequency (hertz, Hz):soundwavefrequency Pitch,sharpness,tonequality,timbre

Acoustic impedance (decibel, dB—note this is accepted for use with SI, but is not an SI unit):attenuationofsoundwavesthroughamedium

Muffled

Tasteandsmell

Chemical composition (mole per metre cubed, mol · m–3)

Flowery,fruity,salty,sweet,bitter,sour….

Table 3.4 Keyphysicalparametersrelevantforinstrumentationusedtomeasurebrainactivityandphysiologicalresponsesforhumanbehavioralstudies

Instrumentation/method Physical parameters

Electroencephalography(EEG) Voltage(volt,V),time(second,s),length(metre,m)Electromyography(EMG) Voltage(volt,V),time(second,s)Event-relatedpotential(ERP) Voltage(volt,V),time(second,s)Functionalmagneticresonanceimaging(FMRI)

Magneticfieldstrength(amperepermetre,A·m–1),time(second,s),length(metre,m)

Functionalnear-infraredimaging(FNIR)

Opticalradiationintensity(watt,W),time(second,s),length(metre,m)

Magnetoencephalography(MEG)


Magneticinductiontomography(MIT)


Positronemissiontomography(PET)

Radionuclideactivity(becquerel,Bq),time(second,s),length(metre,m)

Transcranialmagneticstimulation(TMS)

Magneticfieldstrength(amperepermetre,A·m–1)

Heartrate Frequency(hertz,Hz)Bloodpressure Pressure(pascal,Paornewtonpermetresquared,N·m–2)

Y116134.indb 60 10/10/11 2:11 PM


an instrument.At its simplest level, therefore, calibration ismerelyacomparisonbetweenmeasurements:oneameasurementofknownmagnitudeusingadeviceofknownperformance(thereferenceorstandard)andtheotherusingthetestdevicewhoseperformanceistobecalibrated.

Traceability is the process of linking the calibration values to internationallyrecognizedmeasurementstandards(realizedatnationalmeasurementlaboratories)through an unbroken chain of calibrations. One of the simplest ways of ensuringtraceabilityisbyusinganapprovedcalibrationlaboratorythathasbeenassessedbyanindependentaccreditationbodyagainsttherequirementsofISO17025(InternationalOrganizationforStandardization,2005).Thisisnot theonlyapproach(e.g.,manynationalmeasurement laboratorieshave chosennot tohave a formal accreditationagainstISO17025)butifusinganonaccreditedlaboratory,itisuptotheusertover-ifythatthelinkagetonationalmeasurementstandardscanbeclearlydemonstrated.

Measurement uncertaintyisdefinedas“aparameter,associatedwiththeresultofameasurement,thatcharacterizesthedispersionofthevaluesthatcouldbereason-ablyattributedtothemeasurand”(JointCommitteeonGuidesinMetrology,2008).Itisusuallyexpressedintermsoftheboundsofanintervalwithinwhichthecor-rectresultofthemeasurementmaybereasonablypresumedtolie.Theconceptofmeasurementuncertaintyisusedtorecognizethefactthat,asexplainedinSection3.4.2below,nomeasurementortestiseverperformedperfectlyandtheimperfec-tionsintheprocessandtheinstrumentationusedwillgiverisetoerrorintheresult.Consequently,theresultofameasurementis,atbest,onlyanapproximationtothetruevalueofthequantitybeingmeasured(the“measurand”)andisonlycompletewhenthemeasuredvalueisaccompaniedbyastatementoftheuncertaintyofthatapproximation.Thisuncertaintyevaluationmust includenotonly theuncertaintycontributionsarisingfromthemeasurementprocess,butalso(wherethis isasig-nificantcontributiontotheoveralluncertainty)theuncertaintyassociatedwiththecalibrationoftheinstrumentationused.

3.4.1 Terms related to measurement uncertainty

Thetermsrepeatability,reproducibility,precision,accuracy,anderrorareoftenusedincorrectlyandthereforemeritspecialdiscussion.Althoughtheyareallassociatedinsomewaywithmeasurementuncertainty, theyhavedifferentandveryspecificmeanings:

• Repeatability:Ameasureofthespreadinresultsunderconditionswhereindependent results are obtained with the same method on identical testitemsinthesamelaboratorybythesameoperatorusingthesameequip-mentwithinshortintervalsoftime.Theusualwaytoquantifyrepeatabilityisintermsofthestandarddeviationoftheresults.

• Reproducibility:Ameasureofcloseness-of-agreementbetweentheresultsofmeasurementsofthesamepropertyonidenticaltestitems,whencarriedoutunderchangedconditionsofmeasurement(e.g.,byadifferentopera-tororadifferentmethod,oratadifferenttime).Reproducibilityisusually

Y116134.indb 61 10/10/11 2:11 PM


quantifiedintermsofthestandarddeviationof themeanresultobtainedfromeachoftheindependentsetsofmeasurements.

• Precision:Inscienceandmetrologythisreferstotheclosenessofagreementbetweenindicationsormeasuredquantityvaluesobtainedbyreplicatemea-surementsonthesameorsimilarobjectsunderspecifiedconditions.Itcanbesynonymouswithrepeatabilityorwithreproducibilitydependingontheconditionsunderwhichthereplicatemeasurementsaremade.Precisionisoftenmisused,especiallyincolloquialusage,tomean“accuracy,”“uncer-tainty,”“resolution,”or“finenessofdiscrimination,”anditshouldthereforebeusedwithcaution.

• Measurement accuracy: An expression of the closeness-of-agreementbetween themeasuredvalueand the truevalue.As the truevalue isnotknown,accuracyisaqualitativetermonlyandcannotbegivenanumericalvalue.Ameasurementissaidtobemoreaccuratewhenitoffersasmallermeasurementerror.

• Measurement error:Thisisdefinedasthemeasuredquantityvalueminusareferencequantityvalue.Itshouldnotbeconfusedwithaproductionerrorormistake.Measurementerrorscantaketwoforms:• Systematic measurement error:Thecomponentofmeasurementerror

thatinreplicatemeasurementsremainsconstantorvariesinapredict-able manner. Systematic measurement error, and its causes, can beknownorunknown.Acorrectioncanbeappliedtocompensateforaknownsystematicmeasurementerror.

• Random measurement error: The component of measurement errorthatinreplicatemeasurementsvariesinanunpredictablemanner.

3.4.2 Selection of measurement instruments, methods, and standards

Anumberoffactorscaninfluencetheaccuracyandvalidityofameasurementresultandthesemustbeconsideredwhendecidingonthesuitabilityoftheselectedmea-surementinstrumentfortheintendedpurpose,thereferenceorstandardtobeusedforthecalibrationofthechoseninstrument,thecalibrationandmeasurementproce-durestobeadopted,andtheconfidencethatcanbeplacedinsubsequentmeasure-mentsusingthecalibratedinstrument(i.e.,thefactorsthatmustbeconsideredwhendevelopingameasurementuncertaintybudget;seeSection3.5.4).Someofthemostimportantofthesefactorscanbesummarizedasfollows.(Notethisisnotanexhaus-tivelistandtherelativeimportanceofthese,andotherfactors,willdependontheactualmeasurementbeingperformed.)

1.Eachstepinthetraceabilitychainaffectstheaccuracyandvalidityofthefinalmeasurementresult.Generallythemorestepsinthecalibrationchain,thehigherthemeasurementuncertaintybecomes,anditisthereforeoftendesirabletominimizethenumberofstepsbetweenthenationalmeasure-ment standards and the reference used for the calibration. However, theuncertaintyassociatedwitheachstepinthechainmustalsobeconsidered:

Y116134.indb 62 10/10/11 2:11 PM


asmallnumberofstepseachwith relatively largeuncertaintywilloftenresultinahigheroveralluncertaintythanalongertraceabilitychainwithlowuncertaintyateachstepinthischain.

2.Themostaccurateresultsforanycalibrationareobtainedwhencomparinglikewithlike.Conversely,everydifferencebetweenthereferenceandtestinstruments,orbetween the referenceand testartifacts, introducesaddi-tionaluncertaintyintotheresults.Forexample,fewerpotentialsourcesofmeasurementuncertaintywillbeintroducedwhenmeasuringtheluminanceofavisualdisplayscreenifthemeasuringinstrumenthasbeencalibratedbyreferencetoasimilartypeofdisplayratherthan,say,atungstenfilamentlamp.Ifpossible,therefore,itisgenerallypreferabletocalibrateameasur-inginstrumentusingareferenceinstrumentthathassimilarpropertiestothetestinstrument.Butinanycase,itmustalwaysbeknownwhatreferencewasusedforthecalibration,becausewithoutthisinformationitisnotpos-sibletocarryoutaproperassessmentofthemeasurementuncertainty.

3.Therelationshipbetweenthevaluesdisplayedbythetest instrumentandthe“correct”valuesmayvarybetweendifferentrangesontheinstrument(“range-changeerrors”),orwithinarange(“linearityerrors”).Itisthere-foreimportanttocalibratetheinstrumentoneachrangeoverwhichitwillbeused,andforseveralpointsspanningeachoftheseranges.

4.Strictly speaking, a calibration only applies at the time it is carried out.Instrumentperformancemaydriftwithtimeorwithuse,sothecalibrationmustberepeatedatregularintervalsinordertoassessandallowforsuchdrifts.Thisregularrecalibrationallowsa“calibrationhistory”tobeestab-lished,whichgivesconfidenceinthereliability(orotherwise)oftheresultsofmeasurementsmadeusingtheinstrument.Asuitablerecalibrationintervalshouldthereforebespecifiedinthecalibrationprocedurefortheinstrument.

5.Theresultsofacalibrationdependontheenvironmentalandoperationalconditionsatthetimeofmeasurement(ambienttemperature,humidity,sup-pliedcurrentorvoltage,etc.).Changesintheseconditionsmayhaveasig-nificantimpactontheresultsofmeasurementsmadeusingtheinstrument;themostaccurateresultswillbeobtainedonlyifthecalibrationconditionsarecarefullyreproducedduringuse.

6.Eveniftheenvironmentalandoperationalconditionsarewellcontrolled,repeated measurements may not agree with each other, due to randomeffectsassociatedwiththemeasuringinstrumentorwiththeartifactbeingmeasured.Forexample,onesourceofuncertainty inelectricalmeasure-ments is thermal noise, arising from the equilibrium fluctuations of theelectriccurrent insideanelectricalconductordue to therandomthermalmotionofthechargecarriers.Itisnotpossibletoeliminatesuchrandomeffects by the application of correction factors, but they can be allowedforbymakingseveralrepeatmeasurementsfromwhichameanvalueandstandarddeviationcanbecalculated;increasingthenumberofindicationswillreducetheuncertaintyinthemeanvalueduetotheireffect.

Y116134.indb 63 10/10/11 2:11 PM


7. Instrumentresolutioncanhaveasignificantimpactontheresultsofacalibra-tionandmustbeconsideredwhenchoosingthepointsatwhichtoperformthecalibration.Forexample,iftheinstrumentisonlyabletoreadwithareso-lutionof1unit,performingthecalibrationatapointthatgeneratesareadingof10unitswillmeanthatthebestuncertaintythatcanbeachievedisapprox-imately 10%, even if all other sources of uncertainty are zero. Similarlyresolution influences theuncertainty in themeasurementsmadeusing theinstrument;itisgenerallypreferabletochangetoarangewithhighersensi-tivitywhenmeasuringvaluestowardthelowerendofagivenrange.

3.5 Evaluating measurement uncertainty

Thissectiondescribestheconventionalapproachtoevaluationofmeasurementuncer-tainty,whichisapplicabletoabroadspectrumofmeasurementsandwillbesuitableformostmeasurementsinthefieldofmeasuringtheimpossible.Otherapproachesmaybemoreapplicableinspecificcases,forexample,numericalapproachessuchasaMonteCarlomethodandthosebasedonaBayesiananalysis.Itisonlypossibleheretogiveabriefintroductiontotheprinciplesofuncertaintyevaluation;furtherinformationcanbefoundinJointCommitteeonGuidesinMetrology,2008;UnitedKingdomAccreditationService,2007;CoxandHarris,2006;andBell,2001.

3.5.1 Defining the measurement function

Beforecarryingoutanymeasurement it isnecessary todecidewhatquantityY istobemeasuredandtoselectanappropriatemeasurementprocedurefor thismea-surand.Havingdefined themeasurementprocedure, it is thenpossible to identifywhataspectsofthisproceduremayinfluencetheresultofthemeasurement:thatis,todetermineallsignificantinputquantitiesXithatmayaffecttheoutputY.Basedonthisinformationit ispossibletodevelopamodel, theso-calledmeasurementfunction,thatdescribesthefunctionalrelationfbetweentheinputquantitiesandtheoutput:

Y=f (X1,X2,…,XN). (3.1)

ThusthefunctionfofEquation(3.1)expressesnotsimplyaphysicallaw,butamea-surementprocess,andinparticular,itshouldcontainallquantitiesthatcancontrib-uteasignificantuncertaintytothemeasurementresult.

ThevalueofYcannotbeknownexactly;insteadanestimateofthisvalue,denotedbyy,isobtainedfromEquation (3.1)usinginputestimatesx1,x2,…,xNforthevaluesoftheNinputquantitiesX1,X2,…,XN.Thus,theoutputestimatey,whichistheresultofthemeasurement,isgivenby

y=f (x1,x2,…,xN). (3.2)

3.5.2 Calculating the measurement uncertainty

Theuncertaintyofthemeasurementresultycanbeevaluatedfromknowledgeoftheuncertaintiesu(xi)(sometimesreferredtoasuiforbrevity)oftheinputestimates

Y116134.indb 64 10/10/11 2:11 PM


xithatareusedinEquation (3.2).Theseuncertaintiesfortheinputquantitiescanbedeterminedintwodifferentways:

• Type A uncertaintiesareevaluatedbythestatisticalanalysisofaseriesofindications.

• Type B uncertaintiesareevaluatedbyothermethods,andareusuallybasedonscientificjudgmentusingall therelevantinformationavailable,whichmayinclude:• Previousmeasurementdata• Experiencewith,orgeneralknowledgeof,thebehaviorofthematerials

orinstrumentsused• Manufacturer’sspecifications• Dataprovidedincalibrationreports

Eachcomponentofuncertainty,howeveritisevaluated,isrepresentedbyastan-darddeviation,termedthestandarduncertaintyu(xi),whichisequaltothepositivesquarerootofthevarianceu2(xi).ForaTypeAuncertaintycomponentthevariancecanbedetermineddirectlybystatisticalmethods.ForaTypeBuncertaintycompo-nentthevariance(andthusthestandarduncertainty)isobtainedfromanassumedprobabilitydistributionfunction(PDF),whichdescribestherangeofpossiblevaluesthatthemeasuredvariablecanattainandtheprobabilitythatthevalueofthevari-able iswithinanysubsetof that range. In themajorityofcasesoneof twoPDFscan be assigned (other PDFs may apply under special circumstances; see UnitedKingdomAccreditationService,2007;andCox&Harris,2006):

• AnormalorGaussiandistribution(seeFigure 3.1).Thisisused,forexam-ple,whenallowingfortheuncertaintyassociatedwiththecalibrationofameasuring instrument. The calibration certificate for such an instrumentwilltypicallygivetheuncertaintyforastatedlevelofconfidence,usually

Measured Value, x

Prob

abili

ty, p

Figure 3.1 Thenormal,orGaussian,probabilitydistribution.Thesizeofthedistributionisdescribedintermsofastandarddeviation.Theshadedarearepresents±1standarddeviationfromthecenterofthedistribution.Thiscorrespondstoapproximately68%oftheareaunderthecurve.

Y116134.indb 65 10/10/11 2:11 PM


95%,andthestandarduncertaintycanthenbecalculatedbytreatingthequoteduncertaintyasifanormaldistributionhadbeenusedtocalculateit(unlessotherwiseindicated)anddividingitbytheappropriatefactorforanormaldistribution(1.96fora95%confidencelevel).

• Arectangulardistribution(seeFigure 3.2).ThisisusedinsituationswherethereisnospecificknowledgeaboutthepossiblevaluesofXibetweenanestimatedupperandlowerlimitandonecanonlyassumethatitisequallyprobableforXitotakeanyvaluewithinthesebounds,withzeroprobabilityofbeingoutsidethem.Itisgenerallyareasonabledefaultdistributionintheabsenceofanyotherinformation.Inthiscasethestandarduncertaintyisgivenbythehalf-widthoftherectangularlimits,dividedby√3.Consider,forexample,ameterindicatingadisplayedreadingof101;herethevalueofthequantityhasequalprobabilityoflyinganywherebetween100.5and101.5andthestandarduncertaintyistherefore0.5/√3,or0.29.

Theestimatedstandarddeviationofthemeasurementresultisgivenbythecom-binedstandarduncertaintyuc(y),whichisobtainedbycombiningtheindividualstan-darduncertaintiesu(xi)usingthelawofpropagationofuncertainty.Thisiscommonlyreferredtoastheroot-sum-of-squaresorRSSmethodofcombininguncertaintycom-ponentsestimatedasstandarddeviations,andisgivenbyEquation(3.3)below.(NoteEquation(3.3)givesasimplifiedversionof the lawofpropagationofuncertainty,whichapplieswhentheinputquantitiesareuncorrelated.Formoredetailedinforma-tionthereaderisreferredto:JointCommitteeonGuidesinMetrology,2008;UnitedKingdomAccreditationService,2007;andCox&Harris,2006.)

u y c u xc i

i

N

i2

1

2( ) [ ( )]==

∑ . (3.3)

Measured Value, x

Prob

abili

ty, p

a

xi – a xi + axi

a

Figure 3.2 Therectangularprobabilitydistribution.Thedistributionofpossiblevalueshasahalf-width,orsemi-range,ofa;thereisequalprobabilityofthevalueofxibeinganywherewithintherangexi–a toxi+a,andzeroprobabilityof itbeingoutsidetheselimits.Thestandarddeviationisgivenbya/√3.

Y116134.indb 66 10/10/11 2:11 PM


Theparameterciistermedthesensitivitycoefficient,andquantifieshowchangesintheinputquantityXiaffecttheoutputquantityY.Inotherwords,itdescribeshowtheoutputestimateyvarieswithacorrespondingsmallchangeinaninputestimatexi.ItisobtainedbycalculatingthepartialderivativeofthefunctionfwithrespecttoXianddeterminingthevalueofthispartialderivativeusingtheestimatedvaluesofalltheinputquantities.However,thecalculationsrequiredinordertodeterminesen-sitivitycoefficientsbypartialdifferentiationcanbealengthyprocess,particularlywhentherearemanyinputcontributionsanduncertaintyestimatesareneededforarangeofvalues.Itisoftenbeneficial,therefore,toobtainthesensitivitycoefficientsdirectlythroughthepracticalapproachofchangingoneoftheinputvariablesbyaknownamount,whilekeepingallother inputsconstant,andnoting thechange inthemeasuredoutput(thesensitivitycoefficient is thengivenbythechangein themeasuredoutputdividedbythechangeintheinputquantity).

Itshouldalsobenotedthatifthefunctionalrelationshipbetweentheinputandoutputquantitiesisasimpleadditionorsubtractionoftheinputquantities,thenalltheinputquantitiesaredirectlyrelatedtotheoutputquantityandthepartialderivativeswillallbeunity.Ifthefunctionalrelationshipisaproductorquotient(i.e.,theoutputquantityisobtainedfromonlythemultiplicationordivisionoftheinputquantities),thiscanbesimplifiedbytheuseofrelativevalues,forexample,thoseexpressedinpercentagetermsorinpartspermillion.Thegeneralformofsucharelationshipis:

Y X X Xp pNpN= …1 2

1 2 , (3.4)

wheretheexponentspiareknownpositiveornegativenumbers.Therelativestan-darduncertaintyisthengivenby

u yy

p u xx

c i i

ii

N( ) ( )=

=∑

2

1

. (3.5)

Theuseofrelativeuncertaintiescanoftensimplifythecalculationsandisparticu-larlyhelpful when the input quantities and theuncertainties are alreadygiven inrelativeterms.However,sensitivitycoefficientsmaystillberequiredtoaccountforknown relationships, such as the influence of a temperature coefficient. Relativeuncertaintiesshouldnotbeusedwhenthefunctionalrelationshipisalreadyanaddi-tionorsubtraction.

3.5.3 Reporting measurements and uncertainties

As explained in Section 3.4, the result of a measurement is only complete whenthemeasuredvalue(i.e.,theestimateofthemeasurand)isaccompaniedbyastate-ment of the associateduncertainty.Sometimes this is doneby simply stating themeasuredvaluetogetherwiththecombinedstandarduncertainty.Morecommonly,however,whatisrequiredisameasureofuncertaintythatdefinesanintervalabout

Y116134.indb 67 10/10/11 2:11 PM


themeasuredvaluey withinwhichthetruevalueofthemeasurandY isconfidentlybelievedtolie.ThisistermedtheexpandeduncertaintyU,andisobtainedbymulti-plyinguc(y)byacoveragefactork.Inotherwords:

U =kuc(y) (3.6)

anditisconfidentlybelievedthat

y – U≤Y≤y +U(usuallywrittenasY =y ±U). (3.7)

Most commonly a coverage factorof2 is used,which corresponds to a “levelofconfidence”or“coverageprobability”ofapproximately95% underaGaussianassumption. Inotherwords, the intervaly±U isexpected tocontain95%of thevalues that could be attributed to the output quantity on the assumption that theuncertainty distribution is Gaussian. Where a result is quoted with an expandeduncertaintyinsteadofastandarduncertainty,thecoveragefactorusedmustalwaysbestated.AnexampleisgiveninSection3.5.4below.

3.5.4 Calculating uncertainty: An example

Considertheproblemofmeasuringthecurrentflowingthroughanelectricalcircuitbymeasuring thevoltagedropacross a calibrated reference resistorusing a cali-brateddigitalvoltmeter(DVM).ThecurrentIisgivenbytheequationI=V/RwhereVisthevoltagedropmeasuredacrossthestandardresistorR.Supposethatthefol-lowingconditionsapply:

1.Thestandardresistorwascalibratedatatemperatureof20°C. 2.The calibrated resistance for the standard resistor is 1.060 Ω with an

expandeduncertaintyof0.001Ωforacoveragefactorofk=2. 3.Theoperatingtemperatureoftheresistoratthetimeofmeasurementis30°C

andthetemperaturecoefficientforthisresistorisknowntobe+0.001ΩperdegreeCelsiuswithastandarduncertaintyof0.0001ΩperdegreeCelsius.

4.Themanufacturer’sdatasheetforthestandardresistorstatesamaximumdriftinresistanceof±0.0005Ωperyear;theresistorwascalibratedwithinthepreviousyearandnocorrectionforpossibledrifthasbeenapplied.

5.TheDVMhasbeencalibratedandadjustedsothatitneedsnocorrectiontothedisplayedvaluesonthe1Vrangeforanambienttemperatureof20°C,withastatedexpandeduncertaintyforthecalibrationof0.01%foracover-agefactorofk=2.

6.Theambienttemperatureatthetimeofmeasurementis25°Candthemanu-facturer’sdatasheetfortheDVMstatesthatthetemperaturecoefficientdoesnotexceed±0.001%perdegreeCelsius;nocorrectionisappliedforthedif-ferenceintemperaturebetweentheambientconditionsandthecalibrationconditionsbecauseabestestimateofzeroisimpliedbytheinformation.

7.ThereadoutresolutionoftheDVMis0.1mV.

Y116134.indb 68 10/10/11 2:11 PM


8.Themanufacturer’sdatasheetfortheDVMstatesamaximumdriftincali-brationof+0.8mVperyear;theDVMwaslastcalibratednearlyoneyearagoandithasthereforebeendecidedthatacorrectionforpossibledriftof+0.4mVshouldbeapplied,withamaximumuncertaintyof±0.4mV.

9.A totalof10 independent readingsare taken from theDVMon the1Vrange,withthefollowingresults:0.8081,0.8090,0.8093,0.8096,0.8084,0.8098,0.8083,0.8086,0.8092,0.8099.

Basedonthisinformation,thefollowingmeasurementfunctionapplies:

IV V V TR R R Tm d t V

c d t R

= + ++ +

∆∆

, (3.8)

wheretheinputquantitiesareasfollows:

Vm =voltagemeasuredusing theDVM,whichhasassociateduncertaintiesarisingfrom:

DVMcalibration,Vc

Voltagemeasurementrepeatability,u(V)=s(Vm)DVMresolution,Vr

Vd =driftintheDVMsincecalibration.Vt =temperaturecoefficientoftheDVM.ΔTv =temperaturedifferencebetween theambient temperatureat the timeof

measurementandthetemperatureatthetimeofcalibrationoftheDVM.Rc =calibratedresistancevalueofthestandardresistor.Rd =driftinthestandardresistorsincecalibration.Rt =temperaturecoefficientofthestandardresistor.ΔTR=temperature difference between the operating temperature of the resis-

tor at the time of measurement and the temperature at the time of itscalibration.

ThemeanofthemeasuredvoltageVm,andtheexperimentalstandarddeviationofthemean(alsoknownasthestandarderrorofthemean)s(Vm),areobtainedbystatis-ticalmeansusingthefollowingequations,where nisthenumberofindications:

Vn

Vm m j

j

n

==

∑11

, (3.9)

s Vn n

V Vm m j m

j

n

( )( )

( ),=−

−=

∑11

2

1

(3.10)

ThisgivesVm=0.80902Vands(Vm)=0.20mV.

Y116134.indb 69 10/10/11 2:11 PM


Thismeansthatthebestestimateofthecurrentthroughtheresistoris:

IV V V TR R R Tm d t V

c d t R

= + ++ +

= + +∆∆

0 80902 0 0004 01

. ..0060 0 0 001 10+ + ×( . )

=0.7565A

Thisvalueon itsown isnot sufficient; the associateduncertaintymustbecalcu-latedbyestablishingtheuncertaintybudget(seeTable 3.5).Thisuncertaintybudgetincludesanallowancefor theuncertaintyassociatedwitheachof the inputquan-tities, and also states the form of the probability distribution, the divisor used toconvertfromthestateduncertaintyvaluetothestandarduncertainty,thesensitivitycoefficient,andthedegreesoffreedom.Inmostcasesthedegreesoffreedomisinfi-nite,butforTypeAuncertaintiesthedegreesoffreedomisonelessthanthenumberofindicationsonwhichtheuncertaintyevaluationisbased(i.e., it isgivenbythevalueofn–1inEquation(3.10)).Theeffectivedegreesoffreedomforthecombinedstandarduncertaintyνeffiscalculatedfromthedegreesoffreedomfortheindividualuncertaintycontributionsνi,usingtheWelch–Satterthwaiteformula:

ν

ν

effc

i

ii

N

u y

u y=

=∑

4

4

1

( )

( ), (3.11)

where

u y c u xi i i( ) ( )≡ . (3.12)

Knowledgeoftheeffectivedegreesoffreedomallowstherelationshipbetweenthecoveragefactorandtheconfidenceleveltobecalculated,usingtheStudent’stdistri-bution(tabulatedinUKASM3003,2007).Forνeffgreaterthanabout50,acoveragefactorof2correspondstoaconfidencelevelofapproximately95%.

FortheexamplegivenaboveandinTable 3.5,theexpandeduncertaintyofthecalibrationiscalculatedtobe0.23%,or0.0017A,foracoveragefactork = 2andwitheffectivedegreesoffreedomgreaterthan4000.Thefinalresultofthemeasurementwouldbequotedinoneofthefollowingforms:

Thecurrentflowingthroughthecircuitis0.7565A±0.0017A.Thereportedexpandeduncertaintyisbasedonastandarduncertaintymulti-

pliedbyacoveragefactork=2,providingalevelofconfidenceofapproxi-mately95%.

or

Thecurrentflowingthroughthecircuitis0.7565A±0.23%.Thereportedexpandeduncertaintyisbasedonastandarduncertaintymulti-

pliedbyacoveragefactork=2,providingalevelofconfidenceofapproxi-mately95%.

3.6 Notation

Table 3.6summarizesthemeaningofthenotationusedinthischapter.

Y116134.indb 70 10/10/11 2:11 PM


Tabl

e 3.

5 U

ncer

tain

tyb

udge

tfor

the

exam

ple

give

nin

Sec

tion

5.4

Sym

bol

Sour

ce o

f unc

erta

inty

Valu

e±

Valu

e(%

)P

roba

bili

ty

dist

ribu

tion

Div

isor

c iu i

(y)

Deg

rees

of

free

dom

Vc

DV

Mc

alib

ratio

n0.

01%

Nor

mal

21

0.00

5%∞

u(V

)V

olta

gem

easu

rem

ent

repe

atab

ility

0.20

mV

0.02

5%N

orm

al

11

0.02

5%9

Vr

DV

Mr

esol

utio

n0.

05m

V0.

006%

Rec

tang

ular

√31

0.00

3%∞

Vd

DV

Md

rift

0.40

mV

0.04

9%R

ecta

ngul

ar√3

10.

028%

∞V

tTe

mpe

ratu

re

coef

ficie

nto

fD

VM

5°C

and

tem

pera

ture

coe

ffici

entn

ot

exce

edin

g0.

001%

per

°C

0.

005%

Rec

tang

ular

√31

0.00

3%∞

Rc

Res

isto

rca

libra

tion

0.00

1Ω

0.09

4%N

orm

al2

10.

047%

∞

Rd

Res

isto

rdr

ift

0.00

05Ω

0.04

7%R

ecta

ngul

ar√3

10.

027%

∞

Rt

Tem

pera

ture

co

effic

ient

of

resi

stor

10°C

and

tem

pera

ture

coe

ffici

ent

unce

rtai

nty

0.00

01 Ω

per

°C

(k

=1

)0.

094%

Nor

mal

11

0.09

4%∞

u c(I

)C

ombi

ned

stan

dard

un

cert

aint

yN

orm

al0.

115%

>40

00

UE

xpan

ded

unce

rtai

nty

Nor

mal

(k

=2

)0.

23%

>40

00

Not

e:

The

unc

erta

intie

sin

col

umn

4ar

ere

lativ

eun

cert

aint

ies

fort

here

leva

ntin

putq

uant

ities

(in

this

cas

evo

ltage

orr

esis

tanc

e)a

ndth

ose

inc

olum

n8

are

rela

tive

unce

rtai

ntie

sfo

rth

eou

tput

qua

ntity

(in

this

cas

ecu

rren

t).

Y116134.indb 71 10/10/11 2:11 PM


Table 3.6 Summaryofnotationused

Symbol Meaning

ci SensitivitycoefficientusedtomultiplythevalueofaninputquantityXitoexpressitintermsofthemeasurandY;quantifieshowchangesintheinputquantityXiimpactontheoutputquantityY.Givenby:

cfx

ii

≡ ∂∂

f FunctionalrelationshipbetweenmeasurandYandinputquantitiesXionwhichYdepends,andbetweenoutputestimateyandinputestimatesxionwhichydepends

∂∂fxi

PartialderivativewithrespecttoinputquantityXioffunctionalrelationshipfbetweenmeasurandYandinputquantitiesXionwhichYdepends,evaluatedatXi=xiwherei=1,…N:

∂∂

= ∂∂ …

fx

fXi i x x xN1 2, , ,

k CoveragefactorusedtocalculateexpandeduncertaintyU=kuc(y)ofoutputestimateyfromitscombinedstandarduncertaintyuc(y)

N Numberofinputestimatesxionwhichthevalueofthemeasuranddependsn Numberofrepeatedobservationsq Arithmeticmeanoraverageofnindependentrepeatedobservationsqkof

randomlyvaryingquantityqqj jthindependentrepeatedobservationofrandomlyvaryingquantityqs(q) Experimentalstandarddeviationofthemeanq;standarduncertaintyobtained

fromaTypeAevaluation:

s qs q

n

j( ) =( )

s(qj) Experimentalstandarddeviationdeterminedfromnindependentrepeatedobservationsqjofq:

s qn

q qj j

j

n

( ) =−

−( )=

∑11

2

1

u2(xi) EstimatedvarianceassociatedwithinputestimatexithatestimatestheinputquantityXi;whenxiisdeterminedfromthearithmeticmeanoraverageofnindependentrepeatedobservations,u2(xi)isobtainedbyaTypeAevaluation:

u x s qn n

q qi j

j

n

2 2 2

1

11

( ) = ( ) =−( ) −( )

=∑

u(xi) StandarduncertaintyofinputestimatexithatestimatestheinputquantityXi;whenxiisdeterminedfromthearithmeticmeanoraverageofnindependentrepeatedobservations,u(xi)isobtainedbyaTypeAevaluation:

u x s qi( ) = ( )

Y116134.indb 72 10/10/11 2:11 PM


Acknowledgment and copyright

The preparation of this chapter was jointly funded by the UK Department forInnovation, Universities and Skills (through the National Measurement Office’sInnovationResearchandDevelopmentProgramme)andtheEuropeanCommission(throughNEST-PathfinderProjectnumber043297-MINET).

© Crowncopyright2009.ReproducedbypermissionoftheControllerofHMSOandtheQueen’sprinterforScotland.

References

AmericanAssociationfortheAdvancementofScience.(1892).Science,19(474),127.Bell,S.(2001).NPL measurement good practice guide no. 11 (Issue 2). A beginner’s guide to

uncertainty of measurement.London:NPL.Availablefromwww.npl.co.ukCox, M. G., and Harris, P. M. (2006). Software support for metrology best practice guide

no. 6. Uncertainty evaluation. (NPLReportDEM-ES-011).London:NPL.Availablefromwww.npl.co.uk

International Organization for Standardization. (2005). General requirements for the com-petence of testing and calibration laboratories(ISO/IEC17025:2005).Geneva:Inter-nationalOrganizationforStandardization.

JointCommitteeonGuidesinMetrology.(2008).Evaluation of measurement data—Guide to the expression of uncertainty in measurement (JCGM100:2008).Availablefromwww.bipm.org/en/publications/guides/gum.html

Table 3.6 (continued) Summaryofnotationused

Symbol Meaning

u2c(y) Combinedvarianceassociatedwithoutputestimatey

uc(y) Combinedstandarduncertaintyofoutputestimateyui(y) Componentofcombinedstandarduncertaintyuc(y)ofoutputestimateygenerated

bythestandarduncertaintyofinputestimatexi:

u y c u xi i i( ) = ( )

u x

xi

i

( )Relativestandarduncertaintyofinputestimatexi

U ExpandeduncertaintyofoutputestimateythatdescribesthemeasurandasanintervalY=y±Uhavingahighlevelofconfidence,equaltocoveragefactorktimesthecombinedstandarduncertaintyuc(y)ofy:

U=kuc(y)xi EstimateofinputquantityXi

Xi ithinputquantityonwhichmeasurandYdependsy EstimateofmeasurandY;resultofameasurementY Ameasurand

νi Degreesoffreedom,oreffectivedegreesoffreedom,ofstandarduncertaintyu(xi)ofinputestimatexi

νeff Effectivedegreesoffreedomofuc(y)

Y116134.indb 73 10/10/11 2:11 PM


Organisation Intergouvernementale de la Convention du Mètre. (2006). The International System of Units (SI) (8th edition). Paris: Bureau International des Poids et Mesures.Availablefromwww.bipm.org

UnitedKingdomAccreditationService.(2007).The expression of uncertainty and confidence in measurement (UKASPublicationM3003Edition2).London:UKAS.Availablefromwww.ukas.com

Y116134.indb 74 10/10/11 2:11 PM

75

Meaningful and meaningless statements in epidemiology and public health

Fred S. RobertsCenterforDiscreteMathematicsandTheoreticalComputerScience,RutgersUniversityPiscataway,NJ,USA

4.1 Introduction

The theory of measurement is an interdisciplinary subject that grew out of theattempt to put the foundations of measurement on a firm mathematical founda-tion. Building on classic examples of measurement in the physical sciences, thetheorywasmotivatedbytheattempttomakemeasurementineconomics,psychol-ogy, and other disciplines more precise. The theory traces its roots to the workofHelmholtz(1887/1930),andwaswidelyformalizedin the twentiethcenturyinsuchbooksasKrantz,Luce,Suppes,andTverksy(1971),Luce,Krantz,Suppes,andTversky(1990),Pfanzagl(1968),Roberts(1979/2009),andSuppes,Krantz,Luce,andTversky(1989).Measurementtheoryisnowbeginningtobeappliedinawidevarietyofnewareas.Littleknowninthefieldsofepidemiologyandpublichealth,the theory has the potential to make important contributions to epidemiologicalmeasurement. In turn, problems of epidemiology are posing new challenges formeasurementtheory.

Weseektoanswerquestionssuchasthefollowing:

• Isitmeaningfultosaythatthemalariaparasiteloadhasdoubled?• Is theaveragecoughscore foronesetofTBpatientshigher than that

foranother?• ForcontrollingthespreadofHIV,whichofabstinenceeducation,universal

screening,andcondomdistributionaremoreeffective?

Allofthesequestionshavesomethingtodowithmeasurement.Weprovideabriefintroductiontothetheoryofmeasurement,withanemphasisonthetypesofscalesthatcanarise.

In almost everypractical application in epidemiology, something ismeasured.Yetinmanycases,littleattentionispaidtothelimitationsthatthescalesofmeasure-mentbeingusedmightplaceontheconclusionsthatcanbedrawnfromthem.Scales

4

Y116134.indb 75 10/10/11 2:11 PM


maytosomedegreebearbitrary,involvingchoicesaboutzeropointsorunitsorthelike.Itwouldbeunwisetomakedecisionsthatcouldturnoutdifferentlyifthearbi-trarychoiceofzeropointorunitischangedinsome“admissible”way.Weareinter-estedinexploringtheextenttowhichthiscanhappen,andinlayingdownguidelinesforwhatconclusionsbasedonscalesofmeasurementareallowable.Wemaketheseideasprecisebyintroducingakeymeasurementtheoryconceptofmeaningfulness.Usingexamples from the studyofdiseases suchasHIV,malaria, and tuberculo-sis,wegiveavarietyofexamplesofmeaninglessandmeaningfulstatements.Moresubtleapplicationsincludeaveragingjudgmentsofcoughseverityorjudgmentsoffatigue,findingmeasuresofairpollutioncombiningdifferentpollutants,andevalu-atingalternativeHIVtreatmentsusing“merging”proceduresfornormalizedscores.Wealsodiscussthemeaningfulnessofstatisticaltestsandofanswerstooptimiza-tionquestionsinepidemiologyarisingfromproblemssuchastheeffectofclimatechangeonhealth.Wethendiscussgeneral resultsabouthowtoaveragescores toattainmeasuresallowingmeaningfulcomparisonsandclosewithadiscussionaboutmeasurementissuesarisinginthestudyofbehavioralresponsestohealthevents.

4.2 Scales of measurement

Itseemsclearthatmeasurementhassomethingtodowithnumbers.Inthischapter,itsufficestothinkofassigningrealnumberstoobjects.Ourapproachtoscalesofmeasurement isbasedon thenotion,goingback to thepsychologistS.S.Stevens(1946,1951,1959),thatthepropertiesofascalearecapturedbystudyingadmissible transformations,transformationsthatleadfromoneacceptablescaletoanother.Forexample,wetransformtemperaturemeasurementsfromcentigradeintoFahrenheitandmassmeasurementsfromkilogramsintopounds.Assumingascaleassignsarealnumberf(a)toeachobjectabeingmeasured,anadmissibletransformationofscalecanbethoughtofasafunctionϕthattakesf (a)into(ϕ∘f)(a).

OurapproachtoscalesofmeasurementisbasedonideasintroducedbyStevens.Assuming that a scale assigns a real number to each object being measured, wecall a scale a ratio scale if the admissible transformations are of the form ϕ(x)= αx, α > 0, an interval scale if the admissible transformations are of the formϕ(x) = αx + β, α > 0, an ordinal scale if the admissible transformations are the(strictly) monotone increasing transformations, and an absolute scale if the onlyadmissible transformation is the identity.Fordefinitionsofother scale types, seeRoberts(1979/2009).Thus,inthecaseofratioscales,thescalevalueisdetermineduptochoiceofaunit;inthecaseofintervalscales,itisdetermineduptochoicesofunitandofzeropoint;andinthecaseofordinalscales,itisdeterminedonlyuptoorder.Mass isanexampleofaratioscale.The transformationfromkilogramsintopounds,forexample,involvestheadmissibletransformationϕ(x)=2.2x.Length(inches, centimeters) and time intervals (years, seconds) are two other examplesofratioscales.Itissometimesarguedthatscalesdevelopedforloudness(“sones”)definearatioscale,butthisisnotuniversallyaccepted.Temperature(exceptwherethereisanabsolutezero)definesanintervalscale.Thus,transformationfromcen-tigrade intoFahrenheit involves theadmissible transformationϕ(x)= (9/5)x+32.

Y116134.indb 76 10/10/11 2:11 PM

Meaningful and meaningless statements in epidemiology and public health 77

Time on the calendar is another example of an interval scale (to say that this istheyear2011involvesnotonlyachoiceofunitbutachoiceofthezeroyear).Anexampleofanordinalscaleisanyscaleinwhichwegivegradesofmaterials,suchasleather,lumber,wool,andsoon.Expressedpreferencessometimesleadonlytoordinalscales,ifweknowourpreferencesonlyuptoorder.Subjectivejudgmentsinepidemiology,suchasofcoughorfatigue,oftendefineordinalscales.Countinggivesanexampleofanabsolutescale.

For many scales, the scale type can be determined by showing that the scalearisesfroma(numerical)representation.Intherestofthisparagraph,wesayawordortwoabouthowthisisdoneinthetheoryinsuchbooksasKrantzetal.(1971),Pfanzagl(1968),andRoberts(1979/2009),thoughthedetailsarenotneededforwhatfollows.Specifically,onestudiescertainobserved relationsonasetofobjectsofinterest,relationssuchas“aislongerthanb,”“aislouderthanb,”“Ithinkthevalueofbisbetweenthevalueofaandthevalueofc,”andsoon.Oneidentifiescorre-spondingnumerical relations,relationsonasetofrealnumbers,forinstance,the“greater than” relationor the“betweenness” relation.Thenone studiesmappingsthattakeeachobjectofinterestintoanumbersothatobjectsrelatedinacertainwayinanobservedrelationcorrespondtonumbersrelatedinthesamewayinthecor-respondingnumericalrelation.Forexample,oneseekstoassignnumberstoobjectssothataisjudgedlouderthanbifandonlyifthenumberassignedtoaisgreaterthanthenumberassignedtob.Suchamappingfromobjectstonumbersiscalledahomomorphismfromtheobservedrelationtothenumericalrelation.Inmeasure-ment theory, scales are identified with homomorphisms. Formally, an admissible transformationofascaleisthenatransformationofthenumbersassignedsothatonegetsanotherhomomorphism.Insomecases,onecanderiveacharacterizationoftheclassofadmissibletransformationsbyworkingfroma(numerical)representa-tion.Fordetailsonhowtoformalizetheseideas,seeRoberts(1979/2009).

It shouldbe remarked thatmany scalesbasedon subjective judgments cannotbederivedfroma(numerical)representation.Then,wemustusetheprinciplethattheadmissibletransformationsarethosethatpreservetheinformationcarriedbythescale.Knapp(1990)andThomas(1985)emphasizethedifficultiesinvolvedinidenti-fyingscaletype.AsStevens(1968)argues,itisoftenamatterofempiricaljudgmenttodeterminetheadmissibletransformationsandhencethescaletype.

4.3 Meaningful statements

Inmeasurementtheory,wespeakofastatementasbeingmeaningfulifitstruthorfalsityisnotanartifactoftheparticularscalevaluesused.ThefollowingdefinitionisduetoSuppes(1959)andSuppesandZinnes(1963).

Definition:A statement involving numerical scales is meaningful if its truth or falsity is unchanged after any (or all) of the scales is transformed (inde-pendently?) by an admissible transformation.

Aslightlymoreinformaldefinitionisthefollowing:

Y116134.indb 77 10/10/11 2:11 PM


Alternate Definition:A statement involving numerical scales is meaningful if its truth or falsity is unchanged after any (or all) of the scales is (indepen-dently?) replaced by another acceptable scale.

In some practical examples, for example, those involving preference judgmentsor judgmentsof“louder than”under the“semiorder”model (Roberts,1979/2009,1994),itispossibletohavescaleswhereonecannotgofromonetotheotherbyanadmissibletransformation,soonehastousethisalternatedefinition.

Hereweavoidthelongliteratureofmoresophisticatedapproachestomeaningful-ness.Situationswherethisrelativelysimple-mindeddefinitionmayrunintotroublearedisregarded.Emphasisisonapplicationsoftheinvariancemotivationbehindthetheoryofmeaningfulness.

Considerthefollowingstatement:

Statement S:“ThedurationofsymptomsinaninfluenzavictimnottreatedwithTamifluisthreetimesaslongasthedurationofsymptomsinaninflu-enzavictimwhoissotreated.”

Is this meaningful? We have a ratio scale (time intervals) and we consider thestatement:

f a f b( ) ( )= 3 . (4.1)

Thisismeaningfuliffisaratioscale.For,anadmissibletransformationisϕ(x)=αx,α>0.WewantEquation(4.1)toholdiff

( )( ) ( )( )φ φ f a f b= 3 . (4.2)

ButEquation(4.2)becomes

α αf a f b( ) ( )= 3 (4.3)

and(4.1)iff(4.3)becauseα>0.Thus,thestatementSismeaningful.Nextconsiderthestatement:

Statement T:“Thepatient’stemperatureat9AMtodayis2%higherthanitwasat9AMyesterday.”

Isthismeaningful?Thisisthestatement

f a f b( ) . ( )= 1 02 .

Thisismeaningless.ItcouldbetruewithFahrenheitandfalsewithcentigrade,orviceversa.

Y116134.indb 78 10/10/11 2:11 PM


Ingeneral,forratioscales,itismeaningfultocompareratios:

f a f b f c f d( ) ( ) ( ) ( )/ /> .

Forintervalscales,itismeaningfultocompareintervals:

f a f b f c f d( ) ( ) ( ) ( )− > − .

Forordinalscales,itismeaningfultocomparesize:

f a f b( ) ( )> .

Letusconsideranotherexample.Malariaparasitedensityismainlyobtainedbyreadingslidesundermicroscopes.Considerthestatement:

Statement M:“Theparasitedensityinthisslideisdoubletheparasitedensityinthatslide.”

Isthismeaningful?Densityismeasuredinnumberpermicroliter.So,ifoneslidehas100,000perμLandanother50,000perμL,isitmeaningfultoconcludethatthefirstslidehastwicethedensityofthesecond?Thisismeaningful.Volumeinvolvesratioscalesandcountsareabsolutescales.However,thisdisregardserrorsinmeasure-ment.Astatementcanbemeaningfulinthemeasurementtheorysensebutmeaning-lessinapracticalsense.

Hereisstillanotherexample:

Statement W: “Thesecond tumorweighs20million timesasmuchas thefirstone.”

Thisismeaningful.Itinvolvesratioscales.Itissurelyfalsenomatterwhattheunit.Note thatmeaningfulness isdifferent from truth. Ithas todowithwhatkindsofassertionsitmakessensetomake,whichassertionsarenotaccidentsoftheparticu-larchoiceofscale(units,zeropoints)inuse.

4.4 Averaging judgments of cough severity

Supposewestudytwogroupsofpatientswith tuberculosis.Let f (a)be thecoughseverityofaas judgedononeof thesubjectivecoughseverityscales inuse(e.g.,rateseverityas1to5).Supposethatdatasuggestthattheaveragecoughseverityforpatientsinthefirstgroupishigherthantheaveragecoughseverityofpatientsinthesecondgroup.Isthismeaningful?

Leta1,a2,…,anbepatientsinthefirstgroupandb1,b2,…,bmbepatientsinthesecondgroup.Notethatmcouldbedifferentfromn.Thenweare(probably)assert-ingthat

Y116134.indb 79 10/10/11 2:11 PM


1 1

11n

f am

f bi i

i

m

i

n

( ) ( )>==

∑∑ . (4.4)

Wearecomparingarithmeticmeans.Thestatement(4.4)ismeaningfulifandonlyifunderadmissibletransformationϕ,(4.4)holdsifandonlyif

1 1

11n

f am

f bi i

i

m

i

n

( )( ) ( )( )φ φ >==

∑∑ (4.5)

holds.Ifcoughseveritydefinesaratioscale,then(4.5)isthesameas

1 1

11n

f am

f bi i

i

m

i

n

α α( ) ( )>==

∑∑ , (4.6)

for some positive α. Certainly (4.4) holds if and only if (4.6) does, so (4.4) ismeaningful.

Notethat(4.4)isstillmeaningfuliffisanintervalscale.Forinstance,wecouldbecomparingtemperatures.Itismeaningfultoassertthattheaveragetemperatureofthefirstgroupishigherthantheaveragetemperatureofthesecondgroup.Toseewhy,notethat(4.4)isequivalentto

1 1

1 1n

f am

f bi

i

n

i

i

m

[ ( ) ] [ ( ) ]α β α β+ > += =

∑ ∑ ,

whereα>0.However, (4.4) iseasilyseen tobemeaningless if f is justanordinalscale.To

showthatcomparisonofarithmeticmeanscanbemeaninglessforordinalscales,notethatweareaskingexpertsforasubjectivejudgmentofcoughseverity.Itseemsthatf (a)ismeasuredonanordinalscale,forexample,5-pointscale:5=extremelysevere,4=verysevere,3=severe,2=slightlysevere,and1=nocough.Insuchascale,thenumbersmaynotmeananything;onlytheirordermatters.Supposethatgroup1hasthreememberswithscoresof5,3,and1,foranaverageof3,andgroup2hasthreememberswithscoresof4,4,and2foranaverageof3.33.Thentheaver-agescoreingroup2ishigherthantheaveragescoreingroup1.Ontheotherhand,supposeweconsidertheadmissibletransformationϕdefinedbyϕ(5)=100,ϕ(4)=75,ϕ(3)=65,ϕ(2)=40,ϕ(1)=30.Thenaftertransformation,membersofgroup1havescoresof100,65,30,withanaverageof65,andthoseingroup2havescoresof75,75,40,withanaverageof63.33.Now,group1hasahigheraveragescore.Whichgrouphadahigheraveragescore?Theanswerclearlydependsonwhichversionofthescaleisused.Ofcourse,onecanargueagainstthiskindofexample.AsSuppes(1979)remarksinthecaseofasimilarexamplehavingtodowithgradingapplesin

Y116134.indb 80 10/10/11 2:11 PM


fourorderedcategories,“Surelythereissomethingquiteunnaturalaboutthistrans-formation”ϕ.Hesuggeststhat“thereisastrongnaturaltendencytotreattheorderedcategoriesasbeingequallyspaced.”However,ifwerequirethis,thenthescaleisnotanordinalscaleaccordingtoourdefinition.Noteverystrictlymonotoneincreasingtransformationisadmissible.Moreover,thereisnoreason,giventhenatureofthecategories,tofeelthatequalspacingisdemandedinourexample.Inanycase,theargumentisnotwiththepreceptthatwehavestated,butwiththequestionofwhetherthefive-pointscalewehavegivenisindeedanordinalscaleaswehavedefinedit.Tocompletethisexample,letussimplyremarkthatcomparisonofmediansratherthanarithmeticmeansismeaningfulwithordinalscales:thestatementthatonegrouphasahighermedianthananothergroupispreservedunderadmissibletransformation.

Similarconsiderationsapplytomeasuringaveragefatigue.Fatigueisanimportantvariableinmeasuringtheprogressofpatientswithseriousdiseases.OnescalewidelyusedinmeasuringfatigueisthePiperFatigueScale.Itasksquestionssuchas:onascaleof1to10,towhatdegreeisthefatigueyouarefeelingnowinterferingwithyourabilitytocompleteyourworkorschoolactivities(1=none,10=agreatdeal)?Onascaleof1to10,howwouldyoudescribethedegreeofintensityorseverityofthefatiguewhichyouareexperiencingnow(1=mild,10=severe)?Asimilaranalysisapplies:itismeaninglesstocomparearithmeticmeans,andmeaningfultocomparemedians.

Letusreturntocoughseverity,butnowsupposethateachofnobserversisaskedtorateeachofacollectionofpatientsastotheirrelativecoughseverity.Alternatively,supposeweratepatientsondifferentcriteriaoragainstdifferentbenchmarks.(Asim-ilaranalysisapplieswithperformanceratings,importanceratings,etc.)Letfi(a)betheratingofpatientabyjudgei(orundercriterioni).Isitmeaningfultoassertthattheaverageratingofpatientaishigherthantheaverageratingofpatientb?Asimilarquestionarisesinfatigueratings,ratingsofbrightnessofrash,andsoon.Wearenowconsideringthestatement

1 1

1 1n

f an

f bi

i

n

i

i

n

( ) ( )= =

∑ ∑> . (4.7)

Noteincontrasttostatement(4.4)thatwehavethesamenumberoftermsineachsumandthatthesubscriptisnowonthescalevaluefratherthanonthealternativeaorb.Ifeachfiisaratioscale,wethenaskwhether(4.7)isequivalentto

1 1

1 1n

f an

f bi

i

n

i

i

n

α α( ) ( )= =

∑ ∑> ,

α>0.Thisisclearlythecase.However,wehaveperhapsgonetooquickly.Whatiff1,f2,…,fnhaveindependent

units?Inthiscase,wewanttoallowindependentadmissibletransformationsofthefi.Thus,wemustconsider

Y116134.indb 81 10/10/11 2:11 PM


1 1

1 1n

f an

f bi i

i

n

i i

i

n

α α( ) ( )= =

∑ ∑> , (4.8)

allαi>0. It iseasy tofindα′is forwhich(4.7)holdsbut(4.8)fails.Thus, (4.7) ismeaningless.Doesitmakesensetoconsiderdifferentαis?Itcertainlydoesinsomecontexts.Considerthecasewherethealternativesareanimalsandoneexpertmea-surestheirimprovedhealthintermsoftheirweightgainandasecondmeasuresitintermsoftheirheightgain.

Theconclusionisthatweneedtobecarefulwhencomparingarithmeticmeanratings,evenwhenweareusingratioscales.NormanDalkey(personalcommunica-tion)wasthefirstpersontopointouttotheauthorthat,inmanycases,itissafertousegeometricmeans,aconclusionwhichbynowis“folklore.”

For,considerthecomparison

f a f bi

i

n

n i

i

n

n( ) ( )= =

∏ ∏>1 1

. (4.9)

Ifallαi>0,then(4.9)holdsifandonlyif

α αi i

i

n

n i i

i

n

nf a f b( ) ( )= =

∏ ∏>1 1

.

Thus,ifeachfiisaratioscale,thenevenifexpertschangetheunitsoftheirratingscalesindependently,thecomparisonofgeometricmeansismeaningfuleventhoughthe comparison of arithmetic means is not. An example of an application of thisobservationistheuseofthegeometricmeanbyRoberts(1972,1973).Theproblemaroseinastudyofairpollutionandenergyuseincommutertransportation.(Healtheffectsofairpollutionarediscussedinthenextsection.)Apreliminarystepinthemodelbuildinginvolvedthechoiceof themost importantvariablestoconsiderinthemodel.Eachmemberofapanelofexpertsestimatedtherelativeimportanceofvariablesusingaprocedurecalledmagnitudeestimation.(Here,themostimportantvariableisgivenascoreof100,avariablejudgedhalfasimportantisgivenascoreof50,andsoon.)Thereisastrongbodyofopinionthatmagnitudeestimationleadsto a ratio scale,muchof it goingback toStevens. (See thediscussion inRoberts(1979/2009,pp.179–180).)Howthenshouldwechoosethemostimportantvariables?Bythediscussionabove,itis“safer”tocombinetheexperts’importanceratingsbyusinggeometricmeansandthentochoosethemostimportantvariablesasthosehav-ingthehighestgeometricmeanrelativeimportanceratings,thanitistodothisbyusingarithmeticmeans.ThatiswhyRoberts(1972,1973)usedgeometricmeans.

Y116134.indb 82 10/10/11 2:11 PM


4.5 Measurement of air pollution

There isaclose relationshipbetweenpollutionandhealth.Variouspollutantsarepresent in theair.Somearecarbonmonoxide(CO),hydrocarbons (HC),nitrogenoxides (NOX), sulfuroxides (SOX),andparticulatematter (PM).Alsodamagingareproductsof chemical reactions amongpollutants.For example,oxidants suchasozoneareproducedbyHCandNOXreactinginthepresenceofsunlight.Somepollutantsaremoreseriousinthepresenceofothers:forexample,SOXaremoreharmfulinthepresenceofPM.Intheearlydaysofairpollutionscience,therewasanattempttofindawaytomeasurepollutionwithoneoverallmeasure.Tocomparepollution controlpolicies,weneed to compare effectsofdifferentpollutants.Wemightallowanincreaseofsomepollutantstoachievedecreaseofothers.Onesinglemeasurecouldgiveanindicationofhowbadthepollutionlevelisandmighthelpusdetermineifwehavemadeprogress.Asimpleapproachistocombinetheweightofpollutants.Letusmeasurethetotalweightofemissionsofpollutantioverafixedperiodoftimeandsumoveri.Lete(i, t, k)bethetotalweightofemissionsofpollut-anti(percubicmeter)overthetthtimeperiodandduetothekthsourceormeasuredinthekthlocation.Thenourpollutionindexissimply

A t k e i t ki

n

( , ) ( , , )==

∑1

,

iftherearenpollutantsunderconsideration.Usingthismeasure,Walther(1972)reachedsuchconclusionsas:(i) the largest

source of air pollution is transportation and the second largest is stationary fuelcombustion(especiallybyelectricpowerplants);(ii)transportationaccountsforover50%ofallairpollution;(iii)COaccountsforover50%ofallemittedairpollution.Statement(i)isjust

A t k A t k( , ) ( , )> ′ .

Statement(ii)isjustthestatementthat

A t k A t kr

k kr

( , ) ( , )>≠∑ .

Statement(iii)isjustthestatementthat

e i t k e j t kt k j it k

( , , ) ( , , ), ,

∑ ∑∑>≠

.

Alltheseconclusionsaremeaningfulifwemeasurealle(i, t, k)inthesameunitsofmass(e.g.,milligramspercubicmeter)andsoadmissibletransformationmeansmultiplye(i, t, k)bythesameconstant.

Y116134.indb 83 10/10/11 2:11 PM


However,althoughthesestatementsaremeaningfulinthetechnicalsense,wehavetoaskiftheyaremeaningfulcomparisonsofpollutionlevelinapracticalsense.AunitofmassofCOisfarlessharmfulthanaunitofmassofNOX.U.S.EnvironmentalProtectionAgencystandardsbasedonhealtheffectsfora24-hourperiodallowmanymoreunitsofCOthanunitsofNOXbecauseaunitofmassofCOisfarlessharmfulthanaunitofmassofNOX.Thus,wemightwishtouseaweightingfactorλithatmeasurestheeffectoftheithpollutantandthenuseaweightedsum

λi

i

n

e i t k( , , )=

∑1

. (4.10)

Suchaweightedsum,sometimesknownaspindex,hasbeenusedasacombinedpollutionindex.InearlyusesofthismeasureintheSanFranciscoBayArea(BayAreaPollutionControlDistrict,1968;SauterandChilton,1970;andelsewhere),λiisthereciprocaloftheamountτ(i)ofemissionsofpollutantiinagivenperiodoftimeneededtoreachacertaindanger level,otherwisecalledthe tolerance factor.Thereciprocaliscalledtheseverity factor.Usingthisversionofpindex,Walther(1972)arguesthattransportationisstillthelargestsourceofpollution,butnowaccountingforlessthan50%.Stationarysourcesfalltofourthplace.COdropstothebottomofthelistofpollutants,accountingforjustover2%ofthetotal.Again,thesecon-clusionsaremeaningfulifweusethesameunitsofmassineachcase.Withtheseweightingfactorsλi=1/τ(i),althoughcomparisonsusingpindexaremeaningfulinourtechnicalsense,theindexdoesnotseemtogivemeaningfulnumbersinanyrealsense,becausereaching100%ofthedangerlevelinonepollutantwouldgivethesamepindexvalueasreaching20%ofthedangerleveloneachoffivepollutants.Inconclusion,weshouldstressagainthatthereisadistinctionbetweenmeaningfulnessinthetechnicalsenseandmeaningfulnessinothersenses.

Theseverity tonnageofpollutantiduetoagivensourceisactualtonnagetimestheseverityfactor1/τ(i).Inearlyairpollutionmeasurementliterature,severityton-nagewasconsideredameasureofhowseverepollutionduetoasourcewas.Datafrom Walther (1972) suggest the following. It is an interesting exercise to decidewhichoftheseconclusionsaremeaningfulineitherthetechnicalsenseortheprac-ticalsense:(i)HCemissionsaremoresevere(havegreaterseveritytonnage)thanNOXemissions; (ii)effectsofHCemissions fromtransportationaremoreseverethan those of HC emissions from industry (and the same for NOX); (iii) effectsofHCemissionsfromtransportationaremoreseverethanthoseofCOemissionsfromindustry; (iv)effectsofHCemissions fromtransportationaremore than20timesassevereaseffectsofCOemissionsfromtransportation;(v)thetotaleffectofHCemissionsduetoallsourcesismorethan8timesassevereasthetotaleffectofNOXemissionsduetoallsources.

4.6 Evaluation of alternative HIV treatments

Howdoweevaluatealternativepossibletreatmentplansorinterventionsforagivendisease? One common procedure is the following. A number of treatments are

Y116134.indb 84 10/10/11 2:11 PM


comparedondifferent criteriaor benchmarks.Their scoreson each criterion arenormalizedrelativetothescoreofoneofthetreatments.Thenormalizedscoresofatreatmentarecombinedbysomeaveragingprocedureandaveragescoresarecom-pared.Iftheaveragingisthearithmeticmean,thenthestatement,“Onetreatmenthasahigherarithmeticmeannormalizedscorethananothertreatment,”ismeaning-less:The treatment towhich scores arenormalizedcandeterminewhichhas thehigherarithmeticmean.Similarmethodsareused incomparingperformancesofalternativecomputersystemsorothertypesofmachinery.

Toillustrate,consideranumberoftreatments/interventionsinthecaseofHIV:universalscreening,freecondomdistribution,abstinenceeducation,malecircumci-sion,andthelike.Consideranumberofcriteria/outcomes:CD4count(ameasureofhowwellthebodyisfightingoffHIV),dayswithoutpoorappetite,dayswithoutprofoundfatigue,numberofdayshospitalized,andsoon.

Table 4.1showsthreetreatmentsI,II,III,andfivecriteriaA,B,C,D,E,withthei,jentrygivingthescoreoftheithtreatmentonthejthcriterion.Table 4.2showsthescoreofeachtreatmentnormalizedrelativetotreatmentI,thatis,bydividingbytreatmentI’sscore.Thus,forexample,the1,2entryis83/83=1,andthe2,2entryis70/83=.84.ThearithmeticmeansofthenormalizedscoresineachrowarealsoshowninTable 4.2.WeconcludethattreatmentIIIisbest.

However,letusnownormalizerelativetotreatmentII,obtainingthenormalizedscoresofTable 4.3.Basedonthearithmeticmeannormalizedscoresofeachrowshown inTable 4.3,wenowconclude that treatment I is best.So, the conclusion

Table 4.1 Scoreoftreatmentioncriterionj

Treatment/criterion A B C D E

I 417 83 66 39,449 772II 244 70 153 33,527 368III 134 70 135 66,000 369

Table 4.2 NormalizingrelativetotreatmentI

Treatment/criterion A B C D EArithmetic

meanGeometric

mean

I 1.00 1.00 1.00 1.00 1.00 1.00 1.00II 0.59 0.84 2.32 0.85 0.48 1.01 0.86III 0.32 0.85 2.05 1.67 0.45 1.07 0.84

Table 4.3 NormalizingrelativetotreatmentII

Treatment/criterion A B C D EArithmetic

meanGeometric

mean

I 1.71 1.19 0.43 1.18 2.10 1.32 1.17II 1.00 1.00 1.00 1.00 1.00 1.00 1.00III 0.55 1.00 1.88 1.97 1.08 1.07 0.99

Y116134.indb 85 10/10/11 2:11 PM


thatagiventreatmentisbestbytakingthearithmeticmeanofnormalizedscoresismeaninglessinthiscase.

ThenumbersinthisexamplearetakenfromFlemingandWallace(1986),withdatafromHeath(1984),andrepresentactualscoresofalternative“treatments”inacomputingmachineapplication.

Sometimes,geometricmeanishelpful.ThegeometricmeannormalizedscoresofeachrowareshowninTables 4.2and4.3.Notethatineachcase,weconcludethattreatmentIisbest.Inthissituation,itiseasytoshowthattheconclusionthatagiventreatmenthasthehighestgeometricmeannormalizedscoreisameaningfulconclusion.Itisevenmeaningfultoassertsomethingsuchas:agiventreatmenthasageometricmeannormalizedscore20%higherthananothertreatment.

FlemingandWallacegivegeneralconditionsunderwhichcomparinggeometricmeans of normalized scores is meaningful. It is a research area in measurementtheory,withalonghistoryandlargeliterature,todeterminewhataveragingproce-duresmakesenseinwhatsituations.Wereturntothistopic,andinparticulartheFleming–Wallaceconditions,inSection4.9.

4.7 Meaningfulness of conclusions from statistical tests

Biostatisticsisakeycomponentofepidemiologicalresearch.However,biostatisti-ciansknowverylittleaboutmeasurementtheory.Mosthaveneverheardaboutthetheoryofmeaningfulnessorlimitationsthatmeaningfulnessplacesonconclusionsfromstatisticaltests.Forover50years,therehasbeenconsiderabledisagreementonthelimitationsthatscalesofmeasurementimposeonstatisticalprocedureswemayapply.ThecontroversystemsfromfoundationalworkofStevens(1946,1951,1959,andelsewhere),whodevelopedtheclassificationofscalesofmeasurementwehavedescribedhere.Stevensprovidedrulesfortheuseofstatisticalprocedures,concludingthatcertainstatisticsareinappropriateatcertainlevelsofmeasurement.Theappli-cationofStevens’ideastodescriptivestatisticshasbeenwidelyaccepted.However,theirapplicationtoinferentialstatisticshasbeenlabeledbysomeamisconception.

Toexploretheseideas,supposeP isapopulationwhosedistributionwewouldliketodescribe.WecapturepropertiesofPbyfindingadescriptivestatisticforPortakingasampleSfromPandfindingadescriptivestatisticforS.Ourexamplessug-gestthatcertaindescriptivestatisticsareappropriateonlyforcertainmeasurementsituations.This idea,originallydue toStevens,waspopularizedbySiegel (1956)inhiswell-knownbookNonparametric Statistics for the Behavioral Sciences.Ourexamplessuggesttheprinciplethatarithmeticmeansare“appropriate”statisticsforintervalscales,andmediansforordinalscales.Ontheothersideofthecoin,itisarguedthatitisalwaysappropriatetocalculatemeans,medians,andotherdescrip-tivestatistics,nomatterwhatthescaleofmeasurement.Thewell-knownstatisticianFredericLordmadethisargumentwithafamousexampleofagroupoffirst-yearcollege football players who were upset that the average (arithmetic mean) num-berontheiruniformswaslessthanthatofthemoreadvancedplayers.Hearguedthatitismeaningfulforthefirst-yearplayerstoaverageuniformnumbersbecause

Y116134.indb 86 10/10/11 2:11 PM


“thenumbersdon’trememberwheretheycamefrom.”Marcus-RobertsandRoberts(1987)agree.Itisalwaysappropriatetocalculatemeans,medians,…But,theyask:isitappropriatetomakecertainstatementsusingthesedescriptivestatistics?

The rest of this section summarizes the conclusions of Marcus-Roberts andRoberts.Theyarguethatitisusuallyappropriatetomakeastatementusingdescrip-tivestatisticsifandonlyifthestatementismeaningful.Astatementthatistruebutmeaninglessgivesinformationthatisanaccidentofthescaleofmeasurementused,not information that describes the population in some fundamental way. So, it isappropriatetocalculatethearithmeticmeanofordinaldata.Itisjustnotappropriatetosaythatthemeanofonegroupishigherthanthemeanofanothergroup.

Stevens’ideashavecometobeappliedtoinferentialstatistics,inferencesaboutanunknownpopulationP.Theyhaveledtosuchprinciplesasthefollowing:

• Classicalparametrictests(e.g.,t-test,Pearsoncorrelation,analysisofvari-ance)areinappropriateforordinaldata.Theyshouldbeappliedonlytodatathatdefineanintervalorratioscale.

• Forordinalscales,nonparametrictests(e.g.,Mann–WhitneyU,Kruskal–Wallis,Kendall’stau)canbeused.

Noteveryoneagrees.Thus,therehasbeencontroversy.Marcus-RobertsandRobertsarguethatthevalidityofastatisticaltestdepends

onastatisticalmodel.Thisincludesinformationaboutthedistributionofthepopula-tionandaboutthesamplingprocedure.Thevalidityofthetestdoesnotdependonameasurementmodel.Thatisconcernedwiththeadmissibletransformationsandscaletype.Thescaletypeentersindecidingwhetherthehypothesisisworthtestingatall,whetheritisameaningfulhypothesis.Theissueis:ifweperformadmissibletransformationsofscale,isthetruthorfalsityofthehypothesisunchanged?

Asanexample,supposewehavedataonanordinalscaleandweconsider thehypothesisthatthemeanis0.Thisisameaninglesshypothesis.Canwetestmean-inglesshypotheses?Marcus-RobertsandRobertssay,“Yes.”Buttheyquestionwhatinformationwegetoutsideofinformationaboutthepopulationasmeasured.Tobemorepreciseabout this, considerhowwe testhypothesisH0 aboutP.Wedo thisthroughthefollowingsteps:

• DrawarandomsampleSfromP.• CalculateateststatisticbasedonS.• Calculatetheprobabilitythattheteststatisticiswhatwasobservedgiven

H0istrue.• AcceptorrejectH0onthebasisofthetest.

Calculationofprobabilitydependsonastatisticalmodel,whichincludesinformationaboutthedistributionofPandaboutthesamplingprocedure.But,validityofthetestdependsonlyonthestatisticalmodel,notonthemeasurementmodel.Thus,youcanapplyparametricteststoordinaldata,providedthestatisticalmodelissatisfied.The

Y116134.indb 87 10/10/11 2:11 PM


modelissatisfiedifthedataarenormallydistributed.Wheredoesthescaletypeenter?Itentersindeterminingifthehypothesisisworthtestingatall,thatis,ifitismeaningful.

Forinstance,considerdataonanordinalscaleandletH0bethehypothesisthatthemeanis0.Thehypothesisismeaningless.But,ifthedatameetcertaindistribu-tionalrequirementssuchasnormality,wecanapplyaparametrictest,suchasthet-test,tocheckifthemeanis0.

Similaranalysescanbedevelopedforotherkindsofstatisticaltestsfordataonothertypesofscales.

4.8 Optimization problems in epidemiology

Theimpactofclimatechangeincludespotentialeffectsonthehealthofhumans,ani-mals,plants,andecosystems.Someearlywarningsignsofclimatechangeincludemajorheateventssuchas the1995extremeheatevent inChicago that led to514heat-relateddeathsand3,300excessemergencyroomadmissionsandthe2003heatwaveinEuropethatledto35,000deaths.Withanticipatedchangeinclimatecouldcomeanincreaseinthenumberandseverityofextremeevents,includingmoreheatwaves,floods,hurricanes,andthelike.Oneresponsetoextremeheateventsistheevacuationof themostvulnerableindividualstoclimate-controlledenvironments.Here,therearemodelingchallenges,suchas:wheretolocatetheevacuationcenters,whomtosendwhere,findingwaystominimizetraveltimesfromhometoevacuationcenter,andsoon.Amongtheproblemsarisinghereistheshortestrouteproblem:findtheshortestroutefromhometoevacuationcenter.Thisisanexampleofanopti-mizationproblem,morespecificallyacombinatorialoptimizationproblem.Wenowconsiderthemeaningfulnessofconclusionsaboutoptimalityinsuchproblems.

Consider a network with vertices and edges and numbers on the edges repre-senting some sortof strengthor levelorweightor lengthof the connection.Theproblemistofindtheshortestpathinthenetworkfromvertexxtovertexz,wherethestrengthofapathisthesumoftheweightsofedgesinit.Thisproblemoccurswidelyinpractice.IntheUnitedStates,justoneagencyoftheU.S.DepartmentofTransportationinthefederalgovernmentappliesalgorithmstosolvethisproblemliterallybillionsoftimesayear(Goldman,1981).Considerasimplenetworkwithverticesx,y,andzandedgesfromxtoywithstrength2,ytozwithstrength4,andxtozwithstrength15.Whatistheshortestpathfromxtozinthisnetwork?Theshortestpathisthepaththatgoesfromxtoytoz,withatotal“length”of6.Thealternativepaththatgoesdirectlyfromxtozhastotal“length”15.Istheconclusionthatxtoytozistheshortestpathameaningfulconclusion?

Theconclusionismeaningfulifthestrengthsdefinearatioscale,astheydoifthey are distances or times as in the evacuation problem. However, what if theydefinean interval scale?Consider theadmissible transformationϕ(x)=3x+100.Nowtheweightschangeto106ontheedgefromxtoy,112ontheedgefromytoz,and145ontheedgefromxtoz.Weconcludethatgoingdirectlyfromxtozistheshortestpath.Theoriginalconclusionwasmeaningless.

Theshortestpathproblemcanbeformulatedasalinearprogrammingproblem.Thus,theconclusionthatAisthesolutiontoalinearprogrammingproblemcanbe

Y116134.indb 88 10/10/11 2:11 PM


meaninglessifcostparametersaremeasuredonanintervalscale.Notethatlinearprogrammingiswidelyusedinpublichealthaswellasinotherareasofapplication.Forexample,itisusedtodetermineoptimalinventoriesofmedicines,assignmentsofpatientsordoctorstoclinics,optimizationofatreatmentfacility,andamounttoinvestinpreventivetreatments,amongotherapplications.

Anotherveryimportantpracticalcombinatorialoptimizationproblemisthemin-imumspanning treeproblem.Given a connectedweightedgraphor network,weaskforthespanningtreewithtotalsumofstrengthsorweightsassmallaspossible.(Aspanning treeisatreethatincludesalltheverticesofthenetwork.)Thisproblemhasapplicationsintheplanningoflarge-scaletransportation,communication,anddistributionnetworks.Forexample,givenanetwork,weseektofindusableroadsthatallowonetogofromanyvertextoanyothervertex,minimizingthelengthsoftheroadsused.Thisproblemarisesinthecaseofextremeeventsthatleavesomeroadsfloodedandwhenwerequireroutesthatemergencyvehiclescantake.Again,itisnaturaltoaskiftheconclusionthatagivensetofedgesdefinesaminimumspanningtreeismeaningful.Itissurprisingtoobservethateveniftheweightsontheedgesdefineonlyanordinalscale,thentheconclusionismeaningful.Thisisnotaprioriobvious.However,itfollowsfromthefactthatthewell-knownalgorithmknownasKruskal’salgorithmorthegreedyalgorithmgivesasolution.InKruskal’salgorithm(Kruskal, 1956; Papadimitriou and Steiglitz, 1982), we order edges in increasingorderofweightandthenexamineedgesinthisorder,includinganedgeifitdoesnotformacyclewithedgespreviouslyincluded.Westopwhenallverticesareincluded.Anyadmissibletransformationwillnotchangetheorderinwhichedgesareexam-inedinthisalgorithm;therefore,thesamesolutionwillbeproduced.

Manypracticaldecision-makingproblemsinpublichealthandotherfieldsinvolvethesearchforanoptimalsolutionasintheshortestpathandminimumspanningtreeproblems.Little attention ispaid to thepossibility that the conclusion that apar-ticularsolutionisoptimalmaybeanaccidentofthewaythatthingsaremeasured.Forthebeginningsofthetheoryofmeaningfulnessofconclusionsincombinatorialoptimization,seeMahadev,Pekeč,andRoberts(1998),Pekeč(1996a,1996b),andRoberts(1990,1994,1999).

4.9 How should we average scores?

Wehaveseenthatinsomesituationscomparingarithmeticmeansisnotagoodideaandcomparinggeometricmeansis.Therearesituationswherethereverseistrue.Canwelaydownsomeguidelinesastowhentousewhataveragingprocedure?Abriefdiscussionfollows.

Let a1, a2, …, an be n “scores” or ratings, for example, scores on criteria forevaluatingtreatments.Letu=F(a1,a2,…,an).Fisanunknownaveragingfunction,sometimescalledamerging function,anduistheaverageormergedscore.

FlemingandWallace(1986)takeanaxiomaticapproachtodeterminingappro-priatemergingfunctions.TheytakethecasewherethedomainandrangeofFarethepositiverealnumbersandconsiderthefollowingaxioms:

Y116134.indb 89 10/10/11 2:11 PM


• Reflexivity: F(a,a,…,a)=a.• Symmetry: F(a1,a2,…,an)=F(aπ(1),aπ(2),…,aπ(n))forallpermutationsπof

1,2,…,n.• Multiplicativity: F(a1b1,a2b2,…,anbn)=F(a1,a2,…,an)F(b1,b2,…,bn).

TheyshowthatifFsatisfiesthesethreeaxioms,thenFisthegeometricmean,and conversely. It is fairly simple to understand the first two axioms. Reflexivitysaysthatifallratingsarethesame,thentheiraverageisthesame.Symmetrysaysthat theaverage is independentof thenamesorordergiven to thecriteria(whichmightnotbe true insomeapplications).Themultiplicativepropertysays that theaverageoftheproductsoftheratingsisthesameastheproductoftheaveragesoftheratings.FlemingandWallacemotivatethisaxiombysayingthatifaimeasurestherelativestrengthoftreatmentItotreatmentIIoncriterioni,andbitherelativestrengthoftreatmentIItotreatmentIIIoncriterioni,thenaibimeasurestherela-tivestrengthoftreatmentItotreatmentIIIoncriterioni.If“strength”isspeed,asintheFleming–Wallaceapplications,thenthereissomejustificationforthisconclusionand,moreover,totheconclusionthattheaverageovercriteriaoftherelativestrengthoftreatmentItotreatmentIIIistheproductoftheaverageoftherelativestrengthoverallcriteriaoftreatmentI totreatmentIItimestheaverageoftherelativestrengthoverallcriteriaoftreatmentIItotreatmentIII.However,moregenerally,itishardertobesurethatMultiplicativityisadesiredpropertyofanaveragingprocedure.

Analternativeapproachusesfunctionalequationsandisbasedoneitherassump-tionsaboutscaletypeofsomeofthescalesoraboutmeaningfulnessofsomestate-mentsusingthescales.Consideranunknownfunctionu=F(a1,a2,…,an).WeuseanideaduetoLuce(1959)thatheoncecalledtheprinciple of theory construction:Ifyouknowthescaletypesoftheaiandthescaletypeofuandyouassumethatanadmissibletransformationofeachoftheaileadstoanadmissibletransformationofu,youcanderivetheformofF.(Wedisregardsomeoftherestrictionsonapplicabil-ityofthisprinciple,includingthosegivenbyLuce[1962,1964,1990].)

Toillustratetheideas,letustakeasimplecasewheren=1,a=a1isaratioscale,anduisaratioscale.Anadmissibletransformationofscaleinbothcasesismultiplicationbyapositiveconstant.Bytheprincipleoftheoryconstruction,multiplyingtheinde-pendentvariableabyapositiveconstantαleadstomultiplyingthedependentvariablebyapositiveconstantAthatdependsonα.Thisleadstothefunctionalequation

F A F a Aa( ) ( ) ( ), ( )α α α= > 0 . (4.11)

Bysolvingthisequation,Luce(1959)provesthatiftheaveragingfunctionFiscon-tinuous,atakesonallpositiverealvalues,andFtakesonpositiverealvalues,then

F a cak( ) = .

Thus,iftheindependentanddependentvariablesareratioscales,theonlypossiblewaytorelatethemisbyapowerlaw.

Y116134.indb 90 10/10/11 2:11 PM


Thisresultisverygeneral.InearlywritingsofLuce,itwasinterpretedasverystrictlylimitingthe“possiblescientificlaws”inalldisciplines.Forexample,otherexamplesofpowerlawsaregivenasfollows.Oneis

V r= ( )4 3 3/ π ,

whereVisvolumeandrisradius(bothratioscales).AsecondisNewton’slawofgravitation:

F G mm r= ∗( )/ 2 ,

where F is the force of attraction, G is a gravitational constant, m, m* are fixedmassesofbodiesbeingattracted,andr is thedistancebetween them(everythingbeingaratioscale).AthirdisOhm’slaw:underfixedresistance,voltageispropor-tionaltocurrent(voltageandcurrentbeingratioscales).

Toillustratetheideaswhenthenumberofindependentvariables(ratingsbeingaveraged)islargerthan1,supposethata1,a2,…,anareindependentratioscalesanduisaratioscale.LetFbeamergingfunctiondefinedonalln-tuplesofpositiverealnumbersandoutputtingapositivereal.Bytheprincipleoftheoryconstruction,

F a a a u F a a a un n n( , , , ) ( , , , )1 2 1 1 2 2… …= ↔ =α α α α ,

whereα1>0,α2>0,…,αn>0,α>0,andαdependsonα1,α2,…,αn.Thuswegetthefunctionalequation:

F a a a A F a an n n( , , , ) ( , , , ) ( , , ,α α α α α α1 1 2 2 1 2 1 2… … …= aa An n), ( , , , )α α α1 2 0… > . (4.12)

Luce(1964)showsthatifFiscontinuousandsatisfiesEquation(4.12),then

F a a a a a anc c

ncn( , , , )1 2 1 2

1 2… …= λ (4.13)

forconstantsλ>0,c1,c2,…,cn.AczélandRoberts(1989)showthatif,inaddition,Fsatisfiesreflexivityandsymmetry,thenλ=1andc1=c2=⋯=cn=1/n,soFisthegeometricmean.

TherearealsosituationswhereonecanshowthatthemergingfunctionFisthearithmeticmean.Consider, forexample, thecasewherea1,a2,…,anare intervalscaleswith the sameunit and independentzeropointsandu is an interval scale.Thentheprincipleoftheoryconstructiongivesthefunctionalequation:

F a a a

A

n n

n

( , , , )

( , , , , )

α β α β α β

α β β β

1 1 2 2

1 2

+ + + =…

… FF a a a Bn n( , , , ) ( , , , , )1 2 1 2… …+ α β β β ,

Y116134.indb 91 10/10/11 2:11 PM


where

A n( , , , , )α β β β1 2 0… > .

Evenwithout a continuity assumption,Aczél,Roberts, andRosenbaum (1986)showthatinthiscase,

F a a a a bn i i

i

n

( , , , )1 2

1

… = +=

∑λ ,

whereλ1,λ2,…,λn,barearbitraryconstants.AczélandRoberts(1989)showthatif,inaddition,Fsatisfiesreflexivity,then

λi

i

n

b=

∑ = =1

1 0, .

IfinadditionFsatisfiesreflexivityandsymmetry,thentheyshowthatλi=1/nforalli,andb=0;thatis,Fisthearithmeticmean.

Stillanotherapproachtodeterminingtheformofanappropriatemergingfunc-tionistoreplacescaletypeassumptionswithassumptionsthatcertainstatementsusingscalesaremeaningful.Althoughitisoftenreasonabletoassumethatyouknowthescaletypeoftheindependentvariablesa1,a2,…,an,itisoftennotsoreasonabletoassumethatyouknowthescaletypeofthedependentvariableu.However,itturnsoutthatonecanreplacetheassumptionaboutthescaletypeofuwithanassumptionthatacertainstatementinvolvinguismeaningful.Toreturntothecasewheretheaiareindependentratioscales,insteadofassumingthatuisaratioscale,letusassumethatthestatement

F a a a kF b b bn n( , , , ) ( , , )1 2 1 2… …=

ismeaningfulforalla1,a2,…,an,b1,b2,…,bnandk>0.Thenwegetthesameresultasbefore:Roberts andRosenbaum (1986)prove that under thesehypotheses andcontinuity,FsatisfiesEquation(4.13).Moreover,ifinadditionFsatisfiesreflexivityandsymmetry, thenF is thegeometricmean.Foravarietyofrelatedresults,seeRobertsandRosenbaum(1986).

4.10 Behavioral responses to health events

Governmentsaremakingdetailedplansforhowtorespondtofuturehealththreatssuch as pandemic influenza, H1N1 virus, a bioterrorist attack with the smallpoxvirus, and so on. A major unknown in planning for future disease outbreaks ishowpeoplewillrespond.Willtheyfollowinstructionstostayhome?Willcritical

Y116134.indb 92 10/10/11 2:11 PM


personnelreporttoworkortakecareoftheirfamilies?Willinstructionsforimmu-nizationbefollowed?Mathematicalmodelsareincreasinglyusedtohelpplanforhealtheventsortodevelopresponsestothem.TheyhavebeenespeciallyimportantinplanningresponsestosuchrecenteventsasfootandmouthdiseaseinBritainandSARS.Modelsinepidemiologytypicallyomitbehavioralresponses.Thesearehardtoquantifyandhardtomeasure.Thisleadstochallengesforbehavioralscientistsandforepidemiologicalmodelerswhowanttoworkwiththem.

Inbuildingbehavioralresponsesintoepidemiologicalmodels,wecanlearnsomethings from the studyof responses tovariousdisasters suchas earthquakes,hur-ricanes,andfires.Manybehavioralresponsesneedtobeaddressed.“Compliance”with things such as quarantine instructions is one example. How do we measure“compliance?” In particular, this includes such factors as “resistance” to instruc-tions,willingnesstoseekorreceivetreatment,credibilityofgovernment,andtrustofdecisionmakers.Otherthingsthatneedtobemadepreciseandmeasuredincludemovement, rumor, perception of risk, person-to-person interaction, motivation,social stigmata (such as discrimination against certain social groups), panic, andpeerpressure.Thereisachallengetomeasurementtheoryhere:howdowemeasuresomeofthesefactors?Howdowebringthemintomathematicalmodels?Whatstate-mentsusingthenewscalesofmeasurementaremeaningful?SomeoftheissuesarediscussedinMcKenzieandRoberts(2003),whichsummarizesaworkshopaimedatmodelingsocialresponsestobioterrorisminvolvinginfectiousagents.

Thereismuchmoreanalysisofasimilarnatureinthefieldofepidemiologythatcanbedonewiththeprinciplesofmeasurementtheory.Theissuesinvolvedpresentchallengesbothfortheoryandforapplication.

Acknowledgments

TheauthorgratefullyacknowledgesthesupportoftheNationalScienceFoundationunder grant numberEIA-02-05116 toRutgersUniversity.Anumberof ideas andsomeoftheexamplesandlanguageinthispaperareborrowedfromRoberts(1994),whichexploresmeaningfulandmeaninglessstatementsinoperationsresearch.

References

Aczél,J.,&Roberts,F.S.(1989).Onthepossiblemergingfunctions.Mathematical Social Sciences, 17,205–243.

Aczél, J.,Roberts,F.S.,&Rosenbaum,Z. (1986).Onscientific lawswithoutdimensionalconstants.Journal of Mathematical Analysis & Applications, 119,389–416.

BayAreaPollutionControlDistrict(1968).CombinedpollutantindexesfortheSanFranciscoBayArea.InformationBulletin10-68,SanFrancisco.

Fleming,P.J.,&Wallace,J.J.(1986).Hownottoliewithstatistics:Thecorrectwaytosum-marizebenchmarkresults.Communications of the ACM, 29,218–221.

Goldman, A. J. (1981). Discrete mathematics in government. Lecture presented at SIAM Symposium on Applications of Discrete Mathematics,Troy,NY,June.

Heath,J.L.(1984).Re-evaluationofRISCI.Computer Architecture News, 12,3–10.

Y116134.indb 93 10/10/11 2:11 PM


Helmholtz,H.V.(1887).Zählenundmessen.Philosophische Aufsätze(pp.17–52).Leipzig:Fues’sVerlag(C.L.Bryan,transl,Counting and measuring.Princeton,NJ:VanNostrand,1930.)

Knapp,T.R.(1990).Treatingordinalscalesasintervalscales:Anattempttoresolvethecon-troversy.Nursing Research, 39,121–123.

Krantz,D.H.,Luce,R.D.,Suppes,P.,&Tversky,A.(1971).Foundations of measurement(Vol.I).NewYork:AcademicPress.

Kruskal, J.B. (1956).On theshortest spanning treeofagraphand the travelingsalesmanproblem.Proceedings of the American Mathematical Society, 7,48–50.

Luce,R.D.(1959).Onthepossiblepsychophysicallaws.Psychological Review, 66,81–95.Luce,R.D.(1962).CommentsonRozeboom’scriticismsof“Onthepossiblepsychophysical

laws.”Psychological Review, 69,548–555.Luce, R. D. (1964). A generalization of a theorem of dimensional analysis, Journal of

Mathematical Psychology, 1,278–284.Luce,R.D.(1990).“Onthepsychophysicallaw”revisited:Remarksoncross-modalmatch-

ing.Psychological Review, 97,66–77.Luce,R.D.,Krantz,D.H.,Suppes,P.,&Tversky,A.(1990).Foundations of measurement

(Vol.III).NewYork:AcademicPress.Mahadev,N.V.R.,Pekeč,A.,&Roberts,F.S. (1998).On themeaningfulnessofoptimal

solutionstoschedulingproblems:Cananoptimalsolutionbenon-optimal?Operations Research, 46 suppl,S120–S134.

Marcus-Roberts,H.M.,&Roberts,F.S.(1987).Meaninglessstatistics.Journal of Educational and Behavioral Statistics, 12,383–394.

McKenzie,E.,&Roberts,F.S.(2003).Modelingsocialresponsestobioterrorisminvolvinginfectiousagents.TechnicalReport,DIMACSCenter,RutgersUniversity,Piscataway,NJ,July24.(Availableathttp://dimacs.rutgers.edu/Workshops/Modeling/.)

Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial optimization: Algorithms and complexity,EnglewoodCliffs,NJ:Prentice-Hall.

Pekeč,A.(1996a).Limitations on conclusions from combinatorial optimization,Ph.D.Thesis,DepartmentofMathematics,RutgersUniversity.

Pekeč,A.(1996b).Scalingsinlinearprogramming:Necessaryandsufficientconditionsforinvari-ance.CenterforBasicResearchinComputerScience(BRICS),technicalreportRS-96-50.

Pfanzagl,J.(1968).Theory of measurement.NewYork:Wiley.Roberts,F.S.(1972).BuildinganenergydemandsigneddigraphI:Choosingthenodes.Rept.

927/1—NSF.April.SantaMonica,CA:TheRANDCorporation.Roberts,F.S.(1973).Buildingandanalyzinganenergydemandsigneddigraph.Environment

& Planning, 5,199–221.Roberts,F.S.(1979).Measurement theory, with applications to decisionmaking, utility, and the


Roberts, F. S. (1990). Meaningfulness of conclusions from combinatorial optimization.Discrete Applied Mathematics, 29,221–241.

Roberts, F. S. (1994). Limitations on conclusions using scales of measurement. In S. M.Pollock,M.H.Rothkopf,&A.Barnett(Eds.),Operations research and the public sec-tor,Vol. 6ofHandbooks in operations research and management science(pp.621–671).Amsterdam:North-Holland.

Roberts,F.S.(1999).Meaninglessstatements.InContemporary trends in discrete mathemat-ics,DIMACSSeries (Vol.49,pp.257–274).Providence,RI:AmericanMathematicalSociety.

Y116134.indb 94 10/10/11 2:11 PM


Roberts,F.S.,&Rosenbaum,Z.(1986).Scaletype,meaningfulness,andthepossiblepsy-chophysicallaws.Mathematical Social Sciences, 12,77–95.

Sauter, G. D., & Chilton, E. G. (Eds.). (1970). Air improvement recommendations for the San Francisco Bay Area. The Stanford-Ames NASA/ASEE Summer Faculty Systems Design Workshop, Final Report.October.StanfordCA:StanfordUniversitySchoolofEngineering.PublishedunderNASAContractNGR-05-020-409.

Siegel,S.(1956)Nonparametric statistics for the behavioral sciences,NewYork:McGraw-Hill.Stevens,S.S.(1946).Onthetheoryofscalesofmeasurement.Science, 103,677–680.Stevens,S.S.(1951).Mathematics,measurement,andpsychophysics.InS.S.Stevens(Ed.),

Handbook of experimental psychology(pp.1–49).NewYork:Wiley.Stevens, S. S. (1959). Measurement, psychophysics, and utility. In C. W. Churchman &

P.Ratoosh(Eds.),Measurement: Definitions and theories(pp.18–63).NewYork:Wiley.Stevens,S.S.(1968).Ratioscalesofopinion.InD.K.Whitla(Ed.),Handbook of measurement

and assessment in behavioral sciences.Reading,MA:Addison-Wesley.Suppes,P.(1959).Measurement,empiricalmeaningfulnessandthree-valuedlogic.InC.W.

Churchman&P.Ratoosh(Eds.),Measurement: Definitions and theories(pp.129–143).NewYork:Wiley.

Suppes,P.(1979).Replies.InR.J.Bogdan(Ed.),Patrick Suppes(pp.207–232).Dordrecht,Holland:Reidel.

Suppes,P.,Krantz,D.H.,Luce,R.D.,&Tversky,A.(1989).Foundations of measurement (Vol. II).NewYork:AcademicPress.

Suppes, P., & Zinnes, J. (1963). Basic measurement theory. In R. D. Luce, R. R. Bush, &E.Galanter(Eds.),Handbook of mathematical psychology(Vol.1,pp.1–76).NewYork:Wiley.

Thomas,H.(1985).Measurementstructuresandstatistics.InS.Kotz&N.L.Johnson(Eds.),Encyclopedia of statistical sciences(Vol.5,pp.381–386).NewYork:Wiley.

Walther,E.G.(1972).Aratingofthemajorairpollutantsandtheirsourcesbyeffect.Journal of the Air Pollution Control Association, 22,352–355.

Y116134.indb 95 10/10/11 2:11 PM

Y116134.indb 96 10/10/11 2:11 PM

97

Toward a probabilistic theory of measurement

Giovanni Battista RossiDIMEC,UniversitàdegliStudidiGenovaGenova,Italy

5.1 Basic questions

Measurementplaysafundamentalroleinbothphysicalandbehavioralsciences,asalinkbetweenabstractmodelsandempiricalreality.Soachallengearises,concerningthepossibilityofdevelopingauniquetheoryofmeasurementforthedifferentdomainsofscienceinwhichitisinvolved.Whatwouldweexpectfromonesuchtheory?

Perhapsthreemainquestionsshouldbeaddressed,namely,

• Whatisthemeaningofmeasurement?• Howdowemeasure?• Whatcanbemeasured?

Wepresenthereabasickernel forsucha theory,organized in twomainparts,themeasurementscaleandthemeasurementprocess,dealing,respectively,withthefirsttwoquestions.Ananswertothethirdquestion,concerningmeasurability,comesfromacombinationofthetwomainpartsofthetheory,ratherthanasaseparateissue.Wepresenteachpartindeterministictermsfirst,then,inasmuchasuncertaintyisaninherentfeatureofmeasurement,wereformulatethetheoryinprobabilisticterms.

Beforestarting,letusfixsomelanguage.Wefirstneedatermfordenotingwhat we (want to) measure.Wemaycallthataproperty,characteristic,attribute,orfea-tureofsomething:wechoosecharacteristic.Thenwehavetonamewhat carries(expresses, manifests) the characteristic under investigation: depending upon thediscipline involved, itmaybeanobject,anevent,orevenaperson.Here thedif-ferencesaresubstantial,yetwestilluseauniqueterm,object,butmakeclearthatthisisjustaconventionaltermfordenotingwhatcarriestheproperty.Thenwehavetodistinguishbetweenmeasurable andnonmeasurableproperties: for the formerweuse,againconventionally,thetermquantity.Furthermore,objectsmanifestthecharacteristicofinterestindifferentways(levels,degrees)andwecallstate thewayinwhichanobjectmanifestsaproperty.

5

Y116134.indb 97 10/10/11 2:11 PM


Wearenowreadytointroducetheconceptofmeasurement initself.WerefertoFinkelstein’sdefinition(1982):“Measurementistheprocessofempiricalobjectiveassignmentofnumberstoproperties(characteristics)ofobjectsoreventsoftherealworldinsuchawayastodescribethem.”Otherkeyterms,suchasmeasurementscale,measuringsystem,andmeasurementprocess,arediscussedhere,andattheendofthechapterashortglossaryisincludedforeaseofreference(BIPM,2008b;Rossi,2009b).

5.2 The measurement scale

5.2.1 The meaning of measurement and the notion of measurement scale

Inordertoputmeasurementonafirmfoundation,weneedtostartfromamoreele-mentaryconcept,thenotionofempirical relation.Letusdemonstratethisthroughasimpleexample:themeasurementofhardnessofmineralsaccordingtothemethodproposedbyMohsin1812(Finkelstein,1982).Thekeyideaofhismethodwasthatthehardnessofamineralmaybecharacterizedbyitsabilitytoscratchotherminer-als.Thisistheempiricalrelationinthiscaseand(wemayassumethat)itisaweak order.1He then identifieda seriesof10 referencematerials, collectively suited toexpressthedifferentdegreesofhardnesswemayencounterinnature.Theywere,inincreasinghardnessorder,talc,gypsum,calcite,fluorite,apatite,orthoclase,quartz,topaz,corundum,anddiamond.Thenheassignedthenumbersfrom1to10tothem,therebyfullydefiningareference scale.Wecallastandardeachelementinthescaleandmeasurethecorrespondingnumber.Usingthisasabasis,itispossibletodeter-minethehardnessofanypieceofmineralabycomparingittothescaleinordertoidentifythestandardstowhichitisequivalent,inthesensethatitneitherscratchesnorisscratchedbyit.

Thissimpleexampleisgoodforshowingthemeaningofmeasurement:thenumber(measure)soassignedwillreallyrepresenthardness,inthatweexpectatohavethesamehardnessofanyotherobjectthatobtainsthesamemeasure,tobeharderthananyobjectobtainingalessermeasure,andviceversa.Inotherwords,iftwoobjects,aandb,havehardnessm(a)andm(b),respectively,theorderthatholdsbetweenthemeasurescorrespondstotheorderthatholdsbetweentheobjects;thatis,

a b m a m b ⇔ ( ) ≥ ( ). (5.1)

An expression like (5.1), linking empirical relations to numerical relations, iscalledarepresentation theorem.Wemaynotethatthemeasureofanobjectrevealshowitcompareswithalltheotherobjectsthatcarrythesamecharacteristic:inourexample,howamineralcompareswithallotherminerals,inrespectofhardness.Thealternativeforobtainingthesameinformationwouldbetoactuallycomparetheobjecttobemeasuredwithalltheothersintheclass.Thisshowshowvaluabletheinformationconveyedbymeasurementis.Howisitpossibletoobtainsucharemark-ablesavingineffort?Thisisthankstotheconstructionofthereferencescale(the

Y116134.indb 98 10/10/11 2:11 PM

Toward a probabilistic theory of measurement 99

10standardsobject, in theMohscase),which“materialisesourknowledgeof thequantity under examination” (Mari, 2000). Actually in constructing the scale wehavemadegooduseofourknowledge,consistinginknowingthathardnessmaybecharacterizedbytheabilitytoscratchanothermaterialandthatthemainwaysinwhichitappearsinnaturearewellrepresentedbytheten-standardsseries.

Sowemaynowanswerthequestionaboutthemeaningofmeasurement,bysay-ingthatmeasurementsreproduceempiricalrelationsinanumericaldomain,thankstotheexistenceofameasurementscale.Notethatascalemaybeintendedintwoways,asthesetofconditionsthatenablemeasurement(generalmeaning)oraseriesofstandardobjectswithassignedvalues(specificmeaning).Toavoidconfusion,weusethetermreferencescaleinthelattercase.

Ingeneral,wemayhavedifferenttypesofempiricalrelations,aswehavealreadydiscussedinChapters1and4,correspondingtodifferentempiricalstructuresandrelatedmeasurementscales.AsummaryofthemostcommonandusefultypesofscaleshasbeenreportedinChapter1,Table 1.2.AswehavediscussedinChapter1,scales may be characterized from two different standpoints, either according toempirical properties that they are able to reproduce, expressed by representationtheorems, or on the basis of the admissible transformations that they may safelyundergo.Thissecondpointofview is related tomeaningfulness,whichhasbeenamplydiscussedinChapter4.

These ideas are basic in the representational theory of measurement. For prob-ingthissubjectfurther,werecommendRoberts(1979/2009)asanintroduction,thenKrantz,Luce,Suppes,andTversky(1971),Luce,Krantz,Suppes,andTversky(1990),andSuppes,Krantz,Luce,andTversky(1989)foraremarkablepresentationofkeyresults.AconciseoverviewisprovidedinFinkelsteinandLeaning(1984),andarecentsurveyisofferedbyLuceandSuppes(2002).WealsorecommendreadingtheoriginalessaybyHelmholtz(1971/1887),forunderstandinghowtheseideasoriginated.

Thusfarwehavepresentedmeasurementscalesina“deterministic”way,thatis,byassumingthatalltherelationsinvolvedareeither“true”or“false.”Thisisgoodforclarifying themeaningofmeasurement,butmayno longerbeadequate ifwelookforafaithfuldescriptionofwhathappensinreality.Wenowdiscusswhenthisapproachisnolongersatisfactory.

5.2.2 On the need for a probabilistic approach

Consider again an empirical weak-order relation, as the relation “heavier than,”which occurs in mass measurement. We call a comparator any device that per-formsacomparisonofobjects,inrespectofsomecharacteristicoftheirs.Considernowaclassofobjects,forexample,books,andamasscomparator,forexample,anequal-armbalance.Therearecasesinwhichtheresultofthecomparisonmaybeexpressedbyadeterministicstatement,suchasa≻b,where“≻”means“(empiri-cally)greaterthan,”whichimpliesthatwheneverwecompareawithb,weobservethataisheavierthanb.Forexample,ifaandbaretwodifferentbooksinthesameseries, theywillhaveundergone thesamepublishingstandards, that is, the same

Y116134.indb 99 10/10/11 2:11 PM


kindofpaper,samecover,andsoon,buttheymayhaveadifferentnumberofpages,forexample,500versus400.Thena (thebookwith500pages)willbedefinitelyheavierthanbandwemaycorrectlyrepresenttherealsituationbythedeterministicrelationa≻b.

Butsupposenowthataandbaretwodistinctcopiesofthesamebookinthesameseries;thatis,theyhavethesamekindofpaper,samecover,samenumberofpages.Thentheyshouldbenominallyequivalentandweshouldobservea~b,where“~”means“empiricallyequivalentto.”Yetwecannotbesureofthis,inasmuchastheremaybesmalldifferences,duetotheproductionprocess(inhomogeneityofthepaper,smalldifferencesoccurringinthebindingprocess,etc.)ortotheusage(aging,wear-ing,dust,etc.).Ifweuseacomparatorsensitiveenoughfordetectingsuchdifferencesandiftheyareofthesameorderofmagnitudeofthe“noise”ofthecomparator(due,e.g., to internal friction, vibration, etc.), if we repeat the comparison more timeswemaysometimesobservea≻b,someothersa~bandsomeothersevena≺b.Sowecanno longerdescribe thissituationbyasingle,well-determined relation,ratherwehavetoconsidertheprobabilityofeachofthepossiblerelations;thatis,ℙ(a≻b),ℙ(a~b),andℙ(a≺b),wherethesymbol“ℙ”denotestheprobabilityofarelation.Thefollowingconstraintholdstrue:

P P Pa b a b a b ∼ ≺( )+ ( )+ ( ) = 1. (5.2)

Thisiswhythenecessityofaprobabilisticrepresentationemerges:whenwecom-pare similar objects, the result of the comparison may become uncertain. If thisholds truefor themeasurementofphysicalquantities, it isevenmoreapparent inthecaseofperception.Ifweaskasinglepersonorajurytocompare,say,theloud-nessofsoundsorthebrightnessofobjects,wemayobservesomeintra-andinter-individualvariability.Ontheotherhand,thecaseofawell-determinedorder,suchastheoneoccurringbetweenthetwodifferentbooks,maybeincluded,byassigningℙ(a≻b)=1.Theprobabilisticapproachmaythusbeconsideredasageneralizationofthedeterministicone.Inthischapterweuseonlysimpleprobabilityrules.GoodintroductionstothisdisciplinehavebeenprovidedinItalianbyMontiandPierobon(2000)andinEnglishbyPapoulis(1984).

5.2.3 More on the interpretation of the probabilistic approach

Before proceeding, let us consider some additional points concerning the use ofprobability.Ingeneral,ifwehavetwoobjectsaandb,andacomparatorC,andweobservesomevariabilityintheresultsofthecomparison,howcanweinterpretthisvariability?Shouldweascribethisvariabilitytotheobjects,holdingthatthechar-acteristictheymanifestactuallyvariesatrandomfromonecomparisontoanother,or shouldwe insteadascribe it to thecomparator, assuming that it is affectedbyrandomnoise?

Wesuggestthat,ingeneral,thereisnodefiniteanswertothisquestion:iftheonlyevidencewehaveistheresultofthecomparison,wecannotseparatethecontributionoftheobjectsfromthatofthecomparator,ormoregenerally,thecontributionofthe

Y116134.indb 100 10/10/11 2:11 PM


observerfromthecontributionoftheobservedsystem.Thisisakindofuncertaintyprinciplethat,itisnoteworthy,alreadyemergesfromthisverysimplemodel.

Secondpoint: someonecouldargue that theuseofa less-sensitivecomparatorwouldleadtoadeterminedresultofthecomparison.Unfortunatelythisisonlyanapparentsolution.Infact,supposethatthecomparatorhasalimitedresolution:itisunabletodetectdifferencesintheobjectsthatarelessthanathreshold,sayδ.Nowsupposethatobjectsaandbdifferbylessthanthethreshold;thentheresultwillbea~δb,wherethesymbol“~δ”denotestheindifferenceconditionexpressedbythecomparator.Supposenowthatbandcalsodifferbylessthanthethreshold;thenweobtainb~δc.Yetthereisnoguaranteethataandcarealsowithinthethreshold,sotherelation“~δ”isnottransitive;thatis,itisnolongeranequivalencerelation.Sotheapparentbenefitofhavingabetterrepeatability(i.e.,apropensitytoobtainthesameresultor thesamesetof resultswhenrepeating thesameexperiment in thesameconditions)isbalancedbyaseveredrawback,thelossoftheequivalencerelation.Remedywouldbeworsethanmalady!

Another possible question is whether the limited repeatability of comparisonsis theonly reason for looking for a probabilistic representation. We suggest that,althoughthiscouldbeagoodreasoninitself,itisnottheonlyone.

Supposewehavetwo(andonlytwo)comparators,CandD,andthatwehavethesameconfidenceinbothofthem.SupposethatwecompareaandbwithcomparatorCandthatwealwaysobtain,forexample,a≻Cb;insteadleta~Dbbetheresult,alsoconfirmedthroughrepetitions,ofthecomparisonthroughD.Sowehaveasystem-aticdisagreementbetweenthetwocomparators,andnowayfordecidingbetweenthem.Whatcanwedo?Againwecantreatthiscaseinprobabilisticterms,consider-ingtheresultsofthecomparisonsasconditioned byCandD,respectively,andbyapplyingtherulesofprobabilityaccordingly.

Inprobabilisticterms,theaboveresultmaybeexpressedasfollows:

P a b C |( ) = 1, P a b C∼ |( ) = 0 , P a b C≺ |( ) = 0 ,

P a b D |( ) = 0 , P a b D∼ |( ) = 1 , P a b D≺ |( ) = 0 ,

P C( ) = 0 5. , P D( ) = 0 5. ,

andweobtain

P P P P Pa b a b C C a b D D ( ) = ( ) ( )+ ( ) ( ) =| | .0 5,

P P P P Pa b a b C C a b D D∼ ∼ ∼( ) = ( ) ( )+ ( ) ( ) =| | .0 5 , (5.3)

P P P P Pa b a b C C a b D D≺ ≺ ≺( ) = ( ) ( )+ ( ) ( ) =| | .0 0 .

Last,letusdiscusstheinterpretationoftheabove.

Y116134.indb 101 10/10/11 2:11 PM


Wemaythinkthateithera≻bora~bistrueandoneofthetwocomparatorsiswrong,butwedonotknowwhichone,andsotheonlylogicallyconsistentwayofexpressingwhatweknowiswhatwehavedone.Yetwecanalsoconsider,asanalternativeexplanation,thepossibilitythattheobjectsinteractwiththecomparatorsinsuchaway that their statechanges in the twocomparisons,butweareunabletodefine their stateoutside thesecomparisons, if thesecomparisonsare theonlyevidencethatwehave.Thisisanotherkindoftheuncertainty,orratheroftheinde-terminacy,condition.Ingeneralwemayseethatweareusingprobabilityasalogic,forexpressingdifferentuncertaintysituations.

5.2.4 Probabilistic relations

Weshouldnowbemotivatedenoughtosearchforaprobabilisticrepresentation.Thisapproachisbasedonthenotionofaprobabilistic relation,whichisnotcommoninthetheoryofprobability,and,asfarasweknow,onlyrecentlyhasbeeninvesti-gated,sowediscussitinsomedetail.

Consideradeterministicrelationfirst,forexample,aweakorder,⋟.Sucharela-tionmaybeunderstoodintwoways:

• Inaspecificmeaning:whenwewritea⋟b,weintendthattherelation⋟holdsforthepair(a,b)∈A.

• Inageneralmeaning:whenweconsidertherelation⋟onA,werefertothesetofallthepairsofelementsofAforwhichitholds.

Consider,forexample,thesetA=a,b,candthenumericalexampledisplayedinTable 5.1.Thepossibleweak-orderrelationsthatmayholdonA,thatis,thepossibleweakorderingsofitselements,arelistedincolumn1ofthetable.Inthedetermin-isticcase,oneandonlyoneofsuchorderingsholdstrue.Intheprobabilisticcase,instead,moreorderingsarepossibleandaprobabilityisassignedtoeachofthem,withtheobviousconstraintthatthesumofallsuchprobabilitiesmustbeequaltoone.Thisiswhatwecallaprobabilistic weak orderbecauseweak-orderrelationsonlyhavenonnullprobabilities.Exampleprobabilityvaluesareprovidedinthelastcolumnofthetable.Inordertounderstandhowtheseprobabilitiesmaybeassigned,supposethata,b,andcaresoundsandweaskagroupof,say,100personstocom-parethemwithrespecttotheirloudness(orsharpness,pleasantness,etc.).Then,eachlineinthetablerepresentsonepossibleresponsefromoneperson.Ifresponse⋟1,forexample,a≻b≻c,isprovidedby20personsoutof100,weassignprobability0.2toit,andsoon.

With the numbers provided in this example, qualitatively, a is in the averagegreaterthanbothbandc,andbandcareequivalent.FromthisoverallprobabilisticrelationonA, it ispossibletoderivetheprobabilitiesofthespecificrelationsthatholdforeachpairofelements.Forexample,

P P P P P a b( ) = ( )+ ( )+ ( )+ ( )1 2 8 10 ;

Y116134.indb 102 10/10/11 2:11 PM


thatis,theprobabilityofthe(specific)relationa≻bisthesumoftheprobabilitiesoftheorderingsinwhichitholdstrue.Withthesepremises,wearenowinapositiontoformulateandproveaprobabilisticrepresentationtheorem.

5.2.5 Probabilistic representations

Consideragainthedeterministicrepresentationforweakorder,

a b m a m b ⇔ ≥( ) ( ).

Aswehavejustseen,a⋟bisnownolongerastatementthatmustbeeithertrueorfalse;ratheritexpressesapossibilitythathasadegreeofprobabilityℙ(a⋟b).Asaconsequenceofthis,theassignmentofnumberstoobjectswillnolongerbeunique.Ratherweassociatewitheachelement,saya,arandomvariablexa.Whatwecanrequirenowisthatthisprobabilityofobservingarelationbetweentwoobjectsisthesameastheprobabilityofobtainingthesamekindofrelationbetweentheassociatedrandomvariables.So,inthecaseoforder,therepresentationtheorembecomes,foreacha,binA,

P a b P x xa b( ) = ≥( ) , (5.4)

or,equivalently,

P a b P x xa b( ) = >( )

Table 5.1 AnillustrativeexampleofaprobabilisticorderstructureonA=a,b,c

Weak orders⋟i

Orderings corresponding to weak orders ⋟i xa xb xc ℙ(⋟i)

⋟1 a≻b≻c 3 2 1 0.2

⋟2 a≻c≻b 3 1 2 0.2

⋟3 b≻a≻c 2 3 1 0.0

⋟4 b≻c≻a 1 3 2 0.0

⋟5 c≻a≻b 2 1 3 0.0

⋟6 c≻b≻a 1 2 3 0.0

⋟7 a~b≻c 2 2 1 0.1

⋟8 a~c≻b 2 1 2 0.1

⋟9 b~c≻a 1 2 2 0.0

⋟10 a≻b~c 2 1 1 0.3

⋟11 b≻a~c 1 2 1 0.0

⋟12 c≻a~b 1 1 2 0.0

⋟13 a~b~c 1 1 1 0.1

Y116134.indb 103 10/10/11 2:11 PM


and

P a b P x xa b∼( ) = =( ).

Wenowshow,inaninformalway,howthiscanwork.LookagainatTable 5.1;sup-posethatwehavefirstassignedtheprobabilitiestoeachofthepossibleorderingsviathesoundevaluationexperiment,aswehaveexplainedintheprevioussubsection.

Now,foreachofthepossibleorders,itispossibletofindanumberassignmenttotherandomvariablesxa,xb,andxc,compatiblewiththatorder.Forexample,fororder⋟1,whichimpliesa≻b≻c,averynaturalassignmentisxa=3,xb=2,andxc=1.Actuallytherearemanyotherpossibleassignmentsthatsatisfyempiricalrelations,butsupposethatweadoptsomeconventionalrulesandmakethisassignmentalwaysunique.Forexample,wecouldagreetoalwaysusethefirstnaturalnumbers,1,2,…andinthiswaywegetthenumberassignmentsshownincolumns2–4ofTable 5.1.Eachnumberassignmentcorrespondstoanorderandthenitobtainsthesameprob-abilityof theassociatedorder. In thiswayit ispossible toassociateaprobabilitydistributionwiththerandomvariablesxa,xb,andxc.Consider,forexample,P(xa=1).Theassignmentxa=1occursinassociationwithorders⋟4,⋟6,⋟9,⋟11,⋟12,and⋟13.Consequently,becausesuchorderingsaremutuallyexclusive,theprobabilityofthatassignmentwillbeequaltothesumoftheprobabilitiesoftheordersinwhichitistrue:

P xa =( ) = ( )+ ( )+ ( )+ ( )+ ( )+1 4 6 9 11 12 1P P P P P P 33

0 0 0 0 0 0 0 0 0 0 0 1 0 1

( )= + + + + + =. . . . . . .

Similarly,wemaycalculatetheprobabilitiesofxa=2andxa=3,andtheprobabilitydistributionsoftherandomvariablesxbandxc.TheresultsareshowninFigure 5.1.

Inasimilarwaywemayalsocheck,inthesameexample,thattherepresentationtheorem(5.4)holdstrue,withoutprovidingaformalproof,whichmaybefoundinRossi(2006).

Consider,forexample,therelationa≻b:inourexampleitissatisfiedinorders⋟1,⋟2,⋟5,⋟8,and⋟10and,consequently,

P P P P P P a b( ) = ( )+ ( )+ ( )+ ( )+ ( )

= +

1 2 5 8 10

0 2 0. .. . . . . .2 0 0 0 1 0 3 0 8+ + + =

Ontheotherhand,thenumericalrelationxa>xbisverifiedincorrespondencewiththesameorders(thisisquiteobviousfortheverywayinwhichwehaveassignedthevaluestothevariablesxa,xb,andxc!)andsowealsoobtainP(xa>xb)=0.8,aspredictedby(5.4).

Itisimportanttonotethatthemeasurevalue,whichinthedeterministiccasewasafunctionoftheobject,isnowarandomvariable.Thisisabigchangeinperspec-tive and merits some discussion. In the deterministic case each object manifests

Y116134.indb 104 10/10/11 2:11 PM


thecharacteristicunderinvestigationinone(andonlyone)way.Wecall thestate(of theobject) thewayinwhichanobjectmanifestsacharacteristic.Inthedeter-ministiccasethereisaone-to-onecorrespondencebetweenobjectsandstatesandbetweenstatesand(measure)values.Intheprobabilisticrepresentation,duetotheuncertaintyofempiricalrelations,thereisaone-to-manycorrespondencebetweenobjectsandstates,whereastherestillisaone-to-onecorrespondencebetweenstatesandvalues.

Whatwehavesofardiscussedfororderstructuresmaybeextended,inasimilarway, to other important structures occurring in measurement, such as differenceandextension.AsummaryofresultsispresentedinTable 5.2.Thetablepresentsanoverviewoftheoveralltheorythatisprogressivelypresented,withtherelatednota-tion,intherestofthechapter.

Theprobabilisticapproachtorepresentationsinmeasurementismuchlessdevel-opedthenthedeterministicalgebraicone.IntroductorypresentationsmaybefoundinRoberts(1979/2009,Chapter6)andinSuppesetal.(1989,Chapter17).AnearlyworkbyLeaningandFinkelstein(1980)meritsmention,aswellasaconspicuouscontributionbyFalmagne(1980).ThesubjecthasreceivedadecisivecontributioninRegenwetter(1996),wherethenotionofaprobabilistic(therecalledrandom)rela-tionisinvestigatedindepth.AdditionaltreatmentispresentedinRegenwetterandMarley(2001).ThetheorysketchedhereispresentedindetailinRossi(2006),whereproofisprovidedforalltheresultsinTable 5.2.

1 20

0.5

1Marginal Distributions of xa, xb, xc

P(xc

)

1 2 30

0.5

1

P(xb

)

1 2 30

0.5

1

P(xa

)

Values of the Variables

3

Figure 5.1 Probabilitydistributionsfortherandomvariablesxa,xb,andxc,associatedwithobjectsa,b,andc,respectively,intheexamplereportedinTable 5.1.Allthesevariablesmaytakethevalues1,2,or3.Notethatforthevariablexahighervalues[2,3]havehigherprob-abilities,whereasforvariablesxbandxctheconverseholdstrueandtheyalsohavethesamedistribution.

Y116134.indb 105 10/10/11 2:11 PM


5.2.6 The construction of the reference scale

Wehave considered thenotionof scalemainly in its generalmeaning, as a set ofconditions necessary for measurement to be possible. Yet we have also mentionedanother,morespecific,meaningofthescale,thatwenowhavetodiscuss.Oncewehaveassessed,orevensimplyassumed,theexistenceoftheconditionsformeasurabil-ity,inordertomakemeasurementpossiblewehavetoactuallyconstructareferencescale.Letusseewhatthismeans,bothinabstracttermsandthroughsomeexamples.

We have already encountered an example of a reference scale, the Mohs scaleforhardness,consistingoftenreferenceminerals,withassignedvalues.Whatisthecriterionforchoosingsuchstandards?Inabstractterms,inthedeterministiccase,forafinitesetAwith,say,Nelementsunderanequivalencerelation,wepartitionAintoitsequivalenceclasses,2andthenselectoneelementineachclasstoformtheseriesofstandards.Thisserieshastorepresentthepossiblemanifestationsofthecharacteristicor,equivalently,thepossiblestatesinwhicheachobjectmaybe.Rememberthatinthedeterministiccase,eachobjectwillbeinone(andonlyone)state.

Letusnowdiscussthehypothesisoffiniteness,whichisveryusefulforsimpli-fyingthemathematics.Therationalefor thisassumptionis thatwecannotattain,experimentally,aninfiniteresolution,thatis,theabilityofdetectinginfinitelysmallvariations.Whateverthesensitivityofthecomparator,therewillalwaysbealowerlimittothevariationsthatitisabletodetectandsoitisreasonabletoassumeafinite

Table 5.2 Synopsisofthetheoryconsidered,includingboththedeterministicandtheprobabilisticapproach

The measurement scale

Scale type Deterministic approach Probabilistic approach

Order a⋟b⇔m(a)≥m(b) ℙ(a⋟b)=P(xa≥xb)

Interval ∆ ∆ab cd

m a m b m c m d

⇔

( )− ( ) ≥ ( )− ( )

P ∆ ∆ab cd

a b c dP x x x x

( ) =

= − ≥ −( )Ratio a~b°c⇔m(a)=m(b)+m(c) ℙ(a~b°c)=P(xa=xb+xc)

The measurement process

Process Deterministic approach Probabilistic approach

Observation y=f(x) P(y∙x)

Restitution x=f –1(y) P y P x P y xx yx X

( ) = ( ) ( )

∈

−

∑1

;

x=μ(x∙y)

Measurement x=h(x)=x P x x x x y P y xy Y

ˆ ˆ( ) = − ( ) ( )∈

∑δ µ

Y116134.indb 106 10/10/11 2:11 PM


resolutionforthescale.Ontheotherhand,foranyfixedclassofmeasurementprob-lems,itwillalsobepossibletoassumeamaximumvalueforthecharacteristic.Now,afiniteresolutionplusafinitemaximumvaluemakesa finite set of possible states.Sothehypothesisoffinitenessfortheclassofthepossiblestatesmaybemaintainedinfullgenerality.

Someexamplesmaynowbeuseful.Priortoexaminingthem,inthisregardwealso have to remember that what we have called an object (something that has astableexistence),mayalsobeanevent instead (something thatmaybegeneratedwhenneeded,but thatdoesnotpersist).Asafirst example,weconsider thecaseof mass measurement: here the reference scale is a set of standard masses, withassignedvalues.Howmanymassesmaybeneededinapracticalcase?Ifwewanttorealize,forexample,themassvaluesfrom,say,1gto1kg,withresolutionof1g,whichmakes1,000values,wedonotneedtohave1,000distinctmasses,butratherthefollowingseries:

m m m m

m m m

1 2

10 20

g 2 g g*

5 g

g 20 g g

, , , ,

, , **50 g

g 200 g g*

500 g

, ,

, , , ,

m

m m m m100 200

where the subscript denotes the value of the standard and the asterisk denotes asecondcopyofanalreadyavailablestandard.Itiseasytocheckthat,thankstotheadditionproperty,itispossibletogenerateallthevaluesrequired,withonly12mate-rialstandards!Forexample,wegetm8 gasthesumofthreestandardsintheseries;thatis,m8 g=m1 g°m2 g°m5 g,where“°”denotesphysicaladdition.Notealsothatweneedtwocopiesofm20 g,eventhoughwecouldgetm20 gfromotherterms,becauseotherwisewecouldnotgetm41 g.Thissimpleexampledemonstrateshowadditivityhelpsandthatwedonotneedtohaveall thestandardstoexistat thesametime,butratherwehavetobeabletogeneratethem,byadditioninthiscase,whentheyareneeded.

Anotherexampleistheprimaryscaleforlength.Thisisobtainedbyasophisti-catedlasersystem:herethescaleisanelectromagneticfieldthatpropagatesinthespacethroughparallelplanes.Thedistancebetweensuchplanes,whichisonewave-length,isthebasicreferenceitemforsuchascale,becauseitispossibletorealizemovementsparalleltothepropagationdirectioncorrespondingtoacertainnumberofwavelengths(oroffractionsofawavelength).Inasense, thereferencescale ismaterializedbymovementsofaslideonaguide,soitisacollectionof“events,”thepositionsoftheslidesalongthereferenceaxis,whichinturnareobtainedassumsofelementaryevents,thatis,smallmovementsoftheslide,ofonewavelengthoronefractionofwavelengtheach.Examplesmaybeconsideredintheperceptualdomainalso. In the case of sound perception the reference items are carefully generatedsyntheticsoundshavingawell-definedphysical intensity,spectrum,andduration.Forvision,theymaybereferencecolors,andforsmell,referenceodorsobtainedbycarefullypreparedsolutions.

Y116134.indb 107 10/10/11 2:11 PM


Thusfarwehavediscussedthereferenceelements.Wenowdiscusstheassign-ment of numerical values to them. In the deterministic case we assign a singlevaluetoeachelement.Inthecaseofaprobabilisticrepresentation,eachobjectischaracterized,atleastinprinciple,byaprobabilitydistributionoverasetofpos-siblevalues,ratherthanbyasinglevalue.Inpractice,referencescalesareaccom-paniedbysomestatementabouttheuncertaintyofthesamples:thepointisthatsuchastatementhasafoundationintheprobabilisticrepresentationthatwehaveproposed.Letusconsideragaintheexampleofthemassscale.Onecouldarguethatifwehadagoodcomparator,therewouldbenouncertaintyindistinguish-ingbetweentwomasseshavingadifferentvalue,forexample,m10 gandm20 g,sowhereistheuncertainty?Actuallytherearemanycausesofuncertainty,butjusttohaveanimmediateanswerconsiderthatinthescaletheremaybedifferentwaysofobtainingasamevalue,forexample,a=m20 gisequivalenttob=m10 g°m5 g°m2 g°m2

* g °m1 g.

Now,elementsaandbwillbeveryclosetoeachotherandthenitisreasonablethatthecomparatormaycomparethemonlyinaprobabilisticsense.Finally,forthosesamplesthatmayberealizedonlyinoneway,forexamplem1 g,oneshouldconsiderthepossibilityofhavinganother independentrealizationof thereferencescaleandthen,there,anelement,m1 g′,thatmaybecomparedonlyprobabilisticallywithm1 g.Wehopethatthisoversimplifiedexamplemaygiveafeelingoftheuncertaintyofareferencescaleandhowitmaybecharacterizedintermsofempiricalrelations.

5.3 The measurement process

5.3.1 The measuring system

Oncewehaveconstructeda referencescale,wehave toconsiderhowtomeasureobjectsnotincludedinthescale.Firstweintroducesomeadditionallanguage.Foranyobjecta,weconsiderits(measure) value,x=m(a),asthenumberthatproperlydescribes a, with respect to the characteristic under consideration, that is, as thevaluesatisfyingtherepresentationtheorem(5.1or5.4).AdirectdeterminationofxwouldimplyactuallycomparingawitharepresentativesetofelementsofA:thisiswhatneedsdoing,atleastinprinciple,fortheelementsoftheseriesofstandardsS.Soforeachelements∈S,weobtaina(measure)valuem(s)associatedwithit.ForanobjectanotbelongingtoS,wemaystillconsideritsmeasurevaluex=m(a),asthevaluevirtuallyobtainablebycomparingawitharepresentativesetofelementsofA,butingeneralweproceed,forconvenience,inadifferentway,thatis,throughameasurement process,usuallycarriedoutbyameasuringsystem.Bydoingthis,weobtain,fora,ameasurement value,denotedbyx.So,tosumup,wecallthe“measurevalue,”orsimply“value,”ofanobjectthenumberthatwewouldobtainbycompar-ingitwith(actuallyorvirtually)alltheotherelementsinthesetA,andwecallthe“measurementvalue”thenumberthatweobtainbymeasuringitthroughapropermeasurementprocess.Letusthendiscusshowtoobtainthemeasurementvaluex.

Therearetwomainwaysofdoingthat,directandindirect,illustrated,inthecaseofmassmeasurement,inFigure 5.2.Wefirstdiscusstheminadeterministiccontext,

Y116134.indb 108 10/10/11 2:11 PM


whereweassumethatallthedevicesinvolvedbehaveinaperfectlyrepeatableandreproducibleway,thenweintroduceamorerealisticprobabilisticmodel,capableofaccountingforthebehaviorofreal-worlddevicesthatmaypresentrandomvariationormaybeaffectedbysystematicdeviations.

ThefirstsimplyconsistsincomparingtheunknownobjectawiththeseriesofstandardStofindanelements∈Sequivalenttoa,s~a.ThisisshowninFigure 5.2a,forthemassexample.Letx=m(a)betheunknown(measure)valueofa.Onthebasisoftheresultofthecomparison,inasmuchaswehavefounda~s,weassigntoathemeasurementvaluex=m(s).

Notethatxisunknown,whereasm(s)isknown,becausesisanelementofthereferencescale,soweestimatexbyx=m(s).Intheidealcaseweobtain

x x= ,

thatis,themeasurementvalueequalsthemeasurevalue,whichinturnsatisfiesthedeterministicrepresentationtheorem.

Thesecondwaytomeasureistouseameasuringsystem,orinstrument,thathasbeencalibratedwithrespecttothereferencescale.Inourexample,Figure 5.2b,themeasuringsystemconsistsofalinearspring,orientedaccordingtothegravityfield.Thisdevicemaybecalibratedbyapplyingastandards0,whosevalueisx0=m(s0),andrecordingthecorrespondingdisplacementofthefreeendofthespring,y0.Then,byproportionality, theobjecta such thatm(a)=xwillproduceadisplacementysuchthat

yyxx kx= =0

0

,

wherekisthesensitivityoftheinstrument,whichmustbeestimatedbycalibration.3Wecallthisfunction,thatingeneralreads

(a) (b)

a s s0

OO

y0

y

y

a

Figure 5.2 Direct(a)versusindirect(b)measurementmethod.

Y116134.indb 109 10/10/11 2:11 PM


y f x= ( ),

the calibrationorobservation functionandweassume,bynow,thatitisaone-to-onefunctionandconsequentlythatitisinvertible(weremovethishypothesislateron,intheprobabilisticmodel).Weknowthecalibrationfunctionbecausewehavepreviouslycalibratedthespringdevice,thereforeafterobservingthedisplacementy,wemayassignthemeasurementvalue

x k y f y= = ( )− −1 1 .

Itiseasytocheckthatalsointhiscase

x x= ,

andthetwomeasurements,thedirectandtheindirectone,areconceptuallyequiva-lent.Wethuslookforacommondescriptionofthesetwomethods.

5.3.2 Observation and restitution

Wefirstdefineameasuringsystem(MS)asanempiricalsystemabletointeractwiththeobjectsthatcarrythecharacteristicunderinvestigationandtoproduce,asaresultof such interaction, signs, thatweeventuallycall (instrument) indications,on thebasisofwhichitispossibletoassignameasurementvaluetothecharacteristictobemeasured,inagreementwithapreviouslydefinedreferencescale.Thecharacteristictobemeasuredinagivensituationisusuallycalledthe“measurand”(seetheglos-saryattheendofthechapter)andwealsousethisterminthefollowing.

Inthefirstexampleoftheprevioussubsection,theMSismadebythereferencescaleitselfplusthecomparator,andinthesecondbythecalibratedspring.Weproposetoparsethemeasurementprocessintwosubprocesses, observationandrestitution.

In the observationphase the measurand is input to the measuring system thatinturnproducesanindication.IntherestitutionphasetheindicationoftheMSisinterpretedon thebasisof the calibration functionand themeasurementvalue isobtained.Measurementistheconcatenationoftheobservationandrestitution.Notethat the twophasesareconceptuallydistinct,becauseobservation iswhere infor-mation isproduced, thanks to the interactionbetweentheobjectandtheMSthatproducesrelevantphysicalorpsychologicaltransformations,whereasrestitutionisaninformation-processingphase.Restitutionmaybeverysimple,asinthecaseofdirectmeasurement,whereitjustconsistsinassigningtothemeasurandthesameknown value of the standard that has been selected, in the observation phase, asequivalenttothemeasurand,oritmaybeverycomplicatedandchallenging,asinthecaseof image-basedmeasurement,where itmay involvesophisticated image-processingprocedures.Nonethelessitisconceptuallyalwayspresent,inasmuchasthemeasurandis,byassumption,notdirectlyobservable(otherwisewewouldnot

Y116134.indb 110 10/10/11 2:11 PM


needtomeasure),whereastheindicationis,inthemostgeneralcase,asign,thatis,somethingobservable.Sothemeasurandandtheindicationaretwoinherentlydif-ferententitiesandinordertomakeastatementaboutthemeasurand,basedontheobservationoftheindication,aninterpretationprocessisrequired,whichwehavecalledrestitution.

Letusnowconsideradeterministicmodelofthemeasurementprocess.Aswehaveseenintheexamplesabove,observationmaybedescribedbythecalibrationorobservationfunction,

y f x= ( ), (5.5)

restitutionbyitsinversion,

x f y= ( )−1 , (5.6)

andmeasurementbytheconcatenationofthetwo,

x h x f f x x= ( ) = ( )[ ] =−1 , (5.7)

wherethemeasurementfunction,denotedbyh,inthisidealcasereducestoaniden-tity. It is easy to check that this frameworkholds inboth the cases thatwehaveconsideredintheprevioussection.Indeed,inthecaseofdirectcomparison,ifwemeasure the object a, whose (unknown) value is x = m(a), and we find that it isequivalenttos,weobtain

y f x m s m a x= ( ) = ( ) = ( ) = ,

ˆ .x f y x= ( ) =−1

Intheindirectmeasurementexample,instead,wehave

y f x kx= ( ) = ,

ˆ .x f y k y k kx x= ( ) = = =− − −1 1 1

Letusnowillustrategraphicallythegeneralideaofobservation/restitutioninasimplenumericalexample,inwhichy=f(x)=kx=2x,illustratedinFigure 5.3.Thisexampleisverysimple,eventrivial,butitisinstrumentalinintroducingtheproba-bilisticcase,whichisillustratedsimilarlyatalaterstage.

ThestructureofthemeasurementprocessisillustratedinFigure 5.4.Itisinter-estingtocompare thisgeneralschemeto thosewehavepresentedinChapter1,

Y116134.indb 111 10/10/11 2:11 PM


Figures 1.1–1.5. In the caseofdirectmeasurement (Figure 1.1), theobservationphaseconstitutesthecomparisonoftheobjectwiththereferencescaleandrestitu-tionissimplytheattributionofthevalueoftheselectedstandardtothemeasurand.Magnitudeestimation(Figure 1.5)maybeseenasakindofdirectmeasurement,with an internal reference scale. Indirect measurement occurs both in physics(Figure 1.2)andinpsychophysics(Figure 1.3).Hereobservationincludesthedirectmeasurementoftheinputmeasuredquantity(ies)oroftheinputstimulusandresti-tutionisareinterpretationoftheinputonthebasisofthephysicalorpsychophysi-cal law. Lastly, in measurement in psychometrics (Figure 1.4), and probably inmanyapplicationsofmeasurement in thebehavioral sciences thatusequestion-naires, test itemsplay the roleof theMS, theprocessofadministering them tosubjectsandcollectinganswersisanalogoustoobservation,andtheinterpretationoftheresponses,inordertoobtainameasurementvalue,issimilartorestitution.

So for the above considerations we may perhaps conclude that the proposedschemeofobservation/restitutionisquitegeneralandmayserveasreferencemodel.Useful references for a formal theory of the MS have been provided by Gonella(1988),Mari(2000),Morawski(1994),andRossi(2003).

(a) (b) (c)1 x2 3 4 5 6 7 8 9 10

2

4

68

10

12

14

16

18

20

1 x2 3 4 5 6 7 8 9

2

4

6

8

10

12

14

16

18

20

x10 x1 2 3 4 5 6 7 8 9 10

123456789

10

x = x

x

y = kx y = kx

y y

Figure 5.3 Thephasesofthemeasurementprocess:(a)observation;(b)restitution:incontrastwith(a)notethatheretheinputisyandtheoutputisx,asindicatedbythearrows;(c)measurement.

Object

Characteristic tobe measured

Restitution

Measurementvalue

Observation

Instrumentindication

Measuring system

Figure 5.4 Schemeofthemeasurementprocess.

Y116134.indb 112 10/10/11 2:11 PM


5.3.3 The probabilistic model

Thedeterministicmodeldescribesanidealenvironment.Ifwelookforamorereal-isticaccountofwhathappensinreality,wemayturntoaprobabilisticone.

Letus reconsiderobservationfirst. In contrastwith the idealmodel,whenweinputameasurandwhosevalueisx,andwerepeattheexperimentmoretimes,ingeneralwedonotalwaysobtainthesameindicationybutratheraclusterofindica-tions.Anappropriatewaytodescribethisbehavioristoassignaprobabilitydistribu-tiontotheindications,conditionedonthevalueofthemeasurandP(y∙x),asshowninFigure 5.5a.Intheabscissaswehavethepossiblevaluesxofthequantityunderconsideration;intheordinatestheinstrumentindicationsandthecorrespondingcon-ditionalprobabilityP(y∙x)arerepresentedbycircles:thelargecirclesindicateaprob-abilityequalto0.6,thesmallonescorrespondto0.2.Forexample,iftheinputvalueisx=6,wemayobtainthreeindications,whoseprobabilitiesare

P y x= =( ) =10 6 0 2. ,

P y x= =( ) =12 6 0 6. ,

P y x= =( ) =14 6 0 2. .

(a) (b) (c)x1 2 3 4 5 6 7 8 9 10

2

4

6

8

10

12

14

16

18

20

P(y|x)

P(y|x = 6)

x1 2 3 4 5 6 7 8 9 10

2

4

6

8

10

12

14

16

18

20

P(y|x)

P(x|y = 12)

x1 2 3 4 5 6 7 8 910123456789

10 ˆP(x|x)

ˆP(x|x = 6)

x

y y

Figure 5.5 Aprobabilisticmodelofthemeasurementprocess.Thecirclesrepresentprob-abilityvalues: the largecirclecorresponds to0.6, thesmall to0.2.Phasesof theprocess:(a)observation:notethatheretheinputisanyfixedvalueofx andtheoutputisacorrespond-ingprobabilitydistributionfory:theexampleofx =6isoutlined;(b)restitution:incontrastwith(a),notethatheretheinputisafixedvalueofyandtheoutputisacorrespondingprob-abilitydistributionforx:theexampleofy =12isoutlined;(c)measurement:heretheinputisafixedvalueforxandtheoutputaprobabilitydistributionforx.

Y116134.indb 113 10/10/11 2:11 PM


Thisconditionaldistributionmaystillbeobtained,at least inprinciple,byacalibra-tionexperimentanditreplacestheformercalibrationorobservationfunctiony=f(x).

Howcanwedorestitutionnow?Theidea,mutatis mutandis,isverysimilartothedeterministiccase.Supposethatweobservey=12:fromFigure 5.5bweseethatsuchanindicationmayhavebeencausedbythreevaluesofx,namely5,6,and7,andfromthesamegraphwemayobtaintheprobabilitiesofthesethreecauses,namely

P x y= =( ) =5 12 0 2. ,

P x y= =( ) =6 12 0 6. ,

P x y= =( ) =7 12 0 2. .

Whatwehavejustdoneisaprobabilistic inversionoftheobservationtransforma-tion.Analyticallythismaybeobtainedbyapplyingthewell-knownBayes–Laplacerule,asweshowinamoment.Again,asinthedeterministiccase,wemaycombineobservationandrestitutionforobtainingadescriptionoftheoverallmeasurementprocess,providedbythedistributionP(x∙x).Forexample,weseeinFigure 5.5cthat,ifx=6,themeasurementvaluexprovidedbythemeasurementprocessischaracter-izedbythedistribution

P x xˆ .= =( ) =5 6 0 2 ,

P x xˆ .= =( ) =6 6 0 6 ,

P x xˆ .= =( ) =7 6 0 2.

Wediscusshowthisdistributionmaybeobtainedinamoment.Letusnowformalizetheaboveconsideration.

Observationmaybemodeledby theconditionalprobabilitydistributionof theindications,giventhevalueofthemeasurand;thatis,

P y x( ) . (5.8)

RestitutionistheprobabilisticinversionofobservationanditcanbeobtainedbytheBayes–Laplacerule,as

P x yP y x P x

P y

P y x P x

P y x P xx

( ) = ( ) ( )( ) = ( ) ( )

( ) ( )∑.

Y116134.indb 114 10/10/11 2:11 PM


Iftheprobabilitydistributionfor xisassumedtobeuniform,thentheaboveexpres-sionsimplifiesas

P x yP y x

P y xP y x

x

( ) = ( )( )

∝ ( )∑

. (5.9)

Thisexpressionmaybeexplainedasfollows,havinginmindthepreviousexample.OurgoaldistributionP(x∙y)isthedistributionassignedtothemeasurandx,oncetheindicationyhasbeenobserved.SoP(x∙y)isafunctionofthevariablex,withybeingafixedvalue,anditmaybeobtainedbyinterpretingP(y∙x)asafunctionofx,insteadofasafunctionofy.ThiscorrespondstowhatwehavedoneinFigure 5.5b,wherewehavereadthegraphstartingfromtheordinates.Indoingso,itmaybethatthevaluesweselectinthiswaydonotsumtoone;inthatcasewehavetorescalethemtoensurethatthefinaldistributionsumstoone:

P x yx

( ) =∑ 1.

Soforeachindicationywemayassignaprobabilitydistributiontothemeasurandx.Todescribethemeasurand,asusual,byasinglevalue—themeasurementvalue—wetakeaproperpositionparameterfortherandomvariablex,conditionedbyy,thatwedenoteby

x x y= ( )µ .

Usually,foranintervalorratioscale,thisparameteristheexpected value

µ x y x y xP x yx

( ) = ( ) = ( )∑E ,

whereas for an ordinal scale the median should be used instead (Stevens, 1959).Uncertaintyfigures canalsobeobtained from thefinaldistributionP(x|y) (BIPM,2008a).Finally,wehavetocombineobservationandrestitutiontoobtainadescriptionoftheoverallmeasurementprocess.ThisoperationmaybeunderstoodbylookingatFigure 5.6.

Supposethatx=6;thenwemayobtainy=10,12,14,withprobabilities0.2,0.6,0.2,respectively.If,forexample,y=10,theresultingprobabilitydistributionforxwillbecenteredinx=5andthusthemeasurementvalue(theexpectedvalueofthedistribution)willbex=6.Thiswillhappenwiththesameprobabilityofobservingy=10,thatis,0.2.Analogously,ify=12,weobtainx=6,whichwillhappenwithprobability0.6,andify=14,wewillobtainx=7,whichwillhappenwithprobability0.2.Summarizing,wehave

P x xˆ .= =( ) =5 6 0 2,

P x xˆ .= =( ) =6 6 0 6,

Y116134.indb 115 10/10/11 2:11 PM


P x xˆ .= =( ) =7 6 0 2.

ThisisthewaythegraphofFigure 5.5chasbeenobtained.Thisproceduremaybeexpressedinaformula,

P x x x x y P y xy Y

ˆ ˆ( ) ( )[ ] ( )= −∈

∑δ µ , (5.10)

whereδheredenotesaunitarydiscreteimpulsefunction;thatis,foranyintegeri,δ(i)=1fori=0,andδ(i)=1,fori≠0.Letuspracticeapplyingformula(5.10)tocalculate,forexample,P(x=6∙x=6).Weobtain

P x P xx yˆ =( ) −[ ] =( )= = =6 6 7 146 6δ

+ −[ ] =( )=δ 6 6 12 6P xy

+ −[ ] =( )=δ 6 5 10 6P xy

= =( ) ==P xy 12 0 66 . .

TheprobabilisticdescriptionofthemeasurementprocessissummarizedinthelowerpartofTable 5.2.ForanintroductiontoBayesianinferencewerecommend

x1 2 3 4 5 6 7 8 9

2

4

6

8

10

12

14

16

18

20

P(y|x)

ˆP(x|x = 6)

x10

y

Figure 5.6 Howtoobtainadescriptionoftheoverallmeasurementprocess:seeexplanationinthetext.Thecirclesrepresentprobabilityvalues:thelargecirclecorrespondsto0.6,thesmallto0.2.

Y116134.indb 116 10/10/11 2:11 PM


Press(1989).AdditionalinformationonthemodelpresentedheremaybefoundinCox,Rossi,Harris,andForbes(2008)andinRossi(2003,2006,2009a),includingpossiblegeneralizationsconcerninghowtoaccount forsystematiceffectsand theextensiontovectoranddynamicmeasurements.

5.4 Measurability

Oncethemeaningofmeasurementhasbeenclarifiedbythenotionofmeasurementscaleandthewaywemeasurebytheconceptsofmeasuringsystemandmeasure-ment process, we may discuss a crucial point that has been, as we have seen inChapter1, amatterof strongcontroversy, the issueofmeasurability.There isnoroomhereforathoroughdiscussionofthistopic,andwejustpropose,onthebasisofwhatwehavethusfardiscussed,acriterionformeasurability.Wesuggestthatacharacteristic x ofaclassofobjectsismeasurableifthefollowingfour-stepproce-duremaybesuccessfullyapplied:

1.Definetheclass of objectsthatmanifestthecharacteristic. 2. Identifytheempirical propertiesthatdefinethecharacteristic. 3.Constructareference measurement scale. 4.Deviseatleastonemeasuring systembasedonthatreferencescale.

Simpleasitmayseem,thisprocedureinvolvesverydemandingstepstobetaken;seeRossi(2007)foradiscussion.Herewesimplynotethat,atfirstglance,itmayseem that this procedure applies to fundamentalmeasurement only, according toCampbell’sclassification(Campbell,1957/1920).AswehaveseeninChapter1,forCampbellfundamentalquantitiesareonlythoseforwhichitispossibletoconstructareferencescale,thankstothe“physicaladdition”operation,andthatmaybemea-suredbydirectcomparisonwiththatscale.Ontheotherhand,derivedquantitiesarethosethatmaybemeasuredindirectly,thankstosomephysicallawthatrelatesthemtoother,independentlymeasurable,quantities.

Wemayretainthisclassificationtosomeextent,providedthatwegeneralizeandupdateitonthebasisofrecentirreversibleresultsoftherepresentationaltheory.Sowecall fundamental ascalethatmaybeconstructedonthebasisof the internal properties (relationsandoperations)of thequantityunderconsideration.Wecallderiveda scaleobtainedon thebasisof interrelations linking thequantityunderconsiderationwithotherquantities.Forexample,ascaleformass,constructedbygeneratingthemultiplesandsubmultiplesofaninitial,arbitrary,unitaryelement,byanequal-armbalance,andbyassigningvaluestotheminaccordancewiththeadditivityproperty,maybeconsideredafundamentalscale.Instead,atemperaturescalebasedonagraduatedmercury-in-glassthermometermaybeconsideredderivedfromlengthmeasurement,throughthepropertyofthermaldilatationoffluids.Infact,thispropertyisnot“internal”totemperature(aswouldbethe“warmerthan”orderrelation)butratherrelatestwodistinctquantities,temperatureandlength.

So,whatcanwesay,inafewwords,aboutthemeasurabilityofderivedquantities?

Y116134.indb 117 10/10/11 2:11 PM


Basically,inthecaseofaderivedquantityz,theempiricalpropertiesrequiredforconstructingthescalearederivedfromthepropertiesofanotherquantityxtowhichzislinkedthroughascientificlaw.Butwhatisascientificlaw?

Wemayperhapsdefineit,forthescopeofthisdiscussion,asafunctionalrelationbetweenvariables,say,inourcase,z=g(x), that isacceptedasholdingtrue, inagivendomainofknowledgeandinagivenhistoricmoment.

Now,itmayhappenthatthelawghasbeendetermined,or,atleast,verified,aftermeasuringindependentlyxandz.Inthiscase,wemaystillwishtomeasurex indi-rectlythroughg,forpracticalconvenience,butthefunctionalrelationdoesnothave,inthiscase,afoundationalrole.Forexample,havingpreviouslydefinedtemperaturethroughthermodynamicprinciples(ashappensnowfortheso-calledthermodynamictemperature),wemaydeterminethelawthatlinkstemperaturetotheheightofthemercurycolumninaliquid-in-glassmercurythermometer,realizethatit ishighlylinear,anddecidetomaketemperaturemeasurementbythattypeofthermometer.Completelydifferentisthecaseinwhichwedefinetemperaturebyassumingalin-earrelationbetweenitandtheheightofthemercurycolumn.Inthissecondcase,temperatureisintended,bydefinition,asthatpropertyofabodygivingrisetolinearvariationsofheightinthecolumnofamercurythermometer,whenproperlycoupledwithit(detailsareinessentialhere).Now,inacertainstageofscientificdevelopment,oneofthesetwosolutionsmaybepreferred,inconsiderationoftheoverallbodyofknowledgeinvolved(physics, in thiscase),butbothare, inouropinion,conceptu-allycorrect.

Thus—andthisisakeypoint—afunctionalrelationmaybeusedtofoundthemeasurabilityofsomequantity,bydefiningit,providedthattherearegoodreasonsforacceptingthatrelation.

Atthisstage,asecondproblememerges:whataboutthemeaningofthenumberweassociatewithz, ifempiricalpropertieshavenotbeendirectlyinvolvedinthedefinitionofthescaleofz?Forexample,inthecaseoftemperaturebeingdefinedthroughthemercurythermometer,whatmeaningwouldweassociatewiththeresult-ingnumbers?

Therepresentationaltheoryforderivedmeasurementisnotsoadvancedastheoneconcerningfundamentalmeasurement,yetusefulresultsareavailable.Basically,empiricalrelationsinvolvingbothquantitiescomeintoplay.Intuitively,inourexam-ple,werequirefirstthatthereisanorderbothontemperatureandonheight,andthen that such orders are in concordance. That is to say that if, for example, thetemperatureofaisgreaterthanthatofb,thentheheightofthemercurycolumnofthethermometerincreasesaccordingly.Ifthishappens,wemayderiveanorderscalefortemperature.Ifwelookforanintervalscale,weshouldalsorequirethatequiva-lenttemperatureintervalscorrespondtoequivalentheightintervalsandthatadjacenttemperatureintervalscorrespondtoadjacentheightintervals.

Thereforeasa tentativeconclusion,wemaysay that in thecaseof fundamen-talscalesweconstructthescaleonthebasisofempiricalrelationsinternaltothecharacteristic under consideration, whereas in the case of derived scales we takeadvantageof cross-relationswithother characteristics.Thismaybemore criticalbecausetheempiricalstructureinvolvedismorecomplex,butthebasicideasremain

Y116134.indb 118 10/10/11 2:11 PM


thesame.Sothemeasurabilitycriterionthatwehaveproposedmaystillbeapplied,withproperadaptation.

Measurability has been recently discussed by Finkelstein (2003, 2005, 2008)andbyRossi(2007).MeasurabilityissuesarealsotreatedbyEllis(1966),whoalsoprovidesaninterestingdiscussionoftemperaturemeasurement.AnintroductiontoderivedmeasurementsmaybefoundinRoberts(1979/2009);animportantapplica-tionoftheseideasconcernsmagnitudeestimation,onwhichNarenshaspublishedathoroughstudy(1996).

5.5 Final remarks

Wehavepresentedabasickernelforaprobabilistictheoryofmeasurement,address-ingthreekeyquestions,namelywhatisthemeaningofmeasurement,howdowemeasure,andwhatcanbemeasured?

Wesuggestthattheresponsetothefirstquestionmaybefoundinatheoryofthemeasurementscale,tothesecondinageneralmodelofthemeasurementprocess,andtothethirdinaconceptuallysimplemeasurabilitycriterion.Furtherstudiesarerequiredinmanyregards,includingtheproblemofderivedscales,adeepercharac-terizationofthemeasurementprocess,andtheextensiontomultidimensionalmea-surement,tonamejustafew.

Indevelopingthistheorywehavecarefullyavoidedlinkingittospecialclassesofmeasurementsortospecificmeasuringtechniquesortechnologies.Thuswehopethatitmayconstituteagoodstartingpointforachievingacommonviewofmeasure-mentamongdifferentscientificdisciplines.

Glossary

Some terms in this glossary are also defined in the International Vocabulary ofMetrology(VIM;BIPM,2008b).In thosecases, thedefinitionsprovidedhereareinessentialagreementwiththoseintheVIM,buttheyaredifferentlyphrased,inordertomakethemmoresuitedforafoundationaldiscussionand,itishoped,moreunderstandabletoamultidisciplinaryreadership.

Object:thecarrierofthecharacteristictobemeasured;itmaybeaphysicalobject,anevent,oraperson

Characteristic or property (of an object):whatwewanttomeasureMeasurable characteristic (or quantity):acharacteristicthatmaybemeasuredState (of an object, with respect to a given characteristic):particularwayinwhich

anobjectmanifestsacharacteristicEmpirical relation:arelationthatmaybeobservedbetweentwoormoreobjects,

withreferencetothecharacteristicofinterestComparator: a device (or a person) that performs the comparison of objects, in

respectofsomecharacteristicoftheirsEmpirical structure (or empirical relational system):asetofobjectsandasetof

empiricalrelationsonit

Y116134.indb 119 10/10/11 2:11 PM


Nominal, difference, interval, extensive:differenttypesofempiricalstructuresNumerical structure (or numerical relational system):asetofnumbersandaset

ofnumericalrelationsonitScale:(generalmeaning)thesetofformalconditionsformeasurement(anempiri-

calstructure,anumericalstructureandameasurefunctionconstitutinganhomomorphismbetweenthem)

(Reference) scale:(specificmeaning)aseriesofstandardobjectswithcorrespond-ingnumericalvaluesproperlyassigned

Nominal, ordinal, interval, ratio:differenttypesofscalesMeasurand:acharacteristicofaspecificobjecttobemeasured,inaspecificsituationMeasuring system (or instrument): an empirical system capable of interacting

withobjectscarryingthecharacteristicunderinvestigationand,asaresultofsuchinteraction,ofproducinganobservableoutputaccordingtowhichitispossibletoassignavaluetotheobjecttobemeasured

Calibration:theoperationbywhichthecharacteristicsofthemeasuringsystemareassessed

Measurement process:theprocessbywhichavalueisassignedtoameasurand,normallybasedontheuseofameasuringsystem

(Measure) value:anumberthatmaybeactuallyorvirtuallyassignedtoanobjectinordertoexpresshowtheobjectcompares,withrespecttosomecharacteris-tic,toalltheotherobjectsthatcarrythesamecharacteristic

Measurement value:thevaluethatisassignedtoanobjectastheresultofamea-surementprocess

Model:anabstractsystemthat,tosomeextentandfromacertainstandpoint,rep-resentsarealsystem(oraclassofrealsystems);ascientifictheorymaybesometimesviewedasaverygeneralmodel

(Natural) law:afunctionalrelation(oramodel)linkingoneormorecharacteristicsofrealobjects

Measurement model:amodelusedtofoundthemeasurabilityofcertaincharacter-isticsortoperformmeasurements

Notation

x = characteristic(quantity)tobemeasured,measurand,unknown(measure)valueofthemeasurand

A = setofobjectsmanifestingthecharacteristicx

a, b, c, d = objects,elementsofA

m = measurefunction,m :A→ℝΔab = intervalbetweenelementsaandbofA;itis“positive”ifa≻b

⋟ = empiricalrelationof“greaterthanorequivalentto;”itisaweak-orderrelation;forsimplicity,weusethesamesymbolfordenotinganorderbetweenobjects,e.g.,a⋟b,orbetweenintervals,e.g.,Δab⋟Δcd

⋟i = ithweak-orderrelationdefinedonA

Y116134.indb 120 10/10/11 2:11 PM


~ = “equivalentto,”empiricalequivalencerelationonA defined,fora,b∈Abya~b⇔a⋟bandb⋟a

≻ = “greaterthan,”empiricalrelationofstrictorder

° = binaryempiricaloperationofadditionofelementsofA;itmayalsobeconsideredaternaryrelationonAdefined,fora,b,c∈Aby°(a,b,c)⇔a°b=c

δ = thresholdofacomparator(seebelowforanothermeaningofthesamesymbol)

~δ = indifferencerelationassociatedtoacomparatorwithathreshold;itdiffersfromanequivalencerelationinthatitisnottransitive

S=s1,s2,…sn = seriesofstandards

ℙ = probabilityofarelation

P = probabilityfunction,probabilitydistribution

xa,xb,xc = randomvariablesassociatedtoelementsa,b,c,expressingtheirmeasurevalues

P(x),P(x=xi),P(x≤y) = probabilitydistributionfortherandomvariablex,probabilitythattherandomvariablextakesthespecificvaluexiprobabilitythattherandomvariablexislessthanorequaltorandomvariabley

y = instrumentindication,outputofthemeasuringsystem

P(y/x),P(y=yj /x=xi) = probabilitydistributionofy,conditionedbyx;probabilitythatytakesthevalueyj,giventhatxhastakenthevaluexi

f = calibrationorobservationfunctionofameasuringsystem

x = measurementvalueforx

δ = discreteunitaryimpulsefunction:foranyintegeri,δ(i)=1fori=0,whereasδ(i)=0,fori≠0.

g = functionlinkingtwoquantities,z=g(x),andexpressingascientificlaw

References

BIPM.(2008a).Evaluation of measurement data—Guide to the expression of uncertainty in measurement. (JCGM100:2008).

BIPM. (2008b). International vocabulary of metrology—Basic and general terms (VIM).(JCGM200:2008).

Campbell,N.R.(1957).Foundations of science.NewYork:Dover.(Originalwork,Physics—the elements,published1920).

Cox,M.G.,Rossi,G.B.,Harris,P.M.,&Forbes,A.(2008).Aprobabilisticapproachtotheanalysisofmeasurementprocesses,Metrologia, 45,493–502.

Ellis, B. (1966). Basic concepts of measurement. Cambridge, UK: Cambridge UniversityPress.

Falmagne, J. C. (1980). A probabilistic theory of extensive measurement. Philosophy of Science, 47,277–296.

Finkelstein, L. (1982).Theory and philosophy of measurement. In P. H. Sydenham (Ed.),Handbook of measurement science(Vol.1,pp.1–30).Chichester:Wiley.

Y116134.indb 121 10/10/11 2:11 PM


Finkelstein,L.(2003).Widely,stronglyandweaklydefinedmeasurement,Measurement, 34,39–48.

Finkelstein,L.(2005).Problemsofmeasurementinsoftsystems.Measurement, 38,267–274.Finkelstein,L.(2008,September).Problemsofwidely-definedmeasurement.Paperpresented

at the 12th IMEKO TC1 and TC7 Symposium on Man, Science and Measurement,Annecy,France.

Finkelstein,L.,&Leaning,M.S.(1984).Areviewofthefundamentalconceptsofmeasure-ment.Measurement, 2,25–34.

Gonella,L.(1988).Measuringinstrumentsandtheoryofmeasurement.PaperpresentedattheXI IMEKO World Congress,Houston.

Helmholtz,H.(1971).Anepistemologicalanalysisofcountingandmeasurement.InR.Karl(Ed.andTrans.),Selected writing of Hermann Helmholtz.Middletown,CT:WesleyanUniversityPress.(Originalworkpublished1887)

KrantzD.H.,Luce,R.D.,Suppes,P.,&TverskyA. (1971).Foundations of measurement (Vol.1).NewYork:AcademicPress.

Leaning,M.S.,&Finkelstein,L. (1980).Aprobabilistic treatmentofmeasurementuncer-tainty in theformal theoryofmeasurement. InG.Streker (Ed.),ACTA IMEKO 1979(pp.73–81).Amsterdam:Elsevier.

Luce,R.D.,&Suppes,P.(2002).Representationalmeasurementtheory.InStevens’handbook of experimental psychophysics(Vol.4).NewYork:Wiley.

Luce,R.D.,Krantz,D.H.,Suppes,P.,&Tversky,A.(1990).Foundations of measurement (Vol.3).NewYork:AcademicPress.

Mari,L.(2000).Beyondtherepresentationalviewpoint:Anewformalizationofmeasurement.Measurement, 27,71–84.

Monti,C.M.,&Pierobon,G.(2000).Teoria della probabilità.Bologna:Zanichelli.Morawski,R.Z.(1994).Unifiedapproachtomeasurandreconstruction.IEEE Transactions on

Instrumentation & Measurement, 43,226–231.Narens, L. (1996). A theory of ratio magnitude estimation, Journal of Mathematical

Psychology, 40,109–129.Papoulis, A. (1984). Probability, random variables and stochastic processes (2nd ed.).

Singapore:McGraw-Hill.Press,S.J.(1989).Bayesian statistics.NewYork:Wiley.Regenwetter,M.(1996).Randomutilityrepresentationsoffinitem-aryrelations.Journal of

Mathematical Psychology, 40,219–234.Regenwetter,M.,&Marley,A. J. (2001). Random relations, randomutilities, and random

functions.Journal of Mathematical Psychology, 45, 864–912.Roberts,F.S.(1979).Measurement theory, with applications to decision-making, utility and the


Rossi, G. B. (2003).A probabilistic model for measurement processes. Measurement, 34,85–99.

Rossi,G.B.(2006).Aprobabilistictheoryofmeasurement.Measurement, 39,34–50.Rossi,G.B.(2007).Measurability.Measurement, 40,545–562.Rossi,G.B.(2009a).Probabilityinmetrology.InF.Pavese,&A.Forbes(Eds.),Data model-

ing for metrology and testing in measurement science.Boston:Birkhauser–Springer.RossiG.B.(2009b).Cross–disciplinaryconceptsandtermsinmeasurement.Measurement,

42,1288–1296.

Y116134.indb 122 10/10/11 2:11 PM


Stevens, S. S. (1959). Measurement, psychophysics and utility. In C. W. Churchman &P. Ratoosh (Eds.), Basic concepts of measurements (pp. 1–49). Cambridge, UK:CambridgeUniversityPress.

Suppes,P.,Krantz,D.H.,Luce,R.D.,&Tversky,A.(1989).Foundations of measurement(Vol.2). NewYork:AcademicPress.

Notes

1. Aweak-orderrelation,a⋟b,onasetA,issuchthatitisdefinedforeachpairofele-mentsofAanditistransitive;thatis,ifa⋟bandb⋟c,thenalsoa⋟c.Forthemean-ingofthemathematicalsymbols,seetheGlossaryandNotationsectionsattheendofthechapter.

2. Anequivalence classisasetofelementsallequivalenttooneanother.ThecollectionoftheequivalenceclassesofasetAformsapartitionforA:thismeansthatanysuchclasssharesnoelementwiththeothersandthattheseclasses,alltogether,coverA;thatis,theyincludealltheelementsofA.

3. Calibrationistheprocessofexperimentallycharacterizingthebehaviorofameasuringsystem.Itisusuallydonebyinputtingtheinstrumentwithreferencestandards,whosevaluesareknown,andbyrecordingtheresultinginstrumentindications.Theinstrumentbehaviormay thusbecharacterizedby thecalibrationcurve, that is, thegraphof theindicationsversustheinputvaluesofthestandards,orbysomeanalyticalcurvefittedtothem.

Y116134.indb 123 10/10/11 2:11 PM

Y116134.indb 124 10/10/11 2:11 PM

125

Multivariate measurements

Gerie W. A. M. van der Heijden1 and Ragne Emardson2

1Biometris,WageningenUniversity&ResearchCentreWageningen,TheNetherlands2SPTechnicalResearchInstituteofSwedenBorås,Sweden

6.1 Introduction

Measurementswithpersonsofteninvolvealargenumberofvariablesthathavetobeanalyzedwithstatisticalmethods.Thischaptergivesashortoverviewofvari-ous methods that can be used. The aim is to give readers a basic understandingof the usefulness of and relations between the different methods in a condensedandaccessibleform.Wereferinterestedreaderstonumerousmultivariatestatisticalhandbooksforamoreextensiveandin-depthtreatmentoftheseveralmethods(e.g.,Mardia,Kent,&Bibby,1979;Johnson&Wichern,2002;Härdle&Simar,2007;Tabachnick&Fidell,2007).

6.1.1 Dataset

InthischapterweuseoneexampledatasetoftheEU-projectMONAT(Measure-ment of Naturalness). The aim of the MONAT project is to establish a relation-shipbetween thephysical attributesof anobject and itsperceivednaturalness asassessedbyhumans.Thedatasetweuseconsistsofalimitedsetof30samplesofpanels, representing wood in various degrees of naturalness. An overview of thesamples is given in Figure 6.1. The samples are scored by human observers andaftersomeprocessingthesescoresaretranslatedtoaprobabilityscore(0–1range)fora(possiblyfake)woodenpanelbeingperceivedasnatural.Thisisdoneforthreedifferent perceptual modalities: (1) only by vision (VO), (2) only by touching itwithafinger (TO),and (3)bybothvisionand touch (VT).For furtherdetailsonthepsychophysicalaspectsof thestudy,see, forexample,Overvliet,Soto-Faraco,Whitaker,JohnstoneSorensen,McGlone,andvanderHeijden(2008).Furthermore,physical attributes were measured for all the samples using different instruments(Goodman,Montgomery,Bialek,Forbes,Rides,Whitakeretal.,2008).Withtheseinstruments,vision-relatedattributes suchas features for texture,color, andgloss

6

Y116134.indb 125 10/10/11 2:11 PM


and touch-related attributes such as features for roughness, friction, and thermaleffusivityweremeasured.The30samplesareshowninFigure 6.1.

6.1.2 Correlation and causality

Inmanycasesanapparentcorrelationbetweentwovariablesmaybetheresultofarelationshipbetweenbothofthesetwovariablesanda(possiblyunknown)thirdvariable.Forexample,inmostEuropeancountriesthereisacorrelationovertimebetweenfemalesalariesandmaleaverageheight.Thisdoesnotimplythatthetallerthemenget,themorethewomenarepaidorviceversa.Inthiscasebothparametershaveincreasedwithtimeandthusacorrelationisfoundbetweenthem.Hence,evenifastatisticalcorrelationisfoundbetweentwovariables,itdoesnotalwaysimplythat there is a (direct) causal relationshipbetween them.Furthermore even if thevariablesAandBarerelateditisnotpossiblefromacorrelationanalysistodeter-mineifAcausesBorifBcausesA.

Incontrolledexperiments,alimitednumberoffactorsaresystematicallyvariedandotherfactorsarekeptconstantasfaraspossibleorotherwiserandomized.Itis

Figure 6.1 (See color insert.) Anoverviewofthe30woodsamplesusedintheMONATstudy.Thefirst two rowsare realwood.Rowonecontains tigeroakpanels,ofwhich thethreeleftpanelsareuntreated(raw,weathered,andsanded,respectively)andtheotherfourarewaxed,oiled,varnished,andmanufactured,respectively.Thesecondrowisthesameforcognacoak.Thebottom three rowscontainartificialwoodpanels (3 laminates,1veneer,9vinyl,3papercopy,respectively).

Y116134.indb 126 10/10/11 2:11 PM

Multivariate measurements 127

thenpossibletostudytheeffectdirectlyofacertainfactoronthevariableofinter-estandcausalitycanoftenbeinferredfromsuchanexperiment.However,inmanystudies,wecannotsimplychangeafactorandkeepallotherfactorsconstant.IntheMONATwoodsampleset,forexample,avarietyofwoodenpanelswaschosen,butthesepanelsdidnotsystematicallyvaryforasingleattributesuchas forcolororroughness.Suchastudyisoftenreferredtoasanobservationalstudy.Inobserva-tionalstudies,weneedtobemuchmorecarefulaboutconclusionswecandraw,aseffectscouldbecausedbyotherunknowncorrelatedfactors.

In this chapter,wepresent some tools andmethods for analyzingmultivariatedatasets.Anassumptionoftenmadeintheanalysisofdatais that thedatacomefrom a (multivariate) normal distribution. Therefore we first introduce the multi-variatenormaldistributionasitisneededinseveralofthetechniquesappliedlater.Thisisfollowedbyprincipalcomponentanalysiswhichisoftenusedasafirsttooltofindsystematicpatternsamongobservations.Thelatersectionsdescribeseveralmethods,includingmultivariatelinearregression,discriminantanalysis,andcluster-ingmethods.

6.1.3 Multivariate normal distribution

Arandomvariablecanbedescribedusingdifferentproperties.Generally,themostimportantpropertiesforarandomvariableareitsexpectationvalueandvariance.Givenmeasurementsofarandomvariableitispossibletoformestimatorsoftheseproperties.Theseestimatorscanbeofdifferentquality.

Indeterminingagoodestimator,itisnormallyhelpfultomathematicallymodelthedata.Onecommonapproachistospecifyaprobabilitydensityfunction(PDF)ofarandomvariable.ThePDFisafunctionwhich,foreachpointinthesamplespace,describes the density of probability that the random variable attains that specificvalue.TheprobabilityofarandomvariablefallingwithinagivensetisgivenbytheintegralofitsPDFovertheset.

Inthischapter,wefocusprimarilyonrandomvariableswithPDFscorrespondingtoanormaldistribution.Becauseofitsmanyconvenientproperties,randomvari-ableswithunknowndistributionsareoftenassumedtobenormal.Althoughpoten-tiallywrong,thisisinmanycasesagoodassumptionbecauseanyvariablethatisthesumofalargenumberofindependentfactorsislikelytobenormallydistributed.Theprobabilitydensityfunctionforarandomvectorvariablexthathasa(multivari-ate)normaldistributioncanbewrittenas

f x epx xT

( ) / /=( ) ∑

− −( ) ∑ −( )−1

2 2 1 221

πµ µ ,

where p is the number of variables (dimensionality), μ is the expectation vector,andΣisthecovariancematrixofx.Forascalarvariable(p=1)thisreducestothefamiliarexpression:

Y116134.indb 127 10/10/11 2:11 PM


f x e x( ) = − −( ) 1

2 2

22

πσµ σ .

Anexampleofthetwo-dimensional(bivariate)normalPDFisshowninFigure 6.2.Forarandomvariablexwithanormaldistribution,theprobabilitythatthedis-

tancebetweenavectorsamplexsanditsexpectationvalueμissmallerthanorequaltoaspecificvalueχ2

p(α)is1–α;thatis,

P x xsT

s p−( ) ∑ −( ) ≤ ( ) = −−µ µ χ α α1 2 1 ,

where χ2p(α) is the upper (100α)th percentile of a chi-square distribution with p

degreesoffreedom(see,e.g.,Johnson&Wichern,2002).Inastrictsense,weshoulddistinguishbetweentherandomvariable,sayx,andits

realization,saythesamplevectorxs.However,forreadabilitywedropthesubscriptsandrefertoxbothfortherandomvectoranditsrealization.

Generally,wedonotknowtheexpectationμandthecovarianceΣandweesti-matethesefromthedata.GivenaspecificPDFforarandomvectorvariable,wecan,forexample,findamaximumlikelihoodestimator(MLE),whichis theestimatorof thepropertyof therandomvariablewith thehighestprobabilitygivenasetofmeasurements.Thisisnotnecessarilythebestunbiasedestimatorforthatproperty,butwhenthedatasetislargeenoughittendstothetruevalue(itisanasymptoticallyunbiasedestimator).

Whenwehavennumberofobservations,wecanestimateμasx,thevectorofmeanvaluesofthepvariableswhere

xn

xk ki

i

n

==

∑1

1

fork=1,…,p.

WecanfindanunbiasedestimatorofΣbythesamplecovariancematrixSwitheachelementsklinSas

Y-variate

X-variate

Bivariate Normal Distribution

Figure 6.2 Anexampleofabivariatenormaldistribution.

Y116134.indb 128 10/10/11 2:11 PM


sn

x x x xkl ki k li l

i

n

=−

− −=

∑11

1

( )( ) .

Given the data, we can construct the multidimensional confidence interval for μusingthefollowingprobability.

P n x S xp nn p

FT

p n p−( ) −( ) ≤ −−

( )

=−

−µ µ α1 11

( ), −− α ,

whereFp,n-p(α)istheupper(100α)thpercentileoftheF-distributionwithpandn – pdegreesoffreedom.Theregiondefinedbytheaboveequationisa100(1–α)%con-fidenceregionforμ.

UsingtheMONATdatasetdescribedabove,xcan,forexample,bethescoresofhumanobserverson the threedifferentperceptualmodalities, visiononly, tactileonly,andbybothvisionandtouchforonespecificwoodsample.Hence,inthatcasepequalsthreeandμistheexpectedscorethatanyobserverfromapopulationwouldproduceforthethreemodalities.

6.1.4 Principal component analysis

Principalcomponentanalysis(PCA)isauseful tool forextractinganddisplayingsystematicvariationsinadatamatrixX.ThemainideabehindPCAistoprojectthedatamatrixX,containingnmeasurementsofpvariables,ontoanewspacewithusu-allyalowerdimension,n×k.ThisisachievedbyrewritingXas

X TA ET= + ,

wherethematrixEisthemodelmisfit.ThematricesTandAarefoundbydeter-miningtheeigenvaluesofX.Hence,thematrixAischosenastheeigenvectorsofS,thesamplecovariancematrixofX.Thatis,Aconsistsofthevectorsa1,a2,…,aksatisfyingtheequations

Sa an n n= λ ,

andTisfoundfrom

T X A= ⋅ .

Inorder to reduce thenumberofvariables,wecanchoose to formA of thekfirsteigenvectorscorrespondingtotheklargesteigenvalues.StudyingtheMONAT

Y116134.indb 129 10/10/11 2:11 PM


dataset,wecanusePCAtoreducethedimensionsofthemeasuredparameters.Themeasurements of the physical attributes corresponding to visual features contain58differentvariables.FromaPCA,wefindthatapproximately70%ofthevariationscanbefoundinthefirstthreeprincipalcomponentscorrespondingtothethreelarg-esteigenvalues.Wecanthusreducethematrixfrom58to3dimensionsandillustratethemeasurementsforthe30differentobjectsina3-dimensionalspaceinsteadofin58dimensions(seeFigure 6.3).

PCAisaveryusefultoolinmanymultivariateapplications,whichweshowlater.Ithelpsindealingwithcollinearityandhelpsseparatetheregularitiesfromthenoise.Thesefeaturesareusefulbothinclassificationandinmultivariateregression.

6.2 Regression

6.2.1 Linear regression

Inmanycaseswewanttopredictorexplainaresponsevariableyfromasetofpredictor(orregressor)variablesx1,x2,…,xp.Usuallythegoalistofindarelationshipbetweenaresponsevariablethatisverytimeconsumingtoobtainandpredictorvariablesthatarerelativelyeasytomeasure.Oftenwecanassumethattherelationbetweenyandx1,x2,…, xPislinear.Thenwecanformulatethefollowinglinearregressionmodel:

y x xp p= + + + +β β β ε0 1 1 ,

wherethevectorεistherandomerrorcomponent.Generallytheerrorsareassumedtohavemeanzeroandunknownvarianceσ2.Furthermoretheerrorsareassumedtobeuncorrelated.

First Principal Component

–6–15 –10

–5 0 5 10 –50

510

15

–4

–2

0

2

4

6

ird P

rinci

pal C

ompo

nent

Second Principal Component

Figure 6.3 MONATvisualfeaturesreducedfrom58to3dimensions.

Y116134.indb 130 10/10/11 2:11 PM


Moreconvenientlysuchamodelcanbewritteninmatrixnotationas

y X= +β ε ,

wherethematrixXisthecombinationofthevectors1,x1,x2,…,xpandwhereβ=(β0,β1,…,βp)Tisthevectorofcoefficientsand1indicatesavectorofones.

Ifwehaveasetofnobservationsy=(y1,y2,…,yi,…,yn)TandcorrespondingX,wecanestimatethecoefficientsβusingtheleastsquaresmethod,thatis,bytryingtominimizethesumofsquaredresiduals

RSS y Xi i

i

n

= −[ ]=

∑ β 2

1

.

Thesolutionis

ˆ ( )β = −X X X yT T1 .

Wecanfurthergeneralize this linear regressionbyconsideringmultiple responsevariablessimultaneouslyandwritetherelationas

Y XB E= + ,

whereYandEarematrices.Inanalogywithasingleresponsevariable,wecanfindanestimatorforBusinggenerallinearregressionas

B X X X YT T= ( )−1.

IftheerrormatrixEismultivariatenormaldistributedwithavectorofvariancesσ2

Eanduncorrelatedbetweenthevariables,theleastsquaresestimatorofBisalsonormallydistributedanditisalsotheMLEofB.UsingB,itisnowpossibletopredictnewsetsofobservations.ChoosingnewarbitrarysetsofmeasurementsX0wecanpredicttheresponsevariables:

Y X Bpred = 0ˆ .

Theerrorof thepredictionis thenthedifferencebetweentheprediction,Ypred,andthetruevalueY0;thatis, Ypred– Y0.Normally,wedonotknowthesizeoftheerrorbecauseY0isnotknowntous.Wecan,however,calculatethevarianceofthiserroras

Var Y Y X X X Xpred ET T− = + ( )( )−

02

01

01σ ,

whichwecanusetoevaluatethemeasurementuncertaintyofYpred.

Y116134.indb 131 10/10/11 2:11 PM


Usingtheexamplewith theMONATdataset,wewant toestablish therelationbetweenphysicalattributesofanobjectanditsperceivednaturalness.HenceYcon-tains thevaluesof theperceivednaturalnessandXcontains themeasurementsofthephysicalattributes.FindingarelationbetweentheseXandY(i.e.,anestimateofB)isthenusefulforpredictingperceivednaturalnessofobjectsbasedonphysicalmeasurementonly.

6.2.2 Prediction and validation

Ifwewanttofita(possiblynonlinear)regressionmodeltoasingleresponsevariableywecanwritethemodel:yi=f(Xi,β)+εi,fori=1,…,nobservations,wheref(X,β)indicatesanyfunctionofregressormatrixXandcoefficientsβ.Inanalogywiththelinear regression case, we can estimate the coefficients β asβ by using the leastsquaresmethod,thatis,minimizethesumofsquaredresiduals:

RSS y f Xi i

i

n

= −[ ]=

∑ ( , )β 2

1

.

Themeansquarederror(MSE)ofthemodelcanbecalculatedas

MSE y f X ni i

i

n

= − ( )

=∑ ,β

2

1

andcanbeseenasanestimateoftheerrorrateofthemodel.Oftenweareinterestedinthegeneralapplicabilityofthemodelandwewould

liketoknowhowwellthemodelwillperformwithotherdata;thatis,weareinter-estedintheerrorrateofthemodelinamoregeneralsense.

Ifwehadaccesstoanunlimitednumberofobservations,wecouldfitthemodelontheentirepopulationand,ofcourse,theMSEwouldthenbethetrueerrorrate.Inrealapplicationsweonlyhaveaccesstoafinitesetofobservations.OneapproachtoobtainanestimateoftheerrorrateofthemodelistousetheentiredatasettofitthemodelandestimatetheerrorrateasMSE.However,thiserrorrateestimatewillbeoverlyoptimistic(lowerthanthetrueerrorrate).

Awaytosolvethisproblemistosplitthedatasetintwoparts:onefortrainingthemodelandone for testing it.The testingof themodel isoften referred toasmodelvalidation.IfwehaveanobservationyjatXj,wecangetanimpressionofthelackoffitofthemodelforobservationjbylookingattheresidual,thatis,thedifferencebetweenthetrueobservationanditspredictedvalue:rj=yj–yj,whereyj=f(Xj,β).

Inanalogywiththeabove,wecannowcalculateameansquarederrorofpredic-tion(MSEP)formtestobservationsas

Y116134.indb 132 10/10/11 2:11 PM


MSEP y y mj j

j

m

= − =

∑ ˆ2

1

.

Ifthedatasetislargeenough,wecan,forexample,splitthedataset50%fortrainingand50%fortestingandhenceestimatetheerrorrateonthetestingsetasMSEP.

Inmanycases,wecannotafford todiscarddatafor themodelfitting:wemayneedmoresamples tofit themodelproperlyaswellas toestimate theMSEP. Insuchacase,wecanusecross-validationbyrepeatedlydividingthedatainadisjointtrainingsetandtestset.OnepopularapproachisK-foldcross-validation.Forthis,wedividethedatasetinKdisjointparts.Westartwithusingk =2,…,Kfortrainingthemodelandpartk=1fortestingthemodel.Thenwerepeatthisprocedurebyusingpartk=2fortestingandtheothersfortraining,andsoonforallKparts.EachtimethesquaredresidualsarecalculatedandinthiswaytheMSEPiscalculated.

ThemostextremecaseofK-foldcross-validationiswhenKisequaltoN;thatis,thetestsethasasizeof1.Thisisalsocalledleave-one-outcross-validation.Severalotherapproachesexistforperformingmodelvalidation,includingbootstrapping,butthesefalloutsidethescopeofthisbook.

TheMSEPisamuchmorereliableestimateoftheerrorrateofthemodelthantheMSEofthetrainingsetandisausefulmeasureinordertoevaluatethemeasurementuncertaintywhich isan importantquantity in, forexample,decisionmaking (seealsoChapter16).Notethatvalidationdoesnotonlyapplytolinearmodels,buttoanykindofmodel,suchasneuralnetworks,whicharedescribedinChapter10.

6.2.3 Curse of dimensionality: Small n, large p

Whenwehavealargenumberofregressorvariables(p)andonlyasmallnumberofobservationsn,itisofteneasytoobtainanapparentlygoodfitofthemodel.Acom-monlyusedindicatorofmodelfitis

Ry y

y y

i i

i

2

2

2=

−[ ]−[ ]

∑∑

ˆ,

wherey is theoverallmeanof theobservations.Note thatR2isclosely related toMSE,butitisnormalizedbydividingitbythesumofsquareddifferencesfromtheoverallmean.ThevalueofR2isbetween0and1,whereahighervalueindicatesabetterfit(R2=1indicatesperfectfit).

InFigure 6.4 (left) it canbe seen thatR2 is rather high for perceivednatural-nessVO(n=30)using20arbitraryregressorvariablesfromtheMONATdataset.However, inFigure 6.4(right)wecanseewhathappensifweestimate themodelcoefficientsusing29of the30observationsandpredict theotheronewith itandrepeatthisforall30points(30-foldorleave-one-outcross-validation).Inthiscasethegoodness-of-fitiscalledQ2andissimilartoR2.Themaindifferenceisthatthe

Y116134.indb 133 10/10/11 2:11 PM


observationiwasnotusedinthemodelestimation(similartothedifferencebetweenMSEandMSEP).Ifthemodelfitsthedatausedinthetrainingwell,butistoospe-cificforthedata,thepredictedvalueyicanbefarfromtheobservedvalueyiiftheithobservationwasnotinthemodelfit.ThiscanclearlybeseeninFigure 6.4(right),wheresomepredictionshaveseveralnegativevaluesandonevalueisreallyveryfaroff(–2).HenceQ2cantakenegativevalues(thisisimpossibleforR2),whichindi-catesthatthismodelisevenworsethantheoverallmean.Itisclearthatthemodelwasfitted“tothenoise”ofthedata.

Ifwehavealargenumberofregressorsandonlyalimitednumberofobservations,itisalwayspossibletoobtainagoodfit(R2),becausewehavesomanypossibilitiestofitthedatainthehigh-dimensionalspace.However,themodelthusobtainedcanbetotallyuseless.Soahigh-dimensionalspacegenerallyalsorequiresaveryhighnumberofobservationsforareliablefit,becausethehigh-dimensionalspaceissoempty.Thisisoftenreferredtoasthe“curseofdimensionality.”

Ahighnumberofobservationsisoftennotpossibleandprocedureshavebeendevelopedtocopewithalargenumberofregressorvariablesandalimitednumberofobservations.Particularlyworthmentioningarepartialleastsquares(PLS)andleastabsoluteshrinkageandselectionoperator(LASSO;see,e.g.,Hastie,Tibshirani,&Friedman,2009).WithPLS,allregressorvariablesaretakenintoaccountinthemodel,andthismethodisoftenusedinhighlycorrelatedX-data,forexample,asobtainedfromaspectrometer.WithLASSO,thecriterionistominimizeapenalizedresidualsumofsquares:

ˆ argminβ β λ ββ

LASSOi i j

j

P

y= −[ ] +

=∑Χ 2

1ii

N

=∑

1

,

Perceived Naturalness Score

0.2 –2.0

–1.5

–1.0

–0.5

0.0

0.5

1.0

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

R^2 = 0.88 (20 variables) Q^2 = –4.67 (20 variables)Fi

tted

Valu

es

Pred

ictio

ns (L

eave

-One

-Out

)Perceived Naturalness Score

Figure 6.4 Using20arbitraryvariablesof thevision-relatedX-features and thevisuallyperceivednaturalness(VO),amodelcanbebuiltthatdescribesthedatawell(left),butwhichisaverypoorpredictor(right)whentestedondatathatwerenotusedforfittingthemodel.

Y116134.indb 134 10/10/11 2:11 PM


wheretheRSSofthelinearmodelgetsapenaltyequaltothesumoftheabsolutevaluesofβ1,…,βP,multipliedwithashrinkageparameterλ.Ifλiszero,thisisthenormallinearregression;ifλisinfinite,allcoefficientsexcepttheoverallmean(β0)becomezero.Thisslightmodificationhasaconsiderable impact,as iteffectivelyfunctions as a variable selection procedure. In the MONAT project, the LASSOhasbeenusedtoestablishtherelationbetweentheresponsevariableandthemostrelevantregressorvariables.

6.2.4 Canonical correlation

Givenaset(matrix)XandYconsistingofpandqvariables,respectively,canonicalcorrelationtriestofindthelinearrelationswithinXandwithinY,whichgivemaxi-mumcorrelationbetweenthem.Theobjectiveistofindalinearcombinationofthex-variables,sayu1,anda(different)linearcombinationofthey-variables,sayv1,sothatthecorrelationbetweenu1andv1isthehighestattainablecorrelationofallpos-siblelinearcombinationsuofXandvofY.Inthenextstep,wetrytofindasecondpairoflinearcombinations(u2,v2),uncorrelatedwiththefirstpair,suchthatthecor-relationofthissecondpairis(conditionally)maximal,andsoon,butofcoursenomorethanmin(p,q)pairscanbefound.AnexampleofcanonicalcorrelationisgiveninFigure 6.5,wherethethreey-variables(VO,TO,andVT)werecorrelatedwiththetouch-relatedvariablesoftheX-matrix.

Thefigureshowstherelationbetweenthefirstcanonicalpair(left)andsecondcanonicalpair(right),withthelinearcombinationofXontheX-axisandthelin-earcombinationofYon theY-axis. InTable 6.1 thecorrelationbetween thepair(CA_Corrs)andthe%correlation(%Corrs),respectively,thecumulativepercentagecorrelation(Cum%Corrs)isshown.Obviously,becauseYhasonlythreevariablesinthisexample,therecanbenomorethanthreepairsofcanonicalvariables.

1st Canonical Correlation XT v Y

0.4

0.2

–0.0

–0.2

–0.4

–0.4 –0.2 –0.0 0.2 0.4 –0.4 –0.3–0.2–0.1 0.0 0.1 0.2 0.3

–0.4

–0.3

–0.2

–0.1

0.0

0.1

0.2

0.3

2nd Canonical Correlation XT v Y

Figure 6.5 The relationof thefirst (left) and second (right) canonical correlationpair isshown for the perceived naturalness Y (vision-only, touch only, and visiotactile) and thetouch-relatedfeaturesXTareshownfor30samplesoftheMONATwooddataset.

Y116134.indb 135 10/10/11 2:11 PM


Canonical correlation produces maximally correlated linear combinations ofX and Y, but we should note these can be poor representations of the true struc-tureofXandY.Furthermore,canonicalcorrelationinvolveslinearcombinationsofy-variables,butoftenyouareinterestedinthevariablesthemselvesandinthatcase,youcanbetteruselinearregression.

6.3 Classification and clustering

6.3.1 Supervised classification—Linear discriminant analysis

Inmany cases the response variable is not a continuousvariable, but a groupingvariable.Forexample,wecanclassifytheMONATwoodsamplesinthreegroups:untreatedwood,treatedwood,andartificialproducts(imitationwood).Fortheanal-ysisofdataingroups,classificationmethodscanbeapplied.Themostoftenusedclassificationmethodislineardiscriminantanalysis(LDA).

InLDAweassumethatthedifferentgroupshavedifferentmeans/centroidsfortheX-variables,butthegroupsshareacommoncovariancematrixS.Theaimistomaximizethebetween-groupstowithin-groupsvariation.ThecovariancematrixSiscalculatedbypoolingtheindividualcovariancematriceswithineachgroupg,thatis,byformingap×pmatrixSgpergroupandpoolingthesecovariancematricesSgoverallgroups(takingaccountofdifferencesinng,thenumberofsampleswithineachgroupg).Thebetween-groupscovariancematrixGiscalculated,usingonlythemeansforeachgroupasdata.

Next we try to maximize the ratio of the between-groups over the within-groupscovariance:

maxa

T

T

a Gaa Sa

.

Putdifferently,trytofindthelargesteigenvectorsforthenonsymmetricmatrixS–1Gusinggeneralizedeigenvectordecomposition,takingintoaccountthespecialstruc-tureofthismatrix.Theeigenvaluesandeigenvectorscannowbeusedtorotateandscale the original data to a space that maximizes the between-groups to within-groupsvariation.Thiscanbereferredtoascanonicalvariateanalysisascomparedtoprincipalcomponentanalysis.Inthistransformedspacewecannowcalculatethedistanceofeachobservation to thegroupmeanandclassify thedatapoint to thegroupthathasthesmallestdistancetothispoint.

InFigure 6.6,aplotisshownforthewoodsampledatadividedinthreegroups(untreatedwood,treatedwood,andartificialwood),withthescoresforthefirsttwo

Table 6.1 CoronicalcorrelationofMONATdataset.(SeealsoFigure6.1.)

Pair CA_Corrs %Corrs Cum%Corrs

1 0.9910 33.67 33.672 0.9893 33.61 67.283 0.9629 32.72 100.00

Y116134.indb 136 10/10/11 2:11 PM


canonicalvariatesofthevision-relatedfeaturesoftheX-dataset.Theplotalsoshowsthecorrespondinggroupmeansandtheir95%confidenceintervals.

Ifwehavemany,oftencorrelated,variables(pislarge),thematrixSmightnotbeoffullrank.Thismeansthatitsinverseisnotdefined.Asolutiontothisistoregular-izethematrixSwiththeregularizationparameterγasS(γ)=γS+(1–γ)σ2I,whereσ2IindicatestheelementsonthemaindiagonalofthematrixS,thatis,thevariancesofthepvariables.

Manyotherclassificationmethodsexists,including

• Quadratic discriminant analysis, which allows for different covariancematrices per group instead of a common pooled covariance matrix W.Becausethenumberofparametersisverylarge,p(p–1)/2foreachclassorgroup,regularizationisoftenneeded,forexample,as:Sg(α)=αSg+(1–α)SwithSgthepreviouslymentionedcovariancematrixwithingroupg.

• k-Nearest-neighbor classifier,whichclassifiesanobjecttothesameclassas themajorityofclosestpointsofgroup-sizek.Note that thedefinition

Canonical Variate 1

Discriminant Analysis Wood Samples (Vision Data)

Cano

nica

l Var

iate

2

–1.0

–1.5

–1.0

–0.5

0.0

0.5

1.0

1.5

–0.5 0.0 0.5 1.0 1.5 2.0

Artificial

Untreated

Treated

Figure 6.6 Indiscriminantanalysiswetransformthedataso that thebetween-groups towithin-groupsvariationismaximized.The30samplesoftheMONATwooddatasetweredividedintothreeclasses:class1includesuntreatedwoodpanels(3×2topleftpanelsinFigure6.1),class2includesthetreatedwoodpanels(4×2toprightpanels),andtheartificialwoodsonthebottomthreerowsaregroup3.Thescoresforthefirsttwocanonicalvariatesareshownforthevision-relatedphysicalfeatures.

Y116134.indb 137 10/10/11 2:11 PM


ofdistance(closest)isnotstraightforward.Seealsothenextparagraphonclusteringanddissimilarity.

• Artificial neural networks,which allow fornonlinearityof the classifier.SeealsoChapter10inthisbook.

Mostclassificationmethodswithalargenumberofvariables(highp)arerathersen-sitivetonoise,thereforeitisstronglyadvisedtoalwaysuse(cross-)validation.

6.3.2 Unsupervised classification—Clustering

InsupervisedclassificationsuchasLDA,priorinformationontheclasslabelsoftheobjectsisneeded.Sometimes,suchinformationisnotavailableandclasslabelsneedtobeassignedtotheobjectswithoutpriorinformation;thatis,wewanttoinfertheclasslabelsfromthedatathemselves.Alogicalapproachistoformclustersofdatapointsthatareconsideredtobeclosetogether.Theclusterscanthenberegardedasclassesformedbythedata,hencethetermunsupervisedclassification.

Differentclusteringapproachesexist,butinthisparagraphweonlyconsiderhier-archicalclustering.Inhierarchicalclusteranalysiswewishtoperformtheclusteringinsuchawaythatobjectsinthesameclusteraremorelikeeachother(i.e.,moresimilar) than they are to other objects in other clusters. The notion of similarityordissimilarityiscrucialinthiscontext.Themethodformsclustersconsistingofsubclustersinahierarchicalway,withtheindividualobjectsasthelowestlevelofclustersofsize1.

6.3.3 Dissimilarity measure

Hierarchicalclusteringstartswithameasureofdissimilaritydbetweentheobjectsorclusters.Ifwehavemeasuredattributesofobjects,wecancalculatedissimilaritiesbetweenobjects,takingintoaccountthescaleofthevariables.Manydifferentmea-suresof(dis)similarityexist(Gower,1985).Forintervalandordinalscalevariables,commonmeasuresaretheCityblock(L1)orEuclidean(L2)distance,dividedbytherange(differencebetweenlargestandsmallestobject)toobtainavaluebetween0and1.Forbinaryvariables,theJaccarddissimilarityisoftenused:ifbothobjectshaveascore1,thedissimilarityis0;whenthescoresareopposite,thedissimilarityis1andifbothscoresare0,theentryisignored.Fornominalvariables,themostcommondissimilaritymeasure issimplematching: ifbothobjectshave thesamescore,thedissimilarityis0,elseitis1.

Insomecases,wedonotobtainmeasurementsofattributesoftheobjects,butwedirectlyassessdistancesbetweenobjects.Thisgivesrisetothedirectconstructionofadissimilarityordistancematrix.InChapter9,aspecialmethoddealingwithsubjective(Fechnarian)distancesisdescribed.

Afterhavingobtainedadissimilaritymatrixbetweenallpairsofobjects,wecangroup (clustersof)objects togetherbasedon theirdissimilarity.Different criteriaexistformergingclusters,suchas

Y116134.indb 138 10/10/11 2:11 PM


• Single linkage:Minimumdistancebetweenanytwoobjectsintheclusters.• Complete linkage:Maximumdistancebetweenanytwoobjectsintheclusters.• Group average linkage:Averagedistanceistakenoveralltheobjectsinthe

twomergingclusters.

InFigure 6.7theresultofhierarchicalclusteringisshownfortheMONATdataset.Settingacertainthreshold(verticallineinthedendrogram)cangiveanobjectivecrite-rionforgrouping,butthisisnotalwaysstraightforward.Anotherdisadvantageofden-drogramsisthatthepositionsofclustersandobjectswithinclustersarenotunique.

OtherpopularclusteringapproachesincludeK-means,fuzzyC-meansandneu-ralnetworkapproachesasself-organizingmaps.Ifthedatawithinaclustercanbeassumedtofollowanormaldistribution,powerfulmethodsexisttodealwithsuchmixturesofnormaldistributions,suchastheexpectation-maximizationalgorithm.

3.253.223.233.242.143.263.273.303.163.152.072.132.122.053.282.111.091.081.102.063.182.043.203.293.193.213.171.031.021.01

Dendrogram Single Linkage Wood Data

Figure 6.7 From the MONAT wood samples, the vision-related physical variables havebeenselectedtocalculateaEuclideandistancematrixamongthe30objects.Thedistancematrixisusedforhierarchicalsingle-linkageclustering,whichmeansthatclustersarebasedontheminimumdistancebetweenanytwoobjectsintheclusters.Theintegralpartofthenumbers(e.g.,1in1.01)refersto(1)untreatedwood,(2)treatedwood,and(3)artificialwood.Thedecimalpartreferstothepanelnumber(1,…,30)inFigure6.1.

Y116134.indb 139 10/10/11 2:11 PM


6.3.4 Principal coordinates analysis

Thedissimilaritymatrixthatwascalculatedinthepreviousparagraphcanbevisual-izedinadendrogram.Anotherapproachistomapthematrixintoalower-dimensionalspace,much like trying to reconstruct a2-D roadmap froma city-distance table.Thisisthetopicofmultidimensionalscaling,wheretheobjectiveistofindasetofcoordinateswhoseinterpointdistancesmatch,ascloselyaspossible,thoseofthedis-similaritymatrix.Thecoordinatesystemcanbeinterpretedinmuchthesamewayasamap:forexample,ifthedistancebetweenobjectsinthedatamatrixissmall,theirpointswillappearclosetogetherinagraphusingthesecoordinates.

InFigure 6.8ametricscalingmethodisshownfor thewoodsamplesdissimi-laritymatrix:principal coordinates analysis.The class1wood samples are closetogetherinthismap,althoughthisinformationis,ofcourse,notusedintheconstruc-tionofthemap.Thecalculationofprincipalcoordinatesdoesnotrequireaniterativeapproachandisthereforefastandunique.Itcanbeusedasastartingpointformanyother(nonmetric)multidimensionalscalingmethods.

Inthesamegraph,theminimumspanningtreeisshownconnectingthenobjects,whereeveryobject is linked to thesame tree (connectednetwork).The treedoesnotcontainclosedloopsand,ofalltreeswithoneobjectateverynode,itistheonewhoselinkshaveminimumtotallength.Thelinksincludeallthosethatjoinnearest

First Coordinate

Principal Coordinate and Minimum Spanning Tree

–0.4

–0.2

Seco

nd C

oord

inat

e

0.0

0.2

0.4

–0.2 0.0

1.081.10

1.092.06

3.182.041.01

1.031.02

3.193.293.203.16

3.153.27

3.17

3.21

2.05

2.122.13

3.25

3.22

2.143.23

3.242.113.28

3.26

3.302.07

0.2 0.4 0.6

Figure 6.8 Thedistancematrixusedforclusteringisshownhereinaprincipalcoordinatesystem.Alsoshownistheminimumspanningtree,connectingcloseneighbors.Thetreecanrevealregions in thecoordinatesysteminwhichdistanceisbadlydistortedandiscloselyrelatedtothesinglelinkagedendrogramofFigure6.7.

Y116134.indb 140 10/10/11 2:11 PM


neighborsand theminimumspanning tree isclosely related to thesingle linkagedendrogram.Minimumspanningtreesareusefultorevealregionsinthecoordinatesysteminwhichdistanceisbadlydistorted.

6.4 Conclusions

Intheprevioussectionswehavegivenanoverviewandapplicationofseveralcom-monmultivariatedataanalysismethods.Somemethodsarepredominantlyaimedatgaininginsighttothedataathand,forexample,byreducingthedimensionalityofthedata.PCAisverypopularforthis.Othermethodsareaimedatrelatingonesetof features toanother.Oneapproach is to relateasinglecontinuousy-variatewith many X-measurements, which is the case in linear regression. This can beextendedtomodelmultipley-variatessimultaneously.Ifwehavealargenumberof(possiblyhighlycorrelated)regressorvariablesandasmallnumberofobjects,itisbettertorefertomethodssuchasLASSOorPLS.Iftheobjectsformgroupsandwehavethatinformationavailable,wecanuseclassificationmethods,suchaslineardiscriminantanalysis.

IncanonicalcorrelationtheaimistofindpairsoflinearrelationsinamultivariateX-andY-dataset,insteadoftryingtopredictoneormorey-variatesfromtheX-data.

If noprior information ispresent and the aim is to classifyor cluster objects,clustermethodssuchashierarchicalclusteringcanbeused.Forhierarchicalcluster-ingadissimilaritymatrixisneeded.Suchamatrixcanbeconstructeddirectlyintheexperiment (e.g.,usingpairwisecomparisons)orcanbeformedfromasetofmeasurementdata.Dissimilaritymatricescanbevisualizedandinterpretedusingmultidimensionalscalingmethods.

Whendealingwithmultivariatedata,itisimportanttorealizethathigh-dimen-sional spaces are generally very sparse and special care needs to be taken wheninterpretingtheresults.Methodsspecificallydevelopedforuseinhigh-dimensionaldata,suchasPLSandLASSO,aretobepreferredand(cross-)validationhastobeusedatalltimesifpossible.

References

Goodman,T.,Montgomery,R.,Bialek,A.,Forbes,A.,Rides,M.,Whitaker,A.,Overliet,K.,McGlone,F.,&Heijden,G.W.A.M.vander.(2008).Themeasurementofnatural-ness.In12th IMEKO TC1& TC7 Joint Symposium on Man Science & Measurement,Annecy,France.

Gower, J. C. (1985). Measures of similarity, dissimilarity, and distance. In S. Kotz, N. L.Johnson,&C.B.Read(Eds.),Encyclopaedia of statistical sciences(Vol.V,pp.397–405).NewYork:Wiley.

Härdle,W.,&Simar,L. (2007).Applied multivariate statistical analysis (2nded.).Berlin:Springer.

Hastie,T.,Tibshirani,R.&Friedman,J. (2009).The elements of statistical learning: Data mining, inference, and prediction(2nded.).Springer,Berlin.

Johnson,R.A.,andWichern,D.W.(2002).Applied multivariate statistical analysis.UpperSaddleRiver,NJ:Pearson.

Y116134.indb 141 10/10/11 2:11 PM


Mardia,K.V.,Kent,J.T.,&Bibby,J.M.(1979).Multivariate analysis.London:AcademicPress.Overvliet,K.E.,Soto-Faraco,S.,Whitaker,T.A.,JohnstoneSorensen,L.,McGlone,F.,&

Heijden,G.W.A.M.vander(2008).Measuringperceptionofnaturalness.InA.J.Spink,etal.(Eds.),Proceedings of Measuring Behavior 2008(Maastricht,TheNetherlands,August26–29),pp.86–87.

Tabachnick,B.G.,&Fidell,L.S.(2007).Using multivariate statistics (5thed.).London:Allyn&Bacon.

Y116134.indb 142 10/10/11 2:11 PM

143

The prospects for measurement in infinite-dimensional psychological spacesModern notions for geometric person measurements in finite and infinite dimensional spaces

James T. Townsend, Devin Burns, and Lei PeiDepartmentofPsychologyIndianaUniversityBloomingtonBloomingtonIndiana,USA

7.1 Introduction

Thischapterisaboutcertainspatialtypesofmeasurementofpeople,ascarriedoutinthesocialandbiologicalsciences.Theglobalissueforusisperceptualclassifica-tionandwespecializeonspatial,forinstance,geometricortopological,aspects.Weoffermoredetailbelow,butfornow,wesimplynotethatclassificationistheassign-mentofanobject,typicallypresentedtooneormoreofoursensorymodalities,tooneofsomenumber(perhapsinfinite!)ofclassesordesignatedsets,oftennames.Becauseingeneralterms,thisisneithermorenorlessthanafunctionormappingofasetofthings(stimuli,memories,etc.)intoanother(thenamesorclassifications),thismakesthisconceptextremelybroad.Thus,classificationcoversmanyspecificactivitiesofpsychologicalinterest.Someimportantonesarethefollowing:

1.Afamousformofclassificationisthatofcategorization,whereabunchofobjectsispartitionedsothateachobjectisseparatelyassignedtoasingle(typically)uniquecategory.Therearemanykindsofcategorizationbutoneofspecialimportanceisusuallygivenitsowntitle.

2. Identification is where each category has exactly one member (this mayberareinthereallife).Wecanperformanexperimentwhereeachpersonhasexactlyonenameandeachname isgiven toexactlyoneperson,butofcourse, intherealworld, thenameof“John”isgiventothousandsofpeople.Inmostoftheinterestingpsychologicalsituations,classificationis

7

Y116134.indb 143 10/10/11 2:11 PM


madedifficultbyanynumberof factors.Thiscouldbebecause learningoftheassignmentsisincomplete,suchasinarecognitionmemoryexperi-ment.Inthiscasethepresentedstimulusmaybeveryeasytoperceive,butthelearnedpatternsmayhavebecomequitenoisy,makingtherecognitiontaskchallenging.Othersituationsincludesomekindofaddednoise,ran-domness,orevenverybriefexposureperiods.

3.Yes–no signal detection is the specificationbyanobserverofwhether asignalhasbeenpresented.Thereareonlytworesponses:“YES”versus“NO”couldbeusedeitherwithfaultymemory(thecelebrated“Old”versus“New”memory-recognitionexperiment)orwithsimply“hard-to-detect”physicalstimuli.

4.Typicallywithpsychological scaling,theobservergivesratingsononeormore aspects of the stimuli.For instance, onemight simplybe asked toreport a number that seems to represent the psychological magnitude ofan aspect of a stimulus, such as how happy a particular face looks (seebelowformoreonthis).Or,asisoftenthecaseinmultidimensionalscaling(explainedabitlater),theobservermightbeaskedaboutthepsychologicalsimilarityof twoormoreobjects.Anoldanthology,butonestillheldinhighesteem,withseveralchaptersthatarerelevanttoourpresententerpriseisVolumeIoftheHandbook of Mathematical Psychology(Luce,Bush,&Galanter,1963).Thephilosophyofscienceofthepresentauthorsregardingmeasurementandmodelingofpeopleandanexpansionofsomeofthemes,canbelocatedinTownsendandThomas(1993).

All the chapters in this volumeon “personmeasurement”present anddiscussvaluabletypesofmeasurementofpeople,anumberoftheminvolvingclassification,usuallypertainingtoafinitenumberofaspectsordimensions.Section7.2startsbydiscussingthefoundationalaspectsofmeasurementonpeopleandconsiderssuchfinitesituations.Section7.3concernsmultidimensionalscaling,andalsofocusesonafinitenumberofpsychologicaldimensions.Itismeantnotonlytosetthestageforwhatistocome,butalsoprovideanelementarytutelageonthesubjectandtoguidetheprospectiveuserofthesetoolstosomeoftheclassicalliterature.

That being said, it is our belief that almost none of the interesting aspects ofperceptionreferattheirmostbasicleveltofinitespaces.Rather,wefeelthatinfinite-dimensionalspacesarerequiredforasuitablytheoreticalmilieuforhumanpercep-tionandbeyondthat,tocognitionandactionaswell!Forclaritywesolelyemphasizeperceptioninthischapter.Thetopicoffinite-versusinfinite-dimensionalspacesistakenupinSection7.4.Asaconsequenceofthisstance,animportanttopicisthemathematicalissuesthatmightarisewhenweinquireabouttheabilityofpeopletoextractfinite-dimensionalinformation,eventhevalueonasingledimension,fromobjectsthatbytheirverynatureresideininfinite-dimensionalspaces.ThisquestionisaddressedinSection7.5.

Ahugeliteratureexistsinmathematics(mostlyforphysicsandengineering)oninfinitedimensionalspaceswithparticularrelevancetofunctionsinspacesthatpos-sesscoordinatesatrightanglestoeachother.Itiscalledfunctional analysis.Such

Y116134.indb 144 10/10/11 2:11 PM

The prospects for measurement in infinite-dimensional psychological spaces 145

spaces typicallyare inherentlyflat,withoutcurvature.A revolutionarymovementtookplaceinthelatenineteenthandearlytwentiethcenturiesconcernedwithspacesthatcouldbecurved,evenwithspaceswhosecurvaturecouldchangefrompointtopoint.WerefertoRiemannianmanifolds.Section7.6introducestheconceptofmanifoldsasgentlyaspossible.Thiskindofspaceprovedtobeexactlytheappro-priate setting forEinstein’s theoryofgeneral relativity.Wegooutona limbandsuggestthatmanifoldsmaybeimportantforhumanperceptionandcognitionandactionaswell.

Section7.7takesupthepotentialextensionofprobabilitytheoryandstochasticprocessestoinfinite-dimensionalspaces:Everythingweknowaboutpeople,fromthelevelofasingleneuron(andlower)totheactionsofthepeopleofanentirenationareprobabilistic.Hence,anytheoryormodelwhetherfiniteorinfinitemustsooneror laterbroach this topic.There is aknowledgebase forprobabilities in infinite-dimensionalspacesthatgoesbacktotheearlypartofthetwentiethcentury,withtre-mendousdevelopmentsoccurringmid-century.Thisknowledgeappliesimmediatelytothetypesofspacesfoundinfunctionalanalysis,mentionedjustabove.Thereissomeworkgoingon,inaddition,developinglawsofprobabilityforfinite-dimensionalmanifoldsandreallycutting-edgeworkinprobabilityandstochasticprocessesoninfinite-dimensionalmanifolds.

Section7.8treatstheveryspecialandextraordinarilyimportantcaseofclassifi-cation models based on functional analysis, dynamic systems theory, differentialequations,andstochasticprocesses.WethendelvemoredeeplyintothemathematicsandtakealookatapplyingthesetechniquestotheclassificationfieldinSection7.9.ThechapterconcludeswithSection7.10,whichisabriefreviewofwherewe’vebeenandwhy,andanattempttosetthesenewmethodsintheirpropercontext.

Theearlypartsofthischapterleaveoutthemathematics.Thisisnotonlybecauseofsomewhatlimitedspacebutalsobecausetherearescoresandsometimeshun-dredsofarticlesandbooksonthetopic.Aswemovealong,wedelveintomaterialthat is likelynewtomanyinvestigatorsandstudentsdevoted tomeasurementonpeople. Although our greater use of mathematics as we progress may make thegoinga little tougher for some readers, it alsoaffordsamore rigorousandwell-definedlandscapeforthosewillingtomakethejourney.Weaccompanytheverbalandquantitativedevelopmentswith illustrations toaid in the readers’ interpreta-tionsandintuitions.Inthisway,wehopethateveryonewillgainsomethingfromourpresentation.

7.2 Foundational measurement

Acentralcharacteristicofmeasurementintheseregions,asopposedtovastregionsofphysics,istheabsenceoftheopportunitytogatherdatabasedonstrongmeasure-mentscales(e.g.,Roberts,Patterson,Thomson,&Levey,1969).Thissinglecharac-teristichasbeenthesubjectofhundredsofpapersandbooks,withrootsinfirst,fromthephysicalsciencepointofview,Campbell’sfamousbookinwhichhedevelopsatheoryoffundamentalmeasurementasamappingbetweennumericalsystemsandthosethingswewishtomeasure.Inhistheory,forsomethingtobemeasurable,those

Y116134.indb 145 10/10/11 2:11 PM


thingsmusthaveanorderingandthecombinationof those thingsmustfollowthesamepropertiesastheadditionofnumbers,whichisoftennotsatisfiedbythethingswewishtomeasureinpsychology.

AfundamentalmeasurementsystemasdefinedbyCampbellimpliesaratioscale(Suppes & Zinnes, 1963). One of the earliest proposals that the scales typicallyemployedbyphysics,thatis,“ratioscales”or“extensivemeasurement”mightnotbetheonlypossibilitiesforrigorousapplicationofnumbersinthesocialorbiologicalscienceswasputforthbyStevens(1951).Later, this ideawaspickedupbymath-ematicalpsychologists,mathematicians,andlogicianswiththeensuingdevelopmentofanaxiomaticbasisnotonlyforextensivemeasurementbutalsotheweaker(andinorderofdecreasingstrength),absolutescale,ratioscale(extensivemeasurement),intervalscale,ordinalscale,andso-callednominalscale.Theonlyotherscaleofnoteistheabsolutescale,whichisthestrongestofall(e.g.,formoreonthesetopics,seetheearlyworkandcitationsinSuppes&Zinnes,1963;Krantz,Luce,Suppes,&Tversky,1971;Pfanzagl,1968).

A seminal concept in the theory is that numbers considered as measurementsshould reflect the regularitiesof thenaturalphenomena that arebeingmeasured,whileretainingpropertiesofthenumbersystem(perhapsincludingarithmeticcon-ditionsandthelike).Hence,forsomepurposes,anintervalscale,suchasCelsiusorFahrenheitwillsufficetostudyparticularpropertiesofaphenomenon.However,forwork,sayinthermodynamics,itmaywellbethataformoftheKelvinscalemayberequired.Afascinatingconsequentdualityinthetheoryisthatthestrongerthescale,thefewermathematicaloperationscanbecarriedoutontheobservedmeasurementswithoutdistortiontotherelationshipsbetweenthenumbersandthephenomena.Anexampleofapermittedtransformationonaratioscale,likemass,ismultiplicationbyapositiveconstant(e.g.,changingpoundstokilograms).Thisleavesaratioofanytwomeasurements invariant: it isa“meaningfuloperation.”However,calculatingtheratioofyesterday’stemperaturewithtoday’stemperature,measuredinCelsius,willnotbe thesameaswhen theunitsarealtered toFahrenheit:meaningfulnessislost.Inthisintervalscaleweinsteadneedtotakearatioofdifferencestoretainmeaningfulresults.Toterselycompletethepicture,absolutescalesallownotrans-formationwhatsoeverandtheypossessnounits.Countingisperhapstheprototypi-calexampleofanabsolutescale:onedoesn’tchangea“3”whenonecounts3candybarsoranythingelse.Ofcourse,achangeofthenameof“three”fromonelanguagetoanotherisokay!Thenextstrongestscaleisaffordedthename“ordinalscale.”

Ordinalmeasurementpermitsanymonotonic(order-preserving) transformationtobemadeonthemeasurementnumbers,butonlyordercanbeestablished:com-parisonssuchasratiosordifferencesaremeaningless.Ithasbeendebatedwhetherthesocialorbiologicalsciencesmaybeforeverconfinedtoordinalmeasurement.Fornow,wemainly investigateaspectsofmeasurement thatdonot touchon thisaxiomatic theorybutalsoa few thatdo.Obviously,evenamodestglimpseat theentireedifice(nottomentionthedebates)wouldtakeseveralhundredpages.Oneoftheshortcomingsoftheapproachhasbeentheabsenceofagenerallyacceptederrortheory,whichwouldpermittheerectionofastatisticaltheoryofinferencewithinthe

Y116134.indb 146 10/10/11 2:11 PM


approach.Weshouldalsomentionthatmanystatisticians,psychometricians,andoth-ershaveeitherarguedagainsttheideathatmeasurementscalesareneeded(e.g.,Lord&Novick,1968;Townsend&Ashby,1984)orsimplyignoredit.Nonetheless,fromtheauthors’pointofview,itisimportanttokeeptheseconsiderationsinmind.1

7.3 A brief introduction to multidimensional scaling

Apopularmethodofinferringgeometricalstructurefromdatainpsychology(andrelateddisciplines) is calledMultidimensionalScaling,orMDS.MDS isa setofmathematicaltechniquesthatcanbeusedtorevealtheunderlyingstructureindata.Thesemethodsprovidea spatial configurationof thedatapoints,whichallowsaresearchertointerpretpatternsandregularitiesrelativelyeasily.Therearenumerous(butnotquite infinite!)variationsof these techniques,andhereweonly touchonsomeofthemostimportant,general,andfamousmethods.

Tobeabletorepresentpsychologicaldatainageometricalform,wefirstneedtoestablishanotionof“distance.”Therearemanydifferentwaysofdoingthis,andcareershavebeenspentdebatingwhich is themostappropriate.Oneof themoststraightforwardmethodsistoconductasimilarityexperimentinwhichsubjectsratehowdissimilardifferentstimuliarefromeachother.Thedistanceshouldthenbeamonotonic transformationof thisdissimilarity, so that themore similarobjectsare the lessdistance there isbetweenthem.Afterestablishingamatrixofall thepairwisedistancesbetweenstimuli,computerprogramscanbeusedtoinferwhereeachstimulusislocatedinaspaceinrelationtotheothers.Mostmodernstatisticalcomputingpackages,suchasSPSS,willcontainaprogram(ALSCALisapopularexample)thatdisplaystheseresultsforyou.

Butwhatresultsdoesitdisplayexactly?ThisbringsustoanothertopicofdebateinMDS,thatofdimension(Theconceptofwhichisdiscussedinmoredetailbelow).Whenthecomputerinfersthepositionsofthedatapointsinspace,wehavetotellit in which space to put them. Euclidean space is the most commonly used andsimplest,butthereisnoreasonwhythedatashouldnecessarilybeEuclidean,andvarious spaces andmetrics havebeen imposed for different domain applications.StickingwithEuclideanspacefornow,itmustbedecidedbytheinvestigatorhowmanydimensionstouse.Whenfittingthepoints,MDSprogramsattempttomini-mize the “stress” in the system, which roughly means how well the extrapolatedgeometricdistancesconformto thedistancesderivedfromthedissimilaritydata.Ahighstressvaluemeans that themodel isnotfitting thedata.Clearly,amodelinahigher-dimensionalspacewillalwayshavelessstressthanamoreconstrainedmodel.Wecaneasilyseethatlower-dimensionalmodelsareincludedassubspacesofhigher-dimensionalmodels,sothismustalwaysbetrue.

Sohowdoesoneknowhowmanydimensionstouse?Theoreticalexpectationscanbebroughttobearhere.Ifyourstimulivaryalongonly,say,ndimensions,thenan n-D model may be most appropriate (ignoring more complicated instances ofinteractionfornow).Anexceptioncouldoccurifthebrainisunabletoseparatethendimensions.Inthatcase,asubspaceconsistingofk<ndimensionswillsuffice.In

Y116134.indb 147 10/10/11 2:11 PM


nocasewouldweexpecttheestimateddimensiontobek>n,althoughaninappro-priatemetricmightmakeitappeartobeso.Forexample,supposethepsychologicalspacecorrespondingtotwophysicaldimensionsisa2-Dhemisphereembeddedin3-Dcoordinatesystem.Then,theinappropriateEuclideanmetriccouldmakeitseemthatthreedimensionsarebetterthantwo.Ifonehasthecorrectmetricinhand,thestressof the two-dimensionalmodel shouldbe almost as small as that for three-orgreater-dimensionalmodels.This subjectivecomparisonamong the stressesofdifferentmodelscanbeusedasthedecidingfactoreveninthelackoftheoreticalexpectations.Inthisuninformedcase,theexperimentershouldpickthemodelsuchthatallsubsequentmodelsofgreaterdimensionalityyieldonlyasmallreductioninstressrelativetothepreviousreductions.

IfallofpsychologicalspacewerenicelyEuclideanwithseparabledimensions,thenallofthiswouldbefineandwecouldconstructhigh-fidelitymapsforeverydataset.As I’msureyou’veguessed, this issadly(butmore interestingly)not thecase.Whatifpsychologicaldissimilaritydataarenotdirectlyassociatedwithamet-ric?For instance, it couldbe thatdecisionalbiases intrude thatviolatepropertiesassociatedwithanymetric(e.g.,Townsend,1971).Insuchacase, it issometimespossibletoextricatethetruedissimilarityapartfromthedecisionalinfluences.Inanyevent,weneedtoknowwhatthepropertiesofavaliddistancemeasureare.Thethreeaxiomsthatanymetricmustsatisfyare:

Minimality:d(a,b)≥

d(a,a)=0:Nopairofobjectsismoresimilartoeachotherthananyobjectistoitself,andthelatterdistanceis0.

Symmetry:d(a,b)=d(b,a):Nancy’sfaceisassimilartoJoan’sasJoan’sfaceistoNancy’s.

Thetriangleinequality:d(a,c)≤

d(a,b)+d(b,c):ThepoliticaldistancebetweenObama’s world-view and George W. Bush’s is less than or equal to thedistancebetweenObama’sandFranklinD.Roosevelt’splusthatbetweenRoosevelt’sandBush’s.

Itcouldbethatoneormoreoftheaxiomsisviolatedduetoperturbationbyadeci-sionalbias,oritcouldhappeninamorefundamental,systematicsense.Themathe-maticalpsychologistAmosTverskypointedoutthatallthreeoftheseaxiomscanberoutinelyviolatedinbasicpsychologicalexperiments(Tversky,1977).Minimalityisviolatedwheneverstimulidifferintheirself-similarity.Symmetryisoftenviolatedinsimilaritydataaswell,especiallyifonestimulusisseenasmorebroadorgeneralthan the other, perhaps including the latter as a subclass. For example, the word“poodle”wouldbeseenasmoresimilartotheword“dog”thanthereverse.

Thetriangleinequalityishardertorefute,becauseitisaquantitativestatement,andsimilaritydataisinherentlyordinal.Ithasbeenshownthattrivialmanipulationsofdatacanproducesatisfactionofthefirsttwoassumptions,sothetriangleequalityoftenplaysamajorroleintestingforthepresenceofametric.Althoughtherehavebeenmorerigorousexpositionsofthesubject,Tverskygivesaquickintuitiveargu-mentforwhyweshouldn’tbelievethatthetriangleequalitywillnecessarilyholdinallcases.AssumethattheperceptualdistancefromCubatoRussiashouldbesmall

Y116134.indb 148 10/10/11 2:11 PM


forpolitical reasons (remember,1977), and theperceptualdistance fromCuba toJamaicashouldbesmallforgeographicreasons.ThetriangleinequalitywouldthenforcethedistancefromJamaicatoRussiatoalsobefairlysmall,whichwewouldnotexpect.

One reaction to theseargumentshasbeen to imposeadifferentmetricon thespace.InsteadofusingthestandardEuclideanmetrictocomputedistances,manyotherfunctionshavebeenused.Acommonfamilyofmetricstakestheform:

d x xij im jmr r

m

n

= −( )=

∑1

1

.

These are commonly called power metrics. When r = 2 we have the standardEuclideanmetric,andr=1iswhatisknownasthecityblockmetric.Inthismetricdistances are computed as the sumof theprojecteddistances in eachorthogonaldimension(asinacity,whenyoucanonlytravelinorthogonaldimensions,insteadofasthecrowflies).Inthismetric,thetriangleinequalitybecomesanequality.Ifweconsidervaluesof r <1, the triangle inequality isnow reversed.This canbeinterpreted tosaying that travelingalongonedimension is“faster” than travelingalongadiagonalpath.Thesemetricscanbebetterunderstoodifweconsidergraphsofunitdistance,showninFigure7.1.Inthesegraphseverypointcorrespondstoanequaldistancefromtheorigin.

Tverskyofferedupadifferentapproachwhenherigorouslydevelopedthefeature contrast model.Inthismodel, thesimilaritybetweentwoobjectsisafunctionofthefeaturesthattheobjectshaveincommon,minusthefeaturesthatdifferbetweenthem.The feature contrastmodel captures the intuitive idea that identical sharedfeaturesdecreasepsychologicaldissimilaritybetweentwoobjects,apropertyunat-tainablewithmetric-baseddifferences.Thisrelativelysimplemodelissufficienttoaccountforeachofthepreviouslymentionedviolationsofthemetricaxioms.

Oneof theprimarydifficulties involvedinmodelingpsychologicaldatais thattheperceptualphenomenathatweseektodescribeareconfoundedbythedecisionalprocessesinherentinanyexperimentalsituation.Theclassictheorydesignedtoteasetheseaspectsapart isknownasSignalDetectionTheory(SDT).This theorywasestablishedin1966byJohnSwetsandDavidGreen,buildingonearlierworkdonebyradarresearchers.Inthismethodology,participantsareaskedtodiscern“signal”

r = 1/2

1.00.0x

–1.0

0.0y

1.0

–1.0

r = 1: City Block r = 2: Euclidian

1.00.0x

–1.0

0.0y

1.0

–1.0 1.00.0x

–1.0

0.0y

1.0

–1.0

Figure 7.1 Eachgraphshowsallpointswithaunitdistancefromtheorigin.

Y116134.indb 149 10/10/11 2:11 PM


trialsfrom“noise”trials.Theexperimentercanthenfindvaluesfortheparticipant’sperceptualdiscriminabilityandtheirdecisioncriterion.Thesevariablescanbeinde-pendentlymanipulatedtostudyvariousdecisionalorperceptualqualities.InSDT,stimuliareconsideredtobeperceivedprobabilistically.Insteadofeachstimuluscor-respondingtoasinglefixedpointinsomespace,asinMDSmodels,stimulihavecorrespondingprobabilitydensityfunctions.

AtypicalexampleisshowninFigure7.2.Theleftcurvecorrespondstoanoisetrial, and the right toa signal trial.Theverticaldotted line representsadecisioncriterion,whereaparticipantwillchangefromcallingeverythingontheleft“noise”tocallingeverythingon the righta“signal”.Wecansee that theparticipantwillbeincorrectforthestimulibelongingtothenoisedistributionthatfallontherightof thecriterion (called falsealarms)andalso for the stimuli from the signaldis-tribution that fall to the leftof thecriterion (calledmisses).Shifting thecriterionleftorright(whichcanbeachievedbyalteringtheexperimentalinstructions)willresult in a tradeoff between these twokindsofwrong answers. Theonlyway toincreasethetotalnumberofcorrectanswersistoincreasethedistancebetweenthemeansofthetwodistributions,whichisreferredtoasd′.Thistheoryoffersanaturalexplanationfortheconfusionsbetweenstimuliwhilealsoelucidatingthedifferencesbetweenperceptualanddecisionaleffects,whicharemeasuredwithd′andthecri-terion,respectively.

Alimitationof theSDTmethodas typicallyemployed is thatstimuliareonlyallowedtovaryalongasingledimension,makingthemethodologyapplicabletoapaucityofpsychologicalexperiments.In1986,GregoryAshbyandJamesTownsenddevelopedamultidimensionalextensionofSDTthattheycallGeneralRecognitionTheory (GRT). In GRT, the probability density functions associated with stimuliare multidimensional, so stimuli can vary along as many dimensions as desired.Becauseofthedifficultyinvolvedinmakingfour-ormoredimensionalgraphs,letusconsiderthecasewherestimulivaryalongjusttwodimensions.Figure7.3ashows

0

0.0

0.1

0.2

0.3

0.4

2 4 6 8 10

Figure 7.2 AtypicalSDTgraphofsignalandnoisedensities.Thedottedlineisthedecisioncriterion.

Y116134.indb 150 10/10/11 2:11 PM


theprobabilitydensitiesoftwostimulithatvaryalongdimensionsxandy(Ashby&Townsend,1986).Theplaneshownpassingthroughbothfunctionsdescribestheequalprobabilitycontoursofthetwodistributions.Becausetheshapeoftheinter-section of this plane with a given density (a circle in this case) does not changedependingontheheightoftheplane(itwillonlyexpandorcontract),itisusefultographjusttheseintersections.ThisisshownforourexamplecaseinFigure7.3b.

Wecansee that this lattergraphcaneasilybemappedontoourearlierunder-standingofsignaldetectiontheory.Thedottedlineonceagainrepresentsthedeci-sioncriterion; theonlydifference is that itnowdependsonbothdimensions (thedecisiononthexdimensiondependsontheleveloftheydimension).Eventhoughthedecisionforeachdimensionisdependentupontheother,inthiscasebothstimuliareperceptually independent.Wemeanbythisisthattheperceptualeffectsofonedimensiondonotinfluencethoseoftheotherdimension.Ifthispropertywerevio-lated,wewouldseethatinthegraphofequalprobabilitycontours.Insteadofseeingcircles,whichportendperceptualindependence,wewouldseeskewedellipsesthatwouldpoint up-right for positive correlation anddown-right for negative.Havingpositivelydependentvariablesmeansthatthegreatertheperceptualeffectsareononedimension,thegreatertheywillbeontheother.

Itisimportanttonotethatperceptualindependenceislogicallydistinctfromhav-ingadecisioncriterionononedimensionthatisnotinfluencedbytheotherdimen-sion.ThislatterpropertyisreferredtoinGRTasdecisional separability.Athird,alsologicallydistinctformalizationoftheideaofindependenceiscalledperceptual separability.Thispropertymeansthattheperceptualeffectsofonedimensionarenotdependentontheother.Thisdefinitionsoundsquitesimilartothatofperceptualindependence,butthedifferenceisthatindependenceisawithinstimuluscondition,whileperceptualseparabilityisbetweenstimuli.

AlthoughGRTapproachesthemodelingofpsychologicalphenomenainafun-damentallydifferentwayfromMDS,becauseofitsgeneralityandversatilityithasbeenshownthatstandardEuclideanMDSisactuallyaspecialcasecontainedwithin

x x(a) (b)

y

yfA

fA f (x, y)

fB f (x, y)fB

f (x,

y)

Figure 7.3 (a)Three-dimensionaldensitiesoftwodistributions;(b)anequallikelihoodplotofthesametwodensities.

Y116134.indb 151 10/10/11 2:11 PM


theGRTframework(Ashby,1988).Whenconstrainedinthismanner,GRTwillnec-essarilyberequiredtoassumethemetricaxiomsthatweexaminedearlier.However,inthefullygeneralizedGRTmodelthereisnoneedtomaketheseassumptions.InGRT,theoverlappingregionsofmultiplestimulusdensitiescorrespondtowheretheyareconfusedwitheachother,ratherthanrelyingonadistancemetric.Becausetheseconfusionsareafunctionofboththemeansandstandarddeviationsofthedensities,theprobabilityofacorrectrecognitionisnotnecessarilymonotonicwiththedis-tancebetweentheperceptualmeans.Othermetricviolationscanalsobeaccountedfortoyieldanaccuratedescriptionofthedatainawidevarietyofcircumstances(e.g.,Ashby,1992;Kadlec&Townsend,1992;Thomas,1999,2003).

7.4 Dimensions: Finite and infinite

Theconceptofdimensionreallyonlybegantoassumearigoroustreatmentinthelatenineteenthcentury.Therearenowseveralmathematicalapproachestodimen-sionality.First,letusjustdefinespaceassomesetofpoints,whereapointisaprimi-tive,thatis,anundefinedentity.Thepointmayinfact,begiveninmoreimmediatelycomprehensibleterms,butneednot:oftenitcangainitsmeaningfulnessthroughalistofaxiomsaboutwhatstructureitexistswithin(e.g.,onthespaceitself)oropera-tionsthatcanbedoneonit,andsoon.Theeasiestapproachtounderstandisprob-ablytheonethatdefinesdimensionas“theminimumnumberofrealnumbersthatcanbeemployedtodefineapointinthespace.”4Thenaninfinite-dimensionalspaceisone thatrequiresadenumerable (i.e.,canbeput inone-to-onecorrespondencewithaninfinitesetofintegers)ornondenumerablesetofnumbers(e.g.,theirrationalnumbers,productsofsuchsetsandsoon),toindicateaspecificpointintheoriginalspace.Ofcourse,thisdefinitionrequiressomesortoffunctionthatrelatesthepointsinthespacetonumbers,andthatisfarfromalwaysthemostnaturaltacktotakewithsomespaces.Nonetheless,itisthemoststraightforwarddefinitionforourpurposes.Ourdefinitionofaphysicalorpsychologicaldimensionisthatitberepresentablebyapossiblyboundedintervalontherealline.

Infinite-dimensionalspaces,althoughfrequentlyobeyingmanypreceptsfoundinfinitespaces,sometimesdemandspecialcareandtactics,andoccasionallysimplyactinseeminglybizarrewaysrelativetofinitespaces.Anaturalquestionfromreadersislikelytobewhyweneedinfinite-dimensionalconceptsandespeciallyinspatialterms.Thusthereappeartobeafinite,ifunbelievablyhuge,numberoffundamentalparticlesintheuniverse.(Howeverparticlesaredefinedbymodernphysics,andinspiteoftheparticle-wavedualityofquantumtheory,andthedefinitionof“funda-mental”hasalteredover thepast centurywith, string theorynotwithstanding,noclearendinsight.)However,infinite-dimensionalspacesareanecessityfortheoryinmodernscience,includingphysics.Oneoftheargumentsforinfinite-dimensionalmodels is thatmathematicaldescriptionswhen thepointsnumber in themillionsorbillions,aresimplerormoreelegant,dependingontheusesof themodel.Thesamegoesforspacesofveryhighdimension.Thus,Newtonianmechanicsenjoystheartificeofcontinuoustrajectories(whereatrajectorycontainsanondenumerable

Y116134.indb 152 10/10/11 2:11 PM


numberofpoints)ofobjectsin,say,3-Dspace,althoughamodernquantumdescrip-tionmightlookquitedifferent.

Most functions that even high school students meet are defined on an infinitespace,thatoftherealnumbers(thelatterbeingnondenumerableaswesawabove,forit ismadeupofadenumerableset, theintegersplustherationalnumbersandtheirrationalnumbers).Furthermore,themostusefuloffunctions,suchasthesetofallcontinuousfunctionsontherealline,arethemselvesinfinite,withthelatterset having the same dimensionality as the real line itself! In fact, the ubiquitousappearanceofcontinuous(andoften,smooth,i.e.,differentiabletoahighorinfinitedegree)functionsinsciencebyitselfformsapowerfulargumentfortheinclusionofinfinite-dimensionalspacesinpsychology.Whetherornotthetheorist lookstofunctions, andwewelcome it (e.g., seeTownsend,Solomon,Wenger,&Spencer-Smith,2001;Townsend&Spencer-Smith,2004;Townsend,Aisbett,&Busemeyer,2005),evenakindofcommon-senseconsiderationofsuchobjectsasthesetofallfacesseemstocallforinfinite-dimensionalspaces(seeespeciallythefirstofthelistimmediatelyabove).Eventhoughthenumberoffacesonearthisfinite,itisclearlypossible inprinciple to create an infinitenumberof faces. Interestingly, even themodernapproaches toapproximationandnumericalanalysis,employasa funda-mentalunderpinning, thestructuresof infinite-dimensional functionspaces.And,psychologistsandstatisticiansarebeginningtoworkoutstatisticalproceduresandtheories thatareappropriatefor interestingsetsoffunctions(see,e.g.,Ramsay&Silverman,2006).

7.5 Inserting or extracting finite psychological dimensions into (out of) an infinite-dimensional space

Now,therearemanydirectionswecouldtakefromhere.Animportantoneis:Isitpossible to thinkoffinite-dimensionalspacesassub-partsof infinite-dimensionalspaces?Thiswouldbeaboonforpsychophysicalscalingbecauseitmeansthat itwouldnotbeunnaturaltothinkandworkwithfinitepsychologicalspaces,evenifthe“real”descriptiondemandsinfinitude.Infact,giventhatalmostallrealstimuliappeartobeobjectsfrominfinite-dimensionalspaces,itisclearthatour150yearsorsooffinite-dimensionalpsychophysicswouldlieonaveryshakyfoundationindeedif, say, a dimension such as visual area or hue were not, first of all, mathemati-callyseparableinsomesensefromthemorecomplex(infinite-dimensional)signalofwhich it is apart, and secondly, thebiologicalentity (e.g., the reader)wasnotabletopluckthis informationfromthesignal.Thus,visualareaof thesurfaceofadinosaurisnotonlycomputablefromthecomplicatedspaceofalldinosaursbutthevisual systemofahumancanapproximate that size, thusextricating thesizedimensionfromthispointofthatspace.Inthecaseofhue,theactualsignalmaybeacontinuumofwavelengthsfromthelightspectrum,yetthevisualsystemcomputesacompositehueobeyingthelawsofcolorperception.

Due to space concerns here (not infinite-dimensional!), we only concern our-selveswiththesituationwhereapsychologicaldimensioncorrespondstoaphysical

Y116134.indb 153 10/10/11 2:11 PM


dimension.Ofcourse,aphysicaldimensionitselfmayormaynotbeveryusefulinscience.Forinstance,productsofpowersofmeasurements,withthepowerbeingarationalnumberofbasicphysicaldimensions(usuallymass,length,andtime)consti-tutenewdimensionsbutonlyarelativelysmallnumberoftheseareusefulinphysi-callaws(see,e.g.,Chapter10inKrantzetal.,1971).

ConsideraspaceXthatisinfinite-dimensionalaccordingtotheabovediscussion.Then to elicit a physical dimension thatmight (ormight not)makeup a psycho-logicaldimension,wesimplyneedtomapaninterval(a,b)intoXasinf(x)wherexiscontainedin(a, b)andf(x)iscontainedinX.Wewritethismorecompactlyasf:(a,b)↣X.Althoughperfectlylogical,thisdefinitionofadimensionascontainedinXisnotveryintuitive.Tounpackthissituationabitmore,consideradepictionofinfinite-dimensionalspacesbasedonageneralizationofaso-calledCartesianprod-uctofdimensions.Thus,ℝ2=ℝ×ℝ,thecombinationof2selectionsfromthesetofrealnumbers.Wecanthinkofthesecombinationsasa2-placevector.

Thisconceptcanbeextendedfirsttoarbitraryfinitecombinationsofℝ,whereweusethepisymboltoindicateaCartesian product,thatis,allconceivablecombina-tionsasin

in

1∏

andinthisparticularcase,ℝi=ℝfori=1,2.Ingeneralthen,consideranindexsetthat,insteadofi=1,2,…,n,niseithercountably(i.e.,equaltothenumberofintegers)oruncountably(equaltothenumberofallrealnumbers)infinite.Insteadofusingproductsofℝ,wecanuseanykindofspace;thesespacesdonotevenhavetobethesameasoneanother.Wecaneventakeacombinationofsomefiniteandsomeinfinitespaces.Thenwehaveourspace

X X= ∏ α

α

,

whichmeansthatwetakeallthepossiblecombinationsofthespacesXα,onevalueforeachpossiblevalueofα. Inaddition to theapparentdescriptionofageneral-izedpointinthisspaceasaninfinite-dimensionalvector(withpositioninthevectorindicatedbythevalueofα),wecanalsothinkofitasafunctionf:α↣Xα.Forinstance,wheneachXαisℝandαisalsoinℝ,andfiscontinuous,theneachvec-torisamemberofthesetofallcontinuousfunctionsdefinedontheinterval(a,b).

Nowthatwehaveafairlyintuitiveideaofthishighlyusefultypeofspace,wecansimplyformanewfunctionthatassignsavalueinX foreachmemberoftheinterval(a,b).Inaspecialcase,itmightbethat,sayX237containsa1–1imageof(a,b), f: (a,b)↣X237, and thiseither isausefuldimension forpsychologists (e.g.,loudness)orphysicists,orboth.Sofar,evenwiththeforegoingexplanation,thesitu-ationmayseemabstractandoflittleuse.Butconsiderthemoreinterestingexample

Y116134.indb 154 10/10/11 2:11 PM


ofthespaceofallfacesandcallitF.WepostulatethatFiscontainedinthespaceofall2-Dsurfaces,consideredascontinuousfunctionsofaboundedsubsetofsomecross-productofintervals.Itwon’thurttosimplyposeourmappingasafunctionfromsomedomainDrealizedasthepairofintervals(0,1)×(0,1)=Dintothisfacespace,F.Wefocus,forsimplicity,onthefronthalfoftheface.Basically,wecandothesamekindofthingontherearofthefaces.

Letthefacefunctionbefβ:D↣F,overanon-denumerablesetβ,andwecanwithoutharmthinkofaparticularfaceasbeingassociatedwithaparticularmap,sayfβ.Eachfacewillbeadifferentfβmappingthedomainintothefacespacesuchthateverypointinthedomaincorrespondstoapointontheface.Figure7.4showsafaceconstructedinthisfashion.Now,wemustn’tlosetrackofourfirstgoal,whichwasindicatinghowpsychologicaldimensionscanbeexhibitedinaninfinite-dimensionalspace,suchasourcurrentfacespaceF.Supposewewishtoexperimentwiththepsy-chologicaldimensionofmouthsize(ms)andthesetofallmouthsizes:MS=ms,containednaturallyinℝ.Ofcourse, thereisatruephysicaldimensionofsurfaceareaofthemouthinthiscase,butthatmaywellnotbelinearlyrelatedtothepsy-chologicaldimension.Wecanincludems=0,eventhoughwemayneveractuallyobservethatinreality.Anyhow,weembedMSinFby,forinstance,takingallmem-bersofFbutwithoutamouthandthenformingthetotalF=F–mouth×MS.

This operation constructs all possible faces with all possible mouth sizes. Inactuality,somefacesmaynotaccommodatesome,perhapsparticularlylargemouthsizes.Thistypeofconstraintdoesnorealharmtoourdeliberationshere,andinfact,amoresophisticatedtheorybasedontheideaofmanifoldscanreadilyencompassthistypeofconstraint(see,e.g.,Townsendetal.,2001).Inanyevent,any1-DcurveinFcan,foreveryface-pointtraversed,beindexedbyavalueofms.Infact,anysuchtrajectorythroughFcanbemodifiedsoastomeasuremouthsizeateachofitspoints(i.e.,ateachface).Ofcourse,asubsetoftrajectorieswillpossessaconstantmswhileotherfacetsofthefacesvary,andsoon.Fromthereverseperspective,wecanthink

D

fβ

Figure 7.4 fβisthemappingfromthedomainDontoaspecificfaceinthefacespace.

Y116134.indb 155 10/10/11 2:11 PM


abouthowasattentionisdirectedtowardamouth,perceptualoperatorsarefilteringthepsychologicalmouthsizefromeachfacetowhichitisattending.

Thepresenttheorymayseemintriguing(ortoocomplextoexertanyeffortfor),butisthereanyhopefordisentanglingpsychologicaldimensionsfromaninfinite-dimensionalspace?Theanswerisapparently“yes”inanycaseswehaveseen.TheShepard–Kruskalapproachanditsmanyrelativesmightseemtoapplymostimme-diatelytostimulithatarethemselvesexhibitingandvaryingthefinitedimensionsunder study. A real-life, often employed case is a set of rectangles with varyinglengthandwidth(Schonemann,Dorcey,&Kienapple,1985).AlthoughtheShepard–Kruskalapproachhasneverbeengivenanaxiomaticunderpinning,nonethelessitseemsclear thathumansandprobablymany in theanimalworldcanattend toasmallnumberofdimensionsatatimefromstimulithatareinherentlyfromaninfi-nite-dimensionalspace,especiallywhenafinitesetof1-Ddimensionsarevaryingacross thestimuli. In fact,even thesetof rectanglescanbe interpretedasaveryspecialsubsetofthesetofall2-Dsurfaces.ThedimensionsofpsychologicallengthandwidthwouldthenbeextractedbytheShepard–Kruskalprocedures.Itshouldbenoted,though,thatsomeindividualsabstractthedimensionsofsize(length×width)and shape (length ÷ width). In addition, the fact that infinite-dimensional spacesare not incompatible with 1-D psychological dimensions helps weld together thevariouspsychologicaltasksandprocesses,fromunidimensionalpsychophysics(e.g.,Fechner,Adler,&Boring,1966;Falmagne,2002;Baird,1997;Link,1992)tohigherordermentalfunctionssuchassymbolandlanguageidentification(e.g.,Townsend&Landon,1982,1983;Pelli,Farell,&Moore,2003;Rumelhart&McClelland,1987),tocategorization(e.g.,Nosofsky,1986;Ashby,1992)andbeyond.

Perhapsevenmoreinterestingly,bybringingtobearprincipalcomponentanalysis(closelyrelatedtosingularvaluedecomposition),itisfeasibleinprincipletodissectasetofperceivedpatterns,evenfaces,intoasetofobjectsfromthesamedimensionalspace,thatcan(atleastsothewishgoes)beinterpretedasasetoffacesthatserveasafoundationfromwhichallthefacesinthestimulussetand,onehopes,manymore,canbebuiltup.Intheexamplecaseofourfacespace,thesefoundational“basis”facesarecalledeigenfaces.Morerigorously,thisapproachoriginallystemsfromthetheoryofvectorspaces.

Manyofthepresentreadersmayhavetakenabasiclinearalgebracourse,wheretheylearnedaboutsetsofindependentbasisvectors(possibly,butnotnecessarilyatrightanglestooneanother),weightedsumsofwhichcanproduceanyvectorinthespace(ofcourse,wehavetoomitalotofdetailhere!).Itturnsoutthat,say,thespaceofcontinuousdifferentiablefunctions,whichcanbemultipliedbynumbersandthenadded, subtracted,andsoon, formavalid,but infinite-dimensionalvector space.Nonetheless,thepropertyofinfinitudedoesnotruleoutthepossibilityoffindingasetofeigenvectors(inthiscase,actuallyeigenfunctions),infiniteinnumber,whichcan,withthepropernumberedweights,exactlyreproduceanyoftheoriginalstimu-lusfunctions.

Infact, incontinuedanalogywiththecommonvectorspaces towhichreadersareundoubtedlyaccustomed, therearemanywaysof selectingausually infinite,butdiscrete,setofbasisvectorsthatperformtheforegoingservice.Furthermore,it

Y116134.indb 156 10/10/11 2:11 PM


alsohappensthatthesefunctions(orinthepresentcaseeigenfaces)canbeorderedin their importance for producing human behavior, for instance, similarity judg-ments,confusionprobabilities,andsoforth.Hence,theinvestigatorwilltakeafinitesetoftheseeigenfacesasbeingthemostimportant,downtoacertainbutarbitrarylevelofprecision,anduse themashypothetical representationsof thebasicfacesofwhich“all”otherscanbereconstructed.Naturally,becauseweendwithafinitenumber, thesewillonlyapproximate theoriginal faces to somedegree.Anyhow,thisseeminglykindofexoticandunpromisingapproachhasactuallybeencarriedoutwithgreatsuccessbyanumberoflaboratories.AnauspiciousexampleisfoundintheresearchofAliceO’Tooleandhercolleagues(see,e.g.,O’Toole,Wenger,&Townsend,2001;Deffenbacher,Vetter, Johanson,&O’Toole,1998),whereeigen-faceshavebeenemployedtodiscovermanyintriguingaspectsoffaceperception.

Ouronlywordofcautionhereisthatoftentheeigenfacesmaynotresemblerealfacestoahighdegree.Thisfacetisimportantinouroveralldiscussion,inasmuchaseventhoughthespaceofcontinuousfunctionsisalegitimatevectorspace(e.g.,aweightedsumoftwocontinuousfunctionsisagainacontinuousfunction),thespaceofallfacesisnot.Thisisbecausenotallfaceswouldnecessarilyberepresentableassomesortofcombinationofbasisfaces.Thislackofnicevectorspacequalitiesleadsustoconsideramoregeneralframework,infact,onewherelocally,thatis,withinasmallregion,onehasvectorspacecharacteristics(infact,Euclideanproperties),butwhereglobally,thespacewillnotbeeitherEuclideanoravectorspace.Anaturalsettingtoconsideristhatofamanifold,whichisalsotheframeworkwithinwhichEinstein’s theory of general relativity came to be expressed. In order to help thereaderdevelopsomeintuitionfortheserichspaces,weactuallyneedtotakeastepbacktoratherprimitive,butextremelyusefulconcepts,suchasatopology.

Finallyforthistopic,weobservethat thequestionofindependenceofpsycho-logical dimensions has long been of interest in perceptual science. For instance,thedimensionsof loudnessand intensity(commonly,andsomewhatoddly,called“volume” inEnglish)aremutuallydependent,althoughtheirphysicalsourcesarenot.Withinamorecomplexsetting,socialpsychologistshavefoundthatperceivedintelligence and attractiveness are correlated although, of course, they are not inreality.Ashby&Townsend(1986)proposedageneraltheoryofperceptualdepen-dence among psychological dimensions embedded in a multidimensional patternrecognition setting.Many supplements to the theoryandassociatedmethodologyhave been made and applications in various areas of perception and action (e.g.,Kadlec & Townsend, 1992; Thomas, 2003; Maddox, 2001; Wenger & Ingvalson,2003;Amazeen,1999).

7.6 Manifolds for psychological spaces

Weputasidefornow,thepropertyofhavinganinfinitenumberofdimensionstopursueadifferentcourse.OfthevirtuallyinfinitenumberofpossiblegeneralizationsofEuclideangeometry,thereisanotheronethatstandsoutwhichhasbeenhardlyexplored at all in psychological domains. We are referring to “manifold theory”.Theconceptofaspaceplusatopologyissoimportantthatwehavetodealwithit,

Y116134.indb 157 10/10/11 2:11 PM


atleastinformally,beforebeginninginearnest.First,aspaceisanyset(collection)ofentitieswecallpoints.Thesecan,asinthekindsofspaceswithwhichthereaderis likelymost familiar,beapproximatedbydots. Inmorecomplicated situations,however,suchasthespaceofallcontinuousfunctions,apointwouldbeanentirecontinuousfunction.Thespaceisthenthecollectionofallofthesepoints.Anyhow,amathematicianusuallyquicklyimposesmorestructureonthespace.Thiscatapultsustotheideaoftopologicalspaces.

A topological spaceX isa setofpointswitha specialcollectionof subsetsofpointscalledbasis sets.Onethentakesallpossiblefinitenumbersofintersectionsofthesesetsalongwithallpossiblefiniteorinfiniteunionsofthesesetstoproducetheso-called“setofopensets”(wealsomustincludeX,thesetofallpointsthatmakeupthespace,plustheemptyset,Ø).Thatis,everysetinthispossiblyverybigsetofsetsisbydefinitionopen.Theso-calledclosedsetscanthenbeelicitedbytak-ingthecomplementX–Oα=CαwhereOαisopenandCαisthenclosed.Withthisstructureandlittleelse,onecanimmediatelydefinecontinuousfunctionsfromonetopologicalspaceXtoanotherYalongwithmanyothervaluableconcepts.IfXandYaretopologicallyequivalent,onecanfindabicontinuousfunctionthatcarrieseverypointinXtooneinYandviceversa,andonealreadyhasthejustificationfortheoft-repeatedtopologyjokethat“Atopologistisapersonwhocan’ttellthedifferencebetweenateacupandadoughnut,”because,ofcourse,thestretchingandshrinkingallowedinsuchafunctionpreservesthetopologicalproperties(e.g.,howmanyholesaspacepossesses)ofeitherspace.Topologyisextremelypowerfulfortherelativelysmallsetofassumptionsonwhichitrests.Forevenmoreinterestingproperties,wenowproceedtotopologicalmanifolds.

TheessenceofatopologicalmanifoldMnrestsonthreevitalproperties:

1.MisHausdorff.Thismeansthatforanytwopoints,twosetscanbeformedcontainingthosepointswhichhavenointersection(overlap).

2.Mhasacountablebasis. 3.MislocallyEuclidean.Thatis,anypointinMiscontainedinasmallopen

setOM (calledaneighborhood) thatcanbemappedbicontinuously(boththeoriginalfunctionanditsinversearecontinuous)andina1–1(eachpointinOMmapstoexactlypointinanopensetOEofEn)andonto(allpointsinOhaveaninversepointinOM)fashion.

This means that a small region of our manifold can be treated approximately(andtotallyinthelimitastheoriginalsetgetssmallerandsmaller)likeaEuclideanspace.WecannowusetheEuclideanmetriclocally,

d x x x xi i

n

1 2 1 22

1, , ,( ) = −( )

∑

wherex1=(x1,1,x1,2,…,x1,n),ann-placevectorinEuclideann-Dspace,andsimilarlyforx2.Itcanbeshownthatatopologicalmanifoldcanalwaysbegrantedaglobal

Y116134.indb 158 10/10/11 2:11 PM


metric. However, without more assumptions, that metric may not be the one wewish,aRiemannianmetric,namedafterthegreatmathematicianBernardRiemann.Riemannfurtheredthedrivetowardnon-Euclideangeometry,andworkedoutatypeofmetric thatgeneralized theone inventedbyanothermathematicalgenius,KarlFriedrichGaussforspecialcases,andwhichincludedasspecialcases,thoseposedbyLobachevskyandBolyai(negativecurvature),butlongbeforetheinventionofthecurvatureconcept.Riemann’smanifoldsandhismeasurecapturebothpositiveaswellasnegativecurvature.

Somebrightmathematiciansmetthisneedandbestowedthedriveforageneral-izedmetricwithgreat force,by inventingakindofdifferentiation thatworksonmanifoldsinsteadofjustinEuclideanspace.Thistopicismuchmorereconditethanwecandetailherebutbasicallywecanviewtheoperationsinasimplecasetogetafeelingforwhatisgoingon.Basically(andveryroughly),wecanthinkofbundlesofvectorsthatsitonamanifoldandstartingatanysinglepoint,directustothenextpoint.Withsomecare,wecanprovidedifferentiationoperationson thesevectorssuchthatthe“output”ofthedifferentiationoperatorremainsintheoriginalmani-fold(e.g.,inamanifoldthatappearsasasurfaceinℝ3,thederivativevectorsmightstickoutawayfromthesurfacewearestudying).Withoutcaretakentoconfinethemtothemanifoldathand,wecan’trepresentspeedandaccelerationtoprovideforaNewton-likemechanicsinournovelspace.

Closelyassociatedwithoursetupisamatrix(orquadraticoperator) thatmapsvelocityvectors intoaspeednumber.Thisresultcanbe integratedoverapath toproduce a path length.Evenbetter,we canput conditionson thepaths such thatwe can evoke the closest possible analogue to straight lines in Euclidean spaces(formallycalledgeodesics).WedefinetheRiemannianmetricasann ×nmatrix(wherenisthedimensionofthemanifold),whichisusuallywrittengij(x),withi, jrunningfrom1tonandxbeingapointonthemanifold.Ittellshowfastdistanceisaccumulatingatdifferentplacesinthespace(signifiedbyx)andhowthedifferentdimensions, i and j, affect thataccumulationateachpoint.For instance, supposethatourmanifoldisshapedlikeamountainandthatwewishourmetrictotakeintoaccountnotonlydistanceinourusualsensebutalsoperhapstheeffort(e.g.,power)thatisexpendedinmovingaroundonthismanifold.Hence,whenthegradeisfairlyflat,wecanmakegij(x)smallbutwhenthegoinggetssteepwecanmakeitmuchlarger.Supposetoothatourjourneystartsatapointwhere,whenwemoveinthedirectionofbothincreasingdimensioniaswellasdimensionjweareheadingupthesteepestpartoftheslope.Ifwemomentarilyholdonedimensionconstant,theeffortisconsiderablyless(duetonotactuallyclimbingupward).Thissituationispicturedwitha largegij(x)closeto thestartingpointandgii(x)andgjj(x)beingsmaller(inEuclideanspace,thesearetheonlyentriesinthematrixthatareappropriate,andallareequalto1forallpointsx).

Notethestronginteractionbetweendimensionshere,whichisabsentinEuclideanspace.Interestingly,this(relatively)simplefunctionisallthatisrequiredtoinvesti-gatethecurvatureinourspace,afactthatwasprefiguredbyGaussandmadegeneralandrigorousbyRiemann.(GausswasaseniorprofessorandsatonRiemann’shabili-tationexam,whichissomethinglikeanadvanceddoctoraldissertationinGermany

Y116134.indb 159 10/10/11 2:11 PM


butcurrentlybeingdissolved)Evenwiththisbrief(andquitenonrigorous)tutorial,wecanperhapsseethatmanifoldsincreasethegeneralityandscopeofthekindsofspaceswecouldconsiderforpsychologymagnificently.

Asasimple,finite-dimensionalexample,considerthetraditionalshapeusedincolortheorytocapturesomeimportantaspectsofcolor:A.Hue;B.Brightness;C.Saturation(oritsopposite,Complexity).Whenwedon’tworrytoomuchaboutthepreciseshape,thesurfacelooksliketwoicecreamconeswiththeirtops(thebiggestpartofthecone)stucktogether.Thisisamanifoldthatcan,ofcourse,beembeddedinEuclidean3-Dspace.Themiddle,widestportion isused todepict theoptimalbrightnesslevel,wherethefullrangefromgray(intheverycenterofthecircle)tothebrightesthuesispossible.Aswedescendtowardthebottomapex,allisdark,andupattheotherapexthestimuluslightissobrightthatnohuecanbediscerned.Thislittlespaceisverywellbehaved(exceptattheapexes,wherestrangegeomet-ricthingshappen,soweputthoseasidefornow).UsingRiemann’stechniques,wequicklylearnthatalthough“inthelarge,”thespaceisnotgloballylikeaEuclideanspace(e.g.,itisboundedratherthanunbounded),locallyaroundagivenpoint,itscurvature is0 just like inEuclideanspace. Intuitively, this isbecauseourdoublecone,withacutfrombottomtotop,andonethroughthecenter,unrollstomakeaportionofaEuclideanplane.

7.7 Probability theory in infinite-dimensional spaces

So now we are in possession of elementary topology, manifolds, metrics, differ-entiability(whatmathematicians like tocallsmoothness),andfinite-andinfinite-dimensionality.Nextweneedawaytoinducerandomness, thatisprobability,onourfinite-or infinite-dimensionalmanifolds.Fortunately, there isawell-troddenpathwaythatallowsustoretainourdesirablegenerality.Thiscomesbywayofuti-lizingtheresidenttopologytoourownends.First,weobservethatprobabilityisaformoftherigorousconceptofmeasure.Ameasureisconstitutedbyafunctionthatmapssets(oftencalledclasses)ofsetsintotherealnumbersinaregularfashion.Theconditionstosatisfythetenetsofameasure,andagainwecannotreachdetailhere,arethingssuchas(1)additivityofthemeasureofnon-overlappingsetsinthetopol-ogy,and(2)finitenessofthemeasure(i.e.,themeasureisalwaysboundedbysomefixedrealnumber).Thereareseveralothersthatwedon’ttalkabouthere,butwiththeaddedstipulationthatifwehaveaprobabilitymeasure,themeasureontheentiresetofpoints(i.e.,thepointsinthetopologicalspace)isequalto1(i.e.,somethinghastohappen).Hence,wesimplyformanappropriatetopology,thentakeitssetsandassignaprobabilitymeasuretothem.

Aclassofsets (alongwithcertainoperations)meeting theseandother techni-cal tenets is called a sigma field. Basically, one can assess the probability of aneventbycomputingthemeasureassociatedwithappropriatesetsinthesigmafield.Thisgeneralargumentappliestofinite-orinfinite-dimensionalspaces.Asmightbeexpected,peculiaritiescanariseinthelattercase,butusuallytheydon’tundulyper-turbthepathwaystothetheorist’sgoals.Thesedeliberationsshouldconvinceusthatinfinite-dimensionalspacescanpossessprobabilitydistributions,particularlywhen

Y116134.indb 160 10/10/11 2:11 PM


ametricispresent.Inmanycases,someofwhicharestandardinsuchfieldsaselec-tricalengineering(althoughtherigorousunderpinningsmightbefoundonlyatthegraduate-schoollevel),onemayemploytoolsfromotherfields,suchasfunctionalanalysis,orstochasticprocesses,toavoidtheexplicitproductionofanappropriatesigmafield.Wediscussmoreonthistopiclater.

Anytopologicalmanifoldcandothis,butweareespeciallyinterestedin(infi-nitelydifferentiable)Riemannmanifolds,whichareallmetrizeable,andinfact,wewishtoonlyworkwiththosethatarecompletemetricspaces.Anicepropertyofmetricsisthatanymetricgeneratesatopologybutnotalltopologiesadmitametric.Thefinitecasesaredealtwithelsewhere(e.g.,Townsendetal,2001),sowewanttoseewhathappenswithinfinite-dimensionalmanifoldsorotherinfinite-dimensionalspaces.Perhapsthemoststraightforwardinfinite-dimensionalspaceisthespaceofallcontinuousfunctionsonanintervalIontherealline,whichmayitselfbeinfinite(i.e.,[0,1]),sayf:I↣ℝ.Ofcourse,thegraphofeachofthesefunctionsyieldstheusualpictureofatypicalfunctionastaughttousinelementarymath.Interestingly,ithasbeenshownthatthisclassofobjects(i.e.,thesetoffunctions)isofthesamedimensionality(thejargontermiscardinality)asthepointsonthereallineitself!Therearelotsofmetricswecouldusebutanextremelyusefultypeistheso-calledL2 metric

d f g f x g x dx

i

, ( ) ( )/

( ) = −( ) ∫ 2 1 2

0

,

wherefandgaretwosuchfunctions.ItshouldbeobviousthatthisistheanalogueoftheEuclideanmetricinfinitespaces.Evenmoreintriguingisthefact that it istheonlypowermetricinfunctionspacethatsatisfiestheconditionsrequiredtobeaRiemannianmetric.Thus,theinfinite-dimensionaltwinofthecityblockmetric(seeabove),

d f g f x g x dx, ( ) ( )( ) = −( )∫0

1

isnotatypeofRiemannianmetriconanymanifold.Webackupforamoment inorder toexamine thefiniten-dimensionalmetric

andapathlengthinn-dimensionalspace.Thegeneralinfinitesimaldisplacementinn-dimensionalRiemannianmanifoldis

ds g x x x dx dxij n i j

i j

n

= …( )

=∑ 1 2

1

, , ,,

1 2/

.

Observethatgijissimplyann×nmatrixandthatitcandepend,ingeneral,onthespecificpointinspace,(x1,x2,…,xn),wherewearecurrently.However,inthe

Y116134.indb 161 10/10/11 2:12 PM


specialcasewheregij=I,theidentitymatrixatallpointsinthespace,thisexpres-sionreduces to theEuclideanmetric.Likewise, thepath lengthofapath throughEuclideann-spaceis

g x x x dx dxij n i j

i j

n

1 2

1

, , ,,

…( )

=∑

11 2/

∫∫

wheren integralsaretakentrackingtheappropriatepath.Perhapsit isevenmoreintuitivewhenthedisplacementsdxiareconvertedtovelocities,dxi/dt,forthenwecansimplyconsiderthepathintermsofthevelocitiesofeachcoordinate.Hereweevaluate,saydxi,asbeingatinymotioninthexidirection,fori=1,2,…,n–1,n.Thisobviouslypushesustoanewpoint(x ′1, x ′2, …, x ′n)wheregcantakeonanewvalue, and so on. All this will come in handy in the infinite dimensional casejustfollowing.

Moving back to functions in one variable, inasmuch as it is more intuitive toconfineourselvestothis“spaceofdifferentiablefunctionsinonedimension,”letusindicatethegeneralRiemannianmetriconthiselementarymanifold:

ds g h f t t f t t dx dxa= ∂ ∂( ) ∂ ∂( )∫∫ αβ βα β( ) ( , ) ( , )0

1

0

1

′

∫1 2

0

/

dt

t

.

Now,wehavetoadjustourthinkingabit.Insteadofconsideringdxaasanminis-culemotioninthexadirection,wecanthinkofitinthefollowingway.TakealookatthetwofacialprofilesinFigure7.5,drawnasafunctiononxfrom0to1.Now,foreveryα,xaisanumberbetween0and1.However,wenowneedtothinkofdxaasamotionfrom,sayfacef1tofacef2,thatis,movingupordownverticallybetweenthetwodesignatedfaces.Becauseofthepossibilityofnoise,wecanonlyassumethateachfunctionalongthepath,thatis,his“face-like”,notthatitisnecessarilyatrueface.Indeed,inapatternidentificationsituationoneorbothoff1andf2could

Smaller Mouth size Bigger

Figure 7.5 Hereisaseriesoffacesthatvaryacrossoneisolateddimension,mouthsize,butthereareinfinitelymanyotherdimensionsinthespaceofallfaces.

Y116134.indb 162 10/10/11 2:12 PM


benoisyrenditionsoffaces.Theextracomplexityheremustbeexplainedabitmore.Wearerepresentingafunctionasagigantic(anddense!)vector,γ=f (x), x∈ ℝ,andbecausefisourcurrentpoint,wehavetoletthemetricgdependonit,whichiswhyitappearsintheargumentofg.Inaddition,insteadofdxiforafinitei=1,2,…,n,wemustextendthisnotationtodxawherearunsovertheinterval[0,1],andthesameistrueforβ.So,theoverallideaisthatwearedepictingthemovementoffbywayofhowitchangeswitheachcoordinateof f.And,becauseweareusingageneralquadraticmetricofthecoordinates,whetherfiniteorinfinite,wehavetotakethetwo-wayproductsofallpossiblecoordinatechanges, that is,dxaanddxβ.Finally,thenthepathlengththroughthisfunctionspace,explicitlyusingourvelocityrepre-sentation,isjust

d f g g h dx dt dx dtR , ( )( ) = ( )( )

∫∫ αβ α β

0

1

0

1 1 22

0

dt

t*

∫ .

Ofcourse,whenweseektheshortestpathlengthbetweentwofaces,wecandefinethatasthedistanceandtheensuingpathasthegeodesicbetweenthem.Again,thereadermaywishtoconfirmthatwhenα,βarebothidenticalto1anddon’tdependon f,wegetback to theanalogue to theEuclideanmetric, theL2 metric (andseeTownsendetal,2001,formoreonthisandotherissuespertainingtoface-geodesics).Inanyevent,allthiscanbe(withsometedium)expandedtofunctionsinanyfinite-dimensionalspace.Itisimportanttoobservebeforewegofurtherthatjustbecauseweusetheterm“path”here,asiscommon,wedonotimplyatemporalfactor.Alltheabove,forinstancecomputingthedistancebetweentwofaces,mighttakeplacesimultaneouslyalongthepath,althoughsomepathsmightnonethelesstakelongertocomputethanothers.Amoreabstractandgeneralnotationispossible(e.g.,Boothby,1975;Townsendetal,2001).Suchnotationsareveryusefulbecausetheycapturetheprincipalideasinamarkedlyclearfashionwithoutsometimesmind-numbingprofu-sionsofindices.Ontheotherhand,whenthescientistwishestoactuallycomputesomething,theevilindicesmustbepresentandaccountedfor.

7.8 A special case of great importance

Ofcourse,infinite-dimensionalmanifoldshavenotyetseenmuchapplicationinmostofscience,especiallythelifesciences.Yet,itseemsworthwhiletomentionaspecialcaseofourquantitativeapparatusthathasbeenofenormousvalueinthebasicandappliedsciences.Thetheorygoesbymanynames,butonetermisthe“mathemati-cal theoryofcommunication”(whichwesimplify toMTC),used forexamplebyNorbert Wiener. As worth virtually everything, including mathematics, there areroots in the distant past for this theory, but a real explosion of new results frommathematicians,physicists,andengineers,occurred in themid-twentiethcentury.Toamajorextent,thisworkaccompaniedtheamazingscientificandtechnologicaleffortexpended inandaround the timeofWorldWar II.Werecall thenamesof

Y116134.indb 163 10/10/11 2:12 PM


formidablemathematicianssuchasNorbertWiener,JohnvonNeumann,andelectri-calengineerClaudeShannon.Thesphereofmathematicalcommunicationstheoryintersectshugeregionsofmathematicsandphysics, includingfunctionalanalysis,stochasticprocesses,andprobability,waveformanalysis(e.g.,FourierandLaplacetransforms),electricalengineering,differentialequations,anddeterministicandsto-chasticdynamicsystems,andsoon.Inthepresentwriters’opinion,thewholeofthisworkisasimportanttomoderntechnologyandscience(ifnotusuallysolethalinapplication)asthesplittingoftheatom.

In most applications of MTC, we employ various types of function spaces.These may be deterministic or probabilistic, but usually do not require manifoldtheoryperse.Whenneeded,differentiationcanbeextendedtoso-calledFréchetorGauteauxderivatives,andusuallyordinaryLebesgueorRiemannintegrationsuf-ficesforintegration.Formanypurposes,thefunctionspaceisitselfavectorspace,sometimeswithanorm(asinBanachspaces:thenormofafunctionistheanalogueofthemagnitudeofavectorinafinite-dimensionalvectorspace)oraninnerproduct(theanalogueofthedotproductinaEuclideanvectorspace).Theinnerproduct,theanalogueofthedot(orineptlycalledbynon-mathematiciansthe“crossproduct”)oftwofunctionsofarealvariableisjust

f x g x dx( ) ( )−∞

∞

∫

The accompanying metric, which by now will probably not shock the reader,issimply

d f g f x g x dx, ( ) ( )

/

( ) = −( )

−∞

∞

∫ 2

1 2

,

which naturally looks very much like our good friend, the Euclidean metric, butstretchedoutalongthereal-linecontinuum.Wecanthenthinkof

d f g g f dx dt dx dtR ,( ) = ( )( )( )

∫∫ αβ α β

0

1

0

1 1//* 2

0

dt

t

∫ ,

whichweformulatedabove,asweightingdistinctpartsoffiguressuchasfacesdif-ferently,dependingontheirimportance.Forinstance,itiswellknownthattheeyesplayacriticalroleinfaceperceptionandhencewewouldexpectgαβtobelargewhentransversingthatpartoftheface.

Y116134.indb 164 10/10/11 2:12 PM


7.9 Applying these ideas to classification

Now,supposethatapersonisconfrontedwithapatternfromaninfinite-dimensionalspace.Aspointedoutabove,almosteverythingseenorheardisfromsuchaspace,notafinite-dimensionalone. Inapplying theaboveconcepts to this situation,wehavetomakesurethatourspaceisbigenoughtocontainnotonlypatternsarisinginanoiseless,perfectperceptionenvironment,butalsothepatternsthatareperturbedbysometypeofnoise,orthataresimplyrandomizedinsomeway.Thus,thetheo-reticalsystemweproposeisananaloguetothefactthatprobabilisticidentificationinanoisyenvironment(e.g.,Ashby&Townsend,1986),demandsaverysimilartypeoftheoreticalstructureasdoescategorizationofasetofpatternsthatareassociatedwithaprobabilitydistribution(e.g.,Ashby,1992).First,letusagreethatsomekindof representation isconstructedofeachpattern tobeclassified,whether it iscat-egorizedbyasetofexemplars(seeNosofsky,1988)oratemplate.Inidentification,itmightbeamemoryofeachobject tobeuniquelyidentified.Letusconcentrateonthecaseof identification inasmuchasonecanreadilygeneralize thatsituationtocategorizationorreduceit tosignaldetection.ThereadermayrefertodetaileddevelopmentsanddiscussionofthismaterialforthefinitevectorcaseinTownsend&Landon(1983).

Letusconceiveoftheinputasarandomfunctioninsomen-dimensionalspace(e.g.,aperturbedtwo-dimensionalsurfaceplusnoiseplacedinathree-dimensionalEuclideanspacewithanL2 metric).ItisstandardtouseanL2 metricizedfunctionspaceforphysicalsignalssowecontinuethathere(butcf.Cornish&Frankel,1997).LetUibetherandomfunctiondescribingtheprobabilisticinputforstimulusSiandU =Ui1

NthesetofNrandomizedinputs.WetakeXastheperceptualimagespace.Xisconsideredasaninfinite-dimensionalfunctionspace,butmaybeembeddableinsomefinite-dimensionalspace.So,theperceptualmapisUi→X,andeachrealiza-tionfofthesetF,whereF ⊑ Xisassumedtobecontinuous.ThememorypatternsetY,withwhichtheinputismatched,istakenasadeterministicsetoffunctions.Thatis,g∈Yiswellmemorizedwithnonoiseattached.Incertaincases,nothavingbothberandomcansubstantiallysimplifymatterssuchascalculations.

Next,weneedanevidencespaceZtorepresenttheevidenceforeachresultingcomparisonofan inputwith theNmemory items fromY.The simplest,yetverynaturalspace toemployhere is justℝ+, that is, thepositiverealnumbers.Hence,wemusthaveamape:(F,G)→ℝ+

0,thatis,e( f,g)=r ∈ℝ+0,withthe“0”subscript

indicatingthat0isincluded(naturally,thissetcaneasilybegeneralizedtoincludethenegativerealnumbersifrequired).Theevidencefunctioncouldbecalculatingdistance,Bayesianlikelihood,orsomeothermeasureofacceptabilityofamemoryalternative. In any event, z ∈Z = ℝ+

0 will be a real-valued random variable, andtheimageofaparticular( f,g).Now,withinXeverypointfunction,exceptforties,willbeinfavorofoneresponsealternativeovertheothers,saythej th.Then,foraperceptualsignalfinsuchaset,e( f,gj)=MAX1

n[e( f,gk)],againexceptforties.Thismeansthatoutsideofties,Xwillbepartitionedintosetsofpoints(i.e.,functions)

Y116134.indb 165 10/10/11 2:12 PM


thatfavorsomeoneofthealternatives.ThentheprobabilityofrespondingRjwhenthestimuluswasSirequiresdeterminingtheprobabilityassessedacrossthevariouspercepts,f,thatcanappearwhenSiispresented.ThisprobabilitywillbeP(Rj∙Si)=P[e( f,gj)=MAX1

n[e( f,gk)]Si].Interestingfactsabouttheevidencefunctions:

1.Asnoted,underveryweakconditions(e.g.,thereexistpointsinXwheree( f,gk)favorsRkforallk;andsoon)ewillpartitionthespaceintomutuallyexclusiveregionsthatfavoreachofthealternativespluspointswheretiesmayoccur.

2.Considerpairsof, say,giandgj,distinctmemorypatternsand thesetoff∈Fsuchthath( f,gi,gj)=e( f,gi)–e( f,gj)=0.Then,inmostcircumstances,theborderseparatingwhetherRiwinsversuswhereRjwins,willbeaclosedsubmanifold in X. We can call the set f such that h( f,gi,gj) = e( f,gi) –e( f,gj)=0, thekernelofh, andagain theyare equidistant from facesgiandgj.On either sideof this boundaryone or theotherwins.Now, thesame things happens when any pair is considered and we can also lookfortiesamongthree,fourormoreofthesetofNfaces.Underfairlyweakconditions,thedistinguishedsetf(e.g.,designatedbythetieddistance)willevenbeanicesub-manifoldoftheoriginalfacespace,inheritingitstopologyfromthatoftheparentspace.Sometimestheregionoffacespacewhere,say,giwinsoveritscompetitors,willbeconnected.However,devo-teesofsignaldetectioninasingledimensionwillrecognizethateventhere,the region of points where YES dominates the NO decision will not beconnected:Whenthedistributionsofsignal+noise.versusnoise-alonearenormalwithunequalvariances,thisdisconnectionalwaysoccursunderamaximumlikelihooddecisionrule.Yetanothernicepropertyensuesiftheequidistantboundariesallhaveprobability=0ofoccurring,for thenwedon’thavetoworryaboutjumpingoutsideofourevidencespacetoadjudi-cateties.Thisisquiteanaturaloccurrenceforfinite-dimensionalspaces.

Whathappenswhenthesituationisstochastic?Supposethehumanobserverorsignalprocessor,let’scallherSheila,isatleasttoafirstapproximationalinearfilter,andthatsheisattemptingtorecognizeoneofNpatterns.Supposethattheobserverisdeterministic(i.e.,herfilteringmechanismsactthesamewayeachtimetheyarecalledintoplay)andthataspecificsignal i (i=1,2,…,N) is itselfacontinuousfunction(si(t))withGaussiannoise(–(t))addedin.Then,thesignalpatterncanbeexpressedasU(t)=si(t)+–(t).Theobserver’sfilterforeachsignalpossibility(e.g.,thej th)canbewrittenalsoasafunctionoft,hj(t).Itturnsout(see,e.g.,Luenberger,1979;Padulo&Arbib,1974)thatheroutputonherj thperceptualtemplateis

x t h t t s t N t dtij j i

t

( ) = − ′( ) ′( ) + ′( ) ′∫

0

,

AU: "some" or "one" of the alternatives?

Y116134.indb 166 10/10/11 2:12 PM


undersomereasonableconditions.Thatis,Sheilaisusingatemplateacrosstimerep-resentedbyhtofilterorcomparewiththeinput.Infact,whenthenoisehascertainproperties,hj(t)=sj(t),whichmeansthefilter-templateisareplicaofthej thpatternitselfandthefilteringactionisbasicallyacorrelationoftheinputwitheachoneoftheseNstoredreplicas.Thisactionproduces

x t s t s t N t dtij i i

t

( ) = ′( ) ′( ) + ′( ) ′∫

0

.

There are several decision structures that couldbe imposed, but givenour spaceconsiderations,we take themoststraightforward:wesuppose thatSheilasamplesinformationforafixedinterval,say,[0,t*]andthenselectsthemaximumcorrelationprovidedbyxij(t*),ascalculatedacross j foragivenpresentsignal i.Undersomerestrictions,thisstrategyisoptimalandiscalledamatched filter.Indeed,if–(t′)isstationaryGaussianwhitenoisewithvarianceσN

2=1andmeanμN=0,xij(t)willbeitselfnormalwithmean

µx i

t

s t dt= ′( ) ′∫ 2

0

andvariance

σ x i

t

s t dt2 2

0

= ′( ) ′∫theverysamething!11

Whathappenedtoourmetric?Well,withthesamesamplingrulebutnowusingthesimplestRiemannianmetric,theL2metric,wewouldcompute

d s t x t s t s t N ti ij i j′( ) ( )( ) = ′( ) − ′( ) + ′( ) , * ′

∫

2

0

1 2

dt

t* /

.

Nowthisquantityismonotonicwithitssquarewhichiseasiertodealwithsoweexamined2(si(t′),xij(t*))instead.Wefindthatitsmeanofexpectation(signifiedbytheoperatorE)is

µd i i j j iE s t s t s t s t s t s= ′( ) − ′( ) ′( ) + ′( ) − ′( ) −2 22 2 jj

t

i

t N t N t dt

E s

′( )( ) ′( ) + ′( ) ′

=

∫ 2

0

*

22 2 2

0

2′( ) − ′( ) ′( ) + ′( ) + ′( ) ′t s t s t s t N t dti j j

t

**

∫

Y116134.indb 167 10/10/11 2:12 PM


duetothefactthatthemeanof–(t′)=0.Next,

s ti

t

2

0

′( )∫*

isaconstantandthesameforallcomparisons(recallthats(t′)isthepresentedsig-nal)and

N t dt

t

2

0

′( ) ′∫*

isarandomvariablebutisalsothesameonanyonetrialforallthecomparisonsandsoneitherofthosequantitiescandiscriminatethevariousdecision/responsealterna-tives.Hence,theonlyoperativequantityis

− ′( ) ′( ) + ′( ) ′∫ 2 2

0

s t s t s t dti j j

t*

withmean

− ′( ) ′( ) ′ + ′( ) ′∫ 20

2E s t s t dt E s t di j

t

j

*

tt

t

0

*

∫ .

Notice,inparticularthattheterm

− ′( ) ′( ) ′∫ 20

s t s t dti j

t*

isdirectlyproportionaltothecriticaltermfromourmatchedfilterapproachabove,namely involving the integral of si(t′)sj(t′).Therefore,minimizing thedistance istantamounttomaximizingthecorrelationbetweentheinputandthedecisionalter-natives.Theonlydifferenceisthatourdistanceapproachincludesabiasingterm,

s t dtj

t

2

0

′( ) ′∫*

,

whichprovidesarelativebiasforalternativejaccordingtoitsenergyormagnitude.Wewon’tdetailthederivationofthevariancethistimebutitisalsocloselyrelatedtothatofthematchedfilterexpressions.Interestingly,boththeserulesarealsoequiva-lentundercertainconstraintstoarulebasedonmaximizingthelikelihoodthatthepresentedsignalpatternwasj,giventheobservation.

Y116134.indb 168 10/10/11 2:12 PM


7.10 Conclusion

Ouritineraryonthegeometricaspectsofpsychologicalclassificationhastakenusfromfoundational,axiomaticmeasurementtheorythrough(finite)multidimensionalscaling,toconceptsofdimensionality,includinginfinitedimensionalspaces.Fromthere,wereassuredourselvesthatfinitedimensionalsubspacesofinfinite-dimensionalspacesarelegitimatemathematicalconcepts,meaningthatmodelswhichperformdimensionalreductioncanreadilybeappliedashypothesesaboutthehumanabstrac-tionofpsychologicaldimensions.Thisissomethingthatistakenforgranted,butitstruthisnotatallobvious,althoughwecertainlyfindsomewaytofilterinterestingdimensions.Next,wemovedontoideasthatarestillrelativelynewinmathemat-ics(i.e.,onlyaround150yearsold),thatofnon-EuclideanandRiemanniangeom-etries, and even more ambitious, infinite-dimensional Riemannian spaces. In thesocialandbiological sciences,probabilityandstochasticsareanecessity,andwebriefly surveyed the prospects for placing probability distributions on finite- andinfinite-dimensionalmanifolds. Itcanbesaid thatmathematiciansarestill inhotpursuit of the best ways of carrying out this program. Nonetheless, the next twosectionsexhibit importantspecialcasesandapplicationstoclassificationillustrat-ing theoryandmethodologies thathavebeenaroundsinceat least the1940sand1950sinengineeringandcomputerscience,andhavebeenexpandedanddeepenedinthemeantime.

We take it as a plausible working hypothesis that the gargantuan corpus ofquestionsinsocial,biological,andevensuchareasascognitivescience,includingmachine intelligence and human–machine interactions, ultimately cannot all restcomfortablyor rigorouslywithin the simpler typesofgeometric spaces thathavedominatedthoseareas(withtheexceptionofthespecialcasestreatedinthelasttwosections,whichhave seenextensive implementation in engineeringandcomputerscience).Thatis,inthefinalanalysis,anempiricalquestion,butwebelieveourpres-entandfutureresearchersshouldaccouterthemselveswiththepowerfultoolsthatcanaidinansweringfundamentalquestionssuchasthese.Overthecenturies,thesymbiosisbetweenphysicsandmathematicshastremendouslyenrichedbothfields.Uptonow, thesocialandbiologicalscienceshave largelybeenon theborrowingrather than lending sideof the interactions, but there are strong signs that this ischanging,forthebetter!

Endnotes

1. Theprimarydisciplinesoftheauthorsarepsychology,andcognitivescience.Weshalluse“psychology”asaproxy foranyof thesocialorbiologicalfields thatcannotbereadilyspecifiedintermsofphysics(anexampleofacandidateforonethatcanbesodescribedmightbequantumpropertiesofaneurotransmitterinre-uptakedynamics).

2. Modern psychology now includes a tremendous effort in neuro-sciences, especiallyneuro-imaging and of course, also affords a rich domain of mathematical researchpossibilities–thus,forthelatter,seethewebsitesfortheUSSocietyforMathematicalPsychologyandtheEuropeanMathematicalPsychologyGroup.

Y116134.indb 169 10/10/11 2:12 PM


3. Therehasbeenmuchworkmostlytheoreticalintheinterim.Narens(1985)studiesthecriticalnotionof“meaningfulness”infoundationalmeasurement.Luce&Weber(1986)provideain-depthaccountofaxiomaticdecisionmakingfromanaxiomaticmeasure-menttheoryviewpoint.

4. Mathematicalpsychologyisasubfieldofpsychologywhere,inplaceofverbaltheoriz-ing,mathematicaltheoriesorastheyareoftencalled,“models”,areutilizedtomakeone’sassumptionsrigorous,andtomakestrongpredictionsthataretestablebyobserva-tionaldata.Townsend&Kadlec(1990)offerabriefoverviewofthemajorbranchesofmathematicalpsychologyandTownsend(2008)discusseschallengesandprospectsformathematicalpsychologyinthetwenty-firstcentury.

5. Tverskywasnot thefirst to thinkof identical featuresasaffecting thepsychologicalsimilarityoftwoobjects.HetraceshistheorytoseminalideasofRestle(1961)whopro-posedsettheoreticmeansofassessingsimilaritywhichcouldincludetheformer.OtherusesofthisconceptinmodelsofpatternrecognitioncanbefoundinworkbyTownsend&Ashby(1982).Nonetheless,Tversky’stheorywasbyfarthemostquantitativelythor-oughandwasdevelopedinthecontextofthepreviouslydiscussedfoundationalmea-surementtheory.

6. Thetheory(orrathertheories)ofdimensionalityhasgrownovertheyears.RelativelydeepearliertreatmentsalongsomeavenuescanbefoundinHurewics&Wallman(1941)andNagata(1965).

7. Alimitationofprincipalcomponentsanalysisisthatitassumesthatthebasisvectorsbeorthogonaltooneanother.Itmightwellbethatthebasisvectorsforapsychologicalvec-torspacearelinearlyindependentbutnotorthogonal.Amethodologywhichassumesindependencebutnotorthogonalitywhichhasgainedmuchattentionlately,especiallyintheanalysisoffMRIsignals(literally“functionalmagneticresonanceimaging,”aneuro-imagingmethodbasedonmeasurementofbloodoxygenlevelsandtheirchangesduringpsychologicaltasks)iscalled“independentcomponentsanalysis”(ICA).However,thisapproachtoo,hasitslimitations.

8. Itbearsmentioningthatrelativitytheory,eventhespecialtheory,requiresaddingtimeasanegativenumberinfiguringthedistanceinthespace(e.g.,Minkowski,1908).

9. TheAshbygrouphasdevelopedageneral theoryof categorizationbasedongeneralrecognitiontheory(e.g.,seeChaptersinAshby,1992).

10. Avolumeonapplicationsoftensoranalysis(atypicallyappliedbranchofdifferentialgeometry)bySchouten(1951)isalandmarkintheapplicationsoftensortheorytophys-ics.However,itstandsasoneofthemorechallenging-to-readtreatisesinscience,duetothemazeofindices.

11. Thisniceeventcomesaboutduetostationarity,thepropertiesofwhitenoise,andthevaluesofthenoisemeanandvariance.

References

Amazeen,E.L.(1999).Perceptualindependenceofsizeandweightbydynamictouch.Journal of Experimental Psychology: Human Perception & Psychophysics,25(1),102–119.

Ashby, F. G. (1988). Estimating the parameters of multidimensional signal detectiontheory from simultaneous ratings on separate stimulus components. Perception & Psychophysics,44,195–204.

Ashby, F. G. (1992). Multidimensional models of perception and cognition. Hillsdale, NJ:LawrenceErlbaumAssociates.

Y116134.indb 170 10/10/11 2:12 PM


Ashby,F.G.,&Townsend,J.T.(1986).Varietiesofperceptualindependence.Psychological Review,93,154–179.

Baird, J. C. (1997). Sensation and judgment: Complementarity theory of psychophysics.Hillsdale,NJ:LawrenceErlbaumAssociates.

Boothby,W.M.(1975).An introduction to differentiable manifolds and riemannia.NewYork:AcademicPress.

Cornish,N.J.,&Frankel,N.E.(1997,Aug).Theblackholeandthepea.Physics Review D,56(4),1903–1907.

Deffenbacher,K.A.,Vetter,T.,Johanson,J.,&O’Toole,A.J.(1998).Facialaging,attractive-ness,anddistinctiveness.Perception,27(10),1233–1243.

Falmagne,J.(2002).Elements of psychophysical theory(Vol.1).Oxford:OxfordUniversityPress.

Fechner, G.,Adler, H., & Boring, E. (1966). Elements of psychophysics. NewYork: Holt,RinehartandWinston.

Hurewics,W.,&Wallman,H.(1941).Dimension theory.Princeton,NJ:PrincetonUniversityPress.

Kadlec,H.,&Townsend,J.T. (1992).Signaldetectionanalysisofmultidimensional inter-actions.InF.G.Ashby(Ed.),Probabilistic multidimensional models of perception and cognition.Hillsdale,NJ:LawrenceErlbaumAssociates.

Krantz,D.H.,Luce,R.D.,Suppes,P.,&Tversky,A.(1971).Foundations of measurement (Vol. (). Additive and polynomial representations.NewYork and London:AcademicPress.

Link,S.W. (1992).The wave theory of difference and similarity.Hillsdale,NJ:LawrenceErlbaumAssociates.

Lord,F.M.,&Novick,M.R.(1968).Statistical theories of mental test scores.Reading,MA:Addison-Wesley.

Luce, R., Bush, R., & Galanter, E. (Eds.). (1963). Handbook of mathematical psychology(Vols.I–III).NewYork:Wiley.

Luce,R.D.,&Weber,E.U.(1986).Anaxiomatictheoryofconjoint,expectedrisk.Journal of mathematical psychology,30(2),188–205.

Luenberger,D.G.(1979).Introduction to dynamic systems: Theory, models, and applications.NewYork:Wiley.

Maddox,W.T.(2001).Separatingperceptualprocessesfromdecisionalprocessesinidentifi-cationandcategorization.Perception & Psychophysics,63(7),1183–1200.

Minkowski,H.(1908).Spaceandtime.Nagata,J.(1965).Modern dimension theory.Amsterdam:North-Holland.Narens,L.(1985).Abstract measurement theory.Cambridge,MA:MITPress.Nosofsky,R.M.(1986).Attention,similarity,andtheidentification-categorizationrelation-

ship.Journal of Experimental Psychology. General,115(1),39–57.Nosofsky, R. M. (1988). Exemplar-based accounts of relations between classification, rec-

ognition,andtypicality.Journal of Experimental Psychology: Learning, Memory and Cognition,14(4),700–708.

O’Toole,A.J.,Wenger,M.J.,&Townsend,J.T.(2001).Quantitativemodelsofperceivingandrememberingfaces:Precedentandpossibilities.InM.J.Wenger&J.T.Townsend(Eds.),Computational, geometric, and process perspectives on facial cognition(p.50).Hillsdale,NJ:ErlbaumPress.

Padulo,L.,&Arbib,M.A.(1974).System theory.Philadelphia:Saunders.

AU: "riemonnia" & lower case OK?

AU: Vol. # missing?

AU: Complete info.

Y116134.indb 171 10/10/11 2:12 PM


Pelli,D.G.,Farell,B.,&Moore,D.C.(2003).Theremarkableinefficiencyofword.Nature,423(6941),752–756.

Pfanzagl,J.(1968).Theory of measurement.(p.50).NewYork:Wiley.Ramsay,J.,&Silverman,B.(2006).Functional data analysis.Springer-Verlag,NewYork.Restle,F.(1961).Psychology of judgement and choice.NewYork:Wiley.Roberts,L.D.,Patterson,D.,Thomson,J.O.,&Levey,R.P. (1969,Mar).Solid-stateand

nuclear results fromameasurementof thepressuredependenceof theenergyof theresonancegammarayofau197.Physics Review,179(3),656–662.

Rumelhart,D.E.,&McClelland,J.L. (1987).Parallel distributed processing.Cambridge,MA:MITPress.

Schonemann,P.H.,Dorcey,T.,&Kienapple,K.(1985).Subadditiveconcatenationindissimi-larityjudgements.Perception & Psychophysics,38,1–17.

Schouten,J.(1951).Tensor analysis for physicists.Oxford:ClarendonPress.Stevens, S. (1951). Mathematics, Measurement and psychophysics. S. Stevens (Ed.). New

York:Wiley.Suppes, P., & Zinnes, J. (1963). Basic measurement theory. In R. D. Luce, R. Bush, &

E.Galanter(Eds.).Handbook of mathematical psychology.Stanford,CA.Thomas,R.D.(1999).Assessingsensitivityinamultidimensionalspace:Someproblemsand

adefinitionofagenerald’.Psychonomic Bulletin & Review,6,224–238.Thomas, R. D. (2003). Further consideration of a general d’ in multidimensional space.

Journal of Mathematical Psychology,9,40–50.Townsend,J.T.(1971).Theoreticalanalysisofanalphabeticconfusionmatrix.Perception &

Psychophysics,9,40–50.Townsend, J.T. (2008).Mathematicalpsychology:Prospects for the21st century:Aguest

editorial.Journal of mathematical psychology,52,269–280.Townsend,J.T.,Aisbett,J.,&Busemeyer,J.B.(2005).Generalrecognitiontheoryandmeth-

odologyfordimensionalindependenceonsimplecognitivemanifold.InH.Colonius&E.N.Dzhafarov(Eds.),Measurement and representation of sensations: Recent progress in psychophysical theory.Washington,D.C.:AmericanPsycholgocialAssociation.

Townsend,J.T.,&Ashby,F.G. (1982).Anexperimental testofcontemporarymathemati-calmodelsofvisual letter recognition.Journal of Experimental Psychology: Human Perception and Performance,8,834–864.

Townsend,J.T.,&Ashby,F.G.(1984).Measurementscalesandstatistics:themisconceptionmisconceived.Psychological Bulletin,96(2),394–401.

Townsend, J.T.,&Kadlec,H. (1990).Mathematics andpsychology. InR.Mickens (Ed.),Mathematics and science.Singapore:WorldScientific.

Townsend,J.T.,&Landon,D.E.(1982).Anexperimentalandtheoreticalinvestigationoftheconstantratioruleandothermodelsofvisualletterrecognition.Journal of Mathematical Psychology,25,119–163.

Townsend,J.T.,&Landon,D.E.(1983).Mathematicalmodelsofrecognitionandconfusioninpsychology.International Journal of Mathematical Social Sciences,4,25–71.

Townsend,J.T.,Solomon,B.,Wenger,M.J.,&Spencer-Smith,J.(2001).Theperfectgestalt:Infinite dimensional Riemannian face spaces and other aspects of face cognition. InJ.T.Townsend&M.J.Wenger(Eds.).Computational, geometric and process issues in facial cognition: Progress and challenges.Hillsdale,NJ:ErlbaumPress.

Townsend,J.T.,&Spencer-Smith,J.(2004).Twokindsofglobalperceptualseparabilityandcurvature. In C. Kaernbach, E. Schröger & H. Müller (Eds.), Psychophysics beyond sensation: Laws and invariants of human cognition.Mahwah,NJ:Erlbaum.

AU: Who is the publisher?

Y116134.indb 172 10/10/11 2:12 PM


Townsend,J.T.,&Thomas,R.D.(1993).Foundationsofperceptualtheory.InS.C.Masin(Ed.),(chap.Ontheneedforageneralquantitativetheoryofpatternsimilarity).ElsevierSciencePublishers.

Tversky,A.(1977).Featuresofsimilarity.Psychological Review,84(4),327–352.Wenger,M.J.,&Ingvalson,E. (2003).Preserving informationalseparabilityandviolating

decisional separability in facial perception and recognition. Journal of Experimental Psychology: Learning, Memory and Cognition,29,1106–1118.

Y116134.indb 173 10/10/11 2:12 PM

Y116134.indb 174 10/10/11 2:12 PM

175

Psychophysical linguistics

Stephen LinkDepartmentofPsychology,UniversityofCalifornia,SanDiegoLaJolla,California,USA

8.1 Introduction

“Ihavemeasuredoutmylifewithcoffeespoons,”PrufrockdeclaresinT.S.Eliot’s(1917)famousThe Love Song of J. Alfred Prufrock.Themeasuremaybesomewhatdubiousbut themeaning isplainly clear.Howwould themeaning change if thismeasureoflifealsochanged?Forexample,ifinsteadofcoffeespoonswesubstituteteaspoons,tablespoons,soupspoons,orevensugarspoons.Intermsofpsychophysi-calmeasurement,howfariseachofthesenewmeaningsfromthemeaningestab-lishedbyEliot’spoeticchoiceof“coffeespoons”?Thedevelopmentofameasureofdistancebetweenmeaningsisthefocusofthischapter.

Thetypeofjudgmentrequiredinthiscaseisoneofmatchingorsimilarity.Forexample,does“Ihavemeasuredoutmylifewithtablespoons”havethesameordif-ferentmeaningfromPrufrock’sdeclaration?Ifdifferent,howmuchdifferentisit?Suchjudgmentsofmatchingorsimilarityarecommonlyusedinlinguisticexperi-mentsyet their theoreticalbasis iswidelyunknown.Althoughsuch judgmentsofsimilarity or dissimilarity occur as routinely as judgments of largeness or small-nesstheyarebasedonadifferentprinciple.Judgmentsofsimilarityordissimilarityrequirefortheiranalysisaverydifferenttreatment,adifferenttheoreticalbasis,thandojudgmentsoflargenessorsmallness.

ManyyearsafterFechner’sdevelopmentoftheoriginalmodelforthecompara-tive judgmentof twostimulusmagnitudes,psychophysicistsdiscovered thatsomeofthejudgmentsmadeinpsychologicalexperimentswerenotjudgmentsofdiffer-encesinstimulusmagnitudeatallbutwere,instead,judgmentsofequalityorsame-ness.SeizingonthisdiscoveryUrban(1907)suggestedthatthreedifferentresponsesbe employed in the typical discrimination experiment. A comparison stimulusmightbejudgedtobesmaller,larger,orequalwithrespecttoastandardstimulus.Althoughexperimentalsubjectsdidusethesethreeresponsecategoriesinanumberofexperiments,thetheoryofhowsuchjudgmentsmightoccurrequiredmanyyearsofsubsequentdevelopment(cf.Link,1992).And,asthestudyofcomparativejudg-mentadvanced,psychophysicistsrealizedthatthejudgmentofsmallerorlargerisonetypeofjudgmentandthatthejudgmentofequalityorinequality,ofsameness

8

Y116134.indb 175 10/10/11 2:12 PM


ordifference, is theonlyothertypeofjudgmenttobediscoveredsinceFechner’sintroductionofcomparativejudgmentin1860.

8.2 Discrimination of visual angle

Letusreturntoaonce-famous,butnowvirtuallyunknown,experimentusingjudg-mentsofsamenessordifferencereportedbyDorwinCartwright(1941).Theexperi-mental result that differentiates such judgments from judgments of largeness orsmallnessisillustratedinFigure 8.1.Hereareresultsfromatypicalmethodofcon-stantstimuliexperimentemployingvisualanglesasstimuli.Experimentalsubjectsdecidedifapresentedanglevaryingto160°from10°waswithinalearnedintervalof60°to100°.Bothresponselatency,thetimefromthepresentationofthestimulustotheresponse,alsocalledresponsetime(RT),andresponseproportionsshowtheexistenceofwhatUrban(1910)definedasan“IntervalofUncertainty.”

The“IntervalofUncertainty”isthecontiguousintervalwithinwhichtheprob-abilityofmakingthejudgmentequalisgreaterthan0.50.Cartwright’sresultsshowhow,attheedgesofthis“interval,”atthepointof50%,themeanresponselatency,orresponsetime,reachesapeak.Indeed,meanresponsetimesappeartobeafunctionofthedistanceofacomparisonstimulusfromtheedgesoftheintervalofuncertainty.Thisoneofmanyresults,confirmedovermanydecades,showsthattheapplicationofajudgmentmodelbasedonmonotonicchangesinperformanceasafunctionofstimulusdifferenceisnotapplicabletojudgmentsofsamenessanddifference.

8.3 Numerical comparisons

SimilarresultsfromnumericalcomparisonexperimentsperformedinmylaboratorybyAnnaVentresca(1988)verifytherepeatedoccurrenceofthesefindings.Subjects

Cartwright 1941

0

200

400

600

800

1000

1200

1400

10 40 70 100 130 160Angle of Test Stimulus

Mea

n Re

spon

se T

ime (

mse

c)

0.00

0.25

0.50

0.75

1.00

P (W

ithijn

Inte

rval

)

RTp

Figure 8.1 ResultsfromCartwright(1941)onjudgingwhetheratestanglewaswithintheintervalof60°to100°.Meanresponsetimesinmilliseconds.

Y116134.indb 176 10/10/11 2:12 PM

Psychophysical linguistics 177

weregivenan intervaldefinedby twonumbers andwere to judgewhether a testnumberexistedwithinoroutsidetheinterval.Thesizeoftheintervalandthetestnumberschangedfromtrialtotrial.Figure 8.2showspreviouslyunpublishedresultsforresponseproportionsandmeanresponsetimeswhentheintervalspanned33to77andteststimulirangedfrom12to99exclusiveof33and77.

Asiseasilyseen,thepatternofresultsissimilartothatoftheCartwrightexperi-ment.Responsetimesbecomelargestwhentheresponseproportionisabout0.50.Theresultsderivefromanaverageofmanytrialsfromfourverywell-trainedsubjectswhoweregivenmanydifferentintervalsandteststimuli.TherelationsinFigure 8.2aretypicaloftheresultsacrossallrangesofintervalsemployed,thisrange,33to77,beingthelargest.Atheoreticalbasisfornumericalcomparisonsisdescribedin“Measuringimagelessthought:Therelativejudgmenttheoryofnumericalcompari-sons”(Link,1990).

8.4 Face recognition

Ifanymoreproofsofthiswidelyfoundrelationwereneeded,LinkandLee(2009)reported similar results from experiments on face recognition. Ten subjects weretorespondSameorDifferenttoatestfacepresentedsimultaneouslywithanunchang-ingstandardface.Testfaceswereeitherthesameasthestandardface(50%oftrials)orwerefaceswiththeeyesdisplacedbyvaryingnumbersofpixels(1pixel=0.04cm)rangingto+10pixelsfrom–10pixels,eachpresentedwithprobability0.025.Facesremainedpresentuntilthesubjectresponded.Responsechoiceandresponsetimesweremeasured.Nofeedbackwasgivenduringthe1,000trials.

Test Number within Interval 33–77Link and Ventresca

200

300

400

500

600

700

800

900

1000

12 23 34 45 56 67 78 89Test Number

Mea

n Re

spon

se T

ime

0.00

0.25

0.50

0.75

1.00

P (In

side I

nter

val)

RTp

Figure 8.2 Numericalcomparisonsoftestnumberswithinoroutsideaspecifiedinterval.Theleftaxisshowsmeanresponsetimeinmilliseconds.Theright-handaxisshowsthepro-portionofjudgmentsthatthepresentedtestnumberiswithintheintervalof33to77.

Y116134.indb 177 10/10/11 2:12 PM


OnceagainthepatternofresultsshownintheCartwrightandLinkandVentrescaexperimentsemergesasshowninFigure8.3.Nearthestimulusdifferencegenerating50%responding themean response times reachamaximumand thendecreaseoneitheredgeofthe“IntervalofUncertainty.”Theseresultsfromtensubjectsusetherawdataaveragedacrosssubjects.Similarresultswereobtainedforothergroupsofsub-jectswhostudiedthestandardfaceatthebeginningoftheexperimentforeither10,20,or30secondsandthenweretestedin1,000trialsforrecognitionwithoutfeedback.

8.5 The measurement of meaning

Inthesepsychophysicalexperimentsonlyasingledimensionisalteredtocreatedif-ferentcomparisonstimuli.Inalinguisticspacedefinedbymeaning,butofunknowntopology,wordsmaycreatedimensionsofunknowncharacter.Neverthelesswemayassumethatwhenawordispresentedinthecontextofasentenceitvariesinthelinguisticspacealongsomedimensionsofdifference.Thelesscomplexthecontribu-tionofthewordtothemeaningthefewerdimensionsmayberequiredfordetermin-ingthedistancecontributedbythewordtothemeaning.

Cartwright(1941)providesanexampleofthetypeofexperimentusefulindeter-mining the distance between meanings. In following up on the results shown inFigure 8.1Cartwright tested thechanges inmeaningcausedbychanges inakeywordintwosentences.TheexperimentisdescribedbyCartwright(1941):

AtthebeginningoftheexperimenttheSsweregivenasentencewithinstruc-tionstoarriveataclearformulationofthemeaningofthevariouswordsinit.Then,theywereshownsomenewwords,whichweretobesubstitutedforoneofthewordsofthesentence.Uponseeingeachword,Swasrequiredtodecidewhether the substitution of the new word changed the basic meaning of the

Face Recognition Link and Lee

1.0

1.5

2.0

2.5

–10 –9 –8 –7 –6 –5 –4 –3 –2 –1 0 1 2 3 4 5 6 7 8 9 10Interocular Displacement in Pixels

Mea

n Re

spon

se T

ime (

sec)

0

0.25

0.50

0.75

P (“s

ame”

)

RTp

Figure 8.3 Response times and proportions for the last 400 of 1,000 judgments of facesimilarity.Eyesweredisplacedbyvariousnumbersofpixels(0.04cm/pixel)ontheabscissafromastandardface.ResultsfromLinkandLee(2009).

Y116134.indb 178 10/10/11 2:12 PM


sentence.Twodifferentsentenceswereselectedsothatawordcommontobothwouldrelatetodifferentrangesofmeaning,thuscreatingdifferentdifferentia-tionsofthetestseries.TenSsweregiventhetwosentencesandthetestseriesondifferentdays.SentenceIwas:YesterdayIsawahugebuilding.SentenceIIwas:YesterdayIsawahugeman.Thetest-seriescontainedthefollowing10words:1.immense,2.grand,3.great,4.vast,5.colossal,6.large,7.magnificent,8.big,9,mighty,10.massive.Itwassupposedthata listofwordswhichwouldbeappropriatesubstitutesforhugeinSentenceIwouldpossesscertainwordswhichwouldnotbeappropriate substitutes inSentence II. It isnotproposedthatSentenceIIlimitsthetotalnumberofappropriatesubstitutesforhuge,butratherthenumberofappropriatesubstituteswithinthetestseriesemployedinthisexperiment.ThetestserieswaspresentedsixtimestoeachS.SincetenSsweretestedonbothsentences,1200decisionswererecorded.

Cartwrightmeasuredchoice(“Yes”or“No”)andvocalRT.Hereportedthepercent-ageof“Yes”responsesforeachsubjectandsentencebutonlythesubject’smeanvocalRTforeachprobewordwithoutdistinguishingbetween“Yes”and“No”responses.AttheendoftheexperimentCartwrightrequiredsubjectstorankorderwordsonthebasisofsimilarityoftheresultingsentencestothestandardsentence.Fromtheindi-vidualranksitispossibletocomputetheprobabilitythatawordchangesthesimilaritybetweenastandardandacomparisonsentencebyusingthe60judgmentsforeachrankorder.Althoughthisisnotameasureofdistanceitdoesprovideabasisfordeterminingameaningfuldistance.Table 8.1providestheproportionsforeachrankorderofwordsintermsofsimilarityoftherankorderword’ssentencetothestandardsentence.

8.6 A theory of similarity judgments

ForeachrankordertheseproportionsareconvertedtologitsbasedonthetheoryofcomparativejudgmentproposedbyLink(1975,1992)butextendedheretojudgmentsofsimilarity.Link(1992)showedhowadifferencebetweentworandomvariableswithPoissondistributionscandriveanaccumulationofdifferencesleadingtooneortheotheroftwoalternativeresponses.InthecaseofsimilarityjudgmentsoneofthesePoissondistributions,sayS,measuressimilarityandtheother,sayD,measuresdissimilaritybetweentwostimuli.ThemagnitudeofsimilarityischaracterizedbyaPoissondistributionwithmeanvalueμ.ThedissimilaritieshaveaPoissondistri-butionwithparameterλ.Linkprovedthatwhenthesetwomeasuresarecomparedthrough an accumulation of differences between them, S – D, the probability ofchoosingateststimulustobesimilartothestandardstimulusequals

P=1

1+ −e Aθ , (8.1)

whereAistheamountofsimilarityrequiredbeforeanunbiasedresponsecanoccurandθ=ln(λ/μ)isthelogarithmoftheratioofthemeanvaluescharacterizingthesimi-larityanddissimilarityofthecomparedsentences.

Y116134.indb 179 10/10/11 2:12 PM


Tabl

e 8.

1 P

ropo

rtio

nsp

1an

dp2

of

choo

sing

ate

stw

ord

toc

reat

ea

mea

ning

sim

ilar

toth

est

anda

rds

ente

nce

Sent

ence

Wor

ds

12

34

56

78

910

p10.

995

0.92

90.

896

0.83

10.

615

0.54

90.

385

0.20

20.

087

0.02

2

Aθ1

5.29

2.56

2.15

1.59

0.47

0.20

–0.4

7–1

.37

–2.3

5–3

.82

RT

115

6.8

180.

220

1.3

294.

221

9.5

231.

219

0.1

216.

220

0.4

159.

3p2

0.99

50.

979

0.81

40.

583

0.36

80.

120

0.05

50.

022

0.02

20.

005

Aθ2

5.29

3.82

1.47

0.33

–0.5

4–2

.00

–2.8

5–3

.82

–3.8

2–5

.29

RT

214

5.7

166.

619

9.9

235.

719

2.9

225.

119

7.2

175.

314

9.5

143.

0

Not

e:

Wor

dsa

rein

ran

kor

der

ofs

imila

rity

with

ins

ubje

cts.

Y116134.indb 180 10/10/11 2:12 PM


SomealgebrashowsthattheunknownparametersAandθcanbeestimatedfromdatabyusingexperimentallydeterminedproportions,p,toestimatethetheoreticalprobabilityP,andthencomputing,

A p pθ = −( )ln / ( ) .1 (8.2)

Thisisameasure,calledalogitinthestatisticalliterature,ofthedifferencebetweenthesimilarityanddissimilarityoftheteststimulusandthestandardbecause

Aθ=A[ln(λ)–ln(μ)] (8.3)

isadistancemeasureonthescaleofthenaturallogarithm.NoticethatbyexponentiatingEquation(8.3)weobtainapowerlawthatdepends

onthestrengthsofsimilarityanddissimilarity;thatis,

eAA

θ λµ

=

. (8.4)

But,examiningEquation(8.2)weseethatbyexponentiatingAθthenaturalloga-rithmontheright-handsideofEquation(8.2)isremoved.Therefore,thereexistsapowerlawrelationbetweenproportionsofresponsesandtheunderlyingstrengthsofsimilarityanddissimilarity:

pp

A

1−=

λµ

. (8.5)

ThisiscalledthePowerLawofSimilarity.With respect to response times Link (1975, 1992) split the measured response

time into twoparts.Thefirstpartmeasured the time taken tomake theeventualdecision,thedecisiontime,andthesecondpartcontainedallthosecomponentsthatwerenotrelatedtothedecisionprocessperse.Thustheobservedresponsetimecanbewrittenas

Mean(RT)=Mean(Decisiontime)+Mean(Nondecisioncomponents). (8.6)

Theoreticalanalysisofthisstochasticprocess,oftendescribedasaboundedran-domwalk,revealsthat

Mean(Decisiontime)=A

Pλ µ−( ) −( )2 1

Y116134.indb 181 10/10/11 2:12 PM


whenthereisnoresponsebiasandλ≠μ.Asthedenominatorapproacheszero,θapproaches0,thevalueofPapproaches0.50,andthemeandecisiontimeapproachesamaximumof

A2

λ µ+,

wherethedenominatoristhevarianceofS–D.Inacarefullyrunexperimentthenondecisioncomponentsofresponsetimeshould

becontrolledacrosschangesinstimulusdifferencesothatthechangesinobservedresponsetimearethoseassociatedwiththedecisionprocessitself.Withrespecttothedecisiontimes,themaximumdecisiontimewasshowntooccurwhenP=0.50.Thus,whentheestimatorofP,thatis,p,equals0.50themaximumresponsetimeshouldbeobserved.Thistheoreticalpredictionisconfirmedbymanypreviousexperiments.

8.7 Theoretical analysis of Cartwright’s experiment

Theproportionsof similarityofmeaning judgmentsmadeduring theexperimentappearinTable 8.1.TheseproportionsareusedtodeterminethevaluesofAθshownbeloweachproportion.Theorderlydeclineinthesevaluesfrom5.29to–3.82forSentence1andfrom5.29to–5.29forSentence2issufficientevidencethatvaluesofAθareincloseagreementwiththerankorderingsprovidedaftertheexperiment.Indeed,therankordersmayfollowfromtherememberedmagnitudesofAθ,amag-nitudethatmaybeconsideredasthefeelingofsimilarity.

Althoughthetwosentencesprovidesomewhatdifferentmeasuresbasedontheirrankordersofwords,theagreementbetweenthemeasuresisverygood,producingacorrelation0.97asshowninFigure 8.4.NoticeinFigure 8.4thatthebestlinearfitaccountsfor94%ofthevariance.Furthermore,themeasuresofmeaningforthesetwosentencesarerelatedbytheequation0.78(S2)=S1–1,whereS2isthemeasureofAθforsentence2andS1isthemeasureofAθforsentence1.Thisequationrelatesthesimilarityinmeaningsforthetwosentences.

Cartwrightalsomeasured the individualsubjectvocal response timesforeachjudgment.Althoughthesubjectsvariedintermsofoverallvocalresponsetimesforthesejudgments,thevarioussubjecttimescanbeadjustedforindividualdifferencesbycomputing response timedeviationvalues foreachsubject’s response time foreachjudgment.Theseadjustedvaluesareobtainedbycomputingat-valueforeachresponsetimeforeachsubjectwitheachsentence.Thesevaluesaredeterminedbysubtractingfromasingleresponse time thesubject’smeanresponse timefor thatsentenceanddividingbythestandarddeviationforthatsubject’sjudgmentsforthatsentence. These values are then free from biases due to the between-subject dif-ferences in magnitude and variability of the judgment response times. However,Cartwright’sreportsof theindividualsubjectmeanvocalresponsetimesforeachrank-ordered word allows for computations of mean response times that parallelthecomputationsgeneratingFigures 8.1–8.3.Thereforetheactualmeanvocaltimesenterintotheanalysisofresponsetimesreportedbelow.

Y116134.indb 182 10/10/11 2:12 PM


Tosimplifytheanalysissomewhatthetwosentenceswerecombinedintermsofrankordertoproduceasinglecompositeoftherelationbetweenresponsetimesandresponseproportions.ThequestiontobeanswerediswhetherthepreviousresultsforcomparativejudgmentsofsimilarityarealsoexhibitedinCartwright’sdata.Inprevious figures a physical measure of stimulus difference was the variable con-trolledby the experimenter.Bothpositive andnegativevaluesof stimulusdiffer-enceallowedforatwo-sidedviewofthesubject’sperformance.Herewehaveonlyasinglesidebecausethedirectionofdistanceofthetestwordmeaningsfromthestandardsentencemeaningisunknown.Itisasifinthepreviousfigureswefoldedthefiguresatzerostimulusdifferenceandthenviewedonlyasingleside.

Thesignofstimulusdifferenceisinthiscaseunknown,thusthesingle-sidedgraphsomehowneedstoconveythesameinformationabouttherelationsofresponsepropor-tionandresponsetimesasinthepreviousanalyses.Someofthewordsmaycorrespondtopositivestimulusdifferencesandsometonegativestimulusdifferences,whicharecurrentlyunknown.However,ifweallowsomelatitudeincreatingausefulcomparisonwiththepreviousresultswemayexpanduponthestandardsentenceasiftwosidesofthegraphwereactuallymeasuredinordertoprovideacomparisonwiththepreviousfigures.ThisisshowninFigure 8.5wheretherankordersdecreaseawayfromthestan-dardsentenceonbothsides.Ofcoursethisissymmetricbutthensoarethepreviousfigures,andthisprovesthepointofthiscomparison.Theresultswiththesewordsandsentencesaresimilartoresultsobtainedwithphysicallymeasureablestimuli.

Inconclusion thesevarious resultsprovidean insight intohowpsychophysicaltheoryanddataanalysisprovideabasisforthemeasurementofdifferencesinmean-ingbetween twosentences.This result isasatisfactoryoutcomeofapplyingpsy-chophysicalanalysistoalinguisticquestion.Thenew,fundamental,ideassuchasthePowerLawofSimilarityprovide abasis formore refinedexperiments in themeasurementofmeaning.

Relation Between Meaning Estimates for Sentence 1 and Sentence 2

S2 = 1.2782S1 – 1.2835R2 = 0.9357

–8

–6

–4

–2

0

2

4

6

8

–6 –4 –2 0 2 4 6

Sentence 1

Sent

ence

2

Figure 8.4 Measuresofsimilarityfortestsentencesversustwosimilarstandardsentencescompared.Correlation=0.97.

Y116134.indb 183 10/10/11 2:12 PM


References

Cartwright,D.(1941).Relationofdecision-timetocategoriesofresponse.American Journal of Psychology, 54,174–106.

Eliot,T.S.(1917).Prufrock and other observations.London:TheEgoist.Link, S.W. (1975).The relative judgment theory of two-choice response time. Journal of

Mathematical Psychology,12,114–135.Link,S.W.(1990).Measuringimagelessthought:Therelativejudgmenttheoryofnumerical

comparisons.Journal of Mathematical Psychology, 34,2–41.Link S.W. (1992). The wave theory of difference and similarity. Hillsdale, NJ: Lawrence

Erlbaum.Link,S.,&Lee,K.(2009).Psychophysicsoffacerecognition.InM.A.Elliott,S.Antonijević,

S.Berthaud,P.Mulcahy,B.Bargary,C.Martyn,&H.Schmidt(Eds.),Fechner Day 2009. Proceedings of the 25th Annual Meeting of the International Society for Psychophysics,Galway,Ireland:TheInternationalSocietyforPsychophysics.

Urban,F.M. (1907).On themethodof justperceptibledifferences.Psychological Review,14,244–253.

Urban,F.M.(1910).Themethodofconstantstimulianditsgeneralizations.Psychological Review,17,229–259.

Ventresca,A.(1988).Same/difference judgments of numerical comparisons: A study using an Interval of Equality.B.A.HonoursThesis.McMasterUniversity.Hamilton,Canada:LinkPsychophysicalLaboratory.

Cartwright (1941) Reanalysis

0.0

0.5

1.0

1.5

2.0

2.5

3.0

–10 –9 –8 –7 –6 –5 –4 –3 –2 –1 0 1 2 3 4 5 6 7 8 9 10Rank Order Distance

Mea

n Re

spon

se T

ime

0.00

0.25

0.50

0.75

1.00

P (S

imila

r)

RT (sec)P (similar meaning)

Figure 8.5 ReanalysisofCartwright’s1941judgmentsofsentencesimilarity.Theresultsareflippedacrossthevaluezeroasiftherewerebothpositiveandnegativevaluesofstimulusdifference.Nomeasureisavailableforacomparisonofthestandardsentencewithitself,buttheprobabilityofsimilarity,equalto1.0,isbasedontheory.

Y116134.indb 184 10/10/11 2:12 PM

185

Mathematical foundations of Universal Fechnerian Scaling

Ehtibar N. DzhafarovPurdueUniversityWestLafayette,Indiana

and

SwedishCollegiumforAdvancedStudyUppsala,Sweden

9.1 Introduction

The main idea of Fechner’s original theory (Fechner, 1860, 1877, 1887) can bedescribed as follows (see Figure 9.1). If stimuli are represented by real numbers(measuringstimulusintensities,ortheirspatialortemporalextents),thesubjective distance fromastimulusatoastimulusb> aiscomputedbycumulating fromatob,throughallintermediatevalues,ameasureofdissimilarity ofeverystimulusxfromits“immediate”neighborsontheright.AmodernrenderingofFechner’sthe-ory(Dzhafarov,2001)definesthedissimilaritybetweenxandx+dxas

D x x dx c x x dx, ,+( ) = +( ) −

γ 1

2, (9.1)

whereγ(x,y)isapsychometric function

γ(x,y)=Pr[yisjudgedtobegreaterthanx] (9.2)

withno“constant error” (i.e.,γ (x,x)=1/2), andc is a constant allowed tovaryfromonestimuluscontinuumtoanother.Assumingthatγ(x,y)isdifferentiable,andputting

D x x dx

dx

x y

yF x

y x

, ,+( ) =∂ ( )

∂= ( )

=

γ,

theFechneriandistancefromatob ≥abecomes

G a b F x dxa

b

,( ) = ( )∫ .

9

Y116134.indb 185 10/10/11 2:12 PM


Inparticular,if

F xkx

( ) = ,

whichisarigorousformofWeber’s law,

G a b kba

, log( ) = .

WegetthecelebratedFechner’slawbysettingaatthe“absolutethreshold”x0,

S x kxx

( ) = log0

,

whereS(x)canbereferredtoasthemagnitude of the sensation causedbystimulusx.IfF(x)happenstobedifferentfromk/x,theexpressionsG(a,b)andS(x)aremodifiedaccordingly.Thus,from

F xkx

( ) = −1 β ,1≥β>0,

onegets

G a bkb a,( ) = −( )β

β β

forthesubjectivedistancefromatob,and

S xkx x( ) = −( )β

β β0

Figure 9.1 Fechner’s main idea. To compute the subjective (Fechnerian) distance fromatobonastimuluscontinuum,onecumulates(here,integrates)thedissimilarityofxfromitsinfinitesimallycloseneighborsontherightasxchangesfromatob.

Y116134.indb 186 10/10/11 2:12 PM

Mathematical foundations of Universal Fechnerian Scaling 187

forthesensationmagnitudeofx.InthisrenderingFechner’stheoryisimpervioustothemathematical(Luce&Edwards,1958)andexperimental(Riesz,1933)critiquesleviedagainstit(fordetailsseeDzhafarov,2001,andDzhafarov&Colonius,1999).ThemainideaofthisinterpretationwasproposedbyPfanzagl(1962),andtheninde-pendently reintroduced in Creelman (1967), Falmagne (1971), and Krantz (1971)withintheframeworkoftheso-called“Fechnerproblem”(Luce&Galanter,1963).

Fechner’stheorylaunchedtheworld-view(or“mind-view”)ofclassicalpsycho-physics,accordingtowhichperceptionisessentiallycharacterizedbyacollectionofunidimensionalcontinuarepresentablebyaxesofnonnegativerealnumbers.Eachcontinuumcorrespondstoacertain“sensoryquality”(loudness,spatialextent,satu-ration,etc.)anytwovaluesofwhich,sensory magnitudes,arecomparableintermsof “less than or equal to.” Moreover, each such continuum has a primary physi-cal correlate, an axis of nonnegative reals representing intensity, or spatiotempo-ral extent of a particular physical attribute: the sensory attribute is related to itsphysicalcorrelatemonotonicallyandsmoothly,startingfromthevalueoftheabso-lutethreshold.Thismind-viewhasbeendominantthroughouttheentirehistoryofpsychophysics(Stevens,1975),anditremainsperfectlyviableatpresent(see,e.g.,Luce,2002,2004).

Thereis,however,anothermind-view,alsoderivedfromFechner’sideaofcom-putingdistancesfromlocaldissimilaritymeasures,datingbacktoHelmholtz’s(1891)andSchrödinger’s(1920,1920/1970,1926/1970)workoncolorspaces.Physically,colors are functions relating radiometric energy to wavelength, but even if theirrepresentationbymeansofoneof the traditional colordiagrams (suchasCIEorMunsell)isconsideredtheirphysicaldescription,andevenifthesubjectiverepresen-tationofcolorsisthoughtofintermsofafinitenumberofunidimensionalattributes(suchas, in thecaseofaperturecolors, theirhue, saturation, andbrightness), themappingofphysicaldescriptions intosubjectiveones isclearly thatofonemulti-dimensionalspaceintoanother.Inthiscontextthenotionsofsensorymagnitudesorderedintermsof“greater–less”andofpsychophysicalfunctionsbecomeartificial,ifapplicableatall.Thenotionofsubjectivedissimilarity,bycontrast,acquiresthestatusofanaturalandbasicconcept,whoseapplicabilityallows forbutdoesnotpresupposeanyspecificsystemofcolorcoordinates,eitherphysicalorsubjective.Thenaturaloperationalizationofthediscriminationofsimilarcolorsinthiscontextistheirjudgmentintermsof“same or different,”ratherthan“greaterorless.”(Foradetaileddiscussionofthe“greater–less”versus“same–different”comparisons,seeDzhafarov,2003a.)

Thismind-viewhasbeengeneralizedinthetheoreticalprogramofMultidimen-sional Fechnerian Scaling (Dzhafarov,2002a–d;Dzhafarov&Colonius,1999,2001).Thescopeofthisdifferential-geometricprogramisrestrictedtostimulusspacesrep-resentablebyopen connected regions of Euclidean n-space (refertoFigure 9.2foranillustration.).Thisspaceissupposedtobeendowedwithaprobability-of-differentfunction

ψ(x,y)=Pr[yandxarejudgedtobedifferent].

Y116134.indb 187 10/10/11 2:12 PM


Anytwopointsa,binsuchaspacecanbeconnectedbyacontinuously differen-tiable path x(t)definedonasegmentofreals[a,b].The“length”ofthispathcanbedefinedbymeansofthefollowingconstruction.Assumethat

ψ(x,x)<minψ(x,y), ψ(y,x)

foralldistinctx,y,andthatforanyc∈[a,b]thediscriminationprobabilityψ(x(c),x(t))hasapositiveright-handderivativeatt=c+,

d

d

ψ x xx x

c t

tF c c

t c

( ) ( )( ) = ( ) ( )( )= +

,, .

ThefunctionF(x(t),x(t))isreferredtoasasubmetric function,andthedifferentialF(x(t),x(t))dtservesasthelocaldissimilaritybetweenx(t)andx(t)+x(t)dt.Assum-ingfurtherthatFiscontinuous,wedefinethelengthofthepathx(t)astheintegral

Figure 9.2 Acontinuouslydifferentiablepathx(t)(thickcurve)isshownasamappingofaninterval[a,b](horizontal linesegment) intoanareaofEuclieanspace(grayarea).Foranypointc∈[a,b]thereisafunctiont↦ψ(x(c),x(t))definedforallt∈[a,b](shownbyV-shapedcurvesforthreepositionsofc).Thederivativeofψ(x(c),x(t))att=c+(theslopeofthetangentlineattheminimumoftheV-shapedcurve)istakenforthevalueofF(x(c),x(c)),andtheintegralofthisfunctionfromatobistakenforthevalueoflengthofthepath.Theinsetatthelefttopcornershowsthatoneshouldconsiderthelengthsforallsuchpathsfromatob,andtaketheirinfimumasthe(generallyasymmetric)distanceGab.Theoverall,sym-metricdistanceG*abiscomputedasGab+Gba.[Thelengthsofpathscanbealternativelycomputedbydifferentiatingψ (x(t),x(c))rather thanψ (x(c),x(t)).AlthoughthisgenerallychangesthevalueofGab,itmakesnodifferenceforthevalueofG*ab.]

Y116134.indb 188 10/10/11 2:12 PM


D a b F t t ta

b

x x x, , ( ) = ( ) ( )( )∫ d .

Applyingthistoallcontinuouslydifferentiablepathsconnectingatobandfindingtheinfimum oftheirD-lengths,onedefinesthe(asymmetric) Fechnerian distance Gabfromatob(afunctionwhichsatisfiesallmetricaxiomsexceptforsymmetry).Theoverall (symmetrical) Fechnerian distance G*abbetweenaandbiscomputedasGab+Gba.Althoughthisdescriptionisschematicandincompleteitshouldsuf-fice for introducingone lineof generalizingFechnerianScaling: dispensingwithunidimensionalitybutretainingtheideaofcumulationoflocaldissimilarities.

AfurtherlineofgeneralizationispresentedinDzhafarovandColonius(2005b,2006c).ItisdesignatedasFechnerianScalingofDiscreteObjectSets andappliesto stimulus spaces comprised of “isolated entities,” such as schematic faces, let-tersofanalphabet,andthelike(seeFigure 9.3).Eachpair(x, y)ofsuchstimuliisassignedaprobabilityψ(x,y)withwhichtheyarejudgedtobedifferentfromeachother.Schematizingandsimplifyingasbefore,thelocaldiscriminabilitymeasureisdefinedas

D(x,y)=ψ(x,y)–ψ(x,x),

a = x0

x6 = b

x1

x2

x3

x4

x5

Figure 9.3 Givenachainofpointsx0,x1,…,xk leading froma tob, thedissimilaritiesbetweenitssuccessiveelementsaresummed(cumulated).Inadiscretespace,the(generallyasymmetric)distanceGabfromatobiscomputedastheinfimumofthecumulateddissimi-laritiesoverallchainsleadingfromatob.ThesymmetricaldistanceG*abbetweenaandbiscomputedasGab+Gba.

Y116134.indb 189 10/10/11 2:12 PM


andthe(asymmetric)FechneriandistanceG(a,b)isdefinedastheinfimumof

D i i

i

k

x x, +

=

( )∑ 1

0

computedacrossallpossiblefinitechainsofstimuli

a=x0,x1,…,xk,xk+1=b

connectingatob.HerethedeviationfromFechner’soriginaltheoryisgreaterthanintheMultidimensionalFechnerianScaling:wedispensenotonlywithunidimen-sionality,butalsowiththe“infinitesimality”ofdissimilaritiesbeingcumulated.Buttheideaofcomputingdissimilaritiesfromdiscriminationprobabilitiesandobtain-ingdistancesbysomeformofdissimilaritycumulationisretained.

ThepurposeofthisworkistopresentasweepinggeneralizationofFechner’sthe-orywhichisapplicabletoall possible stimulusspacesendowedwith“same-differ-ent”discriminationprobabilities.Thistheory,calledUniversal Fechnerian Scaling (UFS),ispresentedinthetrilogyofpapersDzhafarovandColonius(2007),Dzhafarov(2008a), and Dzhafarov (2008b). We follow these papers closely, but omit proofs,examples,andtechnicalexplanations.OurfocusisonthemathematicalfoundationsofUFS,which are formedby an abstract theory calledDissimilarity Cumulation (DC):itprovidesageneraldefinitionofadissimilarity function andshowshowthisfunctionisusedtoimposeonstimulussetstopologicalandmetricproperties.

ThepotentialsphereofapplicabilityofUFSisvirtuallyunlimited.Theabilitytodecidewhethertwoentitiesarethesameordifferentisthemostbasicfacultyofalllivingorganismsandthemostbasicrequirementofartificialperceivingsystems,suchasintelligentrobots.Theperceivingsystemmaybeanythingfromanorganismtoapersontoagroupofconsumersorvoterstoanabstractcomputationalproce-dure.Thestimulimaybeanythingfromlettersofalphabet(fromthepointofviewof grammar school children) to different lung dysfunctions represented by X-rayfilms(fromthepointofviewofaphysician)tobrandsofacertainproduct(fromthepointofviewofagroupofconsumers) topoliticalcandidatesorpropositions(from thepointofviewofpotentialvoters) tocompetingstatisticalmodels (fromthepointofviewofastatisticalfittingprocedure).Thus,ifstimuliareseverallungdysfunctionseachrepresentedbyapotentiallyinfinitesetofX-rayfilms,aphysicianoragroupofphysicianscanbeaskedtotelliftworandomlychosenX-rayfilmsdoordonotindicateoneandthesamedysfunction.Asaresulteachpairofdysfunc-tions isassignedtheprobabilitywithwhichtheir respectiveX-rayrepresentationsarejudgedtoindicatedifferentailments.Ifstimuliarecompetingstatisticalmodels,theprobabilitywithwhichmodelsxandyare“judged”tobedifferentcanbeesti-matedbytheprobabilitywithwhichadatasetgeneratedbythemodelxallowsonetorejectthemodely(seeDzhafarov&Colonius,2006a,fordetails).Thequestionstotheperceivingsystemcanbeformulatedinavarietyofforms:“Arexandythesame

Y116134.indb 190 10/10/11 2:12 PM


(overall)?”or“DoxandydifferinrespecttoA?”or“DoxandydifferifoneignorestheirdifferenceinpropertyB?”or“Doxandybelongtooneandthesamecategory(fromagivenlist)?”,andsoon.Notethedifferencefromtheotherknownscalingprocedurebasedondiscriminationprobabilities,ThurstonianScaling (Thurstone,1927a,b).Thisprocedureonlydealswiththeprobabilitieswithwhichonestimulusisjudgedtohavemoreofaparticularproperty(suchasattractiveness,brightness,loud-ness,etc.)thananotherstimulus.Theuseoftheseprobabilitiesthereforerequiresthattheinvestigatorknowinadvancewhichpropertiesarerelevant,thatthesepropertiesbesemanticallyunidimensional(i.e.,assessableintermsof“greater–less”),andthattheperceptionof thestimulibeentirelydeterminedby theseproperties.NosuchassumptionsareneededinUFS.Moreover,intheconcludingsectionofthechapteritismentionedthatthediscriminationprobabilitiesmayverywellbereplacedwithotherpairwisejudgmentsof“subjectivedifference”betweentwostimuli,andthatthetheorycanevenbeappliedbeyondthecontextofpairwisejudgmentsaltogether,for example, to categorization judgments. It is also mentioned there that the dis-similaritycumulationprocedurecanbeviewedasanalternativeto thenonmetricversionsofMultidimensionalScaling,applyingthereforeinallcasesinwhichonecanusethelatter.*

9.2 Psychophysics of discrimination

Weobservethefollowingnotationconventions.Boldfacelowercaseletters,a,b′,x,yn,…,alwaysdenoteelementsofasetofstimuli.Stimuliaremerelynames(qualita-tiveentities),withnoalgebraicoperationsdefinedonthem.Real-valuedfunctionsofoneormoreargumentsthatareelementsofastimulussetareindicatedbystringswithoutparentheses:

ψab, Dabc,DXn,Ψ(ι)ab,….

9.2.1 Regular Minimality and canonical representations

Here, we briefly recapitulate some of the basic concepts and assumptions under-lying the theoryofsame–differentdiscriminationprobabilities.A toyexample inFigure 9.4providesanillustration.AdetaileddescriptionandexamplescanbefoundinDzhafarov(2002d,2003a)andDzhafarovandColonius(2005a,2006a).

Theargumentsxandyofthediscriminationprobabilityfunction

ψ∗xy=Pr[xandyarejudgedtobedifferent]

* Asadata-analyticprocedure,UFSisimplemented(asofSeptember2009)inthreecomputerprograms:the R-language package “fechner” described in Ünlü, Kiefer, and Dzhafarov (2009) and availableatCRAN;aMATLAB-basedprogramFSCAMDSdevelopedatPurdueUniversityandavailableathttp://www1.psych.purdue.edu/~ehtibar/Links.html;andaMATLABtoolboxdevelopedatOldenburgUniversityandavailableathttp://www.psychologie.uni-oldenburg.de/stefan.rach/31856.html.

Y116134.indb 191 10/10/11 2:12 PM


belongtotwodistinctobservationareas,

ψ*:𝔖*1×𝔖*

2→[0,1].

Thus,𝔖*1 (thefirstobservationarea)may represent stimulipresentedchronologi-

callyfirstorontheleft,whereas𝔖*2(thesecondobservationarea)designatesstimuli

presented,respectively,chronologicallysecondorontheright.Theadjectives“first”and“second”refertotheordinalpositionsofstimulussymbolswithinapair(x,y).

Forx,x′∈𝔖*1,wesaythat the twostimuliarepsychologically equal (ormeta-

meric) if ψ*xy = ψ*x′y for any y ∈𝔖*2. Analogously, the psychological equality

fory,y′∈𝔖*2isdefinedbyψ*xy=ψ*xy′,foranyx∈𝔖*

1.Itisalwayspossibleto“reduce”theobservationareas,thatis,relabeltheirelementssothatpsychologicallyequalstimulireceiveidenticallabelsandarenolongerdistinguished.Thediscrimi-nationprobabilityfunctionψ*canthenberedefinedas

~ψ:𝔖1×𝔖2→[0,1].

ThelawofRegularMinimalityisthestatementthattherearefunctionsh:𝔖1→𝔖2andg:𝔖2→𝔖1suchthat

1111

0.60.10.8x7

0.70.70.70.70.50.90.6x5

0.70.70.70.70.50.90.6x3

1110.5y7

1110.5y4

0.10.80.80.9y2

1110.5y6

1110.5y5

0.6111y3

0.80.10.10.6y1

x6x4x2x1

ya

yb

yc

yd

xa xc xb xd

110.70.5yd

0.610.51yc

0.10.80.90.9yb

0.80.10.60.6ya

xdxcxbxa

0.50.711d10.50.61c

0.90.90.10.8b0.60.60.80.1adcba

Observation area 1, xc

a b c dya

xdyb

xbyc

xaydObservation area 2,

Common labels,

21

* **

12

21

Figure 9.4 AtoyexampleusedinDzhafarov&Colonius(2006a).Thetransformationfrom(𝔖*

1,𝔖*2,ψ*)to(𝔖1,𝔖2, ~ψ)istheresultof“lumpingtogether”psychologicallyequalstimuli

(e.g.,thestimuliy4,y5,y6,y7arepsychologicallyequalin𝔖*2,stimulix2andx4arepsycho-

logicallyequalin𝔖*1).Thespace(𝔖1,𝔖2, ~ψ)satisfiestheRegularMinimalitycondition(the

minimumineachrowisalsotheminimuminitscolumn)becauseofwhich(𝔖1,𝔖2, ~ψ)canbecanonicallytransformedinto(𝔖,ψ),bymeansofthetransformationtableshowninbetween.

Y116134.indb 192 10/10/11 2:12 PM


P

P

1

2

( ) ( )[ ] < ≠ ( )

( )

ψ ψ

ψ

x h x xy y h xfor all

gg y y xy x g y( ) < ≠ ( )

( )

ψ for all

P3 h g≡ −1

Stimulusy=h(x)∈𝔖2iscalledthePoint of Subjective Equality (PSE)forx∈𝔖1;analogously,x=g(y)∈𝔖1isthePSEfory∈𝔖2.ThelawofRegularMinimalitystatesthereforethateverystimulusineachofthe(reduced)observationareashasauniquePSEintheotherobservationarea,andthatyisthePSEforxifandonlyifxisthePSEfory.Insomecontextsthelawofregularminimalityisanempiricalassumption,butitcanalsoserveasacriterionforaproperlydefinedstimulusspace.ForadetaileddiscussionofthelawanditscritiquesseeDzhafarov(2002d,2003a,2006),DzhafarovandColonius(2006a),andEnnis(2006).

DuetothelawofRegularMinimality,onecanalwaysrelabelthestimuliin𝔖1or𝔖2sothatanytwomutualPSEsreceiveoneandthesamelabel.Inotherwords,onecanalwaysbijectivelymap𝔖1→𝔖and𝔖2→𝔖sothatx↦aandy↦aifandonlyifx∈𝔖1andy∈𝔖2aremutualPSEs:y=h(x),x=g(y).Thesetoflabels𝔖iscalledacanonically transformed stimulusset.Itselementstoo,forsimplicity,arereferredtoasstimuli.Thediscriminationprobabilityfunction~ψcannowbepresentedinacanonical form,

ψ:𝔖×𝔖→[0,1],

withtheproperty

ψaa<minψab,ψba

foranyaandb≠a.Notethatthefirstandthesecondainψaamayverywellrefertophys-icallydifferentstimuli(equivalenceclassesofstimuli):henceoneshouldexercisecau-tioninreferringtoψaaastheprobabilitywithwhichaisdiscriminatedfrom“itself.”

9.2.2. From discrimination to dissimilarity

Forthecanonicallytransformedfunctionψ,thepsychometric increments of the first and second kind aredefinedas,respectively,

Ψ(1)ab=ψab–ψaa

and

Ψ(2)ab=ψba–ψaa.

Duetothecanonicalformofψthesequantitiesarealwayspositiveforb≠a.

Y116134.indb 193 10/10/11 2:12 PM


ThemainassumptionofUFSaboutthesepsychometricincrementsisthatbothofthemaredissimilarityfunctions.Themeaningofthisstatementbecomesclearlater,afteraformaldefinitionofadissimilarityfunctionisgiven.

DenotingbyD eitherΨ(1) orΨ(2) onecancompute the (generallyasymmetric)FechneriandistanceGabbyconsideringallpossiblefinitechainsofstimulix1…xkforallpossiblekandputting

G D D Dk x x

kk

ab ax x x x b= + +…+[ ]…inf, 1

1 1 2 .

TheoverallFechneriandistanceisthencomputedas

G*ab=G1ab+G1ba.

This quantity can be interpreted as the infimum of D-lengths of all finite closedloops thatcontainpointsaandb.Thatis,

G D D D Dkl

kkl

* inf,,

ab ax x x x b byx xy y

= + +…+ +……11

1 1 2 11 1 2+ +…+[ ]D D ly y y a

ItiseasytoseethattheD-lengthofanygivenloopremainsinvariantifD≡Ψ(1)isreplacedwithD≡Ψ(2)andtheloopistraversedintheoppositedirection.ThevalueofG*abthereforedoesnotdependonwhichofthetwopsychometricincrementsistakenforD.HenceforthwetacitlyassumethatDmaybereplacedwitheitherΨ(1)orΨ(2),nomatterwhich.

9.3 Dissimilarity Cumulation theory

9.3.1 Topology and uniformity

ToexplainwhatitmeansforafunctionD:𝔖×𝔖→ℝtobeadissimilarityfunc-tion,webeginwithamoregeneralconcept.FunctionD :𝔖×𝔖→ℝ isa(uni-form)deviation function ifithasthefollowingproperties:foranya,b∈𝔖andanysequencesan,a′n,bn,b′nin𝔖,

D D

D D

D

1 0

2 0

3

. ;

. ;

. (

[ ]

[ ]

[ ]

≠ ⇒ >

=

a b ab

aa

Uniform CContinuity) If and thenD Dn n n na a b b′ → ′ →0 0, D Dn n n n′ ′ − →a b a b 0.

SeeFigure 9.5foranillustrationofPropertyD3.IfDisasymmetricmetric,thenitisadeviationfunction,withtheuniformcontinuitypropertyholdingasatheorem.

Y116134.indb 194 10/10/11 2:12 PM


IfDisanasymmetricmetric,thenitisadeviationfunctionifandonlyifitaddition-allysatisfiesthe“invertibilityinthesmall”condition,

Dana′n→0⇒Da′nan→0.

Inthefollowingthetermmetric (ordistance),unlessspecificallyqualifiedassym-metric,alwaysreferstoanasymmetricmetric(distance)invertibleinthesmall.

Dinduceson𝔖thenotionofconvergence:wedefinean↔bntomeanDanbn→0.The notation is unambiguous because convergence ↔ is an equivalence relation(i.e.,it is reflexive, symmetric, and transitive). In particular, an ↔ a means bothDaan0andDana→0.Theconvergence(a1

n, …, ank)

↔(b1n, …, bn

k)canbedefined,e.g.,bymaxiDan

ibni→0.

Atopological basis on𝔖isafamilyofsubsetsof𝔖covering𝔖andsatisfyingthefollowingproperty(Kelly,1955,p.47):if𝔞and𝔟arewithinthebasis,thenforanyx∈𝔞∩𝔟thebasiscontainsaset𝔠thatcontainsx.Givenatopologicalbasison𝔖,thetopology on𝔖(afamilyofopen sets “based”onthisbasis)isobtainedbytakingallpossibleunionsofthesubsetscomprisingthebasis(includingtheemptyset,whichistheunionofanemptyclassofsuchsubsets).DeviationDinduceson𝔖atopologybasedon

𝔅D(x,ε)=y∈𝔖:Dxy< ε

takenforallx∈𝔖andallrealε>0.Wecallthistopology(basedon𝔅D-balls)theD-topology.

Thesetopologicalconsiderations,asitturnsout,canbestrengthened:Dinduceson𝔖notonlyatopologybutamorerestrictivestructure,calleduniformity.Recall(Kelly,1955,p.177)thatafamilyofsubsetsof𝔖×𝔖formsabasis for a unifor-mity on𝔖ifitsatisfiesthefollowingfourproperties:if𝔄and𝔅aremembersofthebasis,then

1.𝔄includesasitssubsetΔ=(x,x):x∈𝔖. 2.𝔄−1=(y,x):(x,y)∈𝔄includesasitssubsetamemberofthebasis.

an

a n

b n

bn

Figure 9.5 An illustration for property D3 (uniform continuity). Consider an infinitesequenceofquadrilateralsa1a′1b′1b1,a2a′2b′2b2,…,suchthattheD-lengthsofthesidesana′n

andbnb′n(directedasshownbythearrows)convergetozero.ThenthedifferencebetweentheD-lengthsofthesidesanbnanda′nb′n(inthedirectionofthearrows)convergestozero.

Y116134.indb 195 10/10/11 2:12 PM


3.Forsomememberℭof thebasis,(x,z)∈𝔖2: forsomey, (x,y)∈ℭ∧

(y,z)∈ℭ⊂𝔄. 4.𝔄∩𝔅includesasitssubsetamemberofthebasis.

Givenauniformitybasison𝔖, theuniformityon𝔖 (“based”onthisbasis) isobtainedbytakingeachmemberofthebasisandformingitsunionswithallsubsetsof𝔖×𝔖.Amemberofauniformityiscalledanentourage.DeviationDinduceson𝔖auniformitybasedonentourages

𝔘D(ε)=(x,y)∈𝔖2:Dxy< ε

takenforallrealε >0.Thisuniformitysatisfiestheso-calledseparationaxiom:

∩ε𝔘D(ε)=(x,y)∈𝔖2:x=y.

WecallthisuniformitytheD-uniformity.TheD-topologyispreciselythetopologyinducedbytheD-uniformity(Kelly,1955,p.178):

𝔅D(x,ε)=y∈𝔖:(x,y)∈𝔘D(ε)

istherestrictionofthebasicentourage𝔘D(ε)tothepairs(x=const,y).

9.3.2 Chains and dissimilarity function

Chainsinspace𝔖arefinitesequencesofelements,writtenasstrings:ab,abc,x1…xk,etc.Notethattheelementsofachainneednotbepairwisedistinct.Achainofcardinality k(ak-chain)isthechainwithkelements(vertices),hencewithk–1links(edges).Forcompleteness,wealsoadmitanemptychain,ofzerocardinality.Weusethenotation

D Dk i i

i

k

x x x x1 1

1

1

… = +

=

−

∑ ,

andcallittheD-lengthofthechainx1…xk.Iftheelementsofachainarenotofinterest,itcanbedenotedbyaboldfacecapi-

tal,suchasX,withappropriateornaments.Thus,XandYaretwochains,XY istheirconcatenation,aXbisachainconnectingatob.ThecardinalityofchainXisdenoted∙X∙.Unlessotherwisespecified,withinasequenceofchains,Xn,thecardinal-ity∙Xn∙generallyvaries:Xn=xn

1…xnkn

.AuniformdeviationfunctionDon𝔖isauniformdissimilarity(or,simply,dis-

similarity)functionon𝔖ifithasthefollowingproperty:

[D4.]foranysequenceofchainsanXnbn,

D Dn n n n na X b a b→ ⇒ →0 0.

Y116134.indb 196 10/10/11 2:12 PM


SeeFigure 9.6foranillustration.IfDisametric,thenDisadissimilarityfunctionasatrivialconsequenceofthetriangleinequality.

9.3.3 Fechnerian distance

Thesetofallpossiblechainsin𝔖isdenotedbyC𝔖,orsimplyC.WedefinefunctionGabby

G DC

ab aXbX

=∈inf .

Gabisametric,andG*ab=Gab+Gbaisasymmetricmetric(alsocalled“over-all”).WesaythatthemetricGandtheoverallmetricG*areinducedbythedissimi-larityD.Clearly,G*abcanalsobedefinedby

G D DC C

* inf inf, ,

ab aXbYa bXaYbX Y X Y

= =( )∈ ( )∈2 2

.

9.3.4 Topology and uniformity on (𝔖,G)

Itcanbeshownthat

Danbn→0⇔Ganbn→0,

and

an↔bn⇔Ganbn→0⇔Gbnan→0⇔G*anbn=G*bnan→0.

Asaconsequence,Ginduceson𝔖atopologybasedonsets

𝔅G(x,ε)=y∈𝔖:Gxy< ε

taken forallx∈𝔖 andpositiveε.This topologycoincideswith theD-topology.Analogously,Ginduceson𝔖auniformitybasedonthesets

𝔄G(ε)=(x,y)∈𝔖2:Gxy< ε

takenforallpositiveε.ThisuniformitycoincideswiththeD-uniformity.ThemetricGisuniformlycontinuousin(x,y),i.e.,ifa′n↔anandb′n↔bn,then

Ga′nb′n−Ganbn→0.

...an

bn

Figure 9.6 AnIllustrationforPropertyD4(chainproperty).Consideraninfinitesequenceofchainsa1X1b1,a2X2b2,…,suchthat∙Xn∙increasesbeyondboundswithn→∞,andDanXnbnconvergestozero.ThenDanbn(theD-lengthofthedottedarrow)convergestozerotoo.

Y116134.indb 197 10/10/11 2:12 PM


Thespace(𝔖,D)beinguniformandmetrizable,wegetitsstandardtopologicalcharacterization(see,e.g.,Hocking&Young,1961,p.42):itisacompletely normal space,meaningthatitssingletonsareclosedandanyitstwoseparatedsubsets𝔄and𝔅(i.e.,suchthat𝔄∩𝔅=𝔄∩𝔅=∅)arecontainedintwodisjointopensubsets.

Thefollowingisanimportantfactwhichcanbeinterpretedasthatof internalconsistency of the metric G induced by means of dissimilarity cumulation: onceGabiscomputedastheinfimumoftheD-lengthacrossallchainsfromatob,theinfimumoftheG-lengthacrossallchainsfromatobequalsGab:

DaXnb→Gab⇒GaXnb→Gab,

whereweusethenotationforcumulatedG-lengthanalogoustothatforD-length,

G Gk i i

i

k

x x x x1 1

1

1

… = +

=

−

∑ .

Extending the traditionalusageof the term,onecansay thatG isan intrinsic metric.Thisisanextensionbecausetraditionallythenotionofintrinsicmetricpre-supposestheexistenceofpaths(continuousimagesofsegmentsofreals)andtheirlengths.InsubsequentsectionsweconsiderspecialcasesofdissimilaritycumulationinwhichtheintrinsicalityofGdoesacquireitstraditionalmeaning.

9.4 Dissimilarity Cumulation in arc-connected spaces

9.4.1 Path and their lengths

Becausethenotionofuniformconvergenceinthespace(𝔖,D)iswell-defined,

an↔bn⇔Danbn→0,

wecanmeaningfullyspeakofcontinuousanduniformlycontinuousfunctionsfromrealsinto𝔖.

Let f : [a,b]→𝔖,or f ∙[a,b],be somecontinuous (henceuniformlycontinu-ous)functionwithf(a)=a,f(b)=b,whereaandbarenotnecessarilydistinct.Wecallsuchafunctionapath connectingatob.Aspaceiscalledarc-connected (orpath-connected) ifany twopoints in itcanbeconnectedbyapath.Even thougharcshavenotyetbeenintroduced,theterms“arc-connected”and“path-connected”aresynonymous,because(𝔖,D)isaHausdorffspace,soiftwopointsinitarecon-nectedbyapaththeyarealsoconnectedbyanarc(see,e.g.,Hocking&Young,1961,pp.116–117).Hereafterweassumethat(𝔖,D)isanarc-connectedspace.

Chooseanarbitrarynet on[a,b],

μ=(a=x0≤x1≤…≤xk≤xk+1=b),

wherethexi’sneednotbepairwisedistinct.Wecallthequantity

Y116134.indb 198 10/10/11 2:12 PM


δµ = −( )= …

+max, , ,i k

i ix x0 1

1

thenet’smesh.Asδμn→0,netsμnprovideaprogressivelybetterapproximationfor[a,b].

Givenanetμ=(x0, x1, …, xk, xk+1),anychainX=x0x1…xkxk+1(withtheelementsnotnecessarilypairwisedistinct,andx0andxk+1notnecessarilyequaltoaandb)canbeusedtoformachain-on-net

Xµ=((x0,x0),(x1,x1), …,(xk,xk),(xk+1,xk+1)).

Denotetheclassofallsuchchains-on-netsXµ(forallpossiblepairsofachainXandanetμofthesamecardinality)byℳa

b.Notethatachain-on-netisnotafunctionfromx:xisanelementofμinto𝔖,foritmayincludepairs(xi=x,xi)and(xj=x,xj)withxi≠xj.NotealsothatwithinagivencontextXµandXνdenoteoneandthesamechainontwonets,whereasXµ,Yµdenotetwochainsonthesamenet.

Wedefinetheseparation ofthechain-on-netXµ=((x0,x0), …,(xk+1,xk+1))∈ℳab

fromapathf ∙[a,b]as

σ µ

µf X f x( ) = ( )

∈maxx

i ii

D x .

Forasequenceofpathsfn∙[a,b],anysequenceofchains-on-netsXnµn∈ℳa

bwithδμn→0andσfn(Xn

µn)→0isreferredtoasasequenceconverging with fn.WedenotesuchconvergencebyXn

µn→fn.Inparticular,Xnµn→fforafixedpathf ∙[a,b]means

that δμn→0andσf(Xnµn)→0:inthiscasewecansaythatXn

µnconvergesto f.SeeFigure 9.7foranillustration.

WedefinetheD-length off ∙[a,b]as

D D Df X XX f

Xf

= =→ →

( )→lim inf lim inf

µ

µδµ

σ0

0

,

whereallXµ∈ℳab.

δ

σ

a

ab

b

Figure 9.7 Achain-on-netXµisconvergingtoapathfasσ=σf(Xµ)→0andδ=δμ→0.

Y116134.indb 199 10/10/11 2:12 PM


Given apath f ∙ [a,b], the classof the chains-on-netsXµsuch thatδμ<δ andσf(Xµ)< εisnonemptyforallpositiveδandε,becausethisclassincludesappropri-atelychoseninscribed chains-on-nets

((a,a),(x1,f(x1)), …,(xk,f(xk)),(b,b)).

Here,obviously,σf(Xµ)isidenticallyzero.Note,however,thatwithourdefinitionofD-lengthonegenerallycannotconfineone’sconsiderationtotheinscribedchains-on-netsonly(seeFigure 9.8).

Letusconsidersomebasicpropertiesofpaths.Foranypathf ∙[a,b]connectingatob,

Df≥Gab.

Thatis,theD-lengthofapathisboundedfrombelowbyGab.ThereisnoupperboundforDf; thisquantityneednotbefinite.Thus, it isshownbelowthatwhenDisametric,thenotionofDfcoincideswiththetraditionalnotionofpathlength;andexamplesofpathswhose length, in the traditionalsense, is infinite,arewell-known(see,e.g.,Chapter1inPapadopoulos,2005).WecallapathD-rectifiable ifitsD-lengthisfinite.

...

...

(1, 0)

(a1, a2) (b1, b2)

(b1, b 2)(a1, a 2)

(0, 1)(0, 0)

(1, 0)

(0, 1)(0, 0)

b2

a2

a1 b1

x1,i+1x1,ix 2

,ix 2

,i+1

Figure 9.8 A demonstration of the fact that inscribed chains are not suffcient forD-length computations. The D from (a1,a2) to (b1,b2) is defined as ∙a1 − b1∙+ ∙a2 − b2∙+min∙a1−b1∙,∙a2−b2∙.Itisadissimilarityfunction,asillustratedinthetoppanels.Bottomleft:thestaircasechainhasthecumulateddissimilarity2,and2isthetrueD-lengthofthehypotenuse.Bottomright:theinscribedchainhasthecumulateddissimilarity3.

Y116134.indb 200 10/10/11 2:12 PM


Wenextnotetheadditivitypropertyforpathlength.Foranypathf ∙[a,b]andanypointz∈[a,b],

Df ∙[a,b]=Df ∙[a,z]+Df[z,b].

Dfforanypathf ∙[a,b]isnonnegative,andDf=0ifandonlyiffisconstant(i.e.,f([a,b])isasingleton).

Thequantity

σ f g f g( ) = ( ) ( )∈[ ]max

,x a bD x x

iscalledtheseparation ofpathg∙[a,b]frompathf∙[a,b].Twosequencesofpathsfnandgnaresaidtobe(uniformly)converging toeachotherifσfn(gn)→0.Duetothesymmetryoftheconvergencein𝔖, thisimpliesσgn(fn)→0,sothedefinitionandterminologyarewell-formed.Wesymbolizethisbyfn→gn.Inparticular,iffisfixedthenasequencefnconvergestofifσf(fn)→0.Wepresentthisconvergenceasfn→f.Notethatiffn→f,theendpointsan=fn(a)andbn=fn(b)generallydependonnanddifferfrom,respectivelya=f(a)andb=f(b).

The following very important property is called the lower semicontinuity ofD-length(asafunctionofpaths).Foranysequenceofpathsfn→f,

lim infn

nD D→∞

≥f f .

9.4.2 G-lengths

BecausethemetricGinducedbyDinaccordancewith

G Dab aXbX

= inf

is itselfadissimilarity function, theG-length ofapath f : [a,b]→𝔖 shouldbedefinedas

Gf = liminf GX, Xμ ∈ℳa

b

XμG→f

where(puttingX=x0x1…xkxk+1),

G G i i

i

k

X x x= +

=∑ 1

0

,

andtheconvergenceX fµ →G (whereμistheneta=x0, x1, …, xk, xk+1=bcorrespond-ingtoX)meanstheconjunctionofδμ→0and

Y116134.indb 201 10/10/11 2:12 PM


σ µf X f x*

, ,max( ) = ( ) →

= … +i ki iG x

0 10 .

Itiseasytosee,however,that X fµ →G andXµ→fareinterchangeable:

X f X fµ µ→ ⇔ →G .

BecauseGisametric,wealsohave,byatrivialextensionoftheclassicaltheory(e.g.,Blumenthal,1953),

Gf=supGZ

withthesupremumtakenoverallinscribedchains-on-netsZν;moreover,

G Gn

nf Z=→∞

lim

foranysequenceofinscribedchains-on-netsZnνnwithδνn→0.

As it turns out, these traditional definitions are equivalent to our definition ofG-length.Moreoverthe D-lengthandG-lengthofapatharealwaysequal:foranypathf,

Df=Gf.

9.4.3 Other properties of D-length for paths and arcs

Thepropertiesestablishedinthissectionparallelthebasicpropertiesofpathlengthin the traditional,metric-based theory (Blumenthal,1953;Blumenthal&Menger,1970;Busemann, 2005).Wenotefirst theuniformcontinuity of length traversedalongapath:foranyD-rectifiablepathf∙[a,b]and[x,y]⊂[a,b],Df∙[x,y]isuniformlycontinuousin(x,y),nondecreasinginyandnonincreasinginx(seeFigure 9.9).

Thenext issueweconsider is the (in)dependenceof theD-lengthofapathonthepath’sparametrization.TheD-lengthofapathisnotdeterminedbyitsimagef([a,b])alonebutbythefunctionf:[a,b]→𝔖.Neverthelesstwopathsf∙[a,b]andg∙[c,d]withoneandthesameimagedohavethesameD-lengthiftheyarerelatedtoeachotherinacertainway.Specifically,thishappensiffandgareeachothers’reparametrizations,bywhichwemeanthat,forsomenondecreasingandonto(hencecontinuous)mappingϕ:[c,d]→[a,b],

g(x)=f(ϕ(x)), x∈[c,d].

Notethatweusea“symmetrical”terminology(each other’s reparametrizations)eventhoughthemappingϕisnotassumedtobeinvertible.Ifitisinvertible,thenitisanincreasinghomeomorphism,andthenitiseasytoseethatDf=Dg.Thisequalityextendstothegeneralcase(seeFigure 9.10).

Wedefineanarc asapaththatcanbereparametrizedintoahomeomorphicpath.Inotherwords,g∙[c,d]isanarcifonecanfindanondecreasingandonto(hence

Y116134.indb 202 10/10/11 2:12 PM


continuous)mappingϕ:[c,d]→[a,b],suchthat,forsomeone-to-oneandcontinu-ous(hencehomeomorphic)functionf:[a,b]→𝔖,

g(x)=f(ϕ(x)),

foranyx∈[c,d].Itcanbeshown(byanontrivialargument)thatanypathcontainsanarcwiththesameendpointsandtheD-lengththatcannotexceedtheD-lengthof

a

b

xym

ab

Figure 9.9 Uniformcontinuityoflength:asxandygetclosertoeachother,thelengthofthecorrespondingpieceofthepathconvergestozero.

b

c

d

ba

a

f

Figure 9.10 Thepathfon[a,b]canbereparametrizedwithoutitslengthaffectedintoapathon[c,d]mappedonto[a,b]byanondecreasingfunctionϕ.

Y116134.indb 203 10/10/11 2:12 PM


thepath(seeFigure 9.11).Statedrigorously,letf∙[a,b]beaD-rectifiablepathcon-nectingatob.Thenthereisanarcg∙[a,b]connectingatob,suchthat

g([a,b])⊂f([a,b]),

and

Dg∙[a,b]≤Df∙[a,b],

wheretheinequalityisstrictiff∙[a,b]isnotanarc.Thisresultisimportant,inpar-ticular,inthecontextofsearchingforshortestpathsconnectingonepointtoanother(Section9.4.4):intheabsenceofadditionalconstraintsthissearchcanbeconfinedtoarcsonly.

9.4.4 Complete dissimilarity spaces with intermediate points

Adissimilarityspace(𝔖,D)issaidtobeaspacewith intermediate points ifforanydistincta,bonecanfindanmsuchthatm∉a,bandDamb≤Dab(seeFigure9.12).Thisnotiongeneralizes thatofMenger convexity (Blumenthal, 1953,p.41;thetermitselfisduetoPapadopoulos,2005).IfDisametric,thespaceisMenger-convexif,foranydistincta,b,thereisapointm∉a,bwithDamb=Dab.(Thetraditionaldefinitionisgivenforsymmetricmetricsbutitcanbeeasilyextended.)

Recallthataspaceiscalledcomplete ifeveryCauchysequenceinitconvergestoapoint.Adaptedto(𝔖,D),thecompletenessmeansthatgivenasequenceofpointsxnsuchthat

limkl

k lD→∞→∞

=x x 0 ,

thereisapointxin𝔖suchthat

xn↔x.

Figure 9.11 Onecanremoveclosedloopsfromapathandbeleftwithashorterarc.

ab

m

abm

Figure 9.12 Pointm is intermediate toa andb ifDamb ≤Dab.E.g., ifD isEuclideandistance(rightpanel),anymonthestraightlinesegmentconnectingatobisintermediatetoaandb.

Y116134.indb 204 10/10/11 2:12 PM


Blumenthal(1953,pp.41–43)provedthatifaMenger-convexspaceiscompletethenacanbeconnectedtobbyageodesic arc, that is,anarchwithDh=Dab(whereD isasymmetricmetric).Asitturnsout,thisresultcanbegeneralizedtononmetricdissimilarityfunctions,inthefollowingsense:inacompletespacewithintermediatepoints,anyacanbeconnectedtoanybbyanarcfwith

Df≤Dab.

SeeFigure 9.13foranillustration.ItfollowsthatGabinsuchaspacecanbeviewedastheinfimumoflengthsofallarcsconnectingatob.Putdifferently,inacompletespacewithintermediatepointsthemetricGinducedbyDisintrinsic,inthetradi-tionalsenseoftheword.

9.5 Conclusion

Letussummarize.UniversalFechnerianScalingisatheorydealingwiththecom-putation of subjective distances from pairwise discrimination probabilities. Thetheoryisapplicabletoallpossiblestimulusspacessubjecttotheassumptionsthat(a)discriminationprobabilitiessatisfy the lawofRegularMinimality,and(b) thetwocanonicalpsychometricincrementsofthefirstandsecondkind,Ψ(1)andΨ(2),aredissimilarityfunctions.

0 1 0 1

0 1 0 1

Figure 9.13 Inacompletespacewithintermediatepointsanypointsaandbcanbecon-nectedbychainswhosecardinalityincreasesbeyondboundsandthedissimilaritybetweensuccessive elements converged to zero. As a result the chains converge, pointwise and inlength,toanarcwhoselengthisnotgreaterthanDab.

Y116134.indb 205 10/10/11 2:12 PM


AdissimilarityfunctionDab(whereDcanstandforeitherΨ(1)orΨ(2))forpairsofstimuliinacanonicalrepresentationisdefinedbythefollowingproperties:

D1.a≠b⇒Dab>0;D2.Daa=0;D3.IfDana′n→0andDbnb′n→0,thenDa′nb′n–Danbn→0;andD4.foranysequenceanXnbn,whereXnisachainofstimuli,DanXnbn→0⇒

Danbn→0.

Itallowsustoimposeonthestimulusspacethe(generallyasymmetric)Fechnerianmetric Gab, computed as as the infimum of DaXb across all possible chainsX inserted between a and b. The overall (symmetric) Fechnerian distance G*abbetweenaandbisdefinedasGab+Gba.ThisquantitydoesnotdependonwhetheroneusesΨ(1)orΨ(2)inplaceofD.

ThedissimilarityDimposesonstimulusspaceatopologyandauniformitystruc-turethatcoincidewiththetopologyanduniformityinducedbytheFechnerianmet-ricG(orG*).ThemetricGisuniformlycontinuouswithrespecttotheuniformityjustmentioned.Stimulusspaceistopologicallycharacterizedasacompletelynor-malspace.

TheDissimilarityCumulationtheorycanbespecializedtoarc-connectedspaceswithnoadditionalconstraintsimposedeitheronthesespacesoronthetypeofpaths.Wehaveseenthatthepathlengthcanbedefinedintermsofadissimilarityfunc-tionasthelimitinferiorofthelengthsofappropriatelychosenchainsconvergingtopaths. Unlike in the classical metric based theory of path length, the convergingchainsgenerallyarenotconfinedtoinscribedchainsonly:theverticesofthecon-vergingchainsareallowedto“jitterandmeander”aroundthepathtowhichtheyareconverging.Giventhisdifference,however,mostofthebasicresultsofthemetric-basedtheoryareshowntoholdtrueinthedissimilarity-basedtheory.

The dissimilarity-based length theory properly specializes to the classical onewhenthedissimilarityinquestionisitselfametric(infactwithoutassumingthatthismetricissymmetric).Inthiscasethelimitinferioroverallconvergingchainscoincideswiththatcomputedovertheinscribedchainsonly.Itisalsothecasethatthelengthofanypathcomputedbymeansofadissimilarityfunctionremainsthesameifthedissimilarityfunctionisreplacedwiththemetricitinduces.

Wehaveconsideredaclassofspacesinwhichthemetricsinducedbythedissimi-larityfunctionsdefinedonthesespacesareintrinsic:whichmeansthatthedistancebetweentwogivenpointscanbecomputedastheinfimumofthelengthsofallarcsconnectingthesepoints.Wecallthemspaceswithintermediatepoints,theconceptgeneralizingthatofthemetric-basedtheory’sMengerconvexity.

AllofthisshowsthatthepropertiesD3andD4ofadissimilarityfunctionratherthanthesymmetryandtriangleinequalityofametricareessentialindealingwiththenotionsofpathlengthandintrinsicmetrics.

Inconclusion,itshouldbementionedthatthenotionofdissimilarityandthethe-oryofdissimilaritycumulationhasabroaderfieldofapplicabilitythanjustdiscrimi-nationfunctions.Thus,itseemsplausibletoassumethatmeansormediansofdirect

Y116134.indb 206 10/10/11 2:12 PM


numerical estimates of pairwise dissimilarities,ofthekindusedinMultidimensionalScaling(MDS,see,e.g.,Borg&Groenen,1997),canbeviewedasdissimilarityval-uesinthetechnicalsenseofthepresenttheory.Thiscreatesthepossibilityofusingthedissimilaritycumulationprocedureasadata-analytic techniquealternative to(and,insomesense,generalizing)MDS.Insteadofnonlinearlytransformingdissim-ilarityestimatesDabintodistancesofapreconceivedkind(usually,Euclideandis-tancesinalow-dimensionalEuclideanspace)onecanusedissimilaritycumulationtocomputedistancesG*abfromuntransformedDabandthenseeifthesestimuliareisometrically(i.e.,withoutchangingthedistancesG*abamongthem)embed-dable in a low-dimensional Euclidean space (or another geometric structure withdesirableproperties).Thisapproachcanbeusedevenifthedissimilarityestimatesarenonsymmetric.Avarietyofmodificationsreadilysuggest themselves,suchastakingintoaccountonlysmalldissimilaritiesinordertoreducethedimensionalityoftheresultingEuclideanrepresentation.

Anotherlineofresearchlinksthetheoryofdissimilaritycumulationwithinfor-mation geometry (see,e.g.,Amari&Nagaoka,2000)andappliestothecategoriza-tionparadigm.Here,eachstimulusa ischaracterizedbyavectorofprobabilities(a1, …, ak),

aii

k

==

∑ 11

,

whereai indicates theprobabilitywithwhicha is classified (byanobserveror agroupofpeople)intotheithcategoryamongcertaink >1mutuallyexclusiveandcollectivelyexhaustivecategories.Itcanbeshown,tomentiononeapplication,thatthesquarerootofthesymmetricalversionoftheKullback–Leiblerdivergencemea-sure(Kullback&Leibler,1951),

D a bab

i ii

ii

k

ab ab= = −( )=

∑DivKL log1

,

isa(symmetric)dissimilarityfunctiononanyclosedsubareaofthearea

x = …( ) > … > =

=∑x x x x xk k i

i

k

1 1

1

0 0 1, , : , , , .

The stimuli x can also be viewed as belonging to a (k – 1)-dimensional unitsphere,withcoordinates x1 , …, xk .ThecumulationofDableadstotheclassi-calforinformationgeometrysphericalmetricinanysphericallyconvexareaofthestimulusspace(i.e.,anareawhichwithanytwostimuliitcontainsalsocontainsthesmallerarcofthegreatcircleconnectingthem).Inthosecaseswherethespheri-calconvexityisnotsatisfied(e.g.,ifthespherehasgapswithnostimuli,orstimuliformadiscreteset),thecomputationofthedistancesalonggreatcircleshastobereplacedwithmoregeneralcomputationsusingfinitechainsofstimuli.

Y116134.indb 207 10/10/11 2:12 PM


Acknowledgment

ThisresearchhasbeensupportedbyNSFgrantSES0620446andAFOSRgrantsFA9550-06-1-0288andFA9550-09-1-0252.Iamgratefultomylong-termcollabora-torHansColoniuswho,amongotherthings,openedtomethewealthoftheGerman-languageliteratureonthesubject.IamgratefultoJamesT.TownsendandDevinBurnswhocriticallyreadthefirstdraftofthechapterandsuggestedimprovements.

References

Amari,S.,&Nagaoka,H.(2000).Methods of information geometry.Providence,RI:AmericanMathematicalSociety.

Blumenthal,L.M. (1953).Theory and applications of distance geometry.London:OxfordUniversity.

Blumenthal,L.M.,&Menger,K.(1970).Studies in geometry.SanFrancisco:W.H.Freeman.Borg,I.,&Groenen,P.(1997).Modern multidimensional scaling.NewYork:Springer-Verlag.Busemann,H.(2005).The geometry of geodesics.Mineola,NY:Dover.Creelman,C.D.(1967).Empiricaldetectabilityscaleswithoutthejnd.Perceptual & Motor

Skills, 24,1079–1084.Dzhafarov,E.N.(2001).Fechnerianpsychophysics.InN.J.SmelserandP.B.Baltes(Eds.)

International encyclopedia of the social and behavioral sciences (Vol. 8, pp. 5437–5440).NewYork:PergamonPress.

Dzhafarov,E.N. (2002a).MultidimensionalFechnerianscaling:Regularvariationversion.Journal of Mathematical Psychology, 46,226–244.

Dzhafarov,E.N.(2002b).MultidimensionalFechnerianscaling:Probability-distancehypoth-esis.Journal of Mathematical Psychology, 46,352–374.

Dzhafarov, E. N. (2002c). Multidimensional Fechnerian scaling: Perceptual separability.Journal of Mathematical Psychology,46,564-582.

Dzhafarov,E.N.(2002d).MultidimensionalFechnerianscaling:Pairwisecomparisons,regu-larminimality, andnonconstant self-similarity.Journal of Mathematical Psychology, 46,583–608.

Dzhafarov, E. N. (2003a). Thurstonian-type representations for “same–different” discrimi-nations: Deterministic decisions and independent images. Journal of Mathematical Psychology,47,208–228.

Dzhafarov,E.N.(2003b).Thurstonian-typerepresentationsfor“same-different”discriminations:Probabilisticdecisionsandinterdependentimages.Journal of Mathematical Psychology,47,229–243.[seeDzhafarov,E.N.(2006).Corrigendumto“Thurstonian-typerepresen-tations for ‘same–different’ discriminations:Probabilistic decisions and interdependentimages.” Journal of Mathematical Psychology,50,511.]

Dzhafarov, E. N. (2004). Perceptual separability of stimulus dimensions: A Fechnerianapproach. In C. Kaernbach, E. Schröger, & H. Müller (Eds.), Psychophysics beyond sensation: Laws and invariants of human cognition (pp.9–26).Mahwah,NJ:Erlbaum.

Dzhafarov, E. N. (2006). On the law of regular minimality: Reply to Ennis. Journal of Mathematical Psychology,50,74–93.

Dzhafarov,E.N.(2008a).Dissimilaritycumulationtheoryinarc-connectedspaces.Journal of Mathematical Psychology,52,73–92.[seeDzhafarov,E.N.(2009).Corrigendumto“Dissimilarity cumulation theory in arc-connected spaces”. Journal of Mathematical Psychology, 53,300.]

Y116134.indb 208 10/10/11 2:12 PM


Dzhafarov, E. N. (2008b). Dissimilarity cumulation theory in smoothly connected spaces.Journal of Mathematical Psychology,52,93–115.

Dzhafarov,E.N.,&Colonius,H. (1999).Fechnerianmetrics inunidimensionalandmulti-dimensionalstimulusspaces.Psychonomic Bulletin and Review,6,239–268.

Dzhafarov, E. N., & Colonius, H. (2001). Multidimensional Fechnerian scaling: Basics.Journal of Mathematical Psychology,45,670–719.

Dzhafarov, E. N., & Colonius, H. (2005a). Psychophysics without physics:A purely psy-chological theory of Fechnerian scaling in continuous stimulus spaces. Journal of Mathematical Psychology,49,1–50.

Dzhafarov, E. N., & Colonius, H. (2005b). Psychophysics without physics: Extension ofFechnerianscalingfromcontinuoustodiscreteanddiscrete-continuousstimulusspaces.Journal of Mathematical Psychology,49,125–141.

Dzhafarov,E.N.,&Colonius,H. (2006a).Regularminimality:A fundamental lawofdis-crimination.InH.Colonius&E.N.Dzhafarov(Eds.),Measurement and representation of sensations (pp.1–46).Mahwah,NJ:Erlbaum.

Dzhafarov,E.N.,&Colonius,H.(2006b).GeneralizedFechnerianscaling.InH.Colonius&E.N.Dzhafarov(Eds.),Measurement and representation of sensations (pp.47–88).Mahwah,NJ:Erlbaum.

Dzhafarov,E.N.,&Colonius,H.(2006c).Reconstructingdistancesamongobjectsfromtheirdiscriminability.Psychometrika,71,365–386.

Dzhafarov, E. N., & Colonius, H. (2007). Dissimilarity Cumulation theory and subjectivemetrics.Journal of Mathematical Psychology,51,290–304.

Falmagne, J. C. (1971). The generalized Fechner problem and discrimination. Journal of Mathematical Psychology,8,22–43.

Fechner, G. T. (1860). Elemente der psychophysik [Elements of Psychophysics]. Leipzig:Breitkopf&Härtel.

Fechner,G.T.(1877).In Sachen der psychophysik [In the matter of psychophysics].Leipzig:Breitkopf&Härtel.

Fechner,G.T.(1887).ÜberdiepsychischenmassprinzipienunddasWeberschegesetz[Onthe principles of mental measurement and Weber’s law]. Philosophische Studien, 4,161–230.

Helmholtz,H.von.(1891).VersucheinererweitertenanwendungdesFechnerschengesetzesimfarbensystem[AnattemptatageneralizedapplicationofFechner’sLaw to thecolorsystem].Zeitschrift für die Psychologie und die Physiologie der Sinnesorgane,2,1–30.

Hocking,J.H.&Young,G.S.(1961).Topology.Reading,MA:Addison-Wesley.Kelly,J.L.(1955).General topology.Toronto:VanNostrand.Krantz, D. (1971). Integration of just-noticeable differences. Journal of Mathematical

Psychology, 8,591–599.Kullback,S.,&Leibler,R.A.(1951).Oninformationandsuffciency.Annals of Mathematical

Statistics,22,79–86.Luce,R.D.(2002).Apsychophysicaltheoryofintensityproportions,jointpresentations,and

matches.Psychological Review,109,520–532.Luce,R.D.(2004).Symmetricandasymmetricmatchingofjointpresentations.Psychological

Review,111,446–454.Luce,R.D.,&EdwardsW.(1958).Thederivationofsubjectivescalesfromjustnoticeable

differences.Psychological Review, 65,222–237.Luce,R.D.,&Galanter,E.(1963).Discrimination.InR.D.Luce,R.R.Bush,&E.Galanter

(Eds.),Handbook of mathematical psychology(Vol.1,191–244).NewYork:Wiley.

Y116134.indb 209 10/10/11 2:12 PM


Papadopoulos, A. (2005). Metric spaces, convexity and nonpositive curvature. Zurich,Switzerland:EuropeanMathematicalSociety.

Pfanzagl,J.(1962).ÜberdiestochastischeFundierungdespsychophysischenGesetzes[Onstochasticfoundationsofthepsychophysicallaw].Biometrische Zeitschrift,4,1–14.

Riesz,R.R.(1933).Therelationshipbetweenloudnessandtheminimumperceptibleincre-mentofintensity.Journal of the Acoustical Society of America,4,211–216.

Schrödinger,E.von.(1920).Farbenmetrik[Colormetrics].Zeitschrift für Physik,12,459–466.Schrödinger,E.von.(1920/1970).Outlineofatheoryofcolormeasurementfordaylightvision.

InD.L.MacAdam(Ed.),Sources of color science (pp.397–447,481–520).Cambridge,MA:MITPress.

Schrödinger,E.von.(1926/1970).Thresholdsofcolordifferences.InD.L.MacAdam(Ed.),Sources of color science (pp.183–193).Cambridge,MA:MITPress.

Stevens, S. S. (1975). Psychophysics: Introduction to its Perceptual, Neural, and SocialProspects.NewYork:Wiley.

Thurstone, L. L. (1927a). Psychophysical analysis. American Journal of Psychology, 38,368–389.

Thurstone, L. L. (1927b). A law of comparative judgments. Psychological Review, 34,273–286.

Ünlü,A.,Kiefer,T.,&Dzhafarov,E.N.(2009).FechnerianscalinginR:ThepackageFechner.Journal of Statistical Software,31,1–24.

Y116134.indb 210 10/10/11 2:12 PM

211

Neural networks and fuzzy systems

Christian Eitzinger and Wolfgang HeidlProfactorGmbHSteyr-Gleink,Austria

10.1 Introduction

Neuralnetworksandfuzzysystemsbothbelongtotheareaofsoftcomputing.Theytrytoreproducepropertiesofhumanthinkingandtheneurobiologyofthehumanbraintocreatenewmathematicalstructures.Thisnewwayoflookingatmathematicshasledtodevelopmentsinarangeoffieldssuchasmodelingofhumanvision,con-trolofcomplexdynamicalsystems,andeventhepredictionoffinancialmarkets.

For the purpose of “measurement with persons,” neural networks and fuzzysystemsareofhighrelevance,becausetheybothconsiderthe“humanelement”atdifferentlevelsofabstraction.Thisspecificpropertymakestheminterestingmath-ematicalstructuresformodelinghumanjudgment,behavior,ordecisions.

Bothneuralnetworksandfuzzysystemshavedevelopedintohugefieldsofresearchandwearethuslimitedtoonlyabriefintroduction.Weexplainthebasicelementsthatmakeupneuralnetworksandfuzzysystemsandthentakeabigstepforward,byalmost40yearsofresearch,torecentmodelsofthehumanvisualcortex.

10.2 Neural networks

The development of neural networks was initiated by the wish to reproduce thecapabilityof thehumanbrain todealwithcomplicatedtaskssuchasunderstand-ingspeechoridentifyingobjectsinanimage.Theobservationwasthatthehumanbrainwasoutperformedbyeven thesimplestcalculator in termsofprecisionandspeedwhendoingbasiccalculations,butthatartificialsystemsutterlyfailedinmanytasksthatwouldbeconsideredstraightforwardbyahuman.Itwassoonunderstoodthatcomputingpoweralonewouldnotsolvetheproblemandthatsomethingmorewasneededtoenhancetheperformanceofartificialsystemsoncomplicatedtasks.Consequently, the idea emerged to use the results of neurophysiology and to trytoduplicatetheseinartificialsystems,thuscombiningtheprecisionandspeedofcomputerswiththeflexibilityandlearningcapabilitiesofthehumanbrain.Agoodoverviewcanbefoundin,forexample,LinandLee(1995).

10

Y116134.indb 211 10/10/11 2:12 PM


10.2.1 Properties of biological neural networks

Thereareseveralfeaturesthatmadebiologicalneuralnetworksdifferentfromarti-ficialsystemsatleastbythetimetheideaofneuralnetworkswascreated.Todaywemightlookatthesefeaturesslightlydifferently,inthepresenceoftheWeb2.0andGRIDcomputing,butwithrespecttosmallerentitiesthemainissuesarestillvalid:

Adaptivity:Thehumanbrain isdynamicallychangingandadapting tonewinputs,whichdonotappear inapreviouslydefinedpattern.Morerecentinputswillbemorerelevanttothebrainthaninputsfromthedistantpast.Thebrainisabletolearnfromexamplesbycomplicatedrewardandpun-ishmentprocesses.

Interconnected structure:Thebasiccomponentsofthebrain—theneurons—make up a highly interconnected structure. The brain itself consists ofapproximately 10¹¹ neurons, where each neuron is connected to another10,000neurons. It is in these connections that information is storedandlearningtakesplace.

Structural robustness:Asthenumberofneuronsisverylarge,thefunction-ingofthebraindoesnotdependonasingleneuronandnotevenonalargenumber of neurons. If many neurons cease to function, then the perfor-manceofthebraingraduallydecreasesbutdoesnotdeteriorateabruptly.Weobservethisaspartoftheagingprocessthatisaccompaniedbyaslowdecayof thenumberofneurons.Thisrobustnessachievedbydistributedstoringofknowledgeisapropertyofhugeinterestinartificialsystemsandinparticularinmilitaryapplications.

Hierarchical structure: The information processing from the sensors to thehighercorticalregionsisnotaone-stepprocess,butthebrainissetupinthe formof interconnected layers,whereeach layer fulfills specific tasks.Modern diagnostic technologies such as functional magnetic resonanceimaging(fMRI)allowustoidentifyregionsthatbecomeactivatedduringvarioustasks.Thistechnologyisstillinitsearlystagesandmuchworkneedstobedoneonrefiningexperimentsinordertoprovidemoreinsightnotonlyintothespatialorganizationofbrainareasbutalsotheirtemporalhierarchy.

Size:Intermsofcomputingelementsthehumanbrainishuge.Evencomparedtohigh-performancecomputers,thenumberofneuronsandtheirconnec-tionsisstilllarge.Inadditiontothistheadultbrainhashadmanyyearsordecadestolearnandithasapowerfulsensorysystematitsdisposalthatprovidesinputandpositiveornegativefeedback.Itthuscreatestheexperi-encethatmakeseachindividualunique.

Parallel computation:Processinginthebrainismassivelyparallelandasyn-chronous,whichenablesthebraintoprocesslargeamountsofinformationinacomparativelyshorttime.Evenstructuresofsequentiallayersthatwereusedinearliermodelsofthebrain(e.g.,inhumandecisionmaking)arenowbeingreviewedandreplacedbyhighlyinterconnectedstructures thatareassumedtobeevaluatedinparallel.

Y116134.indb 212 10/10/11 2:12 PM

Neural networks and fuzzy systems 213

Wenowstarttodevelopartificialneuralnetworksandattheendofthe“neuralnetworks”sectiontakealookbackattheirpropertiesandcomparethemtothoseofbiologicalneuralnetworks.

10.2.2 Mathematical representation of artificial neural networks

Thebasicbiologicalcomputingelementsofwhichthebrainisbuiltaretheneurons.Eachneuronconsistsofacellbody(soma),extensionsthattransferincomingsignalstothesoma(dendrites),andasingleoutputnervefiber(axon)whichsendssignalstootherneurons.The junctionpoints that connect the axons to thedendritesof thenextneuronare called synapses; this is theplacewerememory resides. In all ofthesepartscomplicatedelectrochemicalprocessesgeneratethewell-knownelectri-calbehaviorofneurons(GuptaandKnopf,1993).

Intermsofinformationprocessingeachneurontransformstheaggregationofthestimulussignalsreceivedbythedendritesintoasequenceoffrequency-modulatedpulsescalledactionpotentials.Twoimportantpropertiesoftheactionpotentialaredirectlyrelatedtotheencodingabilitiesofaneuron.Thefirstoneistherisetime(latency),whichittakesfortheresultingactionpotentialtoriseafterthestimulushasbeenapplied.Thisresponsetimehasbeenobservedtodecreaseexponentiallywiththeintensityofthestimulus.Thesecondpropertyistheminimumtimethathastopassbeforeanotheractionpotentialcanbegeneratedbytheaxon.Thistime,whichistheminimumtimebetweentwoconsecutivepulses,iscalledtherefractoryperiod.Ifastimulusgreaterthanaspecificthresholdvalueisapplied,boththelatencyandtherefractoryperiodwillcontrolthefrequencyofthegeneratedactionpotentials.Astimuluswithhighintensitywillgenerateashortrisetimeandashortrefractoryperiod,thusproducingahighpulsefrequency.Fromexperimentsithasbeenfoundthattherelationshipbetweentheintensityofthestimulusandthefrequencyoftheactionpotentialsislinearoveralimitedrangewithasaturationcharacteristicathighandlowstimuluslevels.

If the dynamical processes within the neuron are neglected, then it can bedescribedasafunctionf:RN→RthatmapsavectorofNinputvariables(stimuli)tooneoutput(frequencyofactionpotentials).Toresemblethebiologicalconceptofneuronsthisfunctioniscomposedofalinearpart,whichistheaggregationoftheincomingstimuli,where the inputsxundergosomekindof linear transformationsuchas a=wTx (10.1a)

togive theactivationa of thisneuron.This also resembles theprocessingof theinputsinthesynapticconnections,whereconnectionswithpositiveweights(wi>0)areconsideredtobeexcitatoryandconnectionswithnegativeweights(wi<0)areinhibitory.Theoutputisthenfoundbyapplyingausuallynonlinearactivationfunc-tion,ofwhichthestep-functionisthesimplest:

y step aa

a= =

<≥

( )0 0

1 0

if

if (10.1b)

Y116134.indb 213 10/10/11 2:12 PM


Assaidbefore,theneuronalsoincludessomekindofthresholdoperation,wheretheactivationhastoriseaboveaspecificvalue,beforeanoutputisproduced.Tointe-gratethisbehaviorintoourmodel,Equation(10.1a)willbemodifiedto

a=wTx+b, (10.1c)

wherebiscalledbiasoroffsetvalue.DependingonwhetherwTxisgreaterthanbanoutputof1or0isproduced.Thethreeformulas(10.1a–c)constitutedthefirstmodelofaneurondevelopedaround1940.However,thediscontinuousbehaviorofthestep-functionmadeithardtoanalyzelargerstructuresofsuchneurons,inpar-ticulargiventhelimitedcomputingpoweravailableatthattime.About30yearslaterresearchonartificialneuronsandneuralnetworksre-emergedandledtothefieldthatwenowknowasartificialneuralnetworks.

Inadditiontothedevelopmentofcomputers,themostimportantstructuralmodi-ficationwasthereplacementofthestep-functionwithacontinuousmappingofacti-vationstotheintervals[0,1]or[–1,1].Thishasthecomputationaladvantagethattheoutputisnotlimitedtothevalues0and1,andthatthefunctioniscontinuouslydif-ferentiable,whichgivesrisetoefficienttrainingmethods.CommonlyusedactivationfunctionsareshowninFigure 10.1.

Amongthemtherearetheunipolarsigmoidactivationfunction

ye a=

+ −1

1

andthebipolarsigmoidactivationfunctions

ye a=

+−−

21

1 ory=tanh(a).

Anotheractivationfunction,whichdoesnotexactlyfitintotheschemeintroducedabove,istheradialbasisfunction,wheretheoutputsarefoundfrom

y ex wi i

= ∑ −12

2( ).

Activation ActivationActivation

Out

put

Radial basis1.5

10.5

0

–0.5

–1–1.5

–10 –6 –2 0 2 6 10 –10 –6 –2 0 2 6 10 –10 –6 –2 0 2 6 10

Unipolar sigmoid Bipolar sigmoid

Figure 10.1 Shapesofcommonlyusedactivationfunctions.

Y116134.indb 214 10/10/11 2:12 PM


Thisfunctionwillgenerategreateroutputsifthevectorofinputsxissimilartothevectorofweightswandthedistancebetweenthetwovectorsissmall.

Independentof the specificactivation functionused, theneuronsare setup inlayersandthelayersarethencombinedtoformmultilayernetworks.Thisusuallyallows an efficientmathematical description in termsofmatrixoperations. For alayerofNneuronsandMinputstheoutputofneuroni, i = 1,2,…,N withactivationfunctionf:RM→Rcanbefoundby

y f w x bi iT

i= +( )

orforthewholelayerf:RM→RN

y f Wx b= +( ),

whereWisthematrixofweightsandbisthevectorofoffsetvalues.Theactivationfunction f isappliedtoeachelementof theresultingvectorofactivations.Neuralnetworksusuallyconsistofaninputlayer,anumberofhiddenlayers,andanout-putlayer.Forsimplefeed-forwardnetworkseachlayerprocessestheoutputsoftheprecedinglayerandsendsitsownoutputstothefollowinglayer.Whenoutputsaredirectedbacktothesameorprecedinglayers,thenthenetworkisafeedbacknet-work.Feedbacknetworks,whichhaveclosedloops,arecalledrecurrentnetworks.Atypicalstructureofafeed-forwardnetworkisshowninFigure 10.2.

Learninginthesenetworksisdonealmostexclusivelybyvariantsoftheso-calledbackpropagationalgorithm.Thebasicideaisverysimple.Theoutputofthenetworkyiscomparedtoatargetvector t,resultinginanerrore= t – y.BasedonthiserrorchangesfortheweightsΔwarecalculatedsothattheerrordecreases.Anadditionalerrorsignaliscalculatedfortheinputsofthelayer,andbasedonthiserrorsignaltheweightsoftheupstreamlayeraremodified.Thisprocesscanberepeatedrecur-sivelyuntilwe reach the input layer.Quite soon itwas found that thisprocedure

“0”

“1”

“2”

Output layerHidden layerInput layer

24 inputs(4 × 6 gary values)

10 O

utpu

ts

“9”

Figure 10.2 Feed-forwardnetworkforopticalcharacterrecognitionwithaninput,ahidden,andanoutputlayer.

Y116134.indb 215 10/10/11 2:12 PM


isequivalent toagradientdescentmethod,wherethechangesfortheweightsarecalculatedsoastoproceedinthedirectionoftheerrorgradient,whenminimizingthesumofsquareserror.

∆ ∆wt y

dwand w w wk

k

k k k= −∂ −

= ++

12

2

1

( )λ .

Repeatedupdatingoftheweightswithanappropriate,smallstepsizeλfinallyleadstoalocalminimumoftheerrorandthenetworkhaslearnedtoreproducethetargetvectort.

An example of a neural network for optical character recognition is shown inFigure 10.2.The input isasmall imagecontainingacharacter0..9.The image issplitintoa4×6gridandtheaveragegrayvalueiscalculatedforeachgridelement.These24valuesconstitutetheinputtothenetwork,whichconsequentlyhasasizeof24neuronsintheinputlayer.Asmallhiddenlayerwith8neuronsthenfeedsitsoutputs into an output layer with 10 neurons. Each output neuron represents onecharacterfrom0to9andistrainedtoproduceavalueof0.9ifthecharactershowninthebitmapistheonerepresentedbythisneuron.Otherwisethevalueistrainedtobe0.1.Thisnetworkistrainedon3,000images,for5,000iterationsusingtheback-propagationalgorithm,andachievesanaveragerecognitionrateof98%.Ithastobenotedthatthisexampleisjustfordemonstratingtheuseofneuralnetworks;itdoesnotrepresentthestateoftheartwithrespecttoopticalcharacterrecognition.

10.2.3 Properties of artificial neural networks

Adaptivity:Theadaptationofartificialneuralnetworkshasbeenachievedbythe different learning schemes. It can be proven that a sufficiently largeneuralnetwork isable toapproximateanycontinuous function toacho-sendegreeofaccuracy.Clearly, learningbybackpropagation isnotverystrongly related to the natural learning process, but it has proved to beeffectiveanddifferentvariantsofbackpropagationarestillthemaintrain-ingmethodsusedforneuralnetworks.

Interconnected structure:Theinterconnectedstructureoftheneuralnetworkis the property that most closely resembles the natural properties of thebrain. Similar to the brain, the information is stored in the connections(theweights)ratherthantheneurons.Themaindifferencebetweenartifi-cialneuralnetworksandbiologicalnetworksisthenumberofconnections.Inorderforaneuralnetworktobeevaluatedandtrainedinareasonableamount of time, artificial networks are usually quite small compared tobiologicalnetworks.

Structural robustness:Artificialneuralnetworksinprinciplepossesstheprop-erty of structural robustness. Removing a neuron leads to a degradation

Y116134.indb 216 10/10/11 2:12 PM


inperformance,butdoesnotleadtototalfailureofthenetworks.Thisissometimesusedtoreducethesizeofnetworks.

Hierarchical structure:Artificialneuralnetworkspossessalayeredstructureofaninputlayer,avaryingnumberofhiddenlayers,andanoutputlayer.Thislayeredstructureprovedtobeusefulinawiderangeofapplications;however,itdoesnothaveaverystrongresemblancetobiologicalneuralnet-works.Recentdevelopmentsofneuralnetworkstrytorecreatethehumanbrain’sprocessingstepsinamuchmoreprecisewaythanearlierdesigns(Riesenhuber&Poggio,1999).

Size:Themaindifferencebetweenartificialnetworksandbiologicalnetworksissize.Thenumberofconnectionsinthehumanbrainisstillunmatchedbyartificialnetworksanditstillremainstobeseenhowsuchverylargearti-ficialnetworksbehave.Thesecondmajordifferenceistimeandinputusedforlearning.Biologicalnetworkshaveaverylongtime(decades)tolearnandreceivearichspectrumofinputandfeedback,whichisnotpresentinanyimplementationofartificialneuralnetworks.

Parallel computation:This is awell-known featureof artificial neural net-works.Althoughthecomputationswithineachlayercouldbedoneinpar-allel and the evaluation of a whole network would then take only a fewmassively parallel computational steps, almost all neural networks areimplementedonstandardcomputers,wheretheevaluationisdonesequen-tially.Thusthemathematicalmodelhastheabilityofparallelcomputation,buttheusualimplementationdoesnot.

10.2.4 Fuzzy systems

Anothermajortopicintheareaofsoftcomputingisfuzzysets.Thepurposeoffuzzysetsistocapturethevaguenessoflinguisticexpressionsandmakethemmathemati-callyaccessible.Thefollowingsectionsdescribeonlythebasicsoffuzzysets;furtherinformationcanbefoundforexample,inGebhart,Kruse,andKlawonn(1993).

TheinventionoffuzzysetsisattributedtoLotfiZadehandwaspublishedinafamouspaperon“FuzzySets”in1965.Theideaemergedfromsystemstheoryandwasfocusedoncapturingtheinherentimprecisionofbiologicalornaturalsystems.Thefundamentalideaisthatthemembershipofanelementinasetisgradualratherthanbinary(crisp).Thischangegaverisetoawholenewfieldofmathematics,creat-ingfuzzysettheory,fuzzynumbers,fuzzycontrol,andawiderangeofassociatedmathematicalconceptssuchastriangularnorms.

For a crisp set A a unique characteristic function μ:U→0,1 can be defined,whichdivides theuniverseof discourseU into a groupofmembers and a groupofnonmembers.

µA xx A

x A( ) =

∈∉

0

1

if

if

Y116134.indb 217 10/10/11 2:12 PM


Forafuzzysetthemembershipfunctionmapseachelementtoadegreeofmember-shipintheinterval[0,1],suchthatμA(x) ∈ [0,1].

This gradual transition enables mathematicians to adequately describe objectsthatweuseinourdailycommunication.Anillustrativeexampleisthatofa“pileofsand.”Startingfromalargepileweremovethesandgrainbygrain.Wefindthatthereisnosinglemomentintimewhenthepileofsandceasestobeapileofsandaccordingtothecommonlyacceptedmeaning.However,oncethereareonlyafewgrainsleft,itwillnotbeapileanylonger.Consequently,theobjectunderconsider-ationfulfillsthepropertyofbeingapileofsandtoacertaindegreedependingonhowcloselyitmatchesourexpectationsinthecurrentcontext.

TypicalmembershipfunctionsareshowninFigure 10.3.Intheorymembershipfunctionshavetofulfillonlyoneverybasicproperty:thattheymaptheobjectsintheuniverseofdiscoursetomembershipvaluesintherangeof0to1.Inpracticethereareafewfunctionsthatareusedinalmostallapplications,suchastriangular-shapedfunctions, trapezoid functions, andmonotone increasing functions. In somecasessmoothapproximationsofthesefunctionsareusedtoavoidtheproblemsassociatedwiththedifferentiabilityofthecorners.

10.2.5 Fuzzy sets and rules

Similartoconceptsofclassicalsettheoryonecandefineintersections,unions,andotheroperationsonfuzzysets.Giventwofuzzysetsrepresentedbytheirmember-ship function μA and μB, defined on the same universe of discourse,we may, forexample,definetheirintersectionas

µ µ µA B A B∩ = ⋅ .

TheproducthasafewpropertiesthatmakeitaplausibleoperationforcalculatingtheintersectionofAandB;forexample,itwillbenonzeroonlyifμAandμBarenonzero.However,thisisnottheonlypossibility.Onecouldalsodefine

µ µ µA B A B∩ = min( , ),

whichwouldleadtoaslightlydifferentresult,butstillpreservethemostrelevantproperties of an intersection. In fact there is awhole class of functions thatmayserveas intersections; thesearecalled t-norms(Klement,Mesiar,&Papp,2000).

x

Mem

bers

hip

Valu

e

x x

Gaussian Triangular Trapezoid

–10 –6 –2 0 2 6 10

1.5

1

0.5

0

–10 –6 –2 0 2 6 10 –10 –6 –2 0 2 6 10

Figure 10.3 Typicalmembershipfunctionsoffuzzysets.

Y116134.indb 218 10/10/11 2:12 PM


Similarly,wemayconsider thedifferentoptions for calculating theunionof twofuzzysetsAandB.Twosuchoptionswouldbe

µ µ µ µ µA B A B A B∪ = + − or µ µ µA B A B∪ = max( , ) .

Againthereisawholeclassofsuchfunctions,whicharecalledt-conorms.Itshouldbepointedoutthattherearesignificanttheoreticalchallengesthathave

tobeconsideredwhendealingwithintersectionsandunionsoffuzzysets.Fornon-fuzzysetsitisbasicknowledgethattheintersectionofasetAwithitselfisagainA.However,ifwechoosetheproductasourwayofcalculatingtheintersectionoftwofuzzysets,thenwefindthattheintersectionoffuzzysetAwithitselfdoesnotingeneralpreserveitsmembershipfunction,inasmuchas

µ µ µ µA A A A A∩ = ⋅ ≠ .

On theotherhand, ifmin(.,.) ischosenfor the intersection, thenwefind that thispropertyispreserved.

Thebigadvantageof fuzzymethods thatmakes themdifferent fromallothermathematicalconceptsisthepossibilitytocreaterulesinlinguisticform.Suchsetsofrulesarealsocalled“fuzzyinferencesystems.”Therulescanbeeasilyunder-stoodandinterpretedbyhumansandallowthediscoveryofaprioriunknownrela-tionships.Awiderangeofmethodsexisttoautomaticallyextractsuchfuzzyrulesfromexperimentaldatasets (Cox,2005). If, for example,onewould try tomodelrelationships between physical properties and human perception of surfaces, onemightendupwithrulessuchas

(R1)IFsurfaceisROUGHandcolorisREDthenPERCEPTIONisVERYWARM.

(R2)IFsurfaceisSMOOTHandcolorisDARKthenPERCEPTIONisCOLD.

Thecapitalizedwords(“ROUGH”,“DARK”,“COLD”,etc.)arefuzzysetsdefinedonappropriateuniversesofdiscourse.Theinputvariables“surface”and“color”arenonfuzzyquantitiesusuallyrepresentedbyrealnumbers.Theoutput“perception”isafuzzyset,whichforsomeapplicationshastobeconvertedbackintoarealnum-ber.Thisprocess iscalleddefuzzificationandmanydifferentmethodshavebeendevelopedforthispurpose.Wejustmentionthecenter-of-gravitydefuzzificationthatcalculatesthecenterofgravityoftheresultingfuzzymembershipfunction.

TheprocedureforevaluatingsucharulesystemisshowninFigure 10.4andisdoneasfollows.Firstweneedtodeterminethedegreetowhicheachruleisfulfilledbycalculatingthedegreeofmembershipforeachinputvariableintheirrespectivefuzzysets.Wemayfind,forexample,that

μROUGH (surface)=0.8 μSMOOTH (surface)=0.1

μRED (color)=0.6 μDARK (color)=0.5

Y116134.indb 219 10/10/11 2:12 PM


Ifwetakemin(.,.) asfuzzy“and”,thenwefindthatrule(R1)isfulfilledtoadegreeofmin(0.8,0.6)=0.6andrule(R2)isfulfilledtoadegreeofmin(0.5,0.1)=0.1.Theoutput fuzzyset (“VERYWARM”,“COLD”) is thenweightedaccording to thedegreetowhichtherespectiveruleisfulfilled.Thisweightingoperationisusuallydonebyapplyingat-norm,suchasmin(0.6, μVERY_WARM(perception))andmin(0.1, μCOLD(perception)).

Thesecondstepistoaggregateallrulestoobtainasinglefuzzyoutputmember-shipfunctionforPERCEPTION.Theaggregationoperationisusuallyat-conorm,suchasmax(.,.).Theresultingmembershipfunctionisthus

μPERCEPTION(perception)=max(min(0.6,μVERY_WARM(perception)),

min(0.1,μCOLD(perception)))

Occasionally,thereistheneedtoconverttheoutputintoanonfuzzy(crisp)realnum-ber,forexample,ifsuchrulesystemsareusedintechnicalfeedbackcontrolloops.Hereweapplythecenter-of-gravitydefuzzification,sothatthefinalcrispoutputofthefuzzyrulesystemisobtainedby

µ

µ

( )

(

perception perception perception

perc

⋅ ⋅∫ d

eeption perception) ⋅∫ d.

Rough

Smooth

&

Surface Color

Min (.,.)&R1

R2

Dark

Red Very Warm

Cold

Perceived warmth

min (.,.)

max (.,.)Perceivedwarmth

Center of gravity

Perceived warmth

Perception

Figure 10.4 Fuzzy inference with two rules, min/max aggregation and center-of-gravitydefuzzification.EvaluationoftherulesR1andR2forasurfacethatisquiteroughandhasasomewhatdarkredcolor.RuleR1appliestoadegreeof0.6andruleR2appliestoadegreeof0.1.Bycombiningtheresultsofbothrules(bottom,right)weendupawithafuzzymem-bershipfunctionthatdescribestheperceptionofwarmthofthesurface.Thisfuzzymember-ship function may be defuzzified using the center-of-gravity method. See text for furtherexplanation.

Y116134.indb 220 10/10/11 2:12 PM


10.2.6 Fuzziness versus probability

Itisimportanttodistinguishbetweenprobabilityandfuzziness.Bothtrytocaptureadifferenttypeofuncertainty.Probability(atleasttheinterpretationasrelativefre-quenceofoccurrence)appliestotheconceptoflargenumbersofobjectsthatdoordonotpossessacertainproperty.Theuncertaintyliesinthefactthatwedonotknowwhichobjectisactuallychosen.Onceanobjectisselectedfromthislargeset,itcanbedeterminedwithabsolutecertaintywhethertheobjectfulfillstheproperty.Thechanceofchoosinganobjectwiththedesiredpropertyisexpressedbytheprobabil-itydistribution.Fuzzinessdoesnotneedalargenumberofobjects;itcanbeappliedtoasingleobject.However,giventhissingleobject,itisnotfullyclearwhethertheobjectfulfilsthepropertyunderconsideration.Thisfactisexpressedbythevalueofthemembershipfunctionforthatobject.

ThisdifferenceiseasilydemonstratedbytheexampleshowninFigure 10.5.Theobjectshownintheimageissomewhatclosetoasquare.However,wewouldneversaythat“Thisisprobablyasquare.”Forthisparticularobjectitcanbeeasilydeter-mined,that—accordingtothemathematicaldefinition—itisnotasquare.Thereisnoprobabilityinvolved.

Ontheotherhand,asidefromtheexactmathematicaldefinition,ifweadoptahumanlikeinterpretationof“square,”wewouldcallthisa“fuzzysquare.”Tosomedegreeitfulfilsthemostimportantproperties.Ithasfourcorners,althoughnotwithanangleofexactly90°,andedgesofroughlythesamelength.Onecould,forexam-ple,saythatthisobjectfulfilsthepropertiesofa“square”tothedegreeof0.75.

This raises the question of how the value of 0.75 is actually determined. Themainstreaminterpretationoffuzzymembershipfunctionisthatofastatisticalquan-tity.If,forexample,onewantstodeterminethefuzzymembershipfunctionoftheproperty“tall”(ifappliedtomen),onecouldconsidervaluesintherangeof1.6mto2.3m.Anexperimentalprocedurecouldobtaintheopinionsofasufficientlylarge

Figure 10.5 Afuzzysquare.

Y116134.indb 221 10/10/11 2:12 PM


numberofpeopleaboutthemeaningof“tall”inthiscontext.Mostpeoplewillagreethat1.6misnot“tall,”andtherewillalsobeanagreementthat“2.3m”isindeed“tall.”Betweenthesetwoextremestherewillbeatransitionfrom“nottall”to“tall.”Amembershipvalue0.3forthesizeof1.8mcanthusbeinterpretedinawaythat30%oftheparticipantsintheexperimentwouldcallamanof1.8m“tall.”

Clearly,suchexperimentalproceduresarenotoftenused;however,inmanytech-nicalapplicationsitissurprisinglyeasytofindagoodsetofmembershipfunctions.Quiteoftenthecontextandtechnicallimitationsprovideimportantcluesastohowamembershipfunctionshouldbechosen.

10.3 Recent models of object recognition in the visual cortex

Inthissectionwetakeabigstepforward,byalmost40yearsofresearchinmath-ematicalmodelingofthehumanbrainandartificialneuralnetworks.Theoriginalconceptofmodelingtheneuronasanonlinearfunctionofalinearcombinationofinputshasbeensubstantiallyextended.Muchmoreisnowunderstoodaboutthepro-cessingstepsthatgooninlayersofneuronsandthisisparticularlytrueforthelayersthatsetupthehumanvisualcortex.Thispartofthebrainhasreceivedagreatdealofattentioninthepastsothatquitedetailedandsurprisinglyprecisemodelsofthehumanvisualsystemhavenowbeendeveloped.Theoriginalideaofhavingalayeredstructureofneurons,however,isstillpresentintoday’smodels.Inthefollowingwefocusonartificialneuralnetworksthatmodelthetaskofobjectrecognitionandgiveanoverviewofrecentresultscoveringthestepsfromtheperceivedimagethroughvariousbrainareastofinalrecognition(Riesenhuber&Poggio,1999).Wefirststartwithadescriptionof theneurophysiological resultsand thengiveamathematicalrepresentationofthewholemodel.

10.3.1 The visual cortex

Processingof the retinal image isdonealong theventralvisualpathway,startingwiththeprimaryvisualcortex(V1),followedbyvisualareasV2andV4andlead-ingontotheinferotemporalcortex.Theinferotemporalcortexissupposedtoplayanimportantroleinobjectrecognitionandprovidesinputtotheprefrontalcortexthatlinksperceptionandmemory.ThewholeprocessisshowninFigure 10.6anddescribedinthefollowingparagraphs.

Thefirstprocessingstepintheprimaryvisualcortex(V1)isdonebysimplecells(S1)thatareknowntobeorientation-sensitive.Theyrespondtoedgesofhighcon-trastintheimagethatarerunninginacertaindirection.Theactivationsofsimplecellsthatrespondtothesamedirectionareaggregatedbythecomplexcells(C1)overacertainsmallarea.Thisaggregationoverasmallarealeadstoaslightinvariancetoposition.Inthenextstep,theaggregationisdoneovercellswithsimilarposition,butdifferentorientation,whichleadstoso-called“compositefeature”cells(S2).Thefollowinglayer(C2)againcombinestheactivationofS2cellswithsimilarfeatures,butslightlydifferentposition.Theassumptionisthatthereisanalternatingsequence

Y116134.indb 222 10/10/11 2:12 PM


ofsimpleandcomplexcells,whosefinaloutputissensitivetorotationindepth,tochanges in illumination, and to anydeformationof theobject, but is invariant totranslation and scale. The resulting units are thus tuned to a specific view of anobject,whichprovidestheselectivityrequiredforobjectrecognition.

In thesecondpartof themodel invariancewith respect to rotation indepth isobtainedbycombiningseveraldifferentview-tunedunits.Theprocessing is sup-posedtotakeplaceintheanteriorinferotemporalcortex.Atthisstagetheprocessingstepsbecomemorecomplicated,withfeedbackloopsinvolvedandtask-dependentunitsthatperformvisualtaskssuchasidentificationorclassification.Thesehigher-level functions take place in the inferotemporal cortex and the prefrontal cortex.Learningalsotakesplaceintheseareas.

10.3.2 Computational model of the visual cortex

Thefollowingdescriptionofacomputationalmodeloftheobjectrecognitioncapa-bilities of the visual cortex is taken from recent work by Serre, Wolf, Bileschi,Riesenhuber,andPoggio(2007).Wedonot reproduce themodel in thefull levelof detail, but focus on the single neural mechanisms and their computationalrepresentation.

Task-related units

Object-tuned cells

View-tuned cells

Complexcomposite cells C2

Compositefeature cells S2

Complex cells C1

Simple cells S1

O1

V1

O2 O3 On

V2 V3 Vn

Figure 10.6 Thestandardmodelofthehumanvisualcortex.

Y116134.indb 223 10/10/11 2:12 PM


ThefunctionoftheS1simplecellsisusuallymodeledbyGaborfiltersthatprovetobeverygoodatreproducingthesecellsonarangeofexperimentaldata.TheGaborfilter depends on scale (frequency) and orientation. Usually a whole set of Gaborfilters isconsideredwithdifferentscalesandorientations,where64filterswith16differentscalesand4differentorientations(0°,45°,90°,135°)arewidelyused.Eachof these64 images that result from thefilteringprocess hashighvalueson thoseareasthatmatchtothedirectionandscaleoftherespectiveGaborfilters.DependingonthescaleoftheGaborfiltertheimageslookblurred,withlargestscalescausingthestrongestblurringeffect.AnotherpropertyofGaborfiltersisthattheyalsoactasedge-detectors,causinglowvaluesinhomogeneousareasandhighervaluesonedgesorregularpatternsthatmatchtheirfrequency.

Thefirstlayerofcomplexcells(C1)performsanaggregationofthesimplecellsandintroducesaslightinvariancetopositionandscale.C1complexcellscoverabouttwicethesizeofareceptivefieldofsimplecells.Itthusmakessensetocombinethe16scalesto8scalebands,eachofwhichcontainstwoadjacentscales.Theaggre-gationisdonebycalculatingthemaximumoverallactivationsof theS1cells, inthescalesbelongingtothescaleband.Thesizeoftheneighborhoodforthemaxi-mumaggregationisdeterminedbytheaverageofthescalesinthescaleband.Thisaggregationoveraneighborhoodisnotdoneatallpossiblelocations,butthesingleneighborhoodsonlyoverlapby roughlyhalf their size, such thata staggeredgridisobtained.Thisaggregationisdoneindependentlyforallorientations.Theresultis32images(8scalebandstimes4orientations)withsubstantialblur.Themaximumaggregationoveraneighborhoodandthereductionto8scalebandsintroduceacer-tainlevelofinvariancetotranslationandscale.

The followingS2 layerperforms an aggregationover all orientations and alsocontainsthefirsttrainablecomponentsintheformofprototypeimagepatches.TheS2cellsarerepresentedbyGaussianradialbasisfunctions,similartotheoneshowninFigure 10.1.Theactivationwillbelargestiftheimagepatchmatchestheproto-type patch. The prototypes are determined during the learning process. Such S2mapsarecalculatedforallofthe8scalebandsandforeachprototype.Dependingonthecomplexityofthetaskthismaybeseveralthousandmaps.

TheC2units introduceafinalshiftandscale invariancebycalculating theglobal maximum over all scales and positions and independently for all pro-totypes. The resulting maxima for all the prototypes are collected in a singlevector thatconstitutes theoutputofourmodel.Thisvector is in turn input toastandardclassifier,suchasa linearclassifier,asupportvectormachine,orasimilarclassifier,thatfinallyperformstherecognitionbyassigningthevectortoaclassofobjects.

The learning process corresponds to the selection of the prototypes from thetrainingimages.Inthismodeltheselectionisdonebyasamplingprocess,selectingpatchesatrandompositionsandrandomlychosenfromasetoffourdifferentsizes.

Thismodelwasappliedtoasetofimagedatabasesofvaryingcomplexity,con-tainingseveralthousandimages.Themainqualitymeasureistherecognitionrate,thatis,theprobabilitywithwhichthemodelcorrectlyrecognizestheobjects.The

Y116134.indb 224 10/10/11 2:12 PM


benchmarks for this assessment were standard approaches for object recognition(SIFTfeatures).Therecognitionrates[%]aregiveninTable 10.1.

Clearly,forsometypesofobjects thismodelshowsasubstantial improvementoverexistingmethods.Weconsiderthisasoneofthefirstpowerfuldemonstrationsofhowbiologicalmodelsmayachievesignificantprogressinareasthatarecurrentlydominatedbyengineeredmethods.

References

Cox, E. (2005). Fuzzy modeling and genetic algorithms for data mining and exploration,Amsterdam:Elsevier.

Gebhart,J.,Kruse,R.,&Klawonn,F.(1993).Fuzzy-systeme.Stuttgart:B.G.Teubner.Gupta,M.M.,&Knopf,G.K.,Eds.(1993).Neuro-vision systems—Principles and applica-

tions.Washington,DC:IEEEPress.Klement,E.P.,Mesiar,R.,&Papp,E.(2000).Triangular norms.Dordrecht:KluwerAcademic.Lin,C.T.,&Lee,C.S.G.(1995).Neural fuzzy systems: A neuro-fuzzy synergism to intelligent

systems,EnglewoodCliffs,NJ:PrenticeHall.Riesenhuber,M.,&Poggio,T.(1999).Hierarchicalmodelsofobjectrecognitionincortex.

Nature Neuroscience, 2,1019–1025.Serre,T.,Wolf,L.,Bileschi,S.,Riesenhuber,M.,&Poggio,T.(2007).Robustobjectrecogni-

tionwithcortex-likemechanisms.IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3),411–426.

Zadeh,L.A.(1965).Fuzzysets.Information and Control,8,338–353.

Table 10.1 Thehumancortex–basedmodelachievesasignificantimprovementoverbenchmarkmethodsforobjectrecognition(Serreetal.,2007)

Object Benchmark New model

Leaves 84.0 97.0Cars 84.8 99.8Faces 96.4 98.2Airplanes 94.0 96.7Motorcycles 95.0 98.0Faces 90.4 95.9Cars 75.4 95.1

Y116134.indb 225 10/10/11 2:12 PM

Y116134.indb 226 10/10/11 2:12 PM

227

Psychological measurement for sound description and evaluation

Patrick Susini,1 Guillaume Lemaitre,1 and Stephen McAdams2

1InstitutdeRechercheetdeCoordinationAcoustique/MusiqueParis,France2CIRMMT,SchulichSchoolofMusic,McGillUniversityMontréal,Québec,Canada

11.1 Introduction

Severaldomainsofapplicationrequireonetomeasurequantitiesthatarerepresenta-tiveofwhatahumanlistenerperceives.Sound quality evaluation,forinstance,stud-ieshowusersperceivethequalityofthesoundsofindustrialobjects(cars,electricalappliances,electronicdevices,etc.),andestablishesspecificationsforthedesignofthesesounds.Itreferstothefactthatthesoundsproducedbyanobjectorproductarenotonlyevaluatedintermsofannoyanceorpleasantness,butarealsoimportantin people’s interactions with the object. Practitioners of sound quality evaluationthereforeneedmethodstoassessexperimentally,orautomatictoolstopredict,whatusersperceiveandhowtheyevaluatethesounds.Thereareotherapplicationsrequir-ingsuchmeasurement:evaluationofthequalityofaudioalgorithms,management(organization,retrieval)ofsounddatabases,andsoon.Forexample,sound-databaseretrieval systemsoften requiremeasurementsof relevantperceptualqualities; thesearchingprocessisperformedautomaticallyusingsimilaritymetricsbasedonrel-evantdescriptorsstoredasmetadatawiththesoundsinthedatabase.

The“perceptual”qualitiesofthesoundsarecalledtheauditory attributes,whichare defined as percepts that can be ordered on a magnitude scale. Historically,thenotionofauditoryattribute isgrounded in the frameworkofpsychoacoustics.Psychoacoustical researchaims toestablishquantitativerelationshipsbetween thephysical properties of a sound (i.e., the properties measured by the methods andinstrumentsofthenaturalsciences)andtheperceivedpropertiesofthesounds,theauditoryattributes.Thephysicalpropertiesofasoundthatarerelatedtotheauditoryattributescanbecomputed from thesoundsignal.Thesevalues thereforepredicttheauditoryattributesfromthesoundsignalaloneandoncewellunderstoodcanbe substituted for experimental measurements. They are called psychoacoustical

11

Y116134.indb 227 10/10/11 2:12 PM


descriptors.Psychoacousticalresearchhasisolatedseveralauditoryattributes:loud-ness,pitch,duration,andsharpness,amongothers.Methodshavebeendevelopedtomeasuretheseattributesexperimentally,andalgorithmshavebeendevisedtocom-putecorrespondingpsychoacousticaldescriptors.

Hereweuse the term“auditory attribute” in a slightlybroader sense than thepsychoacousticaldefinition. Indeed, listenerscan recovermanykindsof informa-tionfromasound.Notonlydotheyperceiveperceptsthatcanbedirectlymappedtothephysicalpropertiesofthesound,butmostofthetimetheyalsorecognizethesourcethatcausedthesoundandidentifyitsproperties.Gaver(1993a,1993b)ini-tiallyformalizedthisideabyintroducingtheconceptsofmusical listening(focusonthesounditself)andeveryday listening(focusonthepropertiesofthesource).Bymeasuring auditory attributes,wethereforemeanhere“providingquantitiesrepre-sentativeofwhatauserperceives.”

Thepurposeofthischapteristopresentthemeasurementoftheseauditoryattri-butes fromanappliedperspective.Someof theseattributesareeasilyunderstood(andhaveaname)andhavebeenstudiedindepth.Forinstance,loudness,pitch,andduration are auditory attributes forwhich experimentalmethods, and evenmath-ematicalpredictivemodels,areeasilyaccessible.Section11.1brieflysummarizessomeoftheresultsandmethodsassociatedwiththeseattributes.Otherattributesareless easily specified and often require metaphors from other sensory modalitiestobedescribed:brightness (or sharpness), roughness,fluctuationstrength,andsoon.InSection11.2,wepresentmorespecificallythemethodsusedtoexploretheseattributes.Becausetheycannotbeeasilyandunequivocallyspecifiedtoalistener,theseattributesrequireindirectandmultidimensionalmethodsthatallowexplora-tionofsoundperception.Section11.2presentsseveralfamiliesofmethods:semanticscales,similarityjudgmentsandmultidimensionalscaling,sortingtasks,andclusteranalyses.Section11.3presentsexamplesofapplications insoundquality.Finally,perspectivesintherealmofsonicinteractiondesignarebrieflyintroduced.

11.1 Basic knowledge and methods

11.1.1 Peripheral auditory system

Weprovidehereabroadoverviewof theperipheralauditorysystem.*Foramorecompletedescription,interestedreadersshouldrefertoMoore(2003).

11.1.1.1 Description

Thehumanperipheralauditorysystemiscomposedofthreeparts:theouterear,themiddleear,andtheinnerear.Theouter earismainlycomposedofthepinnaandtheauditorycanalbetweenthepinnaandtheeardrum.Theouterearamplifiesthesoundlevelattheeardrumforfrequenciesaround3kHz.Themiddle ear,composedofthree

* AnimationsbyProf.HerbertHudde fromBochumUniversitycanbe foundat the followingURL:http://www.ruhr-unibochum.de/ika/ika/forschung/gruppe_hudde/bohear_en.htm

Y116134.indb 228 10/10/11 2:12 PM

Psychological measurement for sound description and evaluation 229

verysmallossicles,matchesimpedancebetweentheairintheauditorycanal(outerear)andthefluidsinthecochlea(innerear).Italsoimprovessoundtransmissionforfrequenciesintherangeof0.5–4kHz.Fromapsychoacousticalpointofview,themostimportantpartoftheinner earisthebasilarmembrane(BM)thatcanbeconsideredasa“frequencyanalyzer.”AnincomingsoundsetsinmotiontheBMwithamaxi-mumdisplacementatacertainpositionthatdiffersaccordingtothefrequencyofthesound;thepositionofthemaximumvariesfromthebeginning(base)oftheBM(ovalwindow)forhighfrequenciestotheend(apex)oftheBMforlowfrequencies.ThefrequencyproducingamaximumofdisplacementontheBMisthecenterfrequencyofabandpassfilterforthatposition.Becausedifferentfibersoftheauditorynerveareconnectedtodifferentpositionsalongthebasilarmembrane,thefrequencyselectiv-ityofthebasilarmembraneresultsinafrequencydecompositionofthesoundsintheauditorynerve.Thefrequencyselectivityoftheauditorysystemhasveryimportantconsequencesforaudition.Particularly,the“masking”phenomenonhasintroducedtheconceptsofcriticalbands(CB)andauditoryfiltersandhasresultedinamodelthatisthebasisforthecomputationofpsychoacousticaldescriptors.

11.1.1.2 Masking, critical bands, and models

Fletcher(1940)introducedtheconceptofcriticalbandstoaccountformaskingphe-nomena.Forverynarrowbands,heshowedthatthethresholdofdetectionforapuretoneincreasesasthenoisebandwidthincreases.Afteracertainbandwidth,increas-ingthenoisebandwidthnolongerchangesthetonethreshold.Fletcherassumedthatonlyaneffectivepartofthenoisemasker,closetothefrequencyofthetone,hasthepowertomaskthetone.Thecorrespondingfrequencyregionisthecritical band.Furtherinvestigationsshowedthatamodelconsistingofabankofbandpassfilters,thebandwidthofwhichincreaseswiththecenterfrequency,couldaccountformask-ing (Zwicker, 1961;Zwicker&Fastl, 1972;Moore&Glasberg, 1983,1990).Theshapeofeachfilterisasymmetric:roll-offissharpforfrequenciesbelowthecenterfrequency(100dB/octave)andsmoothforfrequenciesabovethecenterfrequencies.Thesteepnessoftheroll-offdecreasesasthelevelofthestimulusincreases.

Thereareseveralmodelsofthesefilters.Third-octavebandpassfilterscanroughlymodeltheauditoryfilters.Fourth-octavebandpassfiltershavealsobeenproposedandshowntoapproximatefairlywelltheauditoryfiltersexceptforlowfrequencies(Hartmann,1997).Amorecomplexmodeluses theGammatonefilters(Patterson&Holdsworth,1991).Finally,basedonthisconceptofcriticalbands,severalscaleshavebeenproposed:theBark scale(Zwicker&Terhardt,1980)andtheEquivalent Rectangular Bandwidth (ERB) scale(Moore&Glasberg,1983).

11.1.1.3 Psychoacoustical descriptors

Modelsoftheauditorysystembasedoncriticalbandsareusedtocomputepsycho-acousticaldescriptors.TheclassicalpsychoacousticaldescriptorsaresummarizedinZwickerandFastl(1999)andMoore(2003).

Y116134.indb 229 10/10/11 2:12 PM


Thedescriptorofloudnessiswidespread.Modelshavebeenstandardized:ISO532-A (Stevens’ model); ISO 532-B for (Zwicker’s model). A BASIC program isalsoavailableinZwicker(1984).ANSIS3.4-2005isarevisionproposedbyMooreand Glasberg (1996) and Moore, Glasberg, and Baer (1997). Corrections of thismodelhavealsobeenproposedallowingabetteraccountof impulsivesounds inabackgroundmaskingnoise(Vos,1998)andoftime-varyingsounds(Glasberg&Moore,2002).Anotherdescriptorofloudness(Meunier,Boulet,&Rabau,2001)hasbeenproposedforenvironmentalandsynthesizedimpulsivesounds.Theloudnessiswellexplainedbyacombinationbetweenthelogarithmofthereleasetimeandtheenergy.

Psychoacousticaldescriptorscorrespondingtootherauditoryattributesarealsocommonly used: spectral centroid and sharpness, roughness (Daniel and Weber,1997),andsoon(seeZwicker&Fastl,1999,andFastl,1997,forsummaries).Theyhave also been implemented in several commercial software packages: BAS andArtemiSbyHeadAcoustics,dBSonicby01dB-Metravib,PULSEbyBrüel&Kjaer,and LEA by Genesis. The available descriptors that have been implemented arebased on experimental results using abstract sounds, thus these psychoacousticaldescriptorssometimesneedtobeadaptedforrealsounds(seetheworkbyMisdariiset al., 2010, on thisquestion).Only the loudnessdescriptorshavebeen standard-ized.Theyprovidereliableresultsforstationarysounds,butfurtherdevelopmentisneededfornonstationarysounds.

11.1.2 Classical psychoacoustical methods

Thetraditionalpsychoacousticalapproachisunidimensional:itaimstoestablishaquantitativerelationshipbetweenasingleauditoryattributeandaphysicalpropertyofthesound.

11.1.2.1 Indirect methods

11.1.2.1.1 Thresholds. The indirect method is based on the measurement ofthresholds.Theabsolute thresholdistheminimumdetectablelevelofasound.Forinstance, for a pure tone it depends on the frequency of the tone. Under normalconditions,ayounglistenercanhearfrequenciesbetween20Hzand20kHz.Formostadults, the threshold increases rapidlyaboveabout15kHz.Thedifferential thresholdordifference limen(DL)isthesmallestchangeinasoundtoproduceajust-noticeable difference(jnd)intherelatedauditoryattribute.

11.1.2.1.2 Confusion scaling. ByvaryingaphysicalparameterandmeasuringtheDL for a given auditory attribute, a confusion scale for this attribute can be set.AssumingthatallDLscorrespondtoequalchangesoftheauditoryattribute(jnd),Fechner’slaw(1860,publishedinEnglishin1966)canbedetermined:

ψ=klog(ϕ)

Y116134.indb 230 10/10/11 2:12 PM


whereψisthemagnitudeoftheauditoryattribute,ϕisthephysicalparameter,andkisaconstantspecifictoeachauditoryattribute.

11.1.2.2 Direct methods

Ratio scalingisadirectmethodrelyingontheabilityofparticipantstomakenumer-ical judgmentsof theratiobetweenthemagnitudesof theirsensations.Theusualmethodsaremagnitude estimation andproduction.Formagnitudeestimation,theparticipants are required to assign a number proportional to their sensation (e.g.,loudness)oftheintensityforsoundspresentedatdifferentlevels.Forthemagnitudeproductionmethod, theparticipant is required in thiscase toadjust the levelofatestsoundtoaspecifiednumberproportionaltoitsloudness.Therelationbetweentheexpressedsensation(e.g.,loudness)usingsuchmethodsandthecorrespondingacousticalvalues(e.g.,soundpressurelevel)leadstothewell-knownpsychophysicallaw,Steven’slaw:

ψ=kϕα

whereψisthemagnitudeoftheauditoryattribute,ϕisthephysicalparameter,andkandαareconstantsspecifictoeachauditoryattribute. Forinstance,fortheloudnessofa1-kHztone,theexponentis0.6:a10-dBincreaseleadstoa2-soneincrease.Fora3-kHztone,theexponentis0.67.Steven’slawforloudnesshasledtothederivationofthesonescale.

The cross-modal matching methodwasproposedbyS.S.Stevens(1959).Thetaskconsists inmatching twosensations (e.g., loudnessandmuscular forcesensation),oneofwhichhasbeencalibratedbeforehandbyadirectestimationmethod(Stevens,1959).Thematchingfunctionbetween thesensations isknownorexperimentallyobtained.Thenratingsrelated to theothersensationaredirectlydeducedbywayofthematchingfunction.Thismethodcanbeusedtoscaletheloudnessoftime-varyingsounds(seethenextsection).

11.1.3 Perspectives: Loudness of time-varying sounds

Theclassicalpsychoacousticalmethodshavebeenbroadlyused to study theper-ceptionofshortandstationarysounds.Everydaysoundeventsandmusicalpieces,however,areusuallynonstationary.Thetemporalfluctuationsanddurations(upto20minutes)of suchnonstationarysoundsdonotallow theuseofclassicalmeth-ods,butrequirecontinuousratingsofthesounds.Theparticipantmustinthiscaserespondinstantaneouslytoanyvariationofthesound.Themethodsandthedevicesusuallyproposedcanbesortedintofivecategories.

1.The method of continuous judgment using categories was proposed byKuwanoandNamba(1978,1985)withtheaimofstudyingtemporalfluc-tuationsofthelevelofurbansounds.Inthisprocedure,participantsjudge

Y116134.indb 231 10/10/11 2:12 PM


theloudnessateachinstantusingaresponseboxwithsevenbuttonscor-respondingtosevencategories:very,veryloud–veryloud–loud–medium–soft–verysoft–very,verysoft.Thisprocessisapplicabletolong-durationstimuli,becausethetaskisnotdifficultandparticipantsexperiencelittlefatigue.Theparticipantsmodifytheirjudgmentassoonastheyperceiveachangeequivalenttothedistancebetweentwocategories.Themaindisad-vantageofthecontinuouscategoryjudgmentmethodisthatitdoesnotallowonetoobtainanalogicalresponsesasafunctionofthesignalcontour.

2.Theaudiovisual adjustment methodwasdevelopedbyKuwanoandNamba(1990).Inthismethod,participantsexpresstheirjudgmentbycontinuouslyadjusting the length of a line with a cursor so that the length is propor-tionaltotheauditorysensation.Themainproblemwiththismethodcomesfrom the clipping or ceiling effect at the top end of the judgment scale,becausethelengthofthelineislimited(computerscreen,sheetofpaper,etc.).Togetaroundthislimitation,KuwanoandNamba(1990)elaboratedadevicewithwhichthelinepresentedontheterminalscreenisprojectedon a large screenwith anoverheadprojector. In a similarmanner,Fastl(1989,1991)performedanexperimentinwhichtheparticipantjudgedtheinstantaneous loudnessbyassociating in real time thedisplacementof apotentiometeronamixingtable.However,thisdeviceprovideslittlefeed-back(asidefromhand/armposition)totheuser.

3.The continuous cross-modal matching method proposed by Susini,McAdams,andSmith(2002)isbasedonthecross-modalmatchingmethodwithaforce-feedbackdevice.Theparticipanthastoadjustamuscularforcesensationtotheperceivedloudness.Thisdevicewasusedtoassess1-kHzpure tones (Susini, McAdams, & Smith, 2002), urban sound sequences(Susini & Maffiolo, 1999), and sounds of accelerating cars (Susini &McAdams, 2008). The method has proved to be a flexible experimentalprocedureallowinganindividualcalibrationofthedeviceasafunctionofeachparticipant’sperceptualscale,withtheaimofavoidingcompressionorsaturationeffectsintheresponses.

4.Theanalog categorical scalingproposedbyWeber(1991)combinesthecategoricalandanalogicalmethods.Participantscanslideacursorcon-tinuouslyalongfivediscretecategorieslabeled(forexample):veryloud–loud–medium–soft–very soft. The distance between each category isconsideredasequivalent.Thismethodhasbeenwidelyused:forloudnessevaluationofvariable-amplitudesinusoidalsounds(Susini,McAdams,&Smith,2002,2007),forassessingspeechquality(Hansen&Kollmeier,1999; Gros & Chateau, 2001), for assessing the comfort of an urbansequenceofarunningbus(Parizet,Hamzaoui,Segaud,&Koch,2003),andforbrightnessratingsofvarioussounds(Hedberg&Jansson,1998).

5.The semantic scale used in real-time was introduced by several authorsto study more complex auditory attributes than loudness, and more spe-cifically to study real-timeemotional response tomusic.Thecontinuousresponse digital interface (CRDI) developed by Madsen (1996) allows a

Y116134.indb 232 10/10/11 2:12 PM


continuoustrackingoftemporalvariationsofmusicalworks,asdoesthetwo-dimensional emotional space (2DES) proposed by Schubert (1996)with which musically evoked emotions are evaluated in real time in atwo-dimensional semantic space. Several authors used continuous ratingtomeasureemotional force inmusicpieces (Sloboda&Lehmann,2001;McAdams,Vines,Vieillard,Smith,&Reynolds,2004).

11.2 Multidimensional and exploratory methods

Itisnotalwayseasytospecifyanauditoryattributeapriori.Apartfrompitchandloudness,veryfewwordsarespecifictosoundoreasilyunderstoodbynonspecial-ists.Therefore,unidimensionaltechniquessuchasdescribedabovecannotbeusedtomeasureauditoryattributesnoteasilycommunicatedtoparticipants,orthosethataresimplyunknowntotheexperimenter.Thissectionreportsmethodstoexploreormeasureunspecifiedauditoryattributesandmoregenerallytodeterminethepsycho-logicalaspectsofsoundperceivedbylisteners.

11.2.1 Judgments on multiple semantic scales

Theuseofmultiplesemanticscalesisafruitfultechniquetoassessdifferentpsycho-logicalaspectsofsounds:auditoryattributes (e.g., loudness, roughness),appraisal(e.g.,preference),emotionalresponse(e.g.,beauty,arousal),andconnotativedimen-sionsofthesoundsource(e.g.,thepowerofasportscar).

Semanticscalesarecategoryscalesdefinedeitherbyasinglesemanticdescrip-tor(unipolarscale)orbyapairofantonymicdescriptors(bipolarscale).Thescalesusuallyhavebetweenthreeandsevencategories.Itisusuallypreferredtouseanoddnumberofintervalstoincludethemiddlepointofthescale.

11.2.1.1 Method and analysis

Themostusedtechniqueisthesemanticdifferential(SD).Participantsareaskedtojudgeeachstimulusdirectlyalongasetofscaleslabeledwithtwoopposedsemanticdescriptors.Usuallytrueantonymlabelsareused(e.g.,good–bad,pure–rich,etc.),butalternativeshavebeenproposed(e.g.,good–notgood).

Thelabelsofthescalesarecalledsemantic descriptors.Theratingsofastimulusonthedifferentsemanticscalesyieldamultidimensionalrepresentationcalledthesemantic profile;anexampleispresentedinFigure 11.1.Afactoranalysiscancom-binesemanticscalesintomainfactors.Amultipleregressionanalysiscanhighlightrelationshipsbetween factors corresponding tocognitiveaspects (e.g.,preference)andfactorscorrespondingtoauditoryattributes(e.g.,loudness,roughness).Thelat-terfactorsareinterpretedbylookingforacousticalorpsychoacousticaldescriptorsthatarecorrelatedwiththem.Eachsemanticdescriptorishypothesizedtobepsy-chologically relevant to thewholesetof stimuliunderexamination.On theotherhand,ithastobeunderstoodbytheparticipantsoftheexperiment.

Y116134.indb 233 10/10/11 2:12 PM


11.2.1.2 Examples of semantic scales used to describe sounds

SinceSolomon(1958)andvonBismarck(1974),thesemanticdifferentialtechniqueproposed by Osgood (1952) has been widely used in the realm of sound percep-tiontodescribethemultidimensionalcharacterofthetimbreofmusicalinstruments(Wedin&Goude,1972;Pratt&Doak,1976;Kendall&Carterette,1992;Stepánek,2006),environmentalsounds(Björk,1985;Zeitler&Hellbrück,2001),andsoundproducts, such as cars, vacuum cleaners, air conditioning noises, or refrigerators(Chouard&Hempel;1999;Kyncl&Jiricek,2001;Siekiersky,Derquenne,&Martin,2001;Jeon,2006).

Typically,resultsfromthedifferentstudieshaveshownthatthesetofsemanticdifferentialscanbecombinedintothreeorfourmainfactorsthataccountforagreatdealofthevarianceinthejudgments.Forinstance,invonBismarck’sstudyonthetimbreofsyntheticsounds, results revealedfour independentscales:“dull–sharp”(44% of the variance explained), “compact–scattered” (26%), “full–empty” (9%),and“colorful–colorless”(2%).Onlythescalesreferringtosharpnesswereconsid-eredascandidates foragenerallyusablescale for themeasurementof timbre. InPrattandDoak(1976),threescales(“dull–bright”,“pure–rich”,“cold–warm”)wereselectedtobethemoresignificantdescriptorsforinstrumentaltimbres.Insummary,resultsfromdifferentstudiesrevealedthatdescriptorsrelatedtosharpness(“sharp,”

Intrusive

Dully

Bright

Speed

Rough

Hard

Vol

Fluct

Hum_L

Hum_P

Whisp_L

Whisp_P

Figure 11.1 (See color insert.) Sensory profiles obtained for three air-conditioningnoisesfromSiekierski,Derquenne,andMartin(2001).Labelsof thesemantic descriptorsare Intrusive, Dully, Brightness, Speed, Roughness, Hardness, Voluminous, Fluctuation,Humming,Whispering.ThelettersLandPcorrespondtotheLevelandPitch,respectively,ofthewhispering(noise)partandthehumming(motor)part.

Y116134.indb 234 10/10/11 2:12 PM


“bright,”“metallic,”orattheopposite,“dull,”“muffled,”“round”)areappropriatetodescribethemostsalientaspectoftimbre.

Kuwano and Namba (2001) report three main factors (“powerful,” “metallic,”and“pleasant”) thathadconsistentlybeenextracted inmostof their formerstud-iesofsoundquality.Furthermore,thesemanticdescriptor“powerful”wasusuallywell correlated with computed loudness (Zwicker’s loudness level based on ISO532B),and thesemanticdescriptor“metallic”waswellcorrelatedwithcomputedsharpness. The “pleasant” factor was related to cognitive and cultural factors aswellastophysicalpropertiesofsounds.InZeitlerandHellbrück’sstudy(2001)onenvironmental sounds, four factorswere linked, respectively, to a hedonic aspect(“ugly”–“beautiful”),timbre(“dark–light”),power(“weak–strong”),andrapidtem-poral variations (“unstable–stable”). The three latter factors were well correlatedwiththreecalculateddescriptors,sharpness,loudness,androughness,respectively.Resultsfromotherstudiesonsounds(speechorsonarsounds)arequitesimilar:themostimportantfactorsweregenerallyinterpretedasrepresentingloudness,timbre(sharpness)orpitch,andanoverallsubjectiveimpression.

11.2.1.3 Prerequisites to use semantic scales

11.2.1.3.1 Controlling loudness, pitch, and duration. First, acoustical param-eterssuchasloudnessandpitch,aswellasvariationsovertime,stronglyaffecttheperceptionoftimbre.Tostudyauditoryattributesindependentlyofthoseobviousparameters, it is thereforerecommendedtocontrol themandtousesteady-statesounds,equalizedinloudness,pitch*andduration.Thisstatementisinagreementwith the current ANSI definition and summarized by Krumhansl (1989, p. 44):timbre is“thewayinwhichmusicalsoundsdifferonce theyhavebeenequatedfor pitch, loudness and duration.” Otherwise, it is recommended to ask partici-pantstoignoretheseparameters,followingtheproposalbyPrattandDoak(1976),whodefinetimbreas“thatattributeofauditorysensationwherebyalistenercanjudgethattwosoundsaredissimilarusinganycriteriaotherthanpitch,loudnessorduration.”

11.2.1.3.2 Selecting an appropriate number of semantic scales. Second,arestrictednumberofsemanticpairssuitablefordescribingtimbrehavetobeselected.Indeed,the preselection of semantic descriptors by the experimenter may strongly affecttheresults,forthesedescriptorsmaynotnecessarilyconformwiththoseapartici-pantwouldusespontaneously.Forinstance,PrattandDoak(1976)investigated(bya questionnaire) what were the most appropriate adjectives for describing timbreofinstrumentsamongalistof19commonlyusedterms.Sevenwordsemergedasfavorites:rich,mellow,colorful,brilliant,penetrating,bright,andwarm.Similarly,

* Forpitch,thiscanbedoneasforloudnessbyanadjustmentprocedureusingthereal-timeSuperVPsoftwareprogrambasedonthephasevocodertechnique;itisthenpossibletotranspose,stretch,orshortensoundsinreal-time.

Y116134.indb 235 10/10/11 2:12 PM


invonBismark(1974),participantswereaskedtorateeachof69semanticscalesintermsoftheirsuitabilityfordescribingtimbre.Finally,28scaleswereconsideredasrepresentative.However,itshouldbenotedthatinvonBismarck’sstudy,the69scaleswereratedindependentlyofthesoundselectedforthestudyandthusmaynotberelevantfordescribingtheperceptualdimensionsofthesesounds.

11.2.1.3.3 Selecting relevant descriptors. Thethirdprerequisiteconsistsinaskingparticipantstojudgetherelevanceofthesemanticdescriptorsconcerningthesoundsusedinthestudy.Faure(2010)gatheredasetofsemanticdescriptorsfromafreever-balizationexperimenton12musicalsounds.Theywereusedtobuildsemanticscales.Therelevanceofthesescaleswasthenjudgedforthesamesetofsounds.Comparisonbetweentherelevancejudgmentsofthescalesandthevocabularyproducedsponta-neouslyshowedthatseveralsemanticdescriptorsthatwerespontaneouslyproduced(suchas“strong”,“loud”,etc.)werenotconsideredasrelevantwhenpresentedwiththescales,evenbytheparticipantswhoproducedthem.Inversely,severalsemanticdescriptorsthatwererarelyusedspontaneouslywerejudgedtobegloballyrelevantby the majority of participants (e.g., “soft,” “muffled/dull sounding,” “metallic,”“nasal”).Inanotherstudy(Kyncl&Jiricek,2001),participantsfreelydescribedsixvacuumcleanersounds.Amongthe33pairsofsemanticoppositionsobtainedfromthevocabularyspontaneouslyproduced,only5wereconsistentlyjudgedasrelevantfordescribingthesounds(“fuzziness,”“atypicality,”“inefficiency,”“loudness,”and“pleasantness”).ThesestudieshighlighttheimportanceofjudgingtherelevanceofdescriptorsusedinSD-scalesforaspecificcorpusofsounds.

11.2.1.3.4 Defining the meaning of the scales. The fourthprerequisiteconcernsthedefinitionofthescales.Indeed,itiscrucialthattheparticipantscorrectlyunder-standthemeaningofthelabels.Forinstance,inFaure(2010),thestimuliwereequal-izedinloudness.Surprisingly,theparticipantsspontaneouslyusedtheword“loud”to describe the sounds. Actually, the participants’ comments revealed that theyused“loud”todescribedifferentperceptions:“stronginsonicpresence,theattack,”“evokesthepowerandpersistenceofthesound.”Similarly,invonBismarck(1974),althoughthesoundswereequalizedinloudness,participantsusedthescale“soft–loud”todescribeattributesotherthanloudness,suchas“unpleasant.”Therefore,theexperimentmustclearlydefinethemeaningofthesemanticscalestoeliminateanyriskofsemanticambiguity.Presentingtheminasentencecanhelpdefinethemean-ingofthedescriptors.ForinstanceParizetandNosulenko(1999)showedthatratingsofinternalnoisesofvehiclesweremorereliablewhenthesemanticdescriptorswerepresentedinasentencethanwhenpresentedinisolation.Susini,Houix,Misdariis,Smith,andLanglois(2009)introducedthesemanticdescriptor“loud”bythesen-tence “The TV is too loud, we can’t have a discussion.” This sentence aimed atclearlyindicatingthat“loud”referredtothesoundlevelandnottheunpleasantness.

Inadditiontotheseveralprerequisitespresentedabove,otherrecommendationsshouldbetakenintoconsiderationwhenusingtheSD-scaletechniquetorateacor-pusofsounds.

Y116134.indb 236 10/10/11 2:12 PM


• Several studieshave shown that subjects feel uncertain ingiving ratingsunlesstheycanreferthemtothewholesampleofsounds.Thustheentirerangeofsoundshastobepresentedbeforethemainexperiment,andpar-ticipantsmustbeinstructedtousethefullrangeofthescale.Inaddition,itisrecommendedthattherangeofsensitivitycorrespondingtoeachseman-ticdescriptoroftheselectedsetofsoundsbebroadenough.

• Manystudiesontimbrehaveusedthetraditionalsemanticdifferentialpara-digm (e.g., dull–sharp).Bipolar adjectivepairs raise thequestionofpre-sentingtherightantonymlabels(isdulltheoppositeofsharpwhenusedtodescribesounds?).InChouardandHempel(1999),clearantonymswerefoundinabout23%ofthecasesforalistof242adjectivesproducedbytheparticipantstodescribeinteriorcarsounds.Thusanimportantproblemintheuseofbipolaroppositesisthatthe“opposite”issometimesunobtainableornotalwaysagoodantipode.Tosolvethisproblem,KendallandCarterette(1992)proposedusingascaleboundedbyanattributeanditsnegation(e.g.,sharp–notsharp)toratethetimbreofmusicalsounds.Theauthorstermedthismethodverbal attribute magnitude estimation(VAME),becausethetaskfor theparticipant is torate thedegree towhichanattribute ispos-sessedbyastimulus.

• Finally,werecommendpresentingthewholesetofsoundsforeachseman-tic descriptor instead of the classical way consisting in presenting onesoundtotheparticipant,whohastoevaluateitonthewholesetofsemanticdescriptors.InastudybyParizetandcolleagues(1999,2005),thecompari-sonofthetwomethodsshowedthattheformerprovedtobemoreaccurateandwithashorterdurationthantheclassicalone,becauselistenerswerefocusedononesemanticdescriptoratatimewhilehearinganewstimulus.Inaddition,tomeasuresubjectreliabilityoraccuracy,randompresentationofthestimulicanberepeated.Cross-correlationcoefficientsarecalculatedbetweenthedatafrombothpresentationsoftherepeatedstimulitocom-putesubjectreliability.

11.2.2 Dissimilarity judgments and multidimensional scaling technique

Semanticscalescomparestimulialongdimensionsdirectlydescribedsemantically.Itisthereforepossibletoassessvariouspsychologicalaspectsofacorpusofsounds,rangingfromelementaryauditoryattributestocognitiveandemotionalaspects.Thedisadvantageisthatthenumberofscalesisoftentoohighand,withtheexceptionofafewstudiesmentionedintheprevioussection,someoftheselectedsemanticdescriptorsarenotperceptuallyrelevant to thecorpusstudiedandaresometimesredundantinrelationtoeachother.However,thisapproachisappropriatetostudytheperceptionofvariousenvironmentalsounds,aslongasseveralprerequisitesaretakenintoaccount.

Incontrast,themultidimensionalscalingtechnique(MDS)isbasedondissimi-larityratingsandthusdoesnotrequireaprioriassumptionsconcerningthenumber

Y116134.indb 237 10/10/11 2:12 PM


ofperceptualdimensionsortheirnature,unlikethemethodsthatuseratingsalongspecifieddimensions.

11.2.2.1 MDS and auditory perception

The multidimensional scaling technique is a fruitful tool for studying perceptualrelationsamongstimuliand foranalyzing theunderlyingauditoryattributesusedbytheparticipantstoratetheperceivedsimilaritybetweentwosounds.MDSrep-resents theperceivedsimilarities ina low-dimensionalEuclideanspace(so-calledperceptual space), so that the distances among the stimuli reflect the perceivedsimilarities (seeMcAdams,Winsberg,Donnadieu,Soete,&Krimphoff,1995, forareviewofthedifferentMDSalgorithms).Eachdimensionofthespace(so-calledperceptual dimension)isassumedtocorrespondtoaperceptualcontinuumthatiscommontothewholesetofsounds.Itisalsoassumedthateachdimensioncanbewellexplainedbyanacousticparameterorapsychoacousticaldescriptor.Inotherwords,theMDStechniqueisappropriatefordescribingsoundsthatarecomparablealongcontinuousauditoryattributes,whichmeansthatitisappropriateforstudyinghomogeneouscorporaofsounds,thatis,thosemadeofsoundsproducedbythesametypeofsource.


Participantsratetheperceiveddissimilaritybetweeneachpairofsoundsundercon-sideration, that is,N(N–1)/2 ratings forN stimuli,onacontinuousscale labeled“VerySimilar”attheleftendand“VeryDissimilar”attherightend.Then,thedis-similaritiesaremodeledasdistancesinaEuclideanspaceofRdimensionsexpectedtobethemostrelevantperceptualdimensionssharedbythesounds.Inthepercep-tual space, a large dissimilarity is represented by a large distance. The final andthemostdifficultpartof thisapproach lies inmatchingperceptualdimensions toacousticalorpsychoacousticaldescriptors.

11.2.2.3 Example of MDS studies to describe timbre of musical sounds

ManyresearchershaveappliedtheMDStechniquetocharacterize theperceptualdimensionsofsounds,sincetheseminalstudiesbyPeters(1960)andPlomp(1970).Peters(1960)startedtoapplytheMDStechniquetoacorpusofsoundswithaknowndimensionality(16puretonescomposedof4frequenciesat4soundpressurelevels:theacousticaldimensionalityistherefore2).Theanalysisofthedissimilarityjudg-mentsfrom39participantssuccessfullyhighlightedthetwoexpectedauditoryattri-butes:pitchandloudness.HethereforeconcludedthattheMDStechniquemightbeusefultoexploresetsofsoundstheauditoryattributesofwhichwouldbeunknown.To test this idea, he applied the technique to other corpora of sounds, for whichthesalientauditoryattributeswereunknown(syntheticcomplexsoundsandspeechsounds).Theresultswerelesseasilyinterpretable(hefoundbetweenthreeandsixdimensionsforthecomplexsounds).Butcomparedtowhatheobtainedwithmore

Y116134.indb 238 10/10/11 2:12 PM


traditionalapproaches(freeverbaldescription,partitionscaling,magnitudeestima-tion), he concluded that “the most promising approach for the isolation and defi-nitionofperceptualdimensionsofcomplexsoundswas theMDSmodel” (p.52).Plomp(1970)appliedMDStosetsofmusicalsounds,whichyieldedthreeorthogo-naldimensions.

Since then, several psychoacoustical studies using MDS have shown clearlythat musical timbre is a multidimensional attribute. Grey (1977) identified threesalient dimensions shared by a corpus of musical sounds. Using a refinement oftheclassicalMDStechnique(EXSCAL,developedbyWinsberg&Carroll,1989),Krumhansl(1989)alsofoundaspacewiththreedimensionssharedbyacorpusofsynthesizedmusical sounds (winds, bowed string, plucked strings,mallet percus-sion).ThesamesetofsoundswasanalyzedbyMcAdams,Winsberg,Donnadieu,Soete,andKrimphoff(1995),whoalsofounda3-Dspace.Thefirstdimensionoftheperceptualspacewascorrelatedwiththecentroidoftheamplitudespectrum.Ithasgenerallybeen reported tocorrespond to the semanticdescriptors“metallic,”“sharp,”or“brilliant.”Theseconddimensionwascorrelatedwiththelogarithmoftheattacktimeoftheamplitudeenvelope,andcorrespondstothesemanticdescrip-tors“fast-slowattack,”“resonant,”or“dry.”Thethirddimensionwascorrelatedwiththespectral irregularity(logarithmof thespectraldeviationofcomponentampli-tudesfromaglobalspectralenvelopederivedfromarunningmeanoftheamplitudesofthreeadjacentharmonics)orthespectralflux(averageofthecorrelationsbetweenamplitudespectrainadjacenttimewindows).

11.2.2.4 Prerequisites for using MDS to study auditory perception

11.2.2.4.1 Controlling loudness, pitch, and duration. Itisimportanttoemphasizethatthemusicalsoundsusedinthestudiespreviouslymentionedwereequalizedinpitch,subjectiveduration,andloudness,sothatratingswouldonlyconcernthedif-ferencesintimbre.Indeed,certainauditoryattributes,suchasloudness,mightdomi-nateandoverpowerlesssalientones,asmentionedinSection11.2.1.3forsemanticscales.Twosoundsthatdiffermainlyintermsofloudnesswillbejudgedobviouslydifferentaccordingtothisdimension,withlittlecontributionfromotherdimensionsofvariationbeingtakenintoaccount.

11.2.2.4.2 Selecting a homogeneous corpus of sounds. Asmentionedearlier,MDSishypothesizedtorepresentacorpusofsoundsbyalimitednumberofcontinuousauditorydimensionsthatarecommontoallthesounds.Thatmeansthecorpushastobecomposedofhomogeneoussoundobjects(soundsproducedbythesametypeofobjectorstimulithatsoundrathersimilar,e.g.,aclassofcarsounds)inordertoavoidaperceptualstructurethatisstronglycategoricalforwhichtheMDSapproachisnotadapted(seenextsection).Aclusteranalysisonthesimilarityratingscanrevealthedegreeofhomogeneityofthesoundcorpus.Ifthetreestructureobtainedrevealsastrongcategorizationofthecorpus,itisadvisabletodeterminewhichcategoriesbestrepresenttheobjectivesofthestudyinordertoobtainappropriatestimuli.

Y116134.indb 239 10/10/11 2:12 PM


11.2.2.4.3 Limiting the number of sounds. Asparticipantsmaybecomefatiguedor losemotivationover time, applicationof theMDS technique is restricted to arathersmallnumberofsounds(moreor less20well-chosensounds),because thenumberofpairs(N(N–1)/2)growsrapidlywiththenumberofsounds(N).Thusapreliminarycategorizationexperimentmaybeadvisableinordertoselectthemostrepresentativesounds(seeSusini,McAdams,Winsberg,Perry,Vieillard,&Rodet,2004).Anotherpossibilitytoavoidbeingconfinedtoasmallnumberofstimuliistousesortingtasks.Indeed,thevalidityofusingsortingtasksforsoundsinsteadofpairedcomparisonshasbeentestedandshowntobeeffectivewithtwodifferentsetsofauditorystimuli(Bonebright,1996).However,furthertestshavetobeperformedinordertoconfirmthevalidityforcollectingdatausingsortingtasks.

11.2.2.4.4 Collecting information from participants. Once the perceptual con-figuration is obtained, it is important to identify the perceptual meaning of eachdimensionoreventolabelthedimensionsusingsemanticdescriptors,andalso,togiveaphysicalinterpretationbyestablishingsystematicrelationsbetweenthestimu-luscharacteristicsandtheirlocationsinthespace.Knowledgeandfamiliaritywiththesoundcorpusandperceptuallyrelevantacousticparametersarethusnecessaryinordertocharacterizethedimensionsofthespaceobjectively.Anotheroptionistodirectlyasktheparticipantstodescribewhichsensationtheyattendedtowhilejudgingthedissimilarities.

11.2.3 Sorting tasks

TheMDS technique isnot appropriate for setsof sounds causedbyverydiffer-ent and obviously identified sources. For instance, Susini, Misdariis, McAdams,& Winsberg (1998) applied an MDS analysis to an extremely heterogeneous setofenvironmentalsounds(trains,cars,andplanes).Theanalysisyieldedastronglycategoricalperceptualstructure:listenersidentifiedthesoundsourcesratherthancomparingthemalongcontinuousdimensions.Therefore,thispredominantcogni-tivefactor—recognition,classification,andidentificationofthesoundsource(seeMcAdams,1993)—violated theassumptionofunderlyingcontinuousdimensionsrequiredby theMDStechnique. In thiscase,otherexperimentalapproachesareneededand,particularly,thesortingtasks.

11.2.3.1 Sorting task, categorization, and auditory cognition

Sortingtasksareverycommonlyusedincognitivepsychologytoaddresstheques-tions of identification and categorization of sound sources. These questions aretightlybound: identifying thesourceofsoundcanbeviewedasconnectingaudi-toryperceptiontoconcepts,andconceptstolanguage,inabidirectionalrelationship(McAdams,1993;Goldstone&Kersten,2003).Severalapproachestotheorganiza-tionandprocessingofconceptsandcategorieshavebeendeveloped(seeGoldstone&Kersten,2003,orKomatsu,1992forareview).Beforepresentingthetechnicalprocedureofsortingtasks,webrieflyrecallthegeneralprinciplesoftheprototypical

Y116134.indb 240 10/10/11 2:12 PM


approachtocategorizationdevelopedbyRosch(1978),whichisveryoftenusedasanunderlyingframeworkinsortingtasks.Thisapproachisbasedonthenotionofsimilarityandisthereforewelladaptedtoaccountforperceptualconceptssuchasthoseusedtodescribesounds.

Rosch’sapproachtocategorizationreliesontwoprinciples.Firstcategorizationisbasedonthecognitive economyprinciple:categoriesalloworganismstohandletheinfinitenumberofstimulibytreatingthemasequivalentwhenthedifferentia-tionisirrelevantforthepurposeathand.Thesecondprincipleisthatthe world has structure.Categorizationoftheworldisthusnotarbitrary,butreliesonitsperceivedstructure(Rosch,Mervis,Gray,Johnson,&Boyes-Braem,1976).

TwoconceptsareoftenborrowedfromRosch’swork:first,Roschandhercol-leagueshaveexperimentallyidentifiedthreelevelsintaxonomiesofobjects:

• Thebaselevel:itemsinthesecategoriessharemanyelementsincommon.• Thesuperordinatelevel:thislevelismoreinclusivethanthebaselevel,but

itemsinthecategoriesatthislevelsharefewerelementsincommon.• Thesubordinatelevel:itemsinthecategoriesatthislevelsharemanyele-

mentsincommon,buttheclassesarelessinclusive.

Second,Roschhasintroducedthenotionofanalogcategorymembership:catego-riesareinternallystructuredintoaprototypeandnonprototypemembers.Fortheselattermembers,thereisagradientofcategory membership(Roschetal.,1978).


Inasorting task,listenersarerequiredtosortasetofsoundsandtogroupthemintoclasses.Whentheexperimenterdoesnotspecifyanyspecificcriteriathatthelisten-ershavetouse,thetaskiscalleda free-sorting task.Usually,thelistenersarealsorequiredtoindicatethemeaningofeachclass.*Sometimes,thelistenersalsohavetoselectaprototypeineachcategory(themostrepresentativemember).

Technically,becausepersonalcomputersarewidespreadinthelab,† theproce-dureamountstoprovidingthelistenerswithaninterfaceallowingthemtolistentothesoundsbyclickingoniconsandmovingtheiconssoastoformgroups.

Toanalyzetheresults,thepartitionofthesoundscreatedbyeachlisteneriscodedinanincidence matrix(inthematrix,0indicatesthattwosoundswereinseparategroupsand1 that theywere in the samegroup).Aco-occurrencematrix is thenobtainedbysummingtheincidencematrices,whichcanbeinterpretedasaproxim-itymatrix(Kruskal&Wish,1978).Therefore,aswithdissimilarityratings,sortingtasks result inestimating similaritiesbetween the sounds.However, the structureofthesedatamightbedifferentdependingontheprocedure.Forinstance,Aldrich,

* Aclassificationofthesoundsistheresultofasortingtask.“Categoriesareequivalenceclassesofdifferent(i.e.,discriminable)entitiesandcategorizationistheabilitytoformsuchcategoriesandtreatdiscriminableentitiesasmembersofanequivalenceclass”(Sloutsky,2003,p.246).

† Thingswererathermorecomplicatedwithoutcomputers;seeVanderveer(1979).

Y116134.indb 241 10/10/11 2:12 PM


Hellier,andEdworthy(2009)showedthatdissimilarityratingsencouragedpartici-pants touseacoustical information,whereasafree-sortingprocedureemphasizedcategorical information. Different techniques are available to visualize the prox-imitydata.Whenthedatafollowthetriangularinequality,butnottheultrametricinequality,*theyarebestrepresentedinalow-dimensionalgeometricalspace(e.g.,byusingMDS).Whentheyalsofollowtheultrametricinequality,theyarebestrep-resentedinatreerepresentation(Legendre&Legendre,1998).Clusteranalysescre-atesuchrepresentations.Themostpopulartreerepresentationis thedendrogram.Itconsistsinrepresentingthedatainahierarchicaltree.Insuchatree,theleavesrepresentthesounds,andtheheightofthenodethatlinkstwoleavesrepresentsthedistancebetweenthetwosounds.Therepresentationishierarchical,wellsuitedtorepresentclassinclusion,andthereforefitswellwithRosch’sframework.

11.2.3.3 Examples of urban soundscape categorization

Sortingtaskshavebeenlargelyusedtostudythecategorizationofeverydaysoundsand soundscapes† (Guyot, 1996;Guyot,Castellengo,&Fabre, 1997;Vogel, 1999;Maffiolo, Dubois, David, Castellengo, & Polack, 1998; Guastavino, 2007; seeSchulte-Fortkamp&Dubois,2006,forareviewofrecentadvances).

Morerecently,Tardieu,Susini,Poisson,Lazareff,andMcAdams(2008)conductedanexperiment thataimed tohighlight thedifferent typesofauditory informationthatareperceivedinthesoundscapesoftrainstations.Thegoalwasalsotodeter-minetheinformationthatparticipantsusedintherecognitionofthespacetypology.Sixty-sixsoundscapesampleswerepresentedtoparticipantsinafree-categorizationtaskwithverbalization.Theresultsshowedthatthelistenersgroupedtogetherthesamplesintoeightglobalcategories.Furtheranalysisaimedtoexplainthecategoriesonthebasisofthefreeverbalizations.Eachverbalizationwasreducedtothewordsthatcontainedadescriptivemeaning.Forexample, the text“Ihavegroupedherethesequencesthattookplaceinaticketoffice.Weclearlyhearpeopletalkingaboutpriceand ticket” is reduced to thewords“ticketoffice,clearlyhear,people, talk-ingaboutprice.”ThisreductionwasmadewiththehelpofthesoftwareLEXICO(2003),whichautomaticallycountseverywordinatext.Then,wordsaregroupedinto semantic fields that are deduced from theverbal descriptions. Five semanticfields were deduced (Figure 11.2): sound sources (e.g., trains, departure boards,ticket-punchingmachines,whistle,etc.),humanactivities(e.g.,conversations,steps,

* ThetriangularinequalitystatesthatforanythreepointsA,B,andC,d(A,C)≥d(A,B)+d(B,C),wheredisthedistancebetweenthetwopoints.InaEuclideanspace,thelengthofanysideofatrianglecan-notbegreaterthanthesumoftheothertwosides.Inanultrametricspace,thisinequalityisreplacedbyd(A,C)≤maxd(A,B),d(B,C).Inthiskindofspace,anygivensidemustbelessthanorequaltothelongeroftheothertwosides.NotethatthisislessconstrainingthantheEuclideancase.Theultra-metricinequalityistomostformsofhierarchicalclusteringwhatthetriangleinequalityistotwo-waymultidimensionalscaling.

† Theterm“soundscape”wasintroducedinthelate1970sbytheCanadiancomposerR.MurraySchafer(1977),whodefinedsoundscapeastheauditoryequivalenttolandscape.BesideSchafer’sproject,thetermsoundscapeperceptionisusedinascientificcontexttocharacterizehowinhabitantsperceive,experience,andappraisetheirsonicenvironment.

Y116134.indb 242 10/10/11 2:12 PM


Part

icip

ants

’ Cat

egor

ies:

Part

icip

ants

’ Ver

baliz

atio

ns:

Soun

d so

urce

sTi

cket

disp

ense

r12

34

67

85

Hum

an ac

tiviti

es

Room

effec

tSm

all s

pace

Conv

ersa

tion,

wai

ting,

step

s

Type

of s

pace

Judg

men

tsQ

uiet

spac

e

Wai

ting

room

,tic

ket o

ffice

Ann

ounc

emen

t mus

ic

Reve

rber

atio

n,co

nfus

ed

Wai

ting

conv

ersa

tions

Unp

leas

ant

Bar,

hall

Whe

eled

suitc

ases

Departure board

Echo

es, r

eson

ance

sEx

tern

al

Plat

form

sCo

rrid

or, h

all

Noi

sy, r

hyth

mic

Step

s, vo

ice

Trai

ns

–

Jingl

es

Arr

ival

s,de

part

ures

Voic

e,ar

rival

s,de

part

ures

Reso

nanc

es,

reve

rbs.

Hal

l

–

Mus

ic, t

rain

sVo

ices

,co

nver

satio

ns

Larg

ere

verb

erat

ion

Hal

l

Noi

sy,

unpl

easa

nt

– Clos

ed sp

ace,

smal

l

Voic

e, pe

ople

Qui

et, i

ntim

ate

Bar,

wai

ting

room

Sem

antic

Fie

lds:

Fig

ure

11.2

(S

ee c

olor

inse

rt.)

Den

drog

ram

repr

esen

ting

the

cate

gori

eso

ftra

ins

tati

ons

ound

scap

esfo

und

inT

ardi

eue

tal.

(200

8).T

hed

escr

ipti

ons

oft

hec

ateg

orie

spr

ovid

edb

yth

epa

rtic

ipan

tsa

rer

epor

ted

ont

het

able

bel

ow,g

roup

edi

nto

five

sem

anti

cfie

lds.

The

mos

tci

ted

verb

aliz

atio

nsa

re

show

nin

ala

rger

siz

efo

nt.

Y116134.indb 243 10/10/11 2:12 PM


transaction,departure,etc.),roomeffect(e.g.,reverberation,confined,exterior/inte-rior,etc.),typeofspace(e.g.,waitingroom,platforms,halls,etc.),andpersonaljudg-ment(e.g.,annoying,pleasant,beautiful,musical,etc.).

11.2.3.4 Prerequisites for using sorting tasks

Thesortingtaskisveryintuitiveforthelisteners,and,inthecaseofthefree-sortingtask,hasthegreatadvantageofleavingthelistenersfreetoarrangethesoundsastheywish.Contrary todissimilarityratings,a largenumberof thesoundscanbehandledbythelistenersinasession.

11.2.3.4.1 Considering a large number of existing sounds. It is possible withsortingtaskstotestmanyexistingsoundsthatarerepresentativeofthevarietyofsoundsunderconsideration.Forinstance74environmentalsoundswerepresentedinBonebright’sstudy(2001),150recordedsoundeffectsinScavone,Lakatos,andHarbke’sstudy(2002),48alarmsoundsinSusini,Gaudibert,Deruty,andDandrel’sstudy(2003),and66trainstationsoundscapesinTardieuetal.’sstudy(2008).

11.2.3.4.2 Collecting information on the type of similarities used for each cate-gory. Fromapracticalpointofview,contrarytotheMDSapproach,categorizationtasksarewelladaptedtodescribeperceptuallyheterogeneouscorporaofsoundsandtorevealdifferentlevelsofsimilaritiesbetweenthesounds.However,greatcarehastobetakenwhenanalyzingthecategoriesbecausethetypeofsimilaritiesusedbytheparticipantsmayvaryfromonecategorytoanother,dependingonthedifficultyinidentifyingthesoundsandontheexpertiseoftheparticipants(moreorlessskillwithsoundevaluation).Indeed,threetypesofsimilaritieshavebeenidentified(Lemaitreetal.,2010),basedonacousticalproperties(loudness,roughness,intensityfluctua-tions,etc.),identifiedphysicalinteractionscausingthesound(impactsoundonglass,rattlesoundonmetal,soundeffect,etc.)andmeaningsassociatedwiththeidentifiedsoundsources(soundsofbreakfast,soundsthatremindoneofchildhood,etc.).

11.2.3.4.3 Selecting the type of similarities. Semanticanalysesoftheverbaldescrip-tionsofthecategoriesproviderichinsightsthatreveal,ontheonehand,thestrategyusedbytheparticipantstoformthecategories,andontheotherhand,thetypeofinformationused.However,semanticanalysesareoftentimeconsumingandhavetobedonerigorouslybyexperts.Lemaitreetal.(2010)proposedanalternative,whichconsistsofaskingtheparticipantstorateforeachcategorywhichtypeofsimilarity(acoustical,causal,semantic)theyhadused.Theresultsmayhelptheexperimentertounderstandtheleveloftheperceptualstructuresunderlyingeachcategory.

11.3 Application: Sound quality of environmental sounds

Thequalityoftheacousticenvironmentiscurrentlyanimportantissue.Effortsarebeingmade toaccount for theannoyancecausedbynoises (Guski,1997).At thesametime,designersareseekingtoimprovethesoundqualityofindustrialproducts.

Y116134.indb 244 10/10/11 2:12 PM


Theideaofsoundqualityhasemergedrelativelyrecently.Itreferstothefactthatthesoundsproducedbyanobjectorproductarenotonlyannoyingorunpleasant,butarealsoawayforpeopletointeractwithanobject.Inthecaseofindustrialproducts,itisthereforeofmajorimportancetodesignsoundstomeetconsumerexpectations.

Sincethebeginningofthe1990s,soundqualityhasbeenconceivedofmainlyintheparadigmofpsychoacoustics.Thishasledtothedesignofexperimentalmeth-odsandauditorydescriptors relevant to soundquality.For instance,ZwickerandFastl(1999)askedparticipantstoratepleasantnessonaunidimensionalscale(e.g.,ratio scale). Then the pleasantness scores were correlated with psychoacousticaldescriptors. Ellermeier, Mader, and Daniel (2004) gathered preference judgmentsof environmental soundsusinga2AFC (twoalternative forcedchoice)procedureandanalyzedthemusingtheBTLtechnique(Bradley–Terry–Luce).Thistechniquerepresentedtheperceivedunpleasantnessonaratioscale.Theunpleasantnessscoreswerethenpredictedbyalinearcombinationofpsychoacousticdescriptors(rough-ness and sharpness). The semantic differential technique is also used to evaluatesoundquality.Ithasbeenlargelyusedforcars(Bisping,1997;Chouard&Hemepl,1999),vacuumcleaners(Ihetal.,2002),andrefrigerators(Jeon,2006).However,asnotedinSection11.2.3.1,definingtheappropriatesemanticdescriptorsofthescalesmustbedonecarefully.

Mostofthestudiesusepsychoacousticaldescriptors(loudness,roughness,etc.)toexplainunpleasantnessscoresorsemanticratings.Thesedescriptorsarecurrentlyincludedinmostsoundqualitysoftwarepackages,yettheyarenotalwaysadaptedtodescribingallkindsofeverydaysounds.Indeed,itappearsthatrelevantperceptualdimensionsaredifferentfromonestudytoanotheraccordingtothecorpusofsoundsunderconsideration.Therefore,thereareno“universal”acousticalorpsychoacousti-caldescriptorsthatcanbeusedtomeasurerelevantauditoryattributesforallcatego-riesofenvironmentalsounds,andwhichwouldthusprovidethesameeffectonthesoundqualityofanyproduct.

11.3.1 Application of the MDS technique to describe environmental sounds

Acrucialaspectfortheresearchinsoundqualityistodeterminetherelevantaudi-tory attributes related to a specific family of environmental sounds. The MDStechniquehasbeenshowntobeafruitfultoolforrevealingandcharacterizingtheunknownperceptualdimensionsunderlyingthetimbreofmusicalsounds.Duringthelastdecade,theMDStechniquehasbeensuccessfullyappliedtodifferentkindsofenvironmentalsounds:everydaysounds(Bonebright,2001),interiorcarsounds(Susini,McAdams,andSmith,1997),air-conditioningnoises(Susinietal.,2004),car door closing sounds (Parizet, Guyader, and Nosulenko, 2006), and car hornsounds(Lemaitre,Susini,Winsberg,McAdams,andLetinturier,2007).Forallthementionedstudies,MDSanalysesledto3-Dperceptualspaces(Figure 11.3pres-entsthe3-Dspaceobtainedforcarsounds)andallthedimensionsexceptoneweredescribedbydifferentacousticalparameters.

Y116134.indb 245 10/10/11 2:12 PM


The spectral centroid* is the acoustical descriptor shared by all the percep-tualspaces related toenvironmentalsounds.Therefore, thisdescriptorappears todescribemusicalsoundsaswellasenvironmentalsoundsandisrelatedtotheseman-ticdescriptors“metallic,”“sharp,”or“brilliant.”Asidefromthespectralcentroid,nouniversalauditoryattributesexisttocharacterizethetimbreofanysound,andaninventoryofthedifferentsalientauditoryattributestodescribethedifferentfam-ilyofsoundsisneeded.Ameta-analysisof10publishedtimbrespacesconductedbyMcAdams,Giordano,Susini,Peeters,andRioux(2006)usingmultidimensionalscaling analyses (CLASCAL) of dissimilarity ratings on recorded, resynthesizedorsynthesizedmusicalinstrumenttones,revealedfourprimaryclassesofdescrip-tors: spectral centroid, spectral spread, spectral deviation, and temporal envelope(effectiveduration/attacktime).

* Thespectralcentroidistheweightedmeanfrequencyofthespectrumofthesignal;eachpartialtoneisweightedbyitscorrespondingamplitude.Thecalculationofthisfeaturecanbemoreorlesscomplex(seetheworkbyMisdariisetal.,2010),butthebasicexpressionis:

SC

A f

A

i ii

ii

=×∑

∑

whereAiandfiaretheamplitudeandfrequencyofthecorrespondingpartial.

0.3

0.2

0.1

0

Dim

. III

0.1

0.2

0.3

0.20

0.20Dim. IDim. II

0.20.2

0.4 0.4

0.4

Figure 11.3 (See color insert.) Three-dimensional space for car sounds: dimension I isexplainedby theenergy ratiobetween theharmonicandnoisyparts,dimension IIby thespectralcentroid,anddimensionIIIbythedecreaseinthespectralenvelope.

Y116134.indb 246 10/10/11 2:12 PM


11.3.2 A general framework for sound quality

Inamoregeneralframework,theMDStechniquemaybecombinedwithanotherapproachbasedonasemanticstudyof thecorpusofsoundsunderconsideration,inordertomappreferencejudgmentsontobothrelevantobjectivedescriptorsandappropriate semantic descriptors. Figure 11.4 presents the framework of the dif-ferent stages of these related approaches. This general framework was appliedusingair-conditioningnoisesasanexampleinathree-partstudybySusini,Perry,Winsberg,Vieillard,McAdams,andWinsberg(2001),Siekierskietal.(2001),andJunker,Susini,andCellard(2001).

ThefirststepconsistsindeterminingtheperceptualspaceusingtheMDStech-nique. Then, in a second step, the acoustical descriptors that are correlated withthepositionsof thesoundsalong theperceptualdimensionsaredetermined. Inaparallelthirdstep,thesoundsareverballydescribedthroughadescriptiveanalysisthatinvolvesasmallnumberoftrainedlisteners.Thisstepprovidesalistofselectedsemanticdescriptors—whichwillbeusedtodefinerelevantsemanticscales—andaverbaldescriptionof theauditorycuesusedby theparticipants tocompare thesounds inorder toguide the researchof theobjectivedescriptorscorrelatedwiththeauditorydimensionsobtainedinthepreviousstage.Inthelaststep,participantsrate their preference (or annoyance) of the sounds. The degree of preference (or,inversely,annoyance)associatedwitheachsoundisrelatedtoafunctionofthesig-nificantobjectivedescriptorsontheonehand,andthesemanticdescriptorsontheother.Theadvantageofthisglobalapproachisthatitdoesnotlimittheexplorationandcharacterizationofthecomponentsofsoundqualitytoacousticalandsemanticdescriptorsthatarealreadyknown.Itprovidesamethodforfindingnewobjective

Semantic description

Relevant objective descriptors ( 1, 2, …)

Acoustical descriptionMultidimensional description

Relevant auditory dimensions

Pref

Preference judgments ( 1, 2, …), (S1, S2, …) S2

S1

2

1

Description of the auditorycues used to compare the

sounds

Relevant semanticdescriptors (S1, S2, …)

Figure 11.4 Frameworkforaglobalsoundqualityapproach,involvingmultidimensional,acoustical,andsemanticdescriptionscombinedwithpreferencejudgments,basedonSusinietal.(2001)andSiekierskyetal.(2001).

Y116134.indb 247 10/10/11 2:12 PM


andsemanticdescriptorsthatareperceptuallyrelevantfordescribingandevaluatingasoundobjectinthedesignprocess.

11.4 Perspectives: Sounds in continuous interactions

Themethodsreportedinthischapteralladdressthemeasurementofquantitiesrep-resentativeofwhatahumanlistenerperceives.Theevaluationoftheperceivedsound qualityofindustrialobjectsisaveryimportantdomaininwhichthesemethodsareapplied.Traditionally,theparadigmofsoundqualityevaluationconsidersalistenerpassively receiving information from the soundsof theproduct.Suchevaluationswould,forinstance,studytheacousticalpropertiesofacarengineroarthatauserprefers(aesthetics)andthatarerepresentativeofasportscar(functionality).

Newtechnologiesforsensingandembeddedcomputation,however,havemadeitpossiblefordesignerstoconsidersonicaugmentationsofamuchwiderarrayofeverydayobjectsthatincorporateelectronicsensingandcomputationalcapabilities.Wherecontinuousauditoryfeedbackisconcerned,thesoundisnolongerproducedinastaticorisolatedway,butisrathercoupledtohumanactioninrealtime.Thisnewdomainofapplicationsiscalledsonic interaction design.

Fromthestandpointofperception,thelevelofdynamicalinteractivityembodiedbysuchartifactsisverydifferentfromthesituationofpassivelisteninginwhichmostofthemethodsreportedarecarriedout.Insonicinteractions,participantsarenotlis-teningtosequencesofstaticsoundsselectedbyanexperimenter,butinsteaddynami-callyexplorethesoundsofaninteractiveobject.Thiscontextmaybethoughttobemorecloselyalliedwithenactiveviewsofperception(e.g.,Bruner,1966)thanwithsomeofthemoretraditionalapproachesfoundinexperimentalauditorypsychology.

Thestudyofsonicinteractionentailsanunderstandingofperceptual–motorbehav-ior,because theseprocessesunderlieanyformofhuman interaction.Newmethodsmay therefore be required. Such methods experimentally study how users performwhenrequiredtodoataskinvolvingsonicinteraction.Aninterestingexampleispro-videdinworkbyRath(Rath&Rocchesso,2005;Rath,2006,2007;Rath&Schleicher,2008).TheydescribetheBallancer,atangibleinterfaceconsistingofawoodenplankthatmaybetiltedbyitsuserinordertodriveavirtualballrollingalongtheplank.Theauthorsusedthisinterfacetostudyparticipants’abilitiestousethisauditoryfeedbackinataskinvolvingguidingtheballtoatargetregionalongthelengthoftheplank,dependingon thekindof soundused.Lemaitreet al. (2009)usedanother tangibleinterface(theSpinotron)implementingthemetaphorofachild’sspinningtoptostudyhowcontinuoussonicinteractionsguidetheuserinmakingaprecisegesture.

References

Aldrich,K.M.,Hellier,E.J.,&Edworthy,J.(2009),Whatdeterminesauditorysimilarity?The effect of stimulus group and methodology, Quarterly Journal of Experimental Psychology, 62(1),63–83.

Bisping, R. (1997). Car interior sound quality: Experimental analysis by synthesis, Acta Acustica United with Acustica, 83,813–818.

Y116134.indb 248 10/10/11 2:12 PM


Björk,E.A.(1985).Theperceivedqualityofnaturalsounds,Acustica, 57,185–188.Bonebright,T.L. (1996).Vocal affect expression: A comparison of multidimensional scal-

ing solutions for paired comparisons and computer sorting tasks using perceptual and acoustic measures,Doctoraldissertation,UniversityofNebraska.

Bonebright,T.L.(2001).Perceptualstructureofeverydaysounds:Amultidimensionalscalingapproach.InICAD,Helsinki,Finland.

Brunner,J.(1966).Toward a theory of instruction.Cambridge,MA:HarvardUniversityPress.Chouard,N.,&Hempel,T.(1999).Asemanticdifferentialdesignespeciallydevelopedfor

theevaluationofinteriorcarsounds.InJoint Meeting: ASA/EAA/DEGA(JASA(105),Berlin),p.1280.

Daniel,P.,&Weber,R.(1997).Psychoacousticalroughness:Implementationofanoptimizedmodel,”Acta Acustica United with Acustica,83,113–123.

Ellermeier,W.,Mader,M.,&Daniel,P.(2004).Scalingtheunpleasantnessofsoundsaccord-ingtotheBTLmodel:Ratio-scalerepresentationandpsychoacousticalanalysis,Acta Acustica United with Acustica, 90,101–107.

Fastl,H.(1989).Averageloudnessofroadtrafficnoise.In InterNoise 89,NewportBeach,CA.Fastl,H.(1991).Evaluationandmeasurementofperceivedaverageloudness.InA.Schick,

J. Hellbrück, and R. Weber (Eds.), Fifth Oldenburg Symposium on Psychological Acoustics(BIS,Oldenburg),205–216.

Fastl,H.(1997).Thepsychoacousticsofsound-qualityevaluation,Acta Acustica United with Acustica, 83,754–764.

Faure, A. (2010). Des sons aux mots, comment parle-t-on du timbre musical? EditionsEdilivre—APARIS,Paris.

Fechner, G.T. (1966). Elements of psychophysics. NewYork: Holt, Rinehart andWinston(Original1860).

Fletcher,H.(1940).Auditorypatterns,Review of Modern Physics,12,47–65.Gaver,W.W.(1993a).Whatdowehearintheworld?Anecologicalapproachtoauditoryevent

perception.Ecological Psychology, 5,1–29.Gaver,W.W.(1993b).Howdowehearintheworld?Explorationsinecologicalacoustics.

Ecological Psychology, 5,285–313.Glasberg,B.R.,&Moore,B.C.J.(2002).Amodelofloudnessapplicabletotime-varying

sounds,Journal of Audio Engineering Society,50,331–342.Goldstone,R.L.,&Kersten,A.(2003).Conceptsandcategorization.InA.F.Healy&R.W.

Proctor(Eds.),Comprehensive handbook of psychology, Vol. 4: Experimental psychol-ogy,chapter22,pp.599–621.NewYork:Wiley.

Grey,J.M.(1977).Multidimensionalperceptualscalingofmusical timbres,Journal of the Acoustical Society of America, 61,1270–1277.

Gros, L., & Chateau, N. (2001). Instantaneous and overall judgments for time-varyingspeechquality:Assessmentsandrelationships,Acta Acustica United with Acustica, 87,367–377.

Guastavino, C. (2007). Catagorisation of environmental sounds, Canadian Journal of Experimental Psychology, 61,54–63.

Guski,R.(1997).Psychophysicalmethodsforevaluatingsoundqualityandassessingacous-ticinformation,Acta Acustica United with Acustica, 83,765–774.

Guyot,F.(1996).Etudedelaperceptionsonoreentermesdereconnaissanceetd’appréciationqualitative: Une approche par la catégorisation. Doctoral dissertation, Université duMaine.

Guyot,F.,Castellengo,M.,&Fabre,B. (1997).Etudede lacatégorisationd’uncorpusdebruitsdomestiques.InD.Dubois(Ed.),Catégorisation et cognition,Paris:Kimé.

Y116134.indb 249 10/10/11 2:12 PM


Hansen,M.,&Kollmeier,B.(1999).Continuousassessmentoftime-varyingspeechquality.Journal of the Acoustical Society of America 106,2888–2899.

Hartmann,W.H.(1997).Signals, sound, and sensation.NewYork:AIPPress.Hedberg,D.,&Jansson,C.(1998).Continuous rating of sound quality.Stockholm:Karolinska

Institutet.Ih,J.-G.,LimD.-H.,JeongH.,&Shin,S.-H.(2002)Investigationonthecorrelationbetween

soundqualityandspectralcompositionvacuumcleanersoundbyusingtheorthogonalarray.InSound Quality Symposium,Dearborn,MI.

Jeon,J.Y.(2006).Soundradiationandsoundqualitycharacteristicsofrefrigeratornoiseinreallivingenvironments,Applied Acoustics, 68,1118–1134.

Junker, F., Susini, P., & Cellard, P. (2001). Sensory evaluation of air-conditioning noise:Comparativeanalysisoftwomethods.InICA,Rome,Italy,p.342.

Kendall,R.A.,&Carterette,E.C.(1992).Verbalattributesofsimultaneouswindinstrumenttimbres.I.vonBismarck’sadjectives.Music Perception, 4,185–214.

Komatsu,L.K.(1992).Recentviewsofconceptualstructure.Psychological Bulletin,112(3),500–526.

Krumhansl,C.L.(1989).Whyismusicaltimbresohardtounderstand?InS.Nielsen,andO.Olsson(Eds.),Structure and perception of electrostatic sound and music(pp.43–53).Amsterdam:Elsevier.

Kruskal,J.B.,&Wish,M.(1978).Multidimensional scaling.BeverlyHills,CA:Sage.Kuwano,S.,&Namba,S. (1978).On the loudnessof road trafficnoiseof longerduration

(20min)inrelationtoinstantaneousjudgment.InASA&ASJjointmeeting.(Journal of the Acoustical Society of America),Berlin, 64,127–128.

Kuwano,S.,&Namba,S.(1985).Continuousjudgementoflevel-fluctuatingsoundsandtherelationshipbetweenoverallloudnessandinstantaneousloudness.Psychology Research, 47,27–37.

Kuwano, S., & Namba, S. (1990). Continuous judgement of loudness and annoyance. InF.Müller(Ed.),Fechner Day, Proceedings of the 6th Annual Meeting of the International Society for Psychophysics,Würzburg,Germany,pp.129–134.

Kuwano,S.,&Namba,S. (2001).Dimensionsofsoundqualityand theirmeasurement. InICA, Rome,Italy,p.56.

Kyncl, L., & Jiricek, O. (2001). Psychoacoustic product sound quality evaluation. In ICA,Rome,Italy,p.90.

Lamalle,C.,Martinez,W.,Fleury,S.,Salem,A.,Fracchiolla,B.,Kuncova,A.,&Maisondieu,A. (2003).LEXICO 3, Outils de statistique textuelle—Manuel d’utilisation,SYLED-CLA2T(Paris3).

Legendre,P.,&Legendre,L.(1998).Numerical ecology.Amsterdam:Elsevier.Lemaitre,G.,Houix,O.,Misdariis,M.,&Susini,P.(2010).Listenerexpertiseandsoundiden-

tificationinfluencethecategorizationofenvironmentalsounds,Journal of Experimental Psychology: Applied,16, 16–32.

Lemaitre,G.,Houix,O.,Visell,Y.,Franinovic,K.,Misdariis,N.,&Susini,P.(2009)Towardthe design and evaluation of continuous sound in tangible interfaces:The spinotron.International Journal of Human Computer Studies,67,976–993.

Lemaitre,G.,Susini,P.,Winsberg,S.,McAdams,S.,&Leteinturier,B. (2007).Thesoundquality of car horns:Apsychoacoustical studyof timbre,Acta Acustica United with Acustica, 93,457–468.

Madsen,C.K.(1996).Empiricalinvestigationofthe“aestheticresponse”tomusic:Musiciansandnonmusicians.In4th International Conference on Music Perception and Cognition,Montreal,Canada,pp.103–110.

Y116134.indb 250 10/10/11 2:12 PM


Maffiolo,V.,Dubois,D.,David,S.,Castellengo,M.,&Polack,J.D.(1998).Loudnessandpleasantness instructurationofurbansoundscapes. InInterNoise,Christchurch,NewZealand.

McAdams,S.,&Bigand,E.(Eds.).(1993).Thinking in sound—The cognitive psychology of human audition,Oxford:UniversityPress.

McAdams,S.,Giordano,B.,Susini,P.,Peeters,G.,&Rioux,V.(2006).Ameta-analysisofacousticcorrelatesoftimbredimensions.InASA,Hawaii.

McAdams,S.,Vines,B.W.,Vieillard,S.,Smith,B.,&Reynolds,S. (2004). Influencesoflarge-scale formoncontinuous ratings in response toacontemporarypiece ina liveconcertsetting.Music Perception, 22,297–350.

McAdams,S.,Winsberg,S.,Donnadieu,S.,Soete,G.D.,&Krimphoff,J.(1995).Perceptualscalingofsynthesizedmusicaltimbres:Commondimensions,specificities,andlatentsubjectclasses.Psychological Research, 58,177–192.

Meunier,S.,Boullet,I.,&Rabau,G.(2001).Loudnessofimpulsiveenvironmentalsounds.InICA,Rome,Italy,p.91.

Misdariis,N.,Minard,A.,Susini,P.,Lemaitre,G.,McAdams,S.,&Parizet,E.,(2010).Environ-mental sound perception: Metadescription and modeling based on independent primarystudies.Journal on Audio, Speech, and Music Processing. doi:10.1155/2010/362013.

Moore,B.C.J. (2003).An introduction to the psychology of hearing (5thed.).NewYork:AcademicPress.

Moore,B.C.J.,&Glasberg,B.R.(1983).Suggestedformulaeforcalculatingauditory-filterbandwidthsandexcitationpatterns.Journal of the Acoustical Society of America,74 (3),750–753.

Moore,B.C.J.,&Glasberg,B.R.(1990).Derivationofauditoryfiltershapesfromnotched-noisedata.Hearing Research, 47,103–138.

Moore,B.C. J.,&Glasberg,B.R. (1996).A revisionofZwicker’s loudnessmodel.Acta Acustica United with Acustica, 82,335–345.

Moore,B.C.J.,Glasberg,B.R.,&Baer,T.(1997).Amodelforthepredictionofthresholds,loudnessandpartialloudness.Journal of the Audio Engineering Society, 45,224–240.

Osgood,C.E.(1952).Thenatureandmeasurementofmeaning.Psychological Bulletin, 49,197–237.

Parizet,E.(1999).Comparisonoftwosemantic-differentialtestmethods.InEAA & ASA joint meeting.Journal of the Acoustical Society of America,Berlin,p.1189.

Parizet,E.,&Nosulenko,V.N.(1999).Multi-dimensionallisteningtest:Selectionofsounddescriptorsanddesignoftheexperiment.Noise Control Engineering Journal, 47,1–6.

Parizet,E.,Guyader,E.,&Nosulenko,V.(2006).Analysisofcardoorclosingsoundquality.Applied Acoustics,69,12–22.

Parizet,E.,Hamzaoui,N.,andSabatié,G.(2005).Comparisonofsomelisteningtestmethods:Acasestudy.Acta Acustica United with Acustica, 91,356–364.

Parizet,E.,Hamzaoui,N.,Segaud,L.,&KochJ.-R.(2003).Continuousevaluationofnoisediscomfortinabus.Acta Acustica United with Acustica,89,900–907.

Patterson,R.D.,&Holdsworth,J.(1991).Afunctionalmodelofneuralactivitypatternsandauditoryimages.Advances in speech, hearing and language processing,Vol. 3.London:JAIPress.

Peters, R. W. (1960). Research on psychological parameters of sound, (pp. 60–249).Hattiesburg:MississippiSouthernCollege.

Plomp, R. (1970). Timbre as a multidimensional attribute of complex tones. In R. Plomp&G.P.Smoorenburg(Eds.),Frequency anaysis and periodicity detection in hearing(pp.397–414).Leiden:A.W.Sijthoff.

Y116134.indb 251 10/10/11 2:12 PM


Pratt,R.L.,&Doak,P.E.(1976).Asubjectiveratingscalefortimbre.Journal of Sound and Vibration, 45,317–328.

Rath,M. (2006).On the relevanceof auditory feedback for subjective andobjectivequal-ityofcontrolinabalancingtask.InProceedings of the 2nd ISCA/DEGA Tutorial and Research Workshop on Perceptual Quality of Systems,Berlin,pp.85–88.

Rath,M.(2007).Auditoryvelocityinformationinabalancingtask.InICAD,Montréal,Canada,pp.372–379.

Rath, M., & Rocchesso, D. (2005). Continuous sonic feedback from a rolling ball. IEEE MultiMedia, 12,60–69.

Rath,M.,&Schleicher,D.R.(2008).Ontherelevanceofauditoryfeedbackforqualityofcontrolinabalancingtask.Acta Acustica United with Acustica, 94,12–20.

Robinson,D.W.,&Dadson,R.S.(1956).Are-determinationoftheequal-loudnessrelationsforpuretones.British Journal of Applied Physics, 7, 166–181.

Rosch,E.(1978).Principlesofcategorisation.InE.RoschandB.B.Lloyd(Eds.),Cognition and categorization,(pp.27–47).Hillsdale,NJ:LawrenceErlbaumAssociates.

Rosch, E., Mervis, C. B., Gray,W. D, Johnson, D. M., & Boyes-Braem, P. (1976). Basicobjectsinnaturalcategories.Cognitive Psychology, 8,382–443.

Scavone,G.P.,Lakatos,S.,&Harbke,C.R.(2002).Thesonicmapper:Aninteractivepro-gramforobtainingsimilarityratingswithauditorystimuli.InICAD, Kyoto.

Schafer,R.M.(1977).The tuning of the world.NewYork:RandomHouse.Schubert,E.(1996).Continuousresponsetomusicusingthetwodimensionalemotionspace.

InICMPC, Montreal,Canada,pp.263–268.Schulte-Fortkamp,B.,&Dubois,D.(2006).Recentadvancesinsoundscaperesearch.Acta

Acustica United with Acustica, 92(6),5–6.Siekierski,E.,Derquenne,C.,&Martin,N. (2001).Sensoryevaluationofair-conditioning

noise:Sensoryprofilesandhedonictests.InICA,Rome,Italy,326.Sloboda,J.A.,&Lehmann,A.C.(2001).Trackingperformancecorrelatesofchangesinper-

ceivedintensityofemotionduringdifferentinterpretationsofaChopinpianoprelude.Music Perception, 19,87–120.

Sloutsky,V.M.(2003)Theroleofsimilarityinthedevelopmentofcategorization.Trends in Cognitive Science, 7,246–251.

Solomon,L.N.(1958),Semanticapproachtotheperceptionofcomplexsounds.Journal of the Acoustical Society of America, 30(5),421–425.

Stepánek,J.(2006).Musicalsoundtimbre:Verbaldescriptionanddimensions.InDAFx-06,Montréal.

Stevens,J.C.,&Tulving,E.(1957).Estimationsofloudnessbyagroupofuntrainedobserv-ers,American Journal of Psychology, 70,600–605.

Stevens,S.S.(1959).Cross-modalityvalidationsofsubjectivescalesforloudness,vibrations,andelectricshock.Journal of Experimental Psychology, 57, 201–209.

Susini,P.,Gaudibert,P.,Deruty,E.,&Dandrel,L.(2003).Perceptivestudyandrecommenda-tionforsonificationcategories.InICAD,Boston.

Susini,P.,Houix,O.,Misdariis,N.,Smith,B.,&Langlois,S.(2009.)Instruction’seffectonsemanticscaleratingsofinteriorcarsounds.Applied Acoustics, 70(3),389–403.

Susini,P.,&Maffiolo,V.(1999).Loudnessevaluationofurbansoundscapesbyacross-modalmatchingmethod. InEAA & ASA joint meeting,Journal of the Acoustical Society of America,Berlin,p.943.

Susini,P.,&McAdams,S.(1998).Globalandcontinuousjudgementsofsoundswithtime-varyingintensity:Cross-modalmatchingwithaproprioceptiveinputdevice.InICA & ASA joint meeting,Journal of the Acoustical Society of America,Seattle,p.2812.

Y116134.indb 252 10/10/11 2:12 PM


Susini,P.,&McAdams,S. (2008).Loudness asymmetry ratingsbetween accelerating anddeceleratingcarsounds,inAcoustics’08, ASA-EAA Joint Conference,Paris.

Susini,P.,McAdams,S.,&Smith,B.(2002).Globalandcontinuousloudnessestimationoftime-varyinglevels.Acta Acustica United with Acustica, 88,536–548.

Susini,P.,McAdams,S.,&Smith,B.(2007).Loudnessasymmetriesfortoneswithincreasinganddecreasinglevelsusingcontinuousandglobalratings.Acta Acustica United with Acustica, 93,623–631.

Susini, P., McAdams, S., and Winsberg, S. (1997). Perceptual characterisation of vehiclenoises. InEEA Symposium: Psychoacoustic, in industry and universities,Eindhoven,TheNetherlands.

Susini,P.,McAdams,S.,Winsberg,S.,Perry,I.,Vieillard,S.,&Rodet,X.(2004).Characteriz-ingthesoundqualityofair-conditioningnoise.Applied Acoustics, 65(8),763–790.

Susini,P.,Misdariis,N.,McAdams,S.,&Winsberg,S.(1998).Caractérisationperceptivedebruits.Acoustique et Techniques,13,11–15.

Susini,P.,Perry,I.,Winsberg,S.,Vieillard,S.,McAdams,S.,&Winsberg,S.(2001).Sensoryevaluation of air-conditioning noise: Sound design and psychoacoutic evaluation. InICA,Rome,Italy,p.342.

Tardieu,J.,Susini,P.,Poisson,F.,Lazareff,P.,&McAdams,S.(2008).Perceptualstudyofsoundscapesintrainstations,Applied Acoustics, 69,1224–1239.

Vanderveer,N.J.(1979)Ecological acoustics: Human perception of environmental sounds.Unpublisheddoctoraldissertation,CornellUniversity,Ithaca,NY.

Vogel, C. (1999). Etude sémiotique et acoustique de l’identification des signaux sonores d’avertissement en contexte urbain.Doctoraldissertation,UniversitéParis6.

vonBismarck,G.(1974).Timbreofsteadysounds:Afactorialinvestigationofitsverbalattri-butes,Acustica, 30,146–158.

Vos,J.(1998).Theloudnessofimpulseandroad-trafficsoundsinthepresenceofmaskingbackgroundnoise,Acta Acustica United with Acustica, 84,1119–1130.

Wedin,L.,&Goude,G.(1972).Dimensionanalysisoftheperceptionoftimbres.Scandinavian Journal of Psychology, 13,228–240.

Weber,R.(1991).Thecontinuousloudnessjudgementoftemporallyvariablesoundswithan “analog” category procedure. InA. Schick, J. Hellbrück, and R.Weber (Eds.),Fifth Oldenburg Symposium on Psychological Acoustics,(pp.267–294)Oldenburg:BIS.

Winsberg,S.,&Carroll,D.(1989).Aquasinon-metricmethodformultidimalscalingviaanextendedEuclidianmodel.Psychometrika, 54,217–229.

Zeitler,A., & Hellbrück, J. (2001). Semantic attributes of environmental sounds and theircorrelationswithpsychoacousticmagnitude.InICA,Rome,Italy,p.114.

Zwicker,E.(1961).Subdivisionoftheaudiblefrequencyrangeintocriticalbands,Journal of the Acoustical Society of America, 33,248.

Zwicker,E.(1984).BASIC-Programforcalculatingtheloudnessofsoundsfromtheir1/3-octbandspectraaccordingtoISO532-B,Acustica, 55,63–67.

Zwicker, E., & Fastl, H. (1972). On the development of the critical band, Journal of the Acoustical Society of America, 52,699–702.

Zwicker,E.,&Fastl,H.(1999).Psychoacoustics: Facts and models (2nded.).Heidelberg:Spinger-Verlag.

Zwicker,R.,andTerhardt,E.(1980).Analyticalexpressionsforcriticalbandrateandcriticalbandwidthasafunctionoffrequency,Journal of the Acoustical Society of America, 68,1523–1525.

Y116134.indb 253 10/10/11 2:12 PM

Y116134.indb 254 10/10/11 2:12 PM

255

Nociception and pain in thermal skin sensitivity

Dieter Kleinböhl,1 Rupert Hölzl,1 and Jörg Trojan2

1Otto–Selz-InstituteforAppliedPsychology,UniversityofMannheimMannheim,Germany2CentralInstituteofMentalHealth,DepartmentofCognitiveandClinicalNeuroscienceMannheim,Germany

12.1 Introduction

Bodyperception,or interoception, ismadeupofamultitudeofsensorysystems,coarselysortedbyfunctionalaspectsintoproprioception,visceroception,andnoci-ception(Sherrington,1906).Proprioception,initsmoderninterpretation, includesthe haptic-somatic sensitivity, with tactile and thermal submodalities of the skinsenses, aswell as the sensitivity formovement andpositionof the joints and thelimbsandforbodyposture.Visceroceptionrelatestotheperceptionofsignalsfromtheintestines,originatingfromreceptorsintheholloworgans,whicharesensitiveforpressureanddistension(Hölzl,Erasmus,&Möltner,1996).Nociception,finally,istheprocessingofintense,noxiousstimulithatmayoriginatefromallpartsofthebody,thusencompassingaspectsofsomatic-hapticsensitivityandvisceroception.

AmodernviewofinteroceptivesensitivityisgivenbyCraig(2003a,2002),whopostulates from neurophysiologic criteria the existence of a phylogenetic ancient“homeostatic”systemofsmallnervefibers,includingnociception,temperaturesen-sitivity,andotherinteroceptivereceptorsystemsmaintainingbodilyfunctions.Thisneurophysiologic coherent system of “homoeostatic afferents” is related to limbicbrainareasinvolvedinaffectiveprocessing(e.g.,anteriorcingulatecortex, insula),and with brain areas modulating affect (e.g., amygdala, orbitofrontal cortex). Theanteriorinsulaisassumedtobecentralfortheintegrationofbodyperceptioninthesenseofan“interoceptivecortex”(Craig,2003b).Thisbrainareamediateshomeo-staticemotions,wherepleasantbodilyfeelingsindicatehomeostasisandunpleasantfeelingsindicateadisturbanceofhomeostasis.Inthisviewofahomeostaticsmallfibersystem,nociceptiveandthermoceptivesensitivityplayacentralrole,beingatthecoreofbodyperceptionitself.Anothercentralfeatureofnociceptionisitsplasticity:nociceptionincludesamultitudeofdynamicmechanismsonperipheralandcentral

12

Y116134.indb 255 10/10/11 2:12 PM


levelsofprocessingthatarecapableofchangingnociceptivetransmissionandpainperception.This feature isnot alwaysadaptive for theorganism,andmaladaptionmayleadtohyperalgesiaandchronicpainunderconditionsnotyetfullyunderstood.

This chapter focuses on nociception, thermoception, and the pain experience,beingacentralcomponentofbodyperception.Itisorganizedinthreeparts:thefirstpartsketchestheneurophysiologyofnociceptionandthephysicalconditionsactivat-ingnociception.Thesecondpartfocusesonmechanismsofdynamicchangeovertimeinnociceptionandpain.Dynamicchangeinnociceptionandpainisexempli-fiedbythreemechanismsondifferentlevelsofprocessing:neuronalplasticityonacellularlevel,automaticprocessesinperception,leadingtosensitizationandhabitu-ation,andassociative learningofalteredpainperception.The thirdpartpresentsthreepsychophysicalprocedures, basedonexperimental contact heat pain stimu-lation,fortheassessmentofthethreeselectedmechanismsofdynamicchangeinnociceptionandpain.

12.2 Neurophysiology of nociception

The neuronal structures involved in the processing of nociceptive information areexplainedhere.Togetherwiththedescriptionofthespecificphysicalstimulielicitingnociception,thisispartoftheobjectivephysiologyofthesenses.Theneurophysiologyofnociceptionisdescribedforthreelevelsofthenervoussystemwhichareorganizedinascendingorder.Eachleveldependsontheloweroneandprovidesmorecomplexcapabilitiesofadaptiveprocessing:first,peripheralnociceptiondealswiththetrans-missionfromthereceptorviaafferentnervefiberstothefirstcentralneuroninthespi-nalcord;second,spinalnociceptionrelatestotheneuronsandnervoustractswithinthespinalcordascendingtothebrainstemandthebrain;andthird,thestructuresofthebrainthatareinvolvedinpainprocessing.Therelationshipoftheneurophysi-ologicalaspectsofnociceptionwithqualitiesofpainperceptionisalsopresented.

12.2.1 Peripheral nociception

Thetermnociceptionwasintroducedtodifferentiatethe“neuralprocessesofencod-ingandprocessingnoxiousstimuli”from“theperceptionofpain”(Loeser&Treede,2008).Peripheralnociceptioncomprisestheneuronaltransmissionfromtheperiph-eral receptor to the presynaptic endings of the peripheral nerve fiber on the firstcentralneuroninthespinalcord.

Noxiouseventsaredetectedandcodedinthenervoussystembyaspecifictypeofreceptor,thenociceptor.Nociceptorsaredescribedassensitivenerveendingswhosemorphologyisnotyetcompletelyclarified.Nociceptorsarefoundmainlyintheskin,intheentiremusculatureofthebody,andinthejoints,aswellasininternalorgansandtheintestines.Specificstimulifornociceptorsarestrongthermal,mechanical,or chemical stimuli that might cause damage to the organism (Torebjörk, 1994).Themajorityofnociceptorsaresimultaneouslyresponsivetomechanical,thermal,andchemicalstimuli.Fortheirmultisensitivitythesenociceptorsaretermedpoly-modal nociceptors.Toamuchlesserdegree,therearealsounimodalmechano-or

Y116134.indb 256 10/10/11 2:12 PM

Nociception and pain in thermal skin sensitivity 257

thermosensitivenociceptors(Heppelmann,Messlinger,Schaible,&Schmidt,1991;Zimmermann&Handwerker,1984).

Activationofnociceptorsisnotsolelyduetoexogenousstimulation.Noxiouseventsmightdamagetissueandinthislinetriggerinflammation,withchemicalmediatorsactingasalgeticsubstances in thesurrounding tissue(e.g.,histamine,prostaglan-dine,bradykinin,andserotonin).Theseproinflammatoryandalgeticsubstancesaresynthesizedintheorganismandcanbeconsideredasendogenousnoxiousevents,whicharecapableof triggeringactionpotentials innociceptivefibers.Thismightoccurwith,butalsowithoutadditionalexogenousnoxiousstimulation(Heppelmannetal.,1991;Mense,Hoheisel,Kaske,andReinert,1997).

ThemostcommonclassofnociceptorsconsistsofthepolymodalC-fibernocicep-torssensibleformechanicalandthermalstimuli(C-fibermechano-heatnociceptors,CMHs;Raja,Meyer,&Campbell,1990).Thethresholdoftheirneuronalresponsetocontactheatstimulation,asmeasuredbymicroneurography,liesinawidephysicaltemperaturerangeof38–50°C.Thismeansthatthesenociceptorsmightbeactiveatnonpainfultemperaturesnormallynotevenperceivedashot.Thereforeitrequiresconsiderable spatial and temporal summation of afferent influx for perception ofthermalpaintooccur(Nielsen&Arendt-Nielsen,1998).

ThesecondcommonclassofnociceptivefibersisamongtheAδ-fibers,coveringthehighintensityrangeofnoxioustemperatures.ThepolymodalAδ-fibernocicep-tors(A-fibermechano-heatnociceptors,AMHs)willrespondonlyabovetempera-turesof48°C.Thesefibersare further subdivided in twoclassesofAMHfibers,accordingtotheirtemporalresponseproperties:TypeIAMHfiberswillcharacteris-ticallyrespond(within600ms)andadaptslowlytoanintensivenociceptivestimulus.IntypeIAMHs,anincreasedsensitivityisobservedafterrepeatednoxiousstimula-tion,whichmediatesatleastinpartthehyperalgesiaandsensitizationobservedaftersuchstimulusseries(Meyer&Campbell,1981;Rajaetal.,1990).TypeIIAMHs,onthecontrary,respondfast(<200ms)andadaptquickly(Rajaetal.,1990),thusmediatingthecharacteristicpinprickingorstingingqualityof“firstpain.”

The specific nociceptive fibers innervating nociceptors belong to the group ofso-calledsmall nerve fibers,constitutingthehomeostaticsmallfibersystempostu-latedbyCraig(2003a).SmallnervefiberscomprisethefastconductingmyelinatedAδ-fibersandslowconductingunmyelinatedC-fibers(Figure 12.1).Inhumans,50%ofallAδ-fibersareestimatedtobenociceptive,whereasforC-fibers,amuchhigherproportionof90%isassumedtobenociceptive.

Asidefromsignalingphysiologicalinformationformaintaininghomeostasis(e.g.,bloodpressure,O2saturation),smallfibersalsoinnervatenonnoxiouswarm-andcoldreceptorsintheskin:warmreceptorsrespondviaafferentC-fibers,andcoldreceptorsviaafferentAδ-fibers(Darian-Smith,1984).Smallfibersthereforemediatetheper-ceptionofawidetemperaturerangeonemightexperienceontheskin,rangingfromnoxiouscoldovernonnoxiouscoldtowarmth,heat,painfulheat,andnoxiousheat.

Thereisconsiderableoverlapofthephysicalresponsepropertiesofthecontrib-utingreceptors(Figure 12.1)andalsohighlyintegrativeprocessingoftheafferentactivationpatterninthedifferentreceptor/fibersystemssubservingtemperatureandpain perception. The basic properties of the small fiber subsystems are therefore

Y116134.indb 257 10/10/11 2:12 PM


classifiedbythesensorymodalityandbytheintensityrangetowhichtheyrespond.Anothercriterionforclassificationistheconductionvelocityoftheafferentfiberanditsresponseproperties(Rajaetal.,1990;Table 12.1).

12.2.2 Spinal nociception

Theperipheralnociceptivenervefibersenterthespinalcordviathedorsalrootgan-glionandthedorsalhornofthespinalcord,wheretheyhavepresynapticendingsoncentraltransmissionneurons.Withinthespinalcord,10histologicallydifferentcelllayers(laminae)aredescribedwhichdiffer,amongothercharacteristics,bytheproportionofafferentinflux(Rexed,1952).Thenociceptiveneuronsofthedorsalhorn lie in thesuperficial layersof thespinalcord,mainly in layersIandII,andalsopartiallyinthedeeplaminaVofthespinalcord(Cervero,1986;Millan,1999).Thedorsalhorntransmissionneuronsaresubdividedinclasses,accordingtotheirsensoryproperties(Mendell,1966;Cervero,1986):

Class 1 of spinal transmission neurons receives input from low-thresholdmechanoceptiveneurons,butshowsnonociceptiveresponses.

Noxious Cold (CMH, AMH)

Noxious Heat (AMH I)

Noxious Heat (CMH)

Warmth (C)

Cold (Aδ)

Pain

Heat Pain

Hot

Warm

Cold0 10 20 30

Temperature (°C)40 50 60

Noxious Heat (AMH II)

Figure 12.1 Responsepropertiesofthermo-nociceptivesmallfiberreceptorsovertempera-ture.Singlecurvesindicatecumulatedreceptoractivitydependentontemperature.Sensitivitycurvesarestackedinascendingordercorrespondingtotheperceptualqualitiesontherightordinate,indicatingwarm,hot,orpainfulsensations.Thereisconsiderableoverlapofsensi-tivityrangesforthedifferentreceptorclasses,especiallyinthenoxioustemperaturerange,startingapproximatelyaround45°C(Datasources:Kleinböhl,1996;Treedeetal.,1998;Rajaetal.,1990).

Y116134.indb 258 10/10/11 2:12 PM


Class 2 of spinal transmission neurons receives convergent afferent influxfromnociceptorsandfromlow-thresholdmechanoreceptors.Thesemulti-receptiveneurons,alsotermed“lowthreshold”or“widedynamicrange”(WDR), may increase their sensitivity under certain stimulus conditionsand,therefore,havebeendiscussedascandidatemediatorsforhyperalgesiaandchronicpain.Class2neuronsarefoundinlaminaeIIandV,theaffer-entinfluxtolaminaII(alsotermedthe“substantiagelatinosa”)originatingparticularlyfromC-fibersoftheskin(Cervero,1986).

Class 3 spinal transmission neurons are innervated exclusively from high-thresholdnociceptors,eitherpurelymechanosensitive(class3a)ormech-ano- and thermosensitive (class 3b). These neurons transfer informationaboutintensenoxiousstimulionly.Therefore,theyarealsocalled“nocicep-tivespecific”(NS)neurons.Theyaremostly,butnotexclusively,locatedinlaminaIandreceivepredominantlyafferentinfluxfromAδ-fibers(Hylden,Hayashi,Bennett,&Dubner,1985).

Nociceptive information is already processed on the level of the spinal cord,allowinganimmediateresponseoftheorganismtonoxiousstimuli.Responsechan-nelsforthereleaseofsympatheticandnocifensivemotorreflexesareactivatedviainterneurons.Thelatterallowaquickmotorresponsesuchaspullingawayofthehandfromahotplate.Vegetativeresponsestonoxiousstimulisuchasaccelerationofbreathingandheartbeatoranincreaseinbloodpressurearemodulatedbyasmallpart of the ascending nociceptive fibers, joining the medulla oblongata (Droste,1988).Finally,vigilanceandawarenessaremodulatedbynociceptiveinputviatheascendingreticularactivatingsystem(ARAS)intheformatioreticularisofthebrainstem(Zimmermann,1993).

12.2.2.1 Ascending nociceptive pathways

The transmission of nociceptive information from the spinal cord to the brain isperformedbytwofunctionallyandanatomicallydistinctpathways(lat.“tractus”),

Table 12.1 Propertiesofperipheralnociceptivenervefibers

Property AMH1-type I AMH1-type II CMH2

Conductionvelocity[m/s] 15.2±9.9 31.1±1.5 0.8±0.1Thermalthreshold[°C] >49 43 43.6±0.6Mechanicalthreshold[bar] 3.5±0.3 1.7 6.0±0.6Rise-timereceptor[ms] Slow:>600 Fast:<200 >50Skintype Hairy,glabrous Hairyonly Hairy,glabrousPerceptualqualities Primaryhyperalgesia Firstpain Secondpain

Note: OverviewcompiledbyKleinböhl(1996),basedonatablefromRajaetal.(1990).1 AMH:A-fiber-mechano-heat,anervefiberwithamyelinsheath,allowingforfast-actionpoten-

tialconductionvelocities.2 CMH:C-fiber-mechano-heat,anunmyelinatednervefiber,andthereforeallowingonlyforslow

conductionvelocities.

Y116134.indb 259 10/10/11 2:12 PM


whicharetermedmedialandlateralsystemaccordingtotheirtargetnucleiinthethalamus.Intheevolutionaryoldermedialsystem(tractuspalaeospinothalamicus),mainlynervetractsfromtheformatioreticularis,butalsofromtractusspinothalam-icusandtractustrigeminothalamicus,gotothemedialnucleiofthethalamus(nucleicentralislateralis,centralismedialis,andparafascicularis).Thesenucleiareassumedtobeconnectedwithlimbicareasofthebrainandwiththehypothalamus.Intheevolutionarilyyoungerlateralsystem(tractusneo-spinothalamicus)thetrajectoriesof the tractus spinothalamicusand tractus trigeminothalamicus join in the lateralsomatosensorynucleiofthethalamus(ventro-basal-nucleus)andthenconnectwiththesomatosensorycortexinthepostcentralgyrus(e.g.,Apkarian,1995).

12.2.2.2 Descending inhibitory pathways

Animportantmechanismofregulatingthenociceptiveinputtothebrainismediatedby a system of brain stem nuclei, exerting inhibition byneuronal fibers descend-ingtothesegmentsofthespinalcord.This“descendinginhibition”isregulatedbythenociceptiveinfluxandisnormallyexertedbyopiate-mediatedinhibitionofthetransmissionneuronsinthespinalcord.Theaxonsofthesenociceptivedorsalhorncells connectvia the anterolateral tract of the spinal cord (tractus anterolateralis)withtheformatioreticularisinthebrainstem(tractusspinoreticularis)andwiththethalamus(tractusspinothalamicus).Collateralfibersfromtheanterolateraltractgototheperiaquaeductalgray(PAG,substantiagrisea)toinitiatetheinhibition.ThereareconnectionsfromthePAGtothelocuscoeruleus,anotherbrainstemnucleus.ItoperatessynergisticallywiththePAGintheopiodergicmodulationofpain(Bodnar,Paul,&Pasternak,1991).Nucleusraphemagnus(NRM)isthethirdbrainstemareainvolvedinpaininhibition(Basbaum,Marley,O’Keefe,&Clanton,1977;Abbott,Melzack,&Leber,1982).Fromthisarea,inhibitoryfibersdescendwithinthedor-solateralfuniculus(DLF)tothenociceptiveneuronsinthedorsalhornofthespinalcordwheretheyinhibittheafferentinflux(Watkins&Mayer,1982).

12.2.3 Brain areas involved in nociception and pain

InCraig’sviewofthehomeostaticsmallfibersystem,thereisaspecificareainthebrainrepresentingbodyperception—theinsulacortexinthedepthofthesylvianfis-sure—however,thisisnotanexclusivecenterofpainrepresentation.Modernimag-ingmethodssuchasmagneticresonanceimagingorpositron-emission-tomographyhave shown that there is no “pain center” in the brain, but a “neuromatrix” offunctional brain areas subserving the various components of the pain experience(Derbyshire,2000;Treede,Kenshalo,Gracely,&Jones,1999).

Theprimary somatosensory cortex (areaSI)isthepartofthebraininthecentralareawheremostoftheafferentinformationfromtheventralposterior-lateralnucleiofthethalamus(VPL)reachesthecortex(Millan,1999;Apkarian,1995,1996).Thisbrainareaisassumedtomediatethesensory-discriminativeaspectsofpainsuchaslocalizationofastimulusorestimatingitsmagnitude.Thesecondary somatosensory cortex (areaSII)receivesparallelnociceptiveinputmainlyfromthemedialnuclei

Y116134.indb 260 10/10/11 2:12 PM


ofthethalamus(ventralposteriorinferior,VPI;ventralmedialposterior,VMPo)asmallerpartfromVPL.TheSIIareaissupposedtobeinvolvedinpain-relatedatten-tionandlearning(Treedeetal.,1999).Theinsula cortexisreciprocallyconnectedwithSII(thereforesometimescalledareaSIII),butreceivesalsoafferentsfromtheposteriorthalamicnucleiandprojectstotheamygdalaandtotheperirhinalcortex.Thereforetheinsulaissupposedtobeahigher-orderintegrationareaforsomatosen-sation,whichalsomodulatesaffectiveandmotorpainresponses(Derbyshire,Jones,Gyulai,Clark,Townsend,andFirestone,1997).

Theanterior cingulate cortex (ACC)influencesvegetativefunctionsandalsopsy-chomotoricundlocomotoricdrive(Treedeetal.,1999;Treede,Apkarian,Bromm,Greenspan,&Lenz,2000).TheACCincludesanactivationsystemwhichislocatedinthemoreposteriorpartsoftheACCandissupposedtobespecificforpain.TheanteriorpartsoftheACCmediategeneralactivationandattention.

Theamygdalaiswellknowntobeinvolvedinaffective-motivationalprocessingofinformationingeneralandespeciallyinmediatingfearandaversivecondition-ing.Theextendedconnectivitybetweentheamygdala,ACC,insula,andprefrontalcortexprovides theneurophysiologicalfoundationfor the integrationofpainwithfear,contextinformation,andhighercognitiveandmotivationalfunctionsmediatedbytheprefrontalareas.

12.2.4 Nociception and perceptual qualities

Somequalitiesofpainperceptionaredependentonspecificpropertiesofthenoci-ceptivestructuresinvolved,suchasthetypeofactivenociceptor,thetypeofafferentfiber,andtheorgansystemaffected(Willis,1988).Thedifferentnerveconductionvelocities in nociceptive Aδ and C-fibers are assumed to determine two specificqualitiesofpain:First pain isawell-localizedprickingpainqualitywhichisduetoactivationofmyelinatedfastconducting(15–31m/s)andfastadaptingAδ-fibers.Second pain ismediatedbyslowC-fibers(0.8m/s),experiencedasaslowlywaxingandwaningpainqualitythathasapronouncedburningcomponentinskinareas,andadull,throbbingqualityinmusculature,skeleton,orintestines(Table 12.1).

The physiological processes underlying the perception of pain have been fur-therspecifiedbycomparativestudiesofhumansandanimals.Inthesestudies,thesubjectivepainjudgmentsinhumanshavebeencomparedwiththedischargefre-quenciesofCMH-fibers,measureddirectlyby smallneedleelectrodesapplied tothenervefiber in animals.This technique is termed “microneurography.” In thisway,thesubjectivepainintensityhasbeenshowntobeconnectedwiththeCMH-fiber activity at comparable physical intensities. This finding is corroborated byselective blockade of AMH-fibers and CMH-fibers during thermal stimulation inhumans:heatpainperceptionmainlydependsontheactivationofCMH-fibers,atleastatstimulusintensitiesnearthepainthreshold(Rajaetal.,1990).Furthermore,theperceivedpainintensityandthedischargefrequencyinCMHsshowaconcur-rentincreasewithstimulusintensity,showingthatacodingofintensityisalsoper-formedinthesefibers(Zimmermann&Handwerker,1984;LaMotte&Campbell,1978a,1978b;Rajaet al.,1990).Thesefindingsdemonstrate thatpainperception

Y116134.indb 261 10/10/11 2:12 PM


isfunctionallyconnectedwithresponsethresholdsandresponsecharacteristicsofC-fibernociceptors(LaMotte&Campbell,1978a,1978b;vanHees&Gybels,1981;Rajaetal.,1990).

However, receptor excitation and subjective pain perception cannot be simplyequated,andthedeviationsinthisrelationshiparenotyetfullyclarified.Forinstance,a low-level excitationofCMH-fibers isnotyetperceivedaspainful (vanHees&Gybels,1981;Rajaetal.,1990).Therefore,thenociceptivereceptorthresholdisnotthesameasthesubjectivepainthreshold.Furthermore,thetimecourseofCMH-fiberactivationdoesnotcorrespondtotheperceivedtimecourseofpaininrepeti-tiveortonicstimulationmodels.ThedischargefrequencyoftheCMHsdecreasesslowly during tonic contact heat; however, pain perception during the same timeintervalincreases(LaMotte,Thalhammer,&Robinson,1983)orremainsconstant(Meyer&Campbell,1981).Thesameisvalidfortonicmechanicalstimulation(Rajaetal.,1990).Theseinconsistenciesbetweenperceivedpainandnociceptiveactivityrequiretheassumptionofneuronalmechanismsastemporalandspatialsummationof afferent signals to explain the constituents of the pain experience (Raja et al.,1990;LaMotteetal.,1982).

12.3 Dynamic processes in nociceptive sensory networks

Duringthelasttwodecades,importantinsightintothephysiologicalandpathologi-calprocessesofthenociceptivesystemhasbeengained.Thishasbeenachievedbydiversemethods,rangingfromsinglecellandnervefiberrecordingstofunctionalneuroimaging.However,acompletepictureofalltheinterwovenprocessesisstillnotinsight.Particularly,theconnectionsbetweenthemicroscopicprocessesonacellularlevelandthemacroscopicprocessesinmediumandlarge-scaleassembliesofneuronsarewidelyunexamined,nottospeakofanintegrationofpsycho(physio)logical aspects. In the following, mechanisms mediating dynamic change in thenociceptivesensorynetworksarediscussedundertheheadingofneuronalplastic-ity, and dynamic change in perception is discussed under the aspect of learning.These mechanisms demonstrate the far-reaching capabilities of the nociceptivesystemtoreorganizeitself,orinotherwords,toadaptcontinuouslytoalteredgen-eralconditions.Normally,theseprocessesofdynamicchangeareadaptiveandhavesomebenefitfortheorganism,forexample,whenafunctionaldeficitcausedbytheimpairmentofonespecificbrainareaisbeingcompensatedbycorticalreorganiza-tion(Flor,2003).Butthevariousmechanismsofdynamicchangemayalsoleadtomaladjustmentandadysfunctionalstate,asinchronicpainsyndromes.

12.3.1 Neuronal mechanisms of dynamic change

Inperipheralandspinalnociception,certainstimulusconfigurationsoractivationpatterns in the afferent nerve may cause short-lived and reversible changes andmodulations,butalsolong-termandirreversiblemodificationsoftheneuronaltrans-missionpathways(seefigureinsetinTable 12.2,representingtheperipheral,spinal,and cerebral levels of the nociceptive pathways). The underlying neurobiological

Y116134.indb 262 10/10/11 2:12 PM


processeshavebeentermed“neuronalplasticity,”whichcanbeclassifiedintothreecategories,accordingtotheirtimecourseandpotentialreversibility(Woolf&Salter,2000;seecolumnsinTable 12.2).

1. Activation-dependent plasticity refers to transient dynamic changes innociceptiveprocessing,being fully reversiblewithinsecondsorminutes.Such mechanisms include both facilitatory and inhibitory processes onalllevelsofnociception.Intheperiphery,thenociceptorexcitabilitymayincrease(autosensitization)ordecrease(fatigue)duringrepetitivestimula-tionofthesameskinarea.Skindamagebyrepetitivenoxiousstimuliorbyburningmaytriggerinflammatoryprocesses,increasingthesensitivityofthenociceptor(heterosensitization)bycontactwiththealgeticsubstances

Table 12.2 Taxonomyofmechanismsmediatingdynamicchangeinnociceptionandpainperceptiona

Activation Modulation Modification

Painperception

Perceptualsensitization

Hyperalgesiab Hyperalgesiab

Allodyniab Allodyniab

Habituation

Brainneuromatrixc

Automaticprocesses

Associativelearning

Associativelearning

Corticalreorganization

Corticalreorganization

Descendinginhibition



Spinalnociceptionc

Wind-upindorsalhornneurons

Centralsensitization

Structuralchangesinspinaltransmission

Facilitation Long-termpotentiation

Long-termdepression

Peripheralnociceptionc

Autosensitizationofnociceptors

Heterosensitizationofnociceptors

Structuralchangesinnociceptors

Fatigueofnociceptors

Fatigueofnociceptors

a ExtendedversionofatablebasedonWoolf&Salter(2000).Facilitativeandinhibitingmechanisms,thelattergiveninshadedlayout.

b Ahypersensitivitytopainfulstimulationiscalledhyperalgesia,whereasahypersensitvitytonormallynonpainfulstimuliiscalledallodynia.

c Threemainlevelsofnociceptivepathways:peripheralcutaneousnociceptorfibersenteringthedorsalhornofthespinalcordandascendingviatheanterolateraltracttothethalamusinthebrain.Fromthereon,aneuromatrixofvariousbrainareasreceivenociceptiveinflux.

Y116134.indb 263 10/10/11 2:12 PM


mediatinginflammation(e.g.,bradykinine,prostaglandine,andothers).Anongoingafferentinputfromaperipheralnociceptormightalsotriggersen-sitizationindorsalhorntransmissionneurons,whichiswellknownfromanimal experiments as the so-called “wind-up” phenomenon (Mendell,1966).Comparativestudiesinhumansandanimalshaveshownthatwind-up is the neuronal correlate of the subjective short-term sensitizationexperiencedunder repetitiveor tonicstimulationconditions (Price,Mao,Frenk,&Mayer,1994;seealsoSection12.2.2).

2. Modulationsaretermeddynamicchangesinnociceptionlastingformin-utes,hours,orevendays,whichmaystillbefullyreversible.Again,thereare facilitating and inhibiting processes at work. Long-term potentiation(LTP)isthestrengtheningofthesynapticconnectivitybetweentwoneu-ronsbyapplyingahigh-frequencystimuluspatterntoonespinalneuroninanimalexperimentsorbycorrespondingelectricalstimulationoftheskininhumanexperiments(Klein,Magerl,Hopf,Sandkühler,&Treede,2004).By changing the stimulus properties to low-frequency stimulation, theconditionscanbereversed:along-termdepression(LTD)inhibitssynap-ticconnectivity,whichmightbeoftherapeuticvaluebymodulatingspinaltransmissionbytranscutaneouselectricnervestimulation(TENS).

3. Modification, finally, includes non- or only slowly reversible changes innociception, caused by altered gene regulation in the cell, which have along-lasting influence on the excitability and connectivity of a neuron.These mechanisms are assumed to be central in establishing persistentpathologicalpain.

Theneurophysiologicalprocessesunderlyingneuronalplasticityinnociceptionarewellexaminedinanimalmodels,usuallycomprisingthesamemolecularprocessesasknownfromtheneurobiologyofmemoryformationinthebrain.TheN-methyl-d-aspartate(NMDA)receptormechanismunderlyingthewind-upphenomenon,ortheLTP,isaprominentexampleoftheseprocesses(e.g.,Kandel,2001).Thisoriginoftheplasticity-in-painresearchhasalsoledtotheterm“painmemories”forthesemechanisms(Sandkühler,2000;Kleinböhl,Baus,Hornberger,&Hölzl,2005).

12.3.2 Psychological mechanisms of dynamic change

Inthetaxonomyoflearningmechanismsandrelatedmemorysystems,thesimplestmechanisms are the so-called automatic processes, represented in perception byhabituationandperceptualsensitization(Milner,Squire,&Kandel,1998).

12.3.2.1 Automatic processes: Perceptual sensitization

Sensitization in perception is defined as an increase of the subjective stimulusintensityduringconstantrepetitiveorcontinuousstimulation.Inbodyperception,sensitization is a characteristic feature of the pain sensation (Greene & Hardy,1962;LaMotte,1979). Infact,sensitizationseemssoclosely linkedwith thepain

Y116134.indb 264 10/10/11 2:12 PM


experience thatattemptsweremade touse theoccurrenceof sensitizationduringtonicstimulationasanobjectivecriterionforthepainthreshold(Severin,Lehmann,&Strian,1985).Perceptualsensitizationisusuallyexaminedwithrepetitivephasicorcontinuoustonicstimulationwiththermal,mechanical,orchemicalstimulationoftheskinorthemusculature.Insuchstimulusmodels,areversibleshorttimesensiti-zationinthetimeintervalbelowoneminuteisprominent,whichcanbemeasuredagainafterashortbreak(Kleinböhl,Hölzl,Möltner,Rommel,Weber,&Osswald,1999;Kleinböhl,Baus,Hornberger,&Hölzl,R.,2005).

Inaccordancewithitsassumedneuralcausethiseffecthasalsobeentermedtem-poral summation.Theunderlyingneuralmechanismhasbeenidentifiedintheabove-mentionedwind-upphenomenonalreadyknownfromanimalexperiments.Thereisanactivation-dependentincreaseofsensitivityofclass2orwidedynamicrangeneu-ronsinthedorsalhornofthespinalcord,whichisfoundwithrepetitivenociceptivestimulationatfrequenciesfasterthan0.3Hz(Mendell,1966).ContinuousexcitationoftheseneuronsbynociceptiveC-fiberafferentsremovestheblockofanionchan-nelwithareceptorsensitiveforNMDA.Thisallowsforaninfluxofcalciumionsintothecell,thusfurtherdepolarizingthecellmembraneandleadingtoanacceler-atedreleaseofactionpotentials,theso-calledwind-up.ThisneuralprocessanditsmodulationorblockadebyNMDAreceptorantagonists (e.g.,MK801,ketamine,memantine,amantadine)arewellexaminedinanimalexperiments(e.g.,Woolf&Thompson,1991;Ren,1994).Psychophysicalexperiments inhumanshaveshownthatsubjectivesensitizationmightbeatleastinpartaperceptualcorrelateofthisspinalmechanism(Priceetal.,1994).

12.3.2.2 Automatic processes: Habituation

Habituation isdefinedasadecreaseof the subjective intensityexperienceduringrepetitiveorcontinuousstimulationatconstantintensities.Thushabituationcanberegardedasaperceptualprocesscomplementarytosensitization,althoughtheunder-lyingneuronalprocessesaresupposedtobeverydifferent.Intheclassicalinterpre-tationofhabituation,asequentialcomparisonprocessbetweenastoredimageofastimulusand its repetitivepresentation isassumed.The resultof thiscomparisonprocessdeterminestheresponsetotheactualstimulus:anagreementofstoredandactualstimulusimageleadstoadecreaseinresponseintensity(Sokolov,1960).Theresponsetonewstimulation(orientingresponse)andhabituationarewellexaminedfornearlyallsensemodalities(e.g.,Lang,Davis,&Öhman,2000).Forpainpercep-tion,thereareonlyalimitednumberofstudiesavailable,probablybecausesensitiza-tionnormallydominatesinpainperception.

12.3.2.3 Associative learning: Operant conditioning

Inthetaxonomyofhigher-orderlearningmechanismsandrelatedmemorysystems,operant and classical conditioning are termed associative learning mechanisms(Milneretal.,1998).SincetheworkofFordyceandothers,operantconditioningisconsideredanimportantpsychologicalmechanismforthedevelopmentandalso

Y116134.indb 265 10/10/11 2:12 PM


for the treatmentofchronicpain(Fordyce,1984;Philips,1987;Philips&Grant,1991). Although this is common knowledge in pain research, studies on operantconditioning of pain are scarce and mostly limited to modifying the observablepainbehavior.

Operantreinforcementofpainhasbeenexaminedsofarparticularlywithexter-nalsocialreinforcement,whererewardandpunishmentwereperformedbymeansofverbalcommentsonpainbehavior(extrinsicreinforcement).Anexampleistheverbalcommentontheparticipant’sratingofpainintensityonaratingscale,suchas“verywelldone”(reinforcement)or“badlyperformed”(punishment).Theresultingdynamicchangehasbeenoperationalizedbyexplicitjudgmentsofperceivedpainintensities, which implies the possibility of various distortions in quantifying thejudgment.Usually,withthistypeofstudy,itcannotbedecidedwhetherpainpercep-tionoronlythelinguisticjudgmentcriterionhasbeenmodifiedbyoperantlearning.ThisissuehasbeenresolvedbytheoperantprocedurepresentedinSection12.4.3.

Severalothermechanismsofdynamicchangeinnociceptionandpainexist,butwe limit the presentation to those for which appropriate validated procedures ofassessmentareavailable,whicharepresentedinthenextsection.

12.4 Measurement of dynamic processes in pain

A thorough examination of the functions of the nociceptive system in humansrequiresaversatile toolboxofpsychophysicalandpsychophysiologicalmeasuringprocedures.Specificstimuluspatternshavetobedefinedasprobes,totapselectedmechanismsofnociceptionandpainonvariouslevelsofprocessing.Thisexperi-mentalpainstimulationprovidesapowerfulframeworktoshedlightonthemulti-dimensionalfunctionalityandinteractionofthesemechanisms.Farfromprovidingacompletecompendiumofpsychophysicalpainassessment,thissectionratherfocusesonaselectionofthreepsychophysicalprocedures,whichcapturedynamicpropertiesofoneormoreofthepreviouslydescribedneuronalmechanismsandlearning.

12.4.1 Assessment of pain dynamics during repetitive phasic stimulation

Measuringsensitizationhasalongtraditioninanimalresearchonpain,withMendell’sdiscoveryoftheneuronalwind-upphenomenonbeingamajormilestone(Mendell,1966).However,psychometricevaluationsoftimecourse–intensityinteractionsinhumanshaveonlybeenaddressedoccasionallyinthelasttwodecades(e.g.,Arendt-Nielsen,Andersen,&Jensen,1996;Nielsen&Arendt-Nielsen,1998).Theclassicalapproachinmeasuringperceptualsensitizationinhumansmakesuseofshortstimu-luspulsesthatarerepetitivelypresented.Infraredlasersemittingthermalradiationarehighlyusefulforthiskindofstimulusmodelbecausetheyprovidestimulusdura-tionsinthemillisecondrangeandselectivelyactivatenociceptivefibers(AMH-IIinhairyskin;Bromm&Treede,1983;1984).Therelatedexperimentalmodelsbasedonthistechniquehavebeentermedtemporal-summation models,representingaprom-isingapproachtoassessdynamicprocessesinthenociceptivesystem.Assessment

Y116134.indb 266 10/10/11 2:12 PM


oftemporalsummationfocusesontheperceptualsensitizationexperiencedduringrepetitivestimulationwithconstantintensity.Thisstrategyleavesasidevariousothermechanismsofdynamicchange,whichareassumedtooverlaytheperceptualmea-suresofhabituationandsensitizationtovaryingdegrees.Therefore,amodifiedtem-poralsummationprocedurehasbeenconceivedwhichcouldcapturethisoverlayofseveraldynamicnociceptiveprocessesworkingconcurrentlytoproducealteredpainperception(Kleinböhl,Trojan,Konrad,&Hölzl,2006).

Inthenewpsychophysicalprocedure,repetitiveheatpulsesof100msdurationwereappliedtotheskinbyaPC-controlledCO2-lasersystem.Thelaserbeamwasprojectedtotheradialnerveareaofthelefthandbyascanningdevicethatallowedautomaticchangeofstimulationareasontheskin.Thischangingofstimulussiteisusuallyperformedtoavoidlocalsensitization(auto-andheterosensitization;c.f.Section 12.3.1 and Table 12.2). A repetitive series of heat pulses is called a trial(Figure 12.2).Participantsestimatedtheperceivedmagnitudeofafirstsinglepulseonascale.Thisjudgmentservedasareferencepointfortheperceivedchangedur-ingthesubsequentrepetitiveseriesoftenpulses.Theserieswasappliedatthesameintensityastheinitialreferencepulseandatoneofthreefrequenciesofrepetition(Figure 12.2).Participantswereinstructedtofocustheirjudgmentonagradualtem-peraturechangeexperiencedbetweenthefirststimulusandthelastoftherepetitivestimulationtrial.

Psychophysicalscalingwasperformedwithvisualanalogscales(VAS)presentedon a computer screen. Participants were asked for four perceptual judgments: anintensityratingofthefirstandthelastsinglelaserpulseoftheseries,anestimateof theperceived intensity changeduring the repetitivepulse series, andfinally, aratingof theaversivenessof thepulseseries.Twomeasuresofperceptualchangewerederivedfromthesejudgments:first,thedifferencebetweentheratingsofthefirstandthelastlaserpulse,reflectingdynamicchangeinfirstpainperception,andsecond,thesubjectiveratingofthedynamicchangeexperiencedduringrepetitivestimulation,ratherconnectedtoslowsecondpain.

Theresultsinhealthyparticipantsshowedthatshort-termsensitizationoccurredatstimulusfrequenciesabove0.3Hz,thetypicaledgefrequencyforspinalwind-uptooccur(Figure 12.3).Moreover,stimulusfrequencyhadanoverallinfluenceonbothmeasuresofperceiveddynamicchange:Thechangeofsensorymagnitude(VAS)andtheperceivedtemperaturechangeΔSbothincreasedwithstimulusfrequencyatall intensitiesused. Incontrast,stimulus intensityeffectsweredifferentfor themeasuresofperceptualchange:thechangeofsensorymagnitude(VAS)dependedonintensityonlyatthehighestfrequency(1.8Hz).Forthedirectratingofperceivedchange(theΔSresponse,“S”standsfor“sensation”),stimulusintensityhadasignifi-canteffect,showinganincreaseofperceptualsensitizationwithincreasingintensityatallrepetitionrates(Figure 12.3).

Thecontrolledstimulationprotocolfurtherallowedassessingtheeffectsofmul-tiplestimulationsofthesameskinareaduringmedium-termtimespans,showinghabituationforthesespecificstimuli.Overtheblocksoftrials,eachonerelatedwithachangeinstimulatedskinarea,nolong-termperceptualchangesoccurred.

Y116134.indb 267 10/10/11 2:12 PM


403020100–10 50 60 70 80 90 (s)

VAS

(A)

∆SVAS

(S2)

VAS

(S1)

Trial

Run

Run 1

531

Run 2

Run 3

Run 4

Run 5

Session

Painful

Pain threshold

NonpainfulTrial:

f (Hz):

f = 0.2/0.6/1.8 Hz

10.6

0 2 4 6 8 10 12 14 16(min)

0 10 20 30 40 50 60 70 80 90(min)

20.2

31.8

40.6

51.8

60.6

71.8

80.2

90.6

Figure 12.2 Psychophysical phasicpainprocedurewith radiant heat stimulusprotocol andpsychophysicalscaling.Onesessionconsistsof5runs,including9trialsperrunand11singlelaserpulsespertrial.Trial:Aseriesof10laserpulsesatoneofthreefrequenciesf(0.2,0.6,or1.8Hz)wasappliedafteraninitialreferencepulse.Withineachtrial,asetofpsychophysicalratingswasobtained:absolutemagnitudeestimationof the initial referencepulse (VAS(S1))andthelastpulse(VAS(S2))oftherepetitiveseries;directestimationofperceivedtemperaturechangeduringrepetitivestimulation(ΔS),andamagnitudeestimateoftheoverall“aversiveness”oftheentiretrial(VAS(A)).Perceptualchangeduringrepetitivestimulationwasassessedfirst,bythedifferenceofstimulusmagnitudeΔVAS=VAS(S2)–VAS(S1),andsecond,bythemea-sureofperceivedtemperaturechangeΔS.Run:Withinafixedpseudorandomsequenceof9tri-als,eachofthe3frequency×3intensityconditionsofthedesignwasappliedonce.Session:Fiverunswereappliedduringoneexperimentalsession.Thestimulationpatchintheleft-handradialnerveareaconsistedof5successivelystimulatedspots,whichwereadditionallymovedlaterallywiththestartofeachrun(startpositions1–3–5intheinset).Alllocationswerepositionedonaregulargrid(1cm).Perceptualmeasureswereanalyzedpertrial(short-termeffects;<1min),perrun(medium-termeffects;1–15min),andpersession(long-termeffects;15-90min).(FigurefromKleinböhletal.,2006:Clinical Neurophysiology 117, p.122,fig.1;ElsevierIrelandLtd.)

Y116134.indb 268 10/10/11 2:12 PM


1.2

1.0PainfulPain thresholdNonpainful

Sensitizationadaptation

0.8

0.6

0.4

Diff

eren

ce R

atin

g ∆S

(°C)

0.2

0.0

–0.2

0.2 0.6Frequency (Hz)

1.8

–0.4

***** ***

60

50PainfulPain thresholdNonpainful

Sensitizationadaptation

40

30

20

Mag

nitu

de D

iffer

ence

∆VA

S (a

.u.)

10

0

–10

0.2 0.6Frequency (Hz)

1.8

–20

***** ***

(a) (b)

100 PainfulPain thresholdNonpainful80

60

40

Sens

ory M

agni

tude

VA

S (S

1) (a

.u.)

20

1 2 3 4 5 6 7 8

Trial Number

9

0

100

80

60

Sens

ory M

agni

tude

VA

S (S

1) (a

.u.)

40

20

1 2 3 4 5Run Number

0

********

PainfulPainful thresholdNonpainful

(c) (d)

********

Figure 12.3 Repetitive phasic pain stimulation and the overlay of dynamic mechanismsof perceptual change. Sensitization and habituation occur concurrently in this procedure,dependingonthetimeintervalobserved,thefrequencyapplied,theintensity,skinarea,andtherepetitivestimulationofthesameskinpatch.(a)DifferenceofmagnitudeestimatesΔVAS(mean ± SD), related to first pain sensations of laser pulses, over stimulus frequency fornon-painfulintensity(),painthresholdintensity(⦁),andpainfulintensity().Perceptualsensitizationincreaseswithrepetitionrateabove0.2Hz.(b)PerceiveddifferenceratingoftemperaturechangeΔS(mean±SD)relatedtosecondpainsensationsduringrepetitivestim-ulation,overstimulusfrequency.Secondpain-relatedperceptualsensitizationincreaseswithfrequencyinthesamedose-dependentmannerastheindexoffirstpain-relatedsensitization,butanadditional intensityeffect isseen(p< .05).Datacollapsedover2sessions,N=10participants,90trialseach.(c)Medium-termhabituationofsensorymagnitudeestimatesofthereferencepulseataconstantstimuluslocationoverthe9pseudorandomizedtrialswithinarun(trialwithinrunmaineffect:F=17.8;p<.05adjusted).Dottedlines:exponentialfitofthehabituationcurveforthetwoupperintensityclasses.(d)Long-termeffectsofsensorymagnitudeestimatesofthereferencepulseover5runsduringasession(runwithinsessionmaineffect:F=1.41;p>.10n.s.).Thebracketsin(c)and(d)indicatetheadjustedsignifi-canceofcontrastsovertrials(nonpainversuspainthreshold:F=31.0;p<.01;painthresholdvs.pain:F=55.9;p<.001;nonpainvs.pain:F=59.6;p<.001).(FiguresfromKleinböhletal.,2006:Clinical Neurophysiology 117,p.124,figs.1,2;ElsevierIrelandLtd.)

Y116134.indb 269 10/10/11 2:12 PM


Theprocedurepermitstheconcurrentassessmentofshort-,medium-,andlong-term dynamic processes of sensitization and habituation in pain processing bydissecting these mechanisms with a sophisticated stimulus protocol of repetitivestimulation(Figure 12.3).Themainunderlyingneuronalmechanismsareassumedtoberelatedtospinalwind-upandcutaneousnociceptivefiberfatigue,respectively.Themethod is therefore suitable forquantitative sensory testingofdynamicpainprocessingoverdifferenttimespans,whichisrelevantinclinicaltestingofpainandindrugassessment.

12.4.2 Assessment of pain dynamics during tonic stimulation

Tonicstimulationisaconstantstimulusintheminutesorsubminuterange,duringwhichdynamicchangesinpainprocessingcanbeobserved.Severalexperimentalpainmeasurementprocedureswithtonicstimulationhavebeendeveloped,butmostofthemarelimitedintheirapplications,becausethecontrolofthestimuluscharac-teristicsisrathercrude.Examplesforthosetonicpainproceduresarethe“submaxi-malefforttourniquettest”andthe“coldpressortest”(Handwerker&Kobal,1993).The tourniquet testmakes themusculatureof a cuffedarmworkunder ischemicconditions,leadingtoadeepachingpain.Thecoldpressorachievessimilarpainbyplacingthearmincoldwaternearthefreezingpoint(approx.4°C).Bothmethodsassess the timeuntilparticipantscannotbear thepainanymore,asa thresholdofpaintolerance.Thesemethodscannoteasilyberepeatedandbycapturingonlyonethresholdparameterintheintensitycontinuumofpain,nodynamicprocessescanbeassessed.Variantsusingcontinuousratingsduringthesetonicstimulationmodesproducehighlyvariableestimates andaredifficult to interpret (e.g.,Davis,Pope,Crawley,&Mikulis,2004).

Based on an earlier method for the measurement of subjective sensitization(Severinetal.,1985),anewtonicheatpaintest,the“dualsensitizationprocedure”(Kleinböhletal.,1999)wasdeveloped(Figure 12.4).Theprocedureincorporatesanassessmentofpainthreshold,andanassessmentofastimulus-responsefunctionfortonicstimuliaswellasatwofoldmeasureofpainsensitizationorhabituation.

Tonic heat stimuli are applied with a contact heat thermode to the skin, forinstance,tothethenarofthehand.Thestimulatorisasmallmetalplatewithapre-ciselyadjustabletemperature,controlledbyaPCsystem.Eachheatstimulusstartsfrombaseline(40°C)toapresettemperature,typicallyoneofseveralvaluesaboveandbelowthepreviouslyassessedpainthreshold.Duringa“referenceinterval”of5sparticipantsareinstructedtomemorizethesensationontheirhand.Duringthenext25sofconstanttonicstimulation,participantswilltypicallyperceivechangesintemperature,anincreaseindicatingsensitizationandadecreaseindicatinghabitu-ation.Attheendofthetonicstimulationtheperceiveddynamicchangeisassessedinatwofoldway.

First,adirectandovertscalingoftheperceivedtemperaturechangeisperformed;participantsareaskedtoratetheamountofchange(in1/10°C)after30sofconstantstimulationbycomparingthepresentsensationwiththesensationinthereferenceinterval. A negative perceived temperature change indicates habituation, whereas

Y116134.indb 270 10/10/11 2:12 PM


positive ratings indicate sensitization (ΔS response, “S” again stands for “sensa-tion”).Second,anindirectandcovertscalingoftheperceivedtemperaturechangeisacquired.Thetaskoftheparticipantsistoactivelyreadjustthethermodetempera-turetoreproducetheirinitialsensationduringthereferenceinterval.Themeasureofperceivedchangeisthencomputedasthedifferenceofend-temperatureT(end)minusinitialtemperatureT(init),negativevaluesindicatingsensitizationandposi-tivevalueshabituation(ΔTresponse,whereTstandsfor“temperature”).Thelattertechnique,termed“behavioraldiscrimination,”avoidsthecommonresponsebiasofsubjectiveratingsbyinferringtheperceptionofparticipantscovertlybytheirbehav-ior(Kleinböhletal.,1999).

Repeatingthisprocedureatseveralintensitiesbelowandabovesubjectivepainthreshold allows the assessment of individual response profiles for sensitizationandhabituation in thedomainofovert subjective evaluationofperceived change(ΔS),aswellasforitscounterpartofbehavioraldiscrimination(ΔT).Thismeasure-mentofdynamicchangeinpainperceptionisabletocaptureprofounddifferencesbetweenhealthyparticipantsandchronicpainpatientswithmusculoskeletalpain(Figure 12.5).Healthyparticipantsarecharacterizedbyhabituationinthefirstplace,whereaspatients display a strong sensitization (Figure 12.5).Both subjective and

50

48

46

44

Tem

pera

ture

(°C

)

42

40

–10 0 10

5s 20s25s

20 30Time (sec)

Max.

T (init)

∆S response

T (end)∆T < 0

Base

line

∆T re

spon

se

Toni

c stim

ulat

ion

Refe

renc

e int

erva

l

30s

40 50 60 70

Figure 12.4 Psychophysical“dualsensitization”procedurefortheassessmentofperceptualchangesinhumans(Kleinböhletal.,1999).Inthisprocedure,thechangeinperceivedinten-sityisassessedduringconstantcontactheatstimuliof30sduration.Asubjectivemeasureofperceived changeΔS is assessedon amagnitude scale (explicit judgment).The secondmeasureofperceivedchange(ΔT)isacquiredbymethodofproduction,whereparticipantsreproduce the same temperature sensationasperceivedduring the reference interval.Thedifferencebetweenthetwotemperaturesisthesecondmeasureofperceivedchange(implicitmeasure).

Y116134.indb 271 10/10/11 2:12 PM


behavioralmeasuresdifferentiatepatientswithnormalphasicpainthresholdsfromhealthypersons.Inparticular, inpatientswithspine-relatedmusculoskeletalpain,enhancedsensitizationappearedalreadywellbelowpain thresholdand increasedfurtherwithstimulusintensity(Figure 12.5).

Theseresultsshowthatchronicmusculoskeletalpainisnotonlycharacterizedbyanincreasedsensitivitytowardpainfulandnonpainfulstimuli,butalsobyalteredcharacteristicsofinformationprocessinginthenociceptivesystem.Thetemperaturewheresensitizationiselicitedisgenerallyshiftedtolowervalues,andtheincreaseofsensitizationwithhighertemperaturesismorepronounced.Thisenhancedsensi-tizationinchronicpainmustreflectanoverlayofdifferentmechanismsofdynamicchangeinpainandnociception.Attheleast,theshorttimedynamicsmediatedbyspinal wind-up seems to be present in the data, obviously amplified in patientsbyothermechanismsonthespinalorevencerebrallevel.

12.4.3 Assessment of pain dynamics mediated by operant learning

Assessingoperant learningmechanisms, thethirdclassofdynamicchangemecha-nismsinnociceptionandpain,requiresacomplexpsychophysicalprocedure,com-binedwithoperantreinforcement.Inthedevelopmentofsuchanoperantconditioningprocedure,astartingpointhasbeenthepathogeneticmarkerofenhancedsensitiza-tion,whichhasbeenfoundmainlyinpatientswithmusculoskeletalpain(Kleinböhletal.,1999).Frompreviouspharmacologicalstudiesithadbecomeclearthatspinal

–2 Unspecificback pain

–1Se

nsiti

zatio

n

0

–1 0Stimulus Intensity

1

Healthy controls

Figure 12.5 Sensitizationtotonicheatasafunctionofrelativetemperatureinchronicbackpainpatientsandcontrols.Meantemperaturegradientsofthebehavioralsensitizationmea-surewithlinearregressions(thicklines)and95%confidenceintervals(thinlines).Symbolsindicatemeanvaluesoverrelativestimulustemperatureforthegroups(–1°C<<painthresh-old=0<<+1°C).Behavioral sensitizationdifferentiatesbackpainpatients fromhealthycontrols,withpatientsshowingenhancedsensitizationalreadyatnonpainfultemperatures.

Y116134.indb 272 10/10/11 2:12 PM


wind-up cannot be the exclusive cause of enhanced pain sensitization (see Section12.3.1onneuronalmechanismsofpain).Amodificationofperceptualsensitizationduringtonicheatpainbyanoperantlearningprocedurewouldbestrongprooffortherelevanceofsuchlearningmechanismsfortheemergenceofpainhyperalgesiaandforamodulationofpainperceptionleadingtochronicpain.Toshowthisexperimentally,the“dualsensitization”proceduredescribedpreviouslyhasbeenmodifiedinseveralways(Figure12.6).

1.Thepathogeneticmarker,enhancedsensitizationmeasuredbybehavioraldiscrimination,hasbeenchosenasthebehaviortobemodifiedbyitscon-sequences,determinedbyanoperantscheduleofreinforcement.

2.Sensitizationhasbeenmeasuredinatwofoldway,asinthedualsensitizationprocedure,byanovertsubjectiveandbyacovertprocedure.Participantswereinstructedtokeepthesensationconstantduringtonicheatstimulationbycontinuousadjustmentofthetemperature.Dynamicchangeduringthisadjustmentphaseisthereforemeasuredindirectlyasthedifferencebetweenstartandendtemperature.

3.Reinforcementhasbeenoperationalizedasintrinsicreinforcement,thatis,within the sensorymodalityunderexamination.A reduction in stimulusintensityorpainservedasnegativereinforcement,andanincreaseofstim-ulus intensityaspunishment.Thiswascombinedwithstandardmethodsofoperantresponseshapingofincreasedsensitizationandhabituation(seeHölzl,Kleinböhl,&Huse,2005,fordetails).

Healthyparticipants(24)wererandomlyassignedtotwogroupstakingpartineitheroperantsensitizationorhabituationlearning.Painfulandnotpainful initialstimulustemperatureswerecomparedinparticipantsintwoseparatesessions,eachincluding 80 trials of thermal stimulation applied with a thermode system to thethenareminence(seeFigure 12.7).

The experimental study demonstrated for the first time an “implicit” operantmodulationofpainperception;bothhabituationandsensitizationlearninghavebeensuccessful.Thesensitizationresponse(ΔT)wasvariedtherebyinonly1–2hoursofoperantconditioninginanorderofmagnituderesemblingtheenhancedsensitizationfoundasapathogeneticmarkerinchronicbackpainpatients.Painperceptionhasbeenchangedbyoperantlearningintwospecificways:first,thespecificlearningconditionsincreasedtheoccurrenceofbehaviorindicatingsensitizationorhabitua-tion.Second,subjectivemagnitudeestimationandactualstimulustemperaturefellapartwithprogressivetrainingduringsensitizationlearning:withstimulustempera-turedecreasing,theperceivedintensityremainedconstant(Figure 12.7).Participantswerenotawareof thecausalitybetween their temperature regulationand thefol-lowingreinforcement; the learning thereforeoccurredwithout theirawareness,orimplicitly.Alltestpersonshadadheredtoprovideexactsensoryjudgments,moti-vatedbythecoverinstructiontoperformaquantitativesensorytesting.

Thestudydemonstratedthatitispossibletoshapeshort-termsensitizationandhabituation to tonic thermo-nociceptivestimuli inhealthyparticipantsby implicit

Y116134.indb 273 10/10/11 2:12 PM


operantreinforcementofdiscriminativebehaviorwithcontingentacutepainreduc-tionandwithoutparticipants’knowledgeofcontingencies.Consequently, implicitlearningofenhancedpainsensitizationmaybeasuitablemodeltoassessoperantplasticityofpainperception,inadditiontobasicsensoryandneuronalmechanismsmediatingdynamicchangeinnociceptionandpain.

12.5 Summary and final considerations

Theassessmentofdynamicmechanismsinnociceptionandpainhasbeenpresentedinthreesections.First,thecomplexneuronalstructuressubservingnociceptionandpainhavebeendescribedonthelevelofperipheralnociception,thelevelofspinalnociception, and, finally, the level of the brain. This first section has been com-plementedby anoverviewof the fewneurophysiological properties that translatedirectly intoperceptualqualities.Second, theprominentmechanismsofdynamicchangeinnociceptionandpainwerepresentedonaneurophysiologicallevel,namelythemechanismsofneuronalplasticity,whicharetriggeredundercertainstimulus

46

PunishmentNo ReinforcementNegative Reinforecement

44

42

40

Tem

pera

ture

(°C

)

38

–10 0 10

6s25s

20 30Time (sec)

Tinit

Tend

T0

∆T = Tend – Tinit

Nex

t tria

l

Rein

forc

emen

t

Perc

eive

d ch

ange

∆S

Hol

d

Cont

inuo

us ad

just

men

t

6s

40 50 60

Figure 12.6 Trialstructureofoperantconditioningprocedure.StimulistartfrombaselinetemperatureT0=37°CuptoapresetinitialstimulustemperatureTinit(0.7°C/s).Participantscontinuouslyadjustthethermodetemperatureundertheinstructiontokeeptemperaturecon-stant.TheadjustedchangeΔTisthedifferenceofendtemperatureTendandstarttemperatureTinit.Reinforcementasaspecified temperaturedecreaseor increase ismadecontingentoncriterionresponses tobeenhancedorweakened(downregulationinsensitizationlearning,upregulationinhabituationlearning).TheperceivedchangeΔSisavisualanalogratingofperceivedtemperaturechangeduringreinforcement(Hölzletal.,2005).(FigurefromHölzletal.,2005,Pain 115,p.14,fig.1;ElsevierIrelandLtd.).

Y116134.indb 274 10/10/11 2:12 PM


conditionsandaffectnociceptionandpainwithindifferenttimespans.Basicmech-anisms of learning were selected as higher-order psychological mechanisms ofdynamicchangeinpain:nonassociativelearningorautomaticprocessesrepresentedbyhabituation and sensitization, and associative learning, represented byoperantconditioning.Third, theassessmentof someof the typicaldynamicpropertiesofnociceptiveprocessinghasbeenexemplifiedbythreepsychophysicalmeasurementproceduresandtheirimplementation.Althoughtherearemanymorepsychophysicalmethodsforassessingvariousaspectsofpainprocessing,thefocushasbeenputonthreeproceduresthatwerespecificallydevelopedformeasuringdynamicchangeinpainperception(Table 12.3).

The procedures introduced here capture several mechanisms mediatingdynamic change in pain to various degrees. The selectivity for circumscribedmechanismsdependson factors suchas thephysical stimulationmodel (phasic,tonic),thespecificproceduralpropertiesandtheoperationalizationoftargeteffectvariables(e.g., ratingscalesforsensitizationorhabituation).Theresultingmea-surementparametershardlyrepresent“pure”propertiesofacertainmechanism.Itmustbeassumedthattheseparameterswillalwaysbeconstitutedbyanover-lay of several mechanisms working in parallel and affecting the perception ofdynamicchange(Table 12.3).Therefore,exactproportionsoftheinfluenceofspe-cificneuronalmechanismsonpaincannoteasilybegiven.However,thedissectionof the specific neuronal and psychological contributions to perceptual phenom-enacanbeimprovedbyvalidatingthepsychophysicalproceduresunderspecific

3 Habituation

Sensitization

1 2 3 4 5Block

6 7 8* *

**

2

1

0

Rela

tive T

empe

ratu

re (°

C)

–1

–2

–3

3 Habituation

Sensitization

1 2 3 4 5Block

6 7 8

**

**

2

1

0

Rel.

Stim

ulus

Mag

nitu

de (a

.u.)

–1

–2

–3 ****

Figure 12.7 Timecourseofstimulustemperaturesinoperantlearningofshort-termsensiti-zationandhabituation.(A)Learningcurvesasblocktoblockmeans(linearregressiontrendwith95%confidenceintervals).Stimulustemperaturesdiffersignificantlyafter50trialsorfiveblocks(*p<.05,**p<.01;lineartrendcontrast:p=.0007**).Implicit(w/oawarenessof contingencies) operant shaping of short-term sensitization or habituation by contingentdecreaseorincreaseofstimulusintensityiseffective(Hölzletal.,2005).(FigurefromHölzletal.,2005,Pain 115,p.16,fig.2b;p.17,fig.3b;ElsevierIrelandLtd.).

Y116134.indb 275 10/10/11 2:12 PM


experimentalconditionsorinspecificsamples.Thisstrategymightbepursuedbypharmacologicalinterventionssuchasblockingthespinalwind-upwithNMDA-antagonists,andalsobyinvestigatingpatientgroupswithchronicpainsyndromes.Themethodologicalprinciplesofmeasuringpaindynamics inhumanare sum-marizedasfollows:

1.Stimulus model:Thespatiotemporalcharacteristicsofexperimentalpainstimulation provide a first selection of the specific nociceptive pathwaystobeaddressed.This isachievedbydefining intensity, timecourse,andstimulationsiteonthebodyappropriately.Mechanismsofdynamicchangeinpaincanbemeasuredwithphasicpainrepetitivestimulationatdifferentfrequenciesorwithconstanttonicstimulationoverdifferenttimespans.

2.Psychophysical scaling: Two types of subjective judgments of perceivedchangeareused.Direct scaling techniques,measuring themagnitudeofperceivedchangebyaskingparticipantstomakeasubjectivejudgmentonascaleareusuallypronetovariouskindsofbias(Poulton,1989).Incontrast,indirectbehavioralmeasurementofperceivedchangecanbeoperational-ized as unbiased no-knowledge procedures, where subjects operate on atask such as keeping apparent temperature constant, giving information

Table 12.3 Assessmentofdynamicchangemechanismsinpainperceptiona

Repetitive stim. (phasic pain)

Dual sensitization (tonic pain)

Operant conditioning (tonic pain)

Painperception

Sensitization Sensitization Sensitization

Habituation Habituation Habituation

Brainneuromatrix

Automatic processes

Automatic processes

Automatic processes

Associative learning




Spinalnociception

Wind-up Wind-upCentralsensitization?

Wind-upCentralsensitization?

(Descending)inhibition



Peripheralnociception




Fatigue of nociceptors

a Psychophysicalproceduresandmechanismsofdynamicchangetheyareassumedtoassessinthefirstplace(indicatedinboldface).Facilitativeandinhibitingmechanismsaregiven(inhibitioninshadedlayout).

Y116134.indb 276 10/10/11 2:12 PM


about perceived change by their temperature regulation behavior duringthetask.

3.Experimental modulation of dynamic change in pain perception can beusedtodissectunderlyingsubprocessesmediatingthechange,forinstance,by applying pharmacological probes, including the topical applicationof capsaicin in peripheral nociception, or the application of NMDA-antagonists (ketamine, amantadine) affecting mainly spinal nociception.Morecomplicatedinterventionsaremadeonhigher-orderprocesses,suchasthemodulationofpainperceptionachievedbyoperantlearning.Thesespecificinterventionsorprobesallowacertaindegreeofdifferentiatingcir-cumscribedcomponentsofpainprocessingandtheunderlyingnociceptivemechanisms(Table 12.3).

4.The experimental pain psychophysical procedures and the most relevantmechanisms they capture are validated by comparing the measures ofdynamicchangeforhealthyparticipantsandchronicpainpatients.Thus,the relevant measurement parameters having differential validity can beisolated, which may point to the pathogenetic relevant mechanisms ofaltereddynamicchangeinchronicpainsyndromes.

References

Abbott,F.V.,Melzack,R.,&Leber,B.F.(1982).Morphineanalgesiaandtoleranceinthetail-flickandformalintests:Dose-responserelationships.Pharmacology Biochemistry Behavior,17,1213–1219.

Apkarian, A. Vania. (1996). Primary somatosensory cortex and pain. Pain Forum, 5(3),188–191.

Apkarian,A.Vania.(1995).Functionalimagingofpain:Newinsightsregardingtheroleofthecerebralcortexinhumanpainperception.Seminars in Neurosciences, 7(4),297ff.

Arendt-Nielsen,L.,Andersen,O.K.,&Jensen,T.S.(1996).Brief,prolongedandrepeatedstimuli applied to hyperalgesic skin areas:A psychophysical study. Brain Research,712(1),165–167.

Basbaum,A.I.,Marley,N.J.,O’Keefe,J.,&Clanton,C.H.(1977).Reversalofmorphineandstimulationproducedanalgesiabysubtotalspinalcordlesions.Pain, 3,43–56.

Bodnar,R.,Paul,D.,&Pasternak,G.W.(1991).Synergisticanalgesicinteractionsbetweentheperiaquaeductalgrayandthelocuscoeruleus.Brain Research, 558,224–230.

Bromm,B.,&Treede,R.D.(1983).CO2laserradiantheatpulsesactivateCnociceptorsinman.Pflugers Archiv: European Journal of Physiology, 399(2),155–156.

Bromm,B.,&Treede,R.D.(1984).Nervefibredischarges,cerebralpotentialsandsensationsinducedbyCO2laserstimulation.Human Neurobiology, 3(1),33–40.

Cervero,F.(1986).Dorsalhornneuronsandtheirsensoryinputs.InT.L.Yaksh(Ed.),Spinal afferent processing(pp.197–216).NewYork:PlenumPress.

Craig,A.D.(2002).Howdoyoufeel?Interoception:Thesenseofthephysiologicalconditionofthebody.National Review of Neuroscience, 3(8),655–666.

Craig,A.D. (2003a). Interoception:The senseof thephysiological conditionof thebody.Current Opinions in Neurobiology, 13(4),500–505.

Craig,A.D.(2003b).Anewviewofpainasahomeostaticemotion.Trends in Neuroscience, 26(6),303–307.

Y116134.indb 277 10/10/11 2:12 PM


Darian-Smith, I. (1984). Thermal sensibility. In I. Darian-Smith (Ed.), Sensory processes II.Handbook of physiology,Sect. 1, Vol. 3 (pp. 879–913).Bethesda,MD:AmericanPhysiologicalSociety.

Davis, K. D., Pope, G. E., Crawley,A. P., & Mikulis, D. J. (2004). Perceptual illusion of“paradoxical heat” engages the insular cortex. Journal of Neurophysiology, 92(2),1248–1251.

Derbyshire,S.W.(2000).Exploring thepain“neuromatrix.”Current Review of Pain, 4(6),467–477.

Derbyshire,S.W.G.,Jones,A.K.P.,Gyulai,F.,Clark,S.,Townsend,D.,&Firestone,L.L.(1997).Painprocessingduringthreelevelsofnoxiousstimulationproducesdifferentialpatternsofcentralactivity.Pain, 73,431–445.

Droste,C.(1988).SchmerzperceptionundperiphereSchmerzlokalisationbeiAnginapectoris.Zeitschrift für Kardiologie, 5,15–33.

Flor, H. (2003). Cortical reorganisation and chronic pain: Implications for rehabilitation.Journal of Rehabilitation Medicine, 41(Suppl),66–72.

Fordyce,W.E.(1984).Behaviouralscienceandchronicpain.Postgraduate Medical Journal, 60(710),865–868.

Greene,L.C.,&Hardy,J.D.(1962).Adaptationofthermalpainintheskin.Journal of Applied Physiology, 17,693–696.

Handwerker,H.O.,&Kobal,G.(1993).Psychophysiologyofexperimentallyinducedpain.Physiological Reviews, 73(3),639–671.

Heppelmann,B.,Messlinger,K.,Schaible,H.G.,&Schmidt,R.F.(1991).Nociceptionandpain.Current Opinion in Neurobiology, 1(2),192–197.

Hölzl,R.,Erasmus,L.,&Möltner,A.(1996).Detection,discriminationandsensationofvis-ceralstimuli.Biological Psychology,42(1–2),199–214.

Hölzl,R.,Kleinböhl,D.,&Huse,E.(2005).Implicitoperantlearningofpainsensitization.Pain, 115(1–2),12–20.

Hylden,J.L.K.,Hayashi,H.,Bennett,G.J.,&Dubner,R.(1985).SpinallaminaIneuronsprojectingtotheparabrachialareaofthecatmidbrain.Brain Research, 336,195–198.

Kandel,E.R.(2001).Themolecularbiologyofmemorystorage:Adialoguebetweengenesandsynapses.Science, 294(5544),1030–1038.

Kenshalo,D.R.,Leonard,R.B.,Chung, J.M.,&Willis,W.D. (1979).Responsesofpri-matespinothalamicneuronstogradedandtorepeatednoxiousheatstimuli.Journal of Neurophysiology, 42,1370–1389.

Klein,T.,Magerl,W.,Hopf,H.C.,Sandkühler,J.,&Treede,R.D.(2004).Perceptualcorre-latesofnociceptivelong-termpotentiationandlong-termdepressioninhumans.Journal of Neuroscience, 24(4),964–971.

Kleinböhl,D.(1996).Psychophysikalische Korrelate von Anpassungsprozessen bei lang andau-ernden Schmerzreizen bei Gesunden und chronischen Schmerzpatienten.Aachen:Shaker.

Kleinböhl, D., Baus, D., Hornberger, U., & Hölzl, R. (2005). Schmerzgedächtnis undSensibilisierung. PsychoNeuro, 31(2),84–91.

Kleinböhl, D., Hölzl, R., Möltner,A., Rommel, C.,Weber, C., & Osswald, P. M. (1999).Psychophysical measures of sensitization to tonic heat discriminate chronic painpatients.Pain, 81,35–43.

Kleinböhl,D.,Trojan, J.,Konrad,C.,&Hölzl,R. (2006).SensitizationandhabituationofAMH and C-fiber related percepts of repetitive radiant heat stimulation. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neuro-physiology, 117(1),118–130.

Y116134.indb 278 10/10/11 2:12 PM


LaMotte,R.H.(1979).Intensiveandtemporaldeterminantsofthermalpain.InD.Kenshalo(Ed.),Sensory functions of the skin in primates.Oxford:PergamonPress.

LaMotte,R.H.,&Campbell,J.N.(1978a).C-fiberafferentsinmonkeywithhumanjudg-mentsofthermalpain.Journal of Neurophysiology, 41,509–528.

LaMotte,R.H.,&Campbell,J.N.(1978b).Comparisonofresponsesofwarmandnocicep-tive c-fiber afferents in monkey with human judgements of thermal pain. Journal of Neurophysiology, 41,509–528.

LaMotte,R.H.,Thalhammer,J.G.,Torebjörk,H.E.,&Robinson,C.J.(1982).Peripheralneu-ralmechanismsofcutaneoushyperalgesiafollowingmildinjurybyheat.The Journal of Neuroscience,2,765–781.

LaMotte,R.H.,Thalhammer,J.G.,&Robinson,C.J.(1983).Peripheralneuralcorrelatesofmagnitudeofcutaneouspainandhyperalgesia:Acomparisonofneuraleventsinmon-keywithsensoryjudgmentsinhuman.Journal of Neurophysiology, 50,1–26.

Lang,P.J.,Davis,M.,&Öhman,A. (2000).Fearandanxiety:Animalmodelsandhumancognitivepsychophysiology.Journal of Affective Disorders, 61(3),137–159.

Loeser,J.D.,&Treede,R.D.(2008).TheKyotoprotocolofIASPbasicpainterminology.Pain, 137(3),473–477.

Mendell,L.M.(1966).Physiologicalpropertiesofunmyelinatedfiberprojectionstothespinalcord.Experimental Neurology, 16,316–332.

Mense,S.,Hoheisel,U.,Kaske,A.,&Reinert,A. (1997).Musclepain:Basicmechanismsandclinicalcorrelates.InT.S.Jensen,J.A.Turner,&Z.Wiesenfeld-Hallin(Eds.),Pro-ceed ings of the 8th World Congress on pain. Progress in Pain Research and Managment (pp.479–496).

Meyer, R.A., & Campbell, J. N. (1981). Myelinated nociceptive afferents account for thehyperalgesiathatfollowsaburntothehand.Science, 213,1527–1529.

Millan,M.J.(1999).Theinductionofpain:Anintegrativereview.Progress in Neurobiology, 57(1),1–164.

Milner,B.,Squire,L.R.,&Kandel,E.R.(1998).Cognitiveneuroscienceandthestudyofmemory.Neuron, 20(3),445–468.

Nielsen,J.,&Arendt-Nielsen,L.(1998).Theimportanceofstimulusconfigurationfortem-poralsummationoffirstandsecondpaintorepeatedheatstimuli.European Journal of Pain, 2(4),329–341.

Philips,H.C.(1987).Avoidancebehaviouranditsroleinsustainingchronicpain.Behaviour Research and Therapy, 25(4),273–279.

Philips,H.C.,&Grant,L.(1991).Theevolutionofchronicbackpainproblems:Alongitudi-nalstudy.Behaviour Research and Therapy, 29,435–441.

Poulton,E.C.(1989).Bias in quantifying judgments.Hove:Erlbaum.Price,D.D.,Mao,J.,Frenk,H.,&Mayer,D.J.(1994).TheN-methyl-d-aspartatereceptor

antagonistdextromethorphanselectivelyreducestemporalsummationofsecondpaininman.Pain, 59,165–174.

Raja,S.N.,Meyer,R.A.,&Campbell,J.N.(1990).Hyperalgesiaandsensitizationofprimaryafferentfibres.InH.L.Fields(Ed.),Pain syndromes in neurology.London:Butterworths.

Ren,K.(1994).Wind-upandtheNMDAreceptor:Fromanimalstudiestohumans.Pain, 59,157–158.

Rexed, B. (1952). The cytoarchitectonic organization of the spinal cord in the cat. Brain Research, 51,415–495.

Sandkühler,J.(2000).Learningandmemoryinpainpathways.Pain, 88(2),113–118.Severin,F.,Lehmann,W.P.,&Strian,F.(1985).Subjectivesensitizationtotonicheatasan

indicatorofthermalpain.Pain, 21,369–378.

Y116134.indb 279 10/10/11 2:12 PM


Sherrington,C.S.(1906).The integrative action of the nervous system.NewHaven,CT:YaleUniversityPress.

Sokolov,E.N.(1960).Neuronalmodelsandtheorientingreflex.InM.A.Brazier(Ed.),The cen-tral nervous system and behavior(pp.187–276).NewYork:JoshuaMacyFoundation.

Torebjörk,E. (1994):Nociceptordynamics inhumans. InG.Gebhart,D.L.Hammond,&T.S.Jensen(Eds.),Progress in pain research and management, Vol. 2: Proceedings of the 7th World Congress on Pain(pp.277–284).Seattle:IASPPress.

Treede,R.D.,Apkarian,A.V.,Bromm,B.,Greenspan,J.D.,&Lenz,F.A.(2000).Corticalrepresentationofpain:Functionalcharacterizationofnociceptiveareasnearthelateralsulcus.Pain, 87(2),113–119.

Treede,R.D.,Kenshalo,D.,Gracely,R.,&Jones,A.(1999).Thecorticalrepresentationofpain.Pain, 79,105–111.

Treede,R.D.,Meyer,R.A.,&Campbell,J.N.(1998).Myelinatedmechanicallyinsensitiveafferentsfrommonkeyhairyskin:Heat-responseproperties.Journal of Neurophysiology, 80(3),1082–1093.

vanHees,J.,&Gybels,J.(1981).Cnociceptoractivityinhumannervesduringpainfulandnonpainfulskinstimulation.Journal of Neurology, 44,600–607.

Watkins,L.R.,&Mayer,D.J.(1982).Organizationofendogenousopiateandnonopiatepaincontrolsystems.Science, 216,1185–1192.

Willis,W.D.(1988).Dorsalhornneurophysiologyofpain.Annals of the New York Academy of Sciences, 531, 76–89.

Woolf,C.J.,&Salter,M.W.(2000).Neuronalplasticity:Increasingthegaininpain.Science, 288(5472),1765–1769.

Woolf,C.J.,&Thompson,S.W.(1991).Theinductionandmaintenanceofcentralsensitiza-tionisdependentonN-methyl-d-asparticacidreceptoractivation:Implicationsforthetreatmentofpost-injurypainhypersensitivitystates.Pain, 44(3),293–299.

Zimmermann,M.,&Handwerker,H.O.(1984).Schmerz—Konzepte und ärztliches Handeln.Berlin:Springer.

Zimmermann, M. (1993). Physiologie von Nozizeption und Schmerz. In H.-D. Basler,C. Franz, B. Kröner-Herwig, H.P. Rehfisch & H. Seemann (Eds.), Psychologische Schmerztherapie.(pp.46–88).Berlin:Springer.

Y116134.indb 280 10/10/11 2:12 PM

281

Measurement-related issues in the investigation of active vision

Boris M. Velichkovsky,1,2 Frans Cornelissen,3 Jan-Mark Geusebroek,4 Sven-Thomas Graupner,1 Riitta Hari,5 Jan Bernard Marsman,3 Sergey A. Shevchik,2 and Sebastian Pannasch1

1AppliedCognitiveResearchUnit,DresdenUniversityofTechnologyDresden,Germany2NationalResearchCenter“KurchatovInstitute”andUniversityMEPHIMoscow,Russia3UniversityMedicalCentreGroningen,theNetherlands4InstituteofInformatics,UniversityofAmsterdamAmsterdam,theNetherlands5BrainResearchUnit,LTL,HelsinkiUniversityofTechnologyEspoo,Finland

13.1 Introduction

Inhumans,asinallhigherprimates,visionisthedominantsensorymodality.Theessentialroleofeyemovementsinvisualperceptionhasbeenwellknownforalongtimeandhasbeenrepeatedlydemonstrated(e.g.,Findlay,1998;Hayhoe&Ballard,2005). This natural sampling of information from the environment requires thatvisualperceptionisinvestigatedwithintheframeworkof“activevision”(Findlay,1998).Duetotheunevendistributionoflight-sensitivereceptorsacrosstheretina,thehighestvisualacuityislimitedtothesmallfovealarea(abouttwodegreesofarcordoublethethumbnailsizeofanextendedarm).Withincreasingeccentricity—inparafoveal andperipheral regions—visionbecomesblurredand theperceptionofcolorisreduced.Theseconstraintsmakeeyemovementsmandatoryforperceivingtheenvironment.Therefore, saccades—fastballisticmovements—areexecuted tobringthegazefromonepointtoanother.Therelativelystableperiodsinbetweenarecalledfixations.Theintakeofvisualinformationoccurswithinfixationsbutislargelysuppressedduringsaccades. Inmanyeverydaysituations, suchas reading

13

Y116134.indb 281 10/10/11 2:12 PM


a text or inspecting an image, oculomotor activity can be described as interplaybetweenfixationsandsaccades.

Sometimes, additional mechanisms are necessary to achieve clear and stableperception.Forinstance,iftheobjectofinterestismovingrelativetoastationarybackground,smooth-pursuitmovements (dynamic fixations)areobserved(Lencer&Trillenberg,2008),whereasmovementsoftheheadorbodyarecompensatedbythevestibulo-ocularreflex(Goldberg&Hudspeth,2000).Withinfixations,theeyesarenotcompletelystationary,but theyundergoseveral typesofmicromovements(forarecentreviewseeCollewijn&Kowler,2008).However,forthepurposeofthischapter,thefocusisonfixationsandsaccades,bothofwhichcanbecharacterizedbyseveralmeasurementparameters.

Inrelationtosaccades,frequentlymeasuredparametersaretheamplitude,maxi-mum(orpeak)velocityanddirection,aswellasthelatencyuntilasaccadeistriggered.With respect to fixations, the prominent parameters are their duration and spatialdistribution.Visualtasksarenormallyaccompaniedbytheexecutionofsaccadiceyemovementsabout3–4timespersecond.Inaneverydaytask,suchastea-making,theamplitudesofsaccadesvaryinsizefromonlyafewdegreesupto130degrees(Land,2004),althoughinthelattercasestheviewingisalsosupportedbycoordinatedmotionoftheheadandshoulders.Theamplitudeofsaccadesisdirectlyrelatedtovariationofthesizeandoverallshapeoftheimage(vonWartburg,Wurtz,Pflugshaupt,Nyffeler,Lüthi,&Müri,2007).Apositivecorrelationbetweentheamplitudeofsaccadesandthesaccadicvelocityhasbeenrepeatedlydemonstratedandisalsoknownasmain sequence,atermthathasbeenadoptedfromastrophysics(Bahill,Clark,&Stark,1975).Inmostsituations,allocationofvisualattentionseemstobeinagoodcorre-spondencewiththedirectionoftheeyes(e.g.,Fischer,1999).

Thetimethattheeyesarestationaryinrelationtoaparticularregionorobjectisdenominatedfixation duration.This timevaries fromless than100ms tosev-eralseconds,resultinginapositivelyskeweddistributionwithapronouncedpeakbetween200and300ms.Becausetheintakeofvisualinformationismostlylimitedtothetimeoffixations,theunderstandingoffactorsthatinfluencethedurationoffixationisessentialinresearchonvisualperceptionandgazecontrol.Thedurationoffixationsisrelatedtothedifficultyoftheongoingtask(Just&Carpenter,1976;Velichkovsky, 1999). Investigations of different tasks, such as reading (Rayner,1978), visual search (Vaughan, 1982), driving (Velichkovsky, Joos, Helmert, &Pannasch,2005;Velichkovsky,Rothert,Kopf,Dornhoefer,&Joos,2002)andstaticscene perception (Henderson, Weeks, & Hollingworth, 1999) have produced dif-ferentmodelstoexplainthecontroloffixationduration.Ithasbeensuggestedthatfixationsarecontrolledbytheextractionofsensoryinformation(Just&Carpenter,1980), cognitive processes such as memory storage (Shebilske, 1975), and by theprocessesofeyemovementprogramming(Zingale&Kowler,1987).

Inthischapter,weprovideanoverviewofthreeclassesofmeasurementissuesthat are central to investigation of eye movements and active vision: (1) physicalmeasurementsrelatedtopotentialopticalhazardstoeyesandtherelatedsafetycon-cerns,aswellasstatisticalevaluationofnatural-imageparameters;(2)approachestotheanalysisandinterpretationofbehavioraldataondistributionofvisualfixations

Y116134.indb 282 10/10/11 2:12 PM

Measurement-related issues in the investigation of active vision 283

across the image; and (3) the need for additional neurophysiological recordings.Theselatterproblemscurrentlyarethefocusofinterestofthescientificcommunityastheirsolutionisaprerequisiteforenhancingourunderstandingofbrainmecha-nismsofvisualcognitionandcontrolofeyemovements.

13.2 Issues in eyetracking, optical measurement, and image statistics

Interestinactivevisionhasalonghistory(forareviewseeWade&Tatler,2005).Thefirstdevicestomakeeyemovementrecordingsweredevelopedover100yearsago(e.g.,Dodge,1900;Stratton,1906).Theearlyuseofeyetracking inpsychologicalresearch was often accompanied by severe inconveniences (e.g., vacuum devicesweredirectlyattachedtotheeyeball).Thusrecordingtimewaslimiteduptoafewminutes.Nowadaysregisteringeyemovementsismorecomfortableandoftendoesnotrequireanycontactofthesystemtotheheadorbody.Theseadvantagesmakeitpossible tomeasureeyemovementsfor longerperiods; infact therearenolon-geranytimelimits.Thistechnologicalprogressmakeseyetrackingmoreandmoreappropriateforpracticalapplicationsinindustrialsettings,aswellasinstudiesofinfantsandofvariouspatientgroups.For instance,patientssufferingfroma lackofpossibilitiestocommunicatewiththeenvironmentcanbenefitfromeyetypingsystemstorecoverapartoftheircommunicativeabilities(Bates,Donegan,Istance,Hansen,&Räihä,2007;Pannasch,Helmert,Velichkovsky,Malischke,&Storch,2008).Withtheeverydayapplicationofeyetracking,questionsconcerningthesafetyofthistechniquearise,especiallywhenitisusedfrequentlyforlongertimeperiodsofdays,months,andyears.

Asarule,today’scontactlesseyetrackingsystemsuseaninfrared(IR)illumina-tionproducedviaIRlight-emittingdiodes(LEDs).Thereare tworeasonsfor thistypeoflightingineyetrackingsystems.Firstofall,theuseofalightsourceinthesceneenhancesthequalityoftheimageintermsofcontrastandintensitylevels,anditfacilitatesdetailedimageanalysisandgazeestimation.Ontheotherhand,IRopti-calradiationisinvisibletothehumaneye,sothatthelightingiscomfortableanddoesnotdistracttheuser’sattention.Inaddition,IRradiationisreflectedbythecornealsurface, creatingbright reflectionpoints, termedglints (seeFigure 13.1). It is fre-quentlyassumedthattheglintpositionsintheimagedonotchangewitheyerotationsbutonlywitheyeballtranslations,thusgivinganeasilymeasuredreferenceforheadposition.Recently,theinitialassumptionhasbeenrevisited(Guestrin&Eizenman,2006;Shih&Liu,2004).Inthesealternativemodelsforgazeestimation,themove-mentoftheglintduetoeyeballrotationsisintroducedandmeasuredaccordingly.

Oneparticularmeasurementproblemisrelatedtobiophysicalaspectsofvideo-basedeyetracking.AsillustratedbyFigure 13.2,currentguidelinesidentifyanumberofpotentialhazardstotheeyefromopticalsources;thesehazardshavetobeevalu-atedtoassuresafety(seeMulvey,Villanueva,Sliney,Lange,Cotmore,&Donegan,2008).Humanvisionisbiologicallyadaptedtoprotectitselfagainstintensebroad-bandopticalradiation(ultraviolet,visible,andinfraredradiantenergy)fromthenat-uralenvironment.Inaddition,humansuseprotection,suchashatsandsunglasses,to shield against the harmful effects upon the eye from intense ultraviolet (UV)

Y116134.indb 283 10/10/11 2:12 PM


radiationandbluelightpresentinsunlightoversnoworsand.Theaversionreflexeffectivelylimitsthelightexposureofretinatoafractionofasecond(about200msorless)andthusprotectstheeyeagainstinjuryfromviewingbrightlightsources,suchasthesun,lamps,andweldingarcs.TheinfraredLEDsofmoderneyetrackersdonot,however,produceanaversionresponse,astheyarebarelyvisible,andtheiremissionislimitedtothenear-infrared(IR-A,from760to1400nm)spectralband.Ifincandescentordischargelampsthathavebeenfilteredtoblockmostvisiblelight

Figure 13.1 Animageofthehumaneyeinvideo-basedmeasurementoftheocularmotilityandthepupildiameter(twoIRlightsourcesareused).

NonionizingRadiation BandWavelength, nm

AdverseEffects Erythema

Cataract

Photokeratitis Retinal Burns

Color VisionNight VisionDegradation

Corneal Burns

Cataracts

ermal Skin Burns

UV–C UV–B UV–A Visible IR–A IR–B IR–C

100 280 315 400 760 1400 3000 106

SkinPenetrationof Radiation(Depth, mm)

0

2

4

Figure 13.2 The photobiological bands of the Commission Internationale de l’Éclairage(CIE)withspectralregionsofopticalhazardstohumaneyesandskintissues(afterMulveyetal.,2008).

Y116134.indb 284 10/10/11 2:12 PM


andtransmitIR-Aareemployed,somenoticeableemissionsarepossibleoutsidetheIR-A.Theseemissionsmustbemeasuredandevaluatedseparately.

Becausethesaccadesarethemovementsthatbringoureyestoanewlocation,thespatialpositionoflandingisofparticularinterest.Theearlyworkoneyemovementsalreadyshowedthatoureyesareguidedtospecificlocationsthatarenotrandomlydistributedoverthewholescene(Buswell,1935).Twopossiblesourcesthatmightinfluencesaccadictargetselectioncanbeassumed:thetaskoftheobserverandthe“saliency”ofobjects.ThestudybyAlfredYarbus(1967)revealedthatpresentingthe same image with various instructions significantly affects the landing pointsof thesaccades.For instance,when theageof thepeople in the imagehad toberated,thefacesweremoreoftentargetedbythegazethanifthematerialcircum-stancesofthedepictedfamilyhadtobeestimated.Inthelattercase,saccadeswereguidedmostlytofurnitureandotherbackgroundobjects(forarecentreplication,seeDeAngelus&Pelz,2009).Theotherinfluencingfactorisrelatedtosaliency,thatis,thephysicalpropertiesofthestimulus:thedistinctivenessandprominenceofphysi-calfeatures(e.g.,contrast,shape,color,etc.)makeanobjectmorelikelytobecomeasaccadetargetthananotherobjectwithotherfeatures(Itti&Koch,2000).

Throughoutevolution,environmentalregularitieshaveshapedsensorysystems,and thereby have had a tremendous influence on visual perception and cognition(Geisler,2008;Gibson,1979).Hence,studyingthestatisticalregularitiespresentinnaturalsceneshasgreatpotentialintheinvestigationofactivevision.Accuratemod-elsofimagestatisticsatthelevelappropriateformachineimplementationmayalsorevealnewmechanismsofvisualprocessinginthebrain(Karklin&Lewicki,2009).Inthecurrentapproaches,saliencymapsofimagesarecomputedonapixel-by-pixelbasis.However,manyperceptualtasksareconcernedwitharougherimpressionofvisualcontentratherthantheexactimagedetails.Forexample,scenecategorization,suchasknowingthataforestisaforest,generalizesoverthevariousvisualquali-tiesof trees.Natural-image statistics are able to capture the appearanceofmanyscenecategories,althoughweonlyhaveavaguesortofunderstandingwhy.Anotherexample,theperceptionoftexturesandmaterials,suchasgrassandtrees,doesnotsortouteverytinyleafandtwig,butcapturessomesortofimpressionofthevisualimage.Asdemonstratedby theworkof impressionists andGestaltpsychologists,muchofwhatweperceiveiscapturedbyaroughimpression(Greene&Oliva,2009;Koffka,1935;Pelli&Tillman,2008),ratherthanbyprecisedepictionorexactpic-torialassessment.However,whatare thestatisticsgatheringan impressionof thescene?Andhowarethesestatisticsusedwhenwedeterminewheretolooknext?

Natural images are highly structured in their spatial configuration. Whereone would expect a different spatial layout, image statistics often follow a gen-eral Weibull-type spatial distribution (Geusebroek & Smeulders, 2005). Scholte,Ghebreab,Waldorp,Smeulders,andLamme(2009)foundahighcorrelationbetweentheWeibullparametersandasimplemodeloftheparvocellularandmagnocellularpathwaysinthevisualsystem.Thusonewouldexpectimagecontrastsaroundfixa-tionlocationstoreflecttheWeibullstatistics.Fourtypesofnatural-imagestatisticsareexpectedaccording to thebehaviorof the integratedWeibulldistribution: the

Y116134.indb 285 10/10/11 2:12 PM


powerlaw,theexponential,theGaussian,andthepossibilitythattheWeibulldistri-butiondoesnotdescribethedatawell(Yanulevskaya&Geusebroek,2009).

To illustrate the different regimes of the integrated Weibull distribution, a setofseveralhundrednaturalimageswasrecentlyanalyzedonglobalstatistics,localstatistics, and statistics around eye fixations, that is, visually attended regions(Yanulevskaya & Geusebroek, 2009). For the global analysis, all images fit theWeibullstatisticwell,andcouldbefurthersubdividedinto20%power-lawandlessthan 5% Gaussian, the remainder fitting best to the exponential distribution. Thepower-lawisoneextreme,indicatingstrongoverallcontrastcausedbyafewedges.TheGaussianistheotherextremeofthesamedistributionshape,indicatinghighlytextured images. Figure 13.3 conveys an impression of the correlation betweenvisualcontentandWeibullparameters.Inafurtherlocalanalysis,basedonsmallerimageparts,13%ofthepatcheswererejectedinthetestforWeibulldistribution.Avisual inspection showed that these images containedonly compression artifacts,or regionswith regularpatterns.The remaining87%of thepatches followed theWeibulldistribution,whereby26%werepower-lawdistributedand14%Gaussian.

Figure 13.3 Scatterplotof theWeibullparameterswithcontrastonhorizontalandshapeon vertical axes and each patch positioned at its respective value (after Yanulevskaya &Geusebroek,2009).Imagesonthebottomleftcorrespondtothepower-lawsubmodel.Theycontainuniform regions separatedby strongedges.Patches in themiddlepart follow theexponentialdistributionshowingsmallerdetails.TwotypesofvisualcontentdemonstratetheGaussiandistribution:smoothpatcheswithGaussian-likenoise(onthetopleftofthescatter-plot)andpatcheswithhighfrequencyandcontrast(onthetoprightand,asenlargedversions,inthebottomrightinset).

Y116134.indb 286 10/10/11 2:12 PM


Thisresultcanbeexpected,aslocalregionsarelikelytocontainfeweredgesandtohavehigher-frequency textures than thewhole images.Wecomparedeyefixa-tionwiththeaprioribaseline,asgivenbylocalanalysis,usingWeibullanalysisofsimilarlysizedlocalpatchesaroundeyefixationlocations.Eyefixationswerethen50%morelikelytoappearatpower-lawdistributedpatchesthancouldbeexpectedfromlocalanalysis.Furthermore,eyefixationswereunlikelytoappearatGaussiandistributedpatchesastheymainlyexhibitedlow-contrastcompressionartifacts.

Thesefindingswerefurtherexploitedineyefixationprediction,orequivalentlysalient regiondetection,bycombiningboth the shapeandcontrastparametersoftheWeibulldistributioninamachine-learningapproach.Eyefixationpredictionhasbeenrelatedtosimplestatisticsofseveraltypesoflow-levelimagefeatures,mainlyrelated tocontrast and local edge frequency.Due to the regularities in the statis-ticsofnaturalimages,manyofthesefeaturesarecorrelated.GiventheabundantlypresentWeibulldistributioninthestatisticsofsuchfeaturesfornaturalimages,westudythelocaldeviationsofthesestatisticscomparedwith“overall”imagestatisticsoverlargercollections.ThesedeviationsbecomeapparentwhenfittingtheWeibulldistributiontothelocalimage(feature)data,andconsecutivelystudyingthedistribu-tionoftheWeibullparametersoverasingleimageincomparisontotheparametervaluesfoundinalargesetofnaturalimages.Theroughideaisthatoutliersintheparametersindicateastatisticallysalientimageregionwithrespectto“commonly”occurringimagepatches.Inotherwords,regionsaresalientiftheirspatialstatisticsbreakwiththegenerallyobservedstatisticsofnaturalscenes.

Indetail,weconsiderthecorrelationbetweeneyefixationsandlocalimagesta-tistics,thelatterbeingcapturedbytheparametersoftheWeibulldistribution.Usingeyetrackingoveralargecollectionofnaturalimagesandformanysubjects,theprob-abilitydistributionoverWeibullparametersoffixatedimageregionscanbemea-sured.Knowingwhich locationshaveattractedattention, a complementary setofregionsindicatethe“nonfixated”regions,yieldinganegativesetofpatchesresultingin theWeibull parameterprobabilitydensities for “not-so-interesting” regions forhumanvision.Then,machine-learningtechniques,suchaslogisticregression,canbedevisedtolearnthedistinctionbetweenfixatedandnonfixatedpatches.Combinedwithpurelybottom-upapproaches,suchasthemethodproposedbyIttiandKoch(2000),amoreadequatemodelforsaliencypredictioncanbeachieved.

Thisstudyillustratesanapproachtoactivevision,whichisaversionofecologicalopticslongagopromulgatedbyJamesJeromeGibson(1979).Itisexclusivelybasedonthemeasurementandstatisticalanalysisofphysicalparametersofnaturalimages.Indeed,ithasoftenbeensuggestedthateyeguidanceismainlycontrolledbystimu-lusproperties(Godijn&Theeuwes,2002).Otherauthorsconsidersaccadictargetselectionasbeingpredominantlycontrolledbythetaskoftheobserver(Williams&Reingold,2001).Recentobservations revealed that taskdemandsareofhigherprioritythanlow-levelsaliencyfeatureswithrespecttogazebehavior(Einhäuser,Rutishauser,&Koch,2008).Itwasalsosuggestedthataflexiblecombinationofbothapproachescanproperlyaccountfortheguidanceofsaccades(vanZoest&Donk,2004;BallardandHayhoe,2009).

Y116134.indb 287 10/10/11 2:12 PM


13.3 Behavioral data analysis and interpretation

Since theearliestattempts, the interpretationofeyemovementbehaviorhasbeenmainlybasedontheanalysisoffixationsandsaccades.Inparticular,analysishasbeenfocusedonparameterssuchasthedurationanddistributionoffixations,aswellastheamplitude,velocity,anddirectionofsaccades.However,aswasalreadydem-onstratedbypioneersinthisdomainofresearch(e.g.,Buswell,1935;Yarbus,1967),fortheunderstandingofeye-movementbehaviorandtheresultingperception,itisessentialtoconsideralsotheinspectedcontent,thatis,textorimage.

A promising approach to combine spatial and temporal characteristics of eyemovementswith the inspectedmaterial is theattentional landscapemethod intro-ducedbyVelichkovsky,Pomplun,andRieser(1996).Insteadofsimplyprojectingthegazetrackingdataontheoriginalvisualmaterial,theseauthorsproposedtoelucidatetheperceptionofambiguouspaintings(byArchimboldo,Dürer,andEscher)byren-deringthesepicturesintermsofthedistributionofvisualfixations(seeFigure 13.4).Withthismethoditwaspossibletotakeintoaccountboththesaccadicsuppression(i.e.,apartialsuppressionofvisualinputbetweentwofixations)andthefactthateachfixationresolvesonlyanarrowareaofthevisualscene.Thelimitationsofvisualspa-tialresolutionwereapproximatedbyaGaussiandistributionwiththestandarddevia-tionof1degreeateachfixationpoint.Inthisway,itwaspossibletovisualizewhenanobserverperceivedoneoranotherversionofambiguouspictures(suchasangelsordevilsinCircle Limit IVbyEscher).Thedescribedattentionallandscapeapproachisasteptowardexplicatingtheidiosyncraticperceptualexperienceofaperson.Itisnowadaysawidelyacceptedinstrumentinusabilityresearch(e.g.,Henderson,2007;Wooding,2002;Wooding,Mugglestone,Purdy,&Gale,2002)andhasalsobeenusedforanexplicationofthegroundsfornonverbaldecisionsinclinicalradiology(Burgertetal.,2007).Althoughtheresultsofthisapproachlookimpressive,theirinterpretationisratherdifficultandastatisticalanalysisofthistypeofinformationiscomplicated.Anopenquestioniswhethertheresultingvisualizationreallysayssomethingaboutattentionorjustaboutfixationdistributions.

Figure 13.4 AnillustrationoftheattentionallandscapeapproachwithCanaletto’sView of Dresden(copyrightGalleryAlteMeister,Dresden,Germany)wheretheoriginalpaintingisinthebackground,anempiricallymeasuredandcomputedFixation Mapisontheright,andthevisualizationissuperimposedonthepainting(imagecourtesyofClaudiaSchmidt).

Y116134.indb 288 10/10/11 2:12 PM


Onecritical issue is theequal treatmentofallfixations for thecreationof theattentionallandscapevisualizations.Besidetaskcomplexityandtheintentionoftheobserver,thequalityandamountofvisualinformationaffectthedurationoffixa-tions.Mannan,Ruddock,andWooding(1995)presentedhigh-andlowpass-filteredandunfilteredphotosandreportedlongestfixationsforthelowpass-filteredscenesandshortestfor theunfilteredmaterial.Withthedevelopmentof thegazecontin-gent paradigm it became possible to manipulate regions of the presented text orscene according to the respective position of the eyes. For instance, van Diepenandd’Ydewalle(2003)usedagaze-contingentmaskduringvisualsearch,prevent-ingeitherfovealorperipheralprocessingofinformation.Prolongedfixationswerereportedforbothmanipulations,suggestingthatbothfovealandperipheralinforma-tionmodulatethelengthsoffixations.DuringsinglefixationsLoschky,McConkie,Yang,andMiller(2005)presentedamultiresolutionaldisplaywithhighresolutionatthefoveathatdecreasedwitheccentricityfromthegazecenter.Longerfixationswerereportedifthemanipulationonthesizeofelementswasperceivable.

Recently, evidenceonqualitativedifferences in informationprocessingduringsinglefixationshasbeencollected.Itisknownthatthedurationoffixationscanberelatedtothelevelofcognitiveprocessing(e.g.,Velichkovsky,2001)andthereforeitseemstobeimplausiblethatallfixationswouldplaythesameroleinactivevisualprocessing.Instudiesofeyemovementbehaviorinstaticanddynamicenvironments,wefoundthataparticularcombinationoffixationdurationandamplitudeofrelatedsaccadesstronglycorrelateswiththeprobabilityofcorrectrecognitionofobjectsandevents(Velichkovsky,Rothert,Kopf,Dornhoefer,&Joos,2002;Velichkovsky,Joos,Helmert,&Pannasch,2005).Accordingtothesefindings,adistinctionisnecessarybetweenfixationsservingthedetailedprocessingofobjectfeaturesandthosethatarerelatedtoadynamicspatiallocalizationoftheobjectsasundifferentiatedblobs(proto-objects).Withoutconsiderationofthisimportantdifferenceinthemodesofactivevision,therepresentationproducedintheattentionallandscapesapproachcanonlyincorrectlyreflecttheperceptualprocessandperceivedinformation.

Anothercriticalissueininvestigationofactivevisionistheinfluenceofsemanticand, in particular, social factors on human oculomotor activity. Even if the low-levelsaliencyfeaturesarecarefullycontrolled,onefindsthatfacialareawitheyesisthedominantattractorforhumaneyefixations(foranexample,seeHari&Kujala,2009). This tendency seems to persist also in nonhuman primates (Tomonaga &Imura, 2009). With a notable exception of autistic observers (Neumann, Spezio,Piven,&Adolphs,2006),theeyesarealwaysaveryspecialtargetofattractioninourenvironment.Thepositionswhereeyesnormallyareintheschemeoffacewillbealsofixatedwhenthereisnoopticalinformationontheminthephysicalimage.Thesedataillustrateoneofthemainfunctionalitiesofhigher-orderformsofvisualattention,namelypayingattentiontotheattentionofanotherperson(Velichkovsky,1995).Overall,theresultstestifytotheexistenceofahigher-ordersocialsaliencyfactor.Theideaofjointattentionstatesastheprerequisiteforcommunicativeandpracticalinteractionoftwopersonsisalsomanifestedinanattentionalemphasisontheobjectsofcommoninterest.ThisVygotskianideaisillustratedbysomerecenteyetrackingstudies,withoneexampleshowninFigure 13.5.

Y116134.indb 289 10/10/11 2:12 PM


Mostof thecurrentattemptsaimedatunderstandinggazecontrol andmecha-nismsofvisualprocessinginfreeviewingconditionscomefromthefieldsofexperi-mentalpsychologyandcognitivescience.Thesestudiesoftenconstrainthemselvesbythemeasurementandanalysesonthebehaviorallevel.Toovercometheselimi-tationsandtorevealtheunderlyingbrainmechanisms,abroaderinterdisciplinaryperspectiveisneeded.Forexample,classicalneuropsychologicaldatademonstratedisturbancesofactivevisualperceptionafterfrontal-lobelesions(Luria,Karpov,&Yarbus,1966).Innonhumanprimatesandotheranimals,invasivemeasuringpro-cedures (e.g., single-cell recordings using implanted electrodes) are common andtheyareusedalsowithcomplexnatural imagesandrealistictasks(Cavanaugh&Wurtz, 2004; Maldonado, Babul, Singer, Rodriguez, Berger, & Grun, 2008). Inhuman subjects, it would be beneficial to combine eyetracking with noninvasiveneurophysiologicalmeasurements.

13.4 Active vision in human electrophysiology and neuroimaging

The idea to combine eyetrackingwithnoninvasivemethodsofmeasuringhumanbrainactivity isnotnew.Several attemptshavebeenmade toanalyzeeyemove-ments togetherwithphysiological techniques, such asEEG (electroencephalogra-phy)andMEG(magnetoencephalography).Twodifferentstrategiescanbeappliedwhenusing such a combination. Because eyemovements produce strong electro-magneticsignals,eyetrackingisusedtocontrolforthosemovementstoremovetheirinfluencesonthebrainsignalsofinterest.Theotherpossibleapproachfollowstheideathateyemovementsaregenuinecomponentsofvisualcognition.Theresearchquestionthenwouldbetoidentifybrainprocessesthatareinvolvedduringdifferentstagesintheinspectionofascene,includingsinglevisualfixations.Inthiscasetheaimistodescribehowvisualperceptionworksduringnaturalvisionandwhataretheunderlyingbrainprocesses.However,eachoftheneurophysiologicalmeasuring

A B

Figure 13.5 A andB:typicaldistributionofgazelocationsonapainting(Lighting PipesbyJuhoRissanen,1902;copyrightAteneumArtMuseum,HannuAaltonen,Finland).Thefixa-tionmapisbasedonthedataof10femalesand10males,whowereallowedtoviewfreelytheimagefor10s(fromHari&Kujala,2009;withpermissionofthepublisher,Physiological Reviews).

Y116134.indb 290 10/10/11 2:12 PM


techniqueshasitsadvantagesanddisadvantages,asdiscussedintheremainderofthischapter.

MeasuringEEGistheoldestandmostbroadlyusedmethodforstudyingongo-ingbrainactivity,notonlyduringvisualperceptionbutalsoduring restand inavarietyofother tasks.CurrentEEG technologyallows themeasurementofbrainactivity with high temporal resolution and relatively good spatial accuracy. ThusEEGcouldbeasuitablemethodtostudybrainactivationsduringnaturalviewingofscenes.However,theintegrationandsimultaneousrecordingsofeyemovementsandEEGhasbeenonlyrarelyaccomplishedinthepast.Asamatteroffact,motil-ityof the eyeballs isoneof the critical sourcesof artifacts inEEG.Because thecorneaispositivelychargedcomparedwiththeretina,theeyebehavesasanelec-tricdipole:eachmovementoftheeyeballgeneratesanelectromagneticfieldthatisdetectedonthescalp(EEG)andoutsidethehead(MEG;Hari,2004).Becausetheeyesmoveaboutthreetimesasecond,thisactivitystronglyinfluencestherawEEGsignal,withamplitudesoftenbeing larger than theactualEEGsignalof interest.Manymathematicalapproacheshavebeenproposedinthepasttoeliminatethecon-taminatinginfluencesoftheeyemovements(e.g.,Delorme,Sejnowski,&Makeig,2007;Gratton,Coles,&Donchin,1983;Vigário,Jousmäki,Hämäläinen,Hari,&Oja,1997).Althoughthesemethodssignificantlycontributetotheimprovementofthesignal,thereareconsiderablelimitationsintheirapplicationastheyeitherdonotremovetheartifactscompletelyormayfilteroutsomesignalsofinterest.

Yagi (1995)wasoneof thefirst tomention thepossiblebenefitsofcombiningeyetrackingwithEEGfortheanalysisofbraindynamicsinmorenaturalsettings.Heintroducedtheterm“eyefixationrelatedpotential”(EFRP),whichdescribesthemethodofanalyzingtheEEGsignalastimelockedtotheoccurrenceoffixations.Themainideaofthismethodistostructuretheelectrophysiologicaldataaccordingtotheeventsofinformationuptakeduringactivevision.Evenifthisapproachdoesnotcompletelysolvetheproblemofeye-movement-relatedartifactsintheEEGdata,itallowsresearchers to investigatebrainactivityandvisionduringmorecomplexandnaturaltasks.Subsequently,theEEGdatacanbeanalyzedinamorecontrolledway,onafixation-by-fixationbasis.Newapproachesuseeyetrackingnotonlyforofflineexperimentalcontrolandsegmentingofthedata,butalsoforonlinemanipu-lationofindependentvariables,suchasingaze-contingentexperiments,withsimul-taneousmeasurementofelectrophysiologicalresponseseithertothefixatedsiteassuchoronaspeciallyselectedteststimulus(Baccino&Manunta,2005;Graupner,Velichkovsky,Pannasch,&Marx,2007).Thelatterparadigmmaybethemethodofchoiceasitallowstestingthevisualsystemwithaspecifieddelayafterthesaccadeand,inthisway,toavoidmuchoftheinterferencewithelectromagneticcontamina-tionfromthemovingeyeballs.

Apart from electromagnetic artifacts caused by the eyes, other effects can beobservedinthebrainwhentheeyesmove.Thesignalspriorto,during,andafterasaccadearerelated to theprocessesofsaccadepreparationandgeneration,signaltransductionalongtheopticnerve,andinformationprocessingwithinthevisualandothercorticalareas.Alltheseactivitiesareofcrucialinterestforunderstandingtheactivevision.AnumberofinterestingsignalshavebeenidentifiedintheEFRPs.

Y116134.indb 291 10/10/11 2:12 PM


Presumablythemostprominentsignalistheso-calledlambda response,whichisascalp-positiveEEGdeflectionpeaking80–100msaftertheoffsetofasaccade(Rémond,Lesèvre,&Torres,1965;Yagi,1979).Thelargestamplitudeofthispeakisusuallyobservedatoccipitalelectrodesites(Graupneretal.,2007).SomeauthorshavearguedthatthelambdaresponseisequivalenttotheP100deflectionofconven-tionalvisualevokedpotentials (VEPs),elicitedbyabruptvisualstimuli (Billings,1989;Kazai&Yagi,2003).ThisinterpretationwassupportedbydipoletracingforthelambdaandP100response,showingsimilarsourcelocations,presumablyinthecalcarinefissure(Kazai&Yagi,2003).However,otherresultsexist,reportingaverylambda-likeresponsegeneratedintheparieto-occipitalsulcusaftereyeblinksandsaccades (Hari,Salmelin,Tissari,Kajola,&Virsu,1994; Jousmäki,Hämäläinen,&Hari,1996;Kazai&Yagi,2003).Thepeakamplitudeof the lambda responsewas found to be correlated with the amplitude of the preceding saccade (Yagi,1979).SimilartotheP100deflection,thelambdaresponsedependsonthephysicalaspects of stimulation, such as luminance, contrast, spatial frequency, and color.AnothercomponentofanEFRPisthespikepotential(Becker,Hoehne,Iwase,&Kornhuber,1972),a scalp-negativedeflectionpeaking8–20msprior to theonsetofasaccade.Itlikelyresultsfromsynchronousfiringofthemotoneuronsofocularmuscles(Riemslag,VanderHeijde,VanDongen,&Ottenhoff,1988;Thickbroom&Mastaglia,1985).Additionally,characteristicEEGpatternshavebeenreportedinrelationtotheprogrammingandexecutionofsaccades(Evdokimidis,Smyrnis,Constantinidis,Gourtzelidis,&Papageorgiou,2001;Skrandies&Laschke,1997).

Foracomparisonoftwosetsoffixations,allthesecontributingfactorsneedtobeconsidered.Forexample,disregardingthematchingofthesaccadicamplitudespriortothefixationwillresultinasystematicbiaswithrespecttothelamda/P100amplitudes (Yagi, 1979) which also might affect later EFRP components (N100,P200,etc.).Similareffectsmustbeexpectedalsoforthesubsequentsaccadeifnotmatchedaccordingly.However,duetoahighvariabilityoffixationdurationsduringnaturalvisualexploration,suchinfluencesofthesubsequentsaccadesmightbelesspronouncedbecauseoftemporalsmearing.Differencesinfixationdurationbetweenthetwodatasetscanintroduceadditionalbiasbecausetheonsetofthefollowingsac-cadewillsystematicallyvary.

Stillanotherinfluencingfactorinparadigmsstudyingactivevisionhastobemen-tioned.Itconcernsthelackofcontrolonthetemporalandspatialdistributionoffix-ations,thatis,wheninthecourseofsceneinspectionaparticularobjectorregionisfixated.Whentwosubjectsfreelyobservethesamenaturalscene,tworatherdiffer-entsetsoffixationmayarisewithrespecttothewhenandwhatofthegazepattern.Thus,imagestatisticsofthefixatedspots,suchaslocalcontrast,spatialfrequency,luminance,andcolormightbedifferentforthetwosetsoffixationswhichwillalsoinfluencetheobtainedEFRP.Eveniftheeyetrackingdataenableresearcherstogainthis information during an experiment either online or offline, it is still an openquestionofhowthesedetailsoftheefficientsensorystimulationcanbeintegratedinthedataanalysis.Inanycase,consideringthementionedpointsismandatoryforaproperinterpretationofresults.

Y116134.indb 292 10/10/11 2:12 PM


Theproblemofeye-movement-relatedartifactsinEEGrecordingsdoesnotonlyapplytotheinvestigationofnaturalviewingofscenes.Therelevanceofsuchinflu-enceshasbeenrecentlyreportedinastudywithparallelrecordingofEEGandeyemovementswheresubjectshadtoperformasimpleobjectidentificationtask.Itwasshownthatsomegamma-bandactivityintheEEGsignalishighlycorrelatedwiththeonsetofsaccades(Yuval-Greenberg,Tomer,Keren,Nelken,&Deouell,2008).Theauthorsexplainedthisfindingbythespikepotentialsthataccompanythesaccadeonsets.Thatmeansthateveninsimplevisualtaskseyemovementbehaviormayvarysystematicallywith the investigatedconditionsand itsoutcomeon theEEGsignalmaytranslateintoapparentbutnotalwaysmeaningfuldifferencesintheresults.

Acomplementaryapproach to theeletrophysiologicalEFRP(EEG)andEFRF(MEG;thelastFtorefertomagneticfield)measurementsisfixation-basedevent-relatedfMRI(FIBERfMRI;seeCornelissen,Marsman,Renken,&Velichkovsky,2008). This is a perspective of deriving brain activation patterns related to rapidhuman visual behavior from a slow blood-oxygenation-level-dependent (BOLD)fMRIresponse.Untilthepresent,fixationshavenotbeenconsideredaseventsofpri-maryinterestinneuroimagingstudies.Nevertheless,wehavebeenabletoidentifydifferencesinbrainactivity,followingdifferenttypesoffixationsineitherviewingtaskorobjectinspected.Specifically,weinvestigatedtheprocessingofvisualinfor-mationfollowingfixationsmadeduringnaturalviewingbehavior.Inthefollowing,webrieflypresentresultsfromarecentfMRIexperiment,wheresubjects’gazewasrecorded tocontrolactualvisualfixations tobuildamarkerevent for theFIBERfMRIanalysisinthreeconditions:freeviewingandtwoversionsofinstructedview-ing,oneofwhichemphasizedlookingatfacesandanotherathouses.Figure 13.6showsthestimulusdisplayandotherrelevantaspectsofthestudy.

First of all, two well-documented brain regions in correspondence with ourstimuli—fusiform face area (FFA) and parahippocampal place area (PPA) bilat-erally (Epstein, Harris, Stanley, & Kanwisher, 1999; Kanwisher, McDermott, &Chun,1997)—were identified inevery subjectby standard localizerexperiments.Furthermore, in themain seriesof experiments, distinguishable featuresbetweentheBOLDresponsesfollowingfixationswereelicited.Theresultwasachievedbyreconstructing the hemodynamic response functions for each type of fixation inthreetaskconditions.

Inall cases, theBOLDresponsepeakedaround3 seconds,which is a shorterlatencythaniscommonlyreportedinconventionalfMRIstudies(Buckner,1998).Itisapparentthatfixationsduringwhichhouseswereinspected(eitherinaspecifichouse-viewing taskorduringfreeexplorationof thestimuli) result in largeracti-vationofPPA.FFAbecomesactivewhenfaceswereinspectedundercomparableconditions.Atthesametime,itseemsthatFFAisalsoactiveduringhouseviewingintheface-viewingtask.DuringfreeviewingthedifferenceinactivationsofhouseandfacefixationsbecomesmoredistinctinPPAincomparisonwithFFAactivation.FFAdoesnotdifferentiatebetweenhousesandfacesonthebasisoftheamplitudeof the response,even though the responsesarestilldifferentlyshaped.Adistinc-tion between the conditions is the existence of a double peak effect, specifically

Y116134.indb 293 10/10/11 2:12 PM


visibleduring instructed (i.e., explicitly task-related) viewingcomparedwith freeviewing. Hence, we argue that this can be a task-related effect that is driven byrepeatedstimulation.Theconclusionwascorroboratedbymultivoxelpatternanaly-ses(MVPA;Norman,Polyn,Detre,&Haxby,2006)wherewewereabletoclassifythesignalssignificantlyabovechancelevelaccordingtotheinspectedobjectandthetask.Interestingly,wefoundthatclassificationofbothhouseandfacetasksismoreefficientthanclassificationoffreeviewingitems.

Theseresultsdemonstratethatbrain-activityanalysescaninanimportantrespectenhancethebehavioralanalysisofeyemovementdata,namelytorevealthenatureofthetaskathandbeyondsimplyshowing—aseyetrackingstudiesoftendo—whichobjectorregionoftheimageiscurrentlyfixated.WehaveshownthatfMRIanalysisbased

PPA (left)

Sign

al C

hang

e (a.u

.)

FFA (left)Face Fixations

House Fixations

Free viewingHouse taskFace task

0 6 12 18 24 32Time (s)

0 6 12 18 24 32

Figure 13.6 (See color insert.) Top left:Example stimulus.Top right: Investigatedbrainregions, FFA (blue) and PPA (red), left hemisphere activations are within the rectangle.Bottom: Estimated hemodynamic responses for fixations toward houses (upper row) andfaces(undersiderow)inleftFFA(leftcolumn)andleftPPA(rightcolumn).Colorsindicatetask;dashedlinesdenotestandarderrorofthemean.Verticalaxesshowpercentagesofthemeanwhole-brainsignal.

Y116134.indb 294 10/10/11 2:12 PM


onfixationsrevealsmeaningfulspatiotemporalpatternsofbrainactivationdespitethesuboptimal temporalcharacteristicsofvisualfixationsandsaccadeswith respect tohemodynamicsinthebrain.ThesefindingsopennewpathstoexplorenaturalviewingbehaviorinthefMRIanalysisofhumanbrainactivity.However,theyalsodemonstratetheimportanceandbenefitsofcombiningneurocognitivemethodsandeyetrackingforthecontrolofexperimentsaswellasforabetterinterpretationofthedata.

13.5 Conclusion and outlook

Asaninterdisciplinaryendeavor,furtherinsightsintobrainmechanismsofhumanactivevisionwilldemandcontinuouseffortsofexpertsfromdifferentdomainsofresearch. In this chapter, we illustrated the scope of relevant measurement prob-lems by three groups of studies ranging from classical physical to psychologicalandneurobiologicalinvestigation.Noneoftheseapproachesisnowadayscompletelysatisfyingandfreeofpotentialcriticalcomments.However,insuchathornyfieldofmeasurementasactivevisiontheoldmethodologicalprincipleofconvergingopera-tions(Garner,Hake,&Eriksen,1956)mightbethebeststrategy.Byusingcomple-mentary,albeit imperfect instruments it is stillpossible to reachsome reasonableconclusions. Several recent examples demonstrate the successful transmethodicalconvergenceinstudiesofactivevision,suchaselucidationofmechanismsofglobalversus local processing (Cant, Arnott, & Goodale, 2009), investigation of visualattentionasexpressedinthepatternsofeyemovements(Velichkovskyetal.,2005),and“socialsalience”incognitive-affectiveneuroscience(Hari&Kujala,2009).

Intheyearstocome,innovativeresearchonvisualperceptionandeyemovementswill be supported by new technological solutions and include new interdisciplin-arydirections.Besidethefurtherimprovementofexistingmeasurementtechniques(based on hard- or software achievements) there will be the development of newmethods. For instance, the emerging method of magnetic induction tomography(MIT;Korjenevsky,Cherepenin,&Sapetsky,2000;Vauhkonen,Hamsch,&Igney,2008)aimsatmappingthelocaldistributionoftheelectricalconductivityinsidethebrain.Eddycurrentsareinducedinthebodybyrelativelyweakmagneticstimula-tionandtheresultingfieldsaremeasuredbyanarrayofreceivercoils.

Moreover,thebasicbiochemicalnatureofthebrain’swetwareandneurotransmit-tersisassumedtobecomemoreinfluentialinthefuture.Theparadigmaticchangesaremotivatedbythedesignofnewapplicationsandservicesundertheinfluenceofincreasinglyfastconvergenceofnano-,bio-,andinformationtechnologieswithcogni-tivescience(forareview,seeBainbridge&Roco,2006).Suchatrans-technologicalconvergencecouldexplodeinthenumberofstudiesdevotedtogeneticandepigeneticmechanismsofhumanvisualprocessingandcognition.Inamiddle-termperspective,newmolecularmethodsofmeasurementwillemergebasedontheuseofnano-sizedsensorsanddyes,suchasinquantum-dotstechnology.Ofcourse,beforethisquantumleapininvestigationwithhumansubjectswillbecomeareality,muchtightersafetyproblemshavetobesolvedthaninthecaseofthevideo-basedeyetrackingdevices.

Y116134.indb 295 10/10/11 2:12 PM


Acknowledgment

Thanks are due to Birgitta Berglund, Jens R. Helmert, Alexander Kaplan, FionaMulvey,JoelNorman,RemcoRenken,andClaudiaSchmidtfortheirhelpinprepa-rationofthemanuscript.OurresearchwassupportedbygrantsfromtheEuropeanCommission (NEST-Pathfinder projects PERCEPT 043261 and MINET 043297,Network of Excellence COGAIN 511598), the Russian Foundation for BasicResearch(Ofi-m09-02-12222and09-06-12003),theRussianMinistryofEducationandScience(P1265),andERCAdvancedGrant#232946.

References

Baccino,T., & Manunta,Y. (2005). Eye-fixation-related potentials: Insight into parafovealprocessing.Journal of Psychophysiology, 19(3),204–215.

Bahill,A.T.,Clark,M.R.,&Stark,L.(1975).Themainsequence,atoolforstudyinghumaneyemovements.Mathematical Biosciences, 24(3–4),191–204.

Bainbridge,W.S.,&Roco,M.C.(2006).Managing nano-bio-info-cogno innovations: Converging technologies for improving human performance.Dordrecht:Springer.

Ballard,D.H.,&Hayhoe,M.M.(2009).Modellingtheroleoftaskinthecontrolofgaze.Visual Cognition, 17(6/7),1185–1204.

Bates, R., Donegan, M., Istance, H. O., Hansen, J. P., & Räihä, K.-J. (2007). IntroducingCOGAIN: Communication by gaze interaction. Universal Access in the Information Society, 6(2),159–166.

Becker,W.,Hoehne,O.,Iwase,K.,&Kornhuber,H.H.(1972).Readinesspotential,pre-motorpositivity and other changes of cortical potential in saccadic eye movements. Vision Research, 12(3),421–436.

Billings,R.J.(1989).Theoriginoftheoccipitallambdawaveinman.Electroencephalography and Clinical Neurophysiology, 72(2),95–113.

Buckner, R. L. (1998). Event-related fMRI and the hemodynamic response. Human Brain Mapping, 6,373–377.

Burgert,O.,Örn,V.,Velichkovsky,B.M.,Gessat,M.,Joos,M.,Straub,G.,etal.(2007).Evaluationofperceptionperformanceinneckdissectionplanningusingeyetrackingandattentionlandscapes—art.no.65150B.Medical imaging 2007: Image perception, observer performance, and technology assessment, 6515,B5150.

Buswell,G.T.(1935).How people look at pictures.Chicago:UniversityofChicagoPress.Cant, J.,Arnott, S., & Goodale, M. (2009). FMRI-adaptation reveals separate processing

regionsfortheperceptionofformandtextureinthehumanventralstream.Experimental Brain Research, 192,391–405.

Cavanaugh,J.,&Wurtz,R.H.(2004).Subcorticalmodulationofattentioncounterschangeblindness.Journal of Neuroscience, 24(50),11236–11243.

Collewijn,H.,&Kowler,E.(2008).Thesignificanceofmicrosaccadesforvisionandoculo-motorcontrol.Journal of Vision, 8(14),1–21.

Cornelissen,F.W.,Marsman,J.B.C.,Renken,R.,&Velichkovsky,B.M.(2008).Predictinggazebehaviorandcognitivetaskfromcorticalactivity:Afixationbasedeventrelated(FIBER)fMRIstudy.Proceeding of the Third Biennial Conference on Cognitive Science(pp.553–557),Moscow:Science.

DeAngelus,M.,&Pelz,J.B.(2009).Top-downcontrolofeyemovements:Yarbusrevisited.Visual Cognition, 17(6/7),790–811.

Y116134.indb 296 10/10/11 2:12 PM


Delorme,A.,Sejnowski,T.,&Makeig,S.(2007).EnhanceddetectionofartifactsinEEGdatausinghigher-orderstatisticsandindependentcomponentanalysis.Neuroimage, 34(4),1443–1449.

Dodge,R.(1900).Visualperceptionduringeyemovement.Psychological Review, 7,454–465.Einhäuser,W.,Rutishauser,U.,&Koch,C.(2008).Task-demandscanimmediatelyreverse

theeffectsofsensory-drivensaliencyincomplexvisualstimuli.Journal of Vision, 8(2),1–19.

Epstein,R.,Harris,A.,Stanley,D.,&Kanwisher,N.(1999).Theparahippocampalplacearea:Recognition,navigation,orencoding?Neuron, 23,115–125.

Evdokimidis, I., Smyrnis, N., Constantinidis, T. S., Gourtzelidis, P., & Papageorgiou, C.(2001).Frontal-parietalactivationdifferencesobservedbeforetheexecutionofremem-bered saccades:An event-related potentials study. Cognitive Brain Research, 12(1),89–99.

Findlay,J.M.(1998).Activevision:Visualactivityineverydaylife.Current Biology, 8(18),R640–R642.

Fischer,M.H.(1999).Aninvestigationofattentionallocationduringsequentialeyemove-menttasks.The Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 3,649–677.

Garner,W.R.,Hake,H.W.,&Eriksen,C.W.(1956).Operationalismandtheconceptofper-ception.Psychological Review, 63,149–159.

Geisler,W.S.(2008).Visualperceptionandthestatisticalpropertiesofnaturalscenes.Annual Review of Psychology, 59,167–192.

Geusebroek,J.M.,&Smeulders,A.W.M.(2005).Asix-stimulustheoryforstochastictex-ture.International Journal of Computer Vision, 62(1–2),7–16.

Gibson,J.J.(1979).An ecological approach to visual perception.Boston:HoughtonMifflin.Godijn,R.,&Theeuwes, J. (2002).Programmingofendogenousandexogenoussaccades:

Evidence for a competitive integration model. Journal of Experimental Psychology: Human Perception & Performance, 28(5),1039–1054.

Goldberg,M.E.,&Hudspeth,A. J. (2000).Thevestibular system. InE.R.Kandel, J.H.Schwartz,&T.M.Jessell(Eds.),Principles of neural science(4thed.,pp.801–815).NewYork:McGraw-Hill.

Gratton,G.,Coles,M.G.,&Donchin,E.(1983).Anewmethodforoff-lineremovalofocularartifact.Electroencephalography and Clinical Neurophysiology, 55(4),468–484.

Graupner,S.-T.,Velichkovsky,B.M.,Pannasch,S.,&Marx,J.(2007).Surprise,surprise:Twodistinctcomponentsinthevisuallyevokeddistractoreffect.Psychophysiology, 44(2),251–261.

Greene, M. R., & Oliva,A. (2009). Recognition of natural scenes from global properties:Seeingtheforestwithoutrepresentingthetrees.Cognitive Psychology, 58(2),137–176.

Guestrin,E.D.,&Eizenman,M.(2006).Generaltheoryofremotegazeestimationusingthepupil center and corneal reflections. IEEE Transactions on Biomedical Engineering, 53(6),1124–1133.

Hari,R.(2004).Magnetoencephalographyinclinicalneurophysiologicalassessmentofhumancorticalfunctions.InE.Niedermeyer&F.LopesdaSilva(Eds.),Electroencephalography: Basic principles, clinical applications, and related fields(5thed.,pp.1165–1197).NewYork:Lippincott;Williams&Wilkins.

Hari,R.,&Kujala,M.V.(2009).Brainbasisofhumansocialinteraction:Fromconceptstobrainimaging.Physiological Reviews, 89(2),453–479.

Hari,R.,Salmelin,R.,Tissari,S.,Kajola,M.,&Virsu,V.(1994).Visualstabilityduringeye-blinks.Nature, 367,121–122.

Y116134.indb 297 10/10/11 2:12 PM


Hayhoe,M.,&Ballard,D.(2005).Eyemovementsinnaturalbehavior.Trends in Cognitive Sciences, 9(4),188–194.

Henderson, J. M. (2007). Regarding scenes. Current Directions in Psychological Science, 16(4),219–222.

Henderson, J. M.,Weeks, P.A., & Hollingworth,A. (1999).The effects of semantic con-sistency on eye movements during complex scene viewing. Journal of Experimental Psychology: Human Perception & Performance, 25(1),210–228.

Itti,L.,&Koch,C.(2000).Asaliency-basedsearchmechanismforovertandcovertshiftsofvisualattention.Vision Research, 40(10–12),1489–1506.

Jousmäki,V.,Hämäläinen,M.,&Hari,R.(1996).Magneticsourceimagingduringavisuallyguidedsaccadetask.NeuroReport1996,7,2961–2964.

Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8(4),441–480.

Just,M.A.,&Carpenter,P.A.(1980).Atheoryofreading:Fromeyefixationstocomprehen-sion.Psychological Review, 87(4),329–354.

Kanwisher,N.,McDermott, J.,&Chun,M.M. (1997).The fusiform facearea:Amoduleinhumanextrastriatecortexspecializedforfaceperception.Journal of Neuroscience, 17(11),4302–4311.

Karklin,Y.,&Lewicki,M.S.(2009).Emergenceofcomplexcellpropertiesbylearningtogeneralizeinnaturalscenes.Nature, 457,83–86.

Kazai,K.,&Yagi,A.(2003).Comparisonbetweenthelambdaresponseofeye-fixation-relatedpotentialsandtheP100componentofpattern-reversalvisualevokedpoten-tials.Cognitive, Affective, & Behavioral Neuroscience, 3(1),46–56.

Koffka,K.(1935).Principles of gestalt psychology.NewYork:Harcourt.Korjenevsky,A., Cherepenin,V., & Sapetsky, S. (2000). Magnetic induction tomography:

Experimentalrealization.Physiological Measurement, 21(1),89–94.Land,M.F.(2004).Thecoordinationofrotationsoftheeyes,headandtrunkinsaccadicturns

producedinnaturalsituations.Experimental Brain Research, 159(2),151–160.Lencer,R.,&Trillenberg,P.(2008).Neurophysiologyandneuroanatomyofsmoothpursuitin

humans.Brain and Cognition, 68(3),219–228.Loschky,L.C.,McConkie,G.W.,Yang,J.,&Miller,M.E.(2005).Thelimitsofvisualresolu-

tioninnaturalsceneviewing.Visual Cognition, 12(6),1057–1092.Luria,A.R.,Karpov,B.A.,&Yarbus,A.L.(1966).Disturbancesofactivevisualperception

withlesionsofthefrontallobe.Cortex, 2,202–212.Maldonado,P.E.,Babul,C.M.,Singer,W.,Rodriguez,E.,Berger,D.,&Grun,S. (2008).

Synchronizationofneuronal responses inprimaryvisual cortexofmonkeysviewingnaturalimages.Journal of Neurophysiology, 100,1523–1532.

Mannan,S.,Ruddock,K.H.,&Wooding,D.S.(1995).Automaticcontrolofsaccadiceyemovementsmadeinvisualinspectionofbrieflypresented2-Dimages.Spatial Vision, 9, 363–386.

Mulvey,F.,Villanueva,A.,Sliney,D.,Lange,R.,Cotmore,S.,&Donegan,M.(2008).Explorationofsafetyissuesineyetracking.InCommunicationbyGazeInteraction,IST-2003-511598:Retrievedfromhttp://www.cogain.org/results/reports/COGAIN-D5.4.pdf

Neumann,D.,Spezio,M.L.,Piven, J.,&Adolphs,R. (2006).Lookingyou in themouth:Abnormalgazeinautismresultingfromimpairedtop-downmodulationofvisualatten-tion.Social Cognitive and Affective Neuroscience, 1,194–202.

Norman,K.A.,Polyn,S.M.,Detre,G.J.,&Haxby,J.V.(2006).Beyondmind-reading:Multi-voxelpatternanalysisoffMRIdata.Trends in Cognitive Sciences, 10(9),424–430.

Y116134.indb 298 10/10/11 2:12 PM


Pannasch,S.,Helmert,J.R.,Velichkovsky,B.M.,Malischke,S.,&Storch,A.(2008).Eyetypinginapplication:AcomparisonoftwosystemswithALSpatients.Journal of Eye Movement Research, 2(4),1–8.

Pelli,D.G.,&Tillman,K.A.(2008).Theuncrowdedwindowofobjectrecognition.Nature Neuroscience, 11(10),1129–1135.

Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85(3),618–660.

Rémond,A.,Lesèvre,N.,&Torres,F.(1965).Etudechrono-topographiquedel’activitéoccip-italemoyennerecueilliesurlescalpchezl’hommeenrelationaveclesdéplacementsdu regard (complexe lambda) [Chrono-topographic studyofmiddleoccipital activityrecordedonthescalpinhumansinrelationtoeyemovements(lambdacomplex)].Revue Neurologique, 113(3),193–226.

Riemslag, F. C.,Van der Heijde, G. L.,Van Dongen, M. M., & Ottenhoff, F. (1988). Onthe origin of the presaccadic spike potential. Electroencephalography and Clinical Neurophysiology, 70(4),281–287.

Scholte,H.S.,Ghebreab,S.,Waldorp,L.,Smeulders,A.W.M.,&Lamme,V.A.F.(2009).BrainresponsesstronglycorrelatewithWeibullimagestatisticswhenprocessingnaturalimages.Journal of Vision, 9(4),29.

Shebilske,W.(1975).Readingeyemovementsfromaninformationprocessingpointofview.In D. Massaro (Ed.), Understanding language (pp. 291–311). NewYork:AcademicPress.

Shih,S.W.,&Liu,J.(2004).Anovelapproachto3-Dgazetrackingusingstereocameras.IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 34(1),234–245.

Skrandies,W.,&Laschke,K. (1997).Topographyofvisuallyevokedbrainactivityduringeyemovements:Lambdawaves,saccadicsuppression,anddiscriminationperformance.International Journal of Psychophysiology, 27(1),15–27.

Stratton,G.M.(1906).Symmetry,linearillusions,andthemovementsoftheeye.Psychologi-cal Review, 13(2),82–96.

Thickbroom,G.W.,&Mastaglia,F.L.(1985).Presaccadic‘spike’potential:Investigationoftopographyandsource.Brain Research, 339(2),271–280.

Tomonaga,M.,&Imura,T.(2009).Facescapturethevisuospatialattentionofchimpanzees(Pan troglodytes):Evidencefromacueingexperiment.Frontiers in Zoology, 6,14–27.

vanDiepen,P.M.J.,&d’Ydewalle,G.(2003).Earlyperipheralandfovealprocessinginfixa-tionsduringsceneperception.Visual Cognition, 10(1),79–100.

van Zoest, W., & Donk, M. (2004). Bottom-up and top-down control in visual search.Perception, 33(8),927–937.

Vaughan,J.(1982).Controloffixationdurationinvisualsearchandmemorysearch:Anotherlook. Journal of Experimental Psychology: Human Perception & Performance, 8(5),709–723.

Vauhkonen, M., Hamsch, M., & Igney, C. H. (2008).A measurement system and imagereconstructioninmagneticinductiontomography.Physiological Measurement, 29(6),445–454.

Velichkovsky,B.M.(1995).Communicatingattention:Gazepositiontransferincooperativeproblemsolving.Pragmatics and Cognition, 3(2),199–222.

Velichkovsky,B.M.(1999).Fromlevelsofprocessingtostratificationofcognition:Convergingevidencefromthreedomainsofresearch.InB.H.Challis&B.M.Velichkovsky(Eds.),Stratification in cognition and consciousness(pp.203–235).Amsterdam:JohnBenjamins.

Y116134.indb 299 10/10/11 2:12 PM


Velichkovsky, B. M. (2001). Levels of processing: Validating the concept. In M. Naveh-Benjamin,M.Moscovitch,&H.L.RoedigerIII(Eds.),Perspectives on human memory and cognitive aging: Essays in honour of Fergus I.M. Craik(pp.48–71).Philadelphia:PsychologyPress.

Velichkovsky,B.M.,Joos,M.,Helmert, J.R.,&Pannasch,S. (2005).Twovisualsystemsandtheireyemovements:Evidencefromstaticanddynamicsceneperception.InB.G.Bara,L.Barsalou,&M.Bucciarelli(Eds.),Proceedings of the XXVII Conference of the Cognitive Science Society(pp.2283–2288).Mahwah,NJ:LawrenceErlbaum.

Velichkovsky,B.M.,Pomplun,M.,&Rieser,H.(1996).Attentionandcommunication:Eye-movement-basedresearchparadigms.InW.H.Zangemeister,H.S.Stiel,&C.Freksa(Eds.),Attention and cognition(pp.125–154).Amsterdam:Elsevier.

Velichkovsky,B.M.,Rothert,A.,Kopf,M.,Dornhoefer,S.M.,&Joos,M.(2002).Towardsan expressdiagnostics for level ofprocessing andhazardperception.Transportation Research, Part F, 5(2),145–156.

Vigário,R.,Jousmäki,V.,Hämäläinen,M.,Hari,R.,&Oja,E.(1997).Independentcomponentanalysisforidentificationofartifactsinmagnetoencephalographicrecordings.InM.I.Jordan,M.J.Kearns,&S.A.Solla(Eds.),Advances on neural information processing systems(Vol.10,pp.229–235).Cambridge,MA:MITPress.

vonWartburg,R.,Wurtz,P.,Pflugshaupt,T.,Nyffeler,T.,Lüthi,M.,&Müri,R.(2007).Sizematters:Saccadesduringsceneperception.Perception, 36(3),355–365.

Wade,N.J.,&Tatler,B.W.(2005).The moving tablet of the eye: The origins of modern eye movement research.Oxford:OxfordUniversityPress.

Williams, D. E., & Reingold, E. M. (2001). Preattentive guidance of eye movements dur-ingtripleconjunctionsearchtasks:Theeffectsoffeaturediscriminabilityandsaccadicamplitude.Psychonomic Bulletin & Review, 8(3),476–488.

Wooding,D.S.(2002).Eyemovementsoflargepopulations:II.Derivingregionsofinterest,coverage,andsimilarityusingfixationmaps.Behavior Research Methods, Instruments, & Computers, 34(4),518–528.

Wooding,D.S.,Mugglestone,M.D.,Purdy,K.J.,&Gale,A.G. (2002).Eyemovementsoflargepopulations:I.Implementationandperformanceofanautonomouspubliceyetracker.Behavior Research Methods Instruments & Computers, 34(4),509–517.

Yagi,A.(1979).Saccadesizeandlambdacomplexinman.Physiological Psychology, 7(4),370–376.

Yagi,A.(1995).Eyefixation-relatedpotentialasanindexofvisualfunction.InT.Kikuch,H. Sakuma, I. Saito, & T. Tsuboi (Eds.), Biobehavioral self-regulation (Vol. 32,pp.177–181).Tokyo:Springer-Verlag.

Yanulevskaya,V.,&Geusebroek,J.M.(2009).SignificanceoftheWeibulldistributionandits sub-models in natural image statistics. Visapp 2009: Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, 1,355–362.

Yarbus,A.L.(1967).Eye movements and vision.NewYork:PlenumPress.Yuval-Greenberg,S.,Tomer,O.,Keren,A.S.,Nelken,I.,&Deouell,L.Y.(2008).Transient

induced gamma-band response in EEG as a manifestation of miniature saccades.Neuron, 58(3),429–441.

Zingale,C.M.,&Kowler,E.(1987).Planningsequencesofsaccades.Vision Research, 27(8),1327–1341.

Y116134.indb 300 10/10/11 2:12 PM

301

Electrical and functional brain imaging

Pasquale Anthony Della Rosa and Daniela PeraniDepartmentofNeuroscience,UniversitàVitaSaluteSanRaffaeleMilan,Italy

14.1 Introduction

InthefourthcenturyBC,Hippocrateswrote

Menoughttoknowthatfromnothingelsebutthebraincomesjoys,delights,laughterandsports,andsorrows,griefs,despondencyand lamentations.Andbythis,inanespecialmanner,weacquirewisdomandknowledge….Andbythesameorganwebecomemadanddelirious,andfearsandterrorsassailus….Allthesethingsweendurefromthebrainwhenitisnothealthy….Intheseways,Iamoftheopinionthatthebrainexercisesthegreatestpowerintheman.(trans.1972)

Thehistoryofourquesttounderstandthebrainiscertainlyaslongashumanhistoryitself.Intheearly1900s,WalterDandyintroducedaprocesscalledpneumoencepha-lographythatinvolveddrainingthecerebrospinalfluidfromaroundthebrainandreplacingitwithair,alteringtherelativedensityofthebrainanditssurroundings,tocauseittoshowupbetteronanX-ray.Thistechniquecarriedsignificantriskstothepatientunderinvestigation;however,thesurgicalinformationgivenbythismethodwasremarkablypreciseandgreatlyenlargedthecapabilitiesandaccuracyofneuro-surgicaltreatment.Atthebeginningofthe1970stheadventofmodernneuroimag-ing techniques brought about a radical change in the application of methods andinstrumentstoassessandquantifybrain–behaviorrelationships.

Magnetic resonance imaging (MRI) and computed tomography (CT) weredevelopedinthe1970sand1980s.Forthefirsttime,thebrainanatomyappearedin vivo with a great power in structural details. Next came SPECT and PETscans,whichallowedscientiststomapbrainfunctionbecause,unlikeMRIandCT, these scans could create more than just static images of the structures inthebrain.These functional imaging invivo techniquescanprovidephysiciansand scientists with several molecular parameters such as brain perfusion andmetabolism,receptordensity,neurotransmitters,andenzymeactivity,measure-ments only partially available in post mortem specimens before the advent ofthesetechniques.

14

Y116134.indb 301 10/10/11 2:12 PM


MRIandPETscanningestablishedamilestonefromwhichscientistswereabletodevelop functionalMRI (fMRI)providing thepossibility to“directly”observecognitive activities and quantify them. In recent years, neurophysiological andfunctionalneuroimagingtechniqueshaveprovidedmeasuresofbrainactivitythathaveincreasedourabilitytostudytheneuralbasisandthecerebralorganizationofsensorimotorandcognitivebrainfunctions.Thesetechniquesfallroughlyintotwoclasses:theelectromagneticapproachmeasuringbrainactivitydirectlybyrecordingtheelectromagneticfieldsgeneratedbycertainneuronalpopulationsandthehemo-dynamicapproachestimatingbrainactivitybydetectingblood-dependentchangesindirectly coupled with modifications in neural activity. These methods differ inanumberofaspectsamongwhicharetheprerequisitesfordetectingasignal, thehomogeneitywithwhichneuralactivityissampledfromdifferentpartsofthebrain,andmoreimportantlytherelativeaccuracyindeterminingwhen versuswhereneuralactivitytakesplace.Thesetwoapproaches,therefore,providecomplementaryviewsofneuralactivityandthedeepinsightwegainedaboutbrainfunctioninthelasttwodecadeslargelyderivesfromscientificstudiesemployingthesetwotechniques.

Inthischapter,weillustratetheessentialsofbothtypesofapproachesandhowtheymaybeemployedtomeasuretherelationshipbetweenthebrainandcognitiveactivitieswhichruleourbehaviorineverydaylife.

14.2 When does brain activity occur?

14.2.1 Electroencephalogram (EEG) and brain rhythms

Electroencephalographyisamedicalimagingtechniquethatreadsscalpelectricalactivitygeneratedbybrainstructures.Theelectroencephalogram(EEG)isdefinedas electrical activity of an alternating type recorded from the scalp surface afterbeing picked up by metal electrodes. Thus electroencephalographic measurementis a completely noninvasiveprocedure that canbe applied repeatedly to patients,healthyadults,andchildrenwithvirtuallynoriskorlimitation.Whenneuronsareactivated,localcurrentflowsareproduced.EEGmeasuresmostlythecurrentsthatflowduringsynapticexcitationsofthedendritesofmanypyramidalneuronsinthecerebralcortex.

Differencesinelectricalpotentialsarecausedbysummedpostsynapticgradedpotentialsfrompyramidalcellsthatcreateelectricaldipolesbetweensoma(bodyofneuron)andapicaldendrites(neuralbranches)andweakelectricalsignalsdetectedbythescalpelectrodesaremassivelyamplified,andstoredincomputermemory.Betweentheelectrodesandneuronallayerscurrentpenetratesthroughskin,skull,and several other layers allowingonly the electrical activitygeneratedby largepopulations of active neurons to be detected and recorded on the head surface.Duetothecapabilityofreflectingbothnormalandabnormalelectricalactivityofthebrain,EEGhasbeenfoundtobeaverypowerfultoolinthefieldofneurologyandclinicalneurophysiologyandindeedinthedomainofcognitiveneuroscience.Withrespecttothelatter,thenumberofneuronsthatdischargesynchronouslyiscaptured,inaphysiologicalsense,bytheEEG“power,”whichisassumedtobea

Y116134.indb 302 10/10/11 2:12 PM

Electrical and functional brain imaging 303

measurethatreflectsthecapacityorperformanceofcorticalinformationprocess-ing.EEGpowercanbemeasuredindifferentfrequencyranges(e.g.,alpha,beta,gamma,orthetawaves;seeFigure 14.1),however,thebest-knownandmostexten-sivelystudiedfrequencyofactivityinthehumanbrainisthenormalalpharhythm.

Alphaactivityisinducedbyclosingtheeyesandbyrelaxation,andreducedbyeyeopeningoralertingbyanymechanism(thinking,calculating).SincetheworkofBerger(1929)itwassuggestedthatvisualorothersensorytaskdemands,specificallyvisualattention,aretheprimaryfactorsthatinduceasuppressionofthealpharhythm(Mulholland,1969;Ray&Cole,1985).Anindividualbrainoperationreliesupontheformation of functionally interconnected neurons in large neuronal assemblies inordertobecarriedout.Theunderlyingmechanismisthesynchronizationofneuronscharacterizedbyarhythmicalpattern.Whenaneuronalassemblyissetuparhyth-mical increase in the totalpotentialoccurs,whereasadisruptionof theneuronalassemblyinducesadecreaseintotalpotential.

Alpha is the dominant frequency in the human scalp EEG of adults and duetoitspropertyofbeinganoscillatorycomponentofthehumanEEGithasledtothe development of techniques such as event-related desynchronization (ERD),capableofinducingchangesinEEGalphaactivityassociatedwithsensoryortask

Beta 14–13 HzAwake, normal alert

consciousness

ALPHA 9–13 HzRelaxed, calm, lucid,

not thinking

THETA 4–8 HzDeep relaxation andmeditation, mental

imagery

Delta 1–3 HzDeep, dreamless

sleep

Figure 14.1 BrainwavefrequenciesmeasuredwithEEG:betawaves(14-30Hz)areseenduring attention to task and focus; alpha waves (9-13 Hz), an idling rhythm, seen duringrelaxationandmeditation;thetawaves(4-8Hz),seeninthedreamlikestatebetweensleep&wakefulness;deltawaves(1-3Hz)occurprimarilyduringsleep.

Y116134.indb 303 10/10/11 2:12 PM


manipulations.Thephasicstructureofalphaactivityreferstothealterationsofsyn-chronization anddesynchronizationperiods and reflecting the formation anddis-ruptionofcorticalneuronalassemblies,respectively.Inthisframework,periodsofalphasynchronizationanddesynchronizationdonotindicateepisodesof“reststate”and“activework,”respectively,butaremarkersoftwotypesofcorticalprocessingequallyactive whichdifferonlyinthewaytheneighboringneuronsareused.

InusingERD(Pfurtscheller&Aranibar,1977), a typical exampleof anEEGepochmaycomefromataskinwhichsubjectswereaskedtoreadavisuallypre-sentedwordandtomakeasemanticjudgmentbyrespondingyestoaworddenotingalivingobjectandnotoaworddenotinganonlivingobject(Klimesch,Doppelmayr,Pachinger,&Russegger,1997).Awarningsignalwaspresentedimmediatelypre-cedingthewordandsubjectshadtojudgethewords.

During thefirstsecondof theepoch,namely the“reference interval,” thesub-jectsshowedpronouncedrhythmicalphaactivity.Aftersubjectswerepresentedwithmanytrials,theanticipationofthewarningsignalalreadyledalphatodesynchro-nizeevenbeforetheactualsignalappeared(seeFigure 14.2).

ThebasicprincipleformeasuringERDisthatatypicalphasicchangeoccursinthealpharhythmoverthetimecourseofatrial.Afteraresponse,thesubjectrelaxesandwaits for thenext stimulus tocomeup.During this stateof relaxedbutalertwakefulness the brain idles, highlighting a pronounced alpha activity during thereferenceintervalbeforeeachtrial.Evenbeforethewarningsignalactuallyappears,thealpharhythmbecomessuppressed,becausethesubjectpreparesandanticipatesthebeginningofthenexttrial.Ifalphasuppressionisregardedasamarkerofdesyn-chronization,itwouldindicatethatlargepopulationsofneuronsnolongeroscillateinsynchronywhileinformationisprocessed.

Thus,ERDcanbecalculatedasthepercentageofabandpowerchangewhileper-formingaspecifictaskwithrespecttoareferenceorrestinginterval(Pfurtscheller&

1 Second AlphaDesynchronization

WarningSignal

Word

Time (secs)Reference

µv

µv

µv

Elec

trod

es

µv

µv

Figure 14.2 AlphadesynchronizationmeasuredwithEEG.Atypicalphasicchangeoccursinthealpharhythmoverthetimecourseofatrialindicatingthatthealpharhythmbecomessuppressed.

Y116134.indb 304 10/10/11 2:12 PM


Aranibar,1977),andrepresentsameasurereflectingtheextenttowhichsynchronyislost.ThestandardmeasureofERDquantifiesashiftinsignalbandpowerintermsofthedifferencebetweenabaselineperiodbeforetheeventoccursandaperiodfollow-ingtheevent.Typically,theERDisevaluatedastheaverageresponseoverasetofsingletrials.OnemethodforestimatingtheERDisthepowermethodwhichallowsthecomputationofthepowerspectraldensity(PSD)ofthedata.PSDisameasureofhowpowerinasignalchangesasafunctionoffrequency.Spectralanalysisdetectsperiodicoscillations(amplitudeandfrequency)andhasbeenemployed inagreatvarietyofsignalprocessingapplications.Thismethodallowsusnotonlytocom-putethePSDforspecifictimesegments,butalsotocomparestatisticallythePSDbetweendata segments.Hencedifferences in thePSDbetween referenceperiodsversusactiveperiodscanbefoundeasily.

ByconventionanERDcorrespondstoanegativevalue(i.e.,adecreaseinpower)whereasevent-relatedsynchronization(ERS)(Pfurtscheller,1992;Klimesh,1996)refers toan increasedsignalpower. It is important tonote thatchangesof signalpowerarequantifiedonlywithrespecttothedeviationfromafixed,constantbase-line level.However, it ispossible togeneralize theconventionalERD frameworkwith respect to thechoiceof referencebymeasuring the relativedeviationof theevent-relatedpowerfromadynamicbaseline.Thedynamicreferencecanbeesti-matedfromcatchtrials,thatis,asaveragedpoweracrosstrialswithoutprocessingtheeventunderstudy.ThemainadvantageofthegeneralizedERD(gERD)measureisduetoitsabilitytoreliablystudyERDresponseseveninthepresenceofdynami-calcorticalstatesbyusingthenaturalrelaxationdynamicsoftheunperturbedEEGrhythmasreferenceobtainedfromcatchtrials.

14.2.2 Evoked potentials (ERPs)

Evokedpotentialsorevent-relatedpotentials(ERPs)aresignificantvoltagefluctu-ations resulting fromanexternalor internal stimulus.Mentaloperations, suchasthoseinvolvedinperception,selectiveattention,languageprocessing,andmemory,proceedovertimerangesintheorderoftensofmilliseconds,forwhichreasonERPsare a suitable methodology for studying cognitive processes of both normal andabnormalnature(e.g.,inneurologicalorpsychiatricdisorders).AmplitudesofERPcomponents are often much smaller than spontaneous EEG components, so theyareextractedfromasetofsinglerecordingsbydigitalaveragingofepochs(record-ingperiods)ofEEGtime-lockedtorepeatedoccurrencesofsensory,cognitive,ormotor events. The spontaneous background EEG fluctuations, which are randomrelativetothetimethestimulioccurred,areaveragedout,leavingtheevent-relatedbrainpotentials.Theseelectricalsignalsreflectonlythatactivitywhichisconsis-tentlyassociatedwiththestimulusprocessinginatime-lockedway.TheERPthusreflects,withhightemporalresolution,thepatternsofneuronalactivityevokedbyastimulus.

AnERPwaveformconsistsofaseriesofpeaksandtroughs;however,thesevolt-agedeflectionsreflectthesumofseveralrelativelyindependentunderlyingorlatent components.Itisextremelydifficulttoisolatethelatentcomponentssothattheycan

Y116134.indb 305 10/10/11 2:12 PM


bemeasuredindependently,andthisisthebiggestbarriertodesigningandinterpret-ingERPexperiments.Consequently,oneofthekeystosuccessfulERPresearchistodistinguishbetweentheobservablepeaksofthewaveformandtheunobservablelatentcomponents.

Afterextractingthesignal,researchersgenerallyfocusonsomespecificfeatureoftheresultingERPwaveform(e.g.,apeakortrough),andthisparticularfeaturethenbecomesthecomponentofinterest.AccordingtoDonchin&Heffley(1978),a“component” isapartof theERPwaveformwithadelimitedscalpdistribution(referringtotheneuronsunderlyingtheERP)andadefinedrelationshiptoexperi-mentalvariables(linkedtothecognitivefunctioncarriedoutbytheactivityofthispopulationofneurons).

TherearethreemeasurableaspectsoftheERPcomponent:amplitude,latency,and scalp distribution (Johnson, 1992). The amplitude of a component providesanindexoftheextentofneuralactivation;namelyitcaptureshowthecomponentrespondsfunctionallytoexperimentalvariables.Thelatencyreferstothepointintimeatwhichthepeakoccursandrevealsthetimingofactivation,andthescalpdis-tributiongivesinformationontheoverallpatternofactivatedbrainareasandreflectsthepatternofvoltagegradientoverthescalpatanypointintime.

Therefore,componentscanbedefinedintermsofacombinationofthespecificoperationbeingperformedandaneuroanatomicalgeneratorsite.Bythisdefinition,acomponentmayoccuratdifferenttimesunderdifferentconditions,aslongasitarisesfromthesamemoduleandrepresentsthesamecognitivefunction.Furthermorethesamecognitivefunctionmayoccurandpeakindifferentpartsofacorticalarealead-ingtochangesinitsscalpdistributionandpolarity.

Different techniques have been developed which use different approaches inordertotracethespecificmarkersofERPcomponentsaccordingtotheirtemporalandspatialcharacteristics.

TechniquessuchasPCA(principalcomponentanalysis)andICA(independentcomponentanalysis;forareviewofbothtechniquesseeDonchinandHeffley,1978)usethecorrelationalstructureofanERPdatasettodefineasetofcomponents,andthesetechniquesthereforederivecomponentsthatarebasedonfunctionalrelation-ships.Specifically,differenttimepointsaregroupedtogetheraspartofasinglecom-ponenttotheextenttheytendtovaryinacorrelatedmanner,undertheassumptionthat timepointswhichreflectacommoncognitiveprocessshouldin theoryshowanoverlappingpatternofvariation.Aswehavenotedabove,variationinERPvolt-ageacrossthescalpisattributedtovariationinthepsychologicalprocessesthatareengagedinasituation.ThepurposeofPCAandICAinthiscontextthenistoiden-tifyaspectsofthewaveformthatshowtemporalcovariationoverbothexperimentalconditionsandscalplocations.

DespitethecomplicationsarisingwhentryingtoidentifythegeneratorsitesofspecificERPcomponents,thescalpdistributionoftheERPcanaddextremelyuse-ful information to component amplitudes and latencies. Comparison of the scalpdistributionsofERPselicitedbydifferentstimuliallowustoinferwhethertwotypesofstimuliengagedifferentpatternsofneuralactivitywhich in turnmayunderliedifferent functional processes (for a review see Johnson, 1993). Hence, given the

Y116134.indb 306 10/10/11 2:12 PM


combinationofbothtemporalandspatialinformationembeddedintheERPwave-formrecordedatmanydifferentsitesoverthescalp,weareabletodefinethetem-poralcharacteristics(bothonsetandduration)ofbrainactivityspecificallyrelatedtoatypeofstimulus.

The clearestwayof representingERPdata, especiallywhen recording fromalarge number of electrodes, is with maps. Two types of maps can be created foranydataset:voltageandcurrentsourcedensity(CSD)ones.Bothtypesofmapsaretraceddirectlyfromtheoriginalamplitudedata;however,eachgivesadifferentper-spectiveofthebrain’sactivity(foracompletetutorial,seePicton,Lins,&Scherg,1995).VoltagemapsareextractedfromtheERPamplitudescollectedateachelec-trodesite,providingapictureofthesummedactivityfromallactivebrainareas,asscalp-recordedERPactivityreflectsthesummationofalltheneuralactivity,bothcortical and subcortical, during any given temporal window. On the other hand,theamplitudesarespatiallyfilteredwithanalgorithmthat“cleansup”theamountofactivityconveyedbysubcorticalanddistantcorticalareasbeforeCSDmapsaregenerated.Theresultingmapsdepictaspatiallysharpened,reference-freedisplayofpositiveandnegativecurrentdensitiesthathighlightslocal(cortical)differencesbetween generators (Nunez, 1981; Picton et al., 1995), and is particularly usefulin forminghypothesesaboutneural sources insuperficialcortex (Perrin,Pernier,Bertrand,&Echallier,1989).

Anessentialdifferencebetweenthesemappingmethodsisthattheyassignadif-ferentweighttothecontributionofsubcorticalactivity.However,itisgenerallyuse-fultocalculatebothtypesofmapsinordertopinpointandtakeintoconsiderationthedifferencesbetweenthevoltageandCSDmaps.Thisisbecause,althoughCSDmapsallowonetoinferneocorticalsources,theactivityofdeepergenerators(e.g.,subcorticalnuclei),wouldonlyberevealedinthevoltagemaps.It is important tonotethatbothtypesofmapsareemployedonlyforvisualinspectionofbrainactivityanddonothingtoquantifytheactivitypatternstheydisplayunlesstheyundergosta-tisticalanalysiswhichmayallowustoinferquantitativedifferencesbetweenthem.

MoredirecttechniquesthatallowERPsourcestobedirectlyinferredfromscalpfieldsthemselveshavealsobeendeveloped.AmongthemostadvancedtechniquesistheBrainElectricalSourceAnalysisProcedure(BESA;Sherg,Grandone,Hoke,&Romani,1990).ThisprocedureisgroundedintheassumptionthattheERPwave-formrepresentsthesummationoftheactivityofanumberofdifferentsourcesoffixedlocationwithinthebrain,andthatthesesourcescanbeappropriatelymodeledas “equivalentdipoles” ina solutionfitted to theobserveddatapattern.ABESAsolutionspecifiesthesesourcesintermsoftheirnumber,location,orientation,andtimecoursesandrelativestrengthsoftheiractivity.Suchsolutionscanbeverifiedbycomputingthescalpfieldsthattheywouldgenerateanddeterminingthegoodness-of-fitbetweenthesepredictedfieldsandthosemeasuredempirically.AnimportantfeatureoftheBESAprocedureisthatthelocationofthesourcescanbedefinedbytheexperimenterbasedonapriorianatomicalknowledgeoronthelocationofbrainactivityderivedfromacomplementarytechniquesuchasfMRIscanning.AsecondimportantadvantageofthisprocedureisthatthecontributionmadebyeachsourcetotheERPcanberegardedasanindependentERPcomponent.Thus,inprinciple

Y116134.indb 307 10/10/11 2:12 PM


thistechniqueallowsustoreduceERPdataintermsofasmallnumberofunderly-ingcomponentseachassociatedwithitsown“defined”sourceinthebrain.

However, the core limitation of analyzing ERP waveforms with the BESAapproach is that the results are constrained by a priori assumptions made by theexperimenter. Although there may be a long history and a robust grounding intheliteratureconcerningthebestorappropriatesourceofacomponentitwillalwaysremainachoiceandthereforeaseedofbiasintroducedbytheexperimenter.

The first step for adding more information and refining EEG/ERP analysesistofindareference-independentmeasure(Dien,1998)andthismaybeaccom-plished by comparing topographies (Lehmann, 1987; Michel, Murray, Lantz,Gonzalez,Spinelli,&GravedePeralta,2004.)That is, theconfigurationof theelectric field at the scalp (i.e., the topographic map) remains constant and theextent of topographic similarity or dissimilarity can be quantified and statisti-callytested.Thus,topographicanalyses(i.e.,quantificationofscalpdistributiondifferences)canbeperformedontheERPamplitudesinordertotestthesignifi-canceofthedifferencesshownbythemaps.Themostimportantfeatureofthistechniqueisthattopographicdifferencescanbetranslatedintermsofneurophysi-ology.Onlychanges in theconfigurationof theunderlying intracranial sources(giventheexclusionofartifactssuchaseyemovements,muscleactivity,etc.)caninducealterationsinthetopographyoftheelectricfieldatthescalp,althoughtheoppositewillnotverify itself (forareviewseeFender,1987andMurrayetal.,2008). Hence, qualitatively and quantitatively distinct ERP scalp topographiesreflectdifferentpatternsofneuralactivityassociatedwiththerespectiveexperi-mental conditions.Topographicdissociations could recall both the engagementofdistinctneuralpopulationsordifferencesintherelativeactivitylevelsinthemembersofacommonpopulation.Suchdifferencespotentiallystandforaneuraldoubledissociation,whichimpliesthattwoexperimentalmanipulationslieuponfunctionallydistinctcognitiveprocesses.

However,alltheseproceduresdonotsolvethe“inverseproblem”:thatis,theydonotprovideauniquesolutiontoaccountforthedistributionofscalpactivityonthebasisoftheactivityofanumberofintracranialsources.ThisissueiscommontobothEEGandMEGandisdiscussedinthenextsection.

14.3 Event-related potential (ERP) studies

14.3.1 ERP components

ERPcomponentscanbegroupedintwomainclasses.Theearly,orsensory,com-ponentsunderliesensorystimuliinallmodalitiesandareassociatedwithaseriesofdeflectionsintheERPthatarerelatedtothetransmissionofsensoryinformationfromtheperipheralsensorysystemtothecortex.Forexample,afterauditorystimuli,onecandetectresponseswithalatencylessthan50msandthesedeflectionshavebeenshowntocorrespondtotheactivationofvariousnucleiinthebrainstemthatareassociatedwiththetransmissionofauditoryinformation.Thesecomponentsare

Y116134.indb 308 10/10/11 2:12 PM


compulsoryinthesensethat theywillbeobservedineveryindividualunless thesensorysystemsinquestionareimpaired.

Thesecondclass is referred toas lateorcognitive components.Theseare thecomponents that occur later in the ERP (from about 100 ms) and are thought torepresentactivityat thelevelofthecortex.ExamplesofsuchcomponentsaretheN100,N200,andP300and theso-calledN400,a largenegativedeflection in theERPelicitedbyanomalouswords.Thenumberroughlyindicatesthetimeinmilli-secondsatwhichthecomponentoccurs.Thesecognitivecomponentscanvaryasafunctionofattention,taskrelevance,andthenatureoftheprocessingrequiredbythestimulus.Onsomeoccasionstheymayevenoccurwhenanexpectedstimulusdoesnotoccur.

However,aclear-cutboundarybetweenearlyandlatecomponentsisdifficulttotraceasmanyearlysensorycomponentshavebeenshowntobemodifiablebycogni-tivemanipulations(e.g.,attention)andmanyofthelatercognitivecomponentshavebeenshowntobeinfluencedbyphysicalattributesoftheelicitingconditions(e.g.,modalityofthestimulus).

Components serve at least three purposes. First, they represent a commondenominator that allows communication across experiments and scientific fields.Second,theyareabridgebetweenERPdataandothermeasuresofbrainactivity.Third,componentscanbeinterpretedasphysiologicalmarkersforspecificcognitiveprocesses.InordertoenhancethesensitivityofERPcomponentstoparticularcog-nitiveorsensorimotorprocesseswecanadoptspecificstrategiestoestablishclearconnectionsbetweenspecificERPcomponentsandspecificprocesses.

14.3.2 Some examples of ERP components

SpecifictolanguageprocessingseveralERPcomponentswithdifferenttemporalandspatialcharacteristicshavebeen identified indicating thatdistinctmechanismsmaymediateatleastsemanticandsyntacticprocesses(Hagoort&Brown,2000a,2000b).

The development of ERP language research began with the seminal study byKutasandHillyard(1980)onthesemanticprocessingofwrittensentences.TheseauthorsexaminedtheERPsproducedwhentheirsubjectsreadsentencesthatendedeitherwithasemanticallycongruentor incongruentword.Theyobservedacom-ponentthatpeakedabout400msaftertheonsetoftheincongruentword.Thisso-calledN400componentisabroadnegativewavedistributedoverposteriorareasofthebrain(seeFigure 14.3a).Moreoverthiscomponenthasbeenobservedunderawiderangeofconditions:indifferentlanguages,includingEnglish,Dutch,German,French,andItalian;indifferentmodalities,includingvisual,auditory,andevensignlanguage;andwithdifferentexperimentalprocedures(Kutas&VanPetten,1994).Thus,itseemsthattheamplitudeoftheN400componentprovidesapropermeasureofthedifficultyencounteredbythereaderinintegratingthelexicalelementintheprecedingcontext.

Interestingly,thepresentationofsyntacticviolationssuchasverbsubcategorizationviolationsdidnotproduceanN400component,butratheranincreaseinthepositivity

Y116134.indb 309 10/10/11 2:12 PM


withabiparietaldistribution(thecentro-posteriorareasofthebrain),severalhundredmillisecondslater.ThiseffecthasbeencalledP600(Osterhout&Holcomb,1992)orlatepositivesyntacticshift(Hagoort,Brown,&Groothusen,1993;seeFigure 14.3b).FurthermorestudiesonmemoryhaveisolatedtwotopographicallydistinctERPcorre-latesofrecognitionmemory,theparietalandmid-frontalold/neweffectsthataredisso-ciatedbyvariablesthatselectivelymodulaterecollectionandfamiliarity,respectively.

5

0

–5

0 0.5Time/s

1

N400Scal

p El

ectr

ical

Pot

entia

l/µV

1.5

5

0

–5

0 0.5Time/s

1

P600

Scal

p El

ectr

ical

Pot

entia

l/µV

1.5

Figure 14.3 (See color insert.) (a)N400component.Thiscomponenthastheformofanega-tivewavepeakingabout400msaftertheonsetofasemanticallyincongruentword.Event-relatedpotentialtracefor“Theshirtwasironed”(blueline)versus“Thethunderstormwasironed”(redline).Thekeywordisironed,andthedeviationiscalledtheN400.(ReprintedfromFriederici,2002.) (b)P600component.Thepresentationof syntacticviolationspro-ducesanincrease in thepositivityataround600msafterstimuluspresentation.Posteriorevent-relatedpotentialtracefor“Theshirtwasironed”(blueline)versus“Theshirtwasonironed”(redline).Thekeywordisironed,andthedeviationiscalledtheP600.(ReprintedfromFriederici,2002.)

Y116134.indb 310 10/10/11 2:12 PM


Ithasbeenmorethan25yearssincethefirstreportsshowedthatERPselicitedbycorrectlyclassifiedold(studied)itemsaremorepositive-goingthanthoseelicitedbycorrectlyclassifiednew(unstudied)testitems(Warren,1980).Investigatorshavemanipulatedseveralvariablesinordertoteaseapart theseERP“old/neweffects”(Yovel&Paller,2004;Duarteetal.2004).Findingsfromthesestudiessuggestedthatrecollection,whichreferstotheretrievalofepisodicinformationinresponsetoarecognitiontestitem,hasadistinctERPsignature,nowoftentermedthe“parietal”old/neweffect(seeFigure 14.4a.).Theeffectisshapedasaphasic,positive-going,parietallymaximalERPmodulationthatoriginatesaround400–500mspost-stim-ulusonsetandcommonlypeaksontheleftsideofthescalp.Itisimportanttonotethat this effect has been functionally and topographically dissociated from otherposteriorly distributed ERP effects that occur in the same timeframe, but ratherrespondtofactorssuchasstimulusprobabilityandresponseconfidence(Woodruffetal.,2006).

InastudybyRuggetal.(1998)subjectsperformedeitheradeep(sentencegen-eration)orashallow(alphabeticjudgment)taskoneachwordstudied,followedbyarecognitiontest.Theauthorsfoundthatthisposterioreffectwaselicitedexclusivelybyolditemssubjectedtodeepstudy.Unliketheposterioreffect,amid-frontaleffect

2 Left parietal

1

0µv

–1

–20 500 ms

(a)

2 Forntal1

0µv

–1–2

(b)

Figure 14.4 (See color insert.) ERPold/neweffects.(a)Grand-averageERPsfromleftpari-etalwiththeparietalold/neweffectindicatedbythepurplearrow.(b)Grand-averageERPsfrom frontal electrodes with the mid-frontal old/new effect indicated by the green arrow.Repeatedmeasuresanalysesofvariance(ANOVAs)focusedonmeanamplitudesfrom300to500msec.DataadaptedfromCurran(2000).

Y116134.indb 311 10/10/11 2:12 PM


wasobservedonlyforitemscorrectlyjudgedasold(seeFigure 14.4b).Ruggetal.(1998), based on this evidence, proposed that the mid-frontal old/new effect wasanERPsignatureof familiarity-drivenrecognitionwhereas recognitionofdeeplystudiedwords reliedonboth recollection (indexedby theparietalold/neweffect)andfamiliarity(indexedbythemid-frontalold/neweffect);shallowlystudiedwordswereinsteadidentifiedlargelyonthebasisoftheirfamiliarity.

14.3.3 A glimpse of MEG

Asmentionedabove,theelectricalactivityofactivecellsinthebrainproducescur-rentsspreadingthroughthehead.Furthermore,thesecurrentsproducemagneticfieldswhichcanbemeasuredabovethescalpsurfacethroughmagnetoencephalography(MEG).Theamplitudeofthemagneticfieldsproducedspontaneouslybythehumanbrainareoftheorderpicotesla(morethanonemilliontimessmallerthantheearth’smagneticfield).Theseverylowfieldstrenghtscanberecordedbyso-calledsuper-conductingquantuminterferencedevices(SQUIDS)throughaspecialpick-upcoilcooleddowntosuperconductivitybyliquidhelium.Themagneticfield,incontrasttotheelectricpotential,hasadirection,usuallyillustratedbymagneticfieldlines.Acurrentflowingalongastraightlineproducescircularmagneticfieldlinesthatareconcentricwithrespecttothecurrentline.Thecoilpicksuponlythestrengthofthemagneticfieldinthedirectionperpendiculartothecoilareaandexploitingaspecificquantumeffectthemagneticfluxthroughthecoilcanbemeasured.ThisimportantfeaturemakesMEGinsensitivetocurrentsourcesthataredirectedtowardorawayfromthescalp(likethetopofacorticalgyrus)referredtoas“radial”sources.MEGmeasuresinsteadtangentialsources,whichareparalleltothescalp.

MEGcanbeconsideredasthemostdirectcorrelateofonlinebrainprocessingobtainablenoninvasivelybecauseitcombinesbothhightemporalprecisionwithahighspatialresolution.MEGcanbeemployedtodisclosebothevokedandongoingbrainactivity.Evokedresponsestovarioussensorystimulicanbewithdrawnfromthebackgroundactivityusingtime-lockedaveraging.Furthermoretheresponsescanbeusedastoolstostudyfunctionsofbrainareasatamillisecondtimescaleoncetheneuralgeneratorsofevokedresponsesareisolated.

OneoftheadvantagesofexploitingthepropertiesofMEGcanbesummarizedintheexamplethatfollows.GiventhefactthatattentionwasknowntoaffectauditoryevokedEEGpotentials(Näätänen,1992),itwasonlyMEGthatallowedustogaindeeper insight on the functioning of the supratemporal auditory cortex, enablingustocharacterizetheroleofthesensory-specificcortexduringauditoryattention.ResultsfromoneMEGstudyshowedthatthemaximumeffectofattentionoccurredaround200msandthesupratemporalauditorycortexprocessedinadifferentwaylow-orhigh-pitchtoneswhenattentionwasorientedtoeitheroftheminamonau-ralcontext(Rifetal.,1991).However,anotherMEGstudyhasfoundevidenceforanearlymodulationofcorticalactivityasearlyas20–40msafterthesoundonsetduetoattention,whichmaysomehowrecallanearlyselectionofinputforfurtherprocessing.Theseresultssuggestedthatmodulationstartsevenearlierandspecifi-callywhenattention isdirected tooneearduringbinauralstimulation thanwhen

Y116134.indb 312 10/10/11 2:12 PM


onefrequencychannelisattendedduringstimulationexclusivelydirectedtooneear(Woodruffetal.,1993).Moreover,MEGallowstrackingofactivationoftheauditorycortices separately inbothhemispheres (Mäkelä et al., 1993),whereas the corre-spondingEEGsignalsfromtheleftandrightauditorycorticessummateatthescalpmidlineandsourcemodelingisrequiredinordertodisentanglethem.

AnotherexampleofthecapabilityheldbyMEGtolocalizegeneratorsofERPcom-ponentsissummarizedbelow.InthesectionabovewediscussedtheN400ERPcom-ponent elicited by semantically incongruent sentences. This component elicits asustainedfieldshowingthesamescalptopographyforseveralhundredmilliseconds.Inthesametimewindow,aslightlydifferentwaveformmorphologyappearstodenoteMEGresponsestovisualwordsbetween200and500msafterword-onset.TheN400timewindowhasbeenexploredwithMEGusingtheclassicalN400paradigmwhereexpectationsforthefinalword,insentencesareviolated(“Hespreadthewarmbreadwithsocks”;Halgrenetal.,2002;forareviewseePylkkanenandMarantz,2003).Forexample,Heleniusandcolleagues(1998)manipulatedthepredictabilityofvisualsentencefinalwordstolocalizethesourceoftheN400ERPandpinpointedseveralareasresponsivetosemanticcongruity,althoughadistinctclusterwaslocalizedinthesuperiortemporallobe,neartheauditorycortex(seeFigure 14.5).

14.3.4 The inverse problem in EEG and MEG

“The deduction of neuronal currents from the measured external electricpotentialormagneticfielddistribution” is referred toas“the inverseproblem”and unfortunately does not have a unique solution (Hämäläinen et al,. 1993).However,somesolutionsaremorelikelythanothersasEEGandMEGsignalsare not due to spurious distributions of electric neuronal generators. They areruledinsteadbyelectrophysiologicalandneuroanatomicalconstraintsandobeythelawsofelectrodynamicsaswell.Hence,wecanreachanapproximatesolu-tion to the inverse problem by establishing constraints derived from anatomy,physiology,andotherdata,inordertolimitthenumberofpossiblesolutionstoasmalleramount,andestimatethesourcetopographiesinaconfidentmanner.Low-resolution brain electromagnetic tomography (LORETA; Pascual-Marqui,

He spread the warmbread with socks

N400(400 mm)

Prob

abili

ty

101

102

Figure 14.5 TheN400componentlocalizedwithMEG.ThesourceoftheN400ERPinMEGappearsadistinctclusterinthesuperiortemporallobe,neartheauditorycortex.

Y116134.indb 313 10/10/11 2:12 PM


Michel,&Lehmann,1994;Pascual-Marqui,1999)isafunctionalimagingmethodbasedontheelectrophysiologicalandneuroanatomicalconstraintsinwhichthecortexcanbemodeledasacollectionofvolumeelements(voxels)inthedigitizedatlasinordertotracesources(seeFigure 14.6).TheLORETAinversesolution(whichisconsistentwiththeEEG/MEGmeasurements)correspondstothe3-Ddistributionofelectricneuronalactivitythathasmaximumsimilarity(i.e.,maxi-mumsynchronization), in termsoforientationandstrength,betweenneighbor-ingneuronalpopulations (representedbyadjacentvoxels).ThecorticalsurfacecanbemodeledasasetofsurfaceelementswithagivenaprioriorientationandLORETAcanlookuptothisneuroanatomicalconstraint,andderivetheinversesolutionthatmaximizesonlythesynchronizationofstrengthbetweenneighbor-ing neuronal populations. According to calculations reviewed by Hämäläinenetal.(1993),atypicalclusterofneuronsmustcoveratleast40to200mm²ofthecorticalsurface.

Figure 14.6 SourcemodelingusingLORETA.TheLORETAinversesolutioncorrespondstothe3Ddistributionofelectricneuronalactivitymodeledonthecortex.

Y116134.indb 314 10/10/11 2:12 PM


TheLORETAsolutionhasbeenvalidatedbyconfrontingtopographieswithdataobtained using other imaging techniques or from lesion studies and intracranialrecordingswherethesourcesitesandsignalwaveformshavebeenlargelyconfirmed(see,forreview,Pascal-Marqui,Esslen,Kochi,andLehmann,2002).

To conclude, MEG and EEG trace the same neuronal activation patterns, justfrom a different perspective, and provide complementary information in order toreconstructthebrain’scurrentdistributionsasaccuratelyaspossible.

14.4 Where do things happen in the brain?

14.4.1 PET and (f)MRI

Event-relatedpotentialsandmagneticfieldscanhelpuslearn“when”dfferentthingshappenin thebrain tocharacterize thespatiotemporalpatternofbrainactivity inbasicallyallareasofneuroscience,buthowdowepreciselyfindoutabout“where”?PETandfMRIcanlocalizewithhighspatialresolutionregionsofactivationwhilethebrainisinvolvedinagivenmentaltask,whereasERPsandMEGcanhelpusindefiningthetimecourseoftheseactivations.PETandfMRIbothfallunderthecategoryofactiverecordingtechniquesdueto thewaysinwhichthey“interfere”withnormalbrainmetabolism.

PETallowsustomeasureindetailthefunctioningofdistinctareasofthehumanbrainwhileapatientoranormalsubjectiscomfortable,conscious,andalert.BeforetheadventofthePETscanner,wecouldonlyinferwhatwentonwithinthebrainfrompostmortemsoranimalstudies.Nowweareabletouseradioactivitytomeasurecerebralmetabolism,cerebralbloodflow,orneuroreceptorsandneurotransmitterstostudythechemicalprocessinvolvedintheworkingofhealthyordiseasedhumanbrainsinawaypreviouslyimpossible.PETcangiveusadetailedpictureofthebrainatworkinvivoallowingscientistsanddoctorstodepicthowitfunctions.

Functionalmagnetic resonance imagingdoesnot require radioactivityandcangiveveryhighspatialresolutionimagesreflectingneuronalactivation.Alargepartofthemetabolicneedsofthebrainreliesuponoxygenandglucosetomaintainsyn-aptic activity and the intake of these energy resources to sustain a constant neu-ralactivityoccurs through thecerebralbloodflow.Thisobservation is importantbecauseitprovesthatchangesinregionalbloodflowcanprovidemoremeaningfulparametersthandirectmeasurementsofcerebralmetabolismandfMRIallowsustopinpointlocalchangesinregionalcerebralbloodflowthatmayoccurafterphysi-ologicalstimulation.

14.4.2 PET—positron emission tomography

PET measures emissions from radioactively labeled chemicals that have beeninjectedintothebloodstreamandusesthedatatoproducetwo-orthree-dimensionalimagesofthedistributionofradiochemicalsthroughoutthebrain.Italsomeasuresthe distribution of particular organic molecules and compounds (e.g., water, glu-cose, neurotransmitters, enzymes) in the brain. However, organic molecules and

Y116134.indb 315 10/10/11 2:12 PM


compounds are not directly detectable because they do not emit electromagneticsignals.Therefore,amachinecalledacyclotronisusedto“label”thesenaturalbodycompounds,suchasglucoseorwater,withsmallamountsofradioactivitytocon-stitutepositron-emittingisotopesofthesemolecules.Thelabeledcompound,whichiscalledaradiotracer,istheninjectedintothebloodstream,whichcarriesittothebrainpassingthroughtheblood-brainbarrierandafterashorttimeperiodtheiso-topesaredispersedthroughoutthebrain.

Theisotopes,alongwiththeblood,flowtotheareasofthebrainwiththehighestmetabolicneedsthatareassumedtobethemostactiveatthegivenpointintime.Accordingtotheirchemicalcharacteristics,theycanalsospecificallybindtoneu-roreceptorsorcompetewithenzymaticactivity.Ataspecifictimepointthenucleioftheisotopesdecay,givingoffpositrons.Whenapositronmeetsanelectron,thecollision(annihilation)producestwogammarayshavingthesameenergy,butgoinginoppositedirections.Thegammaraysleavethepatient’sbodyandaredetectedbysensorsinthePETscanner.Thegreatertheactivationofanarea,themoregammaraysoriginatefromthatarea.

Acomputerusesthedatagatheredbythesensorstoconstructmulticoloredtwo-orthree-dimensionalimagesthatshowwherethecompoundactsinthebrain,provid-ingacomplexpictureofthesubject’sorpatient’sbrain.ThefinalPETimagesshowareasofdifferentcolor tones,eachshaderepresentingdifferentmolecularparam-etersoftheunderlyingbrainstructures(seeFigure 14.7).Usingdifferentcompounds,PETcanshowbloodflow,oxygenandglucosemetabolism,neurotransmission,anddrugconcentrationsinthetissuesofthebrain.Bloodflowandoxygenandglucosemetabolismreflecttheamountofbrainactivityindifferentregionsandenablescien-tiststolearnmoreaboutthephysiologyandneurochemistryoftheworkingbrain.

ForexampleinarecentstudyGaribottoetal.(2008)usedfluorine-18-fluorodeox-yglucosepositron(18FDG)tomeasureglucoseconsumptionandassesstheimpactofeducationandoccupationonbrainglucosemetabolism(rCMRglc)measuredinaMCI(amnesticmildcognitiveimpairment)patientsandinaverylargesampleofsubjectswithprobableAlzheimerdisease(pAD).Theresultsshowedasignificantassociationbetweenhighereducation/occupationand lower rCMRglc inposteriortemporoparietalcortexandprecuneusinpADandaMCIpatientswhichconvertedtoAD,andnocorrelationinaMCInonconvertersandhealthycontrols.ThismeansthatpADandaMCIconverterswithhighereducation/occupationhadamoresevererCMRglcreductionthantheoneswithlowereducation/occupation,giventhesamecognitiveimpairment,.Thisstudysuggeststhathighlyintelligentoreducatedindi-vidualsappeartobeabletocopebetterwiththeonsetofdementia.

InanotherstudyTettamantietal.(2005)used(11C)racloprideandpositronemis-sion tomography tomeasuremodulationsof thedopaminergicsystem inducedbyphonologicalorsyntacticprocessing.Theyfoundthatthelevelofaccuracyinphono-logicalprocessingsignificantlycorrelatedwith tracerbindingpotential in the leftcaudatenucleusand the speed inphonologicalprocessing significantlycorrelatedwithtracerbindingpotentialintheleftputamen.Thesefindingsshowthatthestria-taldopaminergicsystemplaysanessentialroleingrammaticalprocessesthatformthecoreofhumanlanguage.

Y116134.indb 316 10/10/11 2:12 PM


Whenthemeasurementsrefertowatermoleculesandregionalcerebralbloodflow(rCBF),labeledoxygen(15O2;oxygenfromwhichanelectronhasbeenremovedfromitsatomtocreateanunstablecompoundcapableofemittingpositrons)isusedinso-called“tripleoxygen”PETactivationstudies.Inthiscase,areasofhigherbloodflowwillhavealargeramountofradioactivetracer,andthusemitastrongersignal.Bloodflow is an indirectmeasureof local synaptic activity and cognitive functionshavebeenstudiedmainly investigating increases in rCBF(cerebralactivation) linked totheperformanceofcognitivetasks(e.g.,language,memory,attention).Thesestudieshaveusedlabeledoxygen(15O2)astheradiotracerintheformofwater(H2

15O)astheadvantageofthistraceristhatitdecaysinashorttime(approximately2min).Thismeansthatitispossibletomakeseveralscans(upto16approximately)duringasinglesession,thusenablingresearcherstostudydifferentconditionswhilepatientscarryoutdifferenttasks.AtthebeginningofeachPETscan,asmallamountoflabeledwaterisinjectedintoasubject’svein,whileherorhisheadisplacedinsidethePETscanner.Afterabout30seconds,thetracerstartsappearinginthebrainandthenext30secondsconstitutethe“criticalwindow”whenradiationreachesitspeakinthebrain.ImagesofrCBFareobtainedduringthiscriticalwindow.Usingdataanalysismethods,a3-Drepresentationofthebrainisobtainedintheformofamappingofradioactivitydistri-bution,whichindicatescerebralactivitylinkedtothecognitivetask.

Figure 14.7 (See color insert.) ExampleofaPETimageshowingareasofdifferentcolortones, each shade representing different molecular parameters of the underlying brainstructures.

Y116134.indb 317 10/10/11 2:12 PM


PET activation studies commonly employ blocked experimental paradigmsbecauselongintervalsoftime(30secondsormore)arerequiredtocollectsufficientdatatoyieldagoodimage.Inablockdesign,differentconditionsintheexperimentarepresentedasseparateblocksoftrials(i.e.,readingawordinoneblockandnam-ingapictureinadifferentblock),witheachblockrepresentingonescanduringanexperiment.Moreover,inordertoobservetheneuraleffectofsomebrieflyoccurringpsychologicalprocess(e.g.,theactivationduetoaflashingredlight)inaPETexperi-mentthestimulusneedstobepresentedrepeatedlyduringablockoftrialssothatactivationsaccumulateovertherecordingintervalofascan.Onecouldthencom-paretheactivationsinthisscantoanappropriatecontrolscan(baseline)inwhichtheeventdidnotoccur.Inthismanneractivationsrelatedtoslowlychangingfactorssuchasatask-setcanbecaptured.

For example, Perani, Schnur, Tettamanti, Gorno-Tempini, Cappa, and Fazio(1999)investigatedthefunctionalcorrelatesofthesemanticprocessingofpicturesandwordsduringpictureandwordmatching tasks, inwhich theyassessedcere-bralactivationwithPETcomparingtheprocessingofstimulibelongingtodifferentsemanticcategories(animateandinanimate).Regionalcerebralbloodflowwasmea-suredbyrecordingthedistributionofcerebralradioactivityfollowingtheinjectionof labeled oxygen. These results highlighted different brain networks subservingtheidentificationoflivingandnonlivingentitiesindicatingacrucialroleoftheleftfusiformgyrusintheprocessingofanimateentitiesandoftheleftmiddletemporalgyrusfortoolsforbothwordsandpictures.

14.4.3 Magnetic resonance imaging (MRI) and fMRI

UnlikePET,MRIusesmagneticfieldsandradiowavestoproduceimagesofbrainstructureswithoutinjectingradioactivetracers.Patientsorvolunteersareplacedinalargecylindricalmagnetwhichcreatesamagneticfieldaroundtheirhead.Radiowavesaresentthroughthemagneticfieldandsensorsreadthesignalswhicharefedtoacomputer thatuses the information tocreatehigh-dimensional imagesof thebrain.MRIallowsimagingofbothsurfaceanddeepbrainstructureswithahighdegreeofanatomicaldetailanddetectingminutechanges in thesestructures thatoccurovertime.Intheearly1990sOgawaandLee(1990),Ogawa,Lee,Kay,andTank(1990)andKwongetal.(1992)developedatechniquecalledfunctionalbrainimagingthatenablesustouseMRItoimagethebrainasitfunctions.FunctionalMRIrelieson themagneticpropertiesofblood toenablescientists tosee imagesofbloodflowinthebrainasitisoccurring.ThistechniquetakesadvantageoftheBOLD(bloodoxygenleveldependent)effecttotracechangesinthelocalbloodoxy-genationofstimulatedbrainareasinhumans(Bandettini,Wong,Hinks,Tikofsky,&Hyde,1992;Kwongetal.,1992;Ogawaetal.,1992).

Thegeneration of images that are related to bloodflowexploits a property ofhemoglobin thathasdifferentmagneticpropertieswhen it isnotcarryingoxygen(deoxyhemoglobin)thanwhenitiscarryingoxygen(oxyhemoglobin,orsimplyhemo-globin).TherationaleisthatmoredeoxygenatedbloodinanareacausesadecreaseinBOLDsignalwhichissensitivetochangesintheconcentrationofdeoxygenated

Y116134.indb 318 10/10/11 2:12 PM


hemoglobinacrossthevesselsofthebrain.Neuralactivityinsteadisaccompaniedbyincreasedbloodflow,whichreducestheconcentrationofdeoxygenatedhemoglobinandproducesarelativeincreaseinsignal(Ogawa&Lee,1990;Logothetis,2002).Therefore,fMRImeasureschangesinbloodflowinanindirectmanner,throughtheeffectsofchangingpercentagesofdeoxyhemoglobin(Howseman&Bowtell,1999)producingimagesofbrainactivityasfastaseverysecond(seeFigure 14.8).

WearenowabletocharacterizethenatureoftheBOLDcontrastinabetterwayasitdependsnotonlyonbloodoxygenationbutalsooncerebralbloodflowandvol-ume,representingacomplexresponsecontrolledbyseveralparameters(Boxermanetal.,1995;BuxtonandFrank,1997;Ogawa,Menon,Kim,&Ugurbil,1998;seeforareviewLogothetis,2003).However,evidencecomingfromsimultaneousfMRIandelectrophysiological recordingspoints to adirect relationshipbetween theBOLDcontrast mechanism and the neural responses elicited by a stimulus (Logothetis,Pauls,Augath,Trinath,&Oeltermann,2001).Thisspecificfeatureallowsresearch-ers todeterminewithgreatprecisionwhenbrain regionsbecomeactiveandhowlongtheyremainactivewhenpatientsorvolunteersperformataskorareexposedtodifferenttypesofstimuli.

14.4.4 Neuroimaging experiments with fMRI

Functionalneuroimagingexperiments are commonlybuilt to investigate complexpsychologicalprocesses.Theunderlyingassumptionisthatthesecomplexfunctionscanbebrokendownandimaginedascombinationsofconstituentelementaryopera-tions.Therationalebehindneuroimagingistohighlightbrainactivationscommonly

Deoxyhemoglobin

Stimulated tissue

Boldflow

Unstimulated tissue

Oxyhemoglobin

Figure 14.8 TheBOLDeffect.MoredeoxygenatedbloodinanareacausesadecreaseinBOLDsignalwhichissensitivetochangesintheconcentrationofdeoxygenatedhemo-globinacrossthevesselsofthebrain.Neuralactivityinsteadisaccompaniedbyincreasedbloodflow,whichreducestheconcentrationofdeoxygenatedhemoglobinandproducesa relative increase in signal (Ogawa & Lee, 1990; Logothetis, 2002). Therefore, fMRImeasures changes in blood flow in an indirect manner, through the effects of chang-ingpercentagesofdeoxyhemoglobin(Howseman&Bowtell,1999)producingimagesofbrainactivity.

Y116134.indb 319 10/10/11 2:12 PM


associatedwithelementarypsychologicalprocessesandanalyzewhatcombinationsofelementaryprocessesareinvolvedinacognitivetask(fordetailsseeFrackowiak,Friston,Frith,Dolan,&Mazziotta,1997).

PETandfMRIprovideenoughspatialandtemporalresolutiontomeasureneuralactivitysimultaneouslyintheentirebrainallowingustomakemeaningfulconclu-sionsabouttherolesspecificbrainregionsplayduringcognition.Itneedstobeout-linedthatbeforestartingafMRIneuroimagingexperiment,severalimportantissuesmustbetakenintoconsideration(e.g.,aspecificexperimentalhypothesis,appropri-atemethods)andaboveallthenatureofthetaskchosentodrawspecifictypesofinferencefromthestudy.Thetaskmustbedesignedinordertoclearlyteaseaparttheneuralandpsychologicalprocessesunderinvestigationfromtheinfluenceofinter-feringvariables.Nuisancevariablesmaybeneuralprocessesunrelatedtotheques-tionofinterestortechnologicalartefactsorphysiologicalartefacts(e.g.,heartrate,respiration,etc.).Totheextentthatnuisancevariablesinfluencethebrainactivationsinatask,theywillharmandblurtheinterpretabilityofthedata.Onceanadequatetaskischosen,importantdecisionsneedtobemadeconcerningtherightframetofitaroundthetask,namely,theexperimentalparadigm.Thepredominantparadigmsforanalyzingtask-relatedchangesusingfMRIareblockedparadigms,suchasthoseusedforPETactivationstudies(seeabove)andso-calledevent-relateddesigns.

Many fMRI studies have used blocked paradigms, in which subjects alternatebetweenperforminganactive(i.e., readingaword)andacontrol task(i.e.,a restconditionwhere theyare asked to lookat afixationcross) for short timeperiods(e.g.,30s),andthentheimagesacquiredduringtheactivetaskblocksarestatisti-callycomparedtotheimagesacquiredduringthecontroltaskblock.OneadvantageofusingablockeddesignwithfMRIisthatitoffersmorestatisticalpowertodetectachange.

Forexample,Canessaetal.(2005)usedablock-designparadigmtoinvestigatetheeffectsofthecontentofstimulionbrainactivationunderlyingdeductivereasoningwhilesubjectsweresolvingtworeasoningtasks:oneversionofthetaskdescribedanarbitraryrelationbetweentwoactions(descriptive:“Ifsomeonedoes…,thenhedoes…”),whereastheotherdescribedanexchangeofgoodsbetweentwopersons(social-exchange:“Ifyougiveme…,thenIgiveyou…”).Whencomparedtocon-trol,bothtasksactivatedfrontalmedialcortexandleftdorsolateralfrontalandpari-etalregions,confirmingthemajorroleofthelefthemisphereindeductivereasoning.Althoughthetworeasoningconditionswereidenticalinlogicalform,thetaskwithstimulireferringtocontextsandsituationswithasocialcontentwasalsoassociatedwithrightfrontalandparietalactivations,mirroringtheleft-sidedactivationscom-montobothreasoningtasks.Theseresultssuggestthattherecruitmentoftherighthemisphereisdependentonthecontentofthestimulipresented.

Theevent-relatedparadigmhasbeendeveloped to takeadvantageof the rapiddata-acquisition capabilities of fMRI. This technique allows us to create imagesoftheneuralactivityrelatedtospecificstimuliorcognitiveeventswithinatrialbyspacingeventswithaspecific time interval (usually14–16s;Aguirre,Zarahn,&D’Esposito,1998;Dale&Buckner,1997)inwhichthehemodynamicresponsetoasinglestimulusreturnstobaselinebeforetheonsetofthenextstimulus.Signals

Y116134.indb 320 10/10/11 2:12 PM


fromindividualtrialsofthesametask(i.e.,readingsinglewords)canbeaveragedtogether,inordertoidentifythetimecourseofthehemodynamicresponsewithinatrial.Thisparadigmmakesitpossibletorandomizetrialsfromdifferentconditions(i.e., presenting words that refer to animals or tools intermixed), which is essen-tialforcertaintasksandavoidshabituationeffects.Furthermore,activityrelatedtoselectedtypesoftrialsmaybeisolatedinblockswheredifferenttypesoftrialsoccurenablingustodrawinferencesonprocessesthatoccuronlyonsometrials.

The recent advent of “rapid” event-related fMRI techniques has permittedresearcherstoperformexperimentsinwhichsuccessiveeventscanbepresentedwithatimeintervalbetweenthemasshortas750ms(Dale,1999).Notethatrapidevent-relatedfMRIallowsustominimizetheeffectsoffatigueorboredomthatmayoccurwhenlongintervalsseparatethestimuli.Thistypeofdesignislimitedbythespeedoftheunderlyinghemodynamicresponsetoaneuralevent,whichpeaks5–8safterthatneuralactivityhaspeaked.Thus,ifthetimeintervalisnotvariedfromtrialtotrialtheneuraleventswouldoccurtoorapidlytobesampledeffectively.

Accordingtothisframework,Canessaetal.(2008)presentedsubjectswithpic-turesshowingpairsofmanipulableobjectsandaskedwhethertheobjectswithineachpairwereusedwiththesamemanipulationpattern(actionknowledgecondition)orinthesamecontext(functionalknowledgecondition)usinganevent-relateddesign.Directcomparisonsshowedactionknowledge,relativetofunctionalknowledge,toactivatealeftfrontoparietalnetwork,whereasthereversecomparisonyieldedactiva-tionsintheretrosplenialandthelateralanteriorinferotemporalcortex.Theauthorsinterpretedtheseresultsassupportingthehypothesisof theexistenceofdifferenttypesofinformationprocessingintheinternalorganizationofsemanticmemory.

14.5 Statistical analysis of neuroimaging measurements

14.5.1 Functional specialization

Severalstatisticalmethodsareavailablefortheanalysisofneuroimagingdata.Manyfunctionalimagingstudiesusethesubtractionmethod,whichcomparesthepatternofbrainactivationintwodifferentconditions.Themaindeterminantsofthecontrastarethetaskandthematerials.Forexample,inthecaseoftheCanessaetal.(2008)studyinvestigatingsemanticmemory,thetaskiskeptconstant,whereasthecontrastedmaterialsreflectthedimensionofinterest(inthepresentcasetypeofknowledgebuttypicallyitcouldbecategorymembershipormodalityofstimuluspresentation).(SeeCappa,2008,forareviewonimagingstudiesofsemanticmemory.)

Data inneuroimagingexperimentsare in theformofamatrixofsignal inten-sityvaluesineachregionofthebrainexpressedinvoxels(thesmallestdistinguish-ablebox-shapedpartofathree-dimensionalimage).StandardunivariateanalysesoffMRIdataareemployedtoperformstatisticaltestsindependentlyoneachvoxeltoidentifysignificantactivationsinthebrain.Thisapproachinherentlyemphasizesthefunctionallocalizationofcognitivefunctionsoverafunctionalintegration,whichisdiscussedinthenextsection.

Y116134.indb 321 10/10/11 2:12 PM


Once a task is designed and data are collected, the statistical analysis of thedata consists of two important steps: pre-processing of the images and statisticalanalysisoftheresultingactivations(Friston,Holmes,Poline,Price,&Frith,1995a;Frackowiaketal.,1997).Preprocessingreferstoimageprocessinginwhichthevari-ousimagesinasetofdataarepreparedtobefedintoastatisticalanalysisofpos-sibledifferences.Briefly,theimagesmustbealignedtocorrectforheadmotionandfollowingalignmenttheyareoftennormalizedtoastandardbraintemplatesothatresultsfromseveralsubjectscanbeusedforcomparisonwithotherstudiesandinfer-encescanbeextended to thepopulationof interest.Finally imagesaresmoothedwithakernelinordertogivethenoiseintheimagesamoreGaussiandistribution.Followingthesepreprocessingstages,statisticaltestsareperformedonthedata.Anumberofstatisticalstrategiesareavailabletoanalyzethedataobtainedinfunc-tionalactivationstudies.Inprinciple,themeansignalintensitycanbecomparedonavoxelwisebasisbetweenimagesacquiredduring“ConditionA”versus“Rest”(or“ConditionB”).ThesignificanceofanyobserveddifferencecanthenbetestedusingasimpleStudent’st-test.Themagnitude(r)andsignificancelevel(p-value)canthenbedescribedonavoxelwisebasisandusedtobuildcoloroverlaymaps,inwhich“activatedpixels”areidentifiedincoloroverlaidonagrayscalesourceimage,usu-allyanormalizedtemplateimage(seeFigure 14.9).

Researchersprevalentlyemployunivariatetechniquestolocalizetheregionsofactivationbyextractingmeaningful signals and reducingnoise. In this approach,thequestioniswhetherthereisactivationinaspecificvoxelandmostofthedatais

Functional Segregation Functional Integration

Figure 14.9 Statistical approach to neuroimaging measurements. Functional specializa-tionisthecorrespondencebetweenacognitivefunctionandaspecificarea(orareas)inthebrain.Functionalintegrationreferstotheinteractionsamongspecializedareasorneuronalpopulations.

Y116134.indb 322 10/10/11 2:12 PM


regardedasnoise.However,asinthecaseofERPs,someresearchershaveimple-mentedmultivariatetechniquessuchasICAandPCA(seesectiononevokedpoten-tialsandChapter8ofthisbookforfurtherdetailsonthesetechniques),showingthatmeaningfulinformationcouldbeextractednotfromeachvoxelbutfromactivationpatternsacross theentiresetofvoxels(Haynes&Rees,2005;Kamitani&Tong,2005).Incontrasttoaunivariateapproach,themultivariateapproachregardsmostofthedataasmeaningfulsignals.

ManyfMRIexperimentsarecommonlycarriedouttoidentifyactivevoxelsinadataset,however,todeterminemeaningfulactivationthresholdsinfunctionalmag-neticresonanceimagingparadigmsiscomplicatedbyseveralfactors.Theseincludethetime-seriesnatureofthedata,theinfluenceofphysiologicalrhythms(e.g.respi-ration),andvariationsintroducedbytheexperimentaldesign.

Acommonwayofdeterminingsignificanceofastatisticalhypothesistestistoestablishthesignificancelevel(usuallydenotedbya)ortypeIerrorrateofthetest.ThetypeIerrorratecanbeusedtoestablishathresholdasitstandsfortheprobabil-itythat,ifthevoxelistrulynotactive,itsstatistictestwouldsurpassthethreshold,leadingtotheincorrectconclusionthatitisinsteadactive.Thisruleresultsinalargenumber of false positives or voxels declared active when they are truly not. Thereasonforthisproblemisthattherearemultipleindividualvoxelhypothesesbeingtestedcommonlyreferredtoas“themultiplicityproblem”;itoccurswhenmultiplehypothesistestsarecarriedoutatthesametimeandthepossibilityoferrorsoccur-ringoneachofthesetestsmustbetakeninconsideration(Hochberg&Tamhane,1987;Miller,1981;WestfallandYoung,1993).

Theinterpretationofsignificanceinthiscaseishamperedbytheenormousamountofmultiplecomparisonsbeingmadebetweenvoxelsandclearlyevokessomeformofcorrectiontothesimplet-statistic.ThesimplestapproachistheBonferronicorrec-tionmethod,whichapproximatestheactualprobabilityofNcomparisonsasNtimesthep-valueatanyonevoxel.Theuncorrected(orvoxelwise)p-valuesindicatetheproportion ofvoxelsintheimagethatwilllightupbychance,andthecorrected(orfamilywise)p-valuesindicatetheprobability offalsepositivevoxelsoccurringany-whereintheimage(Friston,Holmes,Worsley,Poline,Frith,&Frackowiak,1995b;forhowtodesignaneuroimagingexperiment,seeAbutalebi&DellaRosa,2008).

Regardlessofwhichspecificmethodsareusedtoanalyzethedataandtocreateacorrectedstatisticalparametricmapof thebrain,onecannotentirelyavoid thisproblemasthedichotomizationofeachindividualvoxelasactiveornotactivedoesnotprovideanexhaustiveexplanationforthemajorityofscientificquestionsinvesti-gatingthebrainanditscomplexfunctions.

14.5.2 Functional integration

Inthepastdecadefunctionalneuroimaginghashighlysucceededinprovingfunc-tional segregationas theunderlyingprincipleoforganization in thehumanbrainby revealing the link between a cognitive function and a specific area (or areas)in the brain. Functional segregation is commonly established by identifying thepresence of activation foci in statistical parametricmaps that are directly related

Y116134.indb 323 10/10/11 2:12 PM


tospecificsensorimotororcognitiveprocessesmanipulatedexperimentally.Newerapproacheshaveinsteadfocusedontheintegrationoffunctionallyspecializedareas,defining neurophysiologcal activations in terms of distributed changes (Friston,Frith,Liddle,&Frackowiak,1993a;seeFigure 14.9).

Functionalintegrationreferstotheinteractionsamongspecializedareasorneu-ronalpopulationsand to thedegree towhich these interactionsaredrivenby theexperimentalcontext.Functionalspecializationandintegrationarecomplementaryapproaches,asthelattercanbeinferredonlyinthecontextoftheformer.Functionalspecializationimpliestheidentificationofregionallyspecificeffects inducedbyashiftwithinstimuliortaskconditions.Functionalintegrationisinferredinsteadbyaccountingforthecorrelationsamongactivityindifferentbrainareasorbyfindingacausallinkbetweentheactivityinoneareainrelationtoactivityinanotherarea(Fristonetal.,1993a,1993b;McIntoshandGonzalez-Lima,1994;McIntosh,Grady,Ungerleider,Haxby,Rapoport,&Horwitz,1994).Twoaspectsoffunctionalinter-activitycanbecharacterized:functionalandeffectiveconnectivity(Friston,1994).Twobrainregionsareassumedtobefunctionallyconnectedifactivitywithinbothiscorrelated;correlationscanoccurastheresultoftheassociationbetweenactivitiesfrommanysourceswhichdoesnotnecessarilyimplythattheinteractionbetweentwoareas ismeaningful in termsofconnectivity (e.g., stimulus-evoked transientsintwoneuronalpopulationsthatarenotconnected,orthemodulationoftwocorti-calareasbyacommonsubcortical input).Functionalconnectivitysimply reflectsobserved correlations without providing any direct insight concerning how thesecorrelationsaremediated.Oncefunctionalconnectivityallowsustoestablishinte-grationwithinadistributedsystem,effectiveconnectivityisusedtoinvestigatethenatureof this integration.Effectiveconnectivity examines thedirect influenceofaneuralsourceonasecond.Onekeyaspectofeffectiveconnectivity is that it isalwaysgroundedinanapriorimodelthatdefinesthenatureoftheinfluencethatoneneuronalsystemexertsoveranother.Thisapproachattemptstodisentanglespuriouscorrelations from those mediated by direct or indirect neuronal interactions. Theparameters(usuallytheconnectionstrengths)ofthemodelarethenextractedandarerecognizedasthoseallowingthemodeltoemulate,ascloselyaspossible,theobservedregionalactivities(orinterregionalcorrelations).

Thus,functionalconnectivitydoesnotnecessarilyimplyacausallink,whereaseffective connectivity does. There is a fundamental distinction between simplyobservingcorrelatedactivityanddemonstratingeffectiveconnectivity in termsofthe influence one neuronal system exerts over another in relation to some modelofneuronalinteractions.However,aspointedoutbyFriston(seeLee,Harrison,&Mechelli,2003),effectiveconnectivityismodeldependent,whereasfunctionalcon-nectivity is not. The application of multivariate statistical analyses to fMRI datahasallowedustodisplayclearpicturesofthefunctionalandeffectiveconnectionswithincorticalnetworks.

Structuralequationmodeling(SEM).isastatisticalapproachenablingthechar-acterizationof thedirectionalityof influencebetweenregionsofanetwork.WithSEM,anapriorimodelcomposedof the regions (or ‘‘nodes’’)and theallowable

Y116134.indb 324 10/10/11 2:12 PM


connectionsofinfluencebetweenthemisdefinedandpresumedtobedrivenbythetask.Thestrengthsoftheseconnections(i.e.,pathcoefficients)thatbestfitthemodeltotheobservedcovariancestructurebetweenregionsinthedataarethenestimated.

However,SEMisnotabletotakeintoconsiderationthetemporalchangesinthesignalsacrossregionsandthereforeignoresrelativetimingasafactorinfluencingsignaldirectionality.Tworecenttechniques—dynamiccausalmodeling(DCM)andgrangercausalitymapping(GCM)—overcometheseweaknessesbylimitingaprioriassumptionsaboutdirectionalityandrelyingupontemporalmeasuresacrossregionstoestablisheffectiveconnectivity.

DCM models respond at the neural level by accounting for the hemodynamicparametersofeachregionincludedinanapriorianatomicalnetwork.Thedynam-icsoftheseneuralresponsesarethenfedtoalgorithmsaimedatestimatinghowthecouplingbetweenregionsinthesystemismodulatedbydifferentcognitivecontextsorinputs.Mechelli,Price,Friston,&Ishai(2004)usedDCMtoanalyzefMRIdataspecifictobothavisualperceptiontaskandavisualimagerytask.ADCManalysisonactiveregionssoughttodetectchangesincouplingbetweenregionsinducedbythetaskconditions.Theyconcludedthatduringvisualperception,bottom-upsignalsfromearlyvisualareasspreadtocategory-specificvisualregions(e.g.,fusiformfacearea(FFA)forfaceperception).

GCMisanothertechniqueexplicitlyassessingdirectionalityofsignalsincorticalnetworks.Specificallyitidentifiescausalinteractionsbetweentworegionsbydeter-miningtheextenttowhichthetimeseriesofareaXpredictsthefuturetimeseriesofareaY.Thistechniqueisgroundedinthehypothesisthatnetworkdynamicsevolveovertimeinapredictablemannerthereforewecanmeasureeffectiveconnectivitybetweentworegionsinaconfidentway.Roebroeck,Formisano,andGoebel(2005)appliedGCMtoacomplexvisuomotortaskbyusinganareaoftheprefrontalcortex(PFC)asa“seed”region.TheyhighlightedasignificantconnectionfromthePFCtotheparietalcortex,indicatingadirectinfluenceofthePFCinguidingbehavioralperformance.TheadvantageofGCMoverSEMandDCMisthatitdoesnotrequireanapriorianatomicalnetworkandcanthusbeusedtopinpointareasthroughoutthebrainthatshowsystematiclinkswitharegionofinterest.

14.5.3 A glimpse of structural imaging (VBM and DTI)

Inrecentyears,anumberoftechniqueshavebeendevelopedtodisplayneuroana-tomicaldifferencesinvivo usingMRIimages.Thesetechniquescanbebroadlyclas-sifiedintothosethatlookformacroscopicdifferencesinbrainshapeandthosethataimatidentifyingdissimilaritiesinthelocalstructureofbraintissueafteraccount-ingformacroscopicdifferences.Theformer,suchasdeformation-basedmorphom-etry(DBM),translatetheanatomyofanyindividualbrainintermsofdeformationfieldswhichallowthematchingofeachbrain toastandard reference.The latter,which include voxel-based morphometry (VBM), compare different brains on avoxel-by-voxelbasisafterthedeformationfieldshavebeenusedtospatiallynormal-izetheimages(Wrightetal.,1995).Thesensitivityofeachtechniquefordetecting

Y116134.indb 325 10/10/11 2:12 PM


majororminordifferenceswithinbrainstructureschangeseventhoughbothenableustoexaminethewholebraininanobjectivemanner.Wherethereareglobalpat-ternsofdifference,multivariateapproachessuchasDBMaremorepowerfulasthecovariancesbetweendifferentstructurescanbetakenintoconsideration.Incontrast,massunivariateapproachessuchasVBMaremoresensitiveforisolatingsmall-scaleregionaldifferencesingrayorwhitematter.

Forexample,inonestudy,Brambatietal.(2004)performedaninvivoanatomicstudyofgraymattervolumeinagroupoffamilialdyslexicindividuals,usingvoxel-based morphometry. Focal abnormalities in gray matter volume were observedbilaterallyintheplanumtemporale,inferiortemporalcortex,andcerebellarnuclei,suggestingthattheunderlyinganatomicabnormalitiesmayberesponsiblefordefec-tivewrittenlanguageacquisitioninthesesubjects.InasecondstudybySilanietal.(2005),voxel-basedmorphometrywasusedtoassesstheconsistencyamongbrainand morphometry data, and functional imaging addressed with a PET activationstudy (Paulesu et al., 2001) which revealed a common patternof reduced activa-tionduringreading tasks in the left temporalandoccipital lobes insubjectswithdevelopmental dyslexia. The authors provided evidence that altered activationobservedwithinthereadingsystemwasassociatedwithaltereddensityofgrayandwhitematterofspecificbrainregionssuchastheleftmiddleandinferiortemporalgyriandtheleftarcuatefasciculussupportingtheviewthatdyslexiaisassociatedwithbothlocalgraymatterdysfunctionandwithalteredconnectivityamongphono-logicalareaswithinthelanguagesystem.

Anotherrecentstructuralimagingtechniquethatisbecomingincreasinglypopu-lar inbrain researchandclinicalpractice isdiffusion tensor imaging (DTI).DTIgivesusthepossibilitytovisualizeandtrackwhitematterfascicliintwoandthreedimensions(seeFigure 14.10).ThisMRI-basedmethodology(Basseretal.,1994)exploitsthetranslationaldisplacementofwatermoleculeswithinwhitematterbun-dleswhichismadevisiblethroughdiffusionMRImeasurements(LeBihan,1995;Basseretal.,1994;Basser,1995;Basser&Jones,2002;Basser&Pierpaoli,1996).Watermolecules’motion(diffusion)ismuchfasteralongthewhitematterfibersthanperpendiculartothem(Basseretal.,1994;Basser,1995;Basser&Pierpaoli,1996,1998).Therefore,“diffusionanisotropy,”namelythedifferencebetweenthesetwomotions(parallelandperpendiculartothefibers)isthebasisofDTI(see,fordetails,Basser&Jones,2002).

Thistechniquehasbeenusedtodefinethewhitematterarchitectureofnormalbrainsaswellasthepreservedintegrityofdiseasedbrains(multiplesclerosis,stroke,aging,dementia,schizophrenia,etc.).ForexampleVBMandDTIweresuccessfullycombinedinastudybyBorronietal.(2008)tomeasureselectivestructuralchangesinearlycorticobasaldegenerationsyndrome(CBDS)andtoevaluatethestructuralcorrelatesoflimbapraxia,akeyfeatureofCBDS.ThisapproachtoinvivostructuralanatomycombinedVBMandDTI,thelatterdescribingpatternsofwhitemattercon-nectionsbetweencorticalareas,withneuropsychologicaldataprovidingnewevidenceofgraymatterandfibertractabnormalitiesinearly-phasediseaseandcontributingtoclarifyingtheneuralbasisofaneuropsychologicaldeficit(apraxia)inCBDS.

Y116134.indb 326 10/10/11 2:12 PM


14.6 Final remarks

Inthepasttwodecadesalargebodyoffunctionalneuroimagingstudieshasbeendevoted to the investigationofcognitivefunctionsboth in the intacthumanbrainandinpatients.Thestudiesapplyingthesetechniqueshavenotonlyconfirmedtheanatomicalknowledgegainedfromearlyneuropsychologicallesionstudies,buthavealsofoundthatcognitivefunctionsappeartobesubtendedbymoreextendedpat-ternsofactivityinthebrainandarealsolessfixedtospecificareasaspreviouslyhypothesized.All these techniqueshaveopenedanumberofnewperspectives intheunderstandingof thebrain-function relationship andarewidelyused to char-acterizethespatiotemporalpatternofbrainactivityinbasicallyallareasofneuro-science.NewconceptsandmethodsconstantlyariseandlatestdevelopmentsincludethecombinationofEEG/MEGwithfunctionalmagneticresonanceimaging(PETandfMRI),forcombiningthesuperiorspatialresolutionofthelatterwiththebettertemporalresolutionoftheformer.

References

Abutalebi, J., & Della Rosa, P.A. (2008). Imaging technologies. InWei & Moyer (Eds.),Blackwell guide to research methods in bilingualism.Oxford:Blackwell.

Aguirre,G.K.,Zarahn,E.,&D’Esposito,M.D. (1998).ThevariabilityofhumanBOLDhemodynamicresponses.NeuroImage, 8,360–369.

Figure 14.10 Example of DTI image. DTI enables visualization and characterization ofwhitematterfascicliintwoandthreedimensions.

Y116134.indb 327 10/10/11 2:12 PM


Bandettini,P.A.,Wong,E.C.,Hinks,R.S.,Tikofsky,R.S.,&Hyde,J.S.(1992).TimecourseEPIofhumanbrainfunctionduringtaskactivation.Magnetic Resonance Medicine,25,390–397.

Basser,P.J.(1995).Inferringmicrostructuralfeaturesandthephysiologicalstateoftissuesfromdiffusion-weightedimages.NMR in Biomedicine,7–8,333–344.

Basser,P.J.,&Jones,D.K.(2002).Diffusion-tensorMRI:Theory,experimentaldesignanddataanalysis—Atechnicalreview.NMR in Biomedicine,15,456–467.

Basser,P.J.,Mattiello,J.,&LeBihan,D.(1994).MRdiffusiontensorspectroscopyandimag-ing.Biophysical Journal,66,259–267.

Basser, P. J., & Pierpaoli, C. (1996). Microstructural and physiological features of tissueselucidatedbyquantitative-diffusion-tensorMRI.Journal of Magnetic Resonance Series B, 111,209–219.

Basser,P.J.,&Pierpaoli,C.(1998).AsimplifiedmethodtomeasurethediffusiontensorfromsevenMRimages.Magnetic Resonance in Medicine, 39,928–934.

Berger,H.(1929).UberdasElektroenkephalogrammdesMenschen,Archiv fur Psychiatrie und Nervenkronkheim,87,527–570.

Borroni,B.,Garibotto,V.,Agosti,C.,Brambati,S.M.,Bellelli,G.,Gasparotti,R.,Padovani,A.,&Perani,D.(2008).Whitematterchangesincorticobasaldegenerationsyndromeandcorrelationwithlimbapraxia.Archives of Neurology, 65,796–801.

Boxerman,J.L.,Bandettini,P.A.,Kwong,K.K.,Baker,J.R.,Davis,T.L.,Rosen,B.R.,&Weisskoff,R.M.(1995).TheintravascularcontributiontofMRIsignalchange:MonteCarlomodelinganddiffusion-weightedstudiesinvivo.Magnetic Resonance Medicine,34,4–10.

Brambati,S.M.,Termine,C.,Ruffino,M.,Stella,G.,Fazio,F.,Cappa,S.F.,&Perani,D.(2004).Regionalreductionsofgraymattervolumeinfamilialdyslexia.Neurology,63,742–745.

Buxton,R.B.,&Frank,L.R.(1997).Amodelforthecouplingbetweencerebralbloodflowand oxygen metabolism during neural stimulation. Journal of Cerebral Blood Flow Metabolism,17,64–72.

Canessa,N.,Borgo,F.,Cappa,S.F.,etal.(2008).Thedifferentneuralcorrelatesofactionandfunctionalknowledgeinsemanticmemory:AnFMRIstudy.Cerebral Cortex,18,740–751.

Canessa,N.,Gorini,A.,Cappa,S.F.,Piattelli-Palmarini,M.,Danna,M.,etal.(2005).Theeffectofsocialcontentondeductivereasoning:AnfMRIstudy.Human Brain Mapping,26,30–43.

Cappa,S.F., (2008). Imagingstudiesofsemanticmemory.Current Opinion in Neurology,21(6),669–675.

Curran,T.(2000).Brainpotentialsofrecollectionandfamiliarity.Memory and Cognition,28,923–938.

Dale,A.(1999).Optimalexperimentaldesignforevent-relatedfMRI.Human Brain Mapping,8,109–114.

Dale,A.M.,&Buckner,R.L.(1997).Selectiveaveragingof individual trialsusingfMRI.Human Brain Mapping,5,329–340.

Dien,J.(1998).Issuesintheapplicationoftheaveragereference:Review,critiques,andrec-ommendations.Behavioral Research Methods Instrumentation Computation,30,34–43.

Donchin,E.,&Heffley,E.F.(1978).Multivariateanalysisofevent-relatedpotentialdata:Atutorialreview.InOtto,D.(Ed.),Multidisciplinary perspectives in event-related brain potentials research(pp.555–572).Washington,DC:U.S.EnvironmentalProtectionAgency.

Y116134.indb 328 10/10/11 2:12 PM


Duarte,A.,Ranganath,C.,Winward,L.,Hayward,D.,&Knight,R.T.(2004).Dissociableneuralcorrelates for familiarityand recollectionduring theencodingand retrievalofpictures.Brain Research & Cognitive Brain Research,18,255–272.

Fender, D. H. (1987). Source localisation of brain electrical activity. In Gevins,A. S., &Remond,A. (Eds.). Handbook of electroencephalography and clinical neurophysiol-ogy, Vol. 1: Methods of analysis of brain electrical and magnetic signals(pp.355–399).Amsterdam:Elsevier.

Frackowiak, R. S. J., Friston, K. J., Frith, C. D., Dolan, R. J., & Mazziotta, J. C. (1997).Human brain function.SanDiego:AcademicPress.

Friederici,A.D.(2002).Towardsaneuralbasisofauditorysentenceprocessing.Trends in Cognitive Sciences,6(2),81.

Friston, K. J. (1994). Functional and effective connectivity in neuroimaging:A synthesis.Human Brain Mapping,2,56–78.

Friston,K.J.,Frith,C.D.,Liddle,P.F.,&Frackowiak,R.S.J.(1993a).Functionalconnec-tivity:Theprincipalcomponentanalysisoflarge(PET)datasets.Journal of Cerebral Blood Flow Metabolism,13,5–14.

Friston,K.J.,Frith,C.D.,&Frackowiak,R.S.J.(1993b).Time-dependentchangesineffec-tiveconnectivitymeasuredwithPET.Human Brain Mapping,1,69–80.

Friston,K.J.,Harrison,L.,&Penny,W.(2003).Dynamiccausalmodelling.NeuroImage,19,1273–1302.

Friston,K.J.,Holmes,A.P.,Poline,J.-B.,Price,C.J.,&Frith,C.D.(1995a).Detectingacti-vationsinPETandfMRI:Levelsofinferenceandpower.NeuroImage,4,223–235.

Friston,K.J.,Holmes,A.P.,Worsley,K.J.,Poline,J.-B.,Frith,C.D.,&Frackowiak,R.S.J.(1995b).Statisticalparametricmapsinfunctionalimaging:Agenerallinearapproach.Human Brain Mapping,2,189–210.

Garibotto,V.,Borroni,B.,Kalbe,E.,Herholz,K.,Salmon,E.,Holtoff,V.,Sorbi,S.,Cappa,S.F.,Padovani,A.,Fazib,F.,&Peroni,D.(2008).EducationandoccupationasproxiesforreserveinaMCIconvertersandAD:FDG-PETevidence.Neurology,71,1342–1349.

Hagoort, P., & Brown, C. M. (2000a). ERP effects of listening to speech: Semantic ERPeffects.Neuropsychologia,38,1518–1530.

Hagoort,P.,&Brown,C.M.(2000b).ERPeffectsoflisteningtospeechcomparedtoreading:TheP600/SPStosyntacticviolationsinspokensentencesandrapidserialvisualpresen-tationeffects,Neuropsychologia,38,1531–1549.

Hagoort,P.,Brown,C.M.,&Groothusen, J. (1993).The syntacticpositive shift (SPS) asan ERP-measure of syntactic processing, Language and Cognitive Processing, 8,439–483.

Halgren,E.,Dhond,R.P.,Christenson,N.,VanPetten,C.,Marinkovic,K.,Levine,J.D.,&Dale,A.M.(2002).N400-likemagnetoencephalographyresponsesmodulatedbyseman-ticcontext,wordfrequency,andlexicalclassinsentences.NeuroImage,17,1101–1116.

Hämäläinen, M., Hori, R., Ilmoniemi, R. J., Knuutila, J., & Lounasmaa, O. V. (1993).Magnetoencephalography—theory, instrumentation, and applications to non-invasivestudiesoftheworkinghumanbrain.Review of Modern Physics,65,413–497.

Haynes,J.D.,&Rees,G.(2005).Predictingtheorientationofinvisiblestimulifromactivityinhumanprimaryvisualcortex.Nature Neuroscience,8,686–691.

Helenius,P.,Salmelin,R.,Seivice,E.,&Connolly,J.F.(1998).Distincttimecoursesofwordandcontextcomprehensioninthelefttemporalcortex.Brain,121,1133–1142.

Y116134.indb 329 10/10/11 2:12 PM


Hippocrates.(n.d.)Functionsofthebrain.AdaptedfromFrancisAdams(Trans.),The Genuine Works of Hippocrates(1972).Retrievedfromhttp://www.humanistictexts.org/hippocrates.htm.

Hochberg,Y.,&Tamhane,A.C.(1987).Multiple comparison procedures.NewYork:Wiley.Howseman,A.,&Bowtell,R.W.(1999).Functionalmagneticresonanceimaging:imaging

techniquesandcontrastmechanisms.Philosophical Transactions of the Royal Society of London B,354,1179–1194.

Johnson, R., Jr. (1992). Event-related brain potentials. In Litvan, I., & Agid, Y. (Eds.), Progressive supranuclear palsy: Clinical and research approaches (pp.122–154).NewYork:OxfordUniversityPress.

Johnson,R.,Jr.(1993).OntheneuralcorrelatesoftheP300componentoftheevent-relatedpotential.Psychophysiology,30,90–97.

Kamitani,Y.,&Tong,F.(2005).Decodingthevisualandsubjectivecontentsofthehumanbrain.Nature Neuroscience,8,679–685.

Klimesch, W. (1996). Memory processes, brain oscillations and EEG synchronization.International Journal of Psychophysiology,24,61–100.

Klimesch,W.,Doppelmayr,M.,Pachinger,T.,&Russegger,H.(1997).Event-relateddesyn-chronizationinthealphabandandtheprocessingofsemanticinformation.Cognitive Brain Research,6,83–94.

Kutas, M., & Hillyard, S.A. (1980). Reading senseless sentences: Brain potentials reflectsemanticincongruity.Science,207,203–205.

Kutas,M.,&VanPetten,C.(1994).Psycholinguisticselectrified:event-relatedpotentialinves-tigations.InGernsbacher,M.A.(Ed.),Handbook of psycholinguistics(pp.83–143).SanDiego:AcademicPress.

Kwong,K.K.,Belliveau,J.W.,Chesler,D.A.,Goldberg,I.E.,Weisskoff,R.M.,Poncelet,B.P.,Kennedy,D.N.,Hoppel,B.E.,Cohen,M.S.,&Turner,R.(1992).Dynamicmag-netic resonance imagingofhumanbrainactivityduringprimarysensorystimulation.Proceedings of the National Academy of Sciences USA,89,5675–5679.

LeBihan,D.(Ed.).(1995).Diffusion and perfusion MRI.NewYork:Raven.Lee,L.,Harrison,L.M.,&Mechelli,A.(2003).Areportofthefunctionalconnectivitywork-

shop,Dusseldorf2002.NeuroImage,19,457–465.Lehmann,D.(1987).Principlesofspatialanalysis.InGevins,A.S.,&Remond,A.(Eds.),

Handbook of electroencephalography and clinical neurophysiology, Vol. 1.Methods of analysis of brain electrical and magnetic signals(pp.309–354).Amsterdam:Elsevier.

Logothetis, N. K. (2002). On the neural basis of the BOLD fMRI signal. Philosophical Transactions of the Royal Society of London B Biological Science,357,1003–1037.

Logothetis, N. K. (2003).The underpinnings of the BOLD functional magnetic resonanceimagingsignal[Review].Journal of Neuroscience,23(10),3963–3971.

Logothetis,N.K.,Pauls,J.,Augath,M.,Trinath,T.,&Oeltermann,A.(2001).Neurophysiologi-calinvestigationofthebasisofthefMRIsignal.Nature,412(6843),150–157.

Mäkelä, J.P.,Ahonen,A.,Hämäläinen,M.,Hari,R., Ilmoniemi,R.,Kajola,M.,Knuutila,J., Lounasmaa, O.V., McEvoy, L., Salmelin, R., Salonen, O., Sams, M., Simola, J.,Tesche,C.,&Vasama,J.P.(1993).Functionaldifferencesbetweenauditorycorticesofthetwohemispheresrevealedbywhole-headneuromagneticrecordings.Human Brain Mapping,1,48–56.

McIntosh,A.R.,&Gonzalez-Lima,F.(1994).Structuralequationmodelinganditsapplica-tiontonetworkanalysisinfunctionalbrainimaging.Human Brain Mapping,2,2–22.

Y116134.indb 330 10/10/11 2:12 PM


McIntosh,A.R.,Grady,C.L.,Ungerleider,L.G.,Haxby,J.V.,Rapoport,S.I.,&Horwitz,B.(1994).NetworkanalysisofcorticalvisualpathwaysmappedwithPET.Journal of Neuroscience,14,655–666.

Mechelli,A., Price, C. J., Friston, K. J., & Ishai,A. (2004).Where bottom-up meets top-down:Neuronalinteractionsduringperceptionandimagery.Cerebral Cortex,14,11,1256–1265.

Michel,C.M.,Murray,M.M.,Lantz,G.,Gonzalez,S.,Spinelli,L.,&GravedePeralta,R.(2004).EEGsourceimaging.Clinical Neurophysiology,115,2195–2222.

Miller,R.G.,Jr.(1981).Simultaneous statistical inference,2nded,NewYork:Springer.Mulholland, T. (1969). The concept of attention and the electroencephalographic alpha

rhythm.InEvans,C.R.,&Mulholland,T.B.(Eds).Attention in neurophysiology: An International Conference(pp.100–127).London:Butterworths.

Murray,M.M.,Brunet,D.,&Michel,C.M.(2008).TopographicERPanalyses:Astep-by-steptutorialreview.Brain Topography,20,249–264.

Näätänen,R.(1992).Attention and brain function.Hillsdale,NJ:Erlbaum.Nunez,P.L.(1981).Electric fields of the brain.NewYork:OxfordUniversityPress.Ogawa,S.,&Lee,T.M.(1990).Magneticresonanceimagingofbloodvesselsathighfields:In

vivoandinvitromeasurementsandimagesimulation.Magnetic Resonance Medicine,16,9–18.

Ogawa,S.,Lee,T.M.,Kay,A.R.,&Tank,D.W.(1990).Brainmagneticresonanceimagingwithcontrastdependentonbloodoxygenation.Proceedings of the National Academy of Sciences USA,87,9868–9872.

Ogawa,S.,Menon,R.S.,Kim,S.G.,&Ugurbil,K.(1998).Onthecharacteristicsoffunc-tional magnetic resonance imaging of the brain. Annual Review of Biophysics and Biomolecular Structures,27,447–474.

Ogawa,S.,Tank,D.W.,Menon,R.,Ellermann,J.M.,Kim,S.G.,Merkle,H.,&Ugurbil,K.(1992).Intrinsicsignalchangesaccompanyingsensorystimulation:functionalbrainmappingwithmagnetic resonance imaging.Proceedings of the National Academy of Sciences USA,89,5951–5955.

Osterhout,L.,&Holcomb,P.J. (1992).Event-relatedbrainpotentialselicitedbysyntacticanomaly,Journal of Memory and Language,31,785–806.

Pascual-Marqui, R. D. (1999). Review of methods for solving the EEG inverse problem.International Journal of Bioelectromagnetism,1(1),75–86.

Pascual-Marqui,R.D.,Esslen,M.,Kochi,K.,&Lehmann,D.(2002).Functionalimagingwithlow resolutionbrainelectromagnetic tomography (LORETA):Review,newcompari-sons,andnewvalidation.Japanese Journal of Clinical Neurophysiology,30,81–94.

Pascual-Marqui,R.D.,Michel,C.M.,&Lehmann,D.(1994).Lowresolutionelectromagnetictomography:Anewmethodforlocalizingelectricalactivityinthebrain.International Journal of Psychophysiology,18,49–65.

Paulesu,E.,Demonet,J.F.,Fazio,F.,McCrory,E.,Chanoine,V.,Brunswick,N.,etal.(2001).Dyslexia:Culturaldiversityandbiologicalunity.Science,291,2165–2167.

Perani,D.,Schnur,T.,Tettamanti,M.,Gorno-Tempini,M.,Cappa,S.F.,&Fazio,F.(1999).Wordandpicturematching:APETstudyofsemanticcategoryeffects.Neuropsychologia,37,293.

Perrin,P.,Pernier,J.,Bertrand,O.,&Echallier,J.F.(1989).Sphericalsplinesforscalppoten-tialandcurrentdensitymapping.Electroencephalography in Clinical Neurophysiology,72,184–187.

Y116134.indb 331 10/10/11 2:12 PM


Pfurtscheller,G. (1992).Event-related synchronization_ERS:Anelectrophysiologicalcor-relate of cortical areas at rest. Electroencephalography in Clinical Neurophysiology,83,62–69.

Pfurtscheller,G.,&Aranibar,A.(1977).Event-relatedcorticaldesynchronizationdetectedbypowermeasurementofscalpEEG.Electroencephalography in Clinical Neurophysiology,42,817–826.

Picton,T.W.,Lins,D.O.,&Scherg,M.(1995).Therecordingandanalysisofevent-relatedpotentials.InBoller,F.,&Grafman,J.(Eds).Handbook of neuropsychology(Vol.10,pp.3–73).Amsterdam:Elsevier.

Pylkkanen, L., & Marantz,A. (2003). Tracking the time course of word recognition withMEG.Trends in Cognitive Science,7,187–189.

Ray,W.J.,&Cole,H.W.(1985).EEGalphaactivityreflectsattentionaldemands,andbetaactivityreflectsemotionalandcognitiveprocesses.Science,228,750–752.

Rif, J., Hari, R., Hämäläinen, M. S., & Sams, M. (1991).Auditory attention affects twodifferent areas in the human auditory cortex. Electroencephalography in Clinical Neurophysiology,79,464–472

Roebroeck,A.,Formisano,E.,&Goebel,R.(2005).MappingdirectedinfluenceoverthebrainusingGrangercausalityandfMRI.NeuroImage,25,230–242.

Rugg,M.D.,Mark,R.E.,Walla,P.,Schloerscheidt,A.M.,Birch,C.S.,&Allen,K.(1998).Dissociation of the neural correlates of implicit and explicit memory. Nature, 392,595–598.

Sherg,M.,Grandori,F.,Hoke,M.,&Romani,G.L.(1990).Fundamentalsofdipolesourcepotentialanalysis.(pp.40–69).Basel:S.Karger.

Silani,G.,Frith,U.,Demonet,J.F.Fazio,F.Perani,D.,&Price,C.etal.(2005).Brainabnor-malitiesunderlyingalteredactivation indyslexia:Avoxelbasedmorphometrystudy.Brain, 128(Pt10),2453–2461.

Tettamanti,M.,Moro,A.,Messa,C.,Moresco,R.M.,Rizzo,G.,Carpinelli,A.,Matarrese,M.,Fazio,F.,&Perani,D.(2005).Basalgangliaandlanguage:Phonologymodulatesdopaminergicrelease.NeuroReport,16(4),397–401.

Warren, L. R. (1980). Evoked-potential correlates of recognition memory. Biological Psychology,11,21–35.

Westfall, P.H.,&Young,S.S., (1993).Resampling-based multiple testing: Examples and methods for p-value adjustment.NewYork:Wiley.

Williamson,S.J.,&Kaufman,L.(1981).Biomagnetism.Journal of Magnetic Materials,22,129–202.

Woldorff,M.etal.(1993)Modulationofearlysensoryprocessinginhumanauditorycortexduringauditoryselectiveattention.Proceedings of the National Academy of Sciences USA,90,8722–8725.

Woodruff,C.C. et al. (2006).Electrophysiological dissociationof theneural correlates ofrecollectionandfamiliarity.Brain Research,1100,125–135.

Wright, I. C., McGuire, P. K., Poline, J. B., Travere, J. M., Murray, R. M., Frith, C. D.,Frackowiak,R.S .J.,&Fristan,K. J. (1995).Avoxel-basedmethod for the statisti-calanalysisofgrayandwhitematterdensityappliedtoschizophrenia.NeuroImage,2,244–252.

Yovel,G.,&Paller,K.A.(2004).Theneuralbasisofthebutcher-on-the-busphenomenon:Whenafaceseemsfamiliarbutisnotremembered.Neuroimage,21,789–800.

Y116134.indb 332 10/10/11 2:12 PM


Appendix: General Notation

[11C]Raclopride: Positron-emitting selective dopamine 2 (D2) and dopamine 3(D3)receptorantagonist

15O:Oxygenisotope18FDG:Fluorine-18-fluorodeoxyglucosepositronAD:AlzheimerdiseaseaMCI:AmnesticmildcognitiveimpairmentBESA:BrainelectricalsourceanalysisprocedureBOLD:BloodoxygenleveldependentCBDS:CorticobasaldegenerationsyndromeCSD:CurrentsourcedensityCT:ComputedtomographyDBM:Deformation-basedmorphometryDCM:DynamiccausalmodelingDTI:DiffusiontensorimagingEEG:ElectroencephalogramERD:Event-relateddesynchronizationERPs:Event-relatedpotentialsERS: Event-relatedsynchronizationFDG-PET:F-18-fluorodeoxyglucosepositronemissiontomographyFFA:FusiformfaceareafMRI:FunctionalmagneticresonanceimaginggERD:GeneralizedERDGCM:GrangercausalitymappingH215O:15O-labeledwaterICA:IndependentcomponentanalysisLORETA:Low-resolutionbrainelectromagnetictomographyMEG:MagnetoencephalographyMRI:MagneticresonanceimagingpAD:ProbableAlzheimerdiseasePCA:PrincipalcomponentanalysisPET:PositronemissiontomographyPFC:PrefrontalcortexPSD:PowerspectraldensityrCBF:RegionalcerebralbloodflowrCMRglc:RegionalcerebralmetabolicrateforglucoseSEM:StructuralequationmodelingSPECT:SinglephotonemissioncomputedtomographySQUIDS:SuperconductingquantuminterferencedevicesVBM: Voxel-basedmorphometry

Y116134.indb 333 10/10/11 2:12 PM

Y116134.indb 334 10/10/11 2:12 PM

335

Body languageEmbodied perception of emotion

Charlotte B. A. Sinke,1,2 Mariska E. Kret,1

and Beatrice de Gelder1,3

1CognitiveandAffectiveNeuroscienceLaboratory,TilburgUniversityTilburg,theNetherlands2DepartmentofCognitiveNeuroscience,MaastrichtUniversityMaastricht,theNetherlands3MartinosCenterforBiomedicalImaging,MassachusettsGeneralHospitalCharlestown,Massachusetts

List of abbreviations

AMG =amygdala;almond-shapednucleusinanteriortemporallobeEBA =extrastriatebodyarea;brainarealyingintemporal-occipitalsul-

cuswhichisspecificallyinvolvedinprocessingbodiesEEG =electroencephalography;amethodtomeasureelectricalactivity

fromthescalprelatedtocorticalactivityERP =event-related potential; EEG waves time-locked to specific

stimuliFBA =fusiformbodyarea;brainareainthefusiformgyrusthatisspe-

cificallyinvolvedinprocessingbodiesFFA =fusiformfacearea;brainareainthefusiformgyrusthatisspecifi-

callyinvolvedinprocessingfacesFG =fusiformgyrus;partofthetemporallobethatisinvolvedinvisual

processingfMRI =functional magnetic resonance imaging; brain imaging method

thatmeasuresthehemodynamicresponse(changeinbloodflow)relatedtoneuralactivityinthebrain

hMT+/V5=humanmotionarea;brainareaspecificallyprocessingmovementIOG =inferioroccipitalgyrusIFG =inferiorfrontalgyrus

15

Y116134.indb 335 10/10/11 2:12 PM


MEG=magnetoencephalography; a neuroimaging technique that measuresmagneticfieldsproducedbyelectricalactivityinthebrain

N170 =ERPcomponentoriginatingfromlateraloccipitotemporalcortexspe-cificallyrelatedtoalatestageintheearlyvisualencodingoffaces

OFA =occipitalfacearea;brainareaininferioroccipitalgyrusknowntobeinvolvedinfaceprocessing

P1 =veryearlyERPcomponentrelatedtoveryearlyvisualprocessingPET =positronemissiontomography;brainimagingmethodwherebyradio-

activetracersareinjectedintothebloodstreamPM =premotorcortexSTS =superiortemporalsulcus;posteriorpartisinvolvedinprocessingbio-

logicalmotionTPJ =temporoparietaljunctionV1 =primaryvisualcortex

15.1 Introduction

Ineverydaylife,wearecontinuouslyconfrontedwithotherpeople.Howtheybehaveandmovearoundhasadirect influenceonuswhetherweareawareof itornot.Incommunication,wearegenerallyfocusedontheface.Forthisreason,emotionresearch in the past has focused on faces. Also, facial expressions seem to haveuniversalconsistency.However,bodilyexpressionsare just aswell recognizedasfacialexpressions,theycanbeseenfromadistance,andarefromanevolutionaryperspective much older. Body language therefore has a high communicative rolealbeitwe are less awareof it.Models on facial expressionprocessingmight alsoworkforunderstandingbodilyexpressions.Similarbrainregionsseemtogetacti-vatedforboth,butalthoughfacesshowthementalstatesofpeople,bodyposturesinadditionshowanactionintention.Therefore,seeingbodiesadditionallyactivatesmotionareas.

Inanaturalisticenvironment,facesneverappearalone:theyaremostlyalwaysaccompaniedbyabody,which influenceshow the facial expression isperceived.Thisisalsothecaseforothermodalitiessuchasthevoice.Whichmodalityisdomi-nantdependsonthespecificemotionbeingshown,onthesituation,andmanyotherfactors.Forexample,aggressionseemstobemorepronouncedinbodilyexpressions,whereasshameordisgustcanclearlybeseenfromtheface.Alsothecontext,includ-ingotherpeopleornot,canfacilitaterecognitionofemotions.Moreover,wedonotliveinastaticworld;dynamicstimuligiveus,justasintherealworld,moreinfor-mation.Wealsowouldliketoputforwardthatbrainresponsestoemotionalexpres-sionsarenotdrivenbyexternalfeaturesalonebutaredeterminedbythepersonalsignificanceofexpressionsinthecurrentsocialcontext.Forexample,individualdif-ferencessuchaspersonalitytypeandgenderplayanimportantrole.Moreover,bodylanguageofpeopleinteractingcantellusmuchabouttheirrelationship.

Weargue that thenatureofemotionperceptioncannotbefullyunderstoodbyfocusingseparatelyonsocial,cultural,contextual,individual,orinterpersonalfac-tors. The perception of an emotion is embodied, and its bodily grounded nature

Y116134.indb 336 10/10/11 2:12 PM

Body language 337

providesafoundationforsocialcommunication.“Whatyouseeiswhatyouget,”doesnotapplyhere.Peopledonot“see”thesame,nordotheyattendtothesame.

Furthermore,perceptionandrecognitionofbodilyexpressionsdonotrequirefullattentionnordo they require that thevisual stimulus is consciously seen.This ismostevidentfrompatientswithhemianopia.

Allthesetopicsarediscussedinthischapter.Theyshowusthatbeingabletorec-ognizeemotionalmeaningfromothersisvitalandthatbodylanguageisofcrucialimportance innormal communication.This is clearly impaired indisorders suchasautism.Therefore,investigationsofbodilyexpressionswillenrichbasicclinicalresearchandcanleadtothedevelopmentofnewobservationalanddiagnostictools.

15.2 Similarities and differences in neurofunctional basis of faces and bodies

Forseveralyearstheneuralcorrelatesofbodyshape(Downing,Jiang,Shuman,&Kanwisher,2001)andperceptionofbodilyexpressions(deGelder,Snyder,Greve,Gerard,&Hadjikhani,2004)havebeen the focusof experimental investigations.Althoughmoreorlessneglectedinthepastinfavoroffaces,itisnowincreasinglybelievedthattheperceptionofbodieshasaspecialinfluenceonourbehavior.Tobeabletodothis,theymustbeprocessedfromotherobjectsdistinctly.

The major concept used to argue for the specificity of processing is that ofconfiguration. There is clear evidence that both faces and bodies are processedconfigurally,asawhole,ratherthanasacollectionoffeatures.Thishasbeenshownwiththe“inversioneffect”:recognitionoffacesandbodiespresentedupside-downisrelativelymoreimpairedthaninvertedobjects(Reed,Stone,Bozova,&Tanaka,2003). Besides behaviorally, this effect can also be investigated psychophysicallybylookingatelectrophysiologicalrecordings.Withelectroencephalography(EEG),electricalactivitycomingfromfiringneuronsispickedupatthescalpthroughelec-trodes.Byaveragingbrainactivitytocertainevents,event-relatedpotentials(ERPs)areformed.OnesuchERPcomponentistheN1,whichisthoughttoreflectalatestageinthestructuralencodingofthevisualstimulus(Bentin,Allison,Puce,Perez,&McCarthy,1996;Eimer,2000)andoriginatesfromthelateraloccipitotemporalcortexwhichhousesthefusiformgyrus(FG).Inthecaseoffaceprocessing,theN1peaksatadifferentlatency(around170msafterstimulusonsetandhencecalledtheN170)fromthatforobjects.ThelatencyoftheN170isdelayedwhenpresentedfacesareinverted,whichshowstheinvolvementofFGinprocessingfacesconfigurally.TheN1peakforbodyprocessingalsodiffersfromobjects;itrangesfrom154to228msafterstimulusonset(Gliga&Dehaene-Lambertz,2005;Meeren,vanHeijnsbergen,&deGelder,2005;Righart&deGelder,2007;Stekelenburg&deGelder,2004;Thierryetal.,2006;vanHeijnsbergen,Meeren,Grezes,&deGelder,2007)anditalsoshowsaninversioneffect.Doesthismeanthereisnodifferencebetweenfaceandbodyprocessing?

No,itdoesnot.AlthoughEEGhasaveryhightemporalresolutionandcanthere-fore tellusa lotabout the timingofprocessing, it ishard to linkaspecificbrain

Y116134.indb 337 10/10/11 2:12 PM


areatothefoundactivation.Amethodbettersuitedtodothisismagnetoencepha-lography (MEG).Thiswas recentlydone for investigationof theearliestonsetoftheelectrophysiologicalinversioneffectfordifferentstimuluscategories(Meeren,Hadjikhani, Ahlfors, Hamalainen, & de Gelder, 2008). They indeed found thatthecorticaldistributionofthisearlyeffectwashighlycategoryspecific.Differenttime courses of activation were observed in the common neural substrate in FG.Furthermore,facesactivatedtheinferioroccipitalgyrus(IOG;alsonamedoccipitalfacearea(OFA)),whereasforbodiestheeffectwasobservedintheposteriodorsalmedialparietalareas(precuneus/posteriorcingulate).Hence,whereasfaceinversionmodulatesearlyactivityinface-selectiveareasintheventralstream,bodyinversionevokesactivityindorsalareas,suggestingdifferentearlycorticalpathwaysforcon-figuralfaceandbodyperception.

Inadditiontothisearlyprocessinginperceivingfacesandbodies,moregeneralprocessingonlongertimescalescanbeinvestigatedwithfunctionalmagneticreso-nanceimaging(fMRI).Withthismethod,adistinctionhasactuallybeenfoundintheFGbetween facesandbodies, thereafter called fusiform facearea (FFA)andfusiformbodyarea(FBA)(Schwarzlose,Baker,&Kanwisher,2005).Furthermore,bodiesseemedalsotobeprocessedinanotherarea:theextrastriatebodyarea(EBA)(Downingetal.,2001).Thisarealiesveryclosetothehumanmotionarea(hMT+/V5),andgiventhatbodiesimplyaction,thisfindingisnotpeculiar.Besides,thesupe-riortemporalsulcus(STS)andpremotorcortex(PM)alsogetactivatedforbodies(Grèzes,Pichon,&deGelder,2007);theformerisknowntobeinvolvedinbiologicalmotion(Bonda,Petrides,Ostry,&Evans,1996),thelatteralsobeingamotorarea.

Whendirectlycomparing theneuralcorrelatesof facesandbodies, thesparseevidencepointstoabroadernetworkfortheperceptionofbodies,probablyduetotheactioncomponentinvolvedinthose.Itisremarkablethattheliteratureonisolatedfaceandbodyperceptionismoreextensivecomparedtotheknowledgeofthemoreecologicallyvalidcombinedperceptionofafaceonabody.Thefewstudiesavail-ableaddressingthisissueconsistentlypointtoastrongmutualinfluence(Aviezer,Hassin,Ryan,Grady,Susskind,&Anderson,2008;Meeren,vanHeijnsbergen,&deGelder,2005;VandenStock,2007).

15.3 Emotional modulation of body selective areas

Thatfacesandbodiesareprocessedinadistinctway,beingspecialclassesofobjects,hasprobablytodowiththeirecologicalvalue.Weareexperiencedinrecognizingmanydifferentfacialidentities,andbeingabletoreactappropriatelytointentionsstatedinbodieshassurvivalvalue.Importantsourcesofinformationaboutsome-one’s intentions are facial and bodily expressions. To be able to react quickly tothese,theymustbeeffectivelyprocessedinthebrain.

Evidencewasfoundforfastautomaticprocessingofemotionalbodylanguage.FearexpressedbythebodyaffectedtheresponseoftheP1componentat100–120msafterstimulusonsetandtheN170componentalsoshowedadifference(vanHeijnsbergenetal.,2007).Thismeans thatprocessingof theemotionisfaster than identifyingabody.

Y116134.indb 338 10/10/11 2:12 PM

Body language 339

Thisemotionalprocessingpartlytakesplaceinthefaceandbodyareas,suggest-ing abetter representationof the faces andbodies.Several studieshave reportedemotionalmodulationofface-selectiveareas,fusiformfacearea(FFA),andoccipitalfacearea(OFA;Breiter,Etcoff,Whalen,Kennedy,Rauch,&Buckner,1996;vandeRiet, Grèzes, & de Gelder, 2009; Vuilleumier, Armony, Driver, & Dolan, 2001).However,thiseffectmaybedependentonage(Guyer,Monk,McClure-Tone,Nelson,Roberson-Nay & Adler, 2008), attachment style (Vrticka, Andersson, Grandjean,Sander,&Vuilleumier,2008),personalitytype(Kret,Pichon,Grèzes,&deGelder,2008), and gender of the observer and the observed (Kret, Pichon, Grèzes, &deGelder,2011).Sofar,onlyafewstudieshaveinvestigatedtheeffectsofemotionalinformationprovidedbybodyexpressionsonactivationofbodyareasinthebrain.Thefirstfunctionalmagneticresonanceimaging(fMRI)studyaddressingthisissueobservedanincreasedactivationofFGandamygdala(AMG)forfearfulbodyexpres-sions(Hadjikhani&deGelder,2003).Afollow-upexperimentadditionallyshowedtheinvolvementofmotorareas(deGelderetal.,2004).Alsowhendirectlycompar-ingneutralandemotionalfacesandbodies(vandeRietetal.,2009),weobservedthat emotional bodies activate (sub)cortical motor-related structures, such as theinferiorfrontalgyrus(IFG),caudatenucleus,andputamenwhichprobablyhastodowithbeingabletorespondquicklytoemotionalbodies.

Although our findings of emotional modulation of FBA have been replicated(Peelen, Atkinson, Andersson, & Vuilleumier, 2007), emotional modulation ofEBAisuncertain.Wedidnotobserveadifferencebetweenneutralandemotionalbodyimages(vandeRietetal.,2009)butourdatawithdynamicbodyexpressionsdoshowemotionalmodulation (Grèzes,Pichon&deGelder,2007;Kret,Pichon,Grèzes, & de Gelder, 2011b; Pichon, de Gelder, & Grèzes, 2008; Sinke, Sorger,Goebel,&deGelder,2010).

15.4 Affective gist of the scene influences the perception of emotions

Normally,wedonotseeisolatedpeople,butweseetheminacontext.Howdoesthisinfluenceourperceptionofthebodilyexpressionofasingleindividual?

15.4.1 Emotional context

Becauseofrepetitiveco-occurrenceofobjectsorco-occurrenceofagivenobjectinaspecificcontext,ourbraingeneratesexpectations(Bar&Ullman,1996;Palmer,1975).Acontextcanfacilitateobjectdetectionandrecognition(Boyce,Pollatsek,&Rayner,1989;Palmer,1975),evenwhenglimpsedbrieflyandevenwhentheback-groundcanbeignored(DavenportandPotter,2004).Joubert,Fize,Rousselet,andFabre-Thorpe (2008) also observed that context incongruence induced a drop incorrecthitsandanincreaseinreactiontimes,thusaffectingevenearlybehavioralresponses.Theyconcludedthatobjectandcontextmustbeprocessedinparallelwithcontinuousinteractionspossiblythroughfeedforwardcoactivationofpopulationsofvisualneuronsselectivetodiagnosticfeatures.Facilitationwouldbeinducedbythecustomarycoactivationof“congruent”populationsofneurons,whereasinterference

Y116134.indb 339 10/10/11 2:12 PM


wouldtakeplacewhenconflictingpopulationsofneuronsfiresimultaneously.Bar(2004) proposes a model in which interactions between context and objects takeplaceintheinferiortemporalcortex.

Justasrecognizingobjectsisnotindependentfromothercuessuchascontext,emotionperceptiondoesnotproceedoninformationfromonecue(suchasfacialexpressions)alone(Hunt,1941).Knowledgeof thesocialsituation(Aviezeretal.,2008;Carroll&Russell,1996),bodyposture(Meerenetal.,2005;VandenStocket al., 2007), other emotional faces (Russel & Fehr, 1987), voice (de Gelder &Vroomen,2000),or linguistic labels(Barrett,Lindquist,&Gendron,2007) influ-encesemotionperceptionandevenwhichemotionisseeninthestructuralconfigu-rationoftheparticipants’facialmuscles.Inlinewiththeevolutionarysignificanceoftheinformation,theeffectsoftheemotionalgistofascenemayoccuratanearlylevel.WepreviouslyshowedscenecontextcongruencyeffectsonfacialexpressionsinbehavioralresponsesbutalsoinEEGmeasurements.Itcouldbeobservedwhenparticipantshadtoexplicitlydecodetheemotionalexpressionoftheface(Righart&deGelder,2008)butalsowhentheyfocusedonitsorientation(Righart&deGelder,2006).Thisindicatesthatitreflectsanearlyandmandatoryprocessandsuggestsaperceptualbasis.LookingatanEEG,weseethatthepresenceofafearfulexpres-sioninafearfulcontextenhancedtheface-sensitiveN170amplitudeascomparedtoafaceinaneutralcontext.Thiseffectwasabsentforcontexts-only,indicatingthatitresultedfromthecombinationofafearfulfaceinafearfulcontext(Righart&deGelder,2006).ThatscenesareindeedimportantisalsoshownintworecentfMRIstudies where participants interpreted facial expressions differently and differentbrainareaswereactivateddependingonthecontext(Kim,Somerville,Johnstone,Polis, Alexander, & Shin, 2004; Mobbs, Weiskopf, Lau, Featherstone, Dolan, &Frith,2006).

15.4.2 Social emotional context

Does it influenceouremotional reactionwhenwewatcha single individualflee-ingfromdangerwhilebystandersarepassivelystanding there?Dowe ignore thesocialscenetofocusonlyontheemotionofthetargetfigureorareweunwittinglyinfluencedby the social sceneviewing individual action through thefilter itpro-videsus?Studiesoncrowdbehavior(McDougall,1920)indicatethatsocialscenesprovideacontext inwhich individualactionsarebetterunderstoodpromptinganadaptivereactionin theobserver.Usingpoint-lightdisplays,ThorntonandVuong(2004)haveshownthat theperceivedactionofawalkerdependsuponactionsofnearby“to-be-ignored”walkers.Anotherpoint-lightstudybyClarkeandcolleaguesdemonstratesthattherecognitionofaperson’semotionalstatedependsuponanotherperson’spresence(Clarke,Bradshaw,Field,Hampson,&Rose,2005).

ArecentstudybyKretanddeGelder(2010)reportthatthesocialgroupinwhichweencounteraperson,andespeciallyherbodilyexpressions,influencehowweper-ceivethebodylanguageofthissingleindividual.Inthisstudy,imagesofemotionalbody postures were briefly presented as part of social scenes showing neutral oremotionalgroupactions.Theseweremoreaccuratelyandquicklyrecognizedwhen

Y116134.indb 340 10/10/11 2:12 PM

Body language 341

theactionsinthescenesexpressedanemotioncongruentwiththebodilyexpressionofthetargetfigure.Thesestudiesshowtheimportanceofasocial(emotional)scene.However,otherprocesses thantheonesmeasuredmaycontribute to theobservedeffects, for example, the tendency to automaticallymimicand synchronize facialexpressions,vocalizations,postures,andmovementswiththoseofanotherpersonandtoconvergethememotionally(deGelderetal.,2004;Hatfield,Cacioppo,&Rapson,1994).Similarbrainareasareinvolvedwhensubjectsexperiencedisgust(Wicker,Keysers,Plailly,Royet,Gallese,&Rizzolatti,2003)orpain(Jackson,Meltzoff,&Decety, 2005), as when they observe someone else experiencing these emotions.Suchaprocessmaycontribute toobservers’ability toperceive rapidlyambiguitybetweenaperson’sbodylanguageanditssocial(emotional)context.Suchincongru-itymaycreateaconflict inemotionalcontagionprocesses triggeredby the targetfigureandhelptoexplaintheslowerandlessaccuratereactionintheobserver.

15.4.3 Static versus dynamic

Researchperformedwith facial andbodilypictureshave contributeda lot toourunderstandingofhowourbrainprocesses thesestimuli.However, inreal life,weareconfrontedwithmovingpeople.Although staticbodypostures already implymotion,dynamicstimuliobviouslycontainmore information,whichhelps inbet-ter understanding someone’s intentions and being able to react appropriately tothese.Point-lightdisplaystudiesshowedthatbiologicalmotionisquicklydetected(Johansson,1973).Afewneuroimagingstudiesreporttheimportanceofmovementinprocessingemotionalexpressions(see,forexample,Decety&Chaminade,2003;Grosbras&Paus,2006;LaBar,Crupain,Voyvodic,&McCarthy,2003).Adolphs,Tranel,andDamasio(2003)reportthatapatientwithaventralpathwaylesionisabletoreademotionfromdynamic,butnotfromstaticfacialexpressions.

Inhealthysubjects,Sato,Kochiyama,Yoshikawa,Naito,andMatsumura(2004)found that the AMG, IOG, and FG were more activated by dynamic than staticfearful facial expressions. Studies of bodily expressions also report better recog-nition rates for dynamic versus static stimuli (Atkinson, Dittrich, Gemmell, &Young,2004;deMeijer,1989).Arecentbrainimagingstudylookedatthepercep-tionofangryandneutralhandandfacemovements(Grosbras&Paus,2006).Theauthorsreportedthatregionsknowntobeinvolvedinactionandemotiongenerationinoneselfarealsoactivatedwhenperceivingactionandemotioninthefacesandhands of others. Furthermore, they reported an interaction between emotion andbodypart:whenhandactionswereperformedwithemotion,aregioninthesupra-marginalgyrusrespondedmostlytothis.Becausethisregionhadbeenimplicatedbeforeasbeinginvolvedingettingattentiontowardalimb(Rushworth,Krams,&Passingham,2001),itseemsherethattheemotioninthehandmovementincreasedthisattention.

Thisstudy,however,wasnotdesignedtodetectspecificallywhatadditionalinfor-mationiscontributedbydynamics.Twostudiesthattriedtodothisused3-svideo-clipsofsomeoneopeningadoorineitheraneutralorinafearful(Grèzesetal.,2007)orangryway(Pichonetal.,2008).Fromeachmovie,oneframeatwhichtheemotion

Y116134.indb 341 10/10/11 2:12 PM


was at its peak was taken and also presented for 3 s. Not surprisingly, dynamicversusstaticbodyexpressions(irrespectiveoftheemotionalcontent)causedmotorresonance:bilateralactivationsofPMandparietalcortex,STS,andFG.Mostinter-estingly,aninteractionwasobservedbetweenemotionandmotioninSTSandrightPM.Inhumans,STS,parietal,andPMareinvolvedinactionobservationandprob-ablyalso inactionunderstanding (Grèzes&Decety,2001), so inasmuchas theseareasrepresentedtheemotionalactioninthisstudy,theycouldalsobeinvolvedinemotionunderstanding.

15.5 Individual differences

15.5.1 Gender

Oneaspectthathassofarnotreceivedmuchattentioninthestudiesoffacialandbodilyexpressionsconcernstheroleofgenderinemotionalprocessing.Someiso-latedfindingsindicatethattheremaybegenderdifferencesinemotionalprocesses.Femalestendtoscorehigherthanmalesontestsofempathy,socialsensitivity,andemotionrecognition(seeHall,1978andMcClure,2000forreviews).Butwhereasfemalesshowmorefacialmimicryinresponsetoemotionalmoviefragments,theydidnot report experiencingmore emotion thanmales,which suggests an expres-sive, rather thananexperientialdifference (Kring&Gordon,1998).Testosteronelevelisagoodpredictorofthepresenceofanangertrait,aggressivebehavior,anddominance(vanHonk&Schutter,2007),andattheneuronallevel,AMGresponsetofearandangersignals(Derntl,Windischberger,Robinson,Kryspin-Exner,Gur,&Moser,2009).AlemanandSwart(2008)reportstrongeractivationintheIFGandSTSinmenthanwomeninresponsetofacesdenotinginterpersonalsuperiority.

Adifferent issueiswhether thegenderof thepersonweobserveinfluencesusdifferentlydependingonourowngender.Whenwethinkoftheinterpersonalsupe-riorityeffectinmaleobserversasreportedbyAlemanandSwart(2008),itprobablydoes. Except for very interesting work on gender stereotypes for different emo-tions, thisquestionishardlyexploredinthefieldofsocialneuroscience.ArmonyandSergerie(2007)studiedmemoryforfearful,happy,andneutralexpressionsinrelationtothegenderoftheobserver.TheyreportthatthehemisphericlateralityofAMGformemoryoffacialexpressionswasafunctionofthesexofthesubjectsandthesexofthefacesbeingremembered.TheleftAMGwasmoreactiveforsuccess-fullyrememberedfemalefearfulfacesinwomen,whereasinmentherightAMGwasmoreinvolvedinmemoryformalefearfulfaces.Theseresultsdemonstrateasubtlerelationshipbetweentheobserverandthestimulus.

ArecentstudybyKretetal.(2011a)revealshowface-andbody-specificareasaremodulatedbygender.Twoevent-relatedFMRIexperiments,usinganoddballtask,wereusedtorecordfemaleandmaleparticipants’brainactivitywhiletheyobservedvideosshowingfear,anger,orneutralsignalsexpressedbyfemaleandmaleactors.Inthefirstexperiment,shortvideofragmentsoftheangryandneutralexpressionswereused,inthesecondfearfulandneutralexpressions.TheAMGwasmodulatedmorebyfacialthanbodilyexpressions.FGwasinvolvedinprocessingbodystimuli,more

Y116134.indb 342 10/10/11 2:12 PM

Body language 343

thaninprocessingfaces.Threateningbodyexpressions,whetherfearfulorangry,modulated activity in hMT+/V5-EBA and the parietal and somatosensory cortex(whichmayplayaroleinactionunderstanding).Wealsofoundsignificantinfluencesofthegenderoftheactorsandoftheobservers.AhigheractivationofEBAandSTSwas observed for threatening male versus female actors. Male observers showedmore activation for threatening versus neutral bodies in many visual processingareas,moresothanfemaleobserversandespeciallytomalebodyexpressions.Theseresultsareinlinewithpreviousstudiesthatshowmaleobserversaremorereactivetothreateningsignalsthanfemaleobservers(Aleman&Swart,2008).

Humanemotionperceptiondependstoanimportantextentonwhetherthestimulusisafaceorabodyandalsoonthegenderoftheobserverandobserved.Therefore,thesegendereffectscanalsobeseenintheneurofunctionalmechanismsofemotion.

15.5.2 Personality differences

“Embodiedcognition,”aconceptthathasrecentlybeengettingalotofattentionincognitivescience,suggeststhatourmindandthusourperceptionisshapedasmuchby our body and how we physically interact with the environment as by passivesensoryexperience.Increasedvigilanceandenhancedautonomicactivityarepartofanadaptiveresponsetothreat.Inotherwisehealthyindividualsthiscanbecomemaladaptivewhenstressistoogreat.Invariouspathologicalconditionstheanxietyresponse isdisproportionate to thestress,eitherbecauseofamisinterpretationofthreat,orbecauseofhyper-orhyporesponsivenessatanyofavarietyofpointsinthecomplexnetworkofneuralpathwaysthatservethestressresponse.Imagingtech-niquesofferuniqueopportunitiestoexploretheneurofunctionalbasisofpersonalitydifferencesandindeedshowthatperceivingemotionsisgreatlyregulatedbytop-downprocessesbeingdifferentfrompersontoperson.

PeoplesufferingfromsocialphobiaoranxietygenerallyshowincreasedAMGactivitywhenconfrontedwith threatening faces (forameta-analysis seeEtkin&Wager,2007).However, theroleoftheAMGindepressionislessclear.Whereassome studies report increased AMG response for threatening versus neutralexpressions related todepressivesymptoms(Canli,Cooney,Goldin,Shah,Sivers,& Thomason, 2005; Peluso, Glahn, Matsuo, Monkul, Najt, & Zamarripa, 2009),others report a decrease in activity (Thomas, Drevets, Whalen, Eccard, Dahl, &Ryan,2001),ornodifferenceatall(Davidson&Dalton,2003;Lee,Seok,Lee,Cho,Yoon,&Lee,2008).Severalstudiesreportdecreasedcortico-limbicconnectivityindepressioninresponsetoemotionalstimuli(Anand,Li,Wang,Lowe,&Dzemidzic,2009;Drevets,1998;Fossati,Hevenor,Graham,Grady,Keightley&Craik,2003)butantidepressanttreatmentshowsreciprocaleffects(Anand,Li,Wang,Gardner,&Lowe,2007).Decreasedactivationintheanteriorcingulatecortexhasbeenreportedindepressionaswell(Davidson&Dalton,2003;Fossatietal.,2003).

Recognition of another’s emotion does not suffice for proper communication.Theorbitofrontalcortex(OFC)regulatesappropriatesocialresponses (Kringelbach,O’Doherty, Rolls, & Andrews, 2003; Rolls, 2000). Socially anxious people areafraidofpossiblescrutinyandnegativeevaluationbyothers.Notsurprisingly,many

Y116134.indb 343 10/10/11 2:12 PM


studiesfindanoveractivefrontolimbicsystem(includingOFC,insula,andAMG)inthisgroupduringthreatperception(Shah,Klumpp,Angstadt,Nathan,&Phan,2009;Straube,Mentzel,&Miltner,2005).Moreover,theOFChasbeenconsistentlyinvolvedinthepathophysiologyofmajordepressivedisorderandbipolardisorder(Davidson&Dalton,2003;Drevets,2007).

People with type D (“distressed”) personality (21% of the general population)havehigherscoresondepressionandanxietyscales(Denollet,Schiffer,Kwaijtaal,Hooijkaas,Hendriks&Widdershoven,2009).Theysufferfromemotionaldistress(“negativeaffectivity”),whichtheyconsciouslysuppress(“socialinhibition”).Thispersonality typeisassociatedwithanegativeprognosis indiseaseandarangeofsomaticeffects.ArecentstudybyvandeRietandcolleagues(2009)showedacorre-lationbetweenthenegativeaffectivitysubscaleandAMGhypoactivationforfearfulfacialandbodilyversusneutralexpressions.So,evensmallpersonalitydifferencesinthenormalpopulationaccountforadifferentperceptionofthreat.However,thisstudyfocusedonlyontheAMGastheregionofinterestandneglectedotherpossiblyinterestingeffectsthatcouldhavebeendetectedinawholebrainanalysis.Moreover,thisstudyusedstaticstimuli.

Inafollow-upstudy,weaimedtorevealneuralcorrelatesoftypeDpersonalityandperceivingdynamicthreateningfacialandbodilyexpressions.Weobservedanegative correlation in the temporal pole and cingulate cortexonboth subscales.Furthermore,anegativecorrelationwasobservedbetweennegativeaffectivityandactivationinbrainareascommonlyinvolvedinemotion:AMG,FG,insula,STS,andIFG.TherightOFCcorrelatednegativelywithsocialinhibition.Alsointerestingistherelationbetweensocialinhibitionandincreasedactivationfollowingthreatintheanteriorintraparietalsulcus,leftTPJ,STS,rightIFG,secondarysomatosensorycor-tex,andleftOFC.Theseregionsareallinvolvedintheactiongoaloftheobserved(see,forarecentmeta-analysis,VanOverwalle&Baetens,2009).Whenobservingaction,weneedtotaketheother’sperspectivewhichwedobyactivatingourmir-ror and mentalizing system. The mirror system (anterior intraparietal sulcus andPM)isengagedinperceivingandexecutingmotionsofbodypartsandisimportantforunderstandingactionandemotion(Rizzolatti&Craighero,2004).TPJplaysanimportant role in our mentalizing system and computes the orientation or direc-tionoftheobservedbehaviortopredictitsgoal(VanOverwalle&Baetens,2009).ObservingaswellasimitatingfacialexpressionsactivatestheIFG(Carr,Iacoboni,Dubeau,Mazziotta,&Lenzi,2003).Peoplewhotendtoinhibitsociallyarelikelytooveractivatethemirrorandmentalizingsystem.

Taking the other’s perspective is not enough; we need to empathize and rea-sonhowtoact.TheOFCisconnectedwithareasthatunderlieemotionalfunctionand empathy (Hynes, Baird, & Grafton, 2006) and interprets somatic sensations(Bechara, Damasio,Tranel, &Anderson, 1998)mediated by internallygeneratedsomatosensoryrepresentationsthatsimulatehowtheotherpersonwouldfeelwhendisplayinganemotion(Adolphs,2002).Withouttheserepresentations,appropriatereactivebehaviorwouldbedifficult.Rauch,Savage,Alpert,Miguel,Baer,andBreiter(1995)usedpositronemission tomography(PET) tomeasure thechanges inright

Y116134.indb 344 10/10/11 2:12 PM

Body language 345

cerebralbloodflowinphobicpatientsprovokedbyexposure to the fearedobject.They observed significant increases during the symptomatic compared with thecontrolstateinOFCandsomatosensorycortex.ThecomplexconnectionsbetweentheOFCandareasinvolvedinemotionsuggestimplicationsforitsroleinanxietydisorders(Fischer,Andersson,Furmark,&Fredrikson,1998).Wehypothesizethatpeoplewithhighscoresonsocialinhibitionprefertoavoidsocialsituationsbecauseitgivesthemtoomuchcognitivestress.

15.6 Perceiving interactions

Tryingtogetadditionalinformationgoingfromstatictodynamicfacialandbodilyexpressions,includingacontextandtakingintoaccountgenderstereotypes,thereisanothersteptotaketogettoevenmorenaturalisticsituations.Thisistheperceptionofapersoninteractingwithanotherperson.Theinterplaybetweenthemcaninformusabouttheirrelationship.

In previous stimuli creation, actors always looked into the camera. Therefore,anemotionalexpressionhadanimmediateimpactontheobservingparticipant.Inadirectconfrontation,itmakessensethatyouwanttoreactimmediately.Butwhathappenswhenthethreatisnotdirectedtowardyou?Thisquestionhasbeenstud-iedrecentlybySinkeetal.(2010).Inthisstudy,wewantedtoinvestigatehowthebrainreacts toasituation that is threateningforoneof the twopersons involved.Forthisstudy,wecreated3-svideoclipsinwhichamaleactorgrabbedthehandbagofafemaleactor.Hedidthiseitherinaveryaggressivewaywherebythewomanexpressedfear,orinateasingway,asifthetwokneweachother.Theactorsfacedeachotheranddidnotlooktowardtheobserver.Whenyouwalkonthestreetyoumayhaveyourthoughtsonanupcomingdeadlineinsteadofonthepeopleontheothersideofthestreet.Willyouthenstillbeabletorecognizeathreat?

Toinvestigatethissecondquestion,threesmalldots,presentedonlyfor40ms,wereaddedtoeachmovie.Participantsinthefirsttaskhadtolookexplicitlytothebodiesandcategorizethesituationasthreateningorteasing.Intheothertaskcondi-tion, theyhad tomonitor the randomlyappearingdots andcategorize their color.ResultsshowedfirstofallthattheAMGshowedheightenedactivationforthethreat-eninginteractionsascomparedtotheteasingones.TheAMGseemstoactassomekind of warning signal and possibly passes information through to other regions.Also,duringunattendedthreat,moreprocessingtookplaceinbody-sensitivevisualregions inFG,middleoccipitotemporal gyrus, andSTS than teasing interactions.Furthermore,thisheightenedactivationforunattendedthreatwaspairedwithbetterbehavioralperformanceonthedottaskduringthreateninginteractions.Itseemedasifthethreatheightenedtheirattentionandbecausethedotswerealwaysplacedsome-whereonthebodies,theywereabletoperceivethembetter.Anotherfindingwasthatalthoughthethreatwasclearlynotdirectedtowardtheobserver,regionsknowntobeinvolvedinactionobservation(IFG,TPJ,andinferiorparietallobe)andprepa-ration(PM,putamen)showedincreasedactivationforthreat.Inconclusion,bodily

Y116134.indb 345 10/10/11 2:12 PM


expressionsareeasilyrecognizedeventhoughyourattentionisnotexplicitlyonthesituationandthethreatisnotdirectedtowardyou,whichhashighsurvivalvalue.

15.7 Bodies processed without attention and visual awareness

Studies with hemianopia patients have shown that perception or recognition ofbodilyexpressionsdoesnotrequirefullattention.Patientswithstriatecortexlesionsoranattentionaldisordercanreacttoavisualstimuluseventhoughtheyhavenotconsciouslyseenit.Patientswithlefthemispatialneglectduetoalesionintherightparietalcortexfailtodirectattentiontostimuliintheirleftvisualfield.However,whenthestimulusisanexpressiveincontrasttoaneutralfaceorbodyoraneutralobject,theyarebetterabletoperceiveit.

Theclearestexampleofbeingabletoprocessemotionalsignalshasbeengivenbypatientswith lesions to their primaryvisual cortex (V1).Under stringent test-ingconditions,theywereabletodiscriminatebetweenvisualpropertiesofstimulithey could not consciously see. This phenomenon is called “blindsight.” Later, itwas shown that they were also able to guess correctly the emotional valence offacialstimulipresented in theirblindvisualfield,so-called“affectiveblindsight”(deGelder,Vroomen,Pourtois,&Weiskrantz,1999).In thefirstbehavioralstudyonlymovingstimulibutnotstillimagesoffacialexpressionsappearedtosupportaffectiveblindsight.Ifmovementwerethecriticalaspecttosupportnonconsciousdiscriminationofdifferentemotionalexpressions,onewouldexpectblindsightalsoforotherattributesthatrelyonmovement.However,blindsightwasonlyobservedforemotionalfacialexpressionsandnotfacialspeech(deGelder,Vroomen,Pourtois,&Weiskrantz,2000).Otherfacialattributessuchaspersonalidentityorgenderwerealsotestedwithnegativeresults,suggestingthatneithermovementnornonemotionalfacialattributesarepersedeterminantsofthephenomenon.

Moredirectly,inlaterresearchaffectiveblindsightalsoemergedveryclearlywhen still images of facial expressions were used, especially when tested withindirect methodologies (Anders, Birbaumer, Sadowski, Erb, Mader & Grodd,2004;Pegna,Khateb,Lazeyras,&Seghier,2005).Stillunknowniswhetheraffec-tiveblindsightisinducedbynonconsciousprocessingofoverallfaceconfigura-tionorbyindividualkeyfeatures.Thereisevidencethattheeyeregionismostsalientinconveyingemotioninformation,andthatthemostancientpartsofourvisualandemotionsystemsinthebrainseemtunedtodetectthissimplesignalrather than thewhole faceconfiguration (Kimetal.,2004;Morris,deBonis,&Dolan,2002).

Asidefromfacialexpressions,otherstimuluscategorieshavebeenusedtotestwhetheraffectiveblindsightcouldbeextendedtootherstimuli.Thusfar,themoststudiedcategoriesareaffectivescenesandbodilyexpressions.Generally,negativeresults have been reported for scenes, suggesting that the appraisal of the emo-tionalcontentofcomplexpicturesrequirescognitiveandsemanticprocessingthatdependsonconsciousvisualperception(deGelder,Pourtois&Weiskrantz,2002).Ontheotherhand,behavioralandneuroimagingresultshaveshownthataffective

Y116134.indb 346 10/10/11 2:12 PM

Body language 347

blindsightforbodilyexpressionsmaybeatleastasclearlyestablishedasthatprevi-ouslyreportedforfacialexpressions,andsustainedbyapartlyoverlappingneuralpathway(deGelder&Hadjikhani,2006).Thisimpliesthatimplicitprocessingofemotionsinblindsightisnonspecificforfacesbutspecificforbiologicallyprimitiveemotionalexpressionsingeneral.

15.8 Conclusion

Thereareimportantsimilaritiesanddifferencesintheneurofunctionalbasisoffacesandbodies.Bothareverystrongcues.Theygrabourattentionandcanevenbepro-cessedwithoutattentionandvisualawareness.Whereasit iswidelyacceptedthattheFGplaysaroleintheperceptionofemotions,whetherfromthefaceorbody,emotionalmodulationoftheEBAisstillunderdiscussion.Thesceneinwhichweperceiveemotionscanfacilitateourrecognition,andthepresenceofotherpeopleexpressingthesameemotionnaturallyhelpsusperceiveanother’semotioncorrectly.Moreover,inanaturalsocialscene,weseepeopleinteractingwitheachother.Theperceptionofemotionsisnotapurebottom-upprocess.Severaltop-downprocessessuchasknowledgeofthesocialsituation,gender,andpersonalitytypeplayaroleaswell. Inreal life,peopleexpress theiremotions inadynamicway.Thismove-mentcomponentaddsinformation,therebyfacilitatingrecognition.Toconclude,theperceptionofemotionisnotsostraightforwardandinvolvesmanydifferentkindsofprocesses.

References

Adolphs,R.(2002).Recognizingemotionfromfacialexpressions:Psychologicalandneuro-logicalmechanisms.Behavioral Cognition Neuroscience Review, 1,21–61.

Adolphs,R.,Tranel,D.,&Damasio,A.R.(2003).Dissociableneuralsystemsforrecognizingemotions.Brain Cognition, 52(1),61–69.

Aleman,A.,&Swart,M. (2008).Sexdifferences inneuralactivation to facialexpressionsdenotingcontemptanddisgust.PLoSONE3(11)):e3622.

Anand,A.,Li,Y.,Wang,Y.,Gardner,K.,&Lowe,M.J.(2007).Reciprocaleffectsofanti-depressant treatment on activity and connectivity of the mood regulating circuit:AnfMRIstudy.Journal of Neuropsychiatry and Clinical Neuroscience, 19(3),274–282.

Anand,A.,Li,Y.,Wang,Y.,Lowe,M.J.,&Dzemidzic,M.(2009).Restingstatecorticolim-bicconnectivityabnormalitiesinunmedicatedbipolardisorderandunipolardepression.Psychiatry Research, 171(3),189–198.

Anders,S.,Birbaumer,N.,Sadowski,B.,Erb,M.,Mader,I.,&Grodd,W.(2004).Parietalsomatosensoryassociationcortexmediatesaffectiveblindsight.Nature Neuroscience, 7(4),339–340.

Armony, J. L., & Sergerie, K. (2007). Own-sex effects in emotional memory for faces.Neuroscience Letters, 426,1–5.

Atkinson,A.P.,Dittrich,W.H.,Gemmell,A.J.,&Young,A.W.(2004).Emotionpercep-tion from dynamic and static body expressions in point-light and full-light displays.Perception, 33(6),717–746.

Y116134.indb 347 10/10/11 2:12 PM


Aviezer, H., Hassin, R., Ryan, J., Grady, G., Susskind, J., &Anderson,A. (2008).Angry,disgustedorafraid?Studiesonthemalleabilityofemotionperception.Psychological Science, 19,724–732.

Bar,M.(2004).Visualobjectsincontext.National Review of Neuroscience, 5(8),617–629.Bar,M.,&Ullman,S.(1996).Spatialcontextinrecognition.Perception, 25(3),343–352.Barrett,L.F.,Lindquist,K.A.,&Gendron,M.(2007).Languageascontextintheperception

ofemotion.Trends in Cognitive Sciences, 11,327–332.Bechara, A., Damasio, H., Tranel, D., & Anderson, S. W. (1998). Dissociation of work-

ing memory from decision making within the human prefrontal cortex. Journal of Neuroscience,18(1),428–437.

Bentin,S.,Allison,T.,Puce,A.,Perez,E.,&McCarthy,G.(1996).Electrophysiologicalstud-iesoffaceperceptioninhumans.Journal of Cognitive Neuroscience, 8,551–565.

Bonda, E., Petrides, M., Ostry, D., & Evans, A. (1996). Specific involvement of humanparietalsystemsand theamygdala in theperceptionofbiologicalmotion.Journal of Neuroscience 16, 3737–3744.

Boyce, S. J., Pollatsek, A., & Rayner, K. (1989). Effect of background information onobject identification. Journal of Experimental Psychology: Human Perception and Performance, 15(3),556–566.

Breiter,H.C.,Etcoff,N.L.,Whalen,P.J.,Kennedy,W.A.,Rauch,S.L.,&Buckner,R.L.(1996).Responseandhabituationofthehumanamygdaladuringvisualprocessingoffacialexpression.Neuron, 17(5),875–887.

Canli,T.,Cooney,R.E.,Goldin,P.,Shah,M.,Sivers,H.,&Thomason,M.E.(2005).Amygdalareactivitytoemotionalfacespredicts improvementinmajordepression.Neuroreport, 16(12),1267–1270.

Carr,L.,Iacoboni,M.,Dubeau,M.C.,Mazziotta,J.C.,&Lenzi,G.L.(2003).Neuralmecha-nismsofempathyinhumans:arelayfromneuralsystemsforimitationtolimbicareas.Proceedings of the National Academy of Sciences of the United States of America, 100(9),5497–5502.

Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions?Judgingemotionfromthefaceincontext.Journal of Personal Social Psychology, 70(2),205–218.

Clarke,T.J.,Bradshaw,M.F.,Field,D.T.,Hampson,S.E.,&Rose,D.(2005).Thepercep-tionofemotionfrombodymovementinpoint-lightdisplaysofinterpersonaldialogue.Perception, 34(10),1171–1180.

Davenport,J.L.,&Potter,M.C.(2004).Sceneconsistencyinobjectandbackgroundpercep-tion.Psychological Science, 15,559–564.

Davidson,R. J.,&Dalton,K. (2003).Dysfuction in theneuralcircuitryofemotional faceprocessinginindividualswithautism.Psychophysiology, 40,s3.

Decety,J.,&Chaminade,T.(2003).Neuralcorrelatesoffeelingsympathy.Neuropsychologia, 41(2),127–138.

deGelder,B.,&Hadjikhani,N.(2006).Non-consciousrecognitionofemotionalbodylan-guage. Neuroreport, 17(6),583–586.

deGelder,B.,&Vroomen,J.(2000).Theperceptionofemotionsbyearandbyeye.Cognition and Emotion, 14(3),289–311.

deGelder,B.,Pourtois,G.,&Weiskrantz,L.(2002).Fearrecognitioninthevoiceismodu-latedbyunconsciouslyrecognizedfacialexpressionsbutnotbyunconsciouslyrecog-nizedaffectivepictures.Proceedings of the National Academy of Sciences USA, 99(6),4121–4126.

Y116134.indb 348 10/10/11 2:12 PM

Body language 349

deGelder,B.,Snyder,J.,Greve,D.,Gerard,G.,&Hadjikhani,N.(2004).Fearfostersflight:Amechanismforfearcontagionwhenperceivingemotionexpressedbyawholebody.Proceedings of the National Academy of Sciences USA, 101(47),16701–16706.

deGelder,B.,Vroomen,J.,Pourtois,G.,&Weiskrantz,L.(1999).Non-consciousrecognitionofaffectintheabsenceofstriatecortex.Neuroreport, 10(18),3759–3763.

deGelder,B.,Vroomen,J.,Pourtois,G.,&Weiskrantz,L.(2000).Affectiveblindsight:Areweblindly ledbyemotions?Response toHeywoodandKentridge (2000).Trends in Cognitive Science, 4(4),126–127.

deMeijer,M.(1989).Thecontributionofgeneralfeaturesofbodymovementtotheattributionofemotions.Journal of Nonverbal Behavior, 13,247–268.

Denollet,J.,Schiffer,A.A.,Kwaijtaal,M.,Hooijkaas,H.,Hendriks,E.H.,&Widdershoven,J.W. (2009).Usefulnessof typeDpersonality andkidneydysfunction aspredictorsofinterpatientvariabilityininflammatoryactivationinchronicheartfailure.American Journal of Cardiology, 103(3),399–404.

Derntl, B., Windischberger, C., Robinson, S., Kryspin-Exner, I., Gur, R. C., & Moser, E.(2009).Amygdalaactivitytofearandangerinhealthyyoungmalesisassociatedwithtestosterone.Psychoneuroendocrinology, 34(5),687–693.

Downing,P.E.,Jiang,Y.,Shuman,M.,&Kanwisher,N.(2001).Acorticalareaselectiveforvisualprocessingofthehumanbody.Science, 293(5539),2470–2473.

Drevets,W.C.(1998).Functionalneuroimagingstudiesofdepression:Theanatomyofmel-ancholia.Annual Review of Medicine, 49,341–361.

Drevets,W.C.(2007).Orbitofrontalcortexfunctionandstructureindepression.Annals of the New York Academy of Science, 1121,499–527.

Eimer, M. (2000). The face-specific N170 component reflects late stages in the structuralencodingoffaces.Neuroreport, 11(10),2319–2324.

Etkin,A.,&Wager,T.D. (2007).Functionalneuroimagingofanxiety:Ameta-analysisofemotionalprocessinginPTSD,socialanxietydisorder,andspecificphobia.American Journal of Psychiatry, 164(10),1476–1488.

Fischer,H.,Andersson,J.L.,Furmark,T.,&Fredrikson,M.(1998).Braincorrelatesofanunexpectedpanicattack:Ahumanpositronemissiontomographicstudy.Neuroscience Letters, 251(2),137–140.

Fossati,P.,Hevenor,S. J.,Graham,S. J.,Grady,C.,Keightley,M.L.,&Craik,F. (2003).Insearchoftheemotionalself:AnfMRIstudyusingpositiveandnegativeemotionalwords.American Journal of Psychiatry, 160(11),1938–1945.

Gliga,T.,&Dehaene-Lambertz,G.(2005).Structuralencodingofbodyandfaceinhumaninfantsandadults.Journal of Cognitive Neuroscience, 17,1328–1340.

Grèzes,J.,&Decety,J.(2001).Functionalanatomyofexecution,mentalsimulation,observa-tion,andverbgenerationofactions:Ameta-analysis.Human Brain Mapping, 12(1),1–19.

Grèzes,J.,Pichon,S.,&deGelder,B.(2007).Perceivingfearindynamicbodyexpressions.Neuroimage, 35(2),959–967.

Grosbras,M.H.,&Paus,T.(2006).Brainnetworksinvolvedinviewingangryhandsorfaces.Cerebral Cortex, 16(8),1087–1096.

Guyer,A.E.,Monk,C.S.,McClure-Tone,E.B.,Nelson,E.E.,Roberson-Nay,R.,&Adler,A.D.(2008).Adevelopmentalexaminationofamygdalaresponsetofacialexpressions.Journal of Cognitive Neuroscience.

Hadjikhani,N.,&deGelder,B.(2003).Seeingfearfulbodyexpressionsactivatesthefusiformcortexandamygdala.Current Biology, 13(24),2201–2205.

Y116134.indb 349 10/10/11 2:12 PM


Hall, J.A. (1978). Gender effects in decoding nonverbal cues. Psychological Bulletin, 85,845–857.

Hatfield, E., Cacioppo, J. T., & Rapson, R. L. (1994). Emotional contagion. Cambridge:CambridgeUniversityPress.

Hunt,W.A.(1941).Recentdevelopmentsinthefieldofemotion.Psychological Bulletin, 38,249–276.

Hynes,C.A.,Baird,A.A.,&Grafton,S.T.(2006).Differentialroleoftheorbitalfrontallobeinemotionalversuscognitiveperspective-taking.Neuropsychologia, 44(3),374–383.

Jackson,P.L.,Meltzoff,A.N.,&Decety,J.(2005).Howdoweperceivethepainofothers?Awindowintotheneuralprocessesinvolvedinempathy.Neuroimage, 24(3),771–779.

Johansson,G. (1973).Visualperceptionofbiologicalmotionandamodel for its analysis.Perception & Psychophysics, 14,201–211.

Joubert,O.R.,Fize,D.,Rousselet,G.A.,&Fabre-Thorpe,M.(2008).Earlyinterferenceofcontextcongruenceonobjectprocessinginrapidvisualcategorizationofnaturalscenes.Journal of Vision, 8(13),1–18.

Kim,H.,Somerville,L.H.,Johnstone,T.,Polis,S.,Alexander,A.L.,&Shin,L.M.(2004).Contextualmodulationofamygdalaresponsivitytosurprisedfaces.Journal of Cognitive Neuroscience, 16(10),1730–1745.

Kret,M.E.,&deGelder,B.(2010).Socialcontextinfluencesrecognitionofbodilyexpres-sions.Experimental Brain Research,203(1),169–180.

Kret,M.E.,Pichon,S.,Grèzes,J.,&deGelder,B.(2011a).Menfearothermenmost:Genderspecificbrainactivationsinperceivingthreatfromdynamicfacesandbodies.AnfMRIstudy.Frontiers in Emotion Science, 2, 1–11.

Kret,M.E.,Pichon,S.,Grèzes,J.,&deGelderB.(2011b).Similaritiesanddifferencesinperceiving threat from dynamic faces and bodies.An fMRI study. NeuroImage, 54,1755–1762.

Kring,A.M.,&Gordon,A.H.(1998).Sexdifferencesinemotion:Expression,experience,andphysiology.Journal of Personality and Social Psychology, 74,686–703.

Kringelbach,M.L.,O’Doherty,J.,Rolls,E.T.,&Andrews,C.(2003).Activationofthehumanorbitofrontalcortextoaliquidfoodstimulusiscorrelatedwithitssubjectivepleasant-ness.Cerebral Cortex, 13(10),1064–1071.

LaBar,K.S.,Crupain,M.J.,Voyvodic,J.T.,&McCarthy,G.(2003).Dynamicperceptionoffacialaffectandidentityinthehumanbrain.Cerebral Cortex, 13(10),1023–1033.

Lee,B.T.,Seok,J.H.,Lee,B.C.,Cho,S.W.,Yoon,B.J.,&Lee,K.U.(2008).Neuralcor-relatesof affectiveprocessing in response to sadandangry facial stimuli inpatientswithmajordepressivedisorder.Progress in Neuropsychopharmacology and Biological Psychiatry, 32(3),778–785.

McClure,E.B.(2000).Ameta-analyticreviewofsexdifferencesinfacialexpressionprocess-ingandtheirdevelopmentininfants,children,andadolescents.Psychological Bulletin, 126,424–453.

McDougall,W.(1920).The group mind. NewYork:G.P.Putnam’sSons.Meeren,H.K.,Hadjikhani,N.,Ahlfors,S.P.,Hamalainen,M.S.,&deGelder,B.(2008).

Earlycategory-specificcorticalactivationrevealedbyvisualstimulusinversion.PLoS ONE, 3(10),e3503.

Meeren,H.K.,vanHeijnsbergen,C.C.,&deGelder,B.(2005).Rapidperceptualintegra-tion of facial expression and emotional body language. Proceedings of the National Academy of Sciences USA, 102(45),16518–16523.

Y116134.indb 350 10/10/11 2:12 PM

Body language 351

Mobbs,D.,Weiskopf,N.,Lau,H.C.,Featherstone,E.,Dolan,R.J.,&Frith,C.D.(2006).TheKuleshoveffect:Theinfluenceofcontextualframingonemotionalattributions.Social Cognitive Affect in Neuroscience, 1(2),95–106.

Morris,J.S.,deBonis,M.,&Dolan,R.J.(2002).Humanamygdalaresponsestofearfuleyes.Neuroimage, 17(1),214–222.

Palmer, L. A., & Rosenquist, A. C. (1975). Single-unit studies in the cat. Neuroscience Research Program Bulletin, 13(2),214–220.

Peelen,M.V.,Atkinson,A.P.,Andersson,F.,&Vuilleumier,P.(2007).Emotionalmodula-tionofbody-selectivevisual areas.Social, Cognitive and Affective Neurosciences, 2,274–283.

Pegna,A. J.,Khateb,A.,Lazeyras,F.,&Seghier,M.L. (2005).Discriminatingemotionalfaceswithoutprimaryvisualcorticesinvolvestherightamygdala.Nature Neuroscience, 8(1),24–25.

Peluso,M.A.,Glahn,D.C.,Matsuo,K.,Monkul,E.S.,Najt,P.,&Zamarripa,F.(2009).Amygdalahyperactivationinuntreateddepressedindividuals.Psychiatry Research,173(2),158–161.

Pichon,S.,DeGelder,B.,&Grèzes,J.(2008).Emotionalmodulationofvisualandmotorareasbystillanddynamicbodyexpressionsofanger.Social Neuroscience 3(3),199–212.

Rauch,S.L.,Savage,C.R.,Alpert,N.M.,Miguel,E.C.,Baer,L.,&Breiter,H.C.(1995).Apositronemissiontomographicstudyofsimplephobicsymptomprovocation.Archives of General Psychiatry, 52(1),20–28.

Reed, C. L., Stone, V. E., Bozova, S., & Tanaka, J. (2003). The body-inversion effect.Psychology Science, 14(4),302–308.

Righart,R.,&deGelder,B.(2006).Contextinfluencesearlyperceptualanalysisoffaces:Anelectrophysiologicalstudy.Cerebral Cortex, 16(9),1249–1257.

Righart,R.,&deGelder,B. (2007). Impaired face andbodyperception in developmentalprosopagnosia.Proceedings of the National Academy of Sciences of the United States of America,104(43),17234–17238.

Righart,R.,&deGelder,B.(2008).Recognitionoffacialexpressionsisinfluencedbyemo-tionalscenegist.Cognitive Affect in Behavioral Neuroscience, 8(3),264–272.

Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27,169–192.

Rolls,E.T.(2000).Theorbitofrontalcortexandreward.Cerebral Cortex, 10(3),284–294.Rushworth,M.F.,Krams,M.,&Passingham,R.E.(2001).Theattentionalroleoftheleftpari-

etalcortex:Thedistinctlateralizationandlocalizationofmotorattentioninthehumanbrain.Journal of Cognitive Neuroscience, 13(5),698–710.

Russel,J.A.,&Fehr,B.(1987).Relativityintheperceptionofemotioninfacialexpressions.Journal of Experimental Psychology, 116,233–237.

Sato,W.,Kochiyama,T.,Yoshikawa,S.,Naito,E.,&Matsumura,M.(2004).Enhancedneuralactivity inresponse todynamicfacialexpressionsofemotion:AnfMRIstudy.Brain Research and Cognitive Brain Research, 20(1),81–91.

Schwarzlose,R.F.,Baker,C.I.,&Kanwisher,N.(2005).Separatefaceandbodyselectivityonthefusiformgyrus.Journal of Neuroscience, 25(47),11055–11059.

Shah,S.G.,Klumpp,H.,Angstadt,M.,Nathan,P.J.,&Phan,K.L.(2009).Amygdalaandinsularesponsetoemotionalimagesinpatientswithgeneralizedsocialanxietydisorder.Journal of Psychiatry and Neuroscience, 34(4),296–302.

Sinke,C.B.A.,Sorger,B.,Goebel,R.,&deGelder,B.(2010).Teaseorthreat?Judgingsocialinteractionsfrombodilyexpressions.NeuroImage, 49(2),1717–1727.

Stekelenburg,J.J.,&deGelder,B.(2004).Theneuralcorrelatesofperceivinghumanbodies:AnERPstudyonthebody-inversioneffect.Neuroreport, 15(5),777–780.

Y116134.indb 351 10/10/11 2:12 PM


Straube,T.,Mentzel,H.J.,&Miltner,W.H.(2005).Commonanddistinctbrainactivationtothreatandsafetysignalsinsocialphobia.Neuropsychobiology, 52(3),163–168.

Thierry,G.,Pegna,A.J.,Dodds,C.,Roberts,M.,Basan,S.,&Downing,P.(2006).Anevent-relatedpotentialcomponentsensitivetoimagesofthehumanbody.Neuroimage, 32(2),871–879.

Thomas,K.M.,Drevets,W.C.,Whalen,P. J.,Eccard,C.H.,Dahl,R.E.,&Ryan,N.D.(2001). Amygdala response to facial expressions in children and adults. Biological Psychiatry, 49,309–316.

Thornton,I.M.,&Vuong,Q.C.(2004).Incidentalprocessingofbiologicalmotion.Currents in Biology, 14(12),1084–1089.

vandeRiet,W.A.C.,Grèzes,J.,&deGelder,B.(2009).Specificandcommonbrainregionsinvolvedintheperceptionoffacesandbodiesandtherepresentationoftheiremotionalexpressions.Social Neuroscience, 4(2),101–120.

VandenStock,J.,Righart,R.,&deGelder,B.(2007).Bodyexpressionsinfluencerecognitionofemotionsinthefaceandvoice.Emotion, 7(3),487–494.

vanHeijnsbergen,C.C.,Meeren,H.K.,Grèzes,J.,&deGelder,B.(2007).Rapiddetectionoffearinbodyexpressions,anERPstudy.Brain Research, 1186,233–241.

vanHonk,J.,&Schutter,D.J.(2007).Testosteronereducesconsciousdetectionofsignalsservingsocialcorrection:Implicationsforantisocialbehavior.Psychology and Science, 18(8),663–667.

VanOverwalle,F.,&Baetens,K.(2009).Understandingothers’actionsandgoalsbymirrorandmentalizingsystems:Ameta-analysis.Neuroimage,48(3),564–584.

Vrticka,P.,Andersson,F.,Grandjean,D.,Sander,D.,&Vuilleumier,P. (2008). Individualattachment style modulates human amygdala and striatum activation during socialappraisal.PLoS One, 3(8),e2868.

Vuilleumier,P.,Armony,J.L.,Driver,J.,&Dolan,R.J.(2001).Effectsofattentionandemo-tiononfaceprocessinginthehumanbrain:Anevent-relatedfMRIstudy.Neuron, 30(3),829–841.

Wicker,B.,Keysers,C.,Plailly,J.,Royet,J.P.,Gallese,V.,&Rizzolatti,G.(2003).Bothofusdisgustedinmyinsula:Thecommonneuralbasisofseeingandfeelingdisgust.Neuron, 40(3),655–664.

Y116134.indb 352 10/10/11 2:12 PM

353

Risk assessment and decision making

Leslie R. PendrillSPTechnicalResearchInstituteofSweden,MeasurementTechnologyBorås,Sweden

16.1 Introduction

Oneoftheprincipalmotivesformakinganymeasurementis toprovideobjectiveevidenceonwhichdecisionsofconformityofentitiesofanykind(product,service,etc.)tospecificationscanbebased.Thischapterdealswithdecisionsofconformitybasedonmeasurementsspecificallywithpersons,eitherwheremeasurementsaremadeofapersonorwhereapersoncanbeconsideredasameasurementinstrument.A brief introduction to conformity assessment is followed by two main sections,dealing, respectively, with uncertainty and with the treatment of risk assessmentassociatedwithuncertaintieswhenmakingdecisions.

Specialaspectsofmeasurementswithpersonsincludethecomplexityofthesubjectofmeasurement,rangingfromobjectivetomoresubjectivemeasuresandincludingperception, interpretation, and cognition. Uncertainties, which are often apprecia-ble,needtobeaccountedforinmultivariatemeasurement,exposure/stimulus,andhazard/response.Variability is relatively large,both foreach individualaswellasbetweendifferentindividuals.Basicconceptsofquality-assuredmeasurementfamil-iarin“traditional”metrologyinengineeringandphysics,suchastraceabilitytoref-erence standards and measurement uncertainty, can nevertheless be applied whenmeasuringhumans,ashasbeenexploredinoneofthemainthrustsoftheEUprojectMINET(2009).Indeedsuchconceptsofquality-assuredmeasurementareessentialinanysituation—suchashealthcareandinthecontrolroom—whereahumanbeingcanoftenbethemostcriticalortheweakestlinkintheoverallmeasurementsystem.

The MINET network has the ambition to advance the measurement of multi-dimensionalphenomena thataremediatedbyhumaninterpretationorperception,tobeabletoadvancethefrontiersofthescienceofmeasurement,andtorespondtofuturerequirementsformeasuringpropertiessuchascomfort,naturalness,perceivedquality,feelings,bodylanguage,andconsciousness.Atthesametime,oneisawareofandistryingtobridgethegapbetweenanengineeringtraditionwhichiscriti-cizedforafartooinstrumentalviewofoperatorsandthehumanisticandbehavioralsciencetraditionalltoopreoccupiedwithissuescenteredonhumanoperators.The

16

Y116134.indb 353 10/10/11 2:12 PM


MINETnetworkconsistsofauniquemixofphysicists,engineers,medicalscien-tists,neurologists,psychologists,andsociologists(EUNEST,2007).

MINET is investigatinghow”generic”metrological issuesof (1)measurementconceptsandterminology,(2)measurementtechniques,(3)measurementuncertainty,and(4)decisionmakingandimpactassessmentcanbeappliedspecificallytomeasure-mentofpersonsintermsofmanasameasurementinstrumentandmeasuringman.ThishasbeenstudiedamongothersinaseriesofMINETThinkTanks(2009)andpresentedinanInternet-basedonlineMINETRepository(2009).

Conformityassessmentprovidesconfidencethatrequirementshavebeenmetandisoftenessentialforreasonsofpublicinterest,publichealth,safety,andorder,protectionoftheenvironment,theconsumerandinensuringfairtrade.Assessmentisoftenbasedonmeasurementdata(EUCommission,2004,2005,2006;Codex2004).Measure-mentswithpersons,eitherwheremeasurementsaremadeofapersonorwhereaper-soncanbeconsideredasameasurementinstrument,arethefocusofthischapter.

Measurementandtestingofentitiesprovidevaluable,oftenquantitative,evidenceonwhichdecisionsofconformitycanbebased.Manytestsare,however,madeinpracticalsituationswheretimeandresourcesarelimited.Abalancehastobestruckbetweenexpenditureoninspectionandthepotentialcostsassociatedwithvariousrisks,toboththesupplierandcustomer,associatedwithincorrectdecisionsarisingfromlimitedmeasurementaccuracyandtestuncertainties(ILAC,1996;Williams&Hawkins,1993;Thompson&Fearn,1996;AFNOR,2004;Pendrill,2007).

Twomainsectionsof thischapterdeal, respectively,withuncertainty (Section16.2)andwiththetreatmentofriskassessmentassociatedwithuncertaintieswhenmakingdecisions(Section16.3).Adiscussionintermsofutilityofcommonrulesinconformityassessmentbasedonmeasurement isgiven.Themethodology isofgeneralapplicabilitybutisillustratedinthepresentworkwithexampleswhenmea-surementsanddecisionsaremadewithpersons.Optimumstrategiesforthesupplierareillustratedintermsofminimizingproductionandtestingcosts,whileatthesametimemaintainingsatisfactorylevelsofcustomersatisfaction.

Theapplicationofaunifiedapproachisasteptowardestablishingclearerpro-cedures for settingandspecifying tolerancesandassociateduncertainties, and infacilitatingacceptanceofconformitybybothcustomerandsupplier,eveninmea-surementsofperceptionandcognition.

16.2 Measurement of human performance and reliability

Thepresenceofahumanoperatorcanbeamajorfactorindeterminingtheover-allperformanceandreliabilityofameasurementsystem.Itisthereforeimpor-tanttodescribe,measure,assessconformity,andimprovetheperformanceandreliabilityofahumanasanessentialcomponentinanysystem,alongsideotherkeycomponents.

Theperformancecapabilityandreliabilityofahumanasacomponentofamea-surementsystemcanbedescribedinwayssomewhatsimilartothedescriptionofotherkeycomponentsof thesystem.Theclassicmodelofameasurementsystem(MSA,1995)isshowninFigure 16.1,wheretheobjectistheentitytobemeasured;

Y116134.indb 354 10/10/11 2:12 PM

Risk assessment and decision making 355

andahumanoperatormonitorsandcontrolstheobjectbyinterpretinginformationsignalsfromtheobjectcommunicatedviaameasurementinstrument.Thesethreebasicelements—object,instrument,andoperator—canallbeinfluencedindifferentways by the actual environment of measurement as well as the chosen measure-mentmethod.

Thesamebasicmeasurementsystemmodelcanalsobeappliedinassessingthecapabilityandreliabilityofahuman,byintroducingmanasoneorotherofthebasicelementsofthesystem.Ausefuldistinctioninmeasurementsconcerninghumansistoeitherviewahumanastheobjectonemakesmeasurementsonorasameasure-mentinstrument:

• Measuring Man: Measures of human body language and consciousnesscanbeimportantfactorsinassessinghumanperformanceandreliability,inotherwords,humanalertness,stress,andhealth(Section16.2.1).

• Man as Measurement Instrument: Humanperformanceandreliabilitycanbeunderstoodintermsofhumanperceptionofone’ssurroundingsintermsof perceived comfort, naturalness, quality, feelings, and so on—humanawareness(Section16.2.2)—increatingandmaintainingacorrectmentalmodelofthesystem.

Inthischapter,weusethisdistinctionrepeatedlyanddescribethesetwocasesinthefollowingsections.

16.2.1 Measuring man

In the case of measuring man, the measurement object in the measurement sys-tem (Figure 16.2) is a person and one often aims to assess human performance,health,andreliability.Thiscoversdisciplinesthatincludephysiology,psychology,psychophysics,psychometrics,andsociology.Inmakingmeasurementsonmanhim-self,forinstancewhenattemptingtoassesstheperformanceofanindividualinacontrolroom,awiderangeofmeasurementtechniquescanbeemployed,fromfunc-tionalmagneticresonanceimaging(fMRI)ofbrainactivitytoself-reportingfromthehumanmeasurementobjecthimself.Withall thecomplexity this impliesand

Environment

Instrument

Method

Object Operator

Figure 16.1 Manastheoperatorinameasurementsystem.

Y116134.indb 355 10/10/11 2:12 PM


wherethemeasurementobjectmayreactdifferentlyeachtimethemeasurementismade,uncertaintiescaninmanycasesbeexpectedtobeappreciable.

Table 16.1summarizesdifferentcategoriesofuncertaintyinperceptivemeasure-ment.Measurementuncertaintywilloftenbeadominantfactor,asanessentialfactorforquantifyingthequalityofthemeasurementsandinmakingdecisionsofconfor-mity,andreflectinglimitationsinmeasurementknowledge.Ontheotherhand,haz-ard(orresponse)uncertaintycanbeamainconcernwhenmeasuringman.

16.2.1.1 Example: Different measurements of human fatigue

Fatigueishighlyrelevantforoperatorsasaparameterwhenmeasuringman.Inarecent studyofdriveralertness inavehicle (Schleicher,Galley,Briest,&Galley,

Environment

Instrument

Method

Object Operator

Figure 16.2 Measuringman.

Table 16.1 Informationandmeasurement

Element Person Uncertainty Information Communication

Object Measuringman

Hazardandexposure

“Ambiguity”:nonspecific,one-to-manyrelation,variety,generality,diversity,anddivergence

Actofmeasurement(ormoregenerallyoftransmittingamessage)willreduceourignoranceoruncertainty

Instrument Manasinstrument

Measurement “Vagueness”:fuzziness,unclearness,indistinctness,etc.

Uncertaintyreflectscertainlossofinformationwhencommunicatingmessagefromtransmittertoreceiver

Operator Manasoperator

Measurement Lackofknowledgeorskill

Actofmeasurementwillreduceourignoranceoruncertainty—Bayes’theorem

Y116134.indb 356 10/10/11 2:12 PM


2008), two different measurement methods were employed to gauge the level offatigue.Inadditiontoaself-assessmentbythedriverofhisownlevelofalertnessatvarioustimesduringaparticulardrive,anovelmeasurementtechniquewasused,involvingvideomonitoringof theeyemovements, includingblinksandsaccades.Whencomparingthemeasurementresultsfromtwomethodssuchas in thiscase(Schleicheretal.,2008),itisimportantinassessingapparentdifferencesinalertnessestimatesobtainedbythetwomethodstoensurethateachmethodcanbetracedtocommonreferencestandards.Inaddition,itisdifficulttojudgethesignificanceofanydifferencesinresultsofthetwomethodsunlessmeasurementuncertaintiesforbothsetsofmeasurementresultsarenotonlycalculatedanddisplayedinanycom-parativepresentationbutalsointerpretedintermsofconsequenceandimpact.

16.2.1.2 Example: Risks of exposure and toxicity in measurements with persons

An example of conformity assessment when measuring man concerns the casewhereuncertaintiesinlevelsofexposureandinlevelsoftoxicityhavetobeweighedtogether in assessing overall risks when making measurements with persons. Insuchcases,asillustratedinFigure 16.3,estimatesoftheprobabilityriskofexposureareplottedtogetherwith thepredicted increase inriskof toxicitywith increasingexposure/dose. The dotted lines beside each main curve in Figure 16.3 representtheestimateduncertaintiesineachcumulativeprobabilitycurveforexposureandtoxicity.Theoverallimpactonhumanhealthisdeterminedbyacombinationoftheriskofexposureandtheriskoftoxicityforeachlevelofexposure.Wereturnbelow(Section16.3.2)toageneralapproachtointroducingmeasuresofimpactandriskinconformityassessmentinmeasurementswithman.

10

1 Risk ofExposure

Risk ofToxicity

0.1Prob

abili

ty (%

)

0.010.0001 0.001 0.01

Individual Exposure (IEXP) or Individual Effect Dose (ICED)0.1 1

Figure 16.3 Plottingtheriskofexposureagainsttheriskoftoxicityoverarangeofexposurelevels.ReproducedfromvanderVoetandSlob(2007),withpermission.

Y116134.indb 357 10/10/11 2:12 PM


16.2.2 Man as a measurement instrument

The instrument in the measurement system (Figure 16.4) can in some cases bereplaced by a person as is of interest, for instance, when aiming to understandhumanperformanceandreliabilityasacriticalelementinanysystem.AnexampleofthelatterisaseriesofpaintingsoftheHousesofParliamentinLondonmadebythefamousartistMonet(BakerandThornes2006):acomparisonwithactualinstru-mentalmeasurementsoftheappearanceandprofileofthebuildingsrevealssomeconsiderablevariationinMonet’sperception,bothfrompaintingtopaintingbutalsowithreality!Measurementuncertaintyisaprimeissuewhenthepersonishimselfactingasameasurementinstrument.

When man is the measurement instrument, we have exchanged the traditionalmeasurementinstrument,whichoftenisconsideredasobjective,withasubjectivehumanandperceptionwiththefivesenses.Measuredquantitiesincludeperceivedcomfort,naturalness,quality,feelings,andhumanawareness.Anextradimensionisthatthehumaninstrument(aswiththehumanobjectabove)mightbehighlyvariablewithtimeanditwillsometimesbedifficulttorepeatmeasurementsundersimilarconditions.Inthistypeofwork,thecalibrationoftheperceivedintensityofvari-ousstimulimightbechallenging(BerglundandHarju,2003),althoughthere isalongtraditioninsomeareas,suchasinvisionandhearing,ofquality-assuredmea-surementinphotometryandacoustics,respectively,whereexplicitallowanceforthehumanphysiologicalresponseisalreadyincluded.

16.2.2.1 Example: Operator reaction time

Inacontrolroom,ahumanmaywellintervenebytakingtheroleofameasure-mentinstrumentinmeasuringaparticularcharacteristicofanobjecttobestudied(suchas thesystemtobecontrolledor thedisplayofaninstrument inacontrolpanelconsole).

Oneexample ismeasurementandassessmentofhumanperformance responsetimeinthecaseofasignaler’sinterventioninarailincidentintheUnitedKingdom.Despitebeingidentifiedintheincidentinvestigationasamajorfactor,morerecent

Environment

MeasurementMethod

Object Operator

Figure 16.4 Manasameasurementinstrument.

Y116134.indb 358 10/10/11 2:12 PM


research(StantonandBaber,2008)hasshownthatthe18-secondresponsetimeofthesignalerisinnowayexceptionallylongandthatthereisawidevariabilityofhumanresponsetimeswheninterpretingandactingon,forexample,alarmsignalson instrument displays among different individuals and under different circum-stances.Insuchcases,keyfactorsindicatingthelevelofqualityofmeasurement,suchasmeasurementmethodaccuracyandmeasurementuncertaintywhendeter-mininghumanreactiontimes,canbecrucialinmakingobjectiveandcorrectdeci-sionsofconformity.

16.2.2.2 Example: Risks in nuclear waste storage

Apanelofexpertswasaskedtoestimatetheoverallprobabilityforacertaineventwithincreasingvaluesofaninfluenceparameterxinestimatingtheriskswithlong-termnuclearwastestorage(Helton,Johnson,Sallaberry,&Storlie,2006).Asnotedbytheseauthors,theexpertsdidnotnecessarilyhavetoformulateestimatedprobabilitydistri-butionfunctions(PDF)butonlyhowthecumulativeprobabilities(CDF)varyintheirjudgment.Anacceptableprocedureisthentotakeanaverageofthedifferentoperat-ingcharacteristiccurvesforthevariousexpertsinordertogetaconsensusvalue.

Thetoolcanalsobeemployedwhenuncertaintiesarebasednotonprobabilitytheory,butalsoonevidencetheory(plausibilityandbelief)andpossibilitytheory(possibilityandnecessity)(Heltonetal.,2006).CurvessuchasshowninFigure 16.5

1.0

0.9

0.8

0.7

0.6

0.5

Cum

ulat

ive P

roba

bilit

y

0.4

0.3

0.2

0.1

0.00.0

(b)

5.0 10.0

Mean CDFExpert 1Expert 2Expert 3

15.0Range of Possible Values for x

20.0 25.0

Mean overexperts

Verticalaverage

Figure 16.5 Operating characteristics judged on a number of experts. Reproduced fromHeltonetal.(2006),withpermission.

Y116134.indb 359 10/10/11 2:12 PM


(calledoperatingcharacteristicsorpowercurvesintraditionalstatisticalsignificancetesting;Montgomery,1996)arisewherethereisadistribution—eitherintrinsicorapparent—inquantityvalues;withoutsuchadistribution,associated,forexample,withuncertaintyduetoalessthancompleteknowledgeofexactresponseorexactlevelofexposure,probabilitieswouldshowsharplimitswhere,forinstance,theriskofexposurewouldfallabruptlytozeroataparticularexposurelevel.

Anaccountofstatisticalsignificancetesting(Montgomery,1996),includingthederivationofoperatingcharacteristics(powercurves)fromprobabilitydistributionfunctions,isdeferredtolaterinthischapter(Section16.3).

16.2.3 Measurement scales and measurement uncertainty in perception

Withmeasurementtechniquessodiverse,rangingfromfMRItoself-reporting,andfromobjectivetomoresubjectivejudgments,perceptivemeasurementscoveralargevarietyofclasses.Inordertotreatthesemeasurementscorrectlythereisavarietyof scales: ordinal, interval, and ratio (nominal scales are not usual referred to asmeasurement).Forthedifferentscalestherearedifferentpermissibleoperationsanddifferentappropriatestatistics.

Muchefforthasbeenexpendedinthemetrologycommunityoverthepastdecadeor so in giving guidance about, and harmonizing how to evaluate, measurementuncertainty(JCGM100,2008).Anymeasurementsystem(consistingofaninstru-ment,operator,method,environment,andtheobjectitself,asshowninFigure 16.1)isnotperfectanderrorscausedbylimitationsinthequalityoftheparticularmea-surement systemathandcanaffect the resultof anymeasurement.Measurementuncertaintyherereferstosituationswheretimeandresourcesarelimitedsothatnotallmeasurementerrorsarefullyevaluatedandcorrectedfor;thatis,limitedmeasure-mentqualitycanleadtoanapparentvariabilityinmeasurementdata.Measurementuncertaintycanbeevaluatedindifferentways,withprobabilitytheorybeingoneofthedominantschools.Specialaspectsoftheevaluationofmeasurementuncertaintywithpersonsincludesubjectivityandthatuncertainties,whichareoftenappreciable,need to be accounted for in multivariate measurement (Emardson, Jarlemark, &Floberg,2005;vanderHeijden&Emardson,thisvolume).

Apart frommeasurementuncertainty,other formsofuncertaintywillbeofconcern in general conformity assessment. For instance, in making decisionsof conformity about, say, environmental exposure of persons to some stimu-lus(noise,poison,etc.), therewillbeuncertaintiesboth in theactual levelsofexposureandofpotentialhazardsthatcertainlevelsofexposurecanmeanforpersons,whichmayofcoursevary fromcase tocaseand fromperson toper-son,asexemplifiedabove(Section16.2.1.2).Inmeasurementswithpersonsitisimportanttodistinguishbetweenuncertaintiesintheactuallevelsofexposureandhazardandbetweentheseandmeasurementuncertainties ineachof thesequalitycharacteristics.

Variouskindsofuncertaintycanberegardedasexamplesofamoregeneraltreat-mentofinformationtheory,assummarizedinTable 16.1.Inaprobabilisticapproach

Y116134.indb 360 10/10/11 2:12 PM


tomeasurementuncertainty,itisinterestingtonoteinthecontextofmeasurementswithpersonsthatuseofBayes’theoremallowstheinclusionofpriormeasurementknowledgebytheoperator,inwhichtheactualmeasurementenablesanupdatingofknowledgeabout themeasurementvalue.Priorknowledgecomplementsaregularstatisticalfrequentistkind,byaddingamoreexperience-basedevidence(wherecor-respondingmethodsofevaluationofmeasurementuncertaintyare termedtypeAandtypeB,respectively,intheGUMapproach).

Acceptingthatuncertaintyofallkindsisoftenassociatedwithlimitedcommuni-cationofinformation,KlirandFolger(1988)emphasizethattodealadequatelywithuncertaintyinhumancommunication, it isnecessarytoaccountfor threeaspectsofinformation:

Syntactic:RelationshipamongsignsemployedforcommunicationSemantic:Relationshipbetweensignsand theentities forwhich theystand,

thatis,meaningofsignsPragmatic:Relationshipbetweensignsandtheirutility

Asdiscussedinthecontextofconformityassessmentanddecisionmaking(Section16.3.2), ithasbeen foundusefulnotonly toemploysyntacticandsemantic typesofinformation,butalsotoincludepragmaticmeasuresofutilityinassessingrisksassociatedwithuncertainty.

16.3 Conformity assessment and measuring man

Theaimofconformityassessmentofatypeofentityistoassessconformanceofactualvaluesofacharacteristicwithrespect tospecificationlimits(Montgomery,1996;Joglekar2003).Dispersionincharacteristicentityvalueswillbeduetoactualvariabilityinthemanufacturingprocesswhentheproductismade.Subsequententityvariabilitywillbeduetowearandtearduringtheproductlifetime.

Thevarioustoolsofstatisticscanbeusedindifferentwaysinconformityassess-ment;theycanbeusedtodescribeactualvariabilityaswellasenablemodelingofprobablevariability.Suchstatisticalmodelingisusefulincaseswhereactualvari-abilityisnotknown,wherepriorknowledgeneedstobeincluded,andwhenpredic-tionsaremadeinadvanceinordertoplanformeasurementsinthebestway.

Inevaluatingproductionvariationsofquantityη = X in theentity(orproduct)space,measurementsmademultilaterallymightbeonrepeateditemsinaproductionprocessorbytakingasampleofthepopulationofitemssubjecttoconformityassess-ment.ThecorrespondingprobabilitydistributionfunctionPDF,gentity(x),willhaveaformdeterminedideally(intheabsenceofmeasurementorsamplinguncertainty)bytheintrinsicquantityvariationsofprimeinterestinconformityassessment.

Measurementqualityvariations,expressedwithameasurementuncertaintyPDFgtest,maypartiallymaskobservationsofactualentitydispersion.Assuch,measure-mentvariabilityisjustone,anditishoped,arelativelyminor,sourceofuncertaintythat needs to be accounted for when making decisions of conformity. A general

Y116134.indb 361 10/10/11 2:12 PM


challenge is to find methods that can be used reliably to separate these differentvariabilitycomponents.Uncertaintiesassociatedwithentityvariabilityandstimu-lusvariabilitywilloftenbeofdominantconcern.Overalldecisionsofconformitywillneedtoaccountforeachofthesedifferentkindsofuncertaintyinordertoassesscommensurateriskscorrectly(vanderVoetandSlob,2007).

Independentlyofwhetherconformityassessmentismadeofobjectsorpersons,it is important tospecify theassessment targetasclearlyaspossible:globalcon-formitydenotestheassessmentofpopulationsoftypicalentities,whereasspecificconformityassessmentreferstoinspectionofsingleitemsorindividuals(RossiandCrenna,2006).Specificationlimitsareoftensetinconformityassessmentonactualvaluesofacharacteristicofatypeofentity.

16.3.1 Uncertainties and risks of incorrect decision making

Animportantfactorinmakingdecisionsofconformityassessmentistoallowfortherisksofincorrectdecisionmakingarisingfromuncertainty.Testresultsclearlyinsideoroutsidetheregionsofpermissiblevaluescanleadreadilytounambiguousdecisionsaboutconformityornonconformity.

Uncertaintycanleadto:

• Correctlyconforminghumanoperatorsbeingincorrectlyfailedoninspection• Nonconforminghumanoperatorsbeingincorrectlypassedoninspection

particularly when a test result is close to a specification limit. As shown inFigure 16.6(a),atestresult,apparentlywithinlimits,mightactuallybenonconform-ing inasmuchas the tailof theprobabilitydistribution functionextendsslightlybeyondthelimit.Anexampleisanassessmentofcompliancewithspecifiedmini-mumreactiontimesforahumaninacontrolroom,asexemplifiedinthecaseoftherailwaysignaler(Section16.2.2.1).

Uncertaintycanalsoleadtoambiguitywhenassessingthesignificanceingeneralofanapparentdifferenceinpairsofmeasurementresults,forinstance,asobtainedfrom two different measurement methods (see example of driver fatigue, Section16.2.1.1).AsshowninFigure 16.6(b),twomeasurementresultscanbeexaminedastowhethertheyaresignificantlydifferentbyassessingthedistanceinentityvalueseparatingthetwodistributions’PDF.

Thereis,asiswellknown,acompletesetofstatisticalsignificancetestsfordistri-butionsofindividualandaveragevalues,aswellastestsofvariances.Theseincludeforvariablesthet-testandNormalteststodeterminewhetheranunknownpopulationmeandiffersfromastandardpopulationmean,andtheχ2-testandF-testtodeterminewhetheranunknownpopulationstandarddeviationisgreaterorlessthanastandardvalue (Ferris, Grubbs, & Weaver, 1946: Montgomery, 1996). Corresponding testswhensamplingbyattribute(i.e.,binarydecisionssuchasgo/no-goyieldingpercentnonconforming,forexample)canbebasedonthebinomialandPoissondistributions(Joglekar,2003).Thecomparisonandsignificance testingofmultiplepopulationscanbetackledbyconductinganalysisofvariance(ANOVA;Joglekar,2003).

Y116134.indb 362 10/10/11 2:12 PM


Risksandtheconsequencesofincorrectdecisionmakinginconformityassess-mentofhumanperformanceandreliabilityincontrolroomsshouldbeevaluated.Theycanbeminimizedby setting limitsonmaximumpermissiblemeasurementuncertaintiesandonmaximumpermissibleconsequencecosts.

16.3.2 Introducing impact and cost into conformity assessment risks

“Utility…isonlyadescriptionofthesubjectivejudgementofthedecision-maker….Itthusdoesnotappeartobemeasurement”(Finkelstein,2005).Butwewouldarguethatutilitydoesbecomeakeyfactorinconformityassessmentbasedonpragmaticmeasure-mentwithpersons.Ingeneral,theimpactofawrongdecisioninconformityassessmentisexpressedasariskRisk,definedastheprobabilitypofthewrongdecisionoccurringmultipliedbythecost(utility)Coftheconsequencesoftheincorrectdecision:

Risk=p·C (16.1)

asanexampleof themoregeneral,historicalexpressionofstatisticalexpectation(Meinrath,2008).Inthissection,newexpressionsfordecision-makingrisksinclud-ing costs arepresented, togetherwith anovel tool—theoperating cost character-isticcurve—asanextensionoftraditionalstatisticaltools,withtheadditionofaneconomic decision-theory approach. Complementarity with the optimized uncer-tainty methodology (Thompson and Fearn, 1996) is emphasized in the conclud-ingremarks.

Probability

(a)

Entity value

Probability

(b)Entity value

SL

Figure 16.6 Comparingadistribution(PDF)ofentityvalues:(a)withaspecificationlimit,SL;(b)betweentwosetsofobservations.

Y116134.indb 363 10/10/11 2:12 PM


16.3.3 Conformity Assessment by Variable

Inspectionbyvariableinvolvesmeasurementofthemagnitudeymofacharacteristicofanitem(ISO3951-12005).Takingaccountofvaryingcosts/impactfordifferentquantityvaluesinthecasewhereproductcostvaluevarieswithunitquantityvaluewiththecostfunctionC(η),leadstothefollowing:

Customer’s cost risk:

C y C g y dspecific m testL

mSL

( ) ( ) ( )= ⋅ ⋅<∫ η η η

η (16.2)

withym≥LSL.

Supplier’s cost risk:

C y C g y dspecific m testL

mSL

* ( ) * ( ) ( )= ⋅ ⋅≥∫ η η η

η (16.3)

withym<LSLforspecificconformityassessmentinvolvingtheinspectionofanentitywith respect toa (lower)specification limitLSL.The integral inEquation(16.2), forinstance, corresponds to the tail of the probability distribution function extendingslightlybeyondthespecificationlimitillustratedinFigure 16.6(a)foratestresultappar-entlywithinlimits.

An incorrectaccepton inspectionofanonconformingobjectwill lead tocus-tomercostsassociatedwithanout-of-toleranceproduct.Overallcosts,consistingofasumoftestingcostsandthecostsassociatedwithcustomerrisk(Equation16.2),canbecalculatedwiththeexpression:

E y D y C yD

C g ym m mR

mPV

( , ) ( , ) ( ) ( ) ( )σ σσ

η ηη

= + = + ⋅∉∫2 ⋅⋅dη (16.4)

withym∈RPV,whereRPVdenotestheregionofpermissibleentityvalues,wheretestcostsDaremodeledasvaryinginverselytothesquareddispersionσ.Expression(16.4)canbeappliedtobothspecificandglobalconformityassessment(Pendrill,2007).

Overall costs E(ym,σ), according to (16.4) can be plotted over either of thefollowing:

1.ArangeofquantityvaluesofLSL–h·σ≤ym≤LSL+h·σforagiventestdis-persionσ,andguard-bandfactorh,yieldingan“operatingcostcharacteris-tic”analogoustothetraditional,probability-basedoperatingcharacteristic.

2.Arangeoftestuncertaintiesσforagivenquantityvalueym≥LSL,theso-called“optimizeduncertaintycurve.”

Y116134.indb 364 10/10/11 2:12 PM


Itispossibletoviewthetwotools—theoperatingcostcharacteristics(1)andopti-mizedsampling(2)uncertaintymethodologies—astogetherprovidingacompletebasis for risk assessment in conformity assessment. Overall costs are plotted inthreedimensions,showninFigure 16.7,whereateachentityvalueontheoperatingcharacteristic curve, the corresponding optimized uncertainty curve would crossin the orthogonal direction. The U-shaped optimized uncertainty curve (case 2above,wherethecostsoftestingarebalancedagainstthecostsofincorrectdecisionmaking)isclearlyvisiblealongtheuncertaintyaxisofthe3-DplotinFigure 16.7. Inthisway,theoptimumuncertaintyrequiredatspecificconformityassessmentpoints,suchasthoseforcustomerandsupplierrisk,canbeidentifiedacrossthefullrangeofentityvalues.

16.4 Conclusions

Havingagreedonappropriatedescriptorsofhumanperformanceandreliabilityandhowtomeasureandassessthem,anextstepwouldbetoconsiderwaysofimprov-ingthese.Anexampleofhowtheeffectsoftrainingcanbemeasuredandimprovehumanperformanceand reliability incontrol roomsituations isSauer,Burkolter,Kluge,Ritzmann,andSchüler(2008).

SLEntity Value

Uncertainty

0

1000

2000

3000

4000

5000

Cost

of R

ejec

tion

(₡)

6000

Figure 16.7 OverallcostsE(ym,σ)(Equation16.4)versusuncertaintyσandentityvalueyminthevicinityofaspecificationlimitSL.(AdaptedfromPendrill,2008.)

Y116134.indb 365 10/10/11 2:12 PM


Theworkpresentedhereextendstraditionalsignificancetestingtoincludeutilityandeconomicassessmentsofthecostsofmeasuring,testing,andsamplingtogetherwiththecostsofincorrectdecisionmaking.Thisprovidesthedecisionmakerwithamorecompleteandarguablymorerelevantassessmentofexpectationsandrisksaddi-tionallyincludingmeasuresofimpact,importance,andconsequence.Thisisespeciallyimportantwhenassessingperceptiveorcognitivemeasurementswhereahumanbeingcanoftenbethemostcriticalortheweakestlinkintheoverallmeasurementsystem.

Acknowledgments

ThanksareowingfordiscussionsabouttheperformanceofthetestingofdifferentmeasurementinstrumentstoHåkanKällgren,Jan-ErikElander,JonatanWesterberg,andothercolleaguesatSP.TheauthorhasalsobenefitedfromparticipationintheJointCommitteeforGuidesinMetrology(JCGM),WorkingGroup1.Thisworkhasbeenpartiallyfinancedbygrant38:10NationalMetrologyoftheSwedishMinistryofIndustry,EmploymentandCommunication.

References

AFNOR(2004).Metrologyandstatisticalapplications:Useofuncertainty inmeasurement:Presentationofsomeexamplesandcommonpractice,FDx07-022,2004.

Baker, J. & Thornes, J. E. (2006). Solar position within Monet’s Houses of Parliament.Proceedings of the Royal Society A, 462,3775–3788,doi:10.1098/rspa.2006.1754.

Berglund,B.,&Harju,E.-L.(2003).Masterscalingofperceivedintensityoftouch,coldandwarmth.European Journal of Pain,7,323–334SpecialIssuededicatedtoUlfLindblom,doi:10.1016/S1090-3801(03)00043-0.

Codex(2004).General guidelines on sampling(codexalimentariuscommissionFAO/WHOof the United Nations, CAC/GL 50-2004), www.codexalimentarius.net/download/standards/10141/CXG_050e.pdf.

Emardson,T.R.,Jarlemark,P.O.J.,&Floberg,P.(2005).Uncertaintyevaluationinmultivari-ateanalysis:Atestcasestudy.Journal of Mathematical Modelling and Algorithms, 4,289–305,doi:10.1007/s10852-005-9005-2.

EU Commission (2004). MID Measurement instrument directive PE-CONS 3626/04 MID2004/22/EC(2004),http://europa.eu/eur-lex/pri/en/oj/dat/2004/l_135/l_13520040430en00010080.pdf.

EU Commission (2005). Metrology, pre-packaging, http://ec.europa.eu/enterprise/prepack/metrol_requir/inmetrolog_requir_en.htm;OIML2004R87Quantityofproductinpre-packages;OIML2004E4ThestatisticalprinciplesofthemetrologicalsurveillanceofthenetcontentofprepackagesaslaiddownbytheCEE76/211Directive,InternationalRecommendation,InternationalOrganisationofLegalMetrology(www.oiml.org).

EUCommission(2006).N560-2EN2006-0906AnnextoAHorizontalLegislativeApproachto the Harmonisation of Legislation on Industrial Products, European Commission,http://ec.europa.eu/enterprise/newapproach/review_en.htm.

EU NEST (2007). Measuring the impossible projects. ftp://ftp.cordis.europa.eu/pub/nest/docs/1-nest-measuring-290507.pdf.

Y116134.indb 366 10/10/11 2:12 PM


Ferris,C.D.,Grubbs,F.E.,&Weaver,C.L.(1946).Operatingcharacteristicsforthecommonstatisticaltestsofsignificance.Annals of Mathematical Statistics,17,178–197.

FinkelsteinL(2005).Measurement, 38,267–274.GUM(2008).Evaluation of measurement data: Guide to the expression of uncertainty in

measurement, JCGM 100:2008 (GUM 1995 with minor corrections), http://www.bipm.org/en/publications/guides/gum.html.

Helton,J.C.,Johnson,J.D.,Sallaberry,C.J.,&Storlie,C.B.(2006).Surveyofsampling-based methods for uncertainty and sensitivity analysis. Reliability Engineering and Safety Systems,91,1175–1209.

ILAC(1996). Guidelines on assessment and reporting of compliance with specification,Guideno. 8.

ISO3951-1 (2005).Sampling procedures for inspection by variables—Part 1, ISOGeneva(CH).

JCGM100(2008).Evaluation of measurement data—Guide to the expression of uncertainty in measurement, GUM 1995 with minor corrections.

JCGMWG1(2009).Evaluation of measurement data—The role of measurement uncertainty in deciding conformance to specified requirements,draft GUM Supplement JCGM 106April.

Joglekar, A. M. (2003). Statistical methods for Six Sigma in R&D and manufacturing.Hoboken,NJ:Wiley.

Klir,G.J.,&Folger,T.A.(1988).Fuzzy sets, uncertainty and information.EnglewoodCliffs,NJ:Prentice-Hall.

Meinrath,G.(2008).Lecturesforchemistsonstatistics.I.Belief,probability,frequencyandstatistics:Decision-makinginafloatingworld,Accredited Quality Assurance, 13,3–9,doi:10.1007/s00769-007-0333-y.

MINET(2009).http://minet.wordpress.com.MINETRepository(2009).http://minet3repository.wordpress.com/.MINETThinkTanks(2009).http://lesliependrill.wordpress.com/minet-wp3-think-tanks/.Montgomery,D.C.(1996).Introduction to statistical quality control.Hoboken,NJ:Wiley.MSA(1995).MeasurementsystemanalysisMSA.ChryslerCorp,FordMotorCompanyand

GeneralMotorCorporation,Referencemanual,2ndedition.Pendrill,L.R.(2006).Optimisedmeasurementuncertaintyanddecision-makingwhensam-

plingbyvariablesorbyattribute.Measurement,39,829–840,AdvancedMathematicalTools for Measurement in Metrology and Testing, http://dx.doi.org/10.1016/j.measurement.2006.04.014.

Pendrill,L.R.(2007).Optimisedmeasurementuncertaintyanddecision-makinginconfor-mityassessment,NCSLi MEASURE, 2(2),76–86.

Pendrill, L. R. (2008). Operating ‘cost’ characteristics in sampling by variable and attribute.Accredited Quality Assurance, 13,619–631,doi:10.1007/s00769-008-0438-y.

Pendrill,L.R.,&Källgren,H.(2006a).Decision-makingwithuncertaintyinattributesam-pling.Advanced Mathematical and Computational Tools in Metrology VIIinSeriesonAdvances in Mathematics forApplied Sciences 72, pp. 212– 220, Singapore:WorldScientific.

Pendrill,L.R.,&Källgren,H.(2006b).Exhaust gas analysers and optimised sampling, uncer-tainties and costs, Accredited Quality Assurance 11, 496–505, doi: 10.1007/s00769-006-0163-3.http://dx.doi.org/10.1007/s00769-006-0163-3.

Y116134.indb 367 10/10/11 2:12 PM


Pendrill, L. R., & Källgren, H. (2008). Optimised measurement uncertainty and decision-making in the metering of energy, fuel and exhaust gases. Izmeritelnaya Technika (Measurement Techniques), 4,18–22,April.doi:10.1007/s11018-008-9047-8.

Rossi,G.B.,&Crenna,F.(2006).Aprobabilisticapproachtomeasurement-baseddecisions.Measurement, 39,101–119.

Sauer,J.,Burkolter,D.,Kluge,A.,Ritzmann,S.,&Schüler,K.(2008).Theeffectsofheuris-ticruletrainingonoperatorperformanceinasimulatedprocesscontrolenvironment.Ergonomics,51,953–967,doi:10.1080/00140130801915238.

Schleicher,R.,Galley,N.,Briest,S.,&Galley,L.(2008).Blinksandsaccadesasindicatorsoffatigueinsleepinesswarnings:Lookingtired?Ergonomics, 51(7July),982–1010,doi:10.1080/00140130701817062.

Stanton, N.A., & Baber, C. (2008). Modelling of human alarm response times:A casestudy of the Ladbroke Grove rail accident. Ergonomics, 51(4), 423–440, doi:10.1080/00140130701695419.

Thompson,M.,&Fearn,T.(1996).Whatexactlyisfitnessforpurposeinanalyticalmeasure-ment?Analyst,121,275–278.

vanderHeijden,G.&Emardson,R.(thisvolume).Multivariatemeasurements.InB.Berglund,G.B.Rossi,J.T.Townsend,&L.R.Pendril(Eds.).Measurement With Persons: Theory, Methods, and Implementation Areas(pp.125–142).NewYork:PsychologyPress.

vanderVoet,H.,&Slob,W.(2007).Integrationofprobabilistichazardcharacterization.Risk Analysis,27,351–371.

Williams,R.H.,&Hawkins,C.F.(1993).Theeconomicsofguardbandplacement.Proceedings, 24th IEEE International Test Conference,Baltimore,MD,Oct.17–21.

Y116134.indb 368 10/10/11 2:12 PM

measurement with persons

Documents