Psych 818Psych 818Psychometric Theory and Psychometric Theory and

Measure ConstructionMeasure ConstructionDeShon - 2007DeShon - 2007

PreliminariesPreliminaries Who am I?Who am I?

Your job - Ask questions and find a way to Your job - Ask questions and find a way to become intellectually engagedbecome intellectually engaged Self-handicapping is unproductiveSelf-handicapping is unproductive You have a huge role in the success of the classYou have a huge role in the success of the class

All work is your own!All work is your own!

Syllabus and GradingSyllabus and Grading

PurposePurpose Measure UseMeasure Use Measure InterpretationMeasure Interpretation Measure DevelopmentMeasure Development

Exposure to Measurement thinkingExposure to Measurement thinking Exposure to Measurement papersExposure to Measurement papers Exposure to Measurement StandardsExposure to Measurement Standards

Responsibility!Responsibility! As a Psychologist As a Psychologist YOUYOU will likely have will likely have

substantial effects on the course of many substantial effects on the course of many lives through the use of tests and lives through the use of tests and measuresmeasures Clinical DiagnosisClinical Diagnosis Policy Making & Risk FactorsPolicy Making & Risk Factors Educational OpportunitiesEducational Opportunities Work OpportunitiesWork Opportunities

Example: Police promotion & expert witnessExample: Police promotion & expert witness You will have the power to help or hurt You will have the power to help or hurt

many people (and their families!)many people (and their families!)

Standards for MeasurementStandards for Measurement APA standards for educational and APA standards for educational and

psychological testing and psychological testing and assessmentassessment This is your professional bible.This is your professional bible. You will be held professionally You will be held professionally

accountable to these standardsaccountable to these standards Therefore, you better know them! ;)Therefore, you better know them! ;)

Your responsibility and fair game for test Your responsibility and fair game for test questionsquestions

Beginning QuestionsBeginning Questions Why do we measure?Why do we measure?

What is measurement?What is measurement?

Why do we measure?Why do we measure? Decision makingDecision making

PredictionPrediction

ResearchResearch

What is Measurement?What is Measurement? S.S. StevensS.S. Stevens

The assignment of numerals to objects or events The assignment of numerals to objects or events according to a rule.according to a rule.

Warren TorgersonWarren Torgerson The assignment of numbers to The assignment of numbers to perceived perceived

attributesattributes of objects or events according to a of objects or events according to a rule.rule.

Lyle JonesLyle Jones Measurement is the determination of the Measurement is the determination of the

magnitude of a specified attribute of an object or magnitude of a specified attribute of an object or event in terms of a unit of measurement.event in terms of a unit of measurement.

Classification (including ordering) is not Classification (including ordering) is not measurement.measurement.

What is Measurement?What is Measurement? Cliff’s definitionCliff’s definition GoodGood measurement is the measurement is the

assignment of numbers to perceived assignment of numbers to perceived attributes of objects or events attributes of objects or events according to rules that according to rules that are easily understood; are easily understood; are easily used; are easily used; yield numbers that are as simply related yield numbers that are as simply related

as possible to as many other sets of as possible to as many other sets of measurements as possible. measurements as possible.

History of MeasurementHistory of Measurement Most early measurements occurred in Most early measurements occurred in

physics, chemistry, and much later in physics, chemistry, and much later in biology (e.g., bacteria)biology (e.g., bacteria)

Much attention paid to measuring Much attention paid to measuring specific quantities such as specific quantities such as temperature, pressure, pH, etc...temperature, pressure, pH, etc... But no serious attention given to a But no serious attention given to a

general theory of measurementgeneral theory of measurement Also, little attention paid to a general Also, little attention paid to a general

model of reliability and validitymodel of reliability and validity

Two Physical Measurement Two Physical Measurement Examples Examples

TemperatureTemperature Fundamental to virtually all future Fundamental to virtually all future

discoveries in physicsdiscoveries in physics

PHPH Fundamental to the understanding of the Fundamental to the understanding of the

basic elements (i.e., hydrogen) and basic elements (i.e., hydrogen) and chemistrychemistry

TemperatureTemperature In the beginning there was the PERCEPTION of In the beginning there was the PERCEPTION of

hot and coldhot and cold Aristotle’s fundamental qualitiesAristotle’s fundamental qualities Hot and cold were conceived of as opposing qualities Hot and cold were conceived of as opposing qualities

of objectof object Virtually all measurements are rooted in subjective Virtually all measurements are rooted in subjective

perceptionperception Perceptions are fallible!Perceptions are fallible!

200BC Philo’s air thermoscope200BC Philo’s air thermoscope

Temperature (cont.)Temperature (cont.) 1592 Galileo reinvents 1592 Galileo reinvents

Philo’s (200BC) air Philo’s (200BC) air thermoscope (instrument thermoscope (instrument or indicator)or indicator) This is an indicator with no This is an indicator with no

scalescale Not a measurement?Not a measurement?

Very sensitive to Very sensitive to temperature changestemperature changes

But also open to the air so But also open to the air so sensitive to other variables sensitive to other variables (e.g., Air Pressure)(e.g., Air Pressure)

validity problemvalidity problem

Problems with Air ThermoscopesProblems with Air Thermoscopes

No scale – so a qualitative instrumentNo scale – so a qualitative instrument Construct contamination - ValidityConstruct contamination - Validity

open to the air so sensitive to other open to the air so sensitive to other variables (e.g., Air Pressure)variables (e.g., Air Pressure)

What liquid to use? (water, spirits, What liquid to use? (water, spirits, quicksilver)quicksilver) State changes, sensitivity and linearity of State changes, sensitivity and linearity of

expansion ratesexpansion rates No standardized construction so other No standardized construction so other

instruments are not equivalent instruments are not equivalent Inaccurate conceptualization of heatInaccurate conceptualization of heat

Fix the validity problemsFix the validity problems Seal the thermoscopeSeal the thermoscope

Duke Ferdinand II (1632)Duke Ferdinand II (1632) Pascal (1663) proved that air pressure Pascal (1663) proved that air pressure

caused errors in open air thermometerscaused errors in open air thermometers Are measurement errors reliability or validityAre measurement errors reliability or validity

Use quicksilver (i.e., mercury)Use quicksilver (i.e., mercury) FareinheitFareinheit

Fix the scaling problemsFix the scaling problems Many individuals developed scales to assign numbers to the Many individuals developed scales to assign numbers to the

level of liquid in the level of liquid in the thermometerthermometer (now a thermo – meter)! (now a thermo – meter)! Newton (1700)Newton (1700)

""zeroth degree of heat" as melting snow and "33 degrees of zeroth degree of heat" as melting snow and "33 degrees of heat" as boiling waterheat" as boiling water

Rømer (1708)Rømer (1708) RéaumurRéaumur (1731) (1731)

freezing point of water is 0 degrees, the boiling point 80 freezing point of water is 0 degrees, the boiling point 80 degrees degrees

Fareinheit (1724)Fareinheit (1724) 0 as the melting point of an equal mixture of ice and salt and 0 as the melting point of an equal mixture of ice and salt and

96 as blood heat96 as blood heat Delisle (1732)Delisle (1732)

2400 graduations: 0 degrees as the boiling point and 150 2400 graduations: 0 degrees as the boiling point and 150 degrees as the freezing point of water. degrees as the freezing point of water.

Celcius (1742)Celcius (1742) freezing point of water at 0 °C and the boiling point at 100 °C freezing point of water at 0 °C and the boiling point at 100 °C

Proliferation of ScalesProliferation of Scales

UnitsUnits These scales result in “degrees” or These scales result in “degrees” or

unitsunits

What are some psychological units?What are some psychological units?

Check out these physical units!Check out these physical units! http://www.unc.edu/~rowlett/units/http://www.unc.edu/~rowlett/units/

Comparison of temp. scalesComparison of temp. scales

290944211823−814010440998055265800The surface of the Sun

8831334550−23523494303416681941Titanium melts

6080330671.67212100373.15Water boils (at standard pressure)

26.92529.612.2194.5557.998.236.8310.0Average human body temperature

7.500150491.67320273.15Water freezes (at standard pressure)

−1.83−14.22−5.87176.67459.670−17.78255.37Fahrenheit's ice/salt mixture

−135.90−218.52−90.14²559.7250−459.67−273.150Absolute zero

RømerRéaumurNewtonDelisleRankineFahrenheitCelsiuskelvin

Transformations to Equate scalesTransformations to Equate scales

[K] = 373.15 − [°De] · 2/3[°De] = (373.15 − [K]) · 3/2Delisle

[K] = ([°Rø] − 7.5) · 40/21 + 273.15[°Rø] = ([K] − 273.15) · 21/40 + 7.5Rømer

[K] = [°N] · 100/33 + 273.15[°N] = ([K] − 273.15) · 33/100Newton

[K] = [°Ré] · 5/4 + 273.15[°Ré] = ([K] − 273.15) · 4/5Réaumur

[K] = [°Ra] · 5/9[°Ra] = [K] · 9/5Rankine

[K] = ([°F] + 459.67) · 5/9[°F] = [K] · 9/5 − 459.67Fahrenheit

[K] = [°C] + 273.15[°C] = [K] − 273.15Celsius

Fix the standardization or Fix the standardization or calibration problemscalibration problems

The technology of glass blowing The technology of glass blowing made it very difficult to construct made it very difficult to construct exactly equal instruments (the size of exactly equal instruments (the size of the liquid resivoir, the amount of the liquid resivoir, the amount of liquid, the bore of the tube, etc…all liquid, the bore of the tube, etc…all must be identical to yield identical must be identical to yield identical readings.readings.

Instead, calibrate or equate using Instead, calibrate or equate using fixed reference pointsfixed reference points

Royal Society thermometersRoyal Society thermometers

Theoretical UnderstandingTheoretical Understanding The various scales of measurement and The various scales of measurement and

the calibration of thermometers to fixed the calibration of thermometers to fixed points allowed systematic investigationspoints allowed systematic investigations

That resulted in improved understanding That resulted in improved understanding of what thermometers measure (heat)of what thermometers measure (heat)

Kelvin’s scale of temperature (1848)Kelvin’s scale of temperature (1848) Rational scale with absolute zeroRational scale with absolute zero Based on gas laws and carnot cyclesBased on gas laws and carnot cycles Adopted in 1954!Adopted in 1954!

Example 2: pH - Acids & BasesExample 2: pH - Acids & Bases In the beginning there was … In the beginning there was …

perceptionperception Greeks attempted to characterize natureGreeks attempted to characterize nature One fundamental character of nature One fundamental character of nature

was taste.was taste. SaltySalty Sour Sour

the greek word for sour is the base of our word the greek word for sour is the base of our word for acidfor acid

BitterBitter SweetSweet

Acids and BasesAcids and Bases Next, it turned out that sour tasting Next, it turned out that sour tasting

substances had other regular substances had other regular propertiesproperties changes the color of litmus (a dye changes the color of litmus (a dye

extracted from a lichen commonly extracted from a lichen commonly known as dyer's weed.) known as dyer's weed.)

Corrodes metalsCorrodes metals

No real understanding of acidsNo real understanding of acids

AcidsAcids Huge proliferation of acid types (named by Huge proliferation of acid types (named by

the method used to form the acid)the method used to form the acid) Muratic acid, acetic acid, sulphuric acid, Muratic acid, acetic acid, sulphuric acid,

hydrofluoric acid, phosphoric acid, pyruvic acidhydrofluoric acid, phosphoric acid, pyruvic acid

No clear picture of the cause for the No clear picture of the cause for the common functioning of acidscommon functioning of acids LavoisierLavoisier (1776) thought it was oxygen that (1776) thought it was oxygen that

caused substances to be acidiccaused substances to be acidic Greek word Greek word oxeinoxein meaning 'sour' and meaning 'sour' and gennangennan

meaning 'to generatemeaning 'to generate

AcidsAcids Leibig (1850s) noticed that all acids Leibig (1850s) noticed that all acids

have hydrogen in commonhave hydrogen in common This insight required knowledge of the This insight required knowledge of the

elements, molecules, and weights of elements, molecules, and weights of molecules (another measurement molecules (another measurement problem)problem)

Sorenson (1909) – developed the Sorenson (1909) – developed the scale for indexing the activity of scale for indexing the activity of hydrogen in a substancehydrogen in a substance

AcidsAcids pH = (power of hydrogen ) is a pH = (power of hydrogen ) is a

measure of the activity of hydrogen measure of the activity of hydrogen ions (H+) in a solutionions (H+) in a solution

Log of number of HLog of number of H+ molecules in a molecules in a substance substance

pH value has no unit but it is not an pH value has no unit but it is not an arbitrary scalearbitrary scale

The number arises from a definition based The number arises from a definition based on the activity of hydrogen ions in the on the activity of hydrogen ions in the solution – has an absolute zerosolution – has an absolute zero

Measurement of pHMeasurement of pH Many ways now exist to measure pH.Many ways now exist to measure pH. Early measurements were based on the Early measurements were based on the

color of a substance resulting from a color of a substance resulting from a chemical reactionchemical reaction

pH indicatorpH indicator A A pH indicatorpH indicator is a is a halochromichalochromic chemical chemical

compoundcompound that is added in small amounts to a that is added in small amounts to a solutionsolution so that the so that the pHpH of the solution can be of the solution can be determined easily. Hence a pH indicator is a determined easily. Hence a pH indicator is a chemicalchemical detector for detector for protonsprotons ( (HH+). Normally, +). Normally, the indicator causes the the indicator causes the colorcolor of the solution to of the solution to change depending on the pH.change depending on the pH.

-Wilkpedia -Wilkpedia

pH MeasurementpH Measurement

yellow11.4-13.0blueIndigo carmine

orange-red10.1-12.0yellowAlizarin Yellow R

blue9.4-10.6colorlessThymolphthalein

pink8.2-10.0colorlessPhenolphthalein

blue8.0-9.6yellowThymol blue

red6.6-8.0yellowPhenol red

blue4.5-8.3redLitmus (Azolitmin)

yellow4.2-6.3redMethyl red

red3.0-5.2blueCongo red

yellow2.9-4.0redMethyl yellow

yellow1.2-2.8redThymol blue

blue-violet0.0-1.6yellowMethyl violet

Color - high pHTransition pH rangeColor -

low pHIndicator

pH MeasurementpH Measurement Adding the indicator does not result Adding the indicator does not result

in an absolute shift from one color to in an absolute shift from one color to the other.the other.

Instead, it is a gradual shift and the Instead, it is a gradual shift and the resulting color must be compared to resulting color must be compared to a SCALE to determine the pH.a SCALE to determine the pH.

Scaling pH IndicatorsScaling pH Indicators

pH and IRT similaritiespH and IRT similarities The colormetric approach to pH has a The colormetric approach to pH has a

great deal of overlap with current great deal of overlap with current Item Response approaches to Item Response approaches to measurement – IRTmeasurement – IRT

Keep this in mind when we get to IRTKeep this in mind when we get to IRT

More History of MeasurementMore History of Measurement Representational Measurement Representational Measurement

TheoryTheory Use numbers to “represent” the amount Use numbers to “represent” the amount

of an object characteristic (e.g., weight) of an object characteristic (e.g., weight) Counting in agriculture – how many Counting in agriculture – how many

cows?cows? This simple and natural use of This simple and natural use of

numbers introduced a huge problem numbers introduced a huge problem in the theory of measurementin the theory of measurement Numbers have specific propertiesNumbers have specific properties Do the quantities we want to measure Do the quantities we want to measure

have these same properties?have these same properties?

Properties of numbersProperties of numbers Order/monotonicityOrder/monotonicity CommutativityCommutativity

A + B = B + AA + B = B + A AssociativityAssociativity

(A + B) + C = A + (B + C)(A + B) + C = A + (B + C) TransitivityTransitivity

If A = B and B = C then A = CIf A = B and B = C then A = C

Representational Measurement Representational Measurement TheoryTheory

Helmholtz (1887) and HHelmholtz (1887) and Höölder (1901)lder (1901) 2 properties of measurement 2 properties of measurement

Binary combination operation= Binary combination operation= ºº Ordering relation = ≥Ordering relation = ≥

Weight orderingWeight ordering Balance scaleBalance scale

Length orderingLength ordering RodsRods

Extensive vs. Derived Extensive vs. Derived MeasurementMeasurement

Extensive MeasurementExtensive Measurement Based on addition rule for combinationBased on addition rule for combination Ex: distance, mass, timeEx: distance, mass, time

Derived Measurement:Derived Measurement: Created by mathematical operations Created by mathematical operations

using functions of extensive measuresusing functions of extensive measures Ex: velocity = distance / time (speed of Ex: velocity = distance / time (speed of

light meters per second) , Newtonian light meters per second) , Newtonian constant of gravitation, etc.constant of gravitation, etc.

History of MeasurementHistory of Measurement Campbell (1920; 1940) argued that Campbell (1920; 1940) argued that

psychological measurement was psychological measurement was impossibleimpossible1.1. Prerequisite of measurement is some Prerequisite of measurement is some

form of empirical quantification that can form of empirical quantification that can be experimentally accepted or rejectedbe experimentally accepted or rejected

2.2.Only known form of such quantification Only known form of such quantification that satisfies the axioms of extensive that satisfies the axioms of extensive measurement is the binary operation of measurement is the binary operation of concatenationconcatenation

3.3.Psychology has no extensive Psychology has no extensive measurement measurement

History of MeasurementHistory of Measurement Stevens (1951) argued that this Stevens (1951) argued that this

perspective on measurement was too perspective on measurement was too narrownarrow Various levels or types of measurementVarious levels or types of measurement

Table 1, Stevens 1946, page 678)Table 1, Stevens 1946, page 678)

Scales of MeasurementScales of MeasurementScale Basic Operations Mathematical Group Permissible Statistics

Nominal Determination of Equality

Ordinal Determination of Order

Interval

Ratio Coefficient of variation

Permutation Group – one to one substitution Num ber of cases; mode

Isotonic Group – Monotonic function

Median, percentiles, Range

Determination of Equality of Differences

General linear group xt = ax + b

MeanStandard deviationRank-order correlationProduct-moment correlation

Determination of Equality of Ratios Similarity Group xt = ax

History of MeasurementHistory of Measurement Luce, Narens, TverskyLuce, Narens, Tversky

Conjoint polynomial measurementConjoint polynomial measurement Proved that additivity (interval scaling) Proved that additivity (interval scaling)

can be obtained by using transformations can be obtained by using transformations of two variables that are on an ordinal of two variables that are on an ordinal scalescale

Causality and MeasurementCausality and Measurement Measurement requires causal Measurement requires causal

inferenceinference Changes in the level of the latent entity Changes in the level of the latent entity

cause changes in the level of the cause changes in the level of the indicator variable(s).indicator variable(s).

This inference requires the same This inference requires the same scientific method as any causal research scientific method as any causal research question.question.

Must show that the latent variable is the Must show that the latent variable is the only cause of the indicator (construct only cause of the indicator (construct validity)validity)

ConstructsConstructs What is a construct?What is a construct? Constructs become better defined via Constructs become better defined via

researchresearch Temperature, pH, bacteria, gas pressure, Temperature, pH, bacteria, gas pressure,

electricityelectricity 1. An unobserved 1. An unobserved causecause of variation of variation

in an observable variable(s)in an observable variable(s) 2. A label used to describe a pattern of 2. A label used to describe a pattern of

observed covariancesobserved covariances

Model of MeasurementModel of Measurement

Construct

Construct = an unobservable but real causal variable(aka latent variable)

Indicator / Instrument

Indicator = an observable variable thatis solely caused by the construct andsensitive to changes in the level of the construct

Scale: the rule for assigning numbers to the levels of theindicator or instrument

Key Assumptions for classKey Assumptions for class Psychological Constructs are realPsychological Constructs are real

Reflect actual properties or Reflect actual properties or characteristics of peoplecharacteristics of people

Not just a descriptive listing of Not just a descriptive listing of correlatescorrelates

Measurement error reflects the effect Measurement error reflects the effect of non-random nuisance variablesof non-random nuisance variables No such thing as random measurement No such thing as random measurement

error!error!

What’s your construct?What’s your construct? Over the next week, your primary job Over the next week, your primary job

is to identify a construct/variable that is to identify a construct/variable that interests you.interests you. Feel free to think outside the box!Feel free to think outside the box! Identify some behavior or human Identify some behavior or human

process that interests you and then process that interests you and then think about the cause of that behavior. think about the cause of that behavior. The cause is the construct.The cause is the construct.

Some quotes to think aboutSome quotes to think about "Psychometry, it is hardly necessary to say, means the art of "Psychometry, it is hardly necessary to say, means the art of

imposing measurement and number upon operations of the imposing measurement and number upon operations of the mind...".Galton, 1879. mind...".Galton, 1879.

"I often say that when you can measure what you are speaking "I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may thoughts, advanced to the stage of science, whatever the matter may be.“ Sir William Thomson, Lord Kelvin. 1889 be.“ Sir William Thomson, Lord Kelvin. 1889

““Whatever exists at all, exists in some amount. To know it Whatever exists at all, exists in some amount. To know it thoroughly involves knowing its quality as well as its quantity” thoroughly involves knowing its quality as well as its quantity” Thorndike, 1918Thorndike, 1918