*what is test theory? the study of measurement problems, influence of these measurement problems on...

*What is Test Theory?*What is Test Theory?The study of The study of measurement measurement

problemsproblems, , influenceinfluence of these of these measurement problems on measurement problems on psychological inventories, psychological inventories, and how to create and how to create methodsmethods to minimize these problemsto minimize these problems

11

UNIT IUNIT IINTRODUCTION TO INTRODUCTION TO

MEASUREMENT THEORYMEASUREMENT THEORYCHAP 1: WHAT IS TEST THEORYCHAP 1: WHAT IS TEST THEORY

CHAP 2: STATISTICAL CONCEPTS FOR TEST CHAP 2: STATISTICAL CONCEPTS FOR TEST THEORYTHEORY

CHAP 3: INTRODUCTION TO SCALLINGCHAP 3: INTRODUCTION TO SCALLING

CHAP 4: PROCESS OF TEST CONSTRUCTIONCHAP 4: PROCESS OF TEST CONSTRUCTION

CHAPTER 5: TEST SCORES AS COMPOSITESCHAPTER 5: TEST SCORES AS COMPOSITES

22

UNIT II UNIT II RELIABILITYRELIABILITY

CHAP 6: RELIABILITY AND THE CLASSICAL CHAP 6: RELIABILITY AND THE CLASSICAL TRUE SCORE MODELTRUE SCORE MODEL

CHAP 7: PROCEDURES FOR ESTIMATING CHAP 7: PROCEDURES FOR ESTIMATING RELIABILITYRELIABILITY

CHAP 8: INTRODUCTION TO CHAP 8: INTRODUCTION TO GENERALIZABILITY THEORYGENERALIZABILITY THEORY

CHAP 9: RELIABILITY COEFFICIENTS FOR CHAP 9: RELIABILITY COEFFICIENTS FOR CRITERION-REFERENCED TESTSCRITERION-REFERENCED TESTS

33

UNIT III UNIT III VALIDITYVALIDITY

CHAP 10: INTRODUCTION TO CHAP 10: INTRODUCTION TO VALIDITYVALIDITY

CHAP 11: STATISTICAL CHAP 11: STATISTICAL PROCEDURES FOR PREDICTION PROCEDURES FOR PREDICTION AND CLASSIFICATIONAND CLASSIFICATION

CHAP 12: BIAS IN SELECTIONCHAP 12: BIAS IN SELECTION

CHAP 13: FACTOR ANALYSISCHAP 13: FACTOR ANALYSIS44

UNIT IV UNIT IV ITEM ANALYSIS IN TEST ITEM ANALYSIS IN TEST

DEVELOPMENTDEVELOPMENT

CHAP 14: ITEM ANALYSISCHAP 14: ITEM ANALYSIS

CHAP 15: INTRODUCTION TO CHAP 15: INTRODUCTION TO ITEM RESPONSE THEORYITEM RESPONSE THEORY

CHAP 16: DETECTING ITEM CHAP 16: DETECTING ITEM BIASBIAS

55

UNIT V UNIT V TEST SCORING AND TEST SCORING AND INTERPRETATIONINTERPRETATIONCHAP 17: CORRECTING FOR CHAP 17: CORRECTING FOR GUESSING AND OTHER SCORING GUESSING AND OTHER SCORING METHODSMETHODS

CHAP 18: SETTING STANDARDSCHAP 18: SETTING STANDARDS

CHAP 19: NORMS AND STANDARD CHAP 19: NORMS AND STANDARD SCORESSCORES

CHAP 20: EQUATING SCORES CHAP 20: EQUATING SCORES FROM DIFFERENT TESTSFROM DIFFERENT TESTS 66

Introduction to Introduction to Classical and Modern Classical and Modern

Test TheoryTest Theory

Chapter 1Chapter 1

77

Historic OriginsHistoric Origins

Pioneer countries in Pioneer countries in test theory are:test theory are:

Germany, England, Germany, England, France, France, and the and the United United States States

88

GermanyGermany Wilhelm Wilhelm WundtWundt, Ernest , Ernest Weber, Weber, and and

GustavoGustavo Fechner Fechner used procedures used procedures for collection of observations in a for collection of observations in a standard way for all subjects, such standard way for all subjects, such as as reading the reading the instructionsinstructions at the at the top of the test page top of the test page (see next slide).(see next slide).

99

Germany Germany ContCont.... Multiple ChoiceMultiple Choice Identify the choice that best completes Identify the choice that best completes

the statement or answers the question.the statement or answers the question.1.1. The type of sensation you The type of sensation you

experience depends on which area of the experience depends on which area of the brain is activated. This is known asbrain is activated. This is known as

a. a. ssensory localization. ensory localization. b.b.transduction. transduction. cc.sensory adaptation..sensory adaptation.d.d.cerebralization.cerebralization.

2.2. A hypnic jerk usually occurs duringA hypnic jerk usually occurs during a.a.light sleep.light sleep.b.b.deep sleep.deep sleep.cc.episodes of .episodes of

hypersomnia.hypersomnia.dd.episodes of sleep apnea..episodes of sleep apnea. See p.14 Exercise 4-bSee p.14 Exercise 4-b

1010

GermanyGermany p.14 Exercise 4-b p.14 Exercise 4-b

4.4.Consider the following testing Consider the following testing practices and indicate practices and indicate which which nineteenth-century psychological nineteenth-century psychological researcher probably should be credited researcher probably should be credited with the origin?with the origin?

bb. A teacher about to give a test reads . A teacher about to give a test reads aloud from the test manual: aloud from the test manual: “Please “Please read the instructions at the top of the read the instructions at the top of the page silently while I read them page silently while I read them aloud…..” aloud…..” (see previous slide) (see previous slide)

EnglandEngland

Karl Pearson-----Karl Pearson-----Pearson Pearson CorrelationCorrelation

Charles Spearman-----Charles Spearman-----Spearman Correlation. Spearman Correlation.

Used Factor Analysis in his Used Factor Analysis in his “Theory of Intelligence.”“Theory of Intelligence.”

Galton----Galton----CategorizingCategorizinghalf cousin to Darwinhalf cousin to Darwin

FranceFrance Alfred Binet & Theodore Alfred Binet & Theodore

Simon (1905) Simon (1905) Developed Developed

the first IQ test.the first IQ test.

IQ=MA/CAIQ=MA/CAx100x100

MA=Mental AgeMA=Mental Age

CA= Chronological AgeCA= Chronological Age

**The Difference between The Difference between Ratio IQRatio IQ and and Deviation IQDeviation IQ or Normative IQ or Normative IQ

1313

United StatesUnited States

James McKeen James McKeen CattellCattell “Mental Testing” “Mental Testing”

ThorndikeThorndike -- -- An Introduction An Introduction to the to the Theory of Mental and Theory of Mental and Social Social MeasurementMeasurement

Trail and Error Trail and Error A Theory of Learning A Theory of Learning1414

Key TermsKey Terms TestTest Optimal Optimal

PerformancePerformance Typical Typical

PerformancePerformance Observable Observable

PerformancePerformance ConstructsConstructs MeasurementMeasurement

1515

Key TermsKey Terms Test:Test:

Test is a Test is a ProcedureProcedure for obtaining for obtaining aa sample of an individual’s sample of an individual’s performance.performance.

Optimal Performance:Optimal Performance:

Refers to the performance on Refers to the performance on Aptitude Tests Aptitude Tests (GRE,SAT,ACT), or (GRE,SAT,ACT), or Achievement Tests Achievement Tests (WRAT, WIAT)(WRAT, WIAT)

1616

Key TermsKey Terms Typical Performance:Typical Performance: Refers to the performance on Refers to the performance on questionersquestioners

and and inventoriesinventories to report one’s to report one’s feelings, feelings, attitudes, interests, or reactions to a attitudes, interests, or reactions to a situation.situation.

Observable Observable Performance:Performance: Refers to Refers to performperform in an in an observableobservable behavior behavior (watching children interacting with each (watching children interacting with each

others, natural observation).others, natural observation).

1717

Key TermsKey Terms Measurement:Measurement:

QuantifyingQuantifying an an observable observable behaviorbehavior oror when when quantitative quantitative value value is given is given to a behavior.to a behavior.

See Exercise 1 & 2 on P.14See Exercise 1 & 2 on P.141818

Confounding VariablesConfounding Variables Confounding variables are variables Confounding variables are variables

that the researcher that the researcher failed to control, failed to control, or or eliminate, damaging the eliminate, damaging the internal internal validity validity of an experiment. of an experiment. Also, known Also, known as a as a third variable third variable or a or a mediatormediator variable, variable, can adversely affect the can adversely affect the relation between the independent relation between the independent variable and dependent variable. variable and dependent variable.

Ex. NextEx. Next

2121

Ex. Ex. A research group might A research group might design a study to determine if design a study to determine if heavy drinkers heavy drinkers die at a younger die at a younger ageage. Heavy drinkers may be . Heavy drinkers may be more likely to more likely to smokesmoke, , or eat or eat junkjunk foodfood, , all of which could be all of which could be factors in reducing longevity. factors in reducing longevity. A A third variable may have third variable may have adversely influenced the results.adversely influenced the results.

2222

2323

Heavy drinkers Heavy drinkers die at a younger agedie at a younger age

Intervening VariablesIntervening Variables

A variable that A variable that explains a explains a relation relation or provides a or provides a causal causal link link between other variables.between other variables.

Also called Also called “Mediating “Mediating Variable” Variable” or or “intermediary “intermediary variable.”variable.”

Ex. Next slideEx. Next slide

2424

Intervening VariablesIntervening Variables Ex: The statistical association between Ex: The statistical association between

incomeincome and and longevitylongevity needs to be needs to be explained because just having money explained because just having money does not make one live longer. Other does not make one live longer. Other variables variables intervene intervene between between moneymoney and and long lifelong life. . People with high incomes tend People with high incomes tend to have to have better medical care better medical care than those than those with low incomes. with low incomes. Medical care Medical care is an is an intervening variableintervening variable. . It mediates the It mediates the relation between income and longevity.relation between income and longevity.

2525

Key TermsKey Terms Constructs:Constructs:

Constructs are hypothetical Constructs are hypothetical conceptsconcepts or psychological or psychological attributes/traits, such as attributes/traits, such as personality, personality, anxiety, depression anxiety, depression etc.etc.

They are They are difficult to measuredifficult to measure..

Constructs are Constructs are not not physical physical attributes such as height and weight. attributes such as height and weight.

2626

*Why do we have *Why do we have Measurement Problems in Measurement Problems in

Psychology??Psychology?? 1.There is no single universal way of 1.There is no single universal way of definingdefining psychological psychological constructconstruct

2. Psychological measurements are based on 2. Psychological measurements are based on samplessamples of behavior of behavior

3. 3. SamplingSampling of behavior results in of behavior results in errorserrors in in measurementmeasurement

4.The units 4.The units (scales) of measurements (scales) of measurements are not are not well defined.well defined.

5. The measurements must have demonstrated 5. The measurements must have demonstrated relationship to other variables relationship to other variables to have to have meaning.meaning.

2727

Role of Test Theory in Role of Test Theory in ResearchResearch & Evaluation & Evaluation

Selecting a ProblemSelecting a Problem Operational Definitions of VariablesOperational Definitions of Variables Instruments Instruments Accuracy of the InstrumentsAccuracy of the Instruments Data CollectionData Collection Use of StatisticsUse of Statistics Optometrists and OphthalmologistsOptometrists and Ophthalmologists

2828

Chapter 2Chapter 2 Statistical Statistical

Concepts for Concepts for Test TheoryTest Theory

2929

PopulationPopulation

Sample Sample

3030

PopulationPopulation and and SampleSample Population:Population: Population is the set of all Population is the set of all

individuals of interest for a individuals of interest for a particular study. particular study. Measurements related Measurements related to Population are PARAMETERS.to Population are PARAMETERS.

Sample: Sample: Sample is a set of individuals Sample is a set of individuals

selected from a population.selected from a population. Measurements related to sample are STATISTICS.Measurements related to sample are STATISTICS.

3131

StatisticsStatisticsThe people chosen for a The people chosen for a

study are its study are its subjects or subjects or participantsparticipants, collectively , collectively called a called a samplesample

–The sample must be The sample must be representativerepresentative

3232

StatisticsStatisticsDescriptiveDescriptive DescribesDescribes the distribution of scores the distribution of scores

and values such as and values such as mean, median, and mean, median, and modemode

InferentialInferential InferInfer or draw a conclusion from a or draw a conclusion from a

sample.sample.3333

Key TermsKey Terms Constant Constant I.e. I.e. temptemp in in learninglearning and and hungerhunger

VariableVariable IV IV manipulate manipulate DV DV measure measure Discrete Numbers 1, 2 , 3, 14Discrete Numbers 1, 2 , 3, 14 Continues Numbers 1.3, 3.6Continues Numbers 1.3, 3.6

3434

CONTINUOUS VERSUS CONTINUOUS VERSUS DISCRETE VARIABLESDISCRETE VARIABLES

Discrete variables (categorical)Discrete variables (categorical)– Values are defined by category boundariesValues are defined by category boundaries– E.g., genderE.g., gender

Continuous variablesContinuous variables– Values can range along a continuumValues can range along a continuum– E.g., heightE.g., height

3535

StatisticsStatistics Scales of Measurement Scales of Measurement Frequency Distributions and GraphsFrequency Distributions and Graphs Measures of Central TendencyMeasures of Central Tendency Standard Deviations and Variances Standard Deviations and Variances Z ScoreZ Score 1- Pearson1- Pearson CorrelationsCorrelations 2- Spearman2- Spearman

3636

Scales of Measurement Scales of Measurement ((NOIRNOIR))

Nominal ScaleNominal Scale

Qualities Example What You Can Say

What You Can’t Say

Assignment of labelslabels

Gender— (male ormale or femalefemale))Preference—(like or dislike)Voting record—(for or against)

Each observation

belongs belongs in its in its own own categorcategoryy

An observation represents “more” “more” or “less” or “less” than another observation 3737

ORDINAL SCALEORDINAL SCALE



Assignment of values along some underlying dimension (order)(order)

Rank in Rank in collegecollegeOrder of finishing a race

One observation is ranked above above or below or below another.

The amount amount that one that one variable variable is more is more or less or less than another

3838

INTERVAL SCALEINTERVAL SCALE



Equal Equal distances distances between between pointspoints

arbitrary arbitrary zerozero

Number of words spelled correctly onIntelligence test scoresTemperaturTemperaturee

One One score score differs differs from from another another on some measure that has equally appearing intervals

The The amount of amount of difference difference is an exact is an exact representation of differences of the variable being studied

3939

RATIO SCALERATIO SCALE



Meaningful and non-non-arbitrary arbitrary zerozeroAbsolute Absolute zerozero

AgeAgeWeightWeightTime?Time?

One One value is value is twice as twice as much much as another or no quantity of that variable can exist

Not much!

4141

LEVELS OF LEVELS OF MEASUREMENTMEASUREMENT

Variables are measured at one of these four levelsVariables are measured at one of these four levels Qualities of one level are characteristic of the next level upQualities of one level are characteristic of the next level up The more The more preciseprecise (higher) the level of measurement, the (higher) the level of measurement, the

more more accurateaccurate is the measurement process is the measurement process

Level of Level of MeasurementMeasurement For ExampleFor Example Quality of LevelQuality of Level

RatioRatio Rachael is 5Rachael is 5’’ 10 10”” and Gregory is and Gregory is 55’’ 5 5””

Absolute zeroAbsolute zero

IntervalInterval Rachael is 5Rachael is 5”” taller than Gregory taller than Gregory An inch is an inch is an An inch is an inch is an inchinch

OrdinalOrdinal Rachael is taller than GregoryRachael is taller than Gregory Greater thanGreater than

NominalNominal Rachael is tall and Gregory is Rachael is tall and Gregory is shortshort

Different fromDifferent from

4242

WHAT IS ALL THE FUSS?WHAT IS ALL THE FUSS? Measurement should be as Measurement should be as preciseprecise

as possibleas possible In psychology, most variables are In psychology, most variables are

probably measured at the probably measured at the nominalnominal or or ordinalordinal level level

But—But—how a variable is measured how a variable is measured can determine the level of precisioncan determine the level of precision

4343

Frequency Distributions and Frequency Distributions and GraphsGraphs

4444

histogramhistogram

4545

*Histogram for Test Scores *Histogram for Test Scores

4646

PolygonPolygon

4848

Frequency Distributions and Frequency Distributions and GraphsGraphs

4949

PERCENTILESPERCENTILES When the results of a test for a

specific person are presented in terms of Percentiles, we have direct information about that person’s performance relative to a group.

5353

Quartiles and Z-ScoreQuartiles and Z-Score

5454

Platykurtic Platykurtic Mesokurtic,Mesokurtic, LeptokurticLeptokurtic

5959

Frequency DistributionsFrequency Distributions

Frequency Distributions (ƒ)Frequency Distributions (ƒ)

2, 4, 3, 2, 5, 3, 6, 1, 1, 3, 5, 2, 2, 4, 3, 2, 5, 3, 6, 1, 1, 3, 5, 2, 4, 2 4, 2

Σƒ=N=14Σƒ=N=14

Ρ=ƒ/N P=Proportion Ρ=ƒ/N P=Proportion

%=P x 100%=P x 1006060

Frequency DistributionsFrequency Distributions Frequency Distributions (ƒ)Frequency Distributions (ƒ) X f fX X f fX Ρ=Ρ=ƒ/N %=P x 100 Cum%ƒ/N %=P x 100 Cum%

6 1 6 1/14=.07 7%6 1 6 1/14=.07 7%

5 25 2

4 24 2

3 33 3

2 42 4

1 21 26161

Frequency Distribution TableFrequency Distribution Table

X f fX P=f/n %=px100

Cumulative %

6 1 6 1/14=.07 7% 7%

5 2 10 2/14=.14 14% 21%

4 2 8 2/14=.14 14% 35%

How do you Calculate Cumulative Percent ?

• Add each new individual percent to the running tally of the percentages that came before it.

• For example, if your dataset consisted of the four numbers: 100, 200, 150, 50 then their individual values, expressed as a percent of the total (in this case 500), are 20%, 40%, 30% and 10%.

• The cumulative percent would be:1.Proportion 2.percentage

• 100/500=0.2x100: 20%• 200: (i.e. 20% from the step before + 40%)= 60% • 150: (i.e. 60% from the step before + 30%)= 90% • 50: (i.e. 90% from the step before + 10%) = 100%

63

Frequency DistributionsFrequency Distributions

X=2, f=4, N=14X=2, f=4, N=14 Ρ=Ρ=ƒ/Nƒ/N

P=4/14=.29P=4/14=.29 %=P x 100= 29%%=P x 100= 29% X=3, f=3, N=14X=3, f=3, N=14 P=3/14=.21P=3/14=.21 %= 21%%= 21%

μ=ΣƒX/Σƒμ=ΣƒX/Σƒ6464

Measures of Central Tendency Measures of Central Tendency MeanMean----------------IntervalInterval or or Ratio scale Ratio scale

– The sum of the values divided by the number of The sum of the values divided by the number of values--often called the values--often called the "average." "average." μ=ΣX/Nμ=ΣX/N

– Add all of the values together. Divide by the total Add all of the values together. Divide by the total number of values to obtain the mean. number of values to obtain the mean.

– Example: Example: XX 771212242420201919

????????

6565

StatisticsStatistics

The Mean is:The Mean is:

μ=ΣX/N= 82/5=16.4μ=ΣX/N= 82/5=16.4

(7 + 12 + 24 + 20 + 19) / 5 = (7 + 12 + 24 + 20 + 19) / 5 = 16.4.16.4.

6666

MedianMedian Measures of Central TendencyMeasures of Central Tendency Median or Median or MiddleMiddle ------ ------Ordinal ScaleOrdinal Scale

– Divides the values into two equal halves, with Divides the values into two equal halves, with half of the values being lower than the median half of the values being lower than the median and half higher than the median. and half higher than the median. Sort the values into Sort the values into ascending order. ascending order. If you have an If you have an odd number odd number of values, the of values, the

median is the middle value. median is the middle value. If you have an If you have an even number even number of values, the of values, the

median is the arithmetic mean (see above) of median is the arithmetic mean (see above) of the two middle values. the two middle values.

– Ex: The median of the same five numbers (7, 12, Ex: The median of the same five numbers (7, 12, 24, 20, 19) is ???. 24, 20, 19) is ???.

6767

MModeode The median is 19.The median is 19. MMode ode --------Nominal ScaleNominal Scale

– The most The most frequentlyfrequently-occurring value (or -occurring value (or values). values). Calculate the frequencies for all of the Calculate the frequencies for all of the

values in the data. values in the data. The mode is the value (or values) with The mode is the value (or values) with

the highest frequency. the highest frequency. – Example: For individuals having the Example: For individuals having the

following ages -- 18, 18, 19, 20, 20, 20, 21, following ages -- 18, 18, 19, 20, 20, 20, 21, and 23, the mode is ????and 23, the mode is ????

6868

CHARACTERISTICS OF CHARACTERISTICS OF MODEMODE

Nominal Scale Nominal Scale Discrete VariableDiscrete VariableDescribing ShapeDescribing Shape

6969

The RangeThe Range The Mode is 20The Mode is 20 The Range:The Range: The Range is the difference between The Range is the difference between

the highest number –lowest number +1the highest number –lowest number +1 2, 4, 7, 8, and 10 -> 2, 4, 7, 8, and 10 -> Discrete NumbersDiscrete Numbers 2, 4.6, 7.3, 8.4, and 10 -> 2, 4.6, 7.3, 8.4, and 10 -> Continues Continues

NumbersNumbers The difference between the The difference between the upper real upper real

limit limit of the of the highest number highest number and the and the lower real limit lower real limit of the of the lowest number.lowest number.

VariabilityVariability

7171

VariabilityVariabilityRange, Interquartile Range, Semi-Interquartile Range, Interquartile Range, Semi-Interquartile

Range, Range, Standard Deviation, and Variance are the Standard Deviation, and Variance are the Measures of VariabilityMeasures of Variability

Variability is a measure of Variability is a measure of dispersiondispersion or spreading of or spreading of scores around the mean, and scores around the mean, and has 2 purposes:has 2 purposes:

1. 1. Describes the distributionDescribes the distribution Next slideNext slide

7272

VariabilityVariability 2. 2. How well an individual score (or How well an individual score (or

group of scores) represents the group of scores) represents the entire distribution. i.e. in Z Scoreentire distribution. i.e. in Z Score

Ex. In Ex. In inferential statistics inferential statistics we we collect information from a small collect information from a small samplesample then, then, generalizegeneralize the results the results obtained from the sample to the obtained from the sample to the entire entire population.population.

Next slideNext slide 7373

Variability Variability SS,SS, Standard Deviations Standard Deviations and and VariancesVariances

X X σ² = ss/N σ² = ss/N PopPop 1 1 σ = √σ = √ss/Nss/N 22 4 4 s² = ss/n-1 or ss/df s² = ss/n-1 or ss/df Standard deviationStandard deviation

5 5 s = √ss/df s = √ss/df Sample Sample

SS=SS=ΣxΣx²-(Σx)²/N ²-(Σx)²/N Computation Computation

SS=SS=ΣΣ(( x-x-μμ))² ² Definition Definition

Sum Sum of of SquaredSquared DeviationDeviation from from MeanMeanVarianceVariance (σ²)(σ²) is the is the MeanMean of of SquaredSquared Deviations=M Deviations=MSS

7474

Suppose you earned a score of Suppose you earned a score of X = 54 X = 54 on an exam. Which set of on an exam. Which set of

parameters would give you the parameters would give you the highest grade?highest grade?

a.a. μμ= 50 and = 50 and σσ= 2 = 2 σ²σ²=4=4 b.b. μμ= 50 and = 50 and σσ= 4 = 4 σ²σ²=16=16 c.c. μμ= 54 and = 54 and σσ= 2 = 2 σ²σ²=4=4 d.d. μμ= 54 and = 54 and σσ= 4= 4 σ² σ²=16 =16

7575

Suppose you earned a score of Suppose you earned a score of X = 46 X = 46 on an exam. Which set of on an exam. Which set of

parameters would give you the parameters would give you the highest grade?highest grade?

a.a. μμ= 50 and = 50 and σσ= 2 = 2 σ²σ²=4=4 b.b. μμ= 50 and = 50 and σσ= 4 = 4 σ²σ²=16=16 c.c. μμ= 54 and = 54 and σσ= 2 = 2 σ²σ²=4=4 d.d. μμ= 54 and = 54 and σσ= 4= 4 σ² σ²=16 =16

7676

7777

CovarianceCovariance

7878

CovarianceCovariance CCorrelationorrelation is based on a statistic called is based on a statistic called

CovarianceCovariance (Cov (Cov xy xy or Sor S xy xy) ….. ) ….. COVCOVxyxy=SP/N-1=SP/N-1

Correlation--Correlation-- r=sp/√ssx.ssy r=sp/√ssx.ssy CovarianceCovariance is a number that reflects is a number that reflects

the degree to which the degree to which 2 variables 2 variables varyvary together. together.

Original DataOriginal Data X YX Y 8 18 1 1 01 0 3 63 6 0 10 1

Spearman CorrelationSpearman Correlationrank order rank order data then data then

proceed proceed X X Y Y

1 1 11 2 2 3 3 3 3 22

4 4 4 4

7979

@@Ranking/Monotonic Ranking/Monotonic TransformationTransformation ScoreScore Rank position Rank position Final RankFinal Rank 3 3 1 1 1.51.5 33 2 2 1.51.5 55 3 3 33 66 4 4 5 5 66 5 5 55 66 6 6 55 1212 7 7 7 7

8080

Z ScoresZ Scores

Z=x-μ/ σ Z=x-μ/ σ Single scoreSingle score

Z=M-μ/ Z=M-μ/ σσmm Sample Mean for Sample Mean for researchresearch

σσmm= σ/√n = σ/√n we use Z score when we use Z score when σσ is known. is known.

8282

Z-ScoresZ-Scores

8383

X= X= σσ(Z)+(Z)+µµ µµ= X-= X- σσZZ

σσ= (X-= (X-µ)µ)/Z/Z If X=60If X=60

µµ=50=50

σσ=5 Z=?=5 Z=?

Computations/ Calculations / Computations/ Calculations / Collect Data and Compute test Collect Data and Compute test StatisticsStatisticsZ Score for a Sample Z Score for a Sample M=115, n=25M=115, n=25

8484

Z Score for ResearchZ Score for ResearchStandard Error Standard Error ((σσm m ))

8585

StaninesStanines are used to Stanines are used to comparecompare an individual an individual student’s achievement student’s achievement with the results with the results obtained by a obtained by a national reference sample national reference sample chosen chosen to represent a certain year level i.e. to represent a certain year level i.e. 22ndnd level, 3 level, 3rdrd level level

a nine-point scale used for normalized a nine-point scale used for normalized test scores, test scores, with 1-3 below averagewith 1-3 below average, , 4-64-6 averageaverage, and , and 7-9 above average7-9 above average. It is a . It is a nine-point scale of nine-point scale of standard score standard score with with mean of 5 and SD of 2. mean of 5 and SD of 2.

8888

The Correlational MethodThe Correlational Method

Correlational data can be graphed and a Correlational data can be graphed and a “line of best fit” “line of best fit” can be drawncan be drawn

1- Pearson 1- Pearson CorrelationsCorrelations

2-Spearman2-Spearman

8989


CorrelationCorrelation is the degree to which is the degree to which events or characteristics vary from events or characteristics vary from each other.each other.

–Measures the Measures the strengthstrength of a of a relationshiprelationship

–Does not Does not imply imply cause and effectcause and effect

9090


Correlation has 3 Correlation has 3 characteristics:characteristics:

1. The Form of the Relationship1. The Form of the Relationship 2. The Direction of the 2. The Direction of the

RelationshipRelationship 3. The strength or Consistency 3. The strength or Consistency

of the Relationshipof the Relationship9191

1. The Form of the 1. The Form of the RelationshipRelationship

The most common use of The most common use of correlation is to measure correlation is to measure straight-line (linear form) straight-line (linear form) relationship. However, other relationship. However, other forms of relationships do exist forms of relationships do exist and there are special and there are special correlations used to measure correlations used to measure them.them. 9292

2. The Direction of the 2. The Direction of the RelationshipRelationship

Correlational data Correlational data can be can be graphedgraphed and a and a “line of best “line of best fit” fit” can be drawncan be drawn

9393

Positive Correlation Positive Correlation

Positive correlation Positive correlation = variables change = variables change in the in the same same directiondirection

9494

Positive Correlation Positive Correlation

9595

Negative Correlation Negative Correlation

–Negative correlation =Negative correlation = variables change in variables change in the the opposite direction opposite direction

9696

Negative CorrelationNegative Correlation

9797

No CorrelationNo Correlation

–Unrelated =Unrelated = No No consistent consistent relationshiprelationship

9898

No CorrelationNo Correlation

9999


The magnitude (The magnitude (strengthstrength) of a ) of a correlation is also importantcorrelation is also important

–High magnitude =High magnitude = variables which variables which vary closely together; fall close to vary closely together; fall close to the line of best fitthe line of best fit

–Low magnitude =Low magnitude = variables which do variables which do not vary as closely together; loosely not vary as closely together; loosely scattered around the line of best fitscattered around the line of best fit

100100

3. The strength or Consistency of the 3. The strength or Consistency of the RelationshipRelationship

Direction and magnitude of a Direction and magnitude of a correlation are often calculated correlation are often calculated statisticallystatistically–Called the Called the “Correlation Coefficient,” “Correlation Coefficient,”

symbolized by the letter symbolized by the letter “r”“r” Sign (+ or -) indicates directionSign (+ or -) indicates direction Number (from 0.00 to 1.00) indicates magnitudeNumber (from 0.00 to 1.00) indicates magnitude

0.00 = no consistent relationship0.00 = no consistent relationship +1.00 = +1.00 = perfectperfect positive correlation positive correlation -1.00 = -1.00 = perfectperfect negative correlation negative correlation

Most correlations found in Most correlations found in psychological research fall far short of psychological research fall far short of “perfect”“perfect” 101101

The Correlational MethodThe Correlational Method Correlations can be trusted based on Correlations can be trusted based on

statistical probabilitystatistical probability– ““Statistical significance”Statistical significance” means that the means that the

finding is unlikely to have occurred finding is unlikely to have occurred by by chancechanceBy convention/agreement, if there is By convention/agreement, if there is less less

than a 5% probability that than a 5% probability that findingsfindings are are due to chancedue to chance or (or (pp < 0.05), < 0.05), results are results are considered considered “significant,” “significant,” and thought to and thought to reflect the larger populationreflect the larger population

–Generally, Generally, confidenceconfidence increases with increases with the the size of the sample (n) size of the sample (n) and the and the magnitude of the correlation (r)magnitude of the correlation (r) 1010

22

The Correlational MethodThe Correlational Method Advantages of correlational studies:Advantages of correlational studies:

– Have high Have high external validityexternal validityCan Can generalize generalize findingsfindings

– Can repeat Can repeat (replicate) (replicate) studies on other studies on other samplessamples

Difficulties with correlational studies:Difficulties with correlational studies:

– Lack Lack internal validityinternal validityResults describe but Results describe but do not do not explainexplain a a

relationshiprelationship103103

External & Internal ValidityExternal & Internal Validity *External Validity*External Validity

External validity addresses the ability to External validity addresses the ability to generalize generalize your study to your study to other people and other situations.other people and other situations.

*Internal Validity*Internal ValidityInternal validity addresses theInternal validity addresses the "true" "true" causescauses of the of the

outcomes that you observed in your study. Strong outcomes that you observed in your study. Strong internal validity means that you not only have internal validity means that you not only have reliable measures reliable measures of your of your independentindependent and and dependent variables dependent variables BUT a strong justification that BUT a strong justification that causallycausally linkslinks your independent variables your independent variables (IV) (IV) to to your dependent variablesyour dependent variables (DV). (DV).

104104

The Correlational MethodThe Correlational MethodPearsonPearson

r=sp/√ssx.ssyr=sp/√ssx.ssy Original DataOriginal Data X YX Y 1 31 3 2 62 6 4 44 4 5 75 7

SPSP requires 2 sets of data requires 2 sets of dataSSSS requires only one set of data requires only one set of data

105105

The Correlational MethodThe Correlational MethodSpearmanSpearman

r=sp/√ssx.ssyr=sp/√ssx.ssy Original Data Original Data Ranks Ranks X Y X YX Y X Y 1 3 1 11 3 1 1 2 6 2 3 2 6 2 3 4 4 3 24 4 3 2 5 7 4 45 7 4 4

SPSP requires 2 sets of data requires 2 sets of dataSSSS requires only one set of data requires only one set of data

106106

107107

Regression and Regression and PredictionPrediction

Y=bX+aY=bX+aRegression LineRegression Line

108108

109109

Three Levels of Analysis for Three Levels of Analysis for Prediction/ValidityPrediction/Validity

INPUTSINPUTSPROCESSESPROCESSESOUTCOMESOUTCOMES Ex. Ex. StressStress (INPUT)(INPUT) is an unpleasant psychological is an unpleasant psychological

(PROCESS)(PROCESS) that occurs in response to that occurs in response to environmental pressures environmental pressures (job) (job) and can lead to and can lead to withdrawal/quit jobwithdrawal/quit job (OUTCOME).(OUTCOME). 11

1100

111111

prognosisprognosis

Please read Please read chapter 3 and chapter 3 and 4 for the next 4 for the next weekweek

113113

*what is test theory? the study of measurement problems, influence of these measurement problems on...

Documents