Intangible Measurement

231804-P

231804-P

Social scientists have great need for the development of valid measures,e.g., of the quantity and quality of education services and of the outcomesof those services. Many researchers are frustrated when existinginstruments are not well tailored to the task, since they then cannot expectsensitive, accurate, or valid findings.

This workshop presents the theory and practice of classical test theory,the traditional approach. It then provides an overview of "modern"measurement as practiced using item response theory with a focus onRasch measurement. Rasch analysis provides the social sciences with thekind of measurement that characterizes measurement in the naturalsciences. Since item response theory focuses on the items and the personsrather than the test score, the synthesis of quantitative analysis withqualitative issues is experienced in a way that is rare in social science.Ultimately, Rasch measurement can facilitate more efficient, reliable, andvalid assessment while improving privacy and convenience to users.

The workshop is useful for anyone who wants to understand the role ofmodern measurement in research.

3Measurement..:I N S T R U M E N T A T I O N-The Science of Metrology

4Measurement....: Instrument Usage

Mohd Saidfudin Masodi; IRCA(Lon.) Lead Assessor5

Measurement....: Fatalities

7:01 AM 6

F U N D A M E N T A LPRINCIPLES OF MEASUREMENT

7:01 AM 7

Measurement Overview A Scale Construct Development of Instrument Essentials of Measurement Q & A Session

PRESENTATION STRUCTURE

7:01 AM 8

Measurement Overview-Any measurement call for an instrument.

7:01 AM 9

Instrument Properties:- Construct Validity - Equal interval-Person free - Reliable

7:01 AM 10

Measurement Overview:- Q & A Session: Are you measuring what you want to measure ?

7:01 AM 11

Measurement of Physical Attributes

In physical measurement; a metric systemhas been devised and concept developed; e.g. Length1. Instrument: Ruler, Tape measure, digital2. Unit: inches / feet, meter / km3. Standard definition:

The length of path travelled by light in aninterval of 1/299 792 458 seconds

Can we name others..?

7:01 AM 12

Measurement Units

Two major units:1. Fundamental units

Length : metre mMass : kilogram kgTime : second sTemperature: kelvin oKElectricity: Ampere A

7:01 AM 13

Measurement of Unitscont.:2. Derived units

Area : square metre m2

Volume : cubic meter m3

Density : kg per m3 kg/m3

Velocity : metres per sec m/sAcceleration: metres per sec sq m/s2

Force : Newton kg m/s2

7:01 AM 14

Measurement in Psychology

Recent research has been done in the area of psychometry and in principle has agreed on the approach and criteria of measurement

( Thorndike et al., 1991)

7:01 AM 15

Psychometry Measurement Overview:- Q & A Session: So, how do we measure ability or beauty ?

7:01 AM 16

HOW DO WE CONSTRUCT THE INSTRUMENT IN

SOCIAL SCIENCE STUDY ?

Measurement of Psychological Attributes

A PERFORMER

BEAUTIFUL GLORIOUS

7:01 AM 21

Currently, there are five (5) fundamental issues .

Measurement Construct

1. Sample / Item dependency2. Use of Raw scores or Observed

counts to develop interval measure

3. Response Validity

4. Item Validity

5. Construct Validity

7:01 AM 22

Measurement Overview:- Q & A Session: What is a measurement construct ? Discuss.

7:01 AM 23

1. Sample/Item Dependency

Measurement of Psychological Construct

Have you ever come across a Medical Doctor keeping 10 thermometers in his clinic; one for different gender and size ? Discuss.

a. Items; as agent of measurement, shall be independent of the person; the sample object used in the calibration.

b. Object measures shall be independent of the particular agents used to obtain them.

7:01 AM 24

1a. Specific Objectivity

Measurement of Psychological Construct

Test Free / Sample Free Measures

e.g. Speed test for typist. Result independent of number of candidates and stand on its own merit unless you dont know how to use the instrument. Discuss.

a. Specifivity and objectivity is the cornerstone of all measurement

7:01 AM 25

1b. Sample DependentMeasurement of Psychological Construct

a. More Correct Responses; therefore an Easy test .

High Achievers ?

Low Achievers ?b. Less Correct Responses;

therefore a Hard test .

How does an item dependent on the sample objects; person

?

?

7:01 AM 26

2. Interval MeasureMeasurement of Psychological Construct

Linearity is a requirement for an item to be measurable which can be developed by transforming raw scores into interval scale.

a. Raw scores are observed counts and are not measures

b. Percentages are only numerical data summary but does not give any measure

7:01 AM 27

2b. Interval MeasureMeasurement of Psychological Construct

Limitations of Raw Scorea. Its a concrete set of nominal answers to an item.b. Its a count of person responses pointing to knowledge

& might be taken as ordinal correct answers.

d. Raw score is a description of a bygone event. It does not imply a person ability to take on a given task; item difficulty.

e. Neither set of raw score nor % correct are linear.

c. The quantitative meaning of an ordinal raw score, however, varies with test length; whilst percentage correct varies with item difficulty.

7:01 AM 28

2a. Equal Interval

7:01 AM 29

2c. Linearity of MeasureMeasurement of Psychological Construct

In order to compare persons ability for a given cohort, the measure must be interval data on a linear

scale to enable due statistical analysis done.

6

5

4

3

2

1

Perception assessment=> Ordinal; mathematically= axn

1 4 5 62 3

Poorest

Excellent

Good

V good

Mediocre

Poor

Linearised, transformed by ratio.= log (axn)= log a + n logxLinear regressiony = mx + c,n = m; gradientthus, ratio scale

Numerical series log10 loge1 0.000 0.0002 0.301 0.6945 0.699 1.60910 1.000 2.30320 1.302 2.99750 1.699 3.912

100 2.000 4.6067:01 AM 30

2d. Linearity of MeasureMeasurement of Psychological Construct

Numerical series are only orders with unequal separation which expand exponentially but log maintain equal interval. Proof:

7:01 AM 31

Construct ValidityMeasurement of Psychological Construct

What is the underlying theory ? 1. Sampling Frame: Sampling Unit

2. Item representativeness: Accuracy

3. Focus: Content Validity

4. Item-dependent: Construct Validity

EASY ITEMS DIFFICULT ITEMS11111011111111111 11111111111111111 11111111111100 = 4811111111111111111 11111111111111111 11111001001000 = 4311111111111111111 11111111111100100 01100010000000 = 3311111111111111111 11111111111010100 00110100000000 = 3311010111101111011 10101111101001101 01110110101101 = 3311111111111111111 11110111001000100 01000000000000 = 2711111111111111101 11011011001000100 00000000001000 = 25

7:01 AM 32

Response Validity: Observe the pattern

Measurement of Psychological Construct

SMART

POOR?

ZONE 0ZONE 1

EASY ITEMS DIFFICULT ITEMS

11111011111111111 11111111111111111 111111111111100 = 4811111111111111111 11111111111111111 11011001001000 = 4211111111111111111 11111111111100101 01000010000000 = 3311111111111111111 11111111111010100 10110100000000 = 3311110111111111111 01111111101001100 00110100101100 = 3311111111111111111 10110111011000100 01000000000100 = 2711111111111111101 11011011001000100 00000000000100 = 25

7:01 AM 33

Item Validity : Measurement Direction

Measurement of Psychological Construct

7 6 4 3 0

BI-DIRECTIONAL ITEM

7:01 AM 34

A valid measurement in a psychological construct MUST meet five (5) criteria:

CONCLUSION

1. Linear Scale: AMOUNT2. Reasonable numerical values: ACCURACY3. Empirical Coherence:

Response, Item and Construct VALIDITY4. Incorporate

parameter separation: REPLICABLE

5. Overcome missing data: PREDICTIVE(Wright & Mok, 2004)

7:01 AM 35

Measurement Overview:- Q & A Session: Are you measuring

what you really want to measure ?

Introduction to Rasch Measurement Model: Instrument Construct ValidityIntroduction to Rasch Measurement ModelSlide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16PERFECTIONBEAUTY PAGEANTRATING SCALE: UNSTABLE INTERVALRATING SCALE: UNSTABLE INTERVALSlide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35