scaling. survey research questionnaires and interviews both experimental and nonexperimental...

Scaling

Survey Research

• Questionnaires and Interviews• Both experimental and nonexperimental

research• Read pages 212 through 223 in Martin• Scaling = construction of instruments for

measuring abstract psychological constructs.

Thurstone Scales:Method of Equal-Appearing Intervals

• Define the Concept• Generate Potential Scale Items

– about 100 statements– differ with respect to the extent to which

agreement indicates presence of the attribute to be measured

Thurstone

• Rating the Potential Scale Items– Judges rate the items on 11-point scale– 1 = agreement indicates very low amount of

the attribute– 11 = agreeing indicates very high amount of

the attribute– encourage judges to use entire range of

scale, assigning some statements to each of the 11 values – sort them into 11 piles.

Thurstone

• Computing the Scale Score Values for Each Item– Find median and SD or inter-quartile range– Arrange in table– Sort by median – Within items with same median, sort by SD or

inter-quartile range

Thurstone

• Select the Final Scale Items– 1 (or 2 or 3) item(s) for each possible scale

score value– Prefer items with low variability among judges– End up with 10-30 items– See this example– Items and scale score values are shown on

the next slide.

• People with AIDS deserve what they got. (1)

• AIDS is good because it helps control the population. (2)

• AIDS will never happen to me. (3)• I can't get AIDS if I'm in a monogamous

relationship. (4) • It's easy to get AIDS (5) • Because AIDS is preventable, we should

focus our resources on prevention instead of curing (5)

• People with AIDS are like my parents (6) • If you have AIDS, you can still lead a

normal life (8) • AIDS doesn't have a preference, anyone

can get it (9)• AIDS is a disease that anyone can get if

they are not careful (9) • Aids affects us all (10) • People with AIDS should be treated just

like everybody else. (11)

Thurstone

• Administer the Final Scale– Randomize the order of the items– For each item, respondent chooses Agree or

Disagree– For each item the scale score is the median

from the judges’ ratings– Total Score = mean scale score for items on

which the respondent agreed.

Thurstone

• Thurstone scales are rarely used these days

• They are just too much trouble to create.• Likert scales were developed in response

to this difficulty

Guttman Scaling

• Define the Concept• Generate Potential Scale Items• Evaluating the Potential Items

– For each item, judges are asked if someone high in the attribute would agree with the statement – Yes or No.

Guttman

• Conduct a Scalogram Analysis of Judges’ Responses– use special software to do this– if successful, it will create an ordered list of

items such that• agreeing with the first item indicates you have at

least a little of the measured attribute• agreeing with the second indicates you have at

least a little more of the attribute• etc.

Guttman

– The scalogram analysis also computes a scale score value for each statement.

– See the example in Trochim’s Internet document Guttman Scaling (reproduced on the next slide).

– It is assumed that anybody who would agree with the nth item would also agree with all preceding items.

– The order of the items may be scrambled prior to administering the scale.

• I believe that this country should allow in more immigrants.

• I would be comfortable with new immigrants moving into my community.

• It would be fine with me if new immigrants moved onto my block.

• I would be comfortable if a new immigrant moved next door to me.

• I would be comfortable if my child dated a new immigrant.

• I would permit a child of mine to marry an immigrant.

Guttman

• Administer the Final Scale– Respondents are asked to check items with

which they agree– Respondent’s score = sum of the scale score

values for checked responses.• Like Thurstone scales, Guttman scales are

not often used these days.

Likert Scales

• Define the Concept• Generate Potential Items

– About 100 statements.– On some, agreement indicates being high on

the measured attribute– On others, agreement indicates being low on

the measured attribute

Likert

– Instead of a dichotomous response scale (agree or disagree), use a multi-point response scale like this:

Likert

• Evaluating the Potential Items– Get judges to evaluate each item on a 5-point

scale• 1 -- Agreement = very low on attribute• 2 – Agreement = low on attribute• 3 – Agreement tells you nothing• 4 – Agreement = high on attribute• 5 – Agreement = very high on attribute

– Select items with very high or very low means and little variability among the judges.

Likert

• Alternate Method of Item Evaluation– Ask some judges to respond to the items in

the way they think someone high in the attribute would respond.

– Ask other judges to respond as would one low in the attribute.

– Prefer items that best discriminate between these two groups

– Also ask judges to identify items that are unclear or confusing.

Likert

• Pilot Test the Items– Administer to a sample of persons from the

population of interest– Conduct an item analysis– Prefer items which have high item-total

correlations– Consider conducting a factor analysis

Likert

• Administer the Final Scale– on each item, response which indicates least

amount of the attribute scored as 1– next least amount response scored as 2– and so on– respondent’s total score = sum of item scores

or mean of item scores– dealing with nonresponses on some items– reflecting items (reverse scoring)

Psychometric Analysis

• Whenever you use such an instrument, you should conduct basic psychometrics

• See “Cronbach’s Alpha and Maximized Lambda4”

• Factor analysis can help determine if the instrument is unidimensional or not.

• Factor analysis of the Cultural Values Survey

• Factor analysis of Patel’s SBS.

PCA of Cultural Values Survey

• 45 items• Reduced to seven orthogonal components• Weighted linear combinations of variables• Used as outcome variables in Culture x

Sex x Age Group factorial ANOVA• Loadings used to decide what the

components are

7 Components

• Family Solidarity (respect for the family)• Executive Male (men make decisions,

women are homemakers)• Conscience (important for family to

conform to social and moral standards)• Equality of the Sexes (minimizing sexual

stereotyping)

• Temporal Farsightedness (interest in the future and the past)

• Independence (desire for material possessions and freedom)

• Spousal Employment (each spouse should make decisions about his/her own job)

ANOVA Results

• US Students (especially the women)– Sexually egalitarian– Desire independence– Family not important to younger students

• Taiwanese Students– Temporally farsighted– Men more sexually egalitarian than women– Women more interested in independence

• Mexican Students– Like the Taiwanese, family is important– Like the US students , not temporally

farsighted– Men thought independence more important

than did women

Patel’s SBS

• 21 items, measures aggression toward homosexual persons

• Cronbach’s alpha = .91• FA revealed three factors• Avoidance Behaviors (13 items)

– Moving away from gay– Staring to communicate disapproval of

proximity

• Aggression from a Distance (6 items)– Writing anti-gay graffiti– Damaging gays’ property– Making harassing phone calls

• Up-Close Aggression– Physical fighting