Measurement Validity

Validity

Are you measuring what you are supposed to measure?

Fit between “reality” and “measurement” Fit between “concept” and “operational

definition” Accuracy of measurement in a theoretical

sense

Validity is …

Relative to the purpose of testing How confident can we be that measure indicates

a person’s true score on a trait? Matter of degree, rather than whether or not

Have you ever heard students complain that questions on a test didn’t seem to address the material covered in class?– That’s an issue of lack of validity.

Valid and Reliable

A good measurement Measures what it should measure in a

consistent way

Reliable but Invalid

Your measurement is consistent, but not measuring what it is supposed to measure

Types of Validity

Face validity

Content validity

Construct validity

Predictive Concurrent

Convergent Discriminant

Judgmental Empirical

(Criterion-related)

Face Validity

Appears to measure what it was designed to measure.

Most subjective measure of validity Weakest measure of validity Based on judgement

Content validity Does the measurement include all relevant

factors and exclude irrelevant factors? Judge to what extent the content of measure is

representative of facets of topic being measured? Do the questions cover the range of behaviors/

attitudes considered to be part of what you are assessing?

E.g. Knowledge of Local Culture - language, customs, history - include items from all three to have good content validity

Construct validity

Extent to which instrument measures the theoretical construct it is designed to measure.– (e.g., intelligence, depression, self-esteem)

Questionnaires results agree w predictions based on theory

Extent to which instrument measure a characteristic that cannot be directly observed, but must be inferred from patterns in people’s behavior (e.g., motivation, creativity)

Empirical (Criterion-related) Validity

Compare scores with one or more external variables, criteria…

Correlate with related measure.

Criterion-related validity continued

Examine validity of the measure by comparing to external criterion

Predictive - of future ability

Concurrent - ability to measure present standing

Predictive Validity

- Ability of test to predict some future behavior of future ability

– e.g., SAT and future college performance measured by (operationalized) GPA

– Correlate survey’s results with some behavior that would be expected to occur, given the results

E.g, Questionnaire on political attitudes would have predictive validity if results correctly predicted election outcomes.

Concurrent

- ability to measure present standing– e.g. correlate salespeople’s scores on a sales

aptitude test w/ the $ amount of products they sell

Correlate w another measure of same topic at same time.

Convergent Validity

Results similar to / correlate well with those from different measure on same topic.

– (e.g., high or low shyness and behavior in social situations)

– (e.g., Satisfaction with life scale and

5-item Life satisfaction measure)

Discriminant Validity

Differentiates the construct from others that may be similar.

The new measure and measures of different, unrelated constructs correlate poorly.– Reading ability and IQ.

When a measure is not related to other variables that it theoretically should not be related to, this measure is said to have _______ validity.

A. Face B. Reliable C. Convergent D. Discriminant

A psychologist studied dependency by measuring how many times children in school asked an adult for help.

A critic argued that asking for help is not a good measure of dependency and said,

asking for help shows resourcefulness and initiative. The critic in this situation was disputing the ________ validity of the dependency measure.

A. Face B. Construct C. Measurement error D. Reliability

You are constructing a test of knowledge of research methods.

You use your textbook for this class to devise 100 true-false questions on various aspects of research methods.

Without conducting any research, you can be most certain of the ________ validity of this test.

A. Construct C. Criterion B. Face D. Divergent

When a measure relates consistently to other measures of the same concept it shows _______ validity.

A. Convergent B. Divergent C. Face D. Internal