independent vs dependent variables presumed cause referred to as independent variable (smoking)....

Independent vs Dependent Variables

• PRESUMED CAUSE REFERRED TO AS INDEPENDENT VARIABLE (SMOKING).

• PRESUMED EFFECT IS DEPENDENT VARIABLE (LUNG CANCER). SEEK TO EXPLAIN DEPENDENT VARIABLE.

• DIRECTIONALITY IS STATED OR IMPLIED.

• SMOKING LUNG CANCER.

Internal Validity

• IS THERE A CAUSAL RELATIONSHIP BETWEEN INDEPENDENT VARIABLE & DEPENDENT VARIABLE (COOK & CAMPBELL, 1979, P.38).

• TYPE I ERROR: REJECTING NULL WHEN IT IS TRUE (FALSE POSITIVE). CONCLUDE TREATMENT MORE EFFECTIVE THAN CONTROL WHEN REALLY SAMPLING FLUCTUATIONS ARE RESPONSIBLE.

Internal Validity

• FINDINGS SHOWN TO RESULT ONLY FROM EFFECT OF INDEPENDENT VARIABLE & CAN’T BE INTERPRETED AS REFLECTING EFFECTS OF EXTRANEOUS VARIABLES.

• SUBJECTS’ PERFORMANCE ON DEPENDENT VARIABLE CAN ONLY RESULT FROM EFFECT OF INDEPENDENT VARIABLE.

Internal Validity

• IS INDEPENDENT VARIABLE REALLY RESPONSIBLE FOR DIFFERENCES DETECTED?

• OR ARE THERE COMPETING HYPOTHESES THAT WERE NOT ADDRESSED OR MEASURED.

• THREATS TO INTERNAL VALIDITY SHOULD BE ADDRESSED PRIOR TO THE STUDY.

Threats to Internal Validity

• HISTORY

• MATURATION

• TESTING

• INSTRUMENTATION

• STATISTICAL REGRESSION

• SELECTION

• MORTALITY


• 1. HISTORY:

IN ADDITION TO INDEPENDENT VARIABLE, ANOTHER EVENT MAY HAVE EFFECTED DEPENDENT VARIABLE BETWEEN 1ST & 2ND MEASUREMENT. ADVERTISEMENT. MEDIA COVERAGE.


• 2. MATURATION:

• REFLECTS PASSAGE OF TIME NOT EXPERIMENT.

• EFFECTS OF AGE ON DEPENDENT VARIABLE. AS YOU GET OLDER YOU BECOME WISER. STRENGTH ON ATHLETICISM. PALLIATIVE CARE IN 1ST YEAR OF PROGRAM.


• 3. TESTING:

• FACT THAT YOU ALREADY TOOK TEST MAY LEAD TO SCORE WHEN YOU TAKE TEST AGAIN.

• TEST MAY INFLUENCE OUTCOME, NOT INDEPENDENT VARIABLE.

• EXPERIENCE GAINED THROUGH REPEAT TESTING.


• 4. INSTRUMENTATION:

• CHANGES IN MEASUREMENT OF VARIABLES MAY ACCOUNT FOR DIFFERENCES FOUND.

• CALIBRATION OF MEASUREMENT (DIFFERENT TYPES OF EQUIPMENT, DIFFERENT WAYS OF DIAGNOSING, DIFFERENCES IN RATERS.)


• 5. STATISTICAL REGRESSION

• (REGRESSION TO THE MEAN)

• GROUPS SELECTED FOR EXTREME SCORES WILL, ON REPEAT TESTING WITH SAME OR SIMILAR TEST, HAVE SCORES CLOSER TO MEAN SCORE (BECAUSE THE MORE DEVIANT THE SCORE, THE LARGER MEASUREMENT ERROR).


• 6. SELECTION BIAS:

• BIAS IN SELECTING SUBJECTS IS RESPONSIBLE FOR RESULTS, NOT INDEPENDENT VARIABLE. SMOKING CESSATION CLINIC. PSYCHOLOGY STUDENTS.


• 7. MORTALITY:

• LOSS OF SUBJECTS FOR ANY REASON. THOSE WHO ARE LEFT ARE DIFFERENT THAN THOSE WHO REMAIN & THESE DIFFERENCES ARE RESPONSIBLE FOR THE OUTCOME, NOT THE INDEPENDENT VARIABLE.


• MUST BE ADDRESSED IN THE DESIGN PHASE PRIOR TO IMPLEMENTING THE PROJECT.

External Validity

• GENERALIZABILITY OF FINDINGS TO OTHER SETTINGS OR SAMPLES

• WILL THIS WORK IN ANOTHER SETTING WITH DIFFERENT SUBJECTS?

Threats to External Validity

• 1. SAMPLING

• 2. HAWTHORNE EFFECT

• 3. NOVELTY EFFECTS

• 4. INTERACTION BETWEEN HISTORY & TREATMENT

• 5. EXPERIMENTER EFFECTS


• 1. SAMPLING• LACK OF RANDOM SELECTION

PROCEDURES MAKES IT NOT POSSIBLE TO GENERALIZE TO OTHER POPULATIONS.

• PROBABILITY SAMPLING. CONVENIENCE SAMPLE OF UNIVERSITY STUDENTS OUTSIDE LIBRARY.


• 2. HAWTHORN EFFECT

• EFFECT OF BEING OBSERVED CAUSED CHANGES IN BEHAVIOR NOT INTERVENTION.

• KNOWLEDGE OF BEING IN STUDY CAUSES PEOPLE (BOTH SUBJECTS AND RESEARCHERS) TO CHANGE THEIR BEHAVIOUR.


• 3. NOVELTY EFFECTS

• WHEN TREATMENT IS NEW BOTH SUBJECTS & RESEARCHERS MAY ALTER BEHAVIOUR; ENTHUSIASM, HOPE.

• OVER TIME, THESE EFFECTS DISAPPEAR


• 4. INTERACTION BETWEEN HISTORY & TREATMENT

• SOME EXTERNAL EVENTS MAY INTERACT WITH TREATMENT.

• DIETARY REDUCTION IN CHOLESTEROL AT SAME TIME AS OAT BRAN & CHOLESTEROL.


• 5. EXPERIMENTER EFFECTS

• PERFORMANCE OF SUBJECTS IS INFLUENCED BY CHARACTERISTICS OF RESEARCHER. RESULTS MAY NOT BE SAME WITH OTHER RESEACHERS.


• MUST BE ADDRESSED IN THE DESIGN PHASE PRIOR TO IMPLEMENTING THE PROJECT.

INTERNAL VERSUS EXTERNAL VALIDITY?

• TRY TO ADDRESS BOTH.

• IF THERE IS CONFLICT, OPT FOR STRONEGR INTERNAL VALIDITY AND HANDLE EXTERNAL VALIDITY WITH REPLICATIONS.

• IF FINDINGS NOT INTERNALLY VALID, CAN’T POSSIBLE BE EXTERNALLY VALID.

RELIABILITY

• RELIABILITY OF MEASUREMENT INSTRUMENT IS THE DEGREE OF CONSISTENCY WITH WHICH IT MEASURES WHAT IT PURPORTS TO BE MEASURING.

• WEIGHT ON SCALE.

RELIABILITY

•ARE {EXAM} SCORES OBTAINED FOR YOUR SAMPLE TRUE?

• DO THEY REFLECT THE TRUE DIFFERENCES IN SUBJECTS?

RELIABILITY

•ERROR VARIANCE (MEASUREMENT ERROR) IS THE EXTENT OF VARIABILITY IN TEST SCORES DUE TO ERROR RATHER THAN TRUE MEASUREMENT.

RELIABILITY

NON-SAMPLING MEASUREMENT ERROR.

TOTAL ERRORSAMPLING

ERROR

RELIABILITY

• OBSERVED SCORE = TRUE SCORE + MEASUREMENT ERROR.

• MEASUREMENT ERROR CAN BE DUE TO CHANCE OR RANDOM ERROR =

RELIABILITY.

• CAN ALSO BE SYSTEMATIC ERROR = VALIDITY.

RELIABILITY

• RELIABILITY: EXTENT TO WHICH INSTRUMENTS YIELDS THE SAME

RESULTS ON REPEATED MEASURES.

• CONCERNED WITH CONSISTENCY, ACCURACY, PRECISION, STABILITY, EQUIVALENCE AND HOMOGENEITY

RELIABILITY

• STABILITY:

• ABILITY TO PRODUCE THE SAME RESULTS WITH REPEATED TESTING.

• TEST-RE-TEST RELIABILITY

RELIABILITY

• HOMOGENEITY; INTERNAL CONSISTENCY

• ALL ITEMS IN THE TOOL MEASURE THE SAME CONCEPT. IQ.

• SPLIT-HALF RELIABILITY, KUDER-RICHARDSON, CRONBACH’S ALPHA

RELIABILITY

• EQUIVALENCE:

• TOOL PRODUCES THE SAME RESULTS WHEN PARALLEL TESTS ARE USED. DIFFERENT IQ TESTS.

• DIFFERENT OBSERVERS, RATERS USING INSTRUMENT TO MEASURE SAME PHENOMENA AT SAME TIME. INTER-RATER RELIABILITY

RELIABILITY

• EQUIVALENCE:

• PERCENT AGREEMENT

• RELIABILITY COEFFICIENT

• KAPPA, WEIGHED KAPPA

independent vs dependent variables presumed cause referred to as independent variable (smoking)....

Documents