Types of Research and DesignsThis week and next weekCoveringResearch classificationsVariablesSteps in Experimental ResearchValidityResearch DesignsCommon Sources of Error

VALIDITY

VALIDITYIn Step 4, the validity of measurement for the DVs comes into questionThere are two (2) important classification of validity in experimental research:Internal validityExternal validity

Internal ValidityHow valid the findings are within, or internal to, the study.Did the treatments cause the outcome to occur, or did other extraneous factors cause the outcome?

Internal ValidityEight (8) factors that threat the internal validity of a study:HistoryMaturationTestingInstrumentationStatistical regressionSelectionExperimental mortalityInteraction of Selection and Maturation or History

HistorySpecific things that happen while conducting the research study that affect the final scores of the participants in addition to the effect of the experimental treatment.The effects of linear vs. nonlinear periodization models for in-season volleyball playersResistance training vs. aerobic training for physical fitness in middle-aged men.

MaturationSubjects grow older during the course of an experiment this may affect the outcome of the studyThe time-course for training-induced increases in strength for adolescents during a 25-week resistance training programThe effects of 12 weeks of aerobic training on VO2max (from September to December)

TestingThe learning effectSubjects get better at scoring well on a test the more times they take it.The effects of 2 days of resistance training on muscle strength and power output.

InstrumentationChanges in the adjustment or calibration of the measuring equipment or use of different standards among scorers may cause a difference in outcome scores.

Statistical RegressionThe tendency for groups with extremely high or low scores on one measure to score closer to the mean score of the population on a second measure.Can be mistaken for a treatment effectExample: Low-IQ and high-IQ groups are formed. A treatment is applied to both groups for twelve weeks. A physical ability test is administered to both groups. It is found that both groups are equal in physical ability.

SelectionThe selection of subjects and their assignments to groups can bias the outcome of a study.Subjects that are not representative of the populationKinesiology majors vs. all college-aged males and femalesGroups that are not equal in their abilities prior to the experimentVO2max and muscle strength

Experimental MortalitySubject attritionExcessive loss of subjects throughout the course of a studySample of subjects is no longer representative of the population

Interaction of Selection and Maturation or HistoryThe maturation effect or history effect is not the same for all groups selected for the research study may affect the outcome.Groups that differ in age or background may respond differently to the same treatmentIf a group starts out unequal in ability due to maturation or history, they may not all have the same potential for improvement or change.Disproportional experimental mortality

External ValidityThe degree to which findings in a research study can be inferred or generalized to other populations, settings, or experimental treatments.Can the findings of a particular study be inferred to the entire population?Are the findings unique to the participants of the study, or do they apply to other groups?Examples: A study using prison inmates. Good internal validity.

External ValidityFour threats to external validityInteraction effect of testingInteraction effects of selection bias and experimental treatmentReactive effects of experimental settingMultiple-treatment interference

Interaction Effect of TestingThis effect occurs when the pre-test itself changes the groups response to the treatment.Example: A pre-test may identify deficiencies that the subject may try to correct during the course of the treatement

Interaction Effects of Selection Bias and Experimental TreatmentThe participant or groups selected in a biased manner respond to the experimental treatment in a unique way so they are not representative of any population.Novice vs. intermediate vs. elite weight liftersThis threat generally occurs as a result of a convenient group is tested and an attempt is made at generalizing the results

Reactive Effects of Experimental SettingsThe experimental setting is not the same as a subjects ambient environment this may affect the study outcome.Subjects may respond differently around investigatorsLaboratory vs. natural settingsSubjects may think that a particular treatment is supposed to do something and respond like they are supposed to.

Multiple-Treatment InterferenceMultiple treatments administered at the same time on the same group of subjects.Subject involved in two studies at the same timeCompeting influences of treatments

Control of VariablesControl of all variables operating in an experimental study is highly desirable, but seldom (if ever) accomplished.Sleep, food, exercise, etc.Too much control can harm external validityNot enough control can harm internal validity

Validity in SummaryIn many cases, internal and external validity cannot be obtained.Must decide which is most importantGood internal validity is necessary for external validityInternal validity is the basic minimum (LCD) for an experimental designInvestigators must consider the threats to validity when designing research studies.