Reliability and ValidityReliability and Validity

Research MethodsResearch Methods

RECAP! OBSERVATIONAL RECAP! OBSERVATIONAL METHODSMETHODS

Create 3 behavioural categories for the Create 3 behavioural categories for the following:following:

- Observation of students when teacher Observation of students when teacher leaves roomleaves room

- Observation of dogs on walk in the parkObservation of dogs on walk in the park- Observation of giraffes in the zooObservation of giraffes in the zoo

Learning ObjectivesLearning Objectives

To define reliabilityTo define reliability To define internal and external validityTo define internal and external validity To describe factors which can affect To describe factors which can affect

validityvalidity

ReliabilityReliability

Linked to… Linked to… replicabilityreplicability – if the experiment was – if the experiment was

repeated, would the same results be repeated, would the same results be found?found?

Improving ReliabilityImproving Reliability

Standardised proceduresStandardised procedures – all researchers – all researchers carry out process the samecarry out process the same

Inter-rater reliabilityInter-rater reliability – more than one – more than one researcher to observe behaviour – are they researcher to observe behaviour – are they interpreting it the same?interpreting it the same?

Training needed for all researchers Training needed for all researchers

ValidityValidity

AccuracyAccuracy of an of an experimentexperiment

Does it measure what it Does it measure what it claims to?claims to?

Are the conclusions Are the conclusions drawn justified?drawn justified?

Internal ValidityInternal Validity

Ability of study to test the hypothesisAbility of study to test the hypothesis

How sure are we that it is the IV causing How sure are we that it is the IV causing the effects on the DV? (not extraneous the effects on the DV? (not extraneous variables)variables)

External ValidityExternal Validity

How well can results be generalised How well can results be generalised beyond the study?beyond the study?

Population validity Population validity – sample - can results – sample - can results be generalised to other groups of people?be generalised to other groups of people?

Ecological validity Ecological validity – setting - can results – setting - can results be generalised to other situations?be generalised to other situations?

Your TaskYour Task

Complete the worksheet to determine whether Complete the worksheet to determine whether the research is valid and why/why notthe research is valid and why/why not

Dave – low population validity as only clientele from one shop being viewed – unlikely to be all ages and class/background. Also low ecological validity as it was only one shop that was observed so can’t be applied to “shopping” behaviour in general.

Jeannette – low internal validity – repeated measures design so order effects may me causing results rather than the affect of music on memory

Peter – low internal validity – network login does not mean they are working!!! Could be on games, chatting etc…

Factors which reduce internal Factors which reduce internal validity:validity:-Extraneous variablesExtraneous variables-Demand characteristicsDemand characteristics-Investigator effectsInvestigator effects-Social desirability biasSocial desirability bias

Extraneous variablesExtraneous variables

Anything other than the IV which affects Anything other than the IV which affects the DVthe DV

E.g. participants moodE.g. participants mood

Demand CharacteristicsDemand Characteristics

Participants try to behave in the way they think Participants try to behave in the way they think is demanded (or against what they think is is demanded (or against what they think is demanded)demanded)

Try to guess purpose of research – act in a Try to guess purpose of research – act in a helpful wayhelpful way

Act nervously/out of character as think they are Act nervously/out of character as think they are being judgedbeing judged

Investigator effectsInvestigator effects

The researcher’s behaviour or characteristics The researcher’s behaviour or characteristics influence the research in some wayinfluence the research in some way

Researcher bias – interpret behaviour in a way to fit their Researcher bias – interpret behaviour in a way to fit their own views – pay more attention to data that confirms own views – pay more attention to data that confirms their expectationstheir expectations

Presence of observed – change behaviour of p’sPresence of observed – change behaviour of p’s

Characteristics of investigator – age, gender, Characteristics of investigator – age, gender, appearance, facial expressions etc.appearance, facial expressions etc.

Social desirability biasSocial desirability bias

Participants want others to see them in a Participants want others to see them in a favourable way so respond or behave favourable way so respond or behave differently than they normally woulddifferently than they normally would

Might not give honest answersMight not give honest answers

Your TaskYour Task

Decide whether the statements are Decide whether the statements are examples of demand characteristics, examples of demand characteristics, social desirability bias or investigator social desirability bias or investigator effects.effects.

Learning CheckLearning Check

Define reliabilityDefine reliability State one way in which reliability can be State one way in which reliability can be

improvedimproved Describe the difference between internal and Describe the difference between internal and

external validityexternal validity Describe one factor which can reduce Describe one factor which can reduce

internal validity. Provide an example of this.internal validity. Provide an example of this.