1 research methods techniquesstrengthsproblems. 2 experiments n experiments involve manipulation and...
Post on 20-Dec-2015
213 views
TRANSCRIPT
11
Research MethodsResearch Methods
TechniquesTechniques
StrengthsStrengths
ProblemsProblems
22
ExperimentsExperiments
Experiments involve manipulation and Experiments involve manipulation and control. They use:control. They use:
IVs and DVs.IVs and DVs. Experimental and control groupsExperimental and control groups Pretesting and posttestingPretesting and posttesting They seek to understand causation. The They seek to understand causation. The
IV is thought to lead to, affect, the DVIV is thought to lead to, affect, the DV
33
•Logic of experimentsLogic of experiments
(IV)(IV)
DV Experimental group DVDV Experimental group DV
DV Control group DVDV Control group DV
Ideally, random assignment of subjects to Ideally, random assignment of subjects to experimental and control groupsexperimental and control groups
44
Logic of experiments (continued)Logic of experiments (continued)
If random assignment (randomization) is If random assignment (randomization) is used, the groups are likely to be used, the groups are likely to be equivalent. The P is that the groups will equivalent. The P is that the groups will be the same, differences canceled out.be the same, differences canceled out.
Logic or rationale: If the control and Logic or rationale: If the control and experimental group are the same on the experimental group are the same on the DV at the start of the study, and the only DV at the start of the study, and the only difference is that the experimentaldifference is that the experimental
55
Logic (continued)Logic (continued)
group receives the IV and the control group receives the IV and the control group does not, thengroup does not, then
If there are differences between the If there are differences between the groups on the DV at the end of the groups on the DV at the end of the study, then they must be due to the IV.study, then they must be due to the IV.
66
Equivalent groupsEquivalent groups
Importance of equivalent groupsImportance of equivalent groups Randomization best, but not always Randomization best, but not always
feasiblefeasible Might match on important variables (sex Might match on important variables (sex
race, SES, intelligence, etc..)race, SES, intelligence, etc..) Problem with matching: might not match Problem with matching: might not match
on some important variable, and the on some important variable, and the groups will not be equivalentgroups will not be equivalent
77
Some designsSome designs
Classic experimentClassic experiment One group pre-post testOne group pre-post test Static-group comparisonsStatic-group comparisons
88
Threats to validity (extraneous)Threats to validity (extraneous)
Internal validity: variables which might Internal validity: variables which might affect the DV, aside from the IVaffect the DV, aside from the IV
History: events which occur during the History: events which occur during the course of the studycourse of the study
Maturation: changes that take place Maturation: changes that take place because of the aging processbecause of the aging process
Testing: changes that occur because of Testing: changes that occur because of pretestingpretesting
99
Threats (continued)Threats (continued)
Selection: nonequivalent groupsSelection: nonequivalent groups Regression: problematic when extreme Regression: problematic when extreme
groups are used. People with very low groups are used. People with very low or very high scores tend to score closer or very high scores tend to score closer to the mean on retesting. (holds true for to the mean on retesting. (holds true for other phenomenon, such as inheritance other phenomenon, such as inheritance to traits)to traits)
Mortality: dropping out of a studyMortality: dropping out of a study
1010
External validity (generalization)External validity (generalization)
One study shows that the IV affects the One study shows that the IV affects the DV. Will this be true in other settings DV. Will this be true in other settings with other subjects?with other subjects?
Interaction of testing and IVInteraction of testing and IV Experimenter effects, reactive Experimenter effects, reactive
arrangementsarrangements Hawthorne studyHawthorne study
1111
Advantages of experimentsAdvantages of experiments
We can more directly observe the We can more directly observe the effects of one variable on the other, effects of one variable on the other, holding other variables constantholding other variables constant
Sometimes can be done with little time Sometimes can be done with little time and moneyand money
Experiments can be more easily Experiments can be more easily replicated than other methodsreplicated than other methods
1212
Disadvantages of experimentsDisadvantages of experiments
ArtificialityArtificiality Example: TV and aggressionExample: TV and aggression Example: Zimbardo studyExample: Zimbardo study
1313
SurveysSurveys
Ask people what they think, feel, didAsk people what they think, feel, did Initial steps in surveysInitial steps in surveys Selection of population and sampleSelection of population and sample Development of the questions into a Development of the questions into a
questionnairequestionnaire Pretesting and reviewPretesting and review
1414
Survey methodsSurvey methods
Group administeredGroup administered MailedMailed Face to face interviewsFace to face interviews Telephone interviewsTelephone interviews Each has its advantages and Each has its advantages and
disadvantagesdisadvantages
1515
Group administeredGroup administered
Assemble subjects together, have them Assemble subjects together, have them fill out the questions. A proctor is fill out the questions. A proctor is available to answer questionsavailable to answer questions
Advantages: economical, quick, Advantages: economical, quick, problems can be handled immediatelyproblems can be handled immediately
Problem: it may not be possible to Problem: it may not be possible to assemble subjects. Works best with assemble subjects. Works best with groups who will be together anywaygroups who will be together anyway
1616
Mailed surveysMailed surveys
Mail out: (1) questionnaire; (2) cover Mail out: (1) questionnaire; (2) cover letter explaining the purpose of the letter explaining the purpose of the study, who is doing it, and what the study, who is doing it, and what the results will be used for; and (3) a self-results will be used for; and (3) a self-addressed “stamped” envelope.addressed “stamped” envelope.
bulk ratebulk rate on the return envelope: stamps, on the return envelope: stamps,
metering, business reply permitsmetering, business reply permits
1717
Mailed (continued)Mailed (continued)
business reply permits : more business reply permits : more expensive to purchase, but you only expensive to purchase, but you only pay for those surveys returned.pay for those surveys returned.
Follow-ups--reminder postcards vs. new Follow-ups--reminder postcards vs. new questionnaire, cover letter and envelopequestionnaire, cover letter and envelope
1818
Advantages of mailed surveysAdvantages of mailed surveys
Relatively inexpensiveRelatively inexpensive Fairly fast resultsFairly fast results Geographic accessibility--can reach Geographic accessibility--can reach
almost anyonealmost anyone
1919
DisadvantagesDisadvantages
Low response rates, with the potential Low response rates, with the potential for biased resultsfor biased results
No control over the subjectsNo control over the subjects No control over misinterpretations of the No control over misinterpretations of the
questionsquestions Will subjects be truthful?Will subjects be truthful? More impersonal than other methodsMore impersonal than other methods
2020
Face to face interviewsFace to face interviews
Training of interviewersTraining of interviewers IdentificationIdentification How to dress (depends on subjects)How to dress (depends on subjects) Approaching people (appointment vs. Approaching people (appointment vs.
showing up)showing up) How to administer questions and record How to administer questions and record
responsesresponses
2121
Face to face (continued)Face to face (continued)
Practice sessionsPractice sessions Anticipating as many situations as Anticipating as many situations as
possible and handling problemspossible and handling problems Need to monitor interviewersNeed to monitor interviewers
2222
Advantages of face to faceAdvantages of face to face
High response ratesHigh response rates Control over subjectsControl over subjects Can “probe” and ask more complicated Can “probe” and ask more complicated
questionsquestions Deals better with less educated or more Deals better with less educated or more
illiterate populationsilliterate populations Better for establishing rapport and getting Better for establishing rapport and getting
more informationmore information
2323
Disadvantages of face to faceDisadvantages of face to face
Time-consumingTime-consuming ExpensiveExpensive Interviewer biasInterviewer bias Some people may lie to face to face Some people may lie to face to face
interviewersinterviewers
2424
Telephone interviewsTelephone interviews
Same procedures as face to face, but Same procedures as face to face, but over the telephoneover the telephone
Use of computer systemsUse of computer systems Random digit dialingRandom digit dialing Coding data as questions are answeredCoding data as questions are answered
2525
Advantages of telephoneAdvantages of telephone
FastFast Less expensive than face to faceLess expensive than face to face Can still ask more complicated Can still ask more complicated
questions and probequestions and probe
2626
DisadvantagesDisadvantages
More impersonalMore impersonal Can terminate the interview more easilyCan terminate the interview more easily response rate not as high as face to response rate not as high as face to
face, but higher than mailedface, but higher than mailed Cannot be very longCannot be very long
2727
Observational ResearchObservational Research
Problem with surveys: what people say Problem with surveys: what people say and what they actually do are not and what they actually do are not always the same. always the same.
Additionally, there may be important Additionally, there may be important information that no one thought to askinformation that no one thought to ask
Therefore it is sometimes useful to Therefore it is sometimes useful to observe behavior as it occurs, not ask observe behavior as it occurs, not ask about it or do experimentsabout it or do experiments
2828
Distinctions in observationsDistinctions in observations
Laboratory vs. fieldLaboratory vs. field One subject vs. a groupOne subject vs. a group Unstructured vs. structuredUnstructured vs. structured Unstructured: observe a variety of Unstructured: observe a variety of
aspects of environment and behavioraspects of environment and behavior Structured: interested in a particular Structured: interested in a particular
behaviorbehavior
2929
Distinctions (continued)Distinctions (continued)
Unstructured: Margaret Meade’s Unstructured: Margaret Meade’s studies, prison subculture studiesstudies, prison subculture studies
Structured: police brutality, police Structured: police brutality, police discretion at the time of arrestdiscretion at the time of arrest
Participant vs. non-participant observerParticipant vs. non-participant observer Participant: joins in the activities in the Participant: joins in the activities in the
environment. May be a genuine member environment. May be a genuine member (Becker as a jazz musician)(Becker as a jazz musician)
3030
Distinctions (continued)Distinctions (continued)
Might be pretend to be a member of the Might be pretend to be a member of the group. Problem: is it ethical to group. Problem: is it ethical to deceive? (Rosenhan study, “end of the deceive? (Rosenhan study, “end of the world” study)world” study)
With participant, the observer might With participant, the observer might become too involved to be objective, become too involved to be objective, and might influence the group’s and might influence the group’s behaviorbehavior
3131
Distinctions (continued)Distinctions (continued)
Nonparticipant researcherNonparticipant researcher Might conceal him/herself. Problem: Might conceal him/herself. Problem:
people’s right to privacy (Not a problem people’s right to privacy (Not a problem if observing in a public place, where if observing in a public place, where people do not have an expectation of people do not have an expectation of privacy)privacy)
Might identify self as researcher. Might identify self as researcher.
3232
DistinctionsDistinctions
Problem: Might be rejected. Also, Problem: Might be rejected. Also, people might act differently if they know people might act differently if they know they are being observed.they are being observed.
The pros and cons must be considered The pros and cons must be considered in such observational studiesin such observational studies
3333
What do observers study?What do observers study?
Practices (behaviors)Practices (behaviors) Episodes and encountersEpisodes and encounters RolesRoles RelationshipsRelationships Groups and organizationsGroups and organizations
3434
Steps in observational researchSteps in observational research
LiteratureLiterature InformantsInformants ““Getting in” and rapport, explanation of Getting in” and rapport, explanation of
researchresearch Sampling: often nonprobability--Sampling: often nonprobability--
convenience, quota, or snowball.convenience, quota, or snowball. ObservationsObservations
3535
Steps in observation (continued)Steps in observation (continued)
Unstructured: field notesUnstructured: field notes Structured: record events in categories Structured: record events in categories
as they occuras they occur Unstructured interviews--probe and try Unstructured interviews--probe and try
to determine what is going onto determine what is going on Categorizing observationsCategorizing observations Examples: history of group, biographiesExamples: history of group, biographies
3636
Steps in observation (continued)Steps in observation (continued)
religion, rule-breaking, roles.religion, rule-breaking, roles. Finding patterns, deviations in the Finding patterns, deviations in the
patternspatterns Writing up resultsWriting up results More difficult than other methodsMore difficult than other methods
3737
Advantages of observationAdvantages of observation
Depth of understanding gainedDepth of understanding gained Flexibility of the methodFlexibility of the method Rich source of hypotheses for other Rich source of hypotheses for other
types of studiestypes of studies
3838
Disadvantages of observationDisadvantages of observation
Might misinterpretMight misinterpret Difficulties of interpretationDifficulties of interpretation Small number of subjectsSmall number of subjects Sampling problems--will the Sampling problems--will the
observations be representativeobservations be representative Time-consumingTime-consuming ExpensiveExpensive
3939
Unobtrusive researchUnobtrusive research
Looking at traces left behindLooking at traces left behind Example: looking at well-worn floors in Example: looking at well-worn floors in
museums to determine which exhibits museums to determine which exhibits are most popularare most popular
Our only way of being able to study Our only way of being able to study prehistoric manprehistoric man
Problem: dangers of misinterpretationProblem: dangers of misinterpretation
4040
Record ResearchRecord Research
With the advent of the written word, we With the advent of the written word, we can study behavior. Our only method can study behavior. Our only method for studying the past.for studying the past.
For example, has America always had a For example, has America always had a crime problem, particularly violent crime problem, particularly violent crime, or has this always been part of crime, or has this always been part of our culture?our culture?
Have child rearing practices changed?Have child rearing practices changed?
4141
Content analysisContent analysis
Use of documents to study a behavior Use of documents to study a behavior such as crime: books, magazines, such as crime: books, magazines, songs, newspapers, letters, diariessongs, newspapers, letters, diaries
Example: fear of crime in Switzerland Example: fear of crime in Switzerland studied by examining newspapers and studied by examining newspapers and Parliamentary debates for concern Parliamentary debates for concern about crime (In contrast to the U.S., the about crime (In contrast to the U.S., the topic was not raised often)topic was not raised often)
4242
Content analysis (continued)Content analysis (continued)
Themes put into categoriesThemes put into categories Need to distinguish manifest (actual) Need to distinguish manifest (actual)
and latent (hidden meanings) contentand latent (hidden meanings) content Use of computers to search for words in Use of computers to search for words in
documents (advantages and documents (advantages and disadvantages)disadvantages)
4343
Record (archival) researchRecord (archival) research
Use of existing statistics and recordsUse of existing statistics and records government documentsgovernment documents computer tapes of agenciescomputer tapes of agencies agency filesagency files Examples: Do judges sentence females Examples: Do judges sentence females
differently than males? What percentage differently than males? What percentage of cases are plea bargained? etc..of cases are plea bargained? etc..
4444
Advantages of study of recordsAdvantages of study of records
InexpensiveInexpensive Possible to study processes occurring Possible to study processes occurring
over long periods of timeover long periods of time No effect on those being studied; records No effect on those being studied; records
are being kept anywayare being kept anyway Can be used for historical and Can be used for historical and
comparative usescomparative uses Large numbers of subjectsLarge numbers of subjects
4545
Disadvantages of record Disadvantages of record researchresearch
Records may have gaps, be incompleteRecords may have gaps, be incomplete Ecological fallacy: patterns at a group Ecological fallacy: patterns at a group
level may be different than those at the level may be different than those at the individual. Example: Protestant individual. Example: Protestant countries have higher suicide rates than countries have higher suicide rates than Catholic countries. Are Protestants Catholic countries. Are Protestants committing suicides--could be Catholic committing suicides--could be Catholic minorityminority
4646
Disadvantages--record researchDisadvantages--record research
Problems with statisticsProblems with statistics Official statistics--unreported crimeOfficial statistics--unreported crime Crimes depend on laws, which varyCrimes depend on laws, which vary Numbers depend on discretion of officials Numbers depend on discretion of officials
and their norms, at arrest, plea and their norms, at arrest, plea bargaining and convictions, sentences, bargaining and convictions, sentences, probation/parole revocationprobation/parole revocation
Necessary to look behind the numbersNecessary to look behind the numbers
4747
Evaluation ResearchEvaluation Research
Determining the effects of programs, Determining the effects of programs, agencies, policies and lawsagencies, policies and laws
Four componentsFour components Needs assessmentNeeds assessment MonitoringMonitoring OutcomesOutcomes Cost/benefit and cost/effectivenessCost/benefit and cost/effectiveness
4848
Needs assessmentNeeds assessment
Is the service/change/intervention Is the service/change/intervention needed?needed?
Who is the target population? i.e.., who Who is the target population? i.e.., who do we want to reach the most? What do we want to reach the most? What are their characteristics?are their characteristics?
What problems can be anticipated as What problems can be anticipated as the change is made? i.e.., halfway the change is made? i.e.., halfway houses expect neighborhood resistancehouses expect neighborhood resistance
4949
Needs (continued)Needs (continued)
Can problems be addressed Can problems be addressed successfully?successfully?
Ways of assessing: records, surveys, Ways of assessing: records, surveys, key informants and gatekeeperskey informants and gatekeepers
Without needs assessment, Without needs assessment, inappropriate interventions might be inappropriate interventions might be mademade
5050
MonitoringMonitoring
Looks at the processes taking place as Looks at the processes taking place as the intervention is madethe intervention is made
Are the services being provided? In the Are the services being provided? In the case of a law, is it being utilized?case of a law, is it being utilized?
Is the intervention reaching the target Is the intervention reaching the target population? i.e., if your intervention is population? i.e., if your intervention is school based and your targets are school based and your targets are truants, it will not worktruants, it will not work
5151
Monitoring (continued)Monitoring (continued)
Are there unanticipated side effects?Are there unanticipated side effects? Is the intervention working/happening?Is the intervention working/happening? Assessing: records, interviews with Assessing: records, interviews with
people carrying out the interventions, people carrying out the interventions, interviews with people affected by the interviews with people affected by the interventionintervention
5252
Outcomes researchOutcomes research
What are the goals and outcomes What are the goals and outcomes expected? Usually multiple goalsexpected? Usually multiple goals
Is the intervention achieving those Is the intervention achieving those outcomes?outcomes?
Assessing: experiments or quasi-Assessing: experiments or quasi-experiments (nonequivalent control experiments (nonequivalent control groups, before-after), time series groups, before-after), time series designsdesigns
5353
Cost benefit & cost effectivenessCost benefit & cost effectiveness
Cost-benefit: monetary costs of the Cost-benefit: monetary costs of the intervention are compared to the intervention are compared to the monetary savings as the result of the monetary savings as the result of the programprogram
Cost effectiveness: costs are compared Cost effectiveness: costs are compared to the outcomesto the outcomes
Example: bail programExample: bail program
5454
Cost benefit/cost effectivenessCost benefit/cost effectiveness
Example: Halfway house programExample: Halfway house program Costs must be estimatedCosts must be estimated Effectiveness a more difficult issueEffectiveness a more difficult issue How successful does a program have to How successful does a program have to
be to be “worth it?” a 5% reduction in be to be “worth it?” a 5% reduction in recidivism? 10% ? etc.. compared to recidivism? 10% ? etc.. compared to what costs?what costs?
5555
Strengths of evaluationStrengths of evaluation
Determining what works and what does Determining what works and what does not, rather than just trying intervention not, rather than just trying intervention after interventionafter intervention
Determining what the unanticipated Determining what the unanticipated effects are (ex: net-widening)effects are (ex: net-widening)
Determining why programs do not work, Determining why programs do not work, which can be valuable (example: laws which can be valuable (example: laws and prosecutors)and prosecutors)
5656
Problems in evaluationProblems in evaluation
Problems of uncontrolled experimentsProblems of uncontrolled experiments Measurement problemsMeasurement problems Lack of control over agenciesLack of control over agencies Results ignored, or might be unpopular Results ignored, or might be unpopular
(for example, pornography studies)(for example, pornography studies)