class review week # 3
DESCRIPTION
TRANSCRIPT
Research Methods for Counselors
COUN 597
Saint Joseph College
Class # 3
Copyright © 2007 by R. Halstead. All rights reserved.
Class Objectives
Basic Overview of MeasurementTrochim Chapter 3
Overview of Survey Research Trochim Chapter 4
Review of Correlation CoefficientsSalkind Chapter 5
Some Practice with Concepts
Topics Appropriate for Survey Research
There are a variety of uses for surveysIndividuals are the unit of analysis
Descriptive - (U.S. Census)Explanatory - (Attitudes individuals hold)Exploratory - (Discover some new aspect or
characteristic dimension such as in a needs assessment)
Self-Administered Questionnaires
Mail Distribution and ReturnElectronic SurveysMonitoring ReturnsFollow-up MailingsResponse Rates
Interview Surveys
The Role of the Survey InterviewerGeneral Rules for Survey Interviewing
Appearance and DemeanorFamiliarity with QuestionnaireFollowing Question Wording ExactlyAccurate Recording of ResponsesProbing for Responses
Coordination and Control
Telephone Surveys
Sampling ProblemsPhone ownership used to be a problem now it is cell
phone and access to the numbers which are unlistedOver use by telemarketing and political parties
AdvantagesRandom Digit DialingCost Effective - Computer Controlled Multi-
Number Dialing
Strengths and Weaknesses of Survey Research
StrengthsUseful in describing the characteristics in large
populationsAllows for obtaining very large sample sizesStandardization of the questionnaire allows for greater
control over factors that may serve to bias informant responses
Strengths and Weaknesses of Survey Research
WeaknessesStandardization of the questionnaire may limit the
uniqueness of informant responsesSurvey research rarely allows for an understanding of
the informants’ lives in social contextThere is an assumptive leap that how a person
answers the survey has some bearing on how that person actually operates a situation
Steps in the Survey Research Process
Questionnaire ConstructionSample SelectionSurvey Administration (Data Collection)Data AnalysisDrawing Conclusions
Secondary Analysis
One of the less expensive ways to engage in survey research is to conduct an analysis on research data that has been collected for another purpose.
Data archives allows researchers to do thisSee web link in syllabus to the Murray Center
Also see the ACA Code of Ethics that addresses release of data to researchers
Conceptualization & Measurement
Conceptualization – The birthing of an idea
Operational definitions – Clearly defining terms
Measurement – Adopting a method or method(s) for specifying and collecting data that can be later used for analysis.
Conceptualization
Identify personal conceptionsIdentify public constructsDevelop nominal definitionDevelop operational definition
(Operationalization) Terms: Concepts, Constructs, Indicators,
Dimensions
Conceptualization
Where Do Research Topics Come From?Practical problems in the fieldLiterature in the fieldYour own thinking
Is the Study Feasible?Tradeoff between rigor and practicalityHow long it will takeEthical constraintsNeeded cooperationCosts
Conceptualization
Conducting the Literature ReviewReview the scientific literature – What does it say?Are there inconsistencies that warrant further study?
Do the review early in the process
The literature review can help youSee if your idea has been triedInclude all relevant constructsSelect instrumentsAnticipate common problems
Operational Definitions
Points the way to how a variable will be measured
Specify observation procedures
Specify coding rules
Measurement
Levels of measurement
Measurement Error/Precision
Reliability
Validity
Levels of Measurement
Nominal Scale - Names and CategoriesExample: Gender, Martial Status, Race
Ordinal - Rank ordering Example: 1st, 2nd, 3th
Interval - Equal intervals between levels of an attributeExample: Age expressed in whole years.
Ratio - Continuous data can assume any value between two point along a continuum. Example: Time (2.35 seconds)
Levels of Measurement - So What?
Must be able to chose the level of measurement that will allow you to answer your question of interest. Here is an example.
MaleFemaleAsianInfant+
MaleFemaleAsianInfant? Divided by 4 = ?
Nominal Data
Measurement Error
Systematic Error - reflects a false picture because of some flaw in the system. AcquiescenceSocial desirabilityCulture bias
Random Error - Inconsistencies inherent to any form of measurement
Avoiding Systematic Measurement Error
Select appropriate instruments for your project
Select instruments that have demonstrated reasonable validity and reliability statistics
Look for elements that might suggest biasConduct a small pilot studyBe certain co-researchers are up to the taskAttend environmental factors
Reliability
Defined as the degree to which assessment measures are consistent, dependable, and repeatable.Inter-Rater ReliabilityTest-Retest ReliabilityParallel Forms or Alternate Forms
ReliabilitySplit-Half Reliability
Validity
A multidimensional concept use as a means of expressing the degree to which a certain inference drawn from a test is appropriate and meaningful.
An instrument measures what it purports to measure.
Types of Validity
Content Validity - The content of the items make sense - said to have Face Validity.
Criterion-related Validity - The test score is related to one or more outcome criteria of interest. (Concurrent and Predictive)
Construct Validity - Establishes that the instrument expresses an accurate measure of some construct.
The ProblemThe Problem
Concepts are not mutually exclusive.They exist in a web of overlapping
meaning.To enhance construct validity, you must
show where the construct is in its broader network of meaning.
Take a look at the next slide.
What Is the Goal?What Is the Goal?
The The constructconstruct
Other Other construct: Aconstruct: A
Other Other construct: Cconstruct: C
Other Other construct: Bconstruct: B
Other Other construct: Dconstruct: D
Measure Measure allall of the construct and of the construct and nothing else.nothing else.
Trochim, 2001
Example: You Want to MeasureSelf-EsteemExample: You Want to MeasureSelf-Esteem
Self -Self -esteemesteem
Trochim, 2001
Example: How Would You Distinguish Self-esteem From...Example: How Would You Distinguish Self-esteem From...
Self- Self- esteemesteem
Self-worthSelf-worth
ConfidenceConfidence
Positive Self- Positive Self- disclosuredisclosure
OpennessOpenness
Trochim, 2001
To Establish Construct ValidityTo Establish Construct Validity
You have to set the construct within a semantic (meaning) net.
You have to provide evidence that your data support the theoretical structure. (Constructs that should be more related, are more related and constructs that should be less related, are less related.)
Supply evidence that you control the operationalization of the construct (that your theory has some correspondence with reality).
Trochim, 2001
Summary
Reliability - speaks to the is consistency of measurement. Good reliability suggests that as the measure is used repeatedly, all factors being equal, the results will show little change.
Validity - speaks to the accuracy of the measure in labeling that which it is supposed to be measuring.
Correlation – A Review
Correlation is a statistical measurement that indicates the strength of relationship between two sets of data (two variables).
In essence, a correlation coefficient tells us how two sets of data “co-relate.”
Another way to think about this is how two variables change relative to one another.
Correlation - Continued
Samplesr xy is the correlation between variable x
and variable yr height-weight is the correlation between height
and weightr SAT-GPA is the correlation between SAT
and GPA
Correlation - Continued
9 *8 *7 *6 *5 *4 *3 *2 *1 *0 1 2 3 4 5 6 7 8 9
Correlation - Continued
StrengthPerfect Correlation + 1.00 Perfect Correlation - 1.00None 0.00
DirectionDirection - Positive (+)
and Negative (-)
Absolute Value and Strength
Which is the Stronger of the two correlations below?
+.50 - .70or
Answer: - .70 because the absolute value | .70 | is greater than the absolute value | .50 |.
Understanding What Correlation Means - A Rough Guideline
General Rule
Size of the Correlation General Interpretation.8 to 1.0 ---------------------- Very Strong.6 to .8 ---------------------- Strong.4 to .6 ---------------------- Moderate.2 to .4 ---------------------- Weak.0 to .2 ---------------------- Very Weak or at 0 No Relationship
Good method for a quick assessment
Understanding What Correlation Means - Coefficient of Determination
The Coefficient of DeterminationDefined: The percentage of variance in one
variable that is accounted for by the variance in the other variable.
Remember Variability? (Salkind Chapter 3, page 39)Variability (spread or dispersion) is a measure of how
different scores are from one another.We learned that usually variability is thought of as
measure of how much each score in a distribution differs from the mean.
Understanding What Correlation Means - Coefficient of Determination
Variables that share something in common tend to correlate with one another.Final grade in Appraisal and in Research have a
moderately high correlation for many reasons.Similar ConceptsHours of Study Put Into the Endeavor Consistent Trait Committed to Regular PracticeAptitude for Subject Matter
These factors and others account for the differences in students’ grades - variability
Understanding What Correlation Means - Coefficient of Determination
The more two variables share in common the more they will be related - correlate.
The two variables are said to share variability (the reasons why the final grades in Appraisal and Research tend to be similar).Similar ConceptsHours of StudyRegular PracticeAptitude for Subject Matter
Understanding What Correlation Means - Coefficient of Determination
To determine exactly how much of the variance (the dispersion or spread) one variable can be accounted for by the variance (the dispersion or spread) in another variable you just simply square the correlation coefficient.
Let’s look at an example of this.
Understanding What Correlation Means - Coefficient of Determination
Previously it was stated that we could make rough estimates regarding the strength of a correlation.
Size of the Correlation General Interpretation.8 to 1.0 ---------------------- Very Strong.6 to .8 ---------------------- Strong.4 to .6 ---------------------- Moderate.2 to .4 ---------------------- Weak.0 to .2 ---------------------- Very Weak or No Relationship
Understanding What Correlation Means - Coefficient of Determination
By computing the shared variance (the Coefficient of Determination) we can see why a r = .8 is strong as opposed to a r = .2
r = .8 squared = .64 or 64% of the variance for each variable is shared
r = .2 squared = .04 or 4% of the variance of eachvariable is shared
Understanding What Correlation Means
I know you have been holding back on asking the one question that is burning in the forefront of your mind - so let’s tackle that now.
What about that portion of the variance that is not shared? That portion that can not be explained by a coefficient of determination?
Understanding What Correlation Means - Coefficient of Alienation
The Coefficient of Alienation is the amount of the variance in one variable not explained by the variance in the other variable.
Logic would suggest, then, that the portion of unexplained variance must be due to factors (variables) that have not, as of yet, been taken into account.
Applying Correlation in Practice
Recall our December clients’ intakes.
5 7 14 16 18 20 20 20 24 26 32 33 37
3 5 9 8 7 20 19 24 15 17 10 12 11
Lets look at their levels of depression last August and arrange to data as matched pairs
Applying Correlation in Practice
December clients’ intakes.
Mean = 20.23 Md = 20 Mo = 20 Range = 33 Sd = 10.04
5 7 14 16 18 20 20 20 24 26 32 33 37
3 5 9 8 7 20 19 24 15 17 10 12 11
December clients in August.
Mean = 12.31 Md = 11 Mo = X Range = 22 Sd = 6.2
Applying Correlation in Practice
December clients intakes - Var X.
5 7 14 16 18 20 20 20 24 26 32 33 37
3 5 9 8 7 20 19 24 15 17 10 12 11
December clients in August - Var Y.
Applying Correlation in Practice
December clients intakes - Var X.
December clients in August - Var Y
5 7 14 16 18 20 20 20 24 26 32 33 37
3 5 9 8 7 20 19 24 15 17 10 12 11
r = .38
2
r = .14
CorrelationCoefficient of Determination
Applying Correlation in Practice
Correlation
Coefficient of Determination
r = .38
2
r = .14
What is this results of our data analysis saying?
What research questions might these results generate?