slwk – 609 research methods- online- m. secret. problem idea theory what how who (variables)...
TRANSCRIPT
SLWK – 609 Research Methods- online- M. Secret
Podcast 3: More about Measurement
http://www.screencast.com/t/FkGKEd4Hbh
Problem Idea Theory
What How Who (variables) (methods) (sample)
Data Collection
Data Processing
Data Analysis
Discussion and Application
Backing- up.. Where are we in the research process
Measurement of VARIABLES
Problem <<<>>> Idea <<< >>> Theory What <<<>>> How <<<>>> Who (variables) (methods) (sample)
WHAT ARE YOU GOING TO OBSERVE – or measure -- some characteristic, or some type of knowledge, attitude, or behavior..
conceptualization and operationalization of variables.. How will you measure them…
A. WHAT IS MEASURMENT◦assignment of numbers to in dicators of concepts , or variables, according to rules (mathematical operations of numbers)
◦the link or the -- bridge between the world of concepts and the empirical world Tells you -- quantifies -- how much the variable
varies.. you are going from the very general or theoretical level to the fairly specific or operational level.. .science is deductive....
◦the number tells you, helps you to infer, how much or how little the concept is present
Thinking about the meaning of measurement
A variable is a concept that can be measured:
i.e. Gender is a concept.. When you measure it, it becomes a variable
Depression is a concept… when you measure it, it becomes a variable
Remember:
The variable age can be measured by answer to the question -- How old are you? ..
It can also be measured by collecting data from an existing form that gives you the date of birth
By nature, age varies by numbers of year… so the number is assigned automatically
Examples of how some variables are measured
The variable gender can be measured by the answer to the question, Are you male or female?
Because there is no automatic number associated with the variable gender, you the researcher assigns the number.. An example would be that the value of 1 is assigned to female and 0 is assigned to male
A researcher might also be able to ‘observe’ whether someone is male or female, but we all know that this would not be a particularly valid way to measure gender
The variable stress can be measured by the summed score of the answers to 10 items on the Stress scale
Stress can also be measured by some physiological measure, i.e. blood pressure, where the researcher states that a blood pressure number above a certain level is an indicator of stress
The level of social support can measured by the answer to the question ‘how much social support do you have’ 1 = very much; 2 = some; 3 = very little
Or by asking several questions about social support and adding the answers to these questions together to get one score.. This produces a scaled score, or number, that tells you how much of or how little social support exists.
Some concepts are easier to measure than other concepts because they are, by nature, more concrete
Age is fairly easy to measure by asking just one question --how old are you? Or What is your date of birth?
Depression is more difficult to measure because it’s hard to know what to count, or what exactly to observe, to know whether or not it is present.
Trauma is difficult to measure because you need to be clear about whether you’re going to measure the traumatic event or the trauma symptoms.
For abstract concepts, it is especially important that you define what you mean by a concept before you try to measure it.
i.e. Depression is a feeling of malaise or emotional discomfort that interferes with day to day routine functioning.
Trauma is an external event that causes injury or harm.. OR
Trauma is the condition or response to an event that causes injury or harm.. OR
Trauma is an emotional wound caused by…
Importance of conceptualization
What is the difference between conceptualization and operationalization?
conceptualization ◦Uses words to define◦Dictionary definition◦More abstract
Operationalization◦Uses numbers to define◦How measured◦More concrete – what you count
Measurement and operationalization can be thought of as almost the same thing
When you operationalize a concept you are assigning numbers to a variable
Operationalization = how you are going to measure a concept.
Variation is represented by the characteristics or categories or values of a particular variable
Gender is the variable, the values are male (0) or female (1);
Age is the variable, the values are 1, 2, 3,
Stess is a variable, the values are very much (3), some (2), little (3)
How a variable varies is known as the level of measurement.
Types =
Nominal – variable varies by group or categories◦(Male, female); (married, never married, divorced);
(republican, democrat)◦ variable is either present or not ..
Ordinal – varies by rank or order .. ◦More or less, higher or lower ..( Low, mid, high income
level)
Interval/ratio – varies by specific distance between values ..
◦1 is the same distance from 2 as 2 is from 3, etc etc ◦50 feet is higher than 30 feet -- ◦Difference between values/categories is precise
Levels of measurement
Example: The Group and Individual Assessment Form to “measure” individual and group participation for the research project
consider the form as a measurement tool
INDIVIDUAL ASSESSMENT OF OWN PARTICIPATION: The following scale evaluates level of effort you contributed to your Research activities. Circle the number above the statement that best describes your effort towards the project. Your name _____________________________
5 4 3 2 1 Regular attendance at meetings Attended all meetings, stayed to agreed end, working within timescale, active and attentive,
Missed many meetings, always or often late, left early, digressed, giggled, day-dreamed or gossiped most of the time
5 4 3 2 1 Contribution of ideas for the task Thought about the topic in advance of the meeting, provided workable ideas which were taken up by the group, built on others' suggestions, and were prepared to test out ideas on the group rather than keep quiet
Didn't come prepared. Didn't contribute any ideas. Tended to reject others' ideas rather than build on them
5 4 3 2 1 Researching, analyzing and preparing material for the task Did what you said you would brought materials, did an equal share of the research and helped to analyze and evaluate the material
Did no research. Didn't do what you promised to do. Didn't manage your workload. Didn't get involved with the task and allowed others to provide all the material
5 4 3 2 1 Contribution to the cooperative group process Left personal differences outside the group, willing to review group progress and tackle conflict in the group, took on different roles as needed, kept group on track, willing and flexible but focused on the task
Did not take initiative, waited to be told what to do. Always took the same role (leader, joker etc) regardless of circumstances, created conflict, and not prepared to review group processes
5 4 3 2 1 Supporting and encouraging group members Keen to listen to others, encouraged participation, enabled a collaborative learning environment, sensitive to issues affecting group members, supported group members with special needs
Sought only to complete the task, spoke over others and ignored their opinions, kept ideas and resources to self. Insensitive to individuals' needs and did not contribute to the learning process
5 4 3 2 1 Practical contribution to end product Willing to try new things. Not hogging the tasks, made a high level of contribution, took own initiative, was reliable and produced a high standard work/presentation
Not willing to take on any task, did not take any responsibilities, were unreliable so others felt the need to keep checking up, and made a limited, poor quality contribution
May want to stop the recording and access a copy of this form which is posted on blackboard
Team Assessment Form Measurement application
◦ Develop conceptual/nominal definition(s) for what this form is measuring
◦ Identify the variable(s) and the indicator(s) that the form is measuring
◦ What is the level of measurement for the variables “Regular attendance at meetings” and for “Practical contribution to end product”
◦ What are the values for each of these variables◦ How could you convert these variables to a
nominal level of measurement. ◦ Does it make sense to consider the questions on
this form as a scale. Provide a rationale for your answer.
◦ Create a scale using the items on this form ◦ Applying validity and reliability
Team Assessment Form: What is this form measuring??
In general conceptualization … Participation in group project Own/other contributions to group project
More specific conceptualization… Regular attendance at meetings Contribution of ideas for the task Researching, analyzing and preparing
material for the task Contribution to the cooperative group
process Supporting and encouraging group
members Practical contribution to end product
Regular attendance at meetingsAttended all meetings, stayed to agreed end, working within timescale, active and attentive,
Contribution of ideas for the taskThought about the topic in advance of the meeting, provided workable ideas which were taken up by the group, built on others' suggestions, and were prepared to test out ideas on the group rather than keep quiet
Researching, analyzing and preparing material for the taskDid what you said you would brought materials, did an equal share of the research and helped to analyze and evaluate the material
Contribution to the cooperative group process Left personal differences outside the group, willing to review group progress and tackle conflict in the group, took on different roles as needed, kept group on track, willing and flexible but focused on the task
Supporting and encouraging group membersKeen to listen to others, encouraged participation, enabled a collaborative learning environment, sensitive to issues affecting group members, supported group members with special needs
Practical contribution to end productWilling to try new things. Not hogging the tasks, made a high level of contribution, took own initiative, was reliable and produced a high standard work/presentation
Defining concepts.. More specific, concrete
the definition, better the measurement validity
Thought for the day!!Definition of any one concept rests on shared
understandings of other terms used in the definition
Research is as much about communication as it about anything else
Regular attendance at meetings Extent to which self/others attended all meetings, stayed to agreed end, working within timescale, active and attentive,
Contribution of ideas for the task
How much self/others thought about the topic in advance of the meeting, provided workable ideas which were taken up by the group, built on others' suggestions, and were prepared to test out ideas on the group rather than keep quiet
Researching, analyzing and preparing material for
the task How much did self/others do what he/they said; brought materials, did an equal share of the research and helped to analyze and evaluate the material
Identify the variable(s) and the indicator(s) that the form is measuringVariable
indicator
Variable
Variable
indicator
indicator
5 4 3 2 1
Regular attendance at meetings Attended all meetings, stayed to agreed end, working within timescale, active and attentive,
Missed many meetings, always or often late, left early, digressed, giggled, day-dreamed or gossiped most of the time
5 4 3 2 1 Contribution of ideas for the task Thought about the topic in advance of the meeting, provided workable ideas which were taken up by the group, built on others' suggestions, and were prepared to test out ideas on the group rather than keep quiet
Didn't come prepared. Didn't contribute any ideas. Tended to reject others' ideas rather than build on them
What are the values for each of these variables
5,4,3,2,1, = valuesHow.. or the degree to which
the variable varies
What is the level of measurement for the variables “Regular attendance at meetings” and for “Practical contribution to end product”
5 4 3 2 1
Regular attendance at meetings Attended all meetings, stayed to agreed end, working within timescale, active and attentive,
Missed many meetings, always or often late, left early, digressed, giggled, day-dreamed or gossiped most of the time
5 4 3 2 1 Contribution of ideas for the task Thought about the topic in advance of the meeting, provided workable ideas which were taken up by the group, built on others' suggestions, and were prepared to test out ideas on the group rather than keep quiet
Didn't come prepared. Didn't contribute any ideas. Tended to reject others' ideas rather than build on them
Ordinal5 is more than 4… is more than 3
Values are ordered/ranked
◦How would you create a scale using the items on this form
Add together the values for all the items into ONE score.. The one score tells you how much self/others contributed to the group effort
Application to reliability and validity In the area of measurement, researchers confront
two major issues: validity and reliability. Differentiate between validity and reliability.
Validity – does the variable measure what it is supposed to … is the operational definition the same as the conceptual definition
Reliability – do you get the same answer every time you measure a variable
What does ‘(measurement) reliability’ mean in terms of this form?
A person will answer the same way every time they answer the question -- under the same circumstances
Stable and dynamic traits◦Gender = stable.. Doesn’t change much over time
◦Happiness = dynamic… changes frequently
Reliability and Validity
What does ‘(measurement) validity’ mean in terms of this form?The form is an accurate measure of what you want it to measure….
i.e. it is a good measure of participation in/ contribution to the group effort
Test-retest◦ Will someone fill out the form on two different
days (weeks) the same way – ◦ providing nothing has changed in the situation
Inter-rater Internal Consistency reliability – applies mainly to scaled measurements (alpha or chronbach’s)
◦ Will someone answer the same way (i.e. all 5’s or all 1’s) on each item of the scale
◦ The stronger the association among individual items and the more items included, the higher the reliability of a measure
What are the ways that you could assess the reliability of this form?
Content validity/face validity subjective judgment as to whether or not an instrument is valid based
on whether or not the test items appear to be related. Content validity can be strengthe ned by having experts make the
judgment as to whether or not a measurement measures what it is supposed to measure
What are the ways that you could assess the validity of this form?
Criterion validity◦ How well does your measure compare to another measure .. Another
standard -- or ruler that you trust to be accurate . hold up your test to see how well it compares to some predetermined, already existing standard.
◦ In concurrent validity, the standard, or ruler, is other tests which have already been determined to have validity...
DEPRESSION SCALES...GCS SCALE WITH BECK OR ZUNG....or with DSMIII-R
◦ In predictive validity, the standard you are comparing your test to is what happens in the future. Do social workers who test high on the empowerment scale more likely to
become involved in community activities SAT test as a measure to determine how well students do academically and
thus compares it with how well the student actually does in school... scales to measure high or low risk adoptions placements would be compared
with whether or not the adoption was disrupted. Construct validity
◦ related to a variety of other measures as specified in a theory, used when no clear criterion exists for validation purposes
Do a cheat sheet on this.
Conceptualization and Operationalization
◦ In order to select which concepts to consider and which variables to measure, we must
Examine relevant theories Review previous research Consider the constraints and opportunities Understand that the definition of any one
concept rests on shared understandings of other terms used in the definition
Key Point Re Measurement
FROM LIT REVIEW
• What is treatment fidelity?• Why is it important?
◦ treatment is the independent variable in intervention studies.
◦ need to know the extent to which intervention was implemented in order to understand the outcome of the intervention
• What will you measure to assess treatment fidelity?
• How will you measure treatment fidelity?• What is the connection between treatment
fidelity and evidence based practice.
A Note about Treatment Fidelity