business & graduate studies, trinity washington university qualitative data collection...

Business & Graduate Studies, Trinity Washington University

Qualitative Data Collection Instruments

Research Rescue LabDr. Kelley Wood

A Holistic View of the Research Paper

Dr. Kelley D. Wood, Business & Graduate Studies

2

Topic of Inquiry

Literature Survey

Research Questions Abstract Introductio

nLiterature

ReviewResearch Methods

Results / Findings Discussion

Data Collection Instruments

• Literature Review– This chapter builds the argument, through

logic, that is the basis of the study• The result of this is the Theoretical Framework ,or

Theoretical Construct, or Conceptual Map

• Research Methods– Determine the strategy for collecting data– Develop the Data Collection Instrument

• Derived from the Quantitative Theoretical Framework, or the Qualitative Theoretical Construct, Conceptual Framework, or Literature Map

3Dr. Kelley D. Wood, Business & Graduate Studies

A Holistic View of the Research Paper

• Literature Review– Sections of the chapter

• Preamble – A brief argument, moving from general to specific, that prepares

the reader for engaging in the topic areas you have determined must be covered to support the Theoretical Framework or Theoretical Construct

• Sections and subsections as necessary to the argument– Case studies should include the subject of the case

• Review of other research methods – Used in investigating similar topics

• Theoretical Framework or Theoretical Construct– Describe and define the elements as they will be used in your

study– Including a figure or model as a visual representation



• Research Methods– Preamble – Setting– Population/ Data Source

• Your potential participants– Research Strategy/ Measurements– Intervention Protocol– Data Collection Instrument Protocol

• Observation, Interview, Questionnaire, Survey • Secondary data, content or document analysis - How data

sources will be selected– Data Analysis Strategy– Summary



6

Th

eory

Vari

ab

le o

r Fa

ctor Concep

t

Concept

Concept

Vari

ab

le

or

Fact

or

Concept

Concept

Concept

Becomes the…



7

Data

C

olle

ction

Instru

men

tFa

ctor

Question

Question

Question

Facto

rQuestio

n

Question

Question

Becomes the…



8

Th

eory

Vari

ab

le o

r Fa

ctor Concep

t

Concept

Concept

Vari

ab

le

or

Fact

or

Concept

Concept

Concept

Data

C

olle

ction

Instru

men

tFa

ctor

Question

Question

Question

Facto

rQuestio

n

Question

Question



• Purpose of data collection instrument– To test the hypotheses– Clarify relationships between variables

(concepts) – To test propositions– To discover more about the phenomena of

interest• Which is really to say we are testing

the theoretical framework, construct, or model



• Interviews, Questionnaires, and Surveys– Schensul, Schensul, & LeCompte

(2009, chapters 6, 7, & 8)• Ethnography

– Process is predicated on the understanding that nothing is really known of a new culture to be studied

– Uses scientific methodology to develop a greater understanding



• Ethnography– Process is predicated on the understanding

that nothing is really known of a new culture to be studied

– Moves from • Open-ended Observation • Open-ended Interviews• Semi-structured Interviews• Structured (directed) Surveys

– Follows the Mixed Methods pattern • Qualitative to Quantitative• Development of theory• Testing of theory



• Ethnography– Process is predicated on the understanding

that nothing is really known of a new culture to be studied

– Follows the sequential mixed methods pattern • Qualitative

– Development and testing of propositions– Propositions = local or low level theories

• Testing– Development of theory

• Quantitative– Development of hypotheses– Testing of hypothesis



• Developing the instrument– Follow the continuum of abstraction

• Observation– Description of domains, macro-structural

features, cultural continuities• Interviews

– Description of factors, sub-factors, identification of cultural variation

• Questionnaire – Refined description of factors, sub-factors,

identification of cultural variation• Survey

– Identification of variables, testing of associations and predictive models



• Constructing the instrument– Variable (concept) development

• Domain to Factors to Variables (concepts) • Example

– Domain - Work » Factor - Satisfaction with job

» Variables (concepts)» Compensation, autonomy,

motivation, relationships – coworkers and supervisors or management

» Position, advancement, relevance, decision making power, support

• Schensul, Schensul, & LeCompte (2009, pp. 174-176)



Interviews


Developing the Interview

• Open-ended interviews– Schensul, Schensul, & LeCompte (1999, chapter 6)

• Purpose of the open-ended interview– Explore undefined domains– Identify new domains– Break known domains into factors and subfactors– Obtain contextual and historical information to orient

the research– Build understanding and positive relationships with the

study population– To understand the individual lived experiences of

participants– To understand participant’s meaning schemas



• Think of the interview in the continuum of

ethnographic methodology– Qualitative

• Open-ended observation• Structured observation

– Observe for specific constructs, interactions, behaviors• Open-ended interview

– Open-ended questions• Semi-structured questionnaire

– Semi-directed questions– Quantitative

• Survey – Closed-ended questions



• And the continuum of the data developed– Qualitative

• Open-ended observation– Determine initial constructs and domains

• Structured observation– Refine domains and constructs

• Open-ended interview– Deeper understanding of domains and constructs– Determine factors that comprise the domains and constructs– The factors are your Dependent and Independent Variables

• Semi-structured questionnaire– Specific understanding of factors (variables)– Determine subfactors that comprise the factors

– Quantitative• Survey

– Measuring the subfactors



• Interview questions relate to a factor– Do not go as deep as the variable (concept)– Develop data that relates to your research

questions

• Purpose of the questions– Open-ended questions

• Inspire a narrative account of the participant’s experience

• Inspire the participant to relate naturally• Not to meet an expectation

• Applied to a limited population19Dr. Kelley D. Wood, Business & Graduate

Studies


• Open-ended questions– Represent the factors in the domain – Inclusive of a factor or multiple factors– Do not lead the response or express bias

• Prepare probing and clarifying questions– Neutral agreement– Repeat what is said as a question– Ask for more information– Ask for their opinion– Ask for clarification of discrepancies– Ask for clarification of terms and acronyms


EXERCISE 1

In the handout



Questionnaires


Developing the Questionnaire• Open-ended and semi-structured interviews

– Schensul, Schensul, & LeCompte (1999, chapter 7)• Purpose of the semi-structured interview

– Clarify central domains and factors– Operationalize factors into variables (concepts)– Develop a preliminary hypothesis– Develop the qualitative base for a quantitative survey


Developing the Questionnaire

• Think of the questionnaire in the continuum of ethnographic methodology– Qualitative


– Observe for specific constructs, interactions, behaviors

• Open-ended interview– Open-ended questions

• Semi-structured questionnaire– Semi-directed questions


– Closed-ended questions 24Dr. Kelley D. Wood, Business & Graduate Studies

Developing the Questionnaire• And the continuum of the data developed

– Qualitative• Open-ended observation

– Determine initial constructs and domains• Structured observation

– Refine domains and constructs• Open-ended interview

– Deeper understanding of domains and constructs– Determine factors that comprise the domains and

constructs– The factors are your Dependent and Independent Variables






• Questions link back directly to a variable (concept)– Develop data that relates to your research questions

• Purpose of the questions– Semi-structured questions

• Semi-directed• Provide focus• Not leading

– Question does not contain the expected answer• Not closed

• Applied to a larger population with a limited interpretation


Developing the Questionnaire• Developing semi-structured questions

– Use terms and phrases that the participant • Will understand• Are appropriate to their context

– Keep the length minimal– Avoid leading questions– Avoid ‘two for one’ questions


Developing the Questionnaire• Developing semi-structured questions

– Avoid negatively phrased questions– Avoid rank ordering

• Save for the survey

– Avoid questions that require directions or tasks to complete– Avoid yes or no questions– Be sensitive to cultural context or social meanings in the

questions



• Developing semi-structured questions– Ordering the semi-structured questions

• Temporally – use an arc of time– Earliest to latest

• Complexity – use an arc of complexity– Least to most complex

• Group by topic or domain– Topically similar questions together

• Abstraction – use an arc of abstraction– Most concrete to most abstract

• Threat level– Least sensitive to most threatening


Using both Open-ended and Semi-Structured Questions

• Lead with open-ended questions– Opens the participant up– Refreshes their sense of the experience

• Transition– Briefly review what you have heard to clarify– Allow them to correct or modify– Discuss purpose of the next section without giving too many specifics– The next section may ask the participant to repeat or clarify

• Move to semi-structured questions• Transition

– Briefly review what you have heard to clarify– Allow them to correct or modify

• Closing comments from participant– Ask them if there is anything that they wished you asked– Or that occurred to them that they would like to share



• Research article showing a coding key– Jehn, K. A. (1997) A qualitative

analysis of conflict types and dimensions in organizational groups. Administrative Quarterly 42(3) 530-557 retrieved from ProQuest Social Sciences Journals

– While you are developing your instruments• Interview, Questionnaire, or Survey• Develop a coding key


EXERCISE 2

In the handout



Surveys


Developing the Survey

• Structured (directed) Surveys– Schensul, Schensul, & LeCompte (1999, chapter 8)

• Purpose of the directed question survey– Clarify relationships between factors and

variables (concepts)– Operationalize factors into variables

(concepts)– Test hypotheses– Develop detailed , granular understanding

of factors34Dr. Kelley D. Wood, Business & Graduate

Studies


• Think of the questionnaire in the continuum of ethnographic methodology– Qualitative


– Observe for specific constructs, interactions, behaviors

• Open-ended interview– Open-ended questions

• Semi-structured questionnaire– Semi-directed questions


– Closed-ended questions 35Dr. Kelley D. Wood, Business & Graduate Studies


• And the continuum of the data developed– Qualitative

• Open-ended observation– Determine initial constructs and domains

• Structured observation– Refine domains and constructs

• Open-ended interview– Deeper understanding of domains and constructs– Determine factors that comprise the domains and constructs– The factors are your Dependent and Independent Variables






• Questions link back directly to a variable (concept)– Develop data that relates to your research questions

• Purpose of the questions– Directed questions

• Provide measurement of behaviors at specific points in time• Detailed examination of relationships between dependent

and independent domains (variables)• Not leading

– Question does not contain the expected answer• Closed response

• Larger random and generalizable population



• Constructing the instrument• Considerations

– Exhaustive – no surprise responses– Mutually exclusive – response fits only

one option– Avoid two or more nouns or verbs – you

want to have no confusion of subject or action



• Constructing the instrument– Variables (concepts) to questions

• Question or statement format• Avoid open-ended questions• Each items is stated clearly• Mutually exclusive• Avoid more than one noun or verb



• Constructing the instrument– Variables (concepts) to questions

• Can respondents answer the question• Are questions relevant to the topic• Questions are as short as possible• Avoid alternating positive and negative

questions in the same scale• Avoid items and terms that have hidden or

unclear meanings



• Constructing the instrument– Level of Measurement

• Nominal – number is assigned to represent something as numbers are easier in statistical programs

– Vary by quality but not quantity– Country, gender, etcetera

• Ordinal – values convey order or rank only– Ordered by ranking, not absolute– May chose only one answer

• Interval – values have no true zero and are equidistant

– Distance between ranked intervals, no zero start point– 100 to 120 = 90 to 110 (interval of 20)

• Ratio – has a true zero and values are equidistant– Based on a zero input– Age, income, numbers of a variable (units)



• Constructing the instrument– Types of questions

• Fill in – correct answer• Multiple choice – choose correct answer from

options• Scaled – such as Likert, continuous (on a continuum

with decimals), or discrete (whole numbers or units not on a continuum)

• Semantic Differential – Choose from opposites

• Ranking– Order (<=) 10 items against each other

• Rating– Rate(<=) 10 items from 1 to 10

• Category– Often a yes/no choice of distinct options



• Scales – an aside– Scales can easily convey bias or preferred

response– Choosing a scale of measurement

• When using Median or Mean• Order – does a larger number indicate a greater

value than a smaller number– Ordinal, interval, ratio

• Differences – does subtracting two numbers indicate a meaningful value

– Interval, ratio• Ratio – Does dividing two numbers indicate a

meaningful value – Ratio

• Use the same pattern for consistency – parallelism



• Scales – an aside– Likert is not the only game in town

• The point of Likert is to cause data to be easier to summarize around a central tendency

– Median, Mode, Mean• Using a 5 point scale should encourage the

response to be either positive or negative, unless it is central

– Often encourages the selection of the central value – 1 2 3 4 5

• Some use a 6 point Likert– Middle two values are the central tendency– 1 2 3 4 5 6



• Scales – an aside– When is Central Tendency important,

or when to use Likert?• Ordinal

– One sample sign test, Mann-Whitney U test, Wilcoxon sign-ranks T test, Kruskal-Wallis H test, Friedman test

• Interval/Ratio– One-independent sample t-test, Two-

independent sample t-test, Related samples t-test, One-way between-subjects ANOVA, One-way within-subjects ANOVA, Two-way between-subjects ANOVA


EXERCISE 3

In the handout


business & graduate studies, trinity washington university qualitative data collection...

Documents

kelley wood slide

business graduate studies

data sources

survey secondary data

factor concept variable

theoretical construct

argument case studies

conceptual framework