1 mixed methods research: state of the art (what has developed in mixed methods) by john w....
Post on 24-Jan-2016
218 views
TRANSCRIPT
1
Mixed Methods Research: State of the Art
(What Has Developed In Mixed Methods)
by John W. Creswell, Ph.D.
Department of Educational Psychology, University of Nebraska-LincolnCo-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research
© Please do not duplicate or use these slides without the express permission of the author.
2
The Story of Mixed Methods in the Last 20 Years
Highlight four developments
Why I can tell the story First generation mixed methods – reflect on my role in story Balanced – background in quan and qual 5 books on research methods Applied research methodologist Focused on education, social and human sciences Journal editor
What I am trying not to do “Fix” the history Discourage discourse Speak for all disciplines Limit the discussion to a US perspective Cover all developments
3
Four Developments (last 20 years)
Increasing interest in and advocacy for mixed methods Evolving understanding of what is mixed methods research Emerging philosophical perspectives as a foundation for mixed
methods Developing designs and innovative techniques
4
Development #1: Increasing Interest in and Advocacy for Mixed Methods
Research “movement” Federal funding interest Discipline interest International interest Books Journals Conferences Critics
Key Developments
5
“The emergence of mixed methods as a third methodological movementin the social and behavioralsciences began during the 1980’s.” (p. 697)
-Tashakkori & Teddlie (2003)
Mixed Methods asa “Movement”
6
New NIH Awards Using the Term "Mixed Methods"
0
5
10
15
20
25
30
35
40
45
1998 2000 2002 2004 2006 2008
Year
Nu
mb
er
of
New
Aw
ard
s
NewNIH Awards Using Mixed Methods
7
Development of NIH guidelines (1999)
- Mentioned severalapproaches for combiningqualitative and quantitative research- Advanced considerations for deciding what model to use (literatureavailable, prior studies, realistic design, expertise)- Suggested to describe each method thoroughly
“Combining qualitative and quantitative methods has gained broad appeal in public health research.” (1999)
8
NIH Summer Institute:The Design and Conduct of Qualitative and Mixed-Method Research in Social Work and Other Health Professions
August 4 -8, 2004
Table of Contents
•Course Objectives •Dates and Location •Background •Concept •Institute Co-Chairs •Faculty/Presenters •Speaker Presentations •Eligibility Requirements •Number of Participants •Costs •Application Process and Receipt Date •Additional Information
Course Objectives To provide a thorough grounding in the design and conduct of qualitative and mixed method research to social work researchers and other health professionals interested in developing competence in the planning, design, and execution of these techniques and become more successful in the NIH extramural research program.
Social Work Summer Institute on Mixed-MethodsResearch
9
National Science Foundation Workshop, 2003, Includes Mixed Methods Research
-Provided generalguidelines for developingqualitative research projects-Several papers addressed mixed methods research
10
14+ books on mixed methods research have been written since 1988
Books onMixed MethodsResearch
11
Two New Books FromSage Publications
12
Editors: John W. Creswell and Abbas TashakkoriManaging Editor: Vicki L.Plano Clark
Email: [email protected]
JournalsDevoted toMixed Methods:FieldworkQuality andQuantityMixed MethodApproaches
A NewJournal
13
Conferences
Mixed Methods Conference, Cambridge, England, July 21-24, 2008
Proposed mixed methods conference, Sydney, Australia, 2009
Discipline conferences with mixed methods papers
14
Critics (Creswell, 2007) Is there a post-positivist leaning to mixed methods?
Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10, 42-61.
Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3), 195-2003.
Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive frameworks. Contributed paper for the Mixed Methods Conference, Cambridge, University, July 10-12, 2006.
What are the limits for mixed methods research? Sale, J. E. M., Lohfeld, L. H., Brazil, K. (2002). Revisiting the quantitative-
qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36, 43-53.
Leahey, E. (2007). Convergence and confidentiality? Limits to the implementation of mixed methodology. Social Science Research, 36, 149-158.
Is there a dominant discourse in mixed methods research?• Freshwater, D. (2007). Reading mixed methods research: Contexts for
criticism. Journal of Mixed Methods Research, 1(2), 134-146.
15
Development #2: Evolving Understanding of What is Mixed Methods Research
The emergence of diverse perspectives - Two strands to “mixing” to “methodology” to “mixed methods within designs”
Incorporation of perspectives into current definitions Questions arise about what is “mixing?” Recent assessment of the value of mixed methods research (or
mixing)
Key Developments
16
Framework for Viewing Perspectives on Mixed Methods
Quantitative Data Qualitative Data
MethodologyMethod
ParadigmPerspective Research Design
Procedures
Mixed Methods
17
Mixed methods is… a Method, a Design, a Methodology
Collecting data
Analyzing and interpreting data
Identifying a purpose and stating questions
Reporting and evaluating the study
Identifying a research problem
Reviewing the literature
18
Mixed Methods within Designs
Ethnography Case Study Research Narrative research Experimental research
Qual ----- Quan Qual----- Quan Qual-----Quan Qual---- Quan
19
What is this Method Called?
Multi-method Triangulation Integrated Combined Quantitative and qualitative methods Multi-methodology Mixed methodology Mixed-method Mixed research Mixed methods
20
A Definition
Mixed methods research is a methodology for conducting research that involves collecting, analyzing, and integrating (or mixing) quantitative and qualitative research (and data) in a single study or a longitudinal program of inquiry. The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.
Aspects of this definition: Collecting both quantitative and qualitative data Mixing the data (from merging in which both quan and qual loses identity to keeping them distinct but connected) Single vs multiple-phase project
Raises issues: how, what, where, why, value (of mixing)
21
Mixing (How)
Results
Connect data:
Results
Converge data:
Embed the data:
Quan data
Qual data
Qual Quan
Qual Quan
22
Mixing
What Data (Methods) Qualitative and quantitative research Philosophical assumptions
Where Throughout the process of research Data collection Data analysis Interpretation
23
Mixing – Why? (Bryman, Qualitative Research, 2006)
Validity – to corroborate quan and qual data Offset – offset weaknesses of quan and qual and draw on strengths Completeness – more comprehensive account that qual or quan alone Process – quan provides outcomes; qual, the processes Different question – quan and qual answer different questions Explanation – qual can explain quan results or vice-versa Unexpected results – surprising results from one, other explains Instr development – qual employed to design instr, then instr tested Sampling – one approach facilitates sampling from other approach Credibility – both approaches enhance integrity of findings Context – qual provides context; quan provides general. Illustration – qual data helps develop “depth” for quan data Utility – more useful to practitioners Confirm – quan tests qual generated hypotheses Diversity of views – relationship and meaning; researcher/participant views Enhancement – augmenting or building on one form of data with the other
24
Value of “Mixing”
“What does mixed methods research provide that either quantitative or qualitative research alone does not provide?”
Indicators of “yield” (O’Cathain, Murphy, & Nicholl, JMMR, 2007) Integration as a measure of “yield” Publications as a measure of “yield”
My speculations: Results are useful Stronger impact on policymakers Cost-effective Better solutions through interdisciplinary input More evidence for a study
25
Development #3: Emerging Philosophical Perspectives as a Foundation for Mixed Methods Research
Key developments:
There are many stances taken by researchers. “Purist” stance - Paradigms are incompatible (fixed,
unmoveable), paradigms relate to methods, therefore mixing is impossible.
“Dialectic” stance - Multiple paradigms are possible and can be used, but honor and make explicit the paradigms.
“Single paradigm” stance - There is a single paradigm for mixed methods research: pragmatism, transformative.
“Design” stance - Multiple paradigms can be used – relate them to your design.
“Community” stance – source of belief systems come out of shared meanings and understanding
26
Philosophical Assumptions Behind Mixed Methods Research
Worldview or philosophy (e.g., attitudes and beliefs about knowledge, such as constructivism, post-positivism)
Theoretical lens (e.g., feminist, racial)
Methodological approach (e.g., experimental, survey, ethnography, mixed methods)
Methods of data collection (e.g. interviews, focus groups)
Adapted from Crotty M. (1998).
27
PostpositivismDeterminationReductionismEmpirical observation and measurementTheory verification
ConstructivismUnderstandingMultiple participant meaningsSocial and historical constructionTheory generation
Advocacy/ParticipatoryPoliticalEmpowerment issue-orientedCollaborativeChange-oriented
PragmatismConsequences of actionsProblem-centeredPluralisticReal-world practice oriented
Four Worldviews
28
Smith & Heshusius, (1986)
“Incompatibility thesis”
“The contention of thispaper is that the claim ofcompatibility, let alone one of synthesis, cannot besustained.” (p. 4)
Purist Stance
29
“Dialectic Stance”
•Paradigms – philosophical assumptions,knowledge, methodology, values• Paradigms are different – but are social constructions – not sacrosanct• Differences should be respected,intentionally used together• Used together, tensions achieve dialectical discovery of enhanced,reframed, or new understandings.
Greene (2007)
30
“Single” Paradigm Stance
Pragmatism (Tashakkori & Teddlie, 2003) 13 writers endorse Based on Dewey, Rorty, Pierce, others (Americans) Focus on research question What “works,” consequences Multiple methods – large toolkit
Transformative (Mertens, 2003) Transform lives of underrepresented, marginalized groups Incorporate this focus into all phases of research
31
“Design” Stance (Creswell & Plano Clark, 2007)
Multiple paradigms can be used Different paradigms can provide the foundation for different
phases of research Often post-positivist paradigm provides the foundation for
quantitative aspects of research Often constructivist/advocacy paradigm influences
qualitative aspects of research Often pragmatism paradigm influences both qualitative and
quantitative aspects of research This does not suggest that certain paradigms
are linked to certain methods (paradigm-method fit)
32
“Community” Stance (Morgan, 2007)
Espouses the pragmatic approach Shifts conversation to the importance of
shared meanings and joint actions between researchers.
“To what extent are two people (or two research fields) satisfied that they understand each other, and to what extent can they demonstrate the success of shared meaning by working together on common projects.” (Morgan, 2007, p. 67)
33
Development #4: Developing Designs and Innovative Techniques
Evaluation designs Notation Typologies, and a parsimonious set Diagrams of procedures Complex evaluation models Reconceptualizing designs The practice of research Emergence of innovations in procedures
Unusual blends Methodological issues Data analysis techniques Presentation techniques
Key Developments
34
• Triangulation (convergence,corroboration)• Complementarity(clarify results fromone method to theother)• Development (one method informsthe other method)• Initiation (paradox,contradiction)• Expansion (extendsbreadth of inquiry)
Early Designs(Greene, Caracelli,& Graham, 1989)
35
Preliminary Design Considerations (Morse, 1991)
Approach Type Purpose Limitations Resolutions
QUAL + quan Simultaneous Enrich description of sample
Qualitative sample
Utilize normative data for comparison of results
QUAL Sequential Test emerging H, determine distribution of phenomenon in population
Qualitative sample
Draw adequate random sample from same population
QUAN + qual Simultaneous To describe part of phenomena that cannot be quantified
Quantitative sample
Select appropriate theoretical sample from random sample
QUAN Sequential To examine unexpected results
Quantitative sample
Select appropriate theoretical sample from random sample
quan
qual
36
QUANData & Results
QUANData & Results InterpretationInterpretation
QUALData & Results
QUALData & Results
QUANPre-test Data & Results
QUANPre-test Data & Results
QUANPost-test
Data & Results
QUANPost-test
Data & Results
Intervention
qualProcess
qualProcess
InterpretationInterpretation
Triangulation Design
Embedded Design
Concurrent Mixed Methods Designs
Parsimonious Designs (Creswell & Plano Clark, 2007)
37
Sequential Designs Mixed Methods Designs
QUANData & Results
QUANData & Results
InterpretationInterpretation
qualData & Results
qualData & Results
Following up
QUALData & Results
QUALData & Results
quanData & Results
quanData & Results InterpretationInterpretation
Building to
Before-intervention
qual
Before-intervention
qual
QUANIntervention
Trial
QUANIntervention
Trial
After-
intervention qual
After-
intervention qual InterpretationInterpretation
Exploratory Design
Explanatory Design
Sequential Embedded Design
38
Helpful Tips for Drawing this Visual of the Design:
1. Give a title to the visual model.2. Choose either horizontal or vertical layout for the model.3. Draw boxes for quantitative and qualitative stages of data
collection, data analysis and interpretation of the study results.4. Use capitalized (QUAN) or small letters (quan) to designate priority
of quantitative and qualitative data collection and analysis.5. Use single-headed arrows to show the flow of procedures in the
design. 6. Specify procedures for each quantitative and qualitative data
collection and analysis stage.7. Specify expected products or outcomes of each quantitative and
qualitative data collection and analysis procedure.8. Use concise language.9. Make your visual diagram simple.10. Size your visual diagram to one page.
Source: Ivankova, N. et al. (2006)
39
Example ofA Diagram
Complex Mixed MethodsEvaluation Designs (Nastasi, Hitchcock,Sarkar, Burkholder,Varjas & Jayasena, 2007)
Study of mental healthinterventions for youthin Sri Lanka
Concurrent,Sequential,Recursive Qual/QuanPhases(Nastasi et al.,2007)
42
Reconceptualizing Designs
Three ways to look at mixed methods designs:
Typological approach (Bryman, 1988; Creswell & Plano Clark, 2007)
Systems theory (Maxwell & Loomis, 2003)
Synergistic approach - Hall & Howard (in press), “A Synergistic Approach: Conducting Mixed Methods Research with Typological and Systemic Design Considerations”
Combined effect greater than separate parts Position of equal value of qual and quan Ideology of difference – protect paradigm differences Researcher as collaborator and relationship with design – skilled in
both quan and qual or part of interdisciplinary team
43
The Practice of Conducting A Mixed Methods Study
Cordon & Hirst (in press) “Implementing a mixed methods approach to explore the financial implications of death of a life partner” – Social Policy Researchers, UK
Why they chose mixed methods Reasons for their choice of design Sampling decisions Concept map Discussions among team members
44
Technique - Designing a Quantitative Survey Based on Qualitative Findings
QUAL data analysis
Quotes
Codes
Themes
Quan data analysisinstrument development
Items on a survey
Variables on a survey
Scales on a survey
45
CountAdj.Count** Row PctColumn Pct
PatientsN=2
PhysiciansN=4
Medical AssistantsN=4
Themes
FamiliarityWith the Form
136.541.95.8
174.2527.43.0
194.7530.75.3
4915.5100.0
Reactions to the Form
2311.522.210.2
10025.048.317.4
6115.329.516.9
18451.75100.0
Use for Managing Depression
6733.538.629.8
17744.2551.030.7
369.010.410.0
28086.75100.0
Changes to the Form
11557.537.551.1
19649.032.034.0
18746.830.551.7
498153.3100.0
Situational Use of the Form
73.58.93.1
8621.554.414.9
5814.536.716.1
15139.5100.0
225112.5100.0
576144.0100.0
36190.3100.0
Technique- Matrix with Quantitative and Qualitative Data
46
Technique –Raising and Addressing Methodological Issues in Designs
Creswell, Plano Clark, Garrett (in press) Contradictory findings Data merging Sample selection Sample size Introducing bias Time
47
Technique – Unusual Blends of Data Collection
Qualitative Post-modern philosophy
and Longitudinal data and Qualitative data and Qualitative themes
matched
Qualitative discourse data
Quantitative Quantitative data
Longitudinal data Secondary survey data With items on survey to
produce new quan variables
Survey data
48
Other Developments:
International emphases Discipline differences Research team practices The emergence of bilingual nomenclature
(will employ this thinking in the workshop this afternoon as we “title” projects)
49
Four Developments (last 20 years)
Increasing interest in and advocacy for mixed methods Evolving understanding of what is mixed methods research Emerging philosophical perspectives as a foundation for mixed
methods Developing designs and innovative techniques
•Bryman, A. (1988). Quantity and quality in social research. London: Routledge.•Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research 6(1), 97-113.•Cordon, A., & Hirst, M. (in press). Implementing a mixed methods approach to explore the financial implications of death of a life partner. Journal of Mixed Methods Research. •Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.•Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (in press). Methodological issues in conducting mixed methods research designs. In M. Bergman (Ed.), Advances in mixed methods research. London: Sage.•Crotty M. (1988) The foundations of social research: Meaning and perspective in the research process. London: Sage.•Freshwater, D. (2007). Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research, 1(2), 134-146.•Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3), 195-2003.•Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.•Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.•Hall, B., & Howard, K. (in press). A synergistic approach: Conducting mixed methods research with typological and systemic design considerations. Journal of Mixed Methods Research.•Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive frameworks. Contributed paper for the Mixed Methods Conference, Cambridge, University, July 10-12, 2006.•Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10, 42-61.•Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20.•Leahey, E. (2007). Convergence and confidentiality? Limits to the implementation of mixed methodology . Social Science Research, 36, 149-158•Maxwell, J., & Loomis, D. (2003). Mixed methods design: An alternative approach. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 241-271). Thousand Oaks, CA: Sage.
References
•Mertens, D. M. (2003). Mixed methods and the politics of human research: The transformative-emancipatory perspective. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 135–164). Thousand Oaks, CA: Sage.•Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.•Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120–123.•Nastasi, B. K., Hitchcock, J., Sarkar, S., Burkholder, G., Varjas, K., & Jayasena, A. (2007). Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1(2), 164-182.•National Institutes of Health (2004). NIH Summer Institute: The design and conduct of qualitative and mixed-method research in social work and other health professions, August 4-8, 2004.•National Institutes of Health. Office of Behavioral and Social Sciences Research (1999). Qualitative methods in health research: Opportunities and considerations in application and review. Washington D.C.: NIH.•National Science Foundation (2003). Workshop on scientific foundations of qualitative research. Washington D.C.: NSF.•O’Cathain, A., Murphy, E., & Nicholl, J. (2007). Integration and publications as indicators of “yield” from mixed methods studies. Journal of Mixed Methods Research 1(2), 147-153.•Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.•Sale, J. E. M., Lohfeld, L. H., Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36, 43-53.•Smith, J. K., & Heshusius, L. (1986). Closing down the conversation: The end of the quantitative-qualitative debate among educational inquirers. Educational Researcher, 15(1), 4-12.•Tashakkori, A., & Teddlie, C. (Eds.). (2003a). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.
References (Cont’d)
52
Mixed Methods Research: State of the Art
(What Has Developed In Mixed Methods)
by John W. Creswell, Ph.D.
Department of Educational Psychology, University of Nebraska-LincolnCo-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research
© Please do not duplicate or use these slides without the express permission of the author.