why do we need mixed- methods? should we differentiate integration versus mixed-methods? alan...
TRANSCRIPT
Why do we need mixed-methods? Should we
differentiate integrationversus mixed-methods?
Alan Bryman,
Management Centre,
University of Leicester, UK.
The current situation
• Exciting times
• ‘Paradigm wars’
• Epistemological arguments
• Pragmatism
• Washing machines and questionnaires!
• Not addressing epistemological issues today
My Research on Multi-Strategy Research
• Funded by Economic & Social Research Council – Research Methods Programme
• 2 main strands today:1. Traditional content analysis of journal articles
using mixed-methods research
2. Interviews with social researchers who’ve used it
Research methods
• Wanted to map general characteristics of mixed-methods research – content analysis
• Wanted perspectives of mixed-methods researchers + contingencies involved – semi-structured interviews
Content analysis
• Searched Social Sciences Citation Index for: quantitative and qualitative; triangulation; multi(-)method; mixed method in titles and abstracts of English language articles
• 5 disciplines: sociology; social psychology; organizational behaviour; human and cultural geography; media and cultural studies
• 1994-2003 232 articles
Content analysis
• Articles foregrounded mixed-methods
• Emphasis on mixed-methods in terms of data collection and analysis
• Major focus on rationales for mixed-methods research
Classifying Forms of Mixed-Methods Research
• Distinction between rationale and practice
• Rationale = stated purpose(s) of integrating quantitative and qualitative research
• Practice = actual use(s) made of integrating quantitative and qualitative research
• Used both Greene et al. scheme and my own grounded scheme
Classifying Forms of Mixed-Methods Research
• Greene et al. (triangulation; complementarity; development; initiation; expansion)
• Parsimonious but only 2 rationales coded (primary & secondary)
• Developed alternative scheme
Alternative Scheme
• Triangulation
• Offset
• Completeness
• Process
• Different research questions
• Explanation
• Unexpected results
• Instrument development
• Sampling
• Credibility
• Context
• Illustration
• Utility
• Confirm & discover
• Diversity of views
• Enhancement
• Other/unclear/not stated
Highlights of Findings: Rationale
• No rationale in 27% of all articles
• Main categories in terms of rationale: Enhancement 32% Completeness 13% Sampling 13% Triangulation 13%
Highlights of Findings: Practice
• Main categories in terms of practice: Enhancement 52% (rationale 32%) Triangulation 35% (rationale 13%) Completeness 29% (rationale 13%) Illustration 23% (rationale 2%) Sampling 19% (rationale 13%)
Highlights of Findings: Rationale and Practice
• Rationale and practice not always in lineRationale often not reflected in how multi-
strategy research actually usedPractice often doesn’t chime with rationales
given
• Examples from contingency table analysis
Highlights of Findings: Rationale and Practice –
The Case of Triangulation• Of the 29 articles citing triangulation as a
rationale, 19 used it that way, i.e. one-third of articles citing triangulation as rationale didn’t use multi-strategy research that way or didn’t report doing so. Other prominent uses of articles citing triangulation were: enhance (13); completeness (10); and illustration (8).
Highlights of Findings: Rationale and Practice –
The Case of Triangulation• Other way around
• 80 articles used a triangulation approach but only 19 of them gave it as a rationale, i.e. three-quarters of articles using triangulation didn’t cite it as a rationale
• Suggests triangulation hard to resist when opportunity arises
Highlights of Findings: Rationale and Practice –
The Case of Completeness
• Completeness was a rationale for 31 articles and 84% of them used it that way
• But when practice is examined, 61% of all articles using a completeness approach didn’t specify it as a rationale
Highlights of Findings: Rationale and Practice –
The Case of Enhancement
• 73 articles specified enhance as rationale, a quarter of them didn’t use multi-strategy research this way
• 121 articles used multi-strategy research this way, but over half of them hadn’t specified it as a rationale
• Several other examples of mismatches
Highlights of Findings: Rationale and Practice
• Often mismatch between rationale and practice
• Mixed-methods research a moveable feast
Minority Strategy:The Gatling Gun Approach
Gatling Gun Strategy
4 or more rationales:
• 6 articles in terms of rationale
• 33 articles in terms of practice
Themes from Semi-Structured Interviews
• Similar to content analysis• Mixed-methods research increasingly expected
Concern for many Research questions important Particularistic versus universalistic discourses Not due to confusion – lack of guidelines about mixed-
methods issues; textbook account too simple; ambivalence about role of research questions; lack of prescription
Uncertainty
What do we mean by mixing, etc.?
• Mixing vs. integration
• Use of verbs
• What does bringing together of quantitative and qualitative research entail?
• Mixed-methods research or multi-methodology/multi-method research?
Is Integration Occurring?Content analysis findings
• Content analysis findings
• Genuine integration – 18% of articles
• Parallel presentation – 47% of articles
• Looked for evidence of findings being brought together comprehensive picture interweaving both
Is Integration Occurring?Semi-structured interviews
Most expressed concern. Main themes:• Different audiences• Greater faith in one; also familiarity• Design issues• Time-lines differ• Skill specialisms• One more striking or interesting• Objectivist vs. constructionist accounts• Journal publication issues
Bryman Goes Reflexive
• Mine was a mixed-methods project
• Justified using both content analysis and qualitative interviewing quite well
• Outcomes consistent with rationales
• Mixed-methods research linked to my research questions
• Integration of data not adequately achieved
Back to the Title
Why do we need mixed-methods?
• We often don’t need it
• But difficult to decide when we do
• Good deal of uncertainty about when to use a mixed-methods approach
Should we differentiate integration vs. mixed-methods?
• Don’t get preoccupied with the verbs
• Interviewees saw problem
• Mixing findings like a conversation
• Forging an overall account
• Lack of templates
Should we differentiate integration vs. mixed-methods?
• Tended to emphasize doing mixed-methods research
• Need more attention to representation of mixed-methods findings in publications
• Quality criteria