Summer School 2014

Integrated Mixed-Methods Research Including QCAWendy Olsen and Steph Thomson

Reader in Socio-Economic Research; Research Associate in Education

wendy.olsen@manchester.ac.uk; stephanie.thomson@manchester.ac.uk

1

Systematic Mixed Methods Research session (by Wendy Olsen)Closely related to a Masters level 15-

credit course taught in the University of Manchester

2

Welcome to Systematic Mixed Methods Research Systematic Mixed Methods research has

advantages over traditional frequentist statistics and ALSO over qual+qual mixed methods.

I will: Define SMMR; Note that SMMR acknowledges social structure; Raise some ontological issues; Suggest some good software.

3

A ‘systems’ approach?

Defining SMMR requires definitions of open systems and closed systems an OPEN SYSTEM has permeability, organic

capacity to change itself, and/or multiple causes a CLOSED SYSTEM (e.g. a set of equations) has

routinised workings, has parameters The study of systems requires that we realise

that social systems are open systems. This creates a rationale for trans-disciplinarity.

It is also sociological. But water and farming are also open systems! (Lemon)

4

SMMR is the study of open systems using a mixture of methods of social research which recognise patterns but also acknowledging the tendency of patterns to change at different rates.

The rates of change and the nature of change depend on what factors are causing change and what can cause new changes.

See Sayer 1992 on durable structures See Fay 1987 on embodied habits and rates

of change

5

Retroduction – how to ask ‘why’ See Downward, Mearman, Dow on mixing

methods by doing retroduction = asking why....

Ask why the data look the way they do. Ask why the culture accepts as ‘normal’ what

is thought to be normal Ask why something has happened which

seems deviant; to some agent, this is ethical or right action?

6

Read about Retroduction, Causal Analysis, Causality, and Causes Olsen, 2012, Key Concepts in Data

Collection. Section on retroduction from Danermark, et

al., 2009. Crotty’s book offers an overview of different

philosophies of science. It reflects the current consensus about realism. Non-consensus area: critical realism. Non-consensus area: phenomenology.

7

Traditional Approaches to Mixed Methods 1. Triangulation 2. Pluralism and especially methodological

pluralism 3. Integrated mixed methods vs. sequenced

mixed methods 4. The schism approach

8

1. Triangulation Creative Commons!You may cite these slides as Olsen, W. (2014) mimeo on Mixed Methods Basics, University of Manchester, Methods@Manchester, Manchester UK. Data triangulation Methods triangulation

Methods not the same as methodology Runs into deep problem if schism exists If schism is accepted, but does not exist, then a

schism is created by methods triangulation! Perpetuates the schism concept

Methodological triangulation Theory triangulation

9

Examples

HIV studies in third world contexts epidemiology WITH focus groups

Education studies Classroom visits and focus group

Pluralism in economics is theory triangulation.

10

2. Pluralism

Pluralism of points of view is sometimes thought of as relativism. Be careful: viewpoints make the world seem relatively different, but this can be deceptive.

Pluralism of standpoints is real. Read about standpoints in feminist literature, or Longino.

Methodological pluralism – Roth 1987 argues for a democratic forum for

debate; this will encourage various types of evidence as well as argument

11

Examples

The study of discourse AND alongside that, the study of how the world actually works. discourse is the set of rules and norms that

generally structure how we communicate in a locality or specific organisation/time/group

how the world works depends on the dominant discourse as well as creative and exceptional agency, and on extra-discursive elements.

The best Foucaultians do this.

12

3. Integrated mixed methods

The best methods authors argue that you can integrate your methodological assumptions to make the whole project coherent.

This does not imply a deductivist approach and doing hypothesis testing.

It also does not excuse a pretence of a purely inductivist approach.

Deduction is a mode of logic for smaller parcels of work. Induction is a mode of logic for small moments on insight or

description. Retroduction is a mode of logic too.

13

3. cont’d Sequencing

Qual + quant Quant then qual Qual and another Qual Quant and another Quant

Exploratory vs. confirmatory statistics offers an interesting contrast: This literature introduces the schism concept within statistics! (surprisingly incoherent)

Exploratory then confirmatory stats is a sequenced mixed methods approach

14

You need a warranted argument not just a series of facts. Is sequencing enough? my view about sequencing is that it makes good

science for a person to be a mixed-methods person, and for larger projects to combine methods

however the sequencing argument and pragmatism do not resolve the ontological issues about whether something can exist even if someone denies that it exists

it is important to realise that some respondents can be wrong, too

I would question empirical data, not take it as fact

15

4. The Schism Concept

Debate about the schism in social science Anglo-saxon empiricism vs. french social theory Positivist vs. post-modernism, or

empiricist hypothetico-deductive methods vs. post-structuralism

The data types accepted as interesting by one side are not considered ‘epistemically coherent’ by the other side, hence a schism of epistemology.

The analysis method of the one side is not acceptable to the other, either.

16

Schisms Across Scientific Disciplines Exper. Psychology vs. Anthropology Neoclassical Economics vs. Political Econ.

And within the disciplines: Qualitative Education Research vs. Quantitative Education

Research The methods debate within sociology! 3 poles? Management schools

example of what is the lowest acceptable response rate

17

an illustration about social class

A Structure Example: CLASSAuthors Type of

OntologyAssumes Class is...

Statistical Patterns are Seen As . . .

Giddens, Third Way

Descriptive

Holistic

In a duality with agency

To be described

Realists, e.g. Danermark, Olsen

Depth Ontology

Complex

Open Sys.

Dynamic relationships

To be used in arguments

Social Mobility Studies & Wage Studies

Atomistic An inst’n that mediates with markets

To be described

Foucault Interpretive Biopower Too general

18

A Structure Example: CLASSAuthors KNOWLEDGE is usually construed as: Statistical Patterns

are Seen As . . .

Giddens, Third Way

Factual To be described

Realists, e.g. Danermark, Olsen

Transformative

Progressive, empowering

To be used in arguments

Social Mobility Studies

Factual To be described

Foucault Transformative, progressive, in-depth, political

Too general

19

Your Tasks as a Mixed-Methods Expert:

You must resolve the tensions:

a) between atomistic and holistic assumptions

-- that’s the ontological bit

b) and between factual versus empowering discourses

or assumptions about knowledge – that’s the epistemological bit I advise a critical stance But this can still be a scientific stance with recourse to empirical data and to

experience For the class example, I advise Crompton and Brockmann as excellent mixed-

methods authors who take a critical social science stance.

20

A Simple Example The logic of interpreting some non-parametric

bivariate tests. a. consider the findings fallible. b. condition the findings on the data and where

they came from. c. if not a random sample then don’t claim that

the inference to population is ‘true’. d. if the sample is just of one type of case, pay

attention to the macro and meso environments! e. consider the ethics of deriving findings from

this. HOW WILL YOU WORD YOUR FINDINGS!

21

A Study Task

Pick a very scientific subject, such as: medical tests of dementia subjects’ cognitive skills or how likely is HIV to cause death and what are the

pathways of causality? what tests are best advised for breast lumps? in order to

avoid later death from breast cancer.

Do a quick literature review using Web of Science. Be sure to include both Social Sciences and Sciences, and get 3 papers.

Decide whether mixed methods is advisable!

22

The Textbooks

Creswell, J. W. (2003). Research Design: Qualitative, Quantitative, and Mixed-Methods Approaches London, Sage.

Tashakkori A and Teddlie C. (1998) Mixed Methodology. Combining Qualitative and Quantitative Approaches, Thousand Oaks: Sage.

Bryman, A. (1996 (orig. 1988)). Quantity and Quality in Social Research. London, Routledge.

Also Useful Fisher, A. (1988). The Logic of Real Arguments. Cambridge, NY and Sydney,

Cambridge Univ. Press. Fisher, A. (2001). Critical Thinking: An Introduction. Cambridge, Cambridge

University Press. Flick, U. (1992). "Triangulation Revisited: Strategy of Validation or

Alternative?" Journal for the Theory of Social Behaviour 22(2): 169-197. Singer, P., Ed. (1994). Ethics. Oxford, Oxford Univ. Press. Vaughn, L. (2006). Writing Philosophy: A Student's Guide to Writing Philosophy

Essays. NY, Oxford, Oxford University Press. Weston, A. (2006). A Practical Companion to Ethics. Oxford, NY, Oxford

University Press.

23

Some Readings

Olsen, W.K. (2006), “Pluralism, Poverty and Sharecropping: Cultivating Open-Mindedness in Development Studies”, Journal of Development Studies, 42:7, pgs. 1130-1157.

Dow, JournEconMeth, 2004 “Structured Pluralism”. Vol. 11 no. 3.

Olsen, W.K., and J. Morgan (2005) “A Critical Epistemology Of Analytical Statistics”, Journal for the Theory of Social Behaviour, 35:3, 255-284.

Olsen, W.K. (2007), “Pluralist Methodology for Development Economics”, Journal of Economic Methodology, 14:1, 57-82, March. On ethical values embedded in measurement practices

24

Good Software for SMMR

NVIVO MaxQDA Excel SPSS Word STATA STAT-Transfer QCA (qualitative comparative analysis) fs-QCA (fuzzy set) TOSMANA (good for crisp sets and Multi-valued QCA)

25

Web Sites and Links

www.compasss.org

A Network of Small-N and Medium-N Comparative Researchers

www.jiscmail.ac.uk QUAL-COMPARE This is a JISC List with Archive and a File Storage area online

A web area and shared email list for those who want to get the workshop followup

Special workshop on 15 June 2010, see www.ccsr.ac.uk/events

26

Books by Mixed-Methods Authors Ragin, C. C. (2000). Fuzzy-Set Social Science.

Chicago ; London, University of Chicago Press. Byrne, D., (2002) Interpreting Quantitative Data. Sage. Dale A., Wathan, J. and Higgins, V (2008) “Secondary

Analysis of Quantitative Data Sources”, Ch. in Alasuutari, P. Bickman, L, Brannen, J. eds., Sage Handbook of Social Research Methods SAGE

De Vaus, D. A. (2001). Research Design in Social Research. London, Sage.

27