introduction to qualitative comparative analysis · s 28-03-2017 | 1 28-03-2017 | 1 introduction to...

1 | 28-03-2017

S

1 | 28-03-2017

Introduction to Qualitative Comparative Analysis (QCA)

Invited Presentation for Regioplan | Tuesday, March 28th 2017 | Location: Amsterdam School of Real Estate

2 | 28-03-2017

3 | 28-03-2017

The classical idea of evaluation

beleidseffect

effectniveau na beleid

effectniveau

voor beleid

effect verandering

effectvariable

t

effect met programma

effect zonder programma

VOOR

beleidsinitiatief

TIJDENS

beleidsinitiatief

NA

beleidsinitiatief

Source: Pattyn & Verweij (2014)

4 | 28-03-2017

What is the problem?

› Causality is more complex than “X Y”

Context is not stable but changes occur

Policies interact with other factors (e.g., other policies)

› Okay, in the classical approach we may be measuring the effect of a policy…

…but what are the mechanisms that are activated?

…and under which conditions does the policy work or not?

Source: Verweij & Gerrits (2013)

5 | 28-03-2017

How about case studies then?

Context Interactions between factors Mechanisms Conditions

6 | 28-03-2017

The problem with case studies

› When you study a case, you can show and take into account the complexity, but:

Everything seems to interact with everything… What are the necessary elements for the policy?

What are the relevant context factors for the policy? Which conditions really matter?

› How do we learn from a single case for another case? If every case is unique (cf. case study), how can lessons learned be transferred?

7 | 28-03-2017

Source: Verweij et al. (2013)

[C*I*M] + [~C*I*M] O [C and I and M] or [~C and I and M] O

The truth table is central in QCA

8 | 28-03-2017

Today’s objectives & program

› We will not aim to understand all the techniques and nuances of QCA

› We will focus on:

1) What is QCA, and when and why is using QCA a good idea?

2) How do we get to the truth table?

3) How do we analyze it?

4) How do we interpret the results?

9 | 28-03-2017

QCA: what, when and why?

› “QCA is both a research approach and a data analysis technique”…

› Qualitative comparative analysis as an approach is…

Case-based/oriented

Comparative

Set-theoretic

Source: Schneider & Wagemann (2012)

How do you conduct a QCA?

How do you design a QCA?

10 | 28-03-2017



Case-based/oriented

Comparative

Set-theoretic It strives to “gather in-

depth insight in the different cases and

capturing the complexity of the cases”

Source: Rihoux & Lobe (2009)

11 | 28-03-2017



Case-based/oriented

Comparative

Set-theoretic It strives to “gather in-

depth insight in the different cases and

capturing the complexity of the cases”

… and to “produce some level of generalization”


12 | 28-03-2017



Case-based/oriented

Comparative

Set-theoretic

Source: Ragin (1994)

Many

Qualitative

Research

Comparative

Aspects of Cases Research

Quantitative

Research

Few

Few Number of Cases Many

In QCA, aspects of cases are

called “conditions”

and understood as “sets”

13 | 28-03-2017


Quantitative

Typical method:

regression analysis

QCA

Qualitative

Typical method: single case study

Linear causality Complex causality Complex causality

or non-causal (‘interpretive’)

Variable-based Case-based and

comparative Case-based

Large-N of cases Medium-N of cases Small-N of cases

Pattern recognition,

generalization (small number of variables)

Between generality and complexity

(medium number of conditions)

High level of detail, in-depth case insights

(large number of ‘variables’)

Adapted from: Verweij & Gerrits (2013)

14 | 28-03-2017


Case-based/oriented

Comparative

Set-theoretic


Source: Rihoux et al. (2013)

Advise: study between 7-50 cases,

and between 3-5 conditions

15 | 28-03-2017


Source: Marx & Duşa (2011)

16 | 28-03-2017


Cases can be micro-level, meso-level, or

macro-level

17 | 28-03-2017


› A huge advantage of QCA is that it formalizes and systematizes case comparison; this is important!

› “The problem is that, when it comes to comparing more than, say, two or three cases, in many instances the comparison of the case study material is rather loose or not formalized – hence the scientificity of case studies is often questioned (…).”


18 | 28-03-2017


› Some authors are even more trenchant. Jonathan Aus stated:

› “There can be no doubt that ‘thick descriptions’, as for instance employed in anthropology, may contribute to a better understanding of human behavior in specific social contexts. Yet the interpretation of data gathered in a theoretical vacuum remains largely intuitive (…). Nevertheless, most case studies (…) could maliciously be qualified as a-theoretical ‘data dumps’.”

Source: Aus (2009)

19 | 28-03-2017


› There is another important reason why applying QCA is valuable: it is set-theoretic!


Case-based/oriented

Comparative

Set-theoretic

You can make causal statements

You can systematically unravel complex causality

20 | 28-03-2017


› QCA is a set-theoretic research approach, which makes it fundamentally different from regression-analytical methods

› Causal inference in regression, e.g.:

The more of X, the more of Y

The less of X, the less of Y

The less of X, the more of Y

› Causal inference in QCA, inter alia:

Only if X{1}, then Y{1} X{1} Y{1}

If X{1}, then Y{1} X{1} Y{1}

See for the full argument: Thiem, Baumgartner & Bol (2016)

Covariation vs.

implication

21 | 28-03-2017




If X{1}, then Y{1} X{1} Y{1}

› The first statement indicates a relationship of necessity: condition X has to be present for outcome Y to occur

In other words: Y implies X

Or: outcome Y is a subset of X

22 | 28-03-2017

23 | 28-03-2017




If X{1}, then Y{1} X{1} Y{1}

› The second statement indicates a relationship of sufficiency: condition X can produce the outcome Y by itself

In other words: X implies Y

Or: condition X is a subset of Y

24 | 28-03-2017

25 | 28-03-2017


› However, causality is often more complex than just simple sufficiency or necessity! Conjunctural causation: combinations of conditions (i.e.,

configurations) produce the outcome [Logical-AND]

Equifinality: multiple configurations can produce the outcome [Logical-OR]

Asymmetry: the presence of X for Y does not imply the absence of X (i.e. ~X) for the absence of Y (i.e. ~Y)

Source: Verweij et al. (2013)

26 | 28-03-2017

The truth table: how to get there?

Gaining theoretical and case knowledge

Case construction

Raw data matrix

Truth table

Patterns

Interpretation

Return to the cases/theory

where the QCA

techniques come in

Adapted from: Verweij (2015)

Case study research

QCA is an iterative process

27 | 28-03-2017


› Important to always keep in mind that, conceptually, in QCA, a case = a configuration of conditions plus the outcome

Constructing your cases:

1. Research question

2. Case selection

3. Defining the outcome

4. Selection of conditions

5. Gaining case knowledge

See: Rihoux & Lobe (2009) and Byrne & Ragin (2009)

Y

28 | 28-03-2017




Should fit the set-theoretic nature of QCA

Cf. Berg-Schlosser et al. (2009) in Rihoux & Ragin (2009)

29 | 28-03-2017


› A few examples of research questions from my own research

What combinations of network complexity,

stakeholder involvement, and

network management are necessary and/or sufficient for realizing

stakeholder satisfaction in governance networks?

What are the necessary and/or sufficient (combinations of)

conditions that explain the variation in

environmental justice policy adoption among

states in the US?

Is there a relation between lower bids by

contractors and the size of contract changes?

And what are the sizes of and reasons for contract changes in

transportation infrastructure projects?

30 | 28-03-2017




Should fit the set-theoretic nature of QCA

QCA can be used for multiple purposes, inter alia:

Testing hypotheses or theories

Exploring patterns in data

Summarizing data

Building new theories


31 | 28-03-2017


2. Case selection

Cases can be micro, meso, macro

Cases should be comparable (i.e., defined by the same ‘scope conditions’), but have variation on the conditions

You are allowed to be flexible: it is OK to drop/add cases (but justify)


32 | 28-03-2017



3. Defining the outcome

You must have a clear definition of the outcome you want to explain across cases, prior to starting the analysis

You can only study one outcome per analysis

Include cases with Y and ~Y


33 | 28-03-2017



4. Selection of conditions

Study the same conditions for all the cases but, ideally, cases show variation in terms of their scores (0 or 1) on the conditions

If theory is available, formulate expectations about the effect of conditions on Y or ~Y

Justify your selection of conditions

The n of conditions is kept low


34 | 28-03-2017

› 2 conditions (sets)

22 = 4 possible combinations of conditions (i.e., configurations/truth table rows)

› 3 conditions (sets)

23 = 8 possible combinations of conditions (i.e., configurations/truth table rows)

› 24 = 16

› 32, 64, 128…


Try to have a good balance between n of

cases and n of conditions

Rule of Thumb: 2k

0 0

0 1

1 0

1 1

0 0 0

0 0 1

0 1 0

0 1 1

1 0 0

1 0 1

1 1 0

1 1 1

35 | 28-03-2017




You can use a variety of data sources, both qualitative and quantitative


36 | 28-03-2017




You can use a variety of data sources, both qualitative and quantitative

Trade-off between number of cases and the in-depth knowledge you can have of the cases


37 | 28-03-2017


Gaining theoretical and case knowledge

Case construction

Raw data matrix

Truth table

Patterns

Interpretation

Return to the cases/theory

where the QCA

techniques come in

Adapted from: Verweij (2015)

Case study research

QCA is an iterative process

38 | 28-03-2017


› Once cases are constructed and selected

Data collection

Qualitative coding

Calibrating the coded data

Constructing the data-matrix

› Calibration is “the process of using empirical information on cases for assigning set-membership to them (…).”

Source: Schneider & Wagemann (2012)

39 | 28-03-2017


› Cases have membership in sets (recall: conditions and outcomes = sets)

› There are three main important ‘anchor points’ in calibration:

0.0 full non-membership, out of the set

0.5 ambiguity, cross-over point

1.0 full membership, fully in the set

› Basically, the result of calibration is the grouping of the similar cases and the separating of different cases, per condition

40 | 28-03-2017


› Calibration is “the process of using empirical information on cases for assigning set-membership to them (…).”

An example of

calibration:

Democratic control

in USA state

legislatures

Source: Kim & Verweij (2016)

41 | 28-03-2017


Table: Ragin (2008)

Differences in degree

Differences in degree

Difference in kind

42 | 28-03-2017


Another example of calibration

Source: Verweij (2015)

43 | 28-03-2017


Another example of calibration: contracts, project scope, and project size


Crisp-set calibration (direct) D&C 0.00 DBFM 1.00

Fuzzy set four-value calibration (direct) Renewal 0.00 New build 0.33 Complex 0.67 Integral 1.00

Fuzzy set four-value calibration (cluster analysis)

44 | 28-03-2017


Another example of calibration: management and cooperation


45 | 28-03-2017


A final example of calibration: satisfaction


46 | 28-03-2017

47 | 28-03-2017


› The result of your calibration is a data matrix

› In the case of crisp-set QCA, it only features case memberships of 0 or 1, as shown here

› Each row is a case

48 | 28-03-2017


› In the data matrix, each row is a case

› In the truth table, you group similar cases as combinations of conditions (sets) each row

is now a configuration (combination of sets)

49 | 28-03-2017

Analyzing the truth table

› Size of the truth table is determined by the number of conditions: 2k, where k is number of conditions

› Each truth table row is a statement of sufficiency

Adapted from: Verweij et al. (2013)

Recall: complex causality! C I M Outcome Y Cases

1 1 1 1 ZUID, NOORD, IJSS, SIJT, WEST

1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

Asymmetry

Equifinality

Configurational

50 | 28-03-2017


› First of all: examine the truth table and deal with problems occurring in it

Limited diversity

Contradictory rows


Logical Contradictions

C I M Outcome Y Cases


1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

Limited Diversity

51 | 28-03-2017


› Why are limited diversity and contradictory rows a problem?

› Because: the rule for truth table minimization is:

Pairwise compare configurations that agree on the outcome and differ in but one condition




1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

52 | 28-03-2017


› Dealing with limited diversity

Add cases

Drop conditions

Counterfactual analysis

Recalibration




1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

Limited Diversity

53 | 28-03-2017


› Dealing with contradictory rows

Drop cases

Drop configurations

Add conditions

Recalibration





1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

54 | 28-03-2017


› Notice the interplay between dealing with limited diversity and contradictions

Add cases Drop cases

Drop conditions Add conditions

Counterfactual analysis Drop configurations

Recalibration





1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

Limited Diversity

55 | 28-03-2017


› After the problems have been dealt with, the truth table can really be minimized

› The rule for truth table minimization is:

Pairwise compare configurations that agree on the outcome and differ in but one condition




1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

56 | 28-03-2017


› The rule for truth table minimization is:

Pairwise compare cases that agree on the outcome and differ in but one condition

› This example: analysis for the presence of Y

Note that truth table

analysis is normally

done with software

(see compasss.org)




1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

http://www.compasss.org/software.htm

57 | 28-03-2017


› First, rewrite the rows in a statement of sufficiency; second, pairwise compare

› C*I*M + C*~I*M + ~C*I*M + ~C*~I*~M Y

C*M + I*M + ~C*~I*~M Y




1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

Pairwise compare configurations that

agree on the outcome and differ in but one

condition

58 | 28-03-2017

Interpreting the results

› Our solution is: C*M + I*M + ~C*~I*~M Y

› Complex causality

Sufficient conditions: none Necessary conditions: none Sufficient configurations: three Necessary configurations: none INUS: all conditions

INUS = “Insufficient but Necessary parts of a

configuration which is itself Unnecessary but Sufficient”

Consistency of results: high, because no contradictory rows

are represented by the solution formula Empirical coverage of results is perfectly acceptable, as the

solution formula covers 9 out of 14 cases

59 | 28-03-2017

Interpreting the results

› How about lesson-drawing and policy transfer?

› For instance, looking at this table, what can be learned by comparing PERKT/DELFT with WIER?



1 1 0 C LENT, SCHEL

1 0 1 1 PERK, DELFT

1 0 0 0 WIER

0 1 1 1 BROEK

0 1 0 C WAAL, GOUW

0 0 1 ???

0 0 0 1 DIEF

60 | 28-03-2017

Small exercise

› Construct a truth table from this data matrix and subsequently minimize it

Source: Li et al. (2016)

61 | 28-03-2017

Small exercise (solution)

› Construct a truth table from this data matrix and subsequently minimize it

62 | 28-03-2017

Concluding remarks

› The results need to be interpreted also by going back to the cases and theory

› In that sense is the solution formula often the starting point for the selection and study of typical or deviant (a-typical) cases (see Schneider & Rohlfing, 2013)

› You don’t have to do the whole QCA-process, you can also use elements (e.g., summarizing data or exploring patterns by drafting a truth table)

› More questions? Feel free to contact me! › Publications available via www.stefanverweij.eu or

ResearchGate

http://www.stefanverweij.eu/

https://www.researchgate.net/profile/Stefan_Verweij

63 | 28-03-2017

References

› Aus, J.P. (2009). Conjunctural causation in comparative case-oriented research. Quality & Quantity, 43(2), 173-183.

› Berg-Schlosser, D., De Meur, G., Rihoux, B. & Ragin, C.C. (2009). Qualitative comparative analysis (QCA) as an approach. In B. Rihoux & C.C. Ragin (Eds.), Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques (pp. 1-18). London: Sage.

› Byrne, D.S. & Ragin, C.C. (2009). The Sage handbook of case-based methods. London: Sage.

› Kim, Y. & Verweij, S. (2016). Two effective causal paths that explain the adoption of US state environmental justice policy. Policy Sciences, 49(4), 119-139,

› Li, Y., Koppenjan, J.F.M. & Verweij, S. (2016). Governing environmental conflicts in China: Under what conditions do local governments compromise? Public Administration, 94(3), 806-822,

› Marx, A. & Duşa, A. (2011). Crisp-set qualitative comparative analysis (csQCA), contradictions and consistency benchmarks for model specification. Methodological Innovations Online, 6(2), 103-148.

› Pattyn, V. & Verweij, S. (2014). Beleidsevaluaties tussen methode en praktijk: Naar een meer realistische evaluatiebenadering. Burger, Bestuur & Beleid, 8(4), 260-267.

› Ragin, C.C. (1994). Constructing social research: The unity and diversity of method. Sage: New York.

› Ragin, C.C. (2008). Redesigning social inquiry: Fuzzy sets and beyond. Chicago: University of Chicago Press.

› Rihoux, B., Álamos-Concha, P., Bol, D., Marx, A. & Rezsöhazy, I. (2013). From niche to mainstream method? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Political Research Quarterly, 66(1), 175-184.

› Rihoux, B. & Lobe, B. (2009). The case for qualitative comparative analysis (QCA): Adding leverage for thick cross-case comparison. In D.S. Byrne & C.C. Ragin (Eds.), The Sage handbook of case-based methods (pp. 222-242). London: Sage.

› Rihoux, B. & Ragin, C.C. (Eds.). (2009). Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques. London: Sage.

› Schneider, C.Q. & Rohlfing, I. (2013). Combining QCA and process tracing in set-theoretic multi-method research. Sociological Methods & Research, 42(4), 559-597.

› Schneider, C.Q. & Wagemann, C. (2012). Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis. Cambridge: Cambridge University Press.

› Thiem, A., Baumgartner, M. & Bol, D. (2016). Still lost in translation! A correction of three misunderstandings between configurational comparativists and regressional analysts. Comparative Political Studies, 49(6), 742-774.

› Verweij, S. (2015). Once the shovel hits the ground: Evaluating the management of complex implementation processes of public-private partnership infrastructure projects with qualitative comparative analysis. Rotterdam: Erasmus University Rotterdam.

› Verweij, S. & Gerrits, L.M. (2013). Understanding and researching complexity with qualitative comparative analysis: Evaluating transportation infrastructure projects. Evaluation, 19(1), 40-55.

› Verweij, S., Klijn, E.H., Edelenbos, J. & Van Buuren, M.W. (2013). What makes governance networks work? A fuzzy set qualitative comparative analysis of 14 Dutch spatial planning projects. Public Administration, 91(4), 1035-1055.

introduction to qualitative comparative analysis · s 28-03-2017 | 1 28-03-2017 | 1 introduction to...

Documents