gelo, braakmann & benetka (2008) beyond the debate of quantitative-qualitative

COMMENTARY

Quantitative and Qualitative Research: Beyondthe Debate

Omar Gelo & Diana Braakmann & Gerhard Benetka

Published online: 16 September 2008# Springer Science + Business Media, LLC 2008

Abstract Psychology has been a highly quantitative field since its conception as ascience. However, a qualitative approach to psychological research has gainedincreasing importance in the last decades, and an enduring debate betweenquantitative and qualitative approaches has arisen. The recently developed MixedMethods Research (MMR) addresses this debate by aiming to integrate quantitativeand qualitative approaches. This article outlines and discusses quantitative,qualitative and mixed methods research approaches with specific reference to their(1) philosophical foundations (i.e. basic sets of beliefs that ground inquiry), (2)methodological assumptions (i.e. principles and formal conditions which guidescientific investigation), and (3) research methods (i.e. concrete procedures for datacollection, analysis and interpretation). We conclude that MMR may reasonablyovercome the limitation of purely quantitative and purely qualitative approaches ateach of these levels, providing a fruitful context for a more comprehensivepsychological research.

Keywords Research methods . Quantitative . Qualitative . Mixed methods research

Psychological research has relied heavily on experimental and correlationaltechniques to test theory using quantitative data. This is because psychology, likeother behavioural disciplines, has been dominated by a positivist/post-positivistparadigm. However, criticism toward this way of conducting research during the pastfew decades has emerged. While qualitative research approaches (e.g., Silverman2004) have been developed starting from completely different philosophicalassumptions, such as phenomenology and hermeneutics, some quantitative

Integr Psych Behav (2008) 42:266–290DOI 10.1007/s12124-008-9078-3

O. Gelo (*) :D. BraakmannDepartment of Psychotherapeutic Sciences, Sigmund Freud University—Vienna, Schnirchgasse 9a,1030 Vienna, Austriae-mail: [email protected]

G. BenetkaDepartment of Psychology, Sigmund Freud University—Vienna, Schnirchgasse 9a,1030 Vienna, Austria

researchers (e.g. Michell 1999, 2000; Toomela 2008) have become self-critical abouttheir own research approach. For example, Michell (1999) provided a criticalhistorical overview of the concept of measurement in psychology, identifying twomain issues: (1) most quantitative research is based upon the fact that psychologicalattributes can be measured in a quantitative way rather than upon empiricalinvestigation of the issue; (2) most quantitative researchers adopt a defectivedefinition of measurement, thinking that measurement is simply the assignment ofnumbers to objects and events according to specific rules. In a similar way, Toomela(2008) recently showed how (1) quantitative variables may encode informationambiguously, and how (2) statistical analysis may not always allow a meaningfultheoretical interpretation, because of ambiguity of information encoded in variables,and because of intrinsic limitation of statistical procedures.

According to these authors, there is a fundamental issue which has been oftenignored within quantitative research: the issue of the ontology and epistemology ofvariables (Michell 1999; Toomela 2008). Hence the basic concern is whatinformation is encoded in quantitative variables supposed to represent mentalphenomena (ontology of a variable), and how this kind of information may enlightenus about the relationship between these mental phenomena (epistemology of avariable). Toomela (2008) concludes that without a clear understanding of whatinformation is encoded in a variable, it is not possible to meaningfully interpretevents and their relationship on the basis of any statistical analyses.

Is this a “no-way-out” situation? Do we have to abandon quantitative researchapproaches? We do not think so. Do we have then to improve and refine the existingquantitative methodologies? We think this would be a more favourable solution.However, we believe that a change of perspective is needed, which should primarilyinvolve the way research is conceived. Qualitative research approaches could be aninteresting solution in this regard. Nonetheless, we claim that there is an even moreappropriate alternative, which consists of integrating quantitative and qualitativeresearch approaches.

In the present paper we will in the first place introduce the current debate betweenquantitative and qualitative research approaches. Then we will make a step back andreview respectively quantitative and qualitative research approaches in terms of theirspecific paradigmatic postulates, methodological assumptions and research methods.Thereupon we will describe the Mixed Method Research, a relatively recentapproach which combines and integrates qualitative and qualitative research atdifferent levels. Our aim is to show how such an approach may overcome thelimitations of purely quantitative or qualitative approaches, providing a fruitfulcontext for a more comprehensive psychological research.

The Debate Between Quantitative and Qualitative Research

To study human beings, psychologists have commonly followed either a quantitative orqualitative approach. From an etymological point of view, the former implicatesdetermining how much of an entity there is, while the latter is involved in describing theconstituent properties of an entity. Indeed, much psychological research reflects theessence of this distinction. A great deal of quantitative research is concerned with

Integr Psych Behav (2008) 42:266–290 267

counting occurrences, volumes, or the size of the associations between entities, whilequalitative research aims to provide rich or “thick” (Geertzt 1973) descriptive accountsof the phenomenon under investigation.

Quantitative and qualitative research approaches clearly differ in terms of howdata are collected and analyzed. Quantitative research requires the reduction ofphenomena to numerical values in order to carry out statistical analysis. By contrast,qualitative research involves collection of data in a non-numerical form, i.e. texts,pictures, videos, etc. However, quantitative and qualitative approaches also differ—particularly—in regard to the aims of scientific investigation as well as theunderlying paradigms and meta-theoretical assumptions. According to quantitativeapproaches, psychological and social phenomena have an objective reality. Therelationships between these phenomena are investigated in terms of generalizablecausal effects, which in turn allow prediction. By contrast, qualitative approachesconsider reality as socially and psychologically constructed. The aim of scientificinvestigation is to understand the behaviour and the culture of humans and theirgroups “from the point of view of those being studied” (Bryman 1988, p. 46). Anattempt is usually made to understand a small number of participants’ own frames ofreference or worldviews, rather than trying to test hypotheses on a large sample.

Quantitative approaches have always dominated mainstream psychologicalresearch. Since the conception of psychology as a “science” in the nineteenthcentury, quantitative approaches have prevailed. As stated by Danziger (1985) inanalogy to Kant’s categorical imperative, they have become the “methodologicalimperative”. However, since the 1960s various psychologists, especially thosedealing with social phenomena, have begun to criticize such an approach to theinvestigation of the human nature. They have proposed a naturalistic, contextual-based and holistic understanding of the human being, which has come to be knownas the qualitative approach. Since this approach has gained ground withinpsychology (see e.g. Smith 2003), it sparked a debate about the appropriateness ofeither quantitative or qualitative approaches in psychological research (Patton 1988).Those two diverse approaches could “just” be viable options; instead, they havebecome rather entrenched ideological positions (Todd et al. 2004).

The Quantitative–Qualitative Debate (QQD) has been sustained by several factorswhich can be mainly ascribed to the underlying philosophical and methodologicalassumptions and the related research methods (Bryman 1984; Krantz 1995). Someauthors emphasize the incompatibility of quantitative and qualitative approaches.Their basic argument is that the meta-theoretical paradigms underlying the twoapproaches are so different that any reconciliation between them would destroy thephilosophical foundations of each (Lincoln and Guba 1985; Noblitt and Hare 1988;Rosenberg 1988). As noted by Bryman (1984), the QQD is based to a large extenton epistemological issues, and questions relating to research techniques aresystematically related to these issues. Some other authors, though, assume a morepragmatic position. According to them, it is both possible to subscribe to thephilosophy of one approach and employ the methods of the other (Reichardt andCook 1979; Steckler et al. 1992). Recently, the so called Mixed Methods Research(i.e. Tashakkori and Teddlie 2003b) has been developed, which aims to combine andto some extent integrate different methodological and research method perspectivesof both quantitative and qualitative approaches. Following these emergent trends, the

268 Integr Psych Behav (2008) 42:266–290

current QQD can be re-defined with reference to both a methodologically integratedand an empirically grounded, practice-oriented set of investigations. In this way,controversial philosophical issues may be seemingly bypassed (Krantz 1995) orcombined, and discussions take place at the point of which research strategy is morelikely to investigate specific phenomena.

Quantitative and Qualitative Research

Scientific investigation can be characterized by a set of philosophical and meta-theoretical assumptions concerning the nature of reality (ontology), knowledge(epistemology), and the principles inspiring and governing scientific investigation(methodology), as well as by technical issues regarding the practical implementationof a study (research methods). The latter can be considered as deriving from theformer, i.e. the choice of a particular philosophical position and methodology leadsto a preference for a particular research method on the grounds of its appropriatenesswithin that specific philosophical and methodological orientation. While philosoph-ical and meta-theoretical assumptions underlie the worldviews constraining the kindsof questions we try to answer, and the principles governing our research approach,research methods specify the practical implementation of our scientific investigationin terms of data collection, analysis and interpretation.

The main features characterizing quantitative and qualitative approaches may bedescribed with respective reference to their philosophical foundations, methodolog-ical assumptions, and to the research methods they employ. Differences at each ofthese levels have contributed to sustain the QQD.

Worldviews and Philosophical Foundations

All research needs a foundation for its inquiry, which is provided by worldviews andscientific paradigms. Worldviews imply how we view and, thus, think about researchand go about conducting it. Similarly, scientific paradigms contain a basic set ofbeliefs or assumptions that guide our inquiries (Guba and Lincoln 2005). Withreference to quantitative and qualitative research approaches, three main worldviewsmay be identified: objectivism (according to which reality exists independent fromconsciousness), subjectivism (according to which subjective experience is funda-mental to any knowledge process), and constructivism (according to whichknowledge is a construction resulting from the interaction between individuals andtheir social world)1.

The different worldviews and paradigms underlying quantitative and qualitativeapproaches are reflected in different conceptions about the nature of reality(ontology) and knowledge (epistemology). Quantitative paradigms see reality as

1 Objectivism is often associated with quantitative research approaches and has been articulated at a meta-theoretical and philosophical level in logical positivism and critical rationalism. On the contrary,subjectivism and constructivism are typically associated with qualitative investigation, and have beenexpressed at a meta-theoretical and philosophical level, among others, in phenomenology, hermeneuticsand symbolic interactionism.


single and tangible, where the knower and the known are considered as relativelyseparate and independent. Qualitative paradigms, however, view reality as amultiple, socially and psychologically constructed phenomenon, where the knowerand the known are inextricably connected to each other.

Modern social and psychological sciences developed at the end of the 19thcentury. At that time, the natural sciences were established and well known,accompanied by an enthusiastic faith in scientific progress. Social and psychologicalsciences thus imported the current scientific ideal of an axiomatic knowledge to beexpressed, at best, in a mathematical form, with great emphasis on measures, testsand experiments. This approach, which presupposes quantification, has itsfoundations—from the perspective of philosophy of science—in the logicalpositivism of the so-called ‘first Vienna circle’, and in the critical rationalism ofKarl Popper in the 1930s (Westermann 1987; see also Miller 1994).

Qualitative approaches to the study of the human being have developed since the19th century as an alternative to the dominant social and psychological research.These go back to the philosophical tradition of phenomenology, hermeneutics, andsymbolic interactionism, and reflect the emergent willingness to defend the integrityof human sciences as distinct from the natural sciences. Phenomenology (see Moran2000) deals with the study of mental phenomena as experienced from the first-person point of view (Smith 2003). Hermeneutics can be defined as a specificsystem or method for interpretation (Dilthey 1989), and involves cultivating theability to understand things from somebody else’s point of view. Finally, symbolicinteractionism (see Blumer 1969) claims that human beings act toward things on thebasis of attributed meanings, which are constructed within social interaction.

Methodological Assumptions

General Issues The worldviews and philosophical assumptions described above arereflected in different methodologies. Methodology is the study and logic of researchmethods, and refers to principles governing the research activity; it can be defined asa set of rules, principles and formal conditions which ground and guide scientificinquiry in order to organize and increase our knowledge about phenomena. Morespecifically, methodology establishes which kind of relationship exists between theresearcher’s observation, theory, hypothesis and research methods (see nextparagraph).

Quantitative and qualitative approaches present different methodologies which, asin the case of their paradigmatic foundations, have deeply contributed to maintainthe QQD (see Table 1). The former are usually described to adopt a nomotheticmethodology, while the latter adopt an idiographic methodology. This distinctionwas introduced by Windelband (see Lamiell 1998) in order to differentiate thescience of general laws that govern generality (nomothetic) from the science ofspecific events, which describe the particular, the unique, and the individual(idiographic). Nomothetic science (from the Greek nomos = law, and thesis =proposition) consists of the establishment, collection and assimilation of facts withthe exclusive aim of recognizing and formulating laws that are always and in everycircumstance immutable and universally applicable (tendency to generalize). Thischaracterizes the natural sciences. In contrast, idiographic science (from the Greek


idios = “own”, “private”, and graphein = “to write”, “to describe”) consists of therepresentation of an individual event of singular, temporally limited reality ascompletely as possible with the objective of recording, and comprehending it in itsfactuality (tendency to individualize). This is characteristic of historical and humansciences, which thus reveal their nature as sciences of specific events. It is importantto observe that the same objects of scientific investigation can be made subject ofeither nomothetic or idiographic investigation. There is a very close interplaybetween both approaches: each idiographic science with regard to its generalconcepts must refer back to nomothetic disciplines. On the other hand, every generallaw is based on the observation of many different individual cases. Therefore it issuggested to consider both methodologies as the extremes of a continuum.

The difference between these two approaches can be best outlined with referenceto the dichotomy between explanation and comprehension. Explanation representsthe establishment of connections between facts through regularities that we observe.Comprehension, by contrast, is the reconstruction of how someone else hasestablished connections between facts through regularities they observed (Köckeis-Stangl 1980). Quantitative approaches tend to explain, i.e. to verify if observed

Table 1 Attributes of quantitative and qualitative methodologies

Quantitative approaches Qualitative approaches

Nomothetic IdiographicExtensive IntensiveGeneralizing Individualizing

Explanation ComprehensionPrediction InterpretationGeneralization Contextualization

Deduction InductionTheory-driven Data-drivenHypotheses-testing Hypotheses-generatingVerification-oriented (confirmatory) Discovery-oriented (exploratory)

Experimental NaturalisticTrue-experiments Case-study (narrative)Quasi-experiments Discourse analysis

Conversation analysisNon-experimental Focus groupCorrelational Grounded theoryCorrelational–comparative EthnographicCorrelational–causal–comparativeEx-post-facto

Internal validity Internal validityStatistical conclusion validity Descriptive validity

Interpretative validityConstruct validity Explanatory validityCausal validity

Generalizability GeneralizabilityExternal validity Transferability


phenomena and their systematic relationship confirm the prediction made by atheory. Qualitative approaches, in turn, tend to comprehend, i.e. aspire to reconstructthe personal perspectives, experiences and understandings of the individual actors.Thus, while quantitative approaches are usually deductive and theory-driven (i.e.they observe specific phenomena on the base of specific theories of reference),qualitative ones are inductive and data-driven (i.e. they start from the observation ofphenomena in order to build up theories about those phenomena). In quantitativeapproaches, hypotheses are deductively derived from the theory and have then to befalsified through empirical investigation (confirmatory study). In qualitativeapproaches, however, the development of hypothesis is part of the research processitself, whose aim is to develop an adequate theory according to the observations thathave been made (exploratory study).

Research Designs Each methodology (quantitative vs. qualitative) makes use ofspecific research designs (see Table 1). A research design is the plan of actions orstructure which links the philosophical foundations and the methodological assump-tions of a research approach to its research methods (see next paragraph), in order toprovide credible, accountable and legitimate answers to the research questions.

Rigorous research designs are important as they guide the methods decisions thatresearchers must make during their studies and set the logic by which they makeinterpretations at the end of their studies. Research designs within the quantitativeapproach include experimental and non-experimental designs. Experimental designsmake causal inferences about the relationship between an independent and one ormore dependent variables. They are characterized by the direct manipulation of theindependent variable and by a rigorous control of extraneous variables2. In thosesituations where the independent variable cannot be manipulated, a non-experimen-tal design has to be implemented. The primary aim of such a design is to describethe relationship between two or more variables of interest3.

Contrary to quantitative research approaches—which employ experimental andnon-experimental designs—qualitative approaches make use of naturalistic designs(Lincoln and Guba 1985), whose aim is to study behaviour in natural settings. Thatmeans that phenomena of interest are investigated as they occur naturally, offeringlittle structured context of observations. One fundamental assumption of suchdesigns is that behaviour is best understood as it occurs in its natural contexts,without external constraints or control. The natural context of observation, instead ofbeing regarded as a source of variability to be controlled, is considered essential for adeeper understanding of the phenomena under investigation.

Naturalistic designs include, among others (for more details, see Denzin andLincoln 2005; Silverman 2004): (a) case study designs, which involve an in-depth,longitudinal examination of a single instance or event—called case (e.g. an

2 According to the degree of exerted experimental control, it is possible to distinguish between true-experiments and quasi-experiments (for a detailed description see Polgar and Thomas 2000).

3 Non-experimental designs include correlational designs, correlational–comparative designs, correla-tional–causal–comparative designs, and ex-post-facto designs.


organization, an individual, a specific event)—in order to gain a sharpenedunderstanding of it4; (b) discourse and conversation analysis designs, which sharethe focus on language as medium for interaction; (c) focus group designs, whichanalyze the emergent issues and themes from a group discussion “focused” on aspecific topic5; (d) grounded theory designs, where the field-data is used to generatea “grounded” theory, that is a set of propositions that pertain to a specific experience,situation, or setting; and (e) ethnographic designs, which enable an in-depthdescription and interpretation of shared patterns of beliefs, expectations, andbehaviours within a cultural or social group.

Validity A final important issue of research methodology is that of validity. Validitycan be generally referred to as the level of accountability and legitimacy that isstrived through data collection, analysis and interpretation (see “Research Methods”in the next paragraph) (Onwuegbuzie and Teddlie 2003). At a general level it ispossible to distinguish between internal validity and generalizability (Maxwell andLoomis 2003). These two distinct aspects of validity have been differentlyconceptualized within quantitative and qualitative research approaches (see Table 1).With regard to quantitative research, Cook and Campbell (1979) identified statisticalconclusion validity (i.e. the validity of inferences from the sample to the population),construct validity (i.e. the validity of the theoretical constructs employed), andcausal validity (i.e. the validity of the cause–effect relationship between observedvariables) as specific kinds of internal validity. External validity, on the other hand,can be defined as “the extent to which the results of a study can be generalizedacross populations, settings, and times” (Johnson and Christensen 2000; p. 200)6.

In relation to qualitative approaches, Maxwell (1992) identified four maincategories of validity: descriptive validity (i.e. the validity of the descriptions ofsettings and events), interpretative validity (i.e. the validity of statements about themeanings or perspectives held by participants), explanatory validity (i.e. the validityof claims about causal processes and relationships, including construct validity aswell as causal validity), and generalizability (i.e. the extent to which a researcher cangeneralize the account of a particular situation or population to other individuals,times, setting, or contexts)7.

It is important to observe that causality and causal inference are controversial inqualitative research. Some researchers (e.g. Guba and Lincoln 1989) deny thatcausality is an appropriate concept in qualitative research. Some others (e.g. Sayer2000) argue that causal explanation is relevant also in qualitative research but that itis “based on process rather than variance concept of causality” (Maxwell andLoomis 2003; p. 255).

5 The term focus group may also refer to a specific form of data collection (see next paragraph).6 For a detailed description of different typologies of validity as well as of validity threats and strategiesfor addressing these threats see Campbell and Stanley 1963 and Cook and Campbell 1979.7 Generalizability has also been referred to as transferability (Guba and Lincoln 1989).

4 Case study designs may also be appropriate to the quantitative approach. In this case it is usual to talkabout single-case research designs (see Hilliard 1993 and Kazdin 1982).


Research Methods

Different research designs may be implemented by different research methods.Research methods regard those procedures and techniques involved in datacollection, analysis and interpretation. Collecting and analyzing data are the concretesteps, which allow valid answers to the research questions. Quantitative andqualitative approaches differ in the research methods they apply (see Table 2). Thesewill be described with reference to data sampling, collection, analysis, andinterpretation (for a detailed account see Creswell 2005). These differences havealso contributed to the QQD, although to a lesser extent, compared with theparadigmatic foundations.

Sampling In quantitative research, the intent of sampling is to choose individualsthat are representative of a population, so that results can be generalized to it(external validity). To accomplish this task, quantitative researchers may resort toboth probabilistic (i.e. each member of the population has the same probability to beincluded in the sample) and purposive (i.e. use of some criterions to replace theprinciple of cancelled random errors) sampling (for a detailed overview see Kemper,Stringfield, and Teddlie 2003). Some of the most adopted strategies of probabilistic

Table 2 Attributes of quantitative and qualitative research methods

Quantitative approaches Qualitative approaches

Sampling SamplingProbabilisticSimple random samplingSystematic random samplingStratified random samplingCluster samplingPurposive PurposiveConvenience sampling Convenience sampling

Homogeneous cases samplingExtreme/deviant and Typical case sampling

Data collection Data collectionPrimary data Primary dataTests or standardized questionnaires Open-ended interviewsStructured interviews Focus groupClosed-ended observational protocols Naturalistic observation protocolsSecondary data Secondary dataOfficial documents Official documents

Personal documents

Data analysis Data analysisDescriptive statistics DescriptionInferential statistics Identification of categories/themes

Looking for interconnectedness between categories/themes

Data interpretation Data interpretationGeneralization ContextualizationPrediction based (theory-driven) Interpretation based (data-driven)Interpretation of theory Personal interpretation


sampling are the simple random sampling (i.e. each member of the identifiedpopulation has an equal chance of being included in the sample), the systematicrandom sampling (which involves the selection of each nth unit of the targetpopulation from a randomly ordered list of the population), the stratified randomsampling (which is obtained separating the population into groups so that eachelement belongs to a single group, from which then a random sample is selected),and the cluster sampling (where a random sample of groups—which are naturallyoccurring in the population—is selected). Convenience sampling (whereby elementsare drawn from a subpopulation according to its accessibility and research interests)is a form of purposive sampling usually used within quantitative research designs.

Qualitative approaches, by contrast, make use of almost exclusively purposivesampling strategies. These allow “selecting information-rich cases to be studied indepth” (Patton 1990; p. 169). Purposive sampling strategies include, among others:convenience sampling (see above), homogeneous cases sampling (i.e. pickingelements from a subgroup to study in-depth), snowball sampling (i.e. usinginformants to identify cases that would be useful to include in the study), extreme/deviant and typical case sampling (which involve seeking out respectively the mostoutstanding cases—in order to learn as much as possible about the outliers—or themost average cases from a subpopulation) (see Table 2).

Data Collection Once the sampling is concluded, data has to be collected (see Table 2)(see also Creswell 2005, for a detailed description). Data may be collected directlyfrom the subjects constituting the sample (primary data) or indirectly, e.g. by makinguse of personal and official documents as well as research archives (secondary data).In quantitative research, data has to be collected which are relevant to test theformulated hypotheses. Data collection is attained by using tests or standardizedquestionnaires (which assess performances, attitudes, personality, self-perception,etc.), structured interviews (where the interviewer just reads the pre-defined questionsand records the answers related to one or more issues or phenomena relevant to theresearch questions), and closed-ended observational protocols (which allow classify-ing the behaviour of interest using pre-defined categories8,9). Secondary data may alsobe collected as, for example, referring to official documents (e.g. financial records andcensus data). The resulting data is finally coded by assigning numeric values, andsuccessively introduced into a data matrix, which will be used for the statisticalanalysis (see next section). It is suggested to develop a codebook that lists variablesnames, their definition, and coding values.

In qualitative research, data has to be collected in order to allow an in-depthunderstanding of the participants’ perspective. For that reason, qualitative datacollection procedures display a much lower degree of standardization compared toquantitative data collection. Qualitative data collection is usually accomplished by using

8 Whereby the visual data is usually video-recorded in order to allow the subsequent analysis according tothe specific observational protocol used.9 See e.g. the Analysis and Treatment of Finger Sucking (Ellingson et al. 2000), which allowsinvestigating the reinforcements useful in maintaining finger sucking, or the Strange Situation Protocol(Ainsworth et al. 1978) for the assessment of attachment in infants (12–20 months).


open-ended interviews (which allows investigating the subject’s perspective regardinga pre-defined set of topics10), focus groups (i.e. an in-depth group discussion“focused” on one or more specific issues or topics of interest), and naturalisticobservation protocols (which allow the observation of specific events and/orbehaviours of one or more subjects in real-world situations11). Interviews are usuallyaudio-recorded; naturalistic observation protocols end up in an accurate description ofobserved events and processes and in field notes, which are accounts describingexperiences and observations the researcher has made during the observation. Video-recordings of the observed behaviours and/or situations may help in this process.Qualitative research also makes often use of secondary data, like personal documents(i.e. anything personal written, photographed or recorded for private purposes), officialdocuments (e.g. speeches and video recordings of television shows and advertise-ments) and archived research data (which may e.g. contain results of previouslyconducted qualitative studies). The overall text data obtained in this way must then betranscribed in order to be analyzed (see next section).

Data Analysis Data analysis consists of examining the database to address theresearch questions and hypotheses (see Creswell 2005, for a detailed description). Inquantitative research approaches, the researcher analyzes the data in order to test oneor more formulated hypotheses; however, explorative data analysis is also possible.The aim is to find out if the relationships between the observed variables (either of acausal or correlational nature) in one or more groups are statistically significant, thatis, generalizable to the population the sample is drawn from. The choice of astatistical test is based on the type of questions being asked (e.g. describe trends,compare groups, or relate variables), the types of scales used to measure thevariables (nominal, ordinal, interval or ratio), and whether the population is normallyor non-normally distributed. Confidence intervals and effect sizes may also be usedto provide further evidence. Quantitative analysis proceeds from descriptive toinferential (hypotheses-testing) analysis. Finally, the results of the analysis arepresented in the form of statements summarizing the statistical results. Tables orfigures may also be used.

Qualitative data analysis is carried out on the previously collected text data (i.e.transcriptions, memos and field notes) through content or thematic analysis. Contentor thematic analysis is based on the examination of the data for recurrent instances ofsome kind; these instances are then systematically identified across the data set, andgrouped together by means of a coding system (Silverman 2004). Coding is aprocess of grouping evidence and labelling portions of text so that they reflectincreasingly broader perspectives. The researcher first divides the text to be analyzedinto units (sentences, phrases or passages) and labels them, using terms that shouldcome from exact words of the participant. According to the observed similarities anddifferences between the labelled text units, the researcher groups labels together into

11 The roles of an observer may vary on a continuum: complete participant, to the participant-as-observer,observer-as-participant, and complete observer (Johnson and Turner 2003).

10 Some forms of open-ended interview (i.e. the interview guide approach and the standardized open-ended interview) correspond to some extent to what is generally known as semi-structured interview.


themes (or content categories). These emergent themes are then re-labelled, using alanguage closer to the language of the researcher and to the theory of reference.Finally, the themes (or content categories) are interrelated to each other andabstracted into a set of themes, which will receive new labels. This procedure allowsreaching gradually higher levels of abstraction in the description of the data, andidentifying the constituents of the analyzed texts. The obtained data is thenpresented. Presenting qualitative results essentially involves a discussion of theevidence for the emerged themes and perspectives. The idea is to build a discussionthat persuades the reader that the identified categories and dimensions are effectivelygrounded in the observed data, and not imposed by the researcher. Figures, maps ortables may also be used to represent these results. Table 2 offers a syntheticdescription of the main features of data analysis in quantitative and qualitativeresearch.

Data Interpretation Data interpretation consists of figuring out what the findingsmean, and is part of the overall effort to make sense of the evidence gathered. Inquantitative research, data interpretation consists of giving a meaning to theobtained results with reference to the theory the hypotheses have been developedfrom. This process can also be referred to as deductive inference (Tashakkori andTeddlie 2003b). According to whether the design was experimental or non-experimental, conclusions may be drawn concerning cause–effect relationships orcorrelations between variables in the population the sample was selected from.These conclusions may then enable to confirm, extend or challenge the theory ofreference.

In qualitative research, data interpretation is based on a process of inductiveinference (Tashakkori and Teddlie 2003b), which refers to a process of creatingmeaningful and consistent explanations, understanding, conceptual frameworks,and/or theories drawing on a systematic observation of phenomena. In theseterms, qualitative data interpretation consists of giving a meaning to the obtainedresults with reference to the specific and particular context of the study (e.g.settings, participants). This process of contextualization is necessary to addressthe issue of qualitative internal validity (i.e. descriptive, interpretative andexplanatory validity). In which way may qualitative results (i.e. statements aboutthe meaning and/or perspectives held by the participants concerning a specificissue) help us in increasing our systematic understanding of the issues underinvestigation? According to the kind of naturalistic research design used, theinterrelated themes and/or categories which result from the analysis may be usedto comprise a model (as in grounded theory designs), a chronology (as innarrative research designs), or comparisons between groups (as in ethnographicdesigns). A process of larger sensemaking should then be employed to broadenthe understanding and the theoretical perspectives the results may contribute todevelop. In this way, issues of qualitative external validity (transferability) maybe addressed. Contrary to quantitative research, where results interpretation istheory-driven and may lead to a confirmation, extension or questioning of an alreadyexisting theory, qualitative data interpretation aims at developing data-drivenhypothesis and new theoretical perspectives and understanding of the phenomenaunder investigation.


Mixed Methods Research

We have shown how quantitative and qualitative approaches are profoundly diverseat different levels (i.e. philosophical foundations, methodological assumptions, andresearch methods). Each approach has its strengths and weaknesses, and usually thestrengths of an approach may be considered as the weaknesses of the other approach,and vice versa. These differences have been perceived by the proponents of eachapproach in terms of dichotomy, rather than complementarity. This has stronglycontributed to sustain a debate between quantitative and qualitative researchapproaches (QQD) over the years, leading to epistemological fragmentation,theoretical insularity, and empirical arbitrariness. However, in the past decades anew research approach has been developed, known as Mixed Methods Research(MMR). MMR can be defined as a research approach that combines and integratesquantitative and qualitative research approaches. This research approach is, as in thecase of quantitative and qualitative research approaches, characterized by specificphilosophical foundations, methodological assumptions and research methods.These will be described in the following sections (for a detailed description seeTashakkori and Teddlie 2003a; Creswell and Plano Clark 2007).

Worldviews and Philosophical Foundations

After a formative period between the 1950s and the 1980s, which saw the initialinterest in combining quantitative and qualitative methods in a study (e.g. Sieber1973), a paradigm debate period occurred between the 1970s and the 1980s. Theprevailing issue of this period was the opportunity of integrating the philosophicalfoundations of quantitative and qualitative research. Some (e.g. Smith andHeshiusius 1986) argued that the underlying paradigms of these two researchapproaches were incompatible (see Smith 1983). In 1988, Bryman (1988) challengedthis argument suggesting how the two research paradigms could be combined.Although the debate is still very lively, nowadays there is a consistent agreementabout combining quantitative and qualitative research paradigms. Greene andCaracelli (2003) delineate four meaningful instances in mixing paradigms: (1)thinking dialectically about mixing paradigms, (2) using a new paradigm, (3) beingpragmatic, and (4) putting substantive understanding first. The first two considerparadigms essential for guiding research inquiry, but propose different solutions(Greene and Caracelli 2003). According to the dialectical stance, all paradigms maybe equally valuable to guide scientific research. For this reason, researchers shouldintentionally engage in a dialectical way with multiple sets of philosophicalassumptions toward better understanding (e.g. Greene 2000). The proponents of anew paradigm suggest that paradigms may and should evolve in order to incorporatea broader set of beliefs and assumptions, and therefore welcome more diverse sets ofmethods. One example is the commonsense realism by Putnam (1990), according towhich social reality is both causal and contextual. In this case, the combination ofquantitative and qualitative methods is not only welcomed but actually required.

The third and the fourth instances, by contrast, consider paradigms not primarilyrelevant in guiding research inquiry (Greene and Caracelli 2003). According to thepragmatic (or context-driven) instance, what matters most is the responsiveness to


the demands of the inquiry context. Pragmatists are open by an allegiance to anyparadigm that fits best with the research aims (see e.g. Howe 1988). Finally, theproponents of the concept-driven instance claim, on the other hand, that conceptualor theoretical congruence is the most relevant issue in guiding empirical research.Decisions concerning the research process are made not for their congruence withparticular sets of philosophical assumptions but rather for their ability to enhanceunderstanding of a particular set of concepts in a particular context (see for exampleCooksy et al. 2001).

The philosophical foundations of MMR described above show how this researchapproach allows for multiple worldviews and paradigms. This may enable askingdifferent and more complex questions and, consequently, looking for different andmore complex answers. We suggest that this is the first step to overcome thelimitations connected to the single application of either quantitative or qualitativeapproaches.

Methodological Assumptions

General Issues The methodology of MMR can be described with reference to whatNewman and Benz (1998) called qualitative–quantitative “interactive continuum” ofresearch. As the name suggests, this model considers an interactive continuum, andnot a dichotomy, between qualitative and quantitative methodologies. This model isbased on a unitary vision of science, according to which quantitative and qualitativemethodologies must interact in a continuous way in order to allow researchers toanswer different and complementary research questions. In extending his model,Newman and colleagues (Newman et al. 2003) focus on the researcher’s purpose aseven more fundamental than the researcher’s question. They argue that systemat-ically ordering one’s research purposes may accomplish the linkages betweendifferent research questions and the correspondent methodologies, providing afoundation for MMR methodology.

Shedding light on the “dynamic of research purposes” is necessary to understandMMR’s methodology (Newman et al. 2003). In order to do that, Newman andcolleagues (Newman et al. 2003) present a typology of research purposes, each ofwhich is generally associated with either a quantitative or a qualitative methodology.These nine general purposes (and the correspondent methodologies) are categorized asfollows: (1) predict—through quantitative methodology, (2) add to the knowledge base—through quantitative methodology, (3) have a personal, social, institutional, and/ororganizational impact—through qualitative methodology, (4) measure change—through quantitative methodology, (5) understand complex phenomena—throughqualitative research, (6) test new ideas—through quantitative methodology, (7)generate new ideas—through qualitative methodology, (8) inform constituencies—through qualitative methodology, and (9) examine the past—through qualitativemethodology. It is interesting to observe how each of these different researchpurposes—with the respective quantitative or qualitative methodology—may flowinto, overlap with, and generate other research purposes. This may be characteristic of asingle- or multiple-study research approach. The nine research purposes outline agestalt showing how quantitative and qualitative methodologies may represent aninteractive continuum along which a researcher may plan his study oscillating in a


dynamic way between generalization and contextualization, explanation and under-standing, deduction and induction, and hypotheses-testing and hypotheses-generating.

Research Designs Different research designs inMMR have been identified. Tashakkoriand Teddlie (2003a) have reported nearly 40 different types of mixed methods designsin the literature. Creswell and colleagues (Creswell et al. 2003) have summarized therange of these classifications. Finally, this summary has been updated, leading to a listof 12 classifications which span the past 15 years of scholarly writings about mixedmethods approaches (Creswell and Plano Clark 2007). In order to provide a moresynthetic, parsimonious and functional overview of the different research designsactually existing in MMR, Creswell and Plano Clark (2007) propose four major mixedmethods designs, each of one with its variants: the triangulation design, the embeddeddesign, the explanatory design, and the exploratory design. They can be allocatedeither in one-phase or two-phase approaches (see Table 3).

In one-phase approaches, qualitative and quantitative methods are appliedsimultaneously (for this reason they are also called concurrent designs) and to thesame sample; this is the case of triangulation designs and one-phase embeddeddesigns. In two-phase approaches, the quantitative and qualitative methods areapplied one after the other (for this reason they are also called sequential design) tothe same sample or to different samples in the different stages of the study; this is thecase of explanatory designs, exploratory designs, and two-phase embedded designs.The four main mixed methods research designs are depicted in Fig. 1.

The triangulation design (also called convergence triangulation design) representsthe most and well-known approach to mixing methods (Creswell et al. 2003). Itspurpose is “to obtain different but complementary data on the same topic” (Morse2003; p. 122) (see Fig. 1). The underlying idea is that, to best understand a researchproblem, it is necessary to bring together the differing strengths and non-overlappingweaknesses of quantitative methods (large sample size, trends, generalization) withthose of qualitative methods (small N, details, in-depth) (Creswell and Plano Clark2007). This is especially the case when a researcher wants to directly compare andcontrast quantitative statistical results with qualitative findings, or to validate or expandquantitative results with qualitative data. In the triangulation design, researchersimplement quantitative and qualitative methods during the same timeframe (one-phase

Table 3 Mixed methods research designs and their variants in one-phase and two-phase approaches

One-phase approach Two-phase approach

Triangulation ExplanatoryData transformation model Follow-up explanation modelValidating quantitative data model Participant selection modelMultilevel model

Embedded ExploratoryEmbedded experimental model Instrument developmentCorrelational model Taxonomy development

EmbeddedEmbedded experimental model


design) and with equal weight. It involves the concurrent, but separate, data collectionand analysis (see next paragraph). The two data sets are merged by bringing the resultstogether into one overall or by transforming one data set into the other, and the overallresults are then interpreted.

Some variants exist (Creswell et al. 2003; Creswell and Plano Clark 2007): thedata transformation model, the validating quantitative data model, and the multilevelmodel (see Table 3). The data transformation model is used when a researcher wantsto know to what extent the different types of data confirm each other. After initialdata collection, one data type is transformed into the other data type (by eitherquantifying qualitative findings or qualifying quantitative results (Tashakkori andTeddlie 1998; for application see Pagano et al. 2002). Researchers use the validatingquantitative data model when they want to validate and expand on the quantitative

One-phase approach

(a) Merge the data:

Triangulation design

(b) Embed the data:

Embedded design

Two-phase approach

(a) Connect the data:

Explanatory design

Exploratory design

(b) Embed the data:

Embedded design

or

QUAN Interpretation of

QUAN (qual) results

QUAL Interpretation of

QUAL (quan) resultsqual quan

Interpretation ofQUAL quan

resultsQUAL quan

or

QUAN Interpretation of

QUAN (qual) results

QUAL Interpretation of

QUAL (quan) resultsqual quan

qualInterpretation ofQUAN qual

resultsQUAN

Interpretation ofQUAN + QUAL

resultsQUAN QUAL

→

→

Fig. 1 Mixed methods research designs


findings from a survey by including a few open-ended qualitative questions (see e.g.example Webb et al. 2002). Finally, in the multilevel model (Tashakkori and Teddlie1998), different methods (quantitative and qualitative) are used to address differentlevels within a system. The findings from each level are then merged together intoone overall interpretation. For example, Elliott and Williams (2002) studied anemployee counselling service using qualitative data at the level of clients,counsellors and directors, and quantitative data for the organizational level.

The embedded design is a mixed method design where one data set provides asupportive, secondary role in a study primarily based on the other data type (Creswellet al. 2003; see Fig. 1). This design is used when researchers need to include qualitativeor quantitative data to answer a research question within a largely quantitative orqualitative study. Qualitative data could be embedded within a primarily quantitativemethodology (e.g. an experimental design), or quantitative data could be embeddedwithin a primarily qualitative design (i.e. a grounded theory design).

A variant of this research design is the embedded experimental model, wherequalitative data is embedded within an experimental design (either a true experimentor a quasi-experiment) (see Table 3). This variant can be used either as a one-phaseor a two-phase approach. For example, in a one-phase approach qualitative data canbe embedded during the intervention phase, when the researcher wants to conductin-depth investigation of the participants’ perspective during the process ofintervention. A two-phase approach is instead used when the researcher needsqualitative information before the intervention (e.g. in order to better shape theintervention or to select participants) or after the intervention (e.g. to explore indepth the results of the intervention or to follow up on the experiences of theparticipants about the intervention). For example, Evans and Hardy (2002a, b)conducted an experimental study of goal-setting intervention for injured athletes,followed up by interviewing participants from each of the treatment group to betterinterpret the results of the experimental study. Another variant of the embeddeddesign is the correlational model, in which qualitative data is embedded within aquantitative design. Researchers conduct a quantitative correlational study, and at thesame time collect qualitative data to help explain the obtained results.

The explanatory design is a two-phase mixed methods design. The overallpurpose is to obtain quantitative results, and then explain or build on them usingadditional qualitative data (Creswell et al. 2003; see Fig. 1). In an explanatoryresearch design the researchers start with the collection and analysis of quantitativedata; after that, a qualitative phase of the study is designed so that it follows (orconnects to) the results of the first quantitative phase.

There are two variants of the explanatory design: the follow-up explanation andthe participant selection model (see Table 3). In the follow-up explanation model theresearcher first identifies specific quantitative findings that need additionalexplanation (e.g. significant–non significant, outlier, or surprising results), and thencollect and analyze data from participants that can best help in explaining the results.In the participant selection model quantitative information is used to identify andpurposefully select participants for a follow-up, in-depth qualitative study. In thisvariant the focus is primarily qualitative. For example, May and Etkina (2002)collected quantitative data to identify students with high and low conceptual learninggains, and then completed an in-depth qualitative comparison between these groups.


The last mixed method research design is the exploratory design. The aim of thistwo-phase design is to use the results of the method applied first (qualitative) tofurther develop or inform the results obtained with the second (quantitative) method(Creswell et al. 2003; see Fig. 1). This design is used when exploration of data isneeded (e.g. measures or instruments are not available, little is known aboutvariables that have to be assessed, lack of guiding theory or framework). Researchersstart with qualitative data in order to explore in depth a phenomenon, and then stepto a second, quantitative phase.

This design has two common variants: the instrument development model and thetaxonomy development model. The instrument development model allows developinga quantitative instrument based on qualitative findings. Through a qualitativeinvestigation it is possible to explore the research topic with a few participants. Theseresults are then used to develop items and scales, which will constitute the quantitativesurvey instrument. The taxonomy development model makes use of the initialqualitative phase to identify important variables, develop taxonomy or classificationsystems, elaborate an emergent theory; thereafter, the quantitative phase is used to test orstudy these results in a more detailed way (Tashakkori and Teddlie 1998). This modelallows formulating research questions or hypotheses based on qualitative findings, andtesting them within a quantitative framework (see e.g. Goldenberg et al. 2005).

Validity Mixed methods researchers, as quantitative and qualitative ones, strive forthe accountability and legitimacy of their research results, which is necessary fordrawing valid inferences (see data interpretation in the next paragraph). The issue ofvalidity in MMR is one of the most addressed issues in the literature (e.g. Tashakkoriand Teddlie 2003b).

It is possible to distinguish between inference quality and inference transferability.Inference quality incorporates the quantitative internal validity and the qualitativetrustworthiness and credibility of interpretation. It can be defined as “the degree towhich the interpretations and conclusions made on the basis of the results meet theprofessional standards of rigor, trustworthiness and acceptability as well as thedegree to which alternative plausible explanations for the obtained results can beruled out” (Tashakkori and Teddlie 1998; p. 709). By contrast, inferencetransferability subsumes the quantitative external validity (generalizability) as wellas the qualitative transferability. It can be defined as the “generalizability orapplicability of inferences obtained in a study to other individuals or entities, othersettings or situations, other time periods, or other methods/instruments ofobservation” (Tashakkori and Teddlie 1998; p. 710).

Specific MMR designs may contribute to enhance inference quality and inferencetransferability in different ways. Triangulation design, for example, may allow abroader range of inferences based on the merging of quantitative and qualitativedatasets. In an embedded experimental design, the overall validity of the study isincreased by qualitatively addressing the “process” beside the quantitativeinvestigation of the “product”. In a follow-up explanatory design, the subsequentqualitative analysis may provide additional meaningful information to explain thepreviously obtained quantitative results. Finally, in an exploratory design, apreviously conducted qualitative investigation of a topic in order to develop aquestionnaire may lead to more precise and accurate results.


The specific methodological assumptions of MMR allow to address different andarticulated research questions through a dialectic combination between quantitativeand qualitative approaches. According to us, this represents a second essential step toget over the limitations of purely quantitative or qualitative approaches.

Research Methods

Different mixed methods research designs are characterized by specific proceduresused for data collection (which includes sampling strategies), analysis, andinterpretation. These may present distinct issues according to whether concurrent(one-phase) or sequential (two-phase) research designs are implemented.

Sampling The specific sampling strategies for quantitative and qualitative research (seeTable 2) should be applied also when these two research approaches are used incombination. One supplementary issue concerns participant selection: should the sameor different individuals be selected for the quantitative and qualitative sample? In thecase of triangulation, embedded and explanatory designs, researchers should select thesame individuals for both quantitative and qualitative data collection. If an exploratorydesign has to be implemented, the individuals selected for the first qualitative datacollection are typically not the same as those selected for the following quantitativephase. This is because the aim of such a design is to generalize the results to population.

Another relevant issue is that of sample size: should the same number of individualsbe sampled respectively for the quantitative and qualitative data collection? Generally,the quantitative sample will be bigger than the qualitative one. An exception may beobserved in the case of triangulation design. In this case, the size of both quantitativeand qualitative samples should be as similar as possible, to avoid that differences insample size are reflected in differences in the two datasets.

Data Collection Data collection in MMR can be concurrent (as in triangulation andone-phase embedded designs) or sequential (as in explanatory, exploratory, and two-phase embedded designs) (for a detailed account see Creswell and Plano Clark 2007). Inthe case of concurrent data collection, data is collected during the same timeframe,even though independently from each other (see Fig. 1). The collected data may haveequal or unequal weight (as in triangulation design vs. one-phase embedded designs).

By contrast, sequential data collection involves different stages (see Fig. 1). Thedata is first collected (and then analyzed, see next section) either in a quantitative form (asin explanatory or two-phase embedded designs) or in a qualitative form (as in exploratoryand two-phase embedded designs). Decisions are then made about how the results (eitherquantitative or qualitative) will be used to influence the following data collection (eitherqualitative or quantitative). Finally, a second and complementary phase of data collection(and analysis, see next section) builds on the first one. Either quantitative or qualitativedata collection may be weighted more heavily. Quantitative data collection is moreweighted in the first phase of follow-up explanatory designs, and in the second phase ofinstrument development exploratory designs; qualitative data collection is moreweighted in the second phase of participant selection explanatory designs, and in thefirst phase of taxonomy development exploratory designs. In two-phase embeddeddesigns, quantitative data collection is always more weighted than qualitative one.


Data Analysis As in the case of data collection, also data analysis in MMR may beeither concurrent or sequential (for a detailed account see Creswell and Plano Clark2007). The aim of concurrent mixed methods data analysis is to look forconvergences resulting from merging, or embedding the results from differentdatasets. Concurrent data analysis involves conducting a separate initial analysis foreach of the quantitative and qualitative datasets. After that, the researcher merges orembeds the two datasets, so that a complete picture is developed from both of them(triangulation design), or so that the supportive data set can reinforce or refute theresults of the first dataset (one-phase embedded design).

Two techniques are available for merging quantitative and qualitative datasets inMMR: data transformation and comparison. Data transformation (see Onwuegbuzie andTeddlie 2003) may allow transformation of one form of data into the other.Transforming qualitative data into quantitative ones is usually done in studiesinvolving content analysis (see Sandelowsky 2003). This procedure consistsessentially in reducing qualitative codes, themes and/or content categories to numericinformation, counting the occurrence of each previously identified category.Thereafter, a matrix can be developed, which combines the different qualitativecategories with their occurrences. Transforming quantitative data into qualitative oneshas received much less attention in the literature. An example is, however, provided byPunch (1998), where quantitative data were loaded into factors in a factor analysis, andthe factors were then viewed as aggregated units similar to themes.

Data can be merged also by comparing the results of quantitative and qualitativedata through a matrix or a discussion. In the first case, for example, it is possible toidentify within the text data quotes, which synthetically represent the previouslyidentified qualitative themes. This information can then be introduced into a matrixtogether with the results of quantitative analysis, allowing a comparison between theresults from the two datasets. A discussion may also be used to compare the data. Inthis case, the quantitative results may be displayed and then discussed with referenceto the obtained qualitative results.

The purpose of sequential mixed methods data analysis is to use the results fromthe first data set to inform the results which will be obtained with the second data set.Sequential data analysis therefore involves an initial stage where the first data set isanalyzed following the traditional quantitative (as in explanatory or two-phaseembedded designs) or qualitative (as in exploratory or two-phase embedded designs)procedures of analysis (see Table 2). The resulting information is then used to takedecisions concerning the analysis of the second data set.

Data Interpretation Data interpretation in MMR takes place after the data has beencollected and analyzed either in a concurrent or sequential way12. In MMR, theprocess of making sense of the evidence gathered involves a cyclical combination

12 This is the case of mixed methods designs, which are described in the present paper. In mixed modeldesigns (for on overview see Tashakkori and Teddlie 2003a), by contrast, interpretation takes place afterthe application of each quantitative and qualitative strand of the design. The researcher has then to gothrough a process of meta-interpretation. The inferences developed for each strand of the design are thenintegrated. This process is called meta-inference (Tashakkori and Teddlie 2003a).


between the processes of quantitative deductive inference (theory-driven hypothesistesting, verification oriented) and qualitative inductive inference (data-drivenhypothesis and theory development, exploration oriented). According to whether aconcurrent or a sequential design has been used, there may be a different emphasison either the deductive inference, the inductive inference, or both. A major emphasison quantitative deductive inference processes is characteristic of (a) triangulationdata validating design (in order to find out to what extent the qualitative resultssupport the quantitative ones), (b) embedded experimental and correlational design(to find out how the qualitative results inform and help to explain the experimentalor correlational results), (c) explanatory follow-up design (to find out how thequalitative results help explain the quantitative ones), and of (d) explanatoryinstrument development design (to find out what items and scales represent at bestthe qualitative results).

Some other designs are characterized by an emphasis on qualitative inductiveinference processes. These are the (a) classical embedded design (to find out how thequalitative results support or disconfirm the quantitative ones), and the (b)explanatory taxonomy development design (to find out in what ways the quantitativeresults generalize the qualitative ones).

Finally, emphasis on both quantitative deductive and qualitative inductiveinference processes is placed in the (a) triangulation convergence design (to findout to what extent, how, and why the quantitative and qualitative data converge), (b)triangulation data transformation design (to find out to what extent quantitative andqualitative results confirm each other), and (c) triangulation multilevel design (tofind out how quantitative and qualitative results confirm each other at different levelsof observation).

These different combinations of quantitative deductive and qualitative inductiveinference processes allow addressing in different ways the issues of internal(inference quality) and external (inference transferability) validity in MMR.

The research methods described above provide the possibility of a more reliableand valid data collection, analysis and interpretation. Convergences at the level ofdata collection and analysis (e.g., quantitative and qualitative data are coherent witheach other, the results of quantitative and qualitative analysis support each other)may allow more consistent and meaningful interpretations of the results. Incongru-ities, by contrast, may suggest to refine procedures of data collection and/or analysis,as well as to develop new research questions.

MMR: Toward a More Comprehensive Psychological Research?

Psychological research, developing out of a positivist perspective, has been aremarkably quantitative field. However, since the first half of the century, qualitativeresearch approaches have been developed within social and psychological research.This has led to the development of an enduring debate between these two opposedresearch approaches. The Quantitative–Qualitative Debate (QQD) has been sustainedat the level of philosophical foundations (e.g. objectivism vs. subjectivism andconstructivism), research methodologies (e.g. explanation vs. understanding,prediction vs. interpretation, deduction vs. induction), and research methods (big


vs. small samples, numbers vs. narratives, statistical analysis vs. content analysis,hypothesis testing vs. theory generation).

In order to overcome this debate, Mixed Method Research (MMR) has beenformally developed since the 1980s. Aim of the MMR approach is to combine orintegrate the traditional quantitative and qualitative research approaches in order tomaximize the advantages and minimize the disadvantages connected to the singleapplication of one of the two approaches. Despite the attempt of integratingquantitative and qualitative research approaches, the QQD is still very lively. This istestified, for example, by the recent article of Toomela (2008), which shows thelimitations of variable psychology for the development of a theory of mind due to (a)the inadequacy of quantitative variables to encode in a reliable and externally validway information about mental phenomena, and (b) the related misleadingconclusions statistical analysis may lead to.

The scientific investigation of mind is a very complex issue. It requires thedevelopment of theories, which establish general laws of functioning and, at thesame time, account for the idiosyncratic differences that different individuals maypresent. It also requires the reference to multiple level of analysis, both at an intra-individual level (e.g. the interconnections between biological and psychologicalstructures and functions, the relationships between motivational, emotional,cognitive and behavioural schemes, the different ways of attributing meanings tosituations and events) and at an inter-individual level (e.g. the bio-psychosocialadaptation to the environment, the quality of interpersonal relationships withinfamiliar, social and cultural contexts). For these reasons, we believe that thedevelopment of an adequate theory of mind requires the cycling between approacheswhich, striving for integration, avoid dichotomous (either reductionistic orrelativistic) and therefore partial accounts of phenomena.

We have tried to show how MMR may provide a useful context for a morecomprehensive psychological research, of the extent to which it promotes adialectic interaction of different perspectives at different levels. At a philosoph-ical level, MMR acknowledges the necessity of eventually referring to multipleworldviews and paradigms. This may help in asking more complex questionsfrom different perspectives, which may in turn allow seeking different and morecomplex answers. At the level of methodology, MMR overcomes the dichotomybetween nomothetic and idiographic methodologies which, on the contrary,should be located on an interactive continuum. In this way, a cyclical dynamiccan be established between generalization and contextualization, explanation andunderstanding, deduction and induction, and hypothesis testing and hypothesisgeneration. Finally, at the level of research methods, MMR enables theintegration of data collection and analysis (either concurrent or sequential)which, in turn, may allow (a) overcoming the traditional limitations concerningboth the information encoded in quantitative variables (see Toomela 2008), andthe meaning contained in qualitative accounts, and (b) transcending the rigiddichotomy existing between deductive and inductive inferences, thus leading to anincreased accuracy and meaningfulness of data interpretation. We believe that inthis way it will be possible to overcome the limitations of purely quantitative orqualitative approaches, providing a fruitful context for a more comprehensivepsychological research.


References

Ainsworth, M. D. S., Blehar, M. C., Waters, E., & Wall, S. (1978). Patterns of attachment: apsychological study of the strange situation. Hillsdale, NJ: Erlbaum.

Blumer, H. (1969). Symbolic interactionism: Perspective and method. Berkeley: University of California Press.Bryman, A. (1984). Quantity and quality in social research. London: Unwin Hyman.Bryman, A. (1988). Quantity and quality in social research. London: Routledge.Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on

teaching. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171–246). Chigago: RandMcNally.

Cook, T. D., & Campbell, D. T. (1979). Quasi experimentation: Designs and analysis issues for fieldsettings. Boston: Houghton Miffli.

Cooksy, L. J., Gill, P., & Kelly, P. A. (2001). The program logic model as an integrative framework for amultimethod evaluation. Evaluation and Program Planning, 24, 119–128. doi:10.1016/S0149-7189(01)00003-9.

Creswell, J. W. (2005). Educational research: Planning, conducting, and evaluating quantitative andqualitative research (2nd ed.). Upper Saddle River, NJ: Pearson Education.

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting. Mixed methods research.Thousand Oaks. CA: Sage.

Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methodsresearch designs. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social andbehavioral research (pp. 209–204). Thousand Oaks, CA: Sage.

Danziger, K. (1985). The methodological imperative in psychology. Philosophy of the Social Sciences, 15,1–13. doi:10.1177/004839318501500101.

Denzin, N. K., & Lincoln, S. L. (Eds.). (2005). The Sage handbook of qualitative research (3rd ed).Thousand Oaks, CA: Sage.

Dilthey, W. (1989). Introduction to the human sciences: An attempt to lay a foundation for the study ofsociety and history. In R. A. Makkreel, & F. Rodi (Eds.), Selected works. Princeton, NJ: PrincetonUniversity Press.

Ellingson, S. A., Miltenberger, R. G., Stricker, J. M., Garlinghouse, M. A., Roberts, J., & Rapp, J. T.(2000). Analysis and treatment of finger sucking. Journal of Applied Behavior Analysis, 33, 41–52.doi:10.1901/jaba.2000.33-41.

Elliott, M. S., & Williams, D. I. (2002). A qualitative evaluation of an employee counselling service fromthe perspective of client, counsellor and organization. Counselling Psychology Quarterly, 15(2), 201–208. doi:10.1080/09515070210128991.

Evans, L., & Hardy, L. (2002a). Injury rehabilitation: a goal-setting intervention study. Research Quarterlyfor Exercise and Sport, 73, 310–319.

Evans, L., & Hardy, L. (2002b). Injury rehabilitation: a qualitative follow-up study. Research Quarterlyfor Exercise and Sport, 73, 320–329.

Geertz, C. (1973). The interpretation of cultures: Selected essays. New York: Basic Books.Goldenberg, C., Gallimore, R., & Reese, L. (2005). Using mixed methods to explore Latino children’s

literacy development. In T. S. Weisner (Ed.), Discovering successful pathways in children’sdevelopment: Mixed methods in the study of childhood and family life (pp. 283–303). Chicago:Chicago University Press.

Greene, J. C. (2000). Challenges in practicing deliberative democratic evaluation. In K. R. Ryan, & L.DeStefano (Eds.), Evaluation as a democratic process: Promoting inclusion, dialogue anddeliberation New Directions for Evaluation, No. 85 (pp. 13–26). San Francisco: Jossey-Bass.

Greene, J. C., & Caracelli, V. J. (2003). Making paradigmatic sense of mixed methods practice. In A.Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.91–110). Thousand Oaks, CA: Sage.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.Guba, E. G., & Lincoln, Y. S. (2005). Paradigm controversies, contradictions, and emerging confluences.

In N. K. Denzin, & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (pp. 191–215, 3rded.). Thousand Oaks, CA: Sage.

Hilliard, R. B. (1993). Single-case methodology in psychotherapy process and outcome research. Journalof Consulting and Clinical Psychology, 61, 373–380. doi:10.1037/0022-006X.61.3.373.

Howe, K. R. (1988). Against the quantitative–qualitative incompatibility thesis or dogmas die hard.Educational Researcher, 17(8), 10–16.


http://dx.doi.org/10.1016/S0149-7189(01)00003-9

http://dx.doi.org/10.1016/S0149-7189(01)00003-9

http://dx.doi.org/10.1177/004839318501500101

http://dx.doi.org/10.1901/jaba.2000.33-41

http://dx.doi.org/10.1080/09515070210128991

http://dx.doi.org/10.1037/0022-006X.61.3.373

Johnson, R. B., & Christensen, L. B. (2000). Educational research: Quantitative and qualitativeapproaches. Boston: Allyn and Bacon.

Johnson, R. B., & Turner, L. A. (2003). Data collection strategies in mixed methods research. In A.Tashakkori, & C. Teddlie (Eds.), Handbook on mixed methods in the behavioral and social sciences(pp. 297–320). Thousand Oaks, CA: Sage.

Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York:Oxford University Press.

Kemper, E., Stringfield, S., & Teddlie, C. (2003). Mixed methods sampling strategies in social scienceresearch. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioralresearch (pp. 273–296). Thousand Oaks, CA: Sage.

Köckeis-Stangl, E. (1980). Interpretative Methoden kontrollierten Fremdverstehens. In K. Hurrelmann, &D. Ulrich (Eds.), Handbuch der Sozialisationsforschung. Weinheim: Beltz.

Krantz, D. L. (1995). Sustaining versus resolving the quantitative–qualitative debate. Evaluation andProgram Planning, 18, 89–96. doi:10.1016/0149-7189(94)00052-Y.

Lamiell, J. T. (1998). Nomothetic and idiographic: Contrasting Windelband’s understanding withcontemporary usage. Theory & Psychology, 8(1), 23–38.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard Educational Review,

62(3), 279–300.Maxwell, J. A., & Loomis, D. M. (2003). Mixed methods design: An alternative approach. In A.

Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research.Thousand Oaks, CA: Sage.

May, D. B., & Etkina, E. (2002). College physics students’ epistemological self-reflection and its relationshipto conceptual learning. American Journal of Physics, 70(12), 1249–1258. doi:10.1119/1.1503377.

Michell, J. (1999). Measurement in psychology: A critical history of a methodological concept.Cambridge: Cambridge University Press.

Michell, J. (2000). Normal science, pathological science and psychometrics. Theory & Psychology, 10(5),639–667. doi:10.1177/0959354300105004.

Miller, D. (1994). Critical rationalism. A restatement and defence. Chicago and La Salle: Open Court.Moran, D. (2000). Introduction to phenomenology. London: Routledge.Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori, &

C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 189–208).Thousand Oaks, CA: Sage.

Newman, I., & Benz, C. R. (1998). Qualitative–quantitative research methodology: Exploring theinteractive continuum. Carbondale: Southern Illinois University Press.

Newman, I., Ridenour, C. S., Newman, C., & DeMarco, G. M. P. (2003). A typology of research purposeand its relationship to mixed methods. In N. K. Denzin, & Y. S. Lincoln (Eds.), The Sage handbook ofqualitative research (pp. 167–188, 3rd ed.). Thousand Oaks, CA: Sage.

Noblitt, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park,CA: Sage.

Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods research. InA. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research(pp. 351–383). Thousand Oaks, CA: Sage.

Pagano, M. E., Hirsch, B. J., Deutsch, N. L., & McAdams, D. P. (2002). The transmission of values toschool-age and young adult offspring: race and gender differences in parenting. Journal of FeministFamily Therapy, 14, 13–36. doi:10.1300/J086v14n03_02.

Patton, M. Q. (1988). Paradigms and pragmatism. In D. M. Fetterman (Ed.), Qualitative approaches toevaluation in education: The silent scientific revolution (pp. 116–137). New York: Praeger.

Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.Polgar, S., & Thomas, S. (2000). Introduction to research in the health sciences (4th ed.). Churchill

Livingstone: Edinburgh.Punch, K. F. (1998). Introduction to social research: Quantitative and qualitative approaches. London:

Sage.Putnam, H. (1990). Realism with a human face. Cambridge: Harvard University Press.Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus quantitative methods. In T. D. Cook, &

C. S. Reichardt (Eds.), Qualitative and quantitative methods in evaluation research (pp. 7–32).Beverly Hills, CA: Sage.

Rosenberg, A. (1988). Philosophy of social science. Boulder, CO: Westview.


http://dx.doi.org/10.1016/0149-7189(94)00052-Y

http://dx.doi.org/10.1119/1.1503377

http://dx.doi.org/10.1177/0959354300105004

http://dx.doi.org/10.1300/J086v14n03_02

Sandelowsky, M. (2003). Tables or tableux? The challenges of writing and reading mixed methods studies.In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research(pp. 321–350). Thousand Oaks, CA: Sage.

Sayer, A. (2000). Realism and social science. Thousand Oaks, CA: Sage.Sieber, S. D. (1973). The integration of field work and survey methods. American Journal of Sociology,

78, 1335–1359. doi:10.1086/225467.Silverman, D. (2004). Qualitative research: Theory, method and practice (2nd ed.). London: Sage.Smith, J. K. (1983). Quantitative versus qualitative research: an attempt to clarify the issue. Educational

Researcher, 12(3), 6–13.Smith, J. A. (Ed.). (2003). Qualitative psychology: A practical guide to research methods.(2nd ed).

London: Sage.Smith, J. K., & Heshiusius, L. (1986). Closing down the conversation: the end of the quantitative–

qualitative debate among educational researchers. Educational Researcher, 15(4), 4–12.Steckler, A., McLeroy, K. R., Goodman, R. M., BIrd, S. T., & McCormick, L. (1992). Toward integrating

qualitative and quantitative methods: An introduction. Health Education Quarterly, 19(1), 1–8.Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative

approaches. Thousand Oaks, CA: Sage.Tashakkori, A., & Teddlie, C. (2003a). The past and the future of mixed methods research: From data

triangulation to mixed model designs. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixedmethods in social and behavioral research (pp. 671–701). Thousand Oaks, CA: Sage.

Tashakkori, A., & Teddlie, C. (Eds.). (2003b). Handbook of mixed methods in social and behavioralresearch. Thousand Oaks, CA: Sage.

Todd, Z., Nerlich, B., & McKeown, (2004). Introduction. In Z. Todd, B. Nerlich, S. McKeown, & D.Clarke (Eds.), Mixing methods in psychology: The integration of qualitative and quantitative methodsin theory and practice (pp. 3–16). Hove & New York: Psychology Press.

Toomela, A. (2008). Variables in psychology: a critique of quantitative psychology. IntegrativePsychological & Behavioral Science, 42, 3. doi:10.1007/s12124-008-9059-6.

Webb, D. A., Sweet, D., & Pretty, I. A. (2002). The emotional and psychological impact of mass casualtyincidents on forensic odontologists. Journal of Forensic Sciences, 47(3), 539–541.

Westermann, R. (1987). Wissenschaftstheoretische Grundlagen der experimentellen Psychologie. In G.Lüger (Ed.), Allgemeine experimentelle psychologie (pp. 5–42). Stuttgart: Fischer.

Dr. Omar Gelo is Assistant Professor in the department of Psychotherapeutic Sciences and Co-coordinator of the Doctoral Program in Psychotherapeutic Sciences for foreign students at Sigmund FreudUniversity, Vienna. His research interests in the field of psychotherapy research concern the therapeuticprocess, with particular relevance of metaphorical language, emotional-cognitive regulation, and theapplication of dynamic systems theory to the study of psychotherapy. He is moreover interested in linkingprocess and outcome in different psychotherapeutic orientations.

Dr. Diana Braakmann is Assistant Professor in the department of Psychotherapeutic Sciences atSigmund Freud University, Vienna. She is psychologist and behaviour therapist with a specific training indialectic behaviour therapy. Her psychotherapeutic work during the last years was concentrated on treatingBorderline Personality Disorder and Posttraumatic Stress Disease. Her research interests focus on thephenomenon of dissociation as well as the connection between process and outcome variables inpsychotherapy.

Prof. Gerhard Benetka studied psychology, history, sociology, and philosophy at the University ofVienna, obtaining his Master degree in Psychology in 1989, PhD in Psychology in 1994, and habilitationof Psychology in 1998 at the University of Vienna. He is now Prof. of Psychology and Head of Institute ofPsychology at the Sigmund Freud University, Vienna. His research interests focus on history ofpsychology and psychoanalysis.


http://dx.doi.org/10.1086/225467

http://dx.doi.org/10.1007/s12124-008-9059-6

gelo, braakmann & benetka (2008) beyond the debate of quantitative-qualitative

Documents