introduction to qualitative research-merriam 2002

,~. 1 ,~.Introduction to

Qualitative ResearchSharan B. Merriam

Drawing from a long tradition in anthropology, sociology, and clinical psy-chology, qualitative research has, in the last twenty years, achieved sta-tus and visibility in the social sciences and helping professions. Reports

of qualitative research studies can be found at conferences, on the World WideWeb, and in journals in social work, nursing, counseling, family relations, ad-ministration, health, community services, management, all subfields of educa-tion, and even medicine. In addition, there are numerous methodological textson qualitative research available in fields as disparate as gerontology (Reinharz& Rowles, 1988) and organizational science and management (Lee, 1999).

What is the nature of qualitative inquiry that it has captured the attention ofso many? The purpose of this chapter is to explain what qualitative research is,how it differs from the more familiar positivist or quantitative research, whatvariations exist within the qualitative paradigm itself, and how one goes aboutconducting a qualitative study. This chapter and the following chapter on eval-uating and assessing qualitative research offer the backdrop for exploring thecollection of qualitative studies and author commentaries that follow.

The Nature of Qualitative Research

The key to understanding qualitative research lies with the idea that meaning issocially constructed by individuals in interaction with their world. The world, orreality, is not the fixed, single, agreed upon, or measurable phenomenon that it

assumed to be in positivist, quantitative research. Instead, there are multiple

4 Qualitative Research in Practice

constructions and interpretations of reality that are in flux and that change overtime. Qualitative researchers are interested in understanding what those inter-pretations are at a particular point in time and in a particular context. Learninghow individuals experience and interact with their social world, the meaning ithas for them, is considered an interpretive qualitative approach. If you were in-terested in studying the placement of a child in foster care, for example, you mightfocus on understanding the experience from the perspective of the child, the fos-ter family, the agency involved, or all three.

Drawing from critical social theory, you might also investigate how the socialand political aspects of the situation shape the reality; that is, how larger con-textual factors affect the ways in which individuals construct reality. This wouldbe a criticalqualitative approach. Using the same example of placement of a childin foster care, from a critical qualitative perspective you would be interested inhow the social institution of the placement agency, or the foster family, is struc-tured such that the interests of some members and classes of society are servedand perpetuated at the expense of others. Whose interests are being served bythis placement? How do power, privilege, and oppression play out? Critical socialscience research has its own variations. Much of feminist research draws from

critical theory, as does participatory or participatory action research, a form ofresearch that involves participants in the design and implementation of a study.Some critical research incorporates a strong emancipatory agenda along with cri-tique; that is, the overall objective is to empower participants in the process ofconductingthe investigation. .

Another, more recent, philosophical stance is called postmodem or poststmc-tural. Here researchers question all aspects of the construction of reality, what itis and what it is not, how it is organized, and so on. As Bruner (1993, p. 1) writes,meaning is "radically plural, always open, and . . . politics [is] in every account."For example, a poststructural inquiry would question and "disrupt" the dichoto-mies (for example foster-nonfoster family, child-adult) inherent in the researchproblem above. Lather (1992) lays out these three overarching theoretical per-spectives in terms of understanding (interpretive), emancipation (critical andfeminist are included here), and deconstruction (postmodern). Although I haveincluded examples of critical and postmodern studies in this volume, the emphasisis on interpretive qualitative research studies.

As a qualitative researcher, you can approach an investigation from any of thephilosophical or theoretical stances outlined above. Your particular stance willdetermine the specific research design that you employ for actually carrying outyour study. If your primary interest is in understanding a phenomenon, you havemany options, the most common being grounded theory, phenomenology, narra-tive, ethnography, case study, or just a basic interpretive study. Critical, feminist,postmodern, and participatory studies all have goals that include but go beyondunderstanding.

Several key characteristics cut across the various interpretive qualitative re-search designs (also called forms, types, or genres by various authors). The firstcharacteristic is that researchers strive to understand the meaning people have

Introduction to Qualitative Research 5

onstructed about their world and their experiences; that is, how do people makesense of their experience? As Patton (1985, p. 1) explains: Qualitative research

'~is.an effort to understand situations in their uniqueness as part of a particular'context and the interactions there. This understanding is an end in itself, so that~itis not attempting to predict what may happen in the future necessarily, but tounderstand the nature of that setting-what it means for participants to be in

:)!hatsetting, what their lives are like, what's going on for them, what their mean-jngs are, what the world looks like in that particular setting. . . . The analysis;trives for depth of understanding.".:'.,Asecond characteristic of all forms of qualitative research is that the researcher'~the primary instrument for data collection and data analysis, Sinceunderstand-'hg is the goal of this research, the human instrument, which is able to be im-.iediatelyresponsive and adaptive, would seem to be the ideal means of collectingIld analyzing data. Other advantages are that the researcher can expand his or

ierunderstanding through nonverbal as well as verbal communication, process'~formation (data) immediately, clarify and summarize material, check with re-pondents for accuracy of interpretation, and explore unusual or unanticipated!sponses.

.H(,However,the human instrument has shortcomings and biases that might haveimpact on the study. Rather than trying to eliminate these biases or "subjec-

vities," it is important to identify them and monitor them as to how they may.~ shaping the collection and interpretation of data. Peshkin (1988, p. 18) goesb far as to make the case that one's subjectivities "can be seen as virtuous, for:isthe basis of researchers making a distinctive contribution, one that results:pmthe unique configuration of their personal qualities joined to the data theyave collected."

. : Often qualitative researchers undertake a qualitative study because there is ack of theory or an existing theory fails to adequately explain a phenomenon.lerefore, another important characteristic of qualitative research is that the pro-

:ess is inductive; that is, researchers gather data to build concepts, hypotheses,If theories rather than deductively deriving postulates or hypotheses to be testedas in positivist research), In attempting to understand the meaning a phenom-~nonhas for those involved, qualitative researchers build toward theory fromIbservations and intuitive understandings gleaned from being in the field. 1YP-cally, findings inductively derived from the data in a qualitative study are in the

form of themes, categories, typologies, concepts, tentative hypotheses, and evensubstantive theory,

Finally, the product of a qualitative inquiry is richly descriptive. Words andpiqtures rather than numbers are used to convey what the researcher has learnedabout a phenomenon. There are likely to be descriptions of the context, the par-P-cipantsinvolved, the activities of interest. In addition, data in the form of quotesom documents, field notes, and participant interviews, excerpts from video-

~apes, electronic communication, or a combination thereof are always includedi~ support of the findings of the study. These quotes and excerpts contribute to

tpe descriptive nature of qualitative research.


In summary, qualitative research attempts to understand and make sense of

phenomena from the participant's perspective. The researcher can approach thephenomenon from an interpretive, critical, or postmodern stance. All qualita-tive research is characterized by the search for meaning and understanding, theresearcher as the primary instrument of data collection and analysis, an induc-tive investigative strategy, and a richly descriptive end product.

Distinguishing Among Types of Qualitative ResearchFrom education to anthropology to management science, researchers, students,and practitioners are conducting qualitative studies. It is not surprising, then, thatdifferent disciplines and fields ask different questions and have evolved some-what different strategies and procedures. Writers of qualitative texts have orga-nized the diversity of forms of qualitative research in various ways. Patton (1990),for example, presents ten orientations to qualitative research according to thedifferent kinds of questions researchers from different disciplines might ask.Creswell (1998) has identified five "traditions" -biography, phenomenology,grounded theory, ethnography, and case study. Tesch (1990) lists forty-five ap-proaches divided into designs (for example, case study), data analysis techniques(for example, discourse analysis), and disciplinary orientation (for example,ethnography). Denzin and Lincoln (2000) identify eight research strategies of casestudy, ethnography, phenomenology, grounded theory, biographical, historical,participatory, and clinical. They write that qualitative research "does not belongto a single discipline. Nor does qualitative research have a distinct set of methodsthat are entirely its own" (p. 6).

Given the variety of qualitative research designs or strategies, I have chosen toorganize this resource book around eight of the more commonly used approachesto doing qualitative research: basic interpretive, phenomenology, groundedtheory, case study, ethnography, narrative analysis, critical, and postmodern-poststructural. These and other types of qualitative research do have some at-tributes in common that result in their falling under the umbrella concept of"qualitative." However, they each have a somewhat different focus, resulting invariations in how the research question might be asked, sample selection, datacollection and analysis, and write-up. Following is a short description of each ofthe eight types. More thorough discussions of each type of qualitative research,along with examples and author commentaries, can be found in Part Two.

Basic Interpretive Qualitative Study. A basic interpretive and descriptive qual-itative study exemplifies all the characteristics of qualitative research discussedabove; that is, the researcher is interested in understanding how participants makemeaning of a situation or phenomenon, this meaning is mediated through the re-searcher as instrument, the strategy is inductive, and the outcome is descriptive.In conducting a basic qualitative study, you seek to discover and understand aphenomenon, a process, the perspectives and worldviews of the people involved,or a combination of these. Data are collected through interviews, observations, or

document analysis. These data are inductively analyzed to identify the recurring


patterns or common themes that cut across the data. A rich, descriptive accountof the findings is presented and discussed, using references to the literature thatframed the study in the first place. For example, Levinson and Levinson's (1996)study of women's development is situated in the literature on adult growth anddevelopment. The authors interviewed fifteen homemakers, fifteen corporate busi-

cp.esswomen, and fifteen academics. Findings of women's developmental patterns.parallel their earlier study of male development in which forty men in midlifewere interviewed. Levinson and Levinson found that the basic structure or under-

lying pattern of a woman's life evolves through periods of tumultuous, structure-building phases alternating with stable periods of development.

Phenomenology. Because phenomenology as a school of philosophical thought%Q,nderpinsall qualitative research, some assume that aU qualitative research~';):s.phenomenological,and certainly in one sense it is. However, even though thec" IJlenomenologicalnotions of experience and understanding run through all qual-

ltatiye research, one could also engage in a phenomenological study using itsJwn "tools" or inquiry techniques that differentiate it from other types of qual-

.tive inquiry.cInthe same way that ethnography focuses on culture, a phenomenologicaloy focuses on the essence or structure of an experience. Phenomenologists~interested in showing how complex meanings are built out of simple unitslirect experience. This form of inquiry is an attempt to deal with inner ex-

"iencesunprobed in everyday life. According to Patton (1990), this type of re-:ch is based on "the assumption that there is an essence or essences to shared~rience.. . . The experiences of different people are bracketed, analyzed, andtpared to identify the essences of the phenomenon, for example, the essences

Ion

...

eliness, the essence of being a mother, or the essence of being a partici-tHin a particular program" (p. 70, emphasis in original).

Iliorderto understand the essence or structure of an experience, the researcherllporarilyhas to put aside, or "bracket," personal attitudes or beliefs about the~nomenon. With belief temporarily suspended, consciousness itself becomesght~ned, allowing the researcher to intuit or see the essence of the phenome-.Examples of phenomenological studies include Howard's (1994) study of the~rienceof first-time computer users and Healy's (2001) recent study of insightitation as a transformational learning experience.

unded Theory. It can be argued that Glaser and Strauss' 1967 book, The:pveryof Grounded Theory, launched, or at least was key in the developmentl.tt'fllitativeresearch as a viable research paradigm. The goal of this type of[itative study is to.derive inductively from data a theory that is "grounded"i'e}data-hence, grounded theory. Grounded theory research emphasizes dis-~ty,with description and verification as secondary concerns. Researchers in

bde build substantive theory, which is distinguished from grand or for.leOry.Substantive theory is localized, dealing with particular real-world

tiOM !':l1rh;j~how adults manage school, family, and work life, or what


constitutes an effective counseling program for teen mothers, or how a com-munity allocates its resources.

Data gathered for a grounded theory study are analyzed via the constant com-parative method of data analysis. Other qualitative researchers have adopted thismethod, which involves continually comparing one unit of data with another inorder to derive conceptual elements of the theory, even though they may not bedeveloping theory. This has resulted in some claiming they are doing a groundedtheory study when in fact there is no substantive theory as an outcome of theinquiry. A grounded theory consists of categories, properties, and hypothesesthat state relationships among categories and properties. Unlike hypotheses inexperimental studies, grounded theory hypotheses are tentative and suggestiverather than tested.

Case Study. The case study is an intensive description and analysis of a phe-nomenon or social unit such as an individual, group, institution, or community.The case is a bounded, integrated system (Stake, 1995, Merriam, 1998). By con-centrating upon a single phenomenon or entity (the case), this approach seeks todescribe the phenomenon in depth. The unit of analysis, not the topic of investi-gation, characterizes a case study. For example, a study of women's experiencesin welfare-to-work training programs could be a qualitative study but not a casestudy; the unit of analysis would be the women's experiences, and there could bean indefinite number of women selected for the study. For it to be a case study,one particular program (a bounded system), selected because it was typical,unique, experimental, or highly successful, etc., would be the unit of analysis. Acase study could also be conducted of the experiences of a single woman.

Since it is the unit of analysis that determines whether a study is a case study,this type of qualitative research stands apart from the other types defined here.And in fact, since it is the unit of analysis that defines the case, other types of stud-ies can be and sometimes are combined with case study. Ethnographic case studiesare quite common, for example, wherein the culture of a particular social groupis studied in depth. In addition, one could build grounded theory within a casestudy, or analyze the data from a case study from a critical science perspective, orobtain one person's "story," hence combining narrative with case study, and soon. The examples of case study in Part 1\vo of this book illustrate how the casestudy is a vehicle for in-depth description and analysis.

Ethnographic Study. This form of qualitative research has a long tradition inthe field of anthropology. It was developed by anthropologists specifically tostudy human society and culture. Although culture has been variously defined,it usually refers to the beliefs, values, and attitudes that shape the behavior ofa particular group of people. D'Andrade (1992) writes that culture is somethingbehaviorally and cognitively shared by an identifiable group of people and thatit has "the potential of being passed on to new group members, to exist withsome permanency through time and across space" (p. 230).


Confusion results when the term ethnography is used interchangeably withfieldwork, participant observation, case study, and so on. For a qualitative studyto be an ethnography, it must present a sociocultural interpretation of the data.Therefore, ethnography is not defined by how data are collected, but rather bythe lens through which the data are interpreted. As LeCompte and Preissle (1993)

point out, "ethnographies re-create for the reader the shared beliefs, practices,artifacts, folk knowledge, and behaviors of some group of people" (pp. 2-3). Mostpeople are familiar with ethnographies of foreign and exotic cultures such asMargaret Mead's Coming of Age in Samoa (1973). There are also many ethno-graphies of various social groups within a larger culture, such as Cordeiro andCarspecken's (1993) ethnographic account of twenty successful Hispanic highschool achievers.

Narrative Analysis. The narrative analysis of lives, or life narratives, is currentlya popular form of qualitative research. The key to this type of qualitative researchis the use of stories as data, and more specifically, first-person accounts of ex-perience told in story form. Other terms for this type. of research include biogra-phy, autobiography, life history, oral history, autoethnography, and life narratives.Manning and Cullum-Swan (1994, p. 465) write that "narrative analysis typicallytakes the perspective of the teller, rather than that of the society." Context is im-portant, however, for "if one defines narrative as a story with a beginning, mid-dle, and end that reveals someone's experiences, narratives take many forms, aretold in many settings, before many audiences, and with various degrees of con-nection to actual events or persons" (p. 465).

There are several strategies one can use to do the actual analysis of narrativesor people's stories. The three most common are psychological, biographical, anddiscourse analysis. In the psychological approach, the story is analyzed in termsof internal thoughts and motivations. A more biographical approach attends tothe person in relation to society and takes into account the influences of gender,class, and "family beginnings" (Denzin, 1989, p. 17). Discourse analysis exam-ines the written text of the story for its component parts or assesses the spokenwords by looking for intonation, pitch, and pauses as lens to the meaning of thetext (Gee, 1991). Whatever the approach to analyzing the data, the central defin-ing feature of this type of qualitative research is that the data are in the form ofa story. Part Two contains two examples of narrative analysis.

Critical Qualitative Research. Drawing from critical social science and in par-ticular Habermas' (1972) theory of knowledge, critical qualitative research un-covers, examines, and critiques the social, cultural, and psychological assumptionsthat structure and limit our ways of thinking and being in the world. The ultimateobjective of this type of critique is to free ourselves from these constraints, to be-come empowered to change our social context and ourselves.

Critical research focuses less on individuals than on context. Critical educational

research, for example, queries the context where learning takes place, including

""

.'i


the larger systems of society, the culture and institutions that shape educationalpractice, the structural and historical conditions framing practice. Questions areasked regarding whose interests are being served by the way the educational sys-tem is organized, who really has access to particular programs, who has thepower to make changes, and what are the outcomes of the way in which educa-tion is structured. Critical qualitative research, then, raises questions about theinfluence of race, class, and gender (and their intersections), how power relationsadvance the interests of one group while oppressing those of other groups, andthe nature of truth and the construction of knowledge.

A critical perspective informs other types of research, most commonly par-ticipatory action research (PAR)and some feminist research. PARfocuses uponthe political empowerment of people through group participation in the searchfor and acquisition of knowledge and subsequent action to change the statusquo. Critical feminist research questions and critiques the societal, historical,and cultural assumptions about women that have resulted in their marginal sta-tus compared to men.

Postmodern Research. The most recent development in qualitative research isthe infusion of a postmodern or poststructural perspective. In contrast to the"modern" world, where reality is predictable, research is scientific, and thereare assumed to be universal norms for truth and morality, the postmodern worldis one of uncertainty, fragmentation, diversity, and plurality. There are manytruths, and all generalizations, hierarchies, typologies, and binaries (good/bad,right/wrong, male/female, etc.) are "contested," "troubled," or challenged.

Postmodern research thus challenges the form and categories of traditionalqualitative research. A postmodern research report does not follow a specific for-mat; each has its own rhythm and structure. Data analysis also differs from tra-ditional qualitative research. This has created what Denzin and Lincoln (2000)call a "triple crisis." The first crisis has to do with representation-postmodernresearchers question whether the lived experience of someone else can be cap-tured; "such experience, it is now argued, is created in the social text written bythe researcher" (p. 17). The second crisis has to do with being able to evaluatepostmodern research. What makes a study valid and reliable if traditional qual-itative criteria are inadequate? Because postmodern research is so experimentaland each study unique, there are few if any guidelines about how to do this typeof study, or how to assess its trustworthiness. The third crisis has to do with so-cial action. "If society is only and always a text" (p. 17), how can participatoryaction research, for example, bring about change?

.To summarize this brief overview of the different designs or types of qualitativeresearch, we see that the eight chosen for review vary widely in form and pur-pose. Not all qualitative research is the same; neither can terms such as "grounded

theory," "ethnography," "narrative analysis," and so on be used interchangeably.

---


However, because of the underlying view of reality and the focus on understand-ing and meaning, the forms of qualitative research reviewed here have some char-acteristics in common that allow them to be categorized as "qualitative." A moredetailed discussion of these types can be found in Part Two, along with examplesand author commentaries.

The Design of a Qualitative StudyThe design of a qualitative study focused on interpretation includes shaping aproblemfor this type of study, selectinga sample,collectingand analyzing data,and writing up the findings. An understanding of this process is important forassessingthe rigor and value of individualreports of research (see Chapter '!Woformore discussionon evaluatingand assessingqualitativeresearch). Presentedhere is a brief overview of the component parts of the process of conducting aqualitativeresearch study.

The Research Problem and Sample Selection. A research study begins with yourbeing curious about something, and that "something" is usually related to yourwork, your family, your community, or yourself. A research problem can alsocome from social and political issues of the day or from the literature. Often thesespheres intersect. For example, perhaps you work for a social service agency thatassists the homeless in becoming stabilized in their housing needs. Your work isvery much about a pressing social problem. Or you might have observed howcomfortable your children are with computers and you wonder how people yourage are learning to function in this technological age. In any case, the place to"look" for a research problem is in your everyday experience-ask questionsabout it, be curious as to why things are as they are or how they might be better.

The types of questions that you ask are key to doing a qualitative study. Mar-shall and Rossman (1995) suggest that qualitative research is designed to (1) un-derstand processes, (2) describe poorly understood phenomena, (3) understanddifferences between stated and implemented policies or theories, and (4) dis-

, COverthus far unspecified contextual variables. If you want to understand a phe-.nomenon, uncover the meaning a situation has for those involved, or delineateprocess (how things happen), then a qualitative design would be most appro-priate. For example, with regard to the first topic above, you might ask what theexperience of being homeless is really like, or you might ask what the necessarysteps or stages are in the transition process of moving from homelessness to sta-ble housing. Or from a more critical perspective, you could ask how the social

~rviceagency for which you work reinforces, challenges, or mediates the prob-m of homelessness. Does it further oppress the homeless with its rules and reg-ltions, or does it empower individuals to act on their behalf?The basic question of your study is set within what is called the problem state-

mt. In crafting the research problem, you move from general interest, curios-, or doubt about a situation to a specific statement of the research problem. In'~ct,you have to translate your general curiosity into a problem that can be ad-

_L~ ~ -_n~n +~~ tho


general topic of interest, including key concepts, what has already been studied,why it's an important topic, to the specific question that you have. This specificquestion is most often written as a purpose statement and addresses some gapin the knowledge base on that topic (if previous research has already providedan answer to your question, there is no need to do the study). Using the home-less topic above, one purpose statement might read: "The purpose of this study isto understand the experience of being homeless," or "The purpose of this studyis to identify the process of moving from homelessness to stable housing," or,"The purpose of this study is to uncover how a social service agency both rein-forces and challenges the state of homelessness of its clients." (For more on prob-lem formation, see Merriam, 1998;Merriam&Simpson,2000).

The next step in the design of a qualitative study is to select a sample fromwhich you will collect data. For nearly every study there exist sites that could bevisited, people who could be interviewed, documents that could be read and ana-lyzed. How do you select which sites, people, and documents to be included inyour study? To begin with, since you are not interested in "how much" or "howoften," random sampling makes little sense. Instead, since qualitative inquiry seeksto understand the meaning of a phenomenon from the perspectives of the partic-ipants, it is important to select a sample from which the most can be learned. Thisis called a purposive or purposeful sample. Patton (1990) argues that it is impor-tant to select "information-rich cases for study in depth. Information-rich cases arethose from which one.can learn a great deal about issues of central importance tothe purpose of the research, thus the term purposeful sampling" (p. 169, empha-sis in original). Tobegin purposive sampling, you first determine what criteria areessential in choosing who is to be interviewed or what sites are to be observed.In the study of the experience of homelessness, for example, you would first de-cide whether men and women would be included, what age range would be im-portant, the length of homelessness, and so on.

Data Collection and Analysis. There are three major sources of data for a quali-tative research study-interviews, observations, and documents. The data collec-tion strategy used is determined by the question of the study and by determiningwhich source(s) of data will yield the best information with which to answer thequestion. Often there is a primary method of collecting data with support from an-other. Sometimes only one method is used. For example, in studying how a socialservice agency both reinforces and challenges the status quo of the homeless, youmight interview both homeless people and staff of the agency, conduct observa-tions of the daily operation of the agency, and study internal and external agencydocuments. However, if you were most interested in the experience of homeless-ness, interviews with those who are or have been homeless would yield the mostrelevant information. If at all possible, researchers are encouraged to use morethan one method of data collection as multiple methods enhance the validity of

the findings.Interviews range from highly structured, where specific questions and the

order in which they are asked are determined ahead of time, to unstructured,

Introduction 10 Qualitative Research 13

where one has topic areas to explore but neither the questions nor the order arepredetermined. Most interviews fall somewhere in between. The semistructuredinterview contains a mix of more and less structured questions. Usually, spe-cific information is desired from all the participants; this forms the highly struc-tured section of the interview. The largest part of the interview is guided by alist of questions or issues to be explored, and neither the exact wording nor theorder of the questions is determined ahead of time.

A second major means of collecting data is through observation. Observationaldata represent a firsthand encounter with the phenomenon of interest rather thana secondhand account obtained in an interview. Like interviewing, there is a rangehere also from being a complete observer to being an active participant. A com-plete observer is unknown to those being observed, such as from behind a one-way mirror or in an open, public place. A very active participant observer mightbe someone who is a member of the group or organization who is thus partici-pating while observing. When observation is used in conjunction with interview-ing, the term fieldwork or field study is sometimes used. Observation is the besttechnique when an activity, event, or situation can be observed firsthand, whena fresh perspective is desired, or when participants are not able or willing to dis-cuss the phenomenon under study.

The third major source of data is documents. These can be written, oral, visual(such as photographs), or cultural artifacts. Public records, personal documents,and physical material are types of documents available to the researcher for anal-ysis. The strength of documents as a data source lies with the fact that they al-ready exist in the situation; they do not intrude upon or ci1terthe setting in waysthat the presence of the investigator might. Nor are they dependent upon thewhims of human beings whose cooperation is essential for collecting data throughinterviews and observations. Entire studies can be built around documents. For

example, Abramson's (1992) case study of Russian Jewish emigration is basedsolely on his grandfather's diaries written over a twelve-year period. In contrastto documents already present in the research setting, researcher-generated docu-ments are prepared at the request of the researcher after the study has begun. Par-ticipants might be asked to keep a diary or a log of their activities relevant to thephenomenon being studied, take pictures, write a life history, and so on. Whetherpreexisting or researcher-generated, documents often contain insights and cluesinto the phenomenon, and most researchers find them well worth the effort to lo-cate and examine.

Interviews, observations, and documents are the three traditional sources ofdata in a qualitative research study. With the advent of computer technology andthe World Wide Web, data can also be collected on-line. Web pages, papers avail-able on-line, and so on can be considered documents simply accessed on-line;artifacts in the form of illustrations and games can be downloaded; interviews canbe conducted bye-mail; and researchers can "observe" on-line chat rooms andother forms of interaction. On-line data collection to some extent offers an elec-

tronic extension of familiar data-gathering techniques. However, the medium af-fects the naluTP of thp £i:>b""11,,+,.,-1{-- -- ,,- ~. . -- - --


an in-person interview). As with any form of data collection, researchers need tobe cognizant of the characteristics of each strategy and how those characteristicsshape the nature of the data collected.

In qualitative research, data analysis is simultaneous with data collection. Thatis, one begins analyzing data with the first interview, the first observation, thefirst document accessed in the study. Simultaneous data collection and analysisallows the researcher to make adjustments along the way, even to the point ofredirecting data collection, and to "test" emerging concepts, themes, and cate-gories against subsequent data. To wait until all data are collected is to lose theopportunity to gather more reliable and valid data; to wait until the end is also tocourt disaster, as many a qualitative researcher has found himself or herself fac-ing hundreds of pages of transcripts or field notes without a clue where to begin.

With that caveat in mind, data analysis is essentially an inductive strategy.One begins with a unit of data (any meaningful word, phrase, narrative, etc.)and compares it to another unit of data, and so on, all the while looking for com-mon patterns across the data. These patterns are given names (codes) and arerefined and adjusted as the analysis proceeds.

Although all qualitative data analysis is inductive, different theoretical stancesand different disciplines have evolved particular strategies for data analysis. In anethnographic study, for example, an organizing scheme or typology of categoriesmight be used such as Lofland and Lofland's (i99S) four broad categories for or-ganizing aspects of society (economy, demographics, social structures, the envi-ronment). Narrative analysis might .employpsychological, literary, or sociolinguisticdata analysis strategies. Many qualitative researchers have adopted the constantcomparative method, originally used for developing grounded theory, whether ornot they are seeking to build substantive theory. In a phenomenological study, spe-cific techniques such as epoche, bracketing, imaginative variation, and so on areused to analyze experience. In a postmodern-poststructural study, new forms ofdata analysis are being developed such as deconstruction, rhizoanalysis, geneal-ogy, archaeology, and schizoanalyses (Elizabeth St. Pierre, personal communica-tion, February 2001). These and other data analysis strategies are addressed in abit more detail under the appropriate sections of Part Thlo. However, for more de-tailed discussions, readers are encouraged to refer to some of the resources listedhere and in Part Thlo.

Writing Up Qualitative Research. There is no standard format for reportingqualitative research. Rather, as can be seen from a quick glance at the sixteenreports of qualitative research in Part Thlo of this book, there is a diversity ofstyles, some of which are quite creative. Although not addressed in this book,the presentation of qualitative findings can be through media other than print

(for ~xample, drama, dance, film).In any write-up of qualitative research, what does need to be considered is

the audience for the report. A funding agency or the general public may want


~owthe study was conducted. But colleagues and other researchers will want,detailed description of the methodology in order to assess the study's contri-mtionto the field.'

" Althoughthe relativeemphasis giveneach sectionas well as the overallformIfthe report can vary widely, all write-ups of qualitative research contain at the'ery minimum a discussion of the research problem, the way the investigation

,.,\Tasconducted, and the findings, including a discussion of their importance or rel-i', evance to theory and practice. Since findings are in the form of words rather than:;@\j1urnbers,reports vary widely with regard to the ratio of supporting "raw" data in-~'cluded versus interpretation and analysis. The best guideline is whether enough~r:datain the form of quotes from interviews, episodes from field observations, or~'id.ocumentaryevidence are presented to support adequately and convincingly the~'study's findings. In qualitative research, it is the rich, thick descriptions, the wordsTScCnotnumbers) that persuade the reader of the trustworthiness of the findings.}"'Nevertheless, in any report, there is tension between having the right amount of;; §upporting data versus analysis and interpretation. Another problem is finding the<right voice to present the findings. Write-ups can vary from intimate, first-person

,accounts to more formal presentations to creative experimentation wherein the,,' text is divided by perspective. For example, Lather and Smithies (1997) use a, ~plit-text format wherein the participants' words are presented on the same page~'~para11elto the researchers' interpretation. In another example, Wolf (1992) pre-" sents the same tale of a woman shaman in Taiwan as a short story, as field notes,,..and as an academic journal article.

, Summary

chapter has presented an introductory overview of qualitative research.Qualitative research is an umbrella term that encompasses several philosophi-cal or theoretical orientations, the most common being interpretive, critical, andpostmodern. There are also several designs, types, or genres of qualitative re-search, including a basic interpretive study, phenomenology, grounded theory,case study, narrative analysis, ethnography, critical qualitative research, andpostmodern or poststructural research. All these types of qualitative researchhave in common the search for meaning and understanding, the researcher asthe primary instrument of data collection and analysis, an inductive analysisprocess, and a product that is a rich description of the phenomenon.

Also reviewed in this chapter are the phases of a qualitative research process.One must first shape a research problem that is appropriate for qualitative in-quiry. Next, a purposeful sample is chosen from which data are collected. Thethree primary sources of data are interviews, observations, and documents. Asdata are being collected, data analysis is ongoing and simultaneous. There are avariety of data analysis strategies that can be employed, depending upon the typeof qualitative study. Finally, it is important to present the findings of the study


in a format appropriate to the audience. It is only through the presentation anddissemination of the study's findings that a contribution can be made to theknowledge base of a field and to practice.

References

Abramson, P. R. (1992). A case for case studies. Thousand Oaks, CA: Sage.

Bruner, E. M. (1993). Introduction: The ethnographic self and the personal self.In P. Benson (Ed.), Anthropology and literature (pp. 1-26). Urbana: University ofIllinois Press.

Cordeiro, P. A., & Carspecken, P. F. (1993). How a minority of the minority succeed:A case study of twenty hispanic achievers. Qualitative Studies in Education, 6(4),277-290.

Creswell, J. W. (1998). Qualitative inquiry and research design. Thousand Oaks, CA:Sage.

D'Andrade, R. G. (1992). Afterword. In R. G. D'Andrade & C. Strauss (Eds.), Human

motives and cultural models. Cambridge, England: Cambridge University Press.

Denzin, N. K. (1989). Interpretive biography. Newbury Park, CA: Sage.

Denzin, N. K., & Lincoln, Y.S. (2000). Introduction: The discipline and practice ofqualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualita-tive research (pp. 1-28). Thousand Oaks, CA: Sage.

Gee, J. P. (1991). A linguistic approach to narrative. Journal of Narrative and Ufe His-tory, 1(1), 15-39. .

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. Chicago: AIdine.

Habermas, J. (1972). Knowledge and human interests. Portsmouth, N.H.: HeinemannEducational Books.

Healy, M. F. (2001). The transformationalleaming process within insight (vipassana)meditation: A phenomenological study. Unpublished doctoral dissertation, Depart-ment of Adult Education, The University of Georgia, Athens.

Howard, D.C.P. (1994). Human-computer interactions: a phenomenological examina-tion of the adult first-time computer experience. Qualitative Studies in Education,7(1),33-49.

Lather, P. (1992, Spring). Critical frames in educational research: Feminist and post-structural perspectives. Theory into Practice, 31(2), 87-99.

Lather, P., & Smithies, C. (1997). 1hJubling the angels: Women living with HIVjAIDS.Boulder, CO: Westview.

LeCompte, M. D., & Preissle, J., with Tesch, R. (1993). Ethnography and qualitativedesign in educational research. (2nd ed.) Orlando, FL: Academic Press.

Lee, T. W. (1999). Using qualitative methods in organizational research. ThousandOaks, CA: Sage.

Levinson, D. J., & Levinson, J. D. (1996). The seasons of a woman's life. New York:Ballantine.

Lofland, J., & Lofland, L. H. (1995). Analyzing social settings: A guide to qualitativeobservation and analysis. (3rd ed.) Belmont, CA: Wadsworth.


Manning, P. K., & Cullum-Swan, B. (1994). Narrative, content, and semiotic analysis.In N. K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp. 463-477). Thousand Oaks, CA: Sage.

. Marshall, C., & Rossman, G. B. (1995). Designing qualitative research. (2nd ed.)Thousand Oaks, CA: Sage.

Mead, M. (1973). Coming of age in samoa. (6th ed.) New York: Morrow Hill.

Merriam, S. B. (1998). Qualitative research and case study applications in education.(2nd ed.) San Francisco: Jossey-Bass.

Merriam, S. B., & Simpson, E. L. (2000). A guide to research for educators and trainersof adults. (2nd ed., updated) Malabar, FL: Krieger Publishing Company.

Patton, M. Q. (1985, April). Quality in qualitative research: Methodological principlesand recent developments. Invited address to Division J of the American Educa-tional Research Association, Chicago.

Patton, M. Q. (1990). Qualitative evaluation methods. (2nd ed.) Thousand Oaks, CA:Sage.

Peshkin, A. (1988). In search of subjectivity-one's own. Educational Researcher,17(7),17-22.

Reinharz, S., & Rowles, G. D. (Eds.) (1988). Qualitative gerontology. New York:Springer.

Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.

Tesch, R. (1990). Qualitative research: Analysis types and software tools. New York:Falmer Press.

Wolf, M. A. (1992). A thtice-told tale: Feminism, postmodemism, and ethnographicresponsibility. Stanford, CA: Stanford University Press.

,~., 2 ,~.,Assessing and Evaluating

Qualitative ResearchSharan B. Merriam

Assume for a moment that you are the director of human resource devel-opment for your company and you notice that men seem to get promotedmore readily than women. Or that you are the principal of a high school

with a diverse student body and you have data showing that minority studentshave a higher dropout rate than white, middle-class students. For either of thesequestions you could turn to research studies to better understand the problemand to get ideas for what you might do to change the situation. However, beforeyou implement any changes based on what you discover in the research, youwant to be certain that the changes will help, not exacerbate the situation. Howwill you know which research results are trustworthy? Which studies were donewell? Which changes to implement? These questions are especially important toprofessionals in applied fields such as education and training where practition-ers intervene in people's lives. Lincoln and Guba (2000, p. 178) underscore thispoint by asking whether a study's findings are "sufficiently authentic. , . that Imay trust myself in acting on their implications? More to the point, would I feelsufficiently secure about these findings to construct social policy or legislationbased on them?"

In other words, what constitutes a "good" or "quality" qualitative study, onethat can be trusted should we want to make use of the knowledge generated bythe study? This chapter outlines what to look for in general terms in evaluatingand assessing a qualitative research study. This section is followed by a morespecific discussion on assessing the validity and reliability of qualitative research,including how ethical issues are inextricably intertwined with the trustworthi-ness of the findings.

Assessing and Evaluating Qualitative Research

0'\Vhatto Look for When Evaluating Qualitative Research

,~\Itis impossible to imagine a person leading a life without making judgments;pf;without making discriminations," write Smith and Deemer (2000, p. 888).;rhey go on to point out that "in our roles as inquirers, educators, evaluators,weare always making judgments about papers for publication, presentations,books, dissertations, student papers, and so on, As we approach judgment in

f\~riygiven case, we have in mind or bring to the task a list, . . of characteristics\!1hMwe use to judge the quality of that production" (p. 888). Systematically eval-f~gAtingor critiquing a qualitative study involves considering the overall design~~fthe study, as well as the rigor with which the study was conducted. Drawing~"ffommaterial presented in Chapter One, the following discussion presents "char-*~cteristics" or factors to consider in evaluating specific studies, emphasizingt!@alitative studies that are interpretive rather than critical or postmodern. It is

nportant to note that not all factors will apply equally to all studies.;:The first question to ask is whether the problem is appropriate for qualitative\<1uiry.Recall that this type of research is designed to uncover or discover theleaningspeople have constructed about a particular phenomenon. The researcher:fU1tsto obtain an in-depth understanding of a phenomenon, an individual, a sit-

~

,

ti,

'

,

'

",

on,

. Qualitative researchers are not interested in people's surface opinions as

\survey research, or in cause and effect as in experimental research; rather, they:ant to know how people do things, and what meaning they give to their lives.'uestions of meaning, understanding, and process are appropriate for qualitative"search. One should look for a clear statement of the purpose of the study. Thisioften found in the introductory material or in a section identified as the prob-in statement.

:The problem of the study also needs to be situated in the literature. What re-ITchand theory does this study draw upon? What is the theoretical framework

here the topic is anchored? What theory or literature does it seek to extend andform?What do we already know about the phenomenon, and what is the gap'.our knowledge?Asecond question one could ask is how significant the problem is. The author'eds to make a case that there is some gap in our knowledge about a particularlenomenon, and it is important to answer the questions raised by the study.

his is done by informing the reader of what is known, usually through a review(previous literature and research. Then a case has to be made for the impor-

tce of this particular research. How will this knowledge make a contribution in~world? Who will benefit and in what ways? There should be some sense of

gency surrounding the issue. The mere fact that this topic has not been inves-ated before does not in and of itself justify doing the research; maybe there's:needto know the answers. In applied fields like education and management,example, research is often undertaken for the expressed purpose of improvingtCtlce.How might someone make use of the findings of the study? Who in par-lIar would be most interested? Will this research help someone make better

19


decisions, plan programs, teach, develop policies, engage in social action, em-power others?

In setting up the problem of the study, some authors also identify their par-ticular interest in the topic, their assumptions and biases. In qualitative researchreports it is becoming accepted practice for the researcher to explain his or herperspective on and relationship to the problem. This is sometimes embedded inthe introduction and problem statement, and sometimes it is found in the meth-ods section discussed below.

Moving beyond the problem and significance of the study, we can take a closelook at the methods section of the report. Does the author identify which type ofqualitative research design being used? Is this a basic interpretive study? Anethnography? Grounded theory? Narrative analysis? Each of these genres has itsown purpose and strategies. Is this really the type of qualitative study it purportsto be? Qualitative studies are commonly mislabeled in the literature. Some arelabeled case studies when in fact they are not really an in-depth investigation ofa bounded system; a study is called "grounded theory" because the researcherhas used the constant comparative method but does not build theory; a study islabeled "critical" because there is some analysis, but not of race, class, and gen-der, power and oppression, and so on. Mislabeling signals a superficial under-standing of qualitative research.

Sample selection should be clearly explained. In qualitative research a sampleis selected on purpose to yield the most information about the phenomenon ofinterest. There are usually criteria specified for selection. In a qualitative study(Courtenay, Merriam, & Reeves, 1998) of the transformationalleaming process ofyoung adults diagnosed HIV-positive,for example, the following criteria were usedin selecting a purposive sample: racially diverse men and women under the ageof forty-five with a t-cell count of five hundred or less (at the time of the studybefore protease inhibitors, this measure signaled medical intervention). Not onlyshould we know how the sample was selected, and the rationale for various se-lection criteria, we should be given a description of the final study participants.This description is sometimes placed at the beginning of the findings section. De-pending on the number of participants, the description can be presented in sum-mary form (that is, "ten men and eight women ranging in age from eighteen toforty-five"), or as short portraits or biographies of each participant. In narrativeanalysis studies, each participant's "story" can be quite lengthy and is part of thefindings.

In the methods section we should also be told something about how the datawere collected, managed, and analyzed. It might be recalled that the three pri-mary sources of data are interviews, observations, and documents. If interviewswere used, how were they arranged, where were they held, how long did theytake, and what kind of interview schedule was used? What areas related to thetopic did the interviewer probe? Observations also need to be described. Whereand when were these scheduled? Did the researcher videotape or audiotape aspart of the observation? Was a standardized check sheet used? With regard todocuments, we need to know how the researcher accessed them and what type

of documents they were. For public or official documents, the researcher should

explain the original purpose of the documents.Another source of data is from on-line interaction. While on-line data col-

lection offers an electronic extension of familiar research techniques (one caninterview on-line, observe groups, access documents), the medium itself makesfor some important differences. For example, research suggests that there canbe major discrepancies between real and on-line personalities (Phillips & Barnes,1995). If data were collected on-line, it is important for the researcher to addressits validity and reliability relative to the study's purpose.

Not only how the data were collected, but the system for managing the data aswell as how they were analyzed need to be presented in the methods section. If acomputer software program such as TheEthnograph, NUDIST,or Atlns-ti was usedto manage the data, its use needs to be briefly described as well as any uniqueadaptations the author might have designed. It is commonly understood that thesesoftware programs manage and may facilitate data analysis, but the researcher'sanalytical strategies are still central to deriving categories, themes, or "regularities"across the data set. Just how researchers do this is highly idiosyncratic and intui-tive. Nevertheless, some explanation of how the researcher proceeded, with maybean accompanying example, will be helpful to the reader. Further, different types ofqualitative research use different data analysis strategies. For example, in a phe-nomenological study, "bracketing" and "phenomenological reduction" are strate-gies; in grounded theory, there is "theoretical sampling" and a "core category."

Related to evaluating the methodology of the study, a reader can note whetherany of the strategies for enhancing internal validity, reliability, and external va-lidity were employed (see the next section of this chapter). Do we know anythingabout the researcher's assumptions, biases, or connection to the phenomenon orparticipants of the study? Was there any form of triangulation, if appropriate?Member checks? Peer examination? Do any of the procedures or the conduct ofthe study raise ethical questions in the mind of the reader? Are these addressed?

The explanation of the methods of the study, how the sample was selected,how the data were collected and analyzed, and how validity and reliability wereaddressed constitutes an audit trail. This audit trail or transparency of methodis one strategy for enhancing the study's reliability. How detailed this trail is,how transparent the methodology is, is also one basis for assessing the value ofthe study.

Traditional qualitative studies present the findings of the inquiry as a mixof rich, thick description and interpretation. It should be noted that the field iscurrently experimenting with innovative and creative presentations of researchfindings, a situation that complicates how one assesses the persuasiveness ofpresentations of qualitative research. Open to debate and yet to be developedare appropriate criteria for evaluating poetic, fictional, dramatic, musical, artis-tic, and mixed media presentations of qualitative research findings. However,. . U -- -- "un --n -- ,,- -- n_- -0--

data to adequately and convincingly support the findings of the study. Firestone(1987) notes that a quantitative and qualitative research paradigm each employs


different "rhetoric" to convince readers of its trustworthiness. "The quantitativestudy must convince the reader that procedures have been followed faithfullybecause very little concrete description of what anyone does is provided. Thequalitative study provides the reader with a depiction in enough detail to showthat the author's conclusion 'makes sense'" (p. 19). Further, "the quantitativestudy portrays a world of variables and static states. . . . By contrast the quali-tative study describes people acting in events" (p. 19).

Just how much is "enough" detail and description and how to present it areopen to debate. At the very least, each finding must be supported by the "raw"data from which the finding was derived. These data are in the form of exactquotes from people interviewed, episodes from field observations, and refer-ences from supporting documents. On one hand, a major or unique insight ordiscovery needs more than one quote from a participant to substantiate its pres-ence in the data. On the other hand, presentations of supporting data shouldnot be so long or so dense that the reader loses interest.

Erickson (1986) has made a helpful distinction among description that is par-ticular or general, and interpretive commentary. Particular description consistsof exact quotes from the raw data, general description includes comments to thereader as to whether the quotes and vignettes are typical of the data as a whole,and interpretive commentary guides "the reader to see the analytic type of whichthe instance is a concrete token. . . . Interpretative commentary thus points thereader to those details that are salient for the author, and to the meaning-interpretations ofthe author" (p. 152).

Since there are no agreed-upon guidelines as to the right balance between theparticular and the general, between description and analysis, the reader makesthe judgment as to whether there is "enough" data to support the author's in-terpretation. The bottom line is whether the reader is persuaded that the find-ings make sense in light of the data presented.

By way of summarizing this section on "what to look for when evaluatingqualitative research," Table 2.1 provides a check sheet of the points discussed.This should be used with caution, however, as no single study is likely to meetall of the expectations. Further, all such evaluation forms reflect the author'sorientation to the phenomenon being assessed-in this case, my constructivistand interpretive stance to qualitative research. As Smith and Deemer (2000)point out, "a list of characteristics must be seen as always open-ended, in partunarticulated, and, even when a characteristic is more or less articulated, it isalways and ever subject to constant reinterpretation" (p. 888).

Ensuring for ((Quality"in Qualitative Research

All researchers aspire to produce valid and reliable knowledge in an ethical man-ner. And both producers and consumers of research want to be assured that thef;~rl:~~- _£ -- '-_u _L' .'

Assessing and Evaluating Qualitative Research 23

A. Problem

Table 2.1. Assessing the "Quality" of Qualitative Research.

B. Methods

C. Findings

D. Discussion

1. Is the problem appropriate for qualitative inquiry?Is the question one of meaning, understanding, or process?

2. Is the problem clearly stated?3. Is the problem situated in the literature? That is, is the litera-

ture used to put the problem in context?4. Is the relationship of the problem to previous research

made clear?

S. Is the researcher's perspective and relationship to the problemdiscussed? Are assumptions and biases revealed?

6. Is a convincing argument explicitly or implicitly made for theimportance or significance'of this research? Do we know howit will contribute to the knowledge base and practice?

1. Is the particular qualitative research design identified anddescribed (basic interpretive, grounded theory, phenomenol-ogy, ethnography, and so on)?

2. Is sample selection described including rationale for criteriaused in the selection?

3. Are data collection methods described and are they con-gruent with the problem being investigated and the type ofqualitative design?

4. How were the data managed and analyzed?S. What strategies were used to ensure for validity and reli-

ability? (See Table 2.2.)6. What ethical considerations are discussed?

1. Are the participants of the study described? (This may bein Methods.)

2. Are the findings clearly organized and easy to follow?3. Are the findings directly responsive to the problem of the

study? That is, do they "answer" the question(s) raised bythe study?

4. Do the data presented in support of the findings (quotationsfrom interviews, incidents from field notes, material fromdocuments, and so on) provide adequate and convincingevidence for the findings?

1. Are the findings "positioned" and discussed in terms of theliterature and previous research?

2. Are the study's insights and contributions to the larger bodyof knowledge clearly stated and discussed?

3. Are implications for practice discussed?4. Do the study's implications follow from the data?S. Are there suggestions for future research?


the study. An understanding of these strategies is important for assessingof a particular qualitative study. First discussed are the concepts of Val

~reliability and the strategies that can be employed to ensure each. Si...'worthiness very much hinges on the ethicalconduct of research, ethic -

particularly relevant to qualitative research are also mentioned in this se

~It is not uncommon to hear people ask whether a particular study is a

study. "Good" is of course a relative term open to interpretation, but wha

usually mean by that question is whether the study was conducted in a rigcsystematic, and ethical manner, such that the results can be trusted. Measuensure validity and reliability in positivist research (for example, surveys'petiments) are well developed and accepted by the scientific community. AI

uate students are well acquainted with Campbell and Stanley's (1963) i'

discussion of "threats" to validity and reliability such as survivor bias, test!fects, and selective sampling, for example. Qualitative research also has strafor establishing validity and reliability, strategies based on the different wodand different questions congruent with the philosophical assumptions undethis perspective. ,

Currently, there is much debate and discussion in the literature and at CO]ences as to how to think about validity and reliability in qualitative researcJparticular, writers from postmodern, poststructural, and critical perspective~challenging interpretive/constructivist notions of validity and reliability. Deand Lincoln (2000) consider the postmodern turn in qualitative research, foample, as problematic for evaluating qualitative research. "This is the legitimicrisis. It involves a serious rethinking of such terms as validity, generalizabiand reliability, terms already retheorized in" other types of qualitative resea(p. 17, emphasis in original). Kvale (1996) proposes thinking of validity as"craftsmanship in which the researcher adopts a critical outlook during data anysis, (2) communication where validity is determined in dialogue with others, 3i(3) pragmatic validity,which goes beyond an argument's persuasiveness to asse:ing validity in terms of real-world changes brought about as a result of the rlsearch. Even within an interpretive orientation, Connelly and Clandinin (199

" "

O),

'

"

'

,,j

'

,','

in speaking of narrative inquiry for example, write that the criteria for judging narratives will be different from other qualitative genres, and that at the time of thei ,

writing, these criteria were not yet agreed upon.Nevertheless, most producers and consumers of research do not want to wait

for the scholars of qualitative research to develop a consensus as to the appro-priate criteria for assessing validity and reliability, if indeed that is even possi-ble (see Hammersley, 1990, for example). While the theoretical debate goes on,there are immediate needs to be met regarding these issues. As Stake (2000)notes, knowledge gained in an investigation "faces hazardous passage from the

,~

writer to reader. The writer needs ways of safeguardingthe trip" (p. 443). Fur-,::ther, qualitative researchers need to respond to the concerns of outsiders, many isof whom may be unfamiliar with or blatantly challenging of the credibilityofqualitative research. With these concerns in mind, the following sections ad-dress dealingwith internal validity,reliability,external validity,and ethics in in-teroretive aualitative research.


Internal ValidityInternal validity asks the question, How congruent are one's findings with real-ity? In quantitative research this question is usually construed as, Are we observ-ing or measuring what we think we are observing or measuring? The questionhinges on what we think constitutes reality. As was discussed in Chapter One, re-ality in qualitative inquiry assumes that there are multiple, changing realities andthat individuals have their own unique constructions of reality. In fact, no matterwhich paradigm one is working from, reality is always interpreted through sym-bolic representations such as numbers and words. In qualitative research, the un-derstanding of reality is really the researcher's interpretation of participants'interpretations or understandings of the phenomenon of interest. For example, it

, is wellknown that eyewitnessesto a crime can have widelyvaryingaccounts of" what actuallyhappened. So too, of course,when you ask people how they have'~xperienced a particular phenomenon, how they have made meaning of their, lives, or how they understand certain processes. In qualitative research we arenot interested in how many or the distribution of predefined variables. Rather, itiisimportant to understand the perspectives of those involved, uncover the com-

i

"

.

p..

'

..

'

..

..

..

l

..

.

...

.

..

exI

..

'ty of human behavior in context, and present a holistic interpretation of what

i~happening.'i>Because qualitative researchers are the primary instruments for data collection

d analysis, interpretations of reality are accessed directly through observations[ridinterviews. We are "closer" to reality than if an instrument with predefinedems had been interjected between the researcher and the phenomenon being,tudied. Most agree that when reality is viewed in this manner-that it is alwaysiterpreted-internal validity is considered a strength of qualitative research.iThere are a number of strategies that qualitative researchers can employ toore up the internal validity of a study. Probably the most well known of thesef:triangulation. Foreman (1948) first cited this procedure more than fifty years:0. He recommended using independent investigators "to establish validityrough pooled judgment" and using outside sources to validate case study ma-ials (p. 413). Denzin (1970) presented an extended discussion of triangula-

i,n,identifying four types: multiple investigators, multiple theories, multipleTces of data, or multiple methods to confirm emerging findings. Triangulat-multiple theories is rare, but the other three forms, especially using multi-data collection methods, are commonly found in qualitative studies. In this

mgulation strategy the researcher collects data through a combination of in-"news, observations, and document analysis. For example, what someone tells

in an interview can be checked against what you observe in a field visit or~tyou read or see in documents or artifacts relevant to the investigation. Theof multiple researchers also strengthens the internal validity of a study. This'on of multiple researchers has been discussed in other contexts as collabo-e or team research. In participatory research, where the goal of the research~liticalempowerment, the participants along with the researcher collectivelyethe problem to be addressed, conduct the study, and engage in collective

":'0 \,/UdllIdUYC 1\COCO1\.U'" . "..u~

It should be noted that as with other strategies for ensuring validity and relia-bility, triangulation is being revisited in the literature from a postrnodern perspec-tive. Richardson (2000) points out that triangulation assumes a "'fixed point' or'object' that can be triangulated." But in postmodern research, "we do not trian-gulate; we crystallize. We recognize that there are far more than three sides fromwhich to approach the world" (p. 934). However, from an interpretive perspective,triangulation remains a principal strategy to ensure for validity and reliability.

A second common strategy for ensuring validity in qualitative research is mem-ber checks. Here you ask the participants to comment on your interpretation ofthe data. That is, you take your tentative findings back to some of the participants(from whom you derived the raw data through interviews or observations) andask whether your interpretation "rings true." While you may have used differentwords, participants should be able to recognize their experience in your inter-pretation or suggest some fine-tuning to better capture their perspectives. Somewriters suggest doing member checks throughout the course of the study.

Peerreview is yet another strategy. In one sense, all graduate students havea peer review process built into their thesis or dissertation committee-as eachmember reads and comments on the findings. Peer review or peer examinationcan be conducted by a colleague either familiar with the research or one newto the topic. There are advantages to both, but either way, a thorough peer ex-amination would involve asking a colleague to scan some of the raw data andassess whether the findings are plausible based on the data.

The relationship between the researcher and what and who are being studiedhas been the topic of much scholarly writing in the last decade. Lincoln (1995)in fact suggests that the emerging criteria for quality in interpretive inquiry bebased on considering the relational aspects of the research process (for example,the knower and the known). In so doing, the distinction between quality or rigorand ethics "collapses" (p. 275). In any case, even in journal articles researchersare being called upon to articulate and clarify their assumptions, experiences,worldview, and theoretical orientation to the study. Investigators should explaintheir position vis-a-vis the topic being studied, the basis for selecting participants,the context of the study, and what values or assumptions might affect data col-lection and analysis. This strategy is sometimes labeled "researcher's position"and more recently, "reflexivity"-"the process of reflecting critically on the selfas researcher, the 'human as instrument'" (Lincoln and Guba, 2000, p. 183). Such,a clarification allows the reader to better understand how the individual researcher

might have arrived at the particular interpretation of the data.Finally, it is recommended that the researcher be submerged or engaged in

the data collection phase over a long enough period to ensure an in-depth un-derstanding of the phenomenon. How long one needs to observe or how manypeople need to be interviewed are always difficult questions to answer ahead oftime. The best rule of thumb is that the data and emerging findings must feelsaturated; that is, you begin to see or hear the same things over and over again,and no new information surfaces as you collect more data.

Adequate time in the field should also be coupled with purposefully lookingfor variation in the understanding of the phenomenon. Some writers even suggest


. ratyou should purposefully seek cases that might disconfirm or challenge your. ectations or emerging findings. This strategy has been labeled negative or dis-'epant case analysis (Lincoln & Guba, 1985; Silverman, 1993).

.~liabi1ity

many have pointed out, there is no point in considering reliability without va-'ity. That is, you could have a highly reliable instrument, a thermometer for in-lnce, that records boiling water at 85 degrees Fahrenheit each and every time itplaced in boiling water, but it is not at all valid. Reliability refers to the extentwhich research findings can be replicated. In other words, if the study were re-ted would it yield the same results? Reliability is problematic in the socialnces simply because human behavior is never static, nor is what many expe-

nce necessarily more reliable than what one person experiences. Consider the19icianwho can fool the audience of hundreds but not the stagehand watchingfm the wings. Replication of a qualitative study will not yield the same results,l(this does not discredit the results of any particular study; there can be nu-

~erous interpretations of the same data. The more important question for quali-tive researchers is whether the results are consistent with the data collected.

u::olnand Guba (1985, p. 288) werethe first to conceptualizereliabilityin qual-tive research as "dependability" or "consistency." That is, rather than insistingat others get the same results as the original researcher, reliability lies in others'Incurring that given the data collected, the results make sense-they are con-;tent and dependable.further, since reliability most often has to do with the instrumentation of the

. dy, and since the researcher is the primary instrument of data collection andalysis in qualitative research, the researcher can become a more reliable in-

.rilment through training and practice. Also, the reliability of documents and~rsonalaccounts can be assessed through various techniques of analysis andangulation.

. Strategies that a qualitative researcher can use to ensure for consistency and~pendabilityor reliability are triangulation, peer examination, investigator's po-

:ition, and the audit trail. The first three have been discussed under internal va-

~dity.The use of multiple methods of collecting data, for example, can be seenr~a strategy for obtaining consistent and dependable data as well as data that'emost congruent with reality as understood by the participants. The audit traila method suggested by Guba and Lincoln (1981). Just as an auditor authenti-ltes the accounts of a business, independent readers can authenticate the find-gs of a study by following the trail of the researcher. While "we cannot expecthers to replicate our account," Dey U993, p. 251) writes, "the best we can do

, explain how we arrived at our results." An audit trail in a qualitative study de-:rihes in detail how data were collected, how categories were derived, and how~cisionswere made throughout the inquiry. The audit trail is dependent upone researcher keeping a research journal or recording memos throughout the)nduct of the study. What go into this journal are your reflections, questions,

hddecisions on the Dfohlems. issues. ideas vou encounter in collectim! data. A


interpretation is also recommended. However this journaling is set up, it is im-portant to capture reflections and thoughts about you as a researcher, about datacollection issues, and about interpretations of the data. In a book-length or thesis-length report of the research, the audit trail is really a detailed account of howthe study was conducted and how the data were analyzed. Due to space limita-tions, journal articles tend to have a very abbreviated audit trail, if any at all.

External Validity or GeneralizabilityMore than internal validity or reliability, the issue of external validity or gener-alizability in qualitative research has stimulated substantial discussion and de-bate. It is also a major challenge for novice qualitative researchers to justify theirqualitative inquiry in terms of generalizability. Part of the problem lies with thecommon perception of generalizability derived from positivist-oriented researchwherein one can generalize in a statistical sense from a random sample to a pop-ulation. The basic question even for qualitative research is the extent to whichthe findings of one study can be applied to other situations. But since small, non-random samples are selected purposefully in qualitative research, it is not pos-sible to generalize statistically. A small sample is selected precisely because theresearcher wishes to understand the particular in depth, not to find out what isgenerally true of the many.

Because qualitative research draws from different assumptions about reality,generalizability needs to be thought of differently from quantitative research.There are a number of understandings of generalizability that are more con-gruent with the worldview of qualitative research. Some argue that empiricalgeneralizations are too lofty a goal for social science; instead we should thinkin terms of what Cronbach (1975) calls working hypotheses-hypotheses thatreflect situation-specific conditions in a particular context. Working hypothesesthat take account of local conditions can offer practitioners some guidance inmaking choices-the results of which can be monitored and evaluated in orderto make better decisions in the future. Patton (1990) also promotes thinking of"context-bound extrapolations rather than generalizations" (p. 491).

If one thinks of what can be learned from an in-depth analysis of a particularsituation or incident and how that knowledge can be transferred to another situ-ation, generalizability in qualitative research becomes possible. Erickson (1986)writes that rather than abstract universals arrived at through statistical analysis,what we have in qualitative research are concrete universals. The general lies inthe particular; what we learn in a particular situation we can transfer to similarsituations subsequently encountered. Eisner (1991) points out that more than ab-stractions can be generalized-skills and images can also. For example, we learna skill in one situation and transfer it to another. Images also generalize. "For qual-itative research, this means that the creation of an image-a vivid portrait of ex-cellent teaching, for example-can become a prototype that can be used in theeducation of teachers or for the appraisal of teaching" (Eisner, 1991, p. 199).

Probably the most common way generalizability has been cohceptualized inqualitative research is as reader or user generalizability. In this view, readers


themselves determine the extent to which findings from a study can be appliedto their context. Called case-to-case transfer by Firestone (1993), "It is the readerwho has to ask, what is there in this study that I can apply to my own situation,and what clearly does not apply?" (Walker, 1980, p. 34). Case-to-case or user gen-eralizability is common practice in law and medicine, where the practitioner de-cides whether a previous case is applicable to the present situation. In order tofacilitate the reader (not the researcher) transferring findings from one studyto his or her present situation, the researcher must provide enough detail of thestudy's context so that comparisons can be made.

Providing rich, thick description is a major strategy to ensure for external va-lidity or generalizability in the qualitative sense. This involves providing an ade-quate database, that is, enough description and information that readers will beable to determine how closely their situations match, and thus whether find-ings can be transferred. Multisite designs or maximizing variation in the pur-posely selected sample is another strategy. The logic behind this strategy is thatif there is some diversity in the nature of the sites selected (an urban and a ruralschool, for example) or in participants interviewed, or times and places of fieldvisits, results can be applied to a greater range of situations by readers or con-sumers of the research.

Ethical Issues

A "good" qualitative study is one that has been conducted in an ethical manner.To a large extent, the validity and reliability of a study depend upon the ethics ofthe researcher. Suppose, for example, that you are studying an adult literacy pro-gram reputed to have an unusually high retention rate. Youinterview teachers, ad-ministrators, and students and begin to identify the factors that might account forthe high retention rate. Then you stumble upon some records that appear to havebeen tampered with, inflating attendance and graduation rates. Your decision asto how to handle this discovery will have a direct impact on the trustworthinessof your entire study. While some sense of the researchers' values can be inferredfrom the statement of their assumptions and biases or from the audit trail, read-ers of course are likely never to know what ethical dilemmas were confronted andhow they were dealt with. It is ultimately up to the individual researcher to pro-ceed in as ethically a manner as possible.

In qualitative research, ethical dilemmas are likely to emerge with regard tothe collection of data and in the dissemination of findings. Overlaying both thecollection of data and the dissemination of findings is the researcher-participantrelationship. For example, this relationship and the focus of the research deter-mine how much the researcher reveals about the actual purpose of the study-how informed the consent can actually be-and how much privacy andprotection from harm is afforded the participants. Ethical considerations re-

garding the researcher's relationship to participants ar~ becoming amajor sourceof discussion and debate in qualitative research, especially with the growing in-terest in critical, participatory, feminist, and postmodern research. When the re-search is high Iv collaborative, participatory, and/or political, ethical issues

Qualitative Research in Practice

become prominent. Lincoln (1995), in particular, aligns ethical considerationswith the researcher's relationship with research participants and considers va-lidity to be an ethical question. She suggests seven "standards" for validity, suchas the extent to which the research allows all voices to be heard, the extent ofreciprocity in the research relationship, and so on.

Although qualitative researchers can turn to guidelines, others' experiences, andgovernment regulations for dealing with some of the ethical concerns likely toarise, the burden of producing a study that has been conducted and disseminatedin an ethical manner lies with the individual investigator. No regulation can tell aresearcher when the questioning of a participant becomes an interrogation ratherthan an interview, when to intervene in abusive or illegal situations, or how to en-sure that the study's findings will not be used to the detriment of those involved.As Punch (1994, p. 84) points out, "Acute moral and ethical dilemmas. . . oftenhave to be resolved situationally, even spontaneously." All possibilities cannot beanticipated, nor can one's reaction. Examining the assumptions one carries intothe research process-assumptions about the context, participants, data, and thedissemination of knowledge gained through the study-is at least a starting pointfor conducting an ethical study.

.In summary, there is no simple answer as to what makes a "good" qualitativestudy. Researchers and consumers alike want to be able to trust the results ofany research study, especially in applied fields where we intervene in humanlives. To be trustworthy, a study needs to be valid and reliable and conductedin an ethical manner. There are strategies that researchers can employ that willenhance the trustworthiness of their research, such as triangulation, memberchecks, use of rich, thick description, and so on. Used in conjunction with anawareness of ethical issues, these strategies can build confidence in the valid-ity and reliability of the study. Table 2.2 summarizes the strategies for enhanc-ing validity and reliability in interpretive qualitative research.

Summary

This chapter has addressed two questions: What should you look for when euating qualitative research? and, What makes a good qualitative study? Theare of course highly interrelated. Evaluating a qualitative study means raising q;tions about all aspects of the process and write-up, beginning with whethertopic is appropriate for qualitative inquiry and whether it is an importantquestion to spend time, money, and other resources on answering.

questions are also important, including sample selection and description, datalection and analysis, validity and reliability safeguards, and presentation and


Table 2.2. Strategies for Promoting Validity and Reliability.

review/examination

positionreflexivity

engagementdata collection

Rich, thick descriptions

Description

Using multiple investigators, sources of data, or datacollection methods to confirm emerging findings

Taking data and tentative interpretations back to thepeople from whom they were derived and asking ifthey were plausible

Discussions with colleagues regarding the process ofstudy, the congruency of emerging findings with theraw data, and tentative interpretations

Critical self-reflection by the researcher regardingassumptions, worldview, biases, theoretical orienta-tion, and relationship to the study that may affect theinvestigation

Adequate time spent collecting data such that thedata become "saturated"; this may involve seekingdiscrepant or negative cases of the phenomenon

Purposefully seeking variation or diversity in sampleselection to allow for a greater range of application ofthe findings by consumers of the research

A detailed account of the methods, procedures, anddecision points in carrying out the study

Providing enough description to contextualize thestudy such that readers will be able to determinethe extent to which their situation matches the

research context, and hence, whether findings canbe transferred

trustworthiness has to do with issues of internal validity, reliability, and externalvalidity or generalizability. Strategies ensuring for adequate treatment of each ofthese issues include triangulation, member checks, peer examination, investiga-tor position, audit trail, and rich, thick description. The sixteen articles in Part'TWocan be approached using the guidelines summarized in Tables 2.1 and 2.2.

References

Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designsfor research on teaching. In N. Gage (Ed.), Handbook for research on teaching(pp. 171-246). Chicago: Rand McNally.


Connelly, F. M., & Clandinin, D. J. (1990). Stories of experience and narrative inquiry.Educational Researcher, 9(5),2-14.

Courtenay, B. C., Merriam, S. B., & Reeves, T. (1998). The centrality of meaning-making in transformational learning: How HIV-positive adults make sense oftheir lives. Adult Education Quarterly, 48(2),65-84.

Cronbach, 1. J. (1975). Beyond the two disciplines of scientific psychology. AmericanPsychologist, 30, 116-127.

Denzin, N. K. (1970). The research act: A theoretical introduction to sociological

methods. Chicago: Aldine.

Denzin, N. K., & Uncoln, Y. S. (2000). Introduction: The discipline and practice ofqualitative research. In N. K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualita-tive research. (pp. 1-28). Thousand Oaks, CA: Sage.

Dey, I. (1993). Qualitative data analysis. London: Routledge.

Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement ofeducational practice. Old Tappan, NJ: Macmillan.

Erickson, F. (1986). Qualitative methods in research on teaching. In M. C. Whittrock(Ed.), Handbook of research on teaching. (3rd ed.) Old Tappan, NJ: Macmillan.

Firestone, W. A. (1987). Meaning in method: The rhetoric of quantitative and qualita-tive research. Educational Researcher, 16(7), 16-21.

Firestone, W. A. (1993). Alternative arguments for generalizing from data as applied toqualitative research. Educational Researcher, 22(4), 16-23.

Foreman, P. B. (1948). The theory of case studies. Social Forces,26(4), 408-419.

Guba, E. G., & Lincoln, Y.S. (1981). Effective evaluation. San Francisco: Jossey-Bass.

Hammersley, M. (1990). Reading ethnographic research: A critical guide. London:Longman.

Kvale, S. (1996). InterViews: An introduction to qualitative research methodology.

Thousand Oaks, CA: Sage.

Lincoln, Y.S. (1995). Emerging criteria for quality in qualitative and interpretive re-search. Qualitative Inquiry, 1(1), 275-289.

Lincoln, Y.S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

Lincoln, Y.S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, andemerging confluences. In N. K. Denzin & Y.S. Lincoln (Eds.), Handbook of quali-tative research. (2nd ed.) (pp. 163-188). Thousand Oaks, CA: Sage.

Patton, M. Q. (1990). Qualitative evaluation methods. (2nd ed.) Thousand Oaks, CA:Sage.

Phillips, G. M., & Barnes, S. B. (1995). Is your epal an ax-murderer? InterpersonalComputing and Technology, 3(4), 12-41.

Punch, M. (1994). Politics and ethics in qualitative research. In N. K. Denzin &Y.S. Lincoln (Eds.), Handbook of qualitative research. (pp. 83-97). Thousand

Oaks, CA: Sage.

Richardson, L. (2000). Writing: A method of inquiry. In N. K. Denzin & Y.S. LincolnruA~' Un~rlhnnZ,"f n1JnUtntilJeresearch. (2nd ed.) (pp. 923-948). Thousand


Silverman, D. (1993). Interpreting qualitative data. London: Sage.

Smith, J. K., & Deemer, D. K. (2000). The problem of criteria in the age of relativism.In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research. (2nd ed.)(pp. 877-896). Thousand Oaks, CA: Sage.

Stake, R. E. (2000). Case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook

of qualitative research. (2nd ed.) (pp. 435-454). Thousand Oaks, CA: Sage.

Walker, R. (1980). The conduct of educational case studies: Ethics, theory and proce-dures. In W B. Dockerell & D. Hamilton (Eds.), Rethinking educational research.London: Hodder & Stoughton.

introduction to qualitative research-merriam 2002

Documents