Slide 1

Tim Winchell Analytical Techniques for Public Service The Evergreen State College Winter 2011 Qualitative Data Analysis Slide 2 It wasnt curiosity that killed the cat. It was trying to make sense of all the data curiosity generated. -Halcolm Slide 3 Qualitative Data Written text Conversation, interview, or consultative transcriptions Focus group transcriptions Field notes Diaries Legal transcripts Newspaper clippings Journal or magazine articles Photographs Maps Illustrations Paintings Musical scores Tape recordings Films (McNabb, p. 368) Qualitative Data have been gathered during the conduct of interpretive or postpositivist research studies. They exist most often as some sort of narrative. Examples include: Slide 4 Advantages Grounded in a specific context/situation Real life events/ settings; lived experience Deep layers of meaning; rich description filled with differing perspectives, symbolism, metaphor, and meaning. Descriptions form the bedrock of all qualitative reporting. (Patton, p. 438) The devil is in the details. Slide 5 Difficulties Labor intensive Requires creativity Conceptual sensitivity Non- formulaic (Polit & Beck, p. 570) Research Bias Cost of processing/coding data Small sample size, many variables Very limited generalizability Credibility Slide 6 One Synopsis of the Challenges The challenge of qualitative analyses lies in making sense of massive amounts of data. This involves reducing the volume of raw information, sifting trivia from significance, identifying significant patterns, and constructing a framework for communicating the essence of what the data reveal There are no formulas for determining significance. No ways exist of perfectly replicating the researchers analytical thought processes. No straightforward tests can be applied for reliability and validity. In short, no absolute rules exist except perhaps this: Do your very best with your full intellect to fairly represent the data and communicate what the data reveal given the purpose of the study. (Patton, p. 432-433) Slide 7 Data Management The complexity of the project drives the level of organization needed Format field notes consistently Index notes, so you can find documents easily Make sure you can read them! Have a sensible system for cross referencing your notes Please remember to: Maintain data confidentiality as much as possible Secure your data when not in use Maintain participant confidentiality Slide 8 And Another Data Back Up Reminder! Thomas Carlyle lent the only copy of his handwritten manuscript on the history of the French Revolution, his master work, to philosopher J. S. Mill, who lent it to Mrs. Taylor. Mrs. Taylors illiterate housekeeper thought it was waste paper and burned it. Carlyle behaved with nobility and stoicism, and immediately set about rewriting the book. It was published in 1837 to critical acclaim and consolidated Carlyles reputation as one of the foremost men of letters of his day. Well never know how the acclaimed version compared with the original or what else Carlyle might have written in the year lost after the fireplace calamity. (Patton, p. 441- emphasis added) Slide 9 Your General Approach? Grounded Start data collection with few preconceived notions about whats going on.no pre-formed coding scheme) .or Framed? Specific events, behaviors you intend to look for, with coding scheme already partially developed. Oftentimes use diagrams to explain ideas. All analyses benefit from diagramming and concept mapping, as Babbie discusses (p. 405). Slide 10 Slide 11 Qualitative Analysis: The General Process Data Reduction Coding Data Display Conclusion Drawing These are not linear, but concurrent processes The less framed and more grounded the process, the more they are concurrent: constant comparison Slide 12 Data Reduction First, we transform data from field notes or transcriptions Write up and/or transcribe field notes and print. Which of the data are most useful? Developing some manageable classification system or coding scheme is the first step of analysis. Without classification there is chaos and confusion. Content analysis, then, involves identifying, coding, categorizing, classifying, and labeling the primary patterns in the data. (Patton, p. 463) Slide 13 Consider. For extensive research projects, summarize interviews with a brief cover sheet Who, what, where, when, importance, summary of key contacts Coding schemesmust match the complexity of the project Use similar semantics Identifying concepts, patterns, memos Slide 14 What is Coding? In short, codes are shorthand descriptors of: Setting and context Subjects perspectives, which could include their thinking about people and objects Processes, activities, and/or strategies Relationships and social structures Any preassigned coding schemes (Bogdan & Biklen, 1992, p. 166-172, as quoted in Creswell, p. 193) Creswell recommends analyzing data using codes readers would expect to learn more about, find surprising, and address larger theoretical issues in the literature. (p. 193) Slide 15 Variations. Start categorizing early Or .. Dive deeper into the data and avoid making judgments too early make tentative observations about what might be happening. To further analyze what is happening: Write memos to yourself Use concept mapping (Babbie, p. 405) Build preliminary typologies Try to use outcome/ process matrices (Patton, p. 468-477) Slide 16 Open Coding.One Approach Start with a sample of the data Read responses carefully Keep research questions in mind Make rough categories of these descriptors that seem to belong together and code them with a key word. Utilize constant comparison- similarities and differences. Work to saturation. Slide 17 Farm to School Example Why do local farmers participate in the local farm to school program? Resp.1: It makes the most business sense to me. Possible code: business sense, busin. Resp. 2: It gives me great pride to think of my organic produce being consumed locally by my family members, friends, and church members and their children. Possible code: service, serve Slide 18 Farm to School Example Business Sense (Busin.)Service (Serve) 1. Most business sense 3. Reduces transport costs 3. Ability to hire more 4. Reduces environmental impact- transport 6. Stability of local school district market 1. Belief in organic produce being consumed locally 1. Organic production for nuclear family, friends, & church members & their children 2. Service to local community 5. Some contribution to local school district (lower prices received) Slide 19 Write ongoing memos and abstracts Slide 20 Comprehending: The Basic Goal of this Stage Identify important phenomena Identify broad themes Document codes that emerge Begin to speculate about what might be happening Write ongoing memos and abstracts Slide 21 Axial Coding Explore the relationships between and among codes Look for: Contexts Causal Conditions Phenomenon central ideas Strategies for addressing the phenomenon Intervening conditions Action/ interactions Consequences (Gibbs video) Develop subcategories, linked by a paradigm. Paradigm includes conditions, actions/ interactions, and consequences (Polit & Beck, p. 584) Slide 22 Employee Self Care Example How could agencies promote employee self care in their organizations? Organizational Changes (OrgCh.) Employee Changes (EmpCh.) Policies Management Training Supervisory Best Practices Employee Awareness Health Education Initiatives Medical Coverage Incentives Individual Health Surveys/ Contracts/ Teaming Employee Best Practices Slide 23 Selective Coding Identify core phenomenon Develop story line around the core concept(s) Compare and contrast the core concept(s) to other selective coding categories (Gibbs video) Findings are integrated and refined Include diagrams (Polit & Beck, p. 584) Slide 24 Data Display Playing with typologies and displays is a part of the analysis process See Miles and Huberman, Qualitative Data Analysis Make sense of the data by playing with visual means of representing the patterns that are emerging from the analysis Process and outcome flow charts/ matrices Slide 25 Slide 26 Slide 27 Interpretation, by definition involves going beyond the descriptive data. Interpretation means attaching significance to what was found, making sense of findings, offering explanations, drawing conclusions, extrapolating lessons, making inferences, considering meanings, and otherwise imposing order on an unruly but surely patterned world. (Patton, p. 480) Slide 28 Theorize: Cause and Effect? Classic Conditions for Establishing Cause and Effect Variables Covary Covariance is not spurious Logical time order A lucid explanation is available Or clusters of phenomena, identify things that tend often to show up together, even if the causal connection is not clear Slide 29 Qualitative Analysis- Visually Slide 30 Analysis of Medical Errors Figure 1 classifies the stage in the diagnostic testing process and the transition points within and between stages at which errors can occur, and presents representative occurrences that fall into each of them. (Harris, et al.) Figure 1 Slide 31 Early Introduction of Soft Foods by Young Mothers Slide 32 Verification Triangulate from multiple sources or methods Use several researchers as a reliability check. Use rich, thick description in order to provide for the shared experience Clarify research bias up front Look for disconfirming evidence Spend prolonged time in the field to develop an in-depth understanding Use peer debriefing Use an external auditor to review findings (Creswell, p. 196) Complete several case studies. (Yin, 2003) Review finding with participants. If its just you, double or triple check your data and conclusions Slide 33 Standards Be true to the data Dont get too carried away by particularly eloquent, memorable, or simple respondentsthis creates a cognitive bias Always check and recheck both the data and conclusions you draw from it Slide 34 Qualitative Validity Traditional Criteria for Judging Quantitative Research Alternative Criteria for Judging Qualitative Research Internal validity External validity Reliability Objectivity Credibility Transferability Dependability Confirmability (Trochim, 2006) Slide 35 Drawing Conclusions Summary of data and results of coding analysis Patterns and themes Clusters of similar findings? Case comparisons Powerful metaphors Any data for which your theory cant provide a reasonable explanation? Slide 36 Final Thoughts Data Management and Analysis work hand in hand Coding is technical work, which is improved upon with advanced practice, study, and interpretation Remember to consult additional resource materials (Some are listed at the end of the PowerPoint) Utilize the Internet judiciously Qualitative data software resources are reviewed in many publications and on-line Slide 37 Workshop Case: TESC Alumni Relations Research Interest Why do colleges and universities have alumni programs? Research questions What are TESC graduates perceptions of TESCs alumni programs? What kind of alumni program do they want? How do they recall their experience as TESC students? What connects them to the College? What nourishes that connection? What can AR do to improve those connections? Slide 38 Workshop Methods/ Results Overview Draft questions; approval from Alumni Relations Zoomerang online survey 1647 responses One researcher Pluses: clear conclusions, grounded in data Minus: not validated by second researcher Slide 39 Workshop Exercise Code 2 or 3 pages of the data from the responses to the Alumni survey question. What was the best part of your experience at Evergreen? Code individual responses What are the most common codes? What do these data tell you/us about these alumni ? About Evergreen? Slide 40 Resources YouTube Search qualitative research coding Graham R. Gibbs Qualitative Research Coding Series Open Coding: http://www.youtube.com/watch?v=gn7Pr8M_Gu8 http://www.youtube.com/watch?v=vi5B7Zo0_OE&fe ature=related http://www.youtube.com/watch?v=vi5B7Zo0_OE&fe ature=related http://www.youtube.com/watch?v=n- EomYWkxcA&feature=related http://www.youtube.com/watch?v=n- EomYWkxcA&feature=related http://www.youtube.com/watch?v=AwmDRh5l7ZE& feature=related http://www.youtube.com/watch?v=AwmDRh5l7ZE& feature=related Slide 41 Resources YouTube Search qualitative research coding Graham R. Gibbs Qualitative Research Coding Series Axial Coding: http://www.youtube.com/watch?v=s65aH6So_zY&feature=r elated http://www.youtube.com/watch?v=s65aH6So_zY&feature=r elated Selective Coding: http://www.youtube.com/watch?v=w9BMjO7WzmM&featur e=related http://www.youtube.com/watch?v=w9BMjO7WzmM&featur e=related Grounded Theory: http://www.youtube.com/watch?v=4SZDTp3_New&feature =related http://www.youtube.com/watch?v=4SZDTp3_New&feature =related http://www.youtube.com/watch?v=dbntk_xeLHA&feature= related http://www.youtube.com/watch?v=dbntk_xeLHA&feature= related Morgan, D. L. (1997). Focus Groups as Qualitative Research (2 nd Ed.). Sage Publications: Thousand Oaks, CA. Slide 42 Software Resources Computer Programs: See Babbie, p. 406-416 Data analysis strategies for qualitative research- Research Corner http://findarticles.com/p/articles/mi_m0FSL/is_6_74/ai _81218986/?tag=content;col1 http://findarticles.com/p/articles/mi_m0FSL/is_6_74/ai _81218986/?tag=content;col1 Software for qualitative research http://homepages.vub.ac.be/~ncarpent/soft/soft_softsites.html Software for qualitative research http://www.audiencedialogue.net/soft-qual.html Slide 43 References Babbie, E. (2010). The Practice of Social Research (12 th Ed.). Wadsworth Publishing: Belmont, CA. Creswell, J. W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approached (2 nd Ed.). Sage Publications: Thousand Oaks, CA. Harris, et al. Mixed Methods Analysis of Medical Error Event Reports: A Report from the ASIPS Collaborative http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=aps2&part=A 2024 McNabb, D. E. (2002) Research Methods in Public Administration and Nonprofit Management: Quantitative and Qualitative Approaches. M.E. Sharpe: Armonk, NY. Slide 44 References II Miles, M. B., & A.M. Huberman. (1994). Qualitative Data Analysis. (2 nd Ed.). Sage Publications: Thousand Oaks, CA. Patton, M. Q. (2002). Qualitative Research & Evaluation Methods (3 rd Ed.). Sage Publications: Thousand Oaks, CA. Polit, D. F., & Beck, C. T. (2004). Nursing Research: Principles and Methods (7 th Ed.). Lippincott Williams & Wilkins: New York, NY. Trochim, William M. K. (2006). Research Methods Knowledge Base. http://www.socialresearchmethods.net/kb/qualapp.php http://www.socialresearchmethods.net/kb/qualapp.php Yin, R. K. (2003) Case Study Research (3 rd Ed.). Sage Publications: Thousand Oaks, CA. Slide 45 Acknowledgements Making Sense of Qualitative Data TESC MPA Program ATPS Winter 2010 Geri/Gould/McBride