using socratic questioning to promote critical thinking ... · pdf fileusing socratic...

Using Socratic Questioning toPromote Critical Thinking

Skills Through AsynchronousDiscussion Forums in Distance

Learning Environments

Ya-Ting C. YangInstitute of Education

National Cheng-Kung University

Timothy J. NewbyCollege of Education

Purdue University

Robert L. BillSchool of Veterinary Medicine

Purdue University

This study investigated the effects of using Socratic questioning to en-hance students’ critical thinking (CT) skills in asynchronous discus-sion forums (ADF) in university-level distance learning courses. Theresearch effort empirically examined two coherent subjects: (a) the ef-ficacy of teaching and modeling Socratic questioning for developingstudents’ CT skills in ADF and (b) the persistence of students’ CTskills following the teaching and modeling of Socratic questioning inthe ADF. The results indicate (a) teaching and modeling of Socraticquestioning helped students demonstrate a higher level of CT skillsand (b) students maintained their CT skills after exposure to and mod-eling of Socratic questioning in the ADF.

Encouraging students to develop critical thinking (CT) skills has becomean important issue in higher education. Several teaching strategies, such asclassroom assessment techniques (Angelo 1995), cooperative learning

163

THE AMERICAN JOURNAL OF DISTANCE EDUCATION, 19(3), 163–181Copyright © 2005, Lawrence Erlbaum Associates, Inc.

Correspondence should be sent to Timothy J. Newby, Purdue University, BRNG, Room3138, 100 North University Street, West Lafayette, IN 47907. E-mail: [email protected]

strategies (Cooper 1995), and case study pedagogy (McDade 1995) havebeen proposed to help promote CT. King (1995) and Taba (1966) suggestedthat the level of thinking that occurs is influenced by the level of questionsasked. Asking thoughtful questions plays an important role in inducing stu-dents’ higher-level cognitive processes, such as self-reflection, revision,social negotiation, and conceptual change of student misconceptions, all ofwhich are integral to CT. In addition, when students are asked to generatequestions on their own, factual rather than thought-provoking questions aregenerally posed (Dillon 1988; Flammer 1981; Kerry 1987; King 1990).

Socratic questioning is one of the most popular and powerful teaching ap-proaches that can be used to guide students in generating thoughtful ques-tions, thus fostering their CT skills (Maiorana 1990–91; Paul 1993). Insteadof providing direct answers, the Socratic questioning approach stimulatesstudents’minds by continually probing into the subject with thought-stimu-latingquestions (Paul1993).Asa result, throughactive interactionsbetweeninstructorsandstudentsandamongstudents,Socraticquestioningcan facili-tate students’CT skills by the exchange of ideas and viewpoints, giving newmeaning to content, exploring applications to problems, and providing im-plications for real-life situations (Maiorana 1990–91).

In the context of distance education, however, interactions are often hin-dered because learners and instructors rarely meet face-to-face. Due to thelack of effective two-way communication tools to facilitate interactions fordistance educators, relatively few researchers have explored the presenceof CT within distance education. Emerging technologies such as asyn-chronous discussion forums (ADFs) (i.e., text-based computer-mediatedcommunication tools) have enabled educators to make distance education atruly interactive experience (Anderson and Garrison 1995; Garrison 1993).Since ADFs are promising tools that offer an opportunity for interactivediscussions and also provide an opportunity for the instructor to moderatestudent discussions (Duffy, Dueber, and Hawley 1998; Walker 2004), it ap-pears feasible to integrate Socratic questioning as a part of distance instruc-tion. This has the potential to improve students’ CT skills through an ADF(Hettinger 1995; Newman, Webb, and Cochrane 1995). Gunawardena,Lowe, and Anderson (1997) studied critical discussions in electronic mailand were also able to provide evidence that CT can occur in computerconferencing and that such processes occur as a direct result of exchangesamong participants. Using case study and qualitative research methods,Bullen (1998) concluded that CT can be demonstrated through computerconferencing. Jeong (2000) showed that a structured bulletin board discus-sion can support CT and critical discussions.

164

USING SOCRATIC QUESTIONING

The aforementioned studies have provided valuable information indicat-ing that students’CT skills can be fostered and demonstrated through struc-tured online discussions. Nevertheless, some of the research results, suchas Jeong’s, were merely based on self-reported student surveys or surveysused to gain descriptive information. To truly ascertain the effectiveness ofstructured online discussions in improving students’ CT skills, there is aneed to conduct more true and quasi-experiments. Thus, the first goal ofthis study was to conduct empirical research to ascertain the effectivenessof structured online discussions in improving students’ CT skills.

In this study, online discussion exercises, including debates and casestudies facilitated during the semester, were used in an ADF to monitor stu-dents’ mastery of CT skills. In particular, we were interested in realizingwhether students can maintain their CT skills after the instructor concludedthe facilitation of Socratic questioning. Thus, the second goal of this studywas to investigate the persistence of students’CT skills after the applicationof teaching and modeling Socratic questioning in an ADF.

Specifically, based on these two goals, an interrupted time-seriesquasi-experimental design was employed to test the following researchquestions:

Will students’ CT skills improve after participating inSocratic dialogues, as modeled and facilitated by theinstructor during structured ADF discussions?

After exposure to and modeling of Socratic questioning,will students maintain their higher levels of CT skillswithout the instructor’s further facilitation?

Method

The experimental research was conducted for two consecutive six-teen-week semesters. As shown in Figure 1, planned research procedures(Treatment I and Treatment II), consisting of several observations adminis-tered at appropriate times, were performed to measure and collect the re-quired data for the research analysis. These two treatments were conductedusing the same course, instructor, and course format.

The independent variable (X1) was the teaching and modeling of So-cratic questioning within the ADF. The dependent variables were the stu-dents’ CT skills, measured via the California Critical Thinking Skills Test(CCTST) (Facione 1990a, 1992), and class discussions on the ADF.

165

YANG, NEWBY, BILL

The CCTST is aimed at college/graduate students and adult professionals(Facione1990b).Thetestconsistsof thirty-fourmultiple-choice itemsandtar-gets the core CT skills regarded to be essential elements in a college education.The test reportsanoverall scoreonone’sCTskillsandfivesubscales: analysis,evaluation, inference, deductive reasoning, and inductive reasoning.

There were four online discussions within each treatment. The instruc-tor began the online class discussion by launching into two case studies andtwo controversial issues for debates that were important and relevant to thecourse content and reading assignments. During each discussion, the in-structor asked the students to identify and post at least one argument orstrong example to support their comments about the discussion issue. Eachstudent was required to respond to at least one other student’s posting to ex-plore the issue at hand and widen the discussion. Finally, at the end of thediscussion, students were asked to either summarize the points that weremade during the discussion or to write a short reflection about the discus-sion. The content of the class discussion posted on the ADF was analyzedqualitatively by using the coding scheme for evaluating critical thinking incomputer conferencing, which is discussed in detail in the Data Analysissection.

Participants

Sixteen undergraduate veterinary distance learning students (thirteen fe-males and three males) at a large university in the midwestern United Statesparticipated in the study. Within Treatment I there were eleven participants,while Treatment II had five participants.

166


Figure 1. Interrupted Time-Series Quasi-Experimental StyleNote: X1 = independent variable (the teaching and modeling of Socratic question-ing in ADFs); O1 = scores on CCTST and results of precourse demographic sur-vey; O2, O3, O4, and O5 = CT skills demonstrated in online discussions 1, 2, 3, and4, respectively; O6 = scores on CCTST at the end of the semester.

Procedures

During each semester, the instructor posted four specific discussion top-ics. Discussions 1 and 2 (O2 and O3) took place during the first half of thesemester. Discussions 3 and 4 (O4 and O5) occurred during the second halfof the semester. Of these four discussion exercises, there were two casestudies (Discussions 1 and 3) and two debates (Discussions 2 and 4). Eachdiscussion lasted for two weeks.

The independent variable (X1) in Treatments I and II was administered atdifferent times (see Figure 1). For Treatment I, during the second half of thesemester, the instructor taught and modeled Socratic questioning explicitlyand students practiced these questioning techniques by composing com-ments as well as challenging others in the ADF. The following exemplifiessome Socratic questions the instructor modeled and asked to help studentsexamine their thinking:

(Questions of clarification)Student A: Cardiovect, in all honesty, is probably not the best drug for any

of these cases. It has no real intrinsic activity on the receptor for angiotensinII and aldosterone—which would help dogs 1 & 2.

The instructor: I’m unclear about your point here. Are you saying that nointrinsic activity would help dogs 1 & 2 or that having intrinsic activity forangiotensin II would be beneficial? I can read the comment either way.

(Questions that probe assumptions)Student B: I would tell the owner that at this time Immodium A-D should

not be given. The reason for that is this type of medication decreases gut mo-tility and slows down the intestinal movement. This would prolong the timethe bacteria or enterotoxins are in the intestinal tract and might cause futuredamage.

The instructor: You seem to be assuming that the cause of Clarence’ssickness is the bacteria or enterotoxins in his intestinal tract. By what reason-ing did you come to this assumption?

(Questions that probe reasons and evidence)Student C: Clarence’s abdominal radiographs revealed evidence of possi-

ble hypersecretion of the intestines.The instructor: What evidence would this be?Student D: No, the insecticide would not cause the weight loss and the

chronic diarrhea. Instead it can cause dyspnea because it would inhibit theactivity of acetylcholinesterase, resulting in bronchoconstriction anddyspnea.

167

YANG, NEWBY, BILL

The instructor: Specifically, why wouldn’t it cause weight loss? Anyideas of how to back your statement? Think about your statement concern-ing dyspnea. What compound (neurotransmitter) stimulates the smoothmuscles in the respiratory tract to constrict? What effect would this insecti-cide have on that neurotransmitter? Does this support your argument?

For Treatment II, Socratic questioning was taught and practiced duringthe first half of the semester, and students practiced these questioning tech-niques themselves through online discussions during the second half of thesemester without the instructor’s further facilitation. Treatment I was de-signed to investigate the efficacy of the manipulation of X1 in promotingstudents’ CT skills, while Treatment II was designed to analyze whetherstudents would maintain their CT skills after the manipulation of X1.

Data Analysis

Both quantitative and qualitative methods were used to analyze the datacollected in this study. The quantitative data, scores on the CCTST, wereanalyzed using the Generalized Linear Model analysis of variance proce-dure. The qualitative data, online class discussions on the ADF, were ana-lyzed via content analysis. Online class discussions were analyzed by thecoding scheme for evaluating critical thinking in computer conferencing(see Table 1), which was created by combining the Interaction AnalysisModel for examining CT in computer conferencing, developed byGunawardena, Lowe, and Anderson (1997) (see Table 1, Part A) and amodified coding scheme with indicators of critical (+) and uncritical (–)thinking developed by Newman, Webb, and Cochrane (1995) (see Table 1,Part B). The coded qualitative data were then further analyzed using achi-square test to investigate whether students demonstrated CT skills fromthe online class discussions.

Coding scheme for evaluating critical thinking in computerconferencing. Gunawardena, Lowe, and Anderson (1997) developed theInteraction Analysis Model for the evaluation of “the process of knowledgeconstruction that occurs through social negotiation in computer-mediatedcommunication (CMC)” (400). They focused on assessing the quality ofinteractions in a CMC environment from both the “interactive” and “cogni-tive” dimensions. As shown in Table 1, Part A, their model includes fivephases of interactions and twenty-one operations (subcategories) to studycritical discussions in electronic mail. This model was designed to assess

168


169

Table 1. Coding Scheme for Evaluating Critical Thinking in ComputerConferencing

Part A: Interaction Analysis ModelPhase I: Sharing/comparing of informationA. A statement of observation or opinion [IA]B. A statement of agreement from one or more other participants [IB]C. Corroborating examples provided by one or more participants [IC]D. Asking and answering questions to clarify details of statements [ID]E. Definition, description, or identification of a problem [IE]

Phase II: The discovery and exploration of dissonance or inconsistencyamong ideas, concepts, or statementsA. Identifying and stating areas of disagreement [IIA]B. Asking and answering questions to clarify the source and extent of

disagreement[IIB]

C. Restating the participant’s position, and possibly advancing argumentsor consideration in its support by references to the participant’sexperience, literature, formal data collected, or proposal of relevantmetaphor or analogy to illustrate point of view

[IIC]

Phase III: Negotiation of meaning/co-construction of knowledgeA. Negotiation or clarification of the meaning of terms [IIIA]B. Negotiation of the relative weight to be assigned to types of argument [IIIB]C. Identification of areas of agreement or overlap among conflicting

concepts[IIIC]

D. Proposal and negotiation of new statements embodying compromise,co-construction

[IIID]

E. Proposal of integrating or accommodating metaphors or analogies [IIIE]

Phase IV: Testing and modification of proposed synthesis or co-constructionA. Testing the proposed synthesis against “received fact” as shared by the

participants and/or their culture[IVA]

B. Testing against existing cognitive schema [IVB]C. Testing against personal experience [IVC]D. Testing against formal data collected [IVD]E. Testing against contradictory testimony in the literature [IVE]

Phase V: Agreement statements/applications of newly constructed meaningA. Summarization of agreements [VA]B. Applications of new knowledge [VB]C. Metacognitive statements by the participants illustrating their

understanding that their knowledge or ways of thinking (cognitiveschema) have changed as a result of the conference interaction

[VC]

(continued)

the exchanges made among class members and how these exchangesmoved from the lower to the higher phases of CT. According toGunawardena, Lowe, and Anderson (1997), movement from one phase tothe next shows that knowledge is constructed by the process of social nego-tiation. The transcript analysis procedure involves reading and coding themessages to one or more of the five phases.

Newman, Webb, and Cochrane (1995), as shown in Table 1, Part B,provided a content analysis method to measure CT in computer-mediated

170


Table 1 (Continued)

Part B: Analysis Model for Analyzing Depth of Critical ThinkingR+ – Relevance

R+ Relevant statements to the issue discussedR– Totally irrelevant statements to the issue discussed

I+ – Importance/SignificanceI+ Important/significant points/issuesI– Totally unimportant, trivial points/issue

N+ – NoveltyN+ Provide new information, ideas, or solutions that have never

been mentioned (even they are not important or useful)N– Repeat what has been already said without any further

explorationA+ – Accuracy

A+ The references/literature used or information/data collected tosupport the participant’s position are accurate and true

A– The references/literature used or information/data collected tosupport the participant’s position are clearly false

J+ – JustificationJ+L+ Provide a logical statement of opinion, agreement, or

disagreement with supportingreasons/examples/justifications/proof

J+L– Provide an illogical statement of opinion, agreement, ordisagreement with supportingreasons/examples/justifications/proof

J– Statement with simple agreement, disagreement, or alternativeopinions without elaboration

C+ – Critical AssessmentC+L+ Critical assessment/evaluation of one’s own previous

statements/reflection or others’ contributions toward the issuediscussed with logical thinking process

C+L– Critical assessment/evaluation of one’s own previousstatements/reflection or others’ contributions toward the issuediscussed with illogical thinking process

C– Uncritical or unreasoned acceptance/reject

group learning. To measure the frequencies of specific critical (+) andnoncritical (–) skills demonstrated in the discussions, they developed aset of paired CT indicators (x+ –), such as R+ – and N+ –, where x areCT indicators such as relevance (R) and novelty (N). x+ is the count ofpositive statements of a CT indicator and x– is the count of negativestatements in a transcript. Statements from the discussion transcriptswere analyzed and scored across the list of indicators. Once the scriptswere marked, the totals for each + or – indicator were counted, and adepth of CT ratio (x ratio) was calculated for each of CT indicators: x ra-tio = (x+ – x–)/(x+ + x–), converting the counts to a –1 (all uncritical) to +1(all critical) scale. For example, 48 positive statements of relevance (R+)and three negative statements of relevance (R–) were found in a tran-script. The CT ratio of relevance (x ratio) is 0.88 ((48–3)/(48+3)). Thiswas done to produce a measure that was independent of the quantity ofparticipation, reflecting only the quality of the messages. While the aimof Newman, Webb, and Cochrane’s (1995) coding scheme was to mea-sure the amount and type of CT taking place in group learning to checkon the possibilities of using computer conferencing to promote deeplearning, Newman, Webb, and Cochrane did not attempt to measure theinteractive dimension taking place in the discussion.

Gunawardena, Lowe, and Anderson (1997) and Newman, Webb, andCochrane (1995)both focusedon theproblemofassessing thequalityof inter-actions in a CMC environment. While Newman, Webb, and Cochrane aimedat assessing how much and what kind of CT takes place, Gunawardena, Lowe,and Anderson aimed at evaluating the processes of critical discussions fromthe “interactive” and “cognitive” dimensions. Therefore, the tag codes withcritical (+)anduncritical (–)valencesused inNewman,Webb,andCochrane’scoding scheme were incorporated into Gunawardena, Lowe, and Anderson’sinteractionmodel to formanewcodingsystem, thecodingschemeforevaluat-ingcritical thinking incomputer conferencing, forour study (seeTable1).Thecombined coding system evaluated both whether students demonstrated CTskills in their online discussions and whether students’discussion moved fromthe lower to the higher phases of CT.

Training session for establishing interrater reliability. To establishadequate interrater reliability for this study, the data from two online dis-cussions in the pilot study were evaluated by two raters—the course in-structor and the researcher—using the final coding scheme (Table 1). In thetraining session, the researcher first provided the article, “CriticalThinking: What It Is and Why It Counts” (Facione 1998) and a coding

171

YANG, NEWBY, BILL

manual detailing the exact criteria and procedure for assigning a code fordifferent units. After reading the article and the manual, the raters dis-cussed the concept of critical thinking together, and then reviewed as wellas negotiated the definition of twenty-one CT categories and fourteen dif-ferent tags in the coding theme one by one.

Consequently, the raters separately evaluated and assigned ratings to thefirst online discussion in the pilot study. The coding results were compared bytheraters toreachanoverallagreementonthedefinitionsandcriteriafor these-lection of the codes or tag codes. Using Miles and Huberman’s (1994)interrater reliability formula (reliability = number of agreements / (total num-ber of agreements + disagreements)), the interrater reliability rate for this ini-tial coding was determined to be 83.27% agreement. Of the total 245 codingdecisions made, there were 41 differences. When differences occurred, the re-searcher and the instructor discussed the discrepancies in the coding resultsuntilaconsensuswasreachedthroughmutualconversation.After thefirstcod-ing, the instructorand the researcher independentlycoded theotheronlinedis-cussion in the pilot study and then discussed the results before actually codingthe online discussion postings for the study. The interrater reliability rate forthe second coding results reached 93.12%. Based on this rating, both the accu-racy and reliability of using this coding instrument meet the generalcheck-coding standard that is required in the 90% range (Miles and Huberman1994). This check-coding process was a time-consuming task. However, realrewards were gained by bringing an unequivocal, common vision of what thecodes meant and which blocks of data best fit which codes and by establishingconsistency of judgment among the raters.

Coding procedures. One of the first tasks in the coding procedure wasto parse the discussion transcripts into a unit of analysis—that is, what por-tion of communication will be used as the smallest unit to analyze. A unitof analysis can be a phrase, sentence, paragraph, or message that illustratesany one of the indicators. The instructor and researcher discussed, negoti-ated, and then parsed the discussion transcripts into units of analysis to-gether. Then, they independently rated each unit across the Gunawardena,Lowe, and Anderson (1997) category of interactions and all six ofNewman, Webb, and Cochrane’s (1995) criteria, if applicable.

The following examples are used to explain how each unit of the onlinepostings was coded. For example, if Student A started a new (N+) discus-sion (IA), which is relevant (R+) and important (I+) to the discussion issue,but the statement was inaccurate (A–) and without supporting arguments(J–), the coding was [IA/R+/I+/N+/A–/J–]. Please note that the indicator

172


“critical assessment (C)” was not suitable in this case because Student Awas a starter to the discussion and no other statements/opinions were avail-able for him to evaluate. In a second example, Student B critically evalu-ated Student A’s message with logical thinking process (C+J+) and repliedby stating a contradictory viewpoint (IIA) that no one mentioned before(N+). In addition, Student B’s statement was accurate (A+), relevant (R+),and important (I+) with logical supporting arguments (J+L+), therefore itwas coded [IIA/R+/I+/N+/A+/J+L+/C+J+]. After coding, we counted thetotal number of units of analysis that occurred in each of the phases and op-erations according to the coding scheme for evaluating critical thinking incomputer conferencing (see Table 2). In addition, similar to what Newman,Webb, and Cochrane (1995) did, the totals for each + or – indicator werecounted, and a depth of CT ratio (x ratio) was calculated for each criterion(see Table 3). Both the researcher and the instructor coded all of the fouronline discussions in Treatments I and II and calculated the interraterreliabilities for each discussion separately. Using Miles and Huberman’s(1994) formula, the interrater reliabilities for the eight coded online discus-sions ranged from 90.31% to 95.36%.

Results and Discussion

CCTST

The mean scores on the precourse CCTST were 18.27 (SD = 3.88) and18.40 (SD = 3.36) in Treatments I and II, respectively. The mean scores onthe postcourse CCTST were 19.73 (SD = 4.76) and 21.00 (SD = 4.42) inTreatments I and II, respectively. The p values reveal that both the Treat-ment I group (t(10) = 3.07, p = .0059) and the Treatment II group (t(4) =5.10, p = .0035) showed significant gains on the results of postcourseCCTST. These results indicate that both Treatment I and II groups im-proved their CT skills by the end of the semester.

Asynchronous Online Discussions

Table 2 consists of two parts—Part A and Part B—listing the numberof units of analysis which were coded in each phase and its operationsfor the two treatments. There were a total of 228 and 298 units of analy-sis in the first half of the discussion in Treatments I and II, respectively.There were a total of 216 and 205 units of analysis in the second half ofthe discussion in Treatments I and II, respectively. Students tended to dis-cuss more in the first half of the semester than in the second half. This is

173

YANG, NEWBY, BILL

174

Table 2. Results of Asynchronous Online Discussions by InteractionCategory for Treatment I Group and Treatment II Group

First Half of the Semester(Discussions 1 and 2)

Second Half of the Semester(Discussions 3 and 4)

InteractionCategory

# of Unit ofAnalysis Total (%)

# of Unit ofAnalysis Total (%)

Part A: Treatment I Group

IA 133 132IB 10 1

Phase I IC 7 172 (75.44%) 1 138 (63.89%)ID 15 4IE 7 0

IIA 17 12Phase II IIB 6 23 (10.09%) 20 34 (15.74%)

IIC 0 2

IIIA 2 3IIIB 0 0

Phase III IIIC 3 6 (2.63%) 8 14 (6.48%)IIID 1 3IIIE 0 0

IVA 0 1IVB 0 0

Phase IV IVC 0 0 (0.00%) 0 1 (0.46%)IVD 0 0IVE 0 0

VA 2 3Phase V VB 7 27 (11.84%) 8 29 (13.43%)

VC 18 18

Total 228 228 (100%) 216 216 (100%)

Part B: Treatment II Group

IA 158 118IB 10 10

Phase I IC 2 190 (63.76%) 4 140 (68.29%)ID 20 5IE 0 3

(continued)

likely because the students were busy studying and completing theircourse work toward the end of the semester, so they spent less time dis-cussing online. On average, while each Treatment I participant contrib-uted 40.36 units of analysis ((228 + 216) units/11 participants), each par-ticipant in Treatment II generated 100.60 units ((298 + 205) units/5participants), respectively. The number of units of analysis, in general, isapproximately proportionally equivalent to the number of messages.Therefore, these results imply that if the instructor started to teach andmodel CT skills at the outset of the online discussion (Treatment II)rather than in the middle of the semester (Treatment I), students seemedto be more motivated to participate, and the discussion tended to be moredynamic (100.60 vs. 40.36 units per participant in Treatments II and I,respectively).

Efficacy of teaching and modeling Socratic questioning for developingstudents’ CT skills in ADF. In Treatment I, the total units of analysis inthe first half of the semester (228) were approximately the same as those inthe second half (216) (Table 2, Part A). In the first half of the ADF, there

175

YANG, NEWBY, BILL

Table 2 (Continued)

IIA 17 15Phase II IIB 30 51 (17.11%) 14 29 (14.15%)

IIC 4 0

IIIA 5 5IIIB 0 0

Phase III IIIC 7 16 (5.37%) 4 9 (4.39%)IIID 4 0IIIE 0 0

IVA 0 0IVB 0 0

Phase IV IVC 0 0 (0.00%) 0 0 (0.00%)IVD 0 0IVE 0 0

VA 6 1Phase V VB 1 41 (13.76%) 0 27 (13.17%)

VC 34 26

Total 298 298 (100%) 205 205 (100%)

were 75%, 10%, 3%, 0%, and 12% of units in Phases I, II, III, IV, and V, re-spectively. In the second half of the ADF, 64%, 16%, 6%, 0.5%, and 13% ofthe units were coded in Phases I, II, III, IV, and V, respectively (see Table 2,Part A). These data show that during the second half of the semester, thediscussion tended to move toward higher phases. That is, students seemedto more critically ask and answer questions to clarify ideas and to negotiatemeaning and identify areas of agreement or overlap among conflicting con-cepts, χ2(4, N = 444) = 9.80, p = .0439.

Likewise, a statistically significant difference was found when com-paring the quality of interaction between the first half of the discussion inTreatment II, where Socratic questioning was also facilitated and mod-eled, and that in Treatment I, where no Socratic questioning was facili-tated, χ2(4, N = 526) = 9.77, p = .0444. These two chi-square results indi-cate that students who participated in Socratic dialogues, as modeled andfacilitated by the instructor during ADF (the second half and the first halfof the semester in Treatments I and II, respectively), had a higher qualityof interactions than students who participated in an ADF where Socraticdialogues were not fostered (the first half of the semester in Treatment I).

Persistence of students’ CT skills. In Treatment II, 64%, 17%, 5%,0%, and 14% of units occurred in Phases I, II, III, IV, and V, respectively, inthe first half of the ADF. In the second half of the ADF, 68%, 14%, 4%, 0%,and 13% of the units were coded in Phases I, II, III, IV, and V, respectively(see Table 2, Part B). The data indicate that the first half of the discussionhad slightly higher percentages of units of analysis at higher phases thanthe second half of the discussion. This implies that when the instructor dis-continued asking and modeling Socratic questions during the second halfof the semester in Treatment II, the students’ discussion moved slightlyback to Phase I. However, according to the results of the chi-square test,χ2(4, N = 503) = 1.32, p = .8583, the quality of interactions in terms of theinteractive and cognitive dimensions was not significantly different. Thatis, the quality of interaction in the second half of the discussion, where theinstructor discontinued modeling and facilitating Socratic dialogues, wasnot significantly lower than that in the first half of the discussion in Treat-ment II. In addition, the percentages in higher phases in the second half ofthe Treatment II discussion were still higher than those in the first half ofthe Treatment I discussion, where students were not taught to think criti-cally (see Table 2, Parts A and B).

Table 3 also consists of Parts A and B. The number of codes and thedepth of CT ratio for each of the indicators are presented.


176

177

Table 3. Depth of Critical Thinking by Indicator for Treatment I Groupand Treatment II Group

First Half of the Semester(Discussions 1 and 2)

Second Half of the Semester(Discussions 3 and 4)

Tag CodeNumber of

CodesDepth of CT

RatioNumber of

CodesDepth of CT

Ratio

Part A: Treatment I Group

R (relevance) R+ 163 91.76% 155 96.20%R– 7 3

I (importance) I+ 142 91.89% 151 97.39%I– 6 2

N (novelty) N+ 101 61.60% 96 97.94%N– 24 1

A (accuracy) A+ 90 60.71% 93 89.80%A– 22 5

J (justification) J+L+ 82 88J+L– 12 45.13% 1 89.25%J– 19 4

C (criticalassessment)

C+L+ 12 22

C+L– 0 50.00% 0 91.30%C– 4 1

Total 684 622

Part B: Treatment II Group

R (relevance) R+ 180 97.80% 115 96.58%R– 2 2

I (importance) I+ 152 96.13% 118 98.32%I– 3 1

N (novelty) N+ 126 95.35% 66 94.12%N– 3 2

A (accuracy) A+ 158 90.36% 62 90.77%A– 8 3

J (justification) J+L+ 122 49J+L– 1 86.26% 1 88.46%J– 8 2

C (criticalassessment)

C+L+ 28 17

C+L– 0 93.10% 0 88.89%C– 1 1

Total 792 439

Note: Depth of CT (critical thinking) ratio = (positive indicator – negative indicator) /(positive indicator + negative indicator).

Efficacy of teaching and modeling Socratic questioning for developingstudents’ CT skills in ADF. The depth of CT ratios in Treatment I foreach indicator in the second half of the ADF discussion (see Table 3, PartA) were all higher than those in the first half of the discussion. Both thefirst half and the second half of the discussion had high R (relevance)(92% vs. 96%) and I (importance) (92% vs. 97%) ratios. The partici-pants, in general, brought in relevant materials and seemed to haveadopted a serious style when taking part in the discussion. There werenotable differences for N (novelty) (62% vs. 98%), A (accuracy) (61% vs.90%), J (justification) (45% vs. 89%), and C (critical assessment) (50%vs. 91%). This is possibly because of the Socratic questioning that wasintroduced and taught during the second half of the discussion. Studentsseemed to take more thought before posting their messages. As a result,the number of negative statements for N, A, J, and C was reduced; thus,the depth of CT ratios for these indicators greatly increased. Thechi-square results, χ2(1, N = 1306) = 50.77, p < .0001, statistically affirmthis claim.

A similar result was found when comparing the depth of CT ratios be-tween the first half of the discussion in Treatment II, where students partic-ipated in Socratic dialogues, and that in Treatment I, where students did notparticipate in Socratic dialogues, χ2(1, N = 1476) = 53.76, p < .0001. There-fore, it is concluded that students’ CT skills improved after participating inSocratic dialogues, as modeled and facilitated by the instructor duringADF.

Persistence of students’ CT skills. Upon inspecting the depth of CTratios in the second half of the Treatment II, it is evident that the depthof CT ratios did not decline when compared with those for the first halfof the Treatment II discussion, R (relevance) (98% vs. 97%), I (impor-tance) (96% vs. 98%), N (novelty) (95% vs. 94%), A (accuracy) (90%vs. 91%), J (justification) (86% vs. 88%), and C (critical assessment)(93% vs. 89%). This claim was verified by the results of the chi-squaretest, χ2(1, N = 1231) = .28, p = .5935. In addition, the depth of CT ra-tios in the second half of the Treatment II discussion were much higherthan those in the first half of the Treatment I discussion, where studentswere not taught to think critically (see Table 3, Parts A and B). There-fore, the results imply that without the instructors’ further facilitation toimprove students CT skills, the discussion maintained about the samequality as that facilitated by the instructor.

178


Conclusions

CT is an important issue in higher education, and educators have contin-ued to focus on the development of CT in students (Beyer 1995; Ennis1985; Hilgenberg and Tolone 2000; Paul and Heaslip 1995; Taube 1997).This study focused on investigating the effects of using Socratic question-ing to enhance students’ CT skills in an ADF environment. The empiricalresults of this study indicate that with appropriate course design and in-structional interventions, CT skills can be cultivated and maintained inADF. This is probably because an ADF affords students the time forthoughtful analysis, composition, negotiation, and reflection as their dis-cussion of an issue evolves and allows instructors to model, foster, andevaluate the CT skills exhibited during the discussion. According to the ex-perience from this study, CT skills are difficult to foster, but not impossible.If students are asked to experience, explore, and test their ways of thinking,they will find it to be substantial work; however, positive gains in students’CT skills and attitudes provided evidence that teaching and learning CT isworth the effort. Therefore, it is encouraged that distance educators andcourseware designers take the challenge to create an active learning envi-ronment where CT is valued and where students are motivated and sup-ported in their attempts to think critically via an ADF.

References

Anderson, T., and D. R. Garrison. 1995. Critical thinking in distance edu-cation: Developing critical communities in audio teleconferencing con-text. Higher Education 29 (2): 183–199.

Angelo, T. A. 1995. Beginning the dialogue: Thoughts on promoting criti-cal thinking: Classroom assessment for critical thinking. Teaching ofPsychology 22 (1): 6–7.

Beyer, B. K. 1995. Critical thinking. Bloomington, IN: Phi Delta Kappa.Bullen, M. 1998. Participation and critical thinking in online university dis-

tance education. Journal of Distance Education 13 (2): 1–32.Cooper, J. L. 1995. Cooperative learning and critical thinking. Teaching of

Psychology 22 (1): 7–8.Dillon, J. T. 1988. Questioning and teaching: A manual of practice. New

York: Teachers College Press.Duffy, T. M., B. Dueber, and C. Hawley. 1998. Critical thinking in a dis-

tributed environment: A pedagogical base for the design ofconferencing systems. In Electronic collaborators: Learner-centered

179

YANG, NEWBY, BILL

technologies for literacy, apprenticeship, and discourse, ed. C. J. Bonkand K. S. King, 51–78. Mahwah, NJ: Lawrence Erlbaum Associates,Inc.

Ennis, R. H. 1985. A logical basis for measuring critical thinking skills.Educational Leadership 43 (2): 44–48.

Facione, P. A. 1990a. The California Critical Thinking Skills Test(CCTST): Form A (1990) (1992). Millbrae, CA: California AcademicPress.

———. 1990b. Critical thinking: A statement of expert consensus for pur-poses of educational assessment and instruction. Research findings andrecommendations. ERIC, ED 315423.

———. 1992. The California Critical Thinking Skills Test (CCTST FormB (1992). Millbrae, CA: California Academic Press.

———. 1998. Critical thinking: What it is and why it counts. 1998 Update:California Academic Press. Available online athttp://www.calpress.com/pdf_files/what&why.pdf

Flammer, A. 1981. Towards a theory of question asking. Psychological Re-search 4:407–420.

Garrison, D. R. 1993. Quality and access in distance education theoreticalconsiderations. In Theoretical principles of distance education, ed. D. J.Keegan. New York: Routledge.

Gunawardena, C., C. Lowe, and T. Anderson. 1997. Analysis of global on-line debate and the development of an interaction analysis model for ex-amining social construction of knowledge in computer conferencing.Journal of Educational Computing Research 17 (4): 395–429.

Hettinger, G. 1995. Raising the level of the debate: The effects of computermediated communication on group dynamics and critical thinkingskills. ERIC, ED 383300.

Hilgenberg, C., and W. Tolone. 2000. Student perceptions of satisfactionand opportunities for critical thinking in distance education by interac-tive video. The American Journal of Distance Education 14 (3): 59–73.

Jeong, A. 2000. Bulletin boards support critical thinking in small groupdiscussions. University of Wisconsin-Madison, Learning Technologyand Distance Education. Available online at http://ltde.tripod.com/groupware/bbcriticalthinkingsurvey.htm

Kerry, T. 1987. Classroom questions in England. Questioning Exchange 1(1): 32–33.

King, A. 1990. Enhancing peer interaction and learning in classroomthrough reciprocal questioning. American Educational Research Jour-nal 27 (4): 664–687.

180


———. 1995. Designing the instructional process to enhance criticalthinking across the curriculum: Inquiring minds really do want to know:Using questioning to teach critical thinking. Teaching of Psychology 22(1): 13–17.

Maiorana, V. P. 1990–91. The road from rote to critical thinking. Commu-nity Review 11 (1–2): 53–63.

McDade, S. A. 1995. Case study pedagogy to advance critical thinking.Teaching of Psychology 22 (1): 9–10.

Miles, M. B., and A. M. Huberman. 1994. Qualitative data analysis: An ex-panded sourcebook. Thousand Oaks, CA: Sage.

Newman, D., B. Webb, and C. Cochrane. 1995. A content analysis methodto measure critical thinking in face-to-face and computer supportedgroup learning. Interpersonal Computing and Technology: An Elec-tronic Journal for the 21st Century 3 (2): 56–77.

Paul, R. W. 1993. Critical thinking: What every person needs to survive ina rapidly changing world. Santa Rosa, CA: Foundation for CriticalThinking.

Paul, R. W., and P. Heaslip. 1995. Critical thinking and intuitive nursingpractice. Journal of Advanced Nursing 22 (1): 40–47.

Taba, H. 1966. Teaching strategies and cognitive functioning in elementaryschool children. Cooperative Research Project, No. 2404. San Fran-cisco: San Francisco State College.

Taube, K. T. 1997. Critical thinking ability and disposition as factors ofperformance on a written critical thinking test. Journal of General Edu-cation 46 (2): 129–164.

Walker, S. A. 2004. Socratic strategies and devil’s advocacy in synchro-nous CMC debate. Journal of Computer-Assisted Learning 20 (3):172–182.

181

YANG, NEWBY, BILL

using socratic questioning to promote critical thinking ... · pdf fileusing socratic...

Documents