SCAFFOLDING CRITICAL DISCOURSE IN ONLINE PROBLEM-BASED SCENARIOS : THE ROLE OF ARTICULATION AND EVALUATIVE FEEDBACK

Gihan Osman – The Arab Academy for Science & Technology, Egypt

Thomas Duffy – Indiana University Bloomington, U.S.A

Critical Thinking as an Outcome for Higher Education

Critical thinking is a vital goal for higher education (e.g. Browne & Freeman, 2000; Costa, 1991; Pithers & Soden, 2000)

Critical thinking is not a guaranteed outcome of college (e.g., Halpern, 1996; Pascarella & Terenzini , 2005)

Critical Thinking in Online Settings Dearth of research that focuses on

critical thinking as a distinct skill a willingness (a predisposition) and an

ability to scrutinize and evaluate thinking – one’s own, as well as others’ – in order to determine truth, accuracy, or worth, and to construct logical arguments to justify claims or assertions. (Beyer, 1990)

Questions that guided the study conceptually How can educators create computer-

mediated activities and learning environments that promote critical thinking online?

What processes are relevant to critical thinking in online environments?

What are some of facilitation strategies that are effective in promoting critical thinking?

-

Creating the Appropriate Conditions for Critical Thinking in Online Contexts

Motivation and Relevance

Tasks should be problem-based (Dewey, 1993; Duffy, 2002)

Tasks need to be personally relevant (Barab & Duffy, 1998)

Tasks need to be authentic in their complexity (Brown et al, 1989; Innes, 2004; Resnik, 1987)

Tasks need to encourage use of different contextual resources (Dewey, 1938; Dewey & Bentley, 1949)

Scaffolding

Most learners need scaffolding (Quintana, Reiser, Davis, Krajcik, Fretz, Duncan, Kyza, Edelson, & Soloway,2004)

Scaffolding = instructional techniques that would help learners attain goals and use skills that would otherwise be out of their reach (Davis & Miyake, 2004).

Scaffolding

Most learners need scaffolding for critical thinking

Scaffolding

Most learners need scaffolding for critical thinking

Successful scaffolding = identification of processes that promote critical thinking

EXTERNALISATION

Scaffolding

Most learners need scaffolding for critical thinking

Successful scaffolding = identification of processes that promote critical thinking

EXTERNALISATION

Externalization = manifestation of changes in thought and attitude in the environment

Scaffolding

Most learners need scaffolding for critical thinking

Successful scaffolding = identification of processes that promote critical thinking

EXTERNALISATION

Externalization = manifestation of changes in thought and attitude in the environment

ARTICULATION

EVALUATIVE FEEDBACK

Articulation

Articulation = expression of thought in language – verbally or in written form.

language is critical to organize one’s thinking (Schunk,1999)

the act of choosing words to represent our thoughts allows us to consider our ideas more deliberately (Vygotsky, 1987)

Even in the absence of an audience articulation should promote critical thinking (e.g., journal, self-talk) – implications for unreferenced messages and silence

The presence of an audience, should further promote critical thinking

Articulation

The presence of an audience, should further promote critical thinking

writing for an audience helps learners engage in higher levels thinking by pushing them to ‘compact’ internal speech into a form that is understandable to others (Vygotsky, cited in Pugalee, 2004)

“writers do not simply express thought but transform it in certain complex but describable ways for the needs of a reader “(Flower, 1997; p.20)

Hypothesis 1: Students who articulate their views more often manifest a higher level of critical thinking.

Feedback

Expansive activity an increase in the number of ideas available to the individual (exchange of information) enriches opportunities for discovering areas of cognitive conflict does not necessarily have to lead to cognitive conflict

Learner has collector’s attitude Learner does not discern conflicting ideas Learner decides to bail out

Dialectical activity = individuals asking for clarification challenging ideas, presenting counter-ideas Ensures opportunities for discovering areas of cognitive conflict

Feedback should be dialectical rather than expansive

Hypothesis 2. Looking across all students, those students who had more dialectical feedback on their posts will exhibit greater critical thinking.

Scaffolding: Facilitator Role

Facilitation in this study will focus on: Guidance to participate (articulation) Guidance to evaluate (dialectical feedback)

Questions 3-5: To what extent will learners in the “Guidance to Evaluate” and “Guidance to Participate” scaffolding conditions differ in the:

quantity of contributions they make to online discussions?

quantity of critical thinking contributions they make to online discussions?

proportion of critical thinking contributions they make to online discussions?

Methodology

Participants & Context

74 undergraduate Education students Recruited from four sections from an Educational

Psychology course Mean age: 19.8 97.2% English as mother tongue 35.2% freshmen, 47.9% sophomores, 15.5% juniors,

1.4% seniors 91.5% specialized in secondary education 67.6% had no teaching experience 71.8% had not experienced discussion forums before

study

Tasks

Discussion

scaffolding

Case 1 Case 2

Warm-up No

scaffolding

Recall Test

Group Report

Asynchronous discussion forums Face-to-face

Design

A single factor, between subjects design with two levels of scaffolding was used.

Participants were randomly assigned to one of two conditions - adding up to 10 groups per condition.

The treatments were administered only during the second case.

Conditions

Focus on participation Examples:

“Any new ideas about the case, [name of group member]?”

“[name of group member] and [name of group member] have expressed their thoughts about the case. What do you think?”

Individually prompted via e-mail Every 24 hours as needed

Focus on evaluation & dialectical thinking

Examples: “Do you think these arguments are strong or

weak? Why?”

“Can you think of evidence that could support [name of other member in the group]’s claim?”

Individually prompted via e-mail

Every 24 hours as needed

“Guidance to Participate” Condition

“Guidance to Evaluate” Condition

(Greenlaw & DeLoach, 2003)

Level of critical thinking (Greenlaw & DeLoach, 2003)

Level 6: consideration of ethical issues in decisions

Level 5: use of empirical evidence to strengthen theoretical argument

Level 4: use of theory to make a cohesive argument

Level 3: analysis of an argument or competing arguments and evaluation with evidence

Level 2: unsupported assertions, simplistic one-sided arguments

Level 1: paraphrase, repetition

Level 0: organizational and off-task

Dependent Variables

Level of participation in the forum = total number of ideas units contributed by each student to the discussion.

Critical thinking coded according to Greenlaw and DeLoach (2003) Quantity = number of idea units at Level 3+

Proportion = proportion of idea units at level 3 and above relative to the total number of ideas units

Interviews and post-treatment questionnaire

Semi-structured interviews conducted to provide opportunity for reflection on the experience. 9 participants were interviewed.

Post-treatment questionnaire designed and administered after the interviews to validate emerging themes. Consisted of 30 Likert-scale items +four open-ended

questions 31 of the 74 participants answered the questionnaire.

Findings & Discussion

Data coding

Data was divided into idea units. Budd, Thorp, and Donohue (1967) define idea unit as “a single thought

unit … that conveys a single item of information extracted from a segment of content” (p. 34).

All data coded by two raters Inter-rater reliability = 89% (Cohen’s

Kappa=0.8) for discussion forums.

Was the task motivating? Relevant?

Students found the cases relevant to the course and their future as teachers 71% found that cases very relevant to their

futures as teachers

65% indicated that the cases helped them make the connection between theory and practice

61% suggested that the case studies enhanced their understanding of the course concepts

“It proves that what you’re learning in class is pertinent when you go out and become a teacher. …when ….I started this class, I didn’t see a point of understanding how people memorized things and how they recognize - you know, it just didn’t seem a point to taking the class. I wasn’t very enthusiastic about being in there. But um, to see that you actually will have to um, reflect over , - not really exactly the theories but at least the concepts of the theories …I think is important. And even not just in that class but in other classes. To kind of show that, “Yes, this is relevant and you will be faced with this.” “These are situations that have occurred.”

Karen

What was the level of participation?

About 94% of the participants reported they had read the postings of other members in their group.

Students contributed: 187 messages/week (1149 idea units) 2.5 messages/person/week (15.53

units/person/week)

Participation was average (undergraduate)

Contextual barriers

Lack of familiarity with experience of case study

Lack of clear expectations Demands related to other courses Technological problems with Oncourse

“AHHHHHHHHHHHHHHHHHHH / I am going to kill myself!!!!!! I just added a whole PAGE on ARCS model and now I can't find it to upload it..... I'll keep searching, if not I'll rewrite it.......... damn it!!!!!!!!!!!!”

Joseph

“For me, it was, like, just having to sit down and think critically just…it’s just thinking about it. Like you weren’t gonna find the answer in a book. ….So, that was the hardest part just thinking of a good like, intelligent answer that has reasoning from the class and being able to portray those ideas to my group members. That’s probably why I only posted two or three times. Like, because it was just so hard to think of, to think of a good solution…..yeah.”

Laura

What do students say about their experience? 65% stated that “I supported my ideas with evidence

from theories covered during the course.” Only 13% reported difficulties with supporting their

arguments with evidence Informal analysis of the evidence shows support in the form

of: opinions , examples, and personal experiences

68% indicated that the value of reading others’ posts lies in encountering different perspectives and examples.

95% stated that it is more important to maintain group harmony than reach the best solution.

What did students do in terms of critical thinking?

Level of critical thinking skills was rather lowLevel of critical thinking (Greenlaw & DeLoach,

2003)Percentage

Level 6: consideration of ethical issues in decisions 0%

Level 5: use of empirical evidence to strengthen theoretical argument

0%

Level 4: use of theory to make a cohesive argument 0.1%.

Level 3: analysis of an argument or competing arguments and evaluation with evidence

7%

Level 2: unsupported assertions, simplistic one-sided arguments

57.8%

Level 1: paraphrase, repetition 23.3%

Level 0: organizational and off-task 11.8%

Cri

tica

l Th

inki

ng

Oth

er

What did students do in terms of critical thinking?

Informal analysis of the evidence showed that students’ support of arguments was in the form of: opinions examples personal experiences

Dialectical thinking (challenges, counter-arguments, and requests for clarification) 18 out of 1149 idea units (0.01%).

Was there a relationship between level of participation & critical thinking?

Total number of students

Quantity of Critical Thinking

Proportion of Critical Thinking Idea Units

Total Idea Units

Pearson Correlation

74 .511(**) .054

** Correlation is significant at the 0.01 level (2-tailed).

Is there a relationship between level of participation & critical thinking? Although increase in participation increased the

number of critical thinking contributions in a discussion, it did not enhance the overall quality of discussion in terms of critical thinking density. Significant positive relationship was found between

level of participation and quantity of critical thinking.

No significant relationship was found between level of participation and proportion of critical thinking.

Does “Guidance to Evaluate” result in greater participation and critical thinking?

* Correlation is significant at the 0.05 level.

Facilitator Scaffolding

There was a main effect for facilitator scaffolding on the proportion of critical thinking, but no impact for scaffolding for level of participation in the discussion. Students in both conditions manifested very

similar levels of participation.

Students in the “Guidance to Evaluate” condition had double the proportion of critical thinking compared to participants in the “Guidance Articulate” condition.

Conclusions, Limitations & Future Research

Conclusions

Case-based scenarios were relevant to students, but this was not enough to increase critical thinking and participation.

Participation was average and critical thinking was mediocre Students had different perceptions of what good argumentation and evidence

is. Students had a negative attitude towards evaluation and emphasized

harmony.

The findings provided partial support to the hypothesized relationship between articulation and critical thinking.

Facilitator scaffolding had a significant impact on students’ critical thinking, but not on participation.

Future Research

Use scaffolding techniques that are more specific, explicit, and global

Examining the transfer and development of critical thinking skills

Use tasks that encourage or legitimize the use of dialectical thinking

Use communication tools that are more dynamic Examine factors that impact critical thinking and

participation in problem-based tasks Contextual factors Individual factors Other factors?

Suggestions for teaching

Use tasks that make critical thinking and dialectical thinking necessary and desirable. Create a classroom in which critical thinking is fostered and rewarded

Scaffold critical thinking explicitly and concretely. Show what good evidence is (e.g., modeling, just in time direct

instruction) Focus on what the individual specific feedback.

Focus on the quality of participation. Setting concrete expectations. Engage students in challenging activities for a longer period of

time. Vary communication tools according to the demands of the task.

For more information please contact Gihan Osman at: gosman@aast.edu

Thank you!

Data Analysis

To examine these relationships, two forms of analysis were used. A hierarchical analysis of variance (HANOVA) followed by a one-way analysis of variance (ANOVA). We initially ran a HANOVA, rather than an ANOVA, in order to account for the possible group effect that might have resulted from the nested design used in this study (Walczuch & Watson, 2001).

If the initial HANOVAs revealed no significant group effects, the more powerful ANOVA technique was used.

Limitations of the Study

No control group Technological problems Limited access to and control of

participants Face-to-face meetings