Teaching and Learning Forum 2004 [Proceedings Contents]

Assessment for learning: Some insights from collaboratively constructing a rubric with post graduate education students

Lesley Newhouse-Maiden and Terry de JongEdith Cowan University

Learning to work collaboratively is an important component of successful middle schooling practice (Jackson & Davis, 2000). In their preparation to become middle years teachers, students completing the Postgraduate Diploma in Education (Middle Years) at Edith Cowan University (ECU) are required to engage in a range of collaborative group processes. An example of this is a small group-based project in theFrom Alienation to Engagementunit which necessitated students investigating and presenting their findings on the services offered to adolescents by youth-focussed agencies. As part of this project, students had to reflect upon their individual contribution to the group process. To help facilitate this, students were required to construct an assessment rubric and use this instrument to inform their reflections. This paper describes the rationale for and process used in constructing the rubric. It discusses four key themes of student learning identified from the students' reflection papers.

Introduction

This paper emanates from our desire to "practice what we preach" in educating post-graduate students to be and become effective middle years teachers (Newhouse-Maiden, 2002). We were concerned about aligning our unit of studyFrom Alienation to Engagementwhich is part of ECU's Postgraduate Diploma of Education (Middle Years) to two of theGraduate Attributes at ECU(2002), namely. "problem-solving/decision making" and "teamwork".

The task we set our students was to work in small teams to investigate a community agency that supported the needs of, and challenges faced by, young adolescents. Our students were required to "actively accomplish complex and significant tasks", while bringing to bear their "prior knowledge, recent learning, and relevant skills" (Stowell & McDaniel, 1997, p.142), and to present their findings to their peers. An additional challenge for members of the small groups was to evaluate the effectiveness of their individual contribution to the group.

Our challenge was to develop our students into effective middle school team members and to incorporate teamwork in their middle years pedagogy. In accord with good practice we facilitated a process which created an assessment rubric for each performance task (Jackson & Davis, 2000, p. 60), where a rubric:

... outlines a set of criteria, usually on a four-or six-point scale with performance descriptions that define a range of performance on an authentic assessment task (Stowell & McDaniel, 1997, p. 142).

Our reasoning was that students needed to develop their skills in rubric construction as future middle years teachers. We were guided by McTighe's (1996/1997) contention that "if we expect students to improve their performances on these new, more authentic measures, we need to engage in "performance-based" instruction on a regular basis" (p.7).

We also speculated that if we "crafted a rubric" with the help of students (Jackson & Davis, 2000), they might take more ownership for their performance and gain more "enduring understandings" of group processes as teachers if:

... at every point on the rubric's scale its creators must decide what specific evidence they should see of the understandings and skills the assessment is to incorporate" (p.60)

This rubric was subsequently used as a means of students giving their peers feedback about their contribution to the group. Each student had to collate the feedback (about five per student) and then reflect upon his/her contribution in an individual paper that was submitted as part of the group project assignment.

In sum, there were two key intended outcomes associated with the inclusion of the assessment rubric in theFrom Alienation to Engagementunit:

1. Using a collaborative approach, students should be able to construct an authentic assessment rubric applicable to the middle years of schooling which enhances learning, and reflects the key principles of validity and fairness.

2. Using the assessment rubric as a feedback source and self-reflection tool, students should have a better understanding of the complex dynamics of working in groups, and more insight into their current and potential contribution to group work.

Based on our 2003 course review and formative evaluations of our students completing their Assistant Teacher Practicum, there is clear evidence to suggest that these outcomes are being met. In endeavouring to understand the nature of this evidence better, we were curious to find out what our students appeared to have learnt about the potential of a collaboratively constructed rubric as an assessment "tool". Using Miles and Huberman's (1994) data analysis processes of data reduction, data display, and conclusion drawing we examined the students' reflection papers to respond to this enquiry. The purpose of our paper is to discuss our findings and reflect on what we have learnt from the process too. Before doing so, we briefly describe how the assessment rubric was constructed.

The process of constructing the assessment rubric

In facilitating the construction of the assessment rubric we were guided by four key principles (McTighe, 1996/1997), as follows.

Establish clear performance targets with understanding

It was our firm conviction that setting clear performance targets will encourage students to engage in a more meaningful learning experience. As (McTighe, 1996/1997, p.8) points out,

When students have opportunities to examine their work in the light of known criteria and performance standards they begin to shift their orientation from "What did I get? To "Now I know what I need to do to improve.

Summative outcomes are clearly important. The final grade carries weight. However, engaging students in an "action cycle" is a powerful way of encouraging a more equitable balance between "product" and "process". The rationale for collaboratively constructing the rubric was to ensure that students had a common understanding of the criteria and performance tasks so that the task of establishing group processes was made easier, and the teams' chances of meeting its goals (i.e. performance tasks) were maximised.

Teach strategies explicitly

During their orientation week students were introduced to the importance of building effective groups, and the implication of this for becoming a successful middle years teacher. We built on this experience and skill development by adopting Johnson and Johnson's (2003)The Distributed Actions Theory of Leadershipas a theoretical basis on which to conceptualise 'effective groups'. According to Johnson and Johnson members of effective groups are capable of completing the task and maintaining collaborative relationships among members. Members must engage in both "task-leadership actions" and "maintenance-leadership actions" to maximise successful outcomes for the group. The characteristics of effective groups were highlighted. They included: goals being clarified; two-way communication; distribution of participation and leadership; power is equalised; matching decision-making with procedures; conflict and controversy seen to be positive; high problem-solving adequacy; evaluation of effectiveness of the group; and innovation encouraged (Johnson and Johnson, 2003). We then covered the process of designing a rubric for assessing the collaborative process of small group work (Webb, 1997; Goodrich, 1996/1997).

We encouraged the groups to decide which dimensions of their collaborative work they wished to assess and to list what counts as quality work for the dimensions chosen (Goodrich, 1996/1997). Each of the thirteen groups of students worked on a rubric to "include enough detail to guide students' efforts to succeed" (Jackson & Davis, 2000, p. 60) as an effective contributor to the group.

Use ongoing assessments for feedback and adjustment

Using the guidelines of Webb (1997) and Goodrich (1996/1997), we designed a rubric to assess and give feedback on first drafts from each group (see Appendix A). The following week, we returned their rubrics with a "mark" and feedback for refinement. Each group was encouraged to "rework" their rubric by: checking whether only one element was being assessed in each point; where necessary, re-ordering the list in terms of importance; checking each element against the purpose of the assessment; and defining three gradations of quality for each characteristic using language that could be understood by university students (see Appendix B). Having identified shared characteristics, the class decided to create a common rubric based on the thirteen reworked rubrics. This was considered to enhance validity and reliability. In the third week, the common rubric was presented to the students for final comment before implementation. After their group's presentation, each student used the rubric (see below for the final version) to give their peers feedback about their contribution to the group work.

Rubric: Assessment of individual's contribution and impact on group project

PROJECT TITLE:

NAME:

OVERALL MARK: /10

The individual's ability to be a contributing member of the team is reflected in his/her:

Alwaysdemonstrated9-10

Oftendemonstrated7-8

Sometimesdemonstrated5-6

Notdemonstrated0-4

Participation in the group

Participates actively in group discussion

Has an overwhelmingly positive attitude

Prepared to accept and use others' ideas

Exceptional contribution as a team mem