teaching excellence seminar school of science university of science and technology the design and...
TRANSCRIPT
![Page 1: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/1.jpg)
The Design and Implementation of Outcome-Based Assessment
in Science
Teaching Excellence Seminar Teaching Excellence Seminar School of ScienceSchool of Science
University of Science and TechnologyUniversity of Science and Technology
22 March 201322 March 2013
Isabella Wai-Yin Poon Isabella Wai-Yin Poon
[email protected]@cuhk.edu.hk
Department of StatisticsDepartment of Statistics
Associate Dean of ScienceAssociate Dean of Science
Director of Centre for Learning Enhancement & ResearchDirector of Centre for Learning Enhancement & Research
The Chinese University of Hong KongThe Chinese University of Hong Kong
![Page 2: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/2.jpg)
2
Content
1. Introduction
2. An OBA Assessment Model
3. Assessment as Learning Activities
4. Evaluation of Outcome at Programme Level
5. Recent Initiatives at Science of CUHK
![Page 3: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/3.jpg)
3
Introduction
![Page 4: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/4.jpg)
4
Outcome-based Approach (OBA) OBA
Actively promoted by UGC since 2005 Weave into the (2012) 4-year curriculum
Many professors have no knowledge on this UGC
Not straightjacket Institutions should take ownership Inter-institutional Task Force established
![Page 5: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/5.jpg)
5
OBA: An International Trend? http://www.cwsei.ubc.ca
![Page 6: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/6.jpg)
6
OBA Development @ CUHK General consensus at CUHK
Clearly articulated learning outcomes Curriculum components align with learning
outcomes
• Content
• Learning activities
• Assessment Assessing outcomes and collecting feedback
![Page 7: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/7.jpg)
7
How We Proceeded? OBA has long been used
implicit rather than explicit manner These concepts are part of our culture
Although we are not eloquent in using the language
1st step: articulate programme-level learning outcomes Outcomes developed in 2006
A series of TDG projects (faculty- or department-levels) to implement initiatives that address issues of our concern Front-line initiatives carry the spirit of OBA Formulate in an OBA framework
![Page 8: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/8.jpg)
8
Science Teaching: Areas for improvement
Interaction in class Variety of learning activitiesCultivate the development of learning outcomes
Generic skills Value/attitude
Assessment Align with OBA? Summative vs Formative assessment
• Summative assessment: Provides evidence of student attainment• Formative assessment: Assessment for learning
Criterion- vs norm-referencing• Criterion: Pegged to pre-determined or expected learning otucomes• Norm: Defined by relative performance
Various TDG or other projects Faculty, programme or individual level
![Page 9: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/9.jpg)
9
Faculty-level TDG Project 1
“Engaging Science Students in the Design and Engagement of Assessment” October 2005 – November 2008 Website: http://www.cuhk.edu.hk/sci/TDG/
![Page 10: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/10.jpg)
10
Faculty-level TDG Project 2 “Promoting the Use of the Outcome-Based
Approach to Science Programmes: Development of Learning Outcome Statements and Measurement Instruments” October 2006 – November 2008 Website: http://www.cuhk.edu.hk/sci/OBA/
![Page 11: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/11.jpg)
11
Faculty-level TDG Project 3
“Using Clickers to Enhance Teaching and Learning: Pilot Case Studies in Science Programmes” December 2008 – October 2009 Website: http://www.cuhk.edu.hk/sci/clicker
![Page 12: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/12.jpg)
12
Faculty-level TDG Project 4
The Development of an OBA Assessment Model for Science Subjects” August 2009 to Dec 2012 Website: will soon launch
![Page 13: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/13.jpg)
13
Faculty-level TDG Project 5 “Inter-faculty Academic-related Advising Service”
November 2009 – Website: Restricted access (CUHK students only) http://serverfarm.elearningpro.com/cuhk/softskillmentor
![Page 14: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/14.jpg)
14
Faculty-level TDG Project 6 “Learning by Engaging and Teaching”
January 2013 – Website: http://www.cuhk.edu.hk/cpse/LET/
![Page 15: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/15.jpg)
15
An OBA Assessment Model
![Page 16: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/16.jpg)
16
Norm- vs criterion-referencing Heated debate
Science: Norm-referenced approach CUHK 2008 QAC Audit Report (p.23)
“There is therefore an argument that criterion-referencing as opposed to norm referencing is more consistent with OBA. The Panel encourages CUHK to engage with the pressure placed by OBA on norm-based grading and consider the place of criterion based assessment and the measures which may be introduced, other than norm referencing, to track and correct grade inflation”
Similar statements found in the QAC Audit reports for other UGC-funded institutions
HKBU, 2009, p.18; HKU, 2009, p.22; CITYU, 2010, p.21
![Page 17: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/17.jpg)
17
Two Components
The Assessment Framework How does a course, and in particular its
assessment, support the development of programme LOs ?
The Assessment Chart Develop the grade descriptors Use both norm referencing and criterion
referencing
![Page 18: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/18.jpg)
18
Assessment Framework
![Page 19: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/19.jpg)
19
Assessment Chart
![Page 20: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/20.jpg)
… Assessment Chart
20
![Page 21: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/21.jpg)
21
Characteristics of the Model User-friendly Align assessment with outcomes Scaffold the planning and design of
assessment (components) Communicate clear expectation to students A probabilistic but not a deterministic model
![Page 22: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/22.jpg)
22
A probabilistic but not a deterministic model
Age
Individual
Expected
Height
xy ˆˆˆ
line Regression
![Page 23: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/23.jpg)
23
A Project:Assessment as a Learning Activity
![Page 24: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/24.jpg)
24
Objective
Use assessment as a learning activity More formative assessment
Promote a variety of assessment activities Don’t rely too much on examination
Increase students’ interest in the learning activities
Enhance fairness in assessment
![Page 25: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/25.jpg)
25
Two Strategies
Strategy 1: Students participate in the design of assessment criteria
Strategy 2: Students conduct self- and peer-assessments based on the set of criteria Method introduced to ensure students to seriously
and critically conduct self- and peer-assessments
![Page 26: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/26.jpg)
26
Strategy 1: Assessment Criteria
Ask students to develop the set of criteria through a rigorous process All criteria are proposed by students
• Too time consuming
Provide students with a set of criteria for students’ deliberation and decision Easier and more efficient
![Page 27: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/27.jpg)
27
… Strategy 1: Assessment
Criteria Ownership of the assessment criteria
Students critically think about what constitute quality work
Students know in advance the set of criteria Pay attention to various aspects
• Some aspects may otherwise be ignored Writing skills in addition to content Presentation skills
o Use of IT, tables, graphso Time control
Enhance transparency and fairness on assessment activities
![Page 28: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/28.jpg)
28
Strategy 2: Self- and peer-assessment Self-assessment is graded based on
whether students can assess themselves properly
![Page 29: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/29.jpg)
29
… Strategy 2: Self- and peer-assessments Improve fairness in peer- & self-assessments Students must pay attention to peer’s work in
order to assess themselves properly More critical More serious Learn from peers
Issues to note Benchmark the self- and peer-assessment % of self-assessment contributes to total score More discussion in a paper*
*Poon, W.Y., McNaught, C., Lam P., & Kwan H.S. (2009). Improving assessment methods in university science education with negotiated self- and peer-assessment. Journal of Assessment in Education: Principles, Policy & Practice, 16:3, 331-346.
![Page 30: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/30.jpg)
30
Experiences in Science Used by many science teachers in
Group presentation Individual presentation Group project written report Individual written assignment Laboratory experiment report Poster presentation In-class activities in student-oriented courses
• Debate
• Role-play
• Portfolio of activities designed to explore a topic
![Page 31: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/31.jpg)
31
Evaluation Examples of students’ positive comments
“Quite interesting and innovative.” “Innovative, can allow students to identify and think about
what really matters.” “This can force students to pay attention to the
presentations of the others instead of daydreaming or chatting to each other.”
“It allows students to be more aware of their performance towards their peers, such that the presentation is more interesting and interactive.”
“It helps us to focus on the aspects other than the content itself, like the presentation skills and how the questions from audience are answered. Through this process, we could learn much more from each other.”
![Page 32: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/32.jpg)
32
…Evaluation
Examples of students’ suggestions “As an undergraduate student, it is a bit difficult for
us to assess other’s work.” “Students may not have enough knowledge to
comment other’s group topic.” “A weighting can be assigned to each marking
criterion to better reflect the importance of each criterion.”
“There may be strong disagreement among group members in deciding the scores … I think the assessment will be much fairer if it is carried out individually.”
![Page 33: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/33.jpg)
33
Teachers’ Views
In general well-received by teachers Also Students Easy to use Challenges in classes of large size
Applicable to a variety of activities Students paying more attention to peer’s works Can address the issue of free-riders in group
project Promote the use of group project
• as an effective assessment method
• as learning activity
![Page 34: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/34.jpg)
34
…Teachers’ Views
Most effective for assessing generic skills Some students still prefer teacher’s assessment
• Content knowledge or professional skills Ask students to peer- and self-assess generic skills but
not content or professional skills Can nurture the development of a wide range of (skill-
and value-domain) outcomes Self-managed learning, life-long learning, team-work,
active learning, critical thinking, honesty, integrity…
![Page 35: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/35.jpg)
Evaluation of Outcome at Programme Level
Student Experience Questionnaire (SEQ)
35
![Page 36: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/36.jpg)
Student Experience Questionnaire Administer to all undergraduate programmes @
CUHK 3-year cohort: 1st & final year 4-year cohort: 2nd & final year
Two parts, 16 constructs, 33 questions Part 1: Students’ Capability Development
7 constructs, 14 questions Part 2: Teaching & Learning Environment
9 constructs, 19 questions All questions are on 5-point scale
from (1) strongly disagree to (5) strongly agree
36
![Page 37: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/37.jpg)
… Student Experience Questionnaire The 7 constructs for “Capability Development”
Critical thinking Creative thinking Self-managed learning Adaptability Problem solving Communication skills Interpersonal skills and groupworks
37
![Page 38: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/38.jpg)
… Student Experience Questionnaire The 9 constructs for “Teaching and Learning
Environment” Active learning Teaching for understanding Feedback to assist learning Assessment Relationship between teachers and students Workload Relationship with other students Cooperative learning Coherence of curriculum
38
![Page 39: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/39.jpg)
Areas for Attention Capability
Communication skills Interpersonal skills and groupwork
Teaching & Learning Environment Active learning
• My teaching staff use a variety of teaching methods
• We are given the chance to participate in class Workload
TDG projects developed to improve in these areas
39
![Page 40: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/40.jpg)
… Science SEQ Results (2003 – 2012)
Improvement observed Communication skills Active learning
Many factors contribute to the improvement Many front-line teaching and learning
initiatives
40
![Page 41: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/41.jpg)
41
Recent Initiatives at the Science Faculty at CUHK
![Page 42: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/42.jpg)
42
Project Information: Book & Website Published “Recent
Initiatives in Teaching and Learning: Faculty of Science 2010” Overview a number
of T&L projects
![Page 43: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/43.jpg)
43
Website: http://www.cuhk.edu.hk/sci/ritl
![Page 44: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/44.jpg)
44
![Page 45: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/45.jpg)
45
More details for each project
![Page 46: Teaching Excellence Seminar School of Science University of Science and Technology The Design and Implementation of Outcome-Based Assessment in Science](https://reader031.vdocuments.us/reader031/viewer/2022032802/56649e025503460f94aecc62/html5/thumbnails/46.jpg)
46
Q & A
Thank you very much.