1

Chapter Six - The Consider C The Consider C is concerned with the reflective elements of learning and the ways in which learners can demonstrate how they have achieved their learning outcomes; in other words it is about the assessment elements of learning. Reflection and assessment are key drivers for learning and hence appropriate design of these elements is crucial. This chapter will begin by providing an overview of the key concepts of reflection and assessment and will then describe a number of ways in which these can be promoted.

Reflection Reflection is a metacognitive strategy to help learners to reflect on experiences actions and decisions, in other words the active application of experiences to gain new or better understanding. Reflection is more than just thinking, it s an intentional and intellectual activity. The importance of reflection was highlight by Dewey (1916). Dewey believed that education and learning are social and interactive processes. He argued that:

Reflection involves not simply a sequence of ideas, but a consequence—a consecutive ordering in such a way that each determines the next as its proper outcome, while each in turn leans back on its predecessors (Dewey 1933 pg 1).

He defined reflection as:

… an active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends (Dewey, 1933, pg. 6).

He describes five aspects of reflection: leaping to possible solutions, intellectualisation of a problematic experience into a problem to be solved, the use of hypotheses to guide further observation and information gathering , the elaboration of ideas in to a reasoned supposition and the experimentation and testing of hypotheses. Dewey articulates that there are two kinds of experiential processes that result in learning. The first is trial and error, which leads to a ‘rule of thumb’ decision. The second was, what he termed, ‘reflective activity’, which involves the perception of relationships and connections between the parts of the experience. He considered reflection on experience as a learning loop, continually oscillating between the experience and the relationship being inferred. Moon defines reflection as:

Reflection is a form of mental processing – like a form of thinking – that we use to fulfil a purpose or to achieve some anticipated outcome. It is applied to

2

relatively complicated or unstructured ideas for which there is not an obvious solution and is largely based on the further processing of knowledge and understanding and emotions that we already possess (Moon 1999).

Tate and Sills define reflection as:

We learn through critical reflection by putting ourselves into the experience & exploring personal & theoretical knowledge to understand it & view it in different ways (Tate and Sills 2004).

Reflection can be both informal and formal. Informal reflection involves self -questioning and develops our awareness of our own assumptions. Formal reflection draws on theory and practice, and provides guidelines for practice. Mezirow argues that we need to make an interpretation of an experience to make sense of it and in doing so we learn (Mezirow 1990). He goes on to argue that critical reflection is about critiquing the presuppositions on which our beliefs are built. In other words:

Learning may be defined as a process of making a new or revised interpretation of the meaning of an experience, which guides subsequent understanding, appreciation, and action (Mezirow, 1990, pg. 1).

Mezirow suggests there are three levels of reflection. The first is non-reflection, i.e. the absence of reflective thought. The second is lower level reflection, which involves the awareness of judgments, observations and descriptions, evaluations of planning, and assessment of decisions. The final level is critical reflection, which is the process of reflection and includes assessment of the need for further Sherwood et al. argue that reflection is a way of bridging the gap between theory and practice and that reflection enables tacit knowledge to be made explicit (Sherwood, Freshwater et al. 2005). They argue that reflection is a metacognitive strategy concerned with: self-monitoring, self-evaluating, and self-reinforcing goal-orientated behaviours. They suggests a number of ways of facilitating reflection (Sherwood, Freshwater et al. 2005). Firstly, through writing reflective diaries of blogs, to reflect on a learner’s experience and how this has transformed their learning. To this I would add peer critiquing of blog posts to foster dialogue between learners and the development of shared co-construction. Secondly, through the use of video and audio tapes. For example, learners might engage in a role-play activity, which is recorded. Afterwards the learners watch the video and discuss particular episodes to further their understanding of what happened. Thirdly, the use of the critical incident technique, which identifies particular helpful or unhelpful behaviours in a specific critical situation. Fourthly, discussion, for example in class or through a discussion forum. Learners might be given a topic to discuss or use the space to reflect on their learning to date. Fifthly, the use of visual technical to map out concepts and their relationships, for example through drawing or concept maps. Sixthly, through role play, for example in a face-to-face situation or in a virtual world. Role play allows learners to act out a particular event, problem, or situation. For example, learners might take different roles (such as nurses, doctors, administrators, patients, etc.) in a medical ward simulation, where a series of events occur that they have to respond to. Wills et al. have written a book which describes a range of role-play designs (Wills, Leight et al. 2010). Finally, through storytelling, which is a power discourse, which has a long history.

3

Boud et al. argue that it is important to provide opportunities for learners to build on their prior experience and to enable them to be actively engaged in what they are learning (Boud, Keogh et al. 1985). Schön defines reflection as ‘the capacity to reflect on action so as to engage in a process of continuous learning’ (Schön 1987). He introduced the concept of the reflective practitioner and argued that reflective practice is an important facet of professional practice, enabling individuals to reflect on their own practice and hence develop. It is about lifelong learning, whereby a practitioner analyses their experiences in order to learn from them. He states that reflective practice is ‘ a dialogue of thinking and doing through which I become more skillful’. He argues that ‘reflection in action’ concerns thinking about something whilst engaged in doing it, having a feeling about something & practicing according to that feeling. It is concerned with practicing critically. ‘Reflection-on-action’ on the other hand, occurs after the activity has taken place when a person thinks about what happened, judging how successful it was and whether any changes to what occurred could have resulted in different outcomes.

Assessment Assessment is one of the key drivers for learning. Boud and Falchikov (2007) argue that assessment directs attention to what is import and acts as an incentive to study. It affects what students study and how they study. The Quality Assurance Agency (QAA) defines assessment as:

… any processes that appraise an individual's knowledge, understanding, abilities or skills (QAA 2012).

The American Association for Higher Education defines it as:

Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education (Angelo 1995).

There are four types of assessment: diagnostic assessment (to ascertain a leaner’s current level of understanding of a topic), formative assessment by a tutor or by peers (to provide feedback on a learner’s work), and summative assessment (which contributes to the learner’s final grade. Assessments can be norm or criterion referenced. Norm referencing is where the students are placed in rank order and the grades are assigned by comparison to other students’ performance rather than upon the absolute quality of their performance. Criterion referencing is where achievements are judged in relation to objectives or criteria irrespective of other students’ performance.

4

Race et al. argue that:

Nothing that we do to, or for, our students is more important than our assessment of their work and the feedback we give them on it. The results of our assessment influence our students for the rest of their lives and careers – fine if we get it right, but unthinkable if we get it wrong (Race, Brown et al. 2005).

Spurlin suggests there are three facets to assessment: a focus on student learning, the collection, analysis and interpretation of information, and application for the purpose of improvement (Spurlin 2006). The Jisc publication ‘Effective assessment in a digital age’ states that:

Assessment lies at the heart of the learning experience: how learners are assessed shapes their understanding of the curriculum and determines their ability to progress. At the same time, assessment and feedback form a significant part of practitioners’ workloads and, with increased numbers, reduced budgets and higher learner expectations, continue to be a matter of concern for many institutions (Jisc 2010).

The report argues that technology can be used in a number of ways for assessment and it lists the following benefits:

Greater variety and authenticity in the design of assessments

Improved learner engagement, for example through interactive formative assessments with adaptive feedback

Choice in the timing and location of assessments Capture of wider skills and attributes not easily assessed by other means,

for example through simulations, e-portfolios and interactive games

Efficient submission, marking, moderation and data storage processes Consistent, accurate results with opportunities to combine human and

computer marking

Immediate feedback Increased opportunities for learners to act on feedback, for example by

reflection in e-portfolios Innovative approaches based around use of creative media and online peer and self-assessment

Accurate, timely and accessible evidence on the effectiveness of curriculum design and delivery

It is it is now understood that learning opportunities that provide enable learners to acquire skills of self-monitoring and self-regulation (for example by assessing their own work against defined criteria) prompt deeper and more effective learning. Boud (2007) lists the following principles for good assessment:

They must provide whatever information is necessary for judgements to be recorded of student performance

5

They must have a positive influence on what students do from prior to the act of assessment to their subsequent learning afterwards

They must be valued by students as worthwhile activities In terms of the learning activity the student engages with, he argues that:

The activity should be a learning experience The activity is seen as valid and worthwhile

It actively promotes learning and skills beyond the act itself

The student is an active agent throughout It has a strong positive backwash effect on learning during the course

It enables students to celebrate and portray achievements It is part of a sequence of great designs over the course as a whole It arises from a great learning environment It is not excessively resource-intensive

It requires and prompts informed judgement The Re-Engineering Assessment Practice (REAP) project1 provides a framework for discussing how assessment and feedback can have a beneficial impact on learning. Nicol et al. list the following good principles of assessment and feedback (Nicol and Macfarlane-Dick 2006; Nicol and Draper 2009). Good feedback processes should:

Help clarify what good performance is (goals, criteria, standards) Facilitate the development of self-assessment and reflection in learning

Deliver high quality information to students about their learning that helps them self-correct

Encourage teacher-student and peer dialogue around learning Encourage positive motivational beliefs and self-esteem

Provide opportunities to act on feedback Provide information to teachers that can be used to help shape their

teaching

Effective assessment tasks should: Capture sufficient study time in and out of class

Distribute student effort evenly across topics and weeks Engage students in productive learning activity

Communicate clear and high expectations to students Lombardi argues that Educators who strive to bring authentic learning experiences to their students must devise appropriate and meaningful measures to assess student learning and mastery of concepts at hand (Lombard 2008). She provides a comparison of traditional versus authentic assessment2 as outlined in Table 1.

1 http://www.reap.ac.uk/ 2 A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills from http://jfmueller.faculty.noctrl.edu/toolbox/whatisit.htm#definitions

6

Table 1: The difference between traditional and authentic assessment

Brown et al. (2005) argue that assessment should be: valid, reliable, transparent, authentic, should motivate students to learn, promote deep learning, fair, equitable, timely, incremental, demanding, enable the demonstration of excellence, efficient and manageable. They list the following reasons to assess:

To guide students’ improvement

To help students decide on which options to choose To help students to learn from their mistakes or difficulties To allow students to check out how well they are developing as learners To classify or grade students To allow students to make realistic decisions about whether they are up

to the demands of the course or module

To determine fitness for entry to a programme To give teachers feedback on their teaching

To enable students to do serious (deep) learning To translate intended learning outcomes into reality To add variety to students’ learning experience To help teacher structure their teaching and constructively align learning

outcomes to assessments To allow students to place benchmark their learning against that of their

peers To gather statistics on the course To lead to appropriate qualifications

Race3 lists the following mechanisms for assessing: unseen, time-constrained written exams, open book exams, open notes exams, structured exams, essays, reviews and annotated bibiligraphies, reports, practical work, portfolios,

3 Adapted from http://www.port.ac.uk/departments/services/dcqe/developingyouracademicpractice/downloads/filetodownload,177017,en.pdf

7

presentations, vivas, student projects, posters, dissertations and thesis, and work-based learning.

Kolb’s experiential learning model Kolb, built on the work of Dewey (Dewey 1933), Lewin (Lewin 1942) others. He argues that ‘learning is the process whereby knowledge is created through the transformation of experience’ (Kolb 1984 pg. 38). Kolb’s experiential4 learning model consists of four stages (Figure 1): the concrete experience stage, i.e. the ‘do’ aspect of learning, the reflective observation stage, the abstract conceptualisation stage, i.e. the ‘think’ aspect of learning, and the active experimentation stage, i.e. the plan aspect of learning. It is possible to begin anywhere in the cycle and learning occurs through iteration between the four stages. In the first stage, concrete experience, the learner actively experiences something, the second stage, reflective observation, is where the learner consciously reflects back on that experience, the third stage, abstract conceptualisation, is where the learner conceptualises a theory or model to explain what was observed. The fourth stage, active experimentation, is where the learner plans how to test their theory or model

Figure 1: Kolb's experiential learning cycle

Kolb argues that there are a number of key characteristics of experiential learning. Firstly, that learning is a process, it is not solely about outcomes. Secondly, learning is a continuous process, grounded in experience. Thirdly, the process of learning requires the resolution of conflicts between dialectically opposite modes of adaptation to the world. Fourthly, learning is a holistic process of adaptation to the world. Fifthly, learning involves transactions between the learner and their environment. Sixthly, learning is a process of creating knowledge.

Taylor’s reflective approach Taylor (2000) argues that there are eight stages to reflection:

4 To emphasis the importance of experience in learning

8

Identify the issues, problems or phenomenon for reflection, Decide on the reflective method and clarify its intent, Plan the stages in a research proposal,

Follow the method and use the process, Generate insights, Institute changes and improvements and continue to reflect on outcomes,

Report on outcomes, Use the outcomes in practice as evidence.

Technologies can be used in a variety of ways to support assessment. The JISC roadmap for e-assessment report states:

The variety of applications of e-assessment reported and their innovation and general effectiveness indicate the potential of e-assessment to significantly enhance the learning environment and the outcomes for students in a wide range of disciplines and applications (Jisc 2006).

The JISC/QCA provide the following definition: e-Assessment is the end-to-end electronic assessment processes where ICT is used for the presentation of assessment activity, and the recording of responses. This includes the end-to-end assessment process from the perspective of learners, tutors, learning establishments, awarding bodies and regulators, and the general public (Jisc 2007).

The following are important considerations when designing effective assessment practices: appropriateness, timeliness, relevance, accessibility, validity and the quality of supporting systems (Jisc 2007). E-assessment can provide more authentic learning experiences, through the use of e-portfolios, reflective diaries, blogs and role-play in virtual worlds. Multiple choice questions and computer-marked assessment can provide timely formative feedback to students. Students can provide each other with peer feedback and review, for example on blog post or in discussion forums.

Constructive alignment Biggs developed the concept of constructive alignment, which consists of two parts: learners construct meaning from what they do, and the teacher aligns the planned learning activities with the learning outcomes (Biggs 1999). The central premise is that the curriculum is designed so that the learning activities and assessment tasks are aligned with the learning outcomes that are intended in the course or module (Figure 2),

9

Figure 2: Biggs' constructive alignment

There are three aspects to this:

Define the learning outcomes.

Select learning and teaching activities likely to enable the students to

attain the outcomes.

Assess the students' outcomes and grade the students learning.

Biggs states that constructive alignment starts:

with the outcomes we intend students to learn, and align teaching and assessment to those outcomes. The outcome statements contain a learning activity, a verb, that students need to perform to best achieve the outcome, such as “apply expectancy-value theory of motivation”, or “explain the concept of … “. That verb says what the relevant learning activities are that the students need to undertake in order to attain the intended learning outcome (Biggs n.d.).

Learning outcomes should have three components: a measurable verb, the

condition under which the performance will occur and the criteria for

acceptable performance, i.e. how the learner demonstrates their learning.

Learning outcomes can be mapped to Bloom’s taxonomy (Bloom 1956),

which was later adapted by Anderson and Krathwohl (2001). Table 2 lists the

10

seven levels of Bloom’s taxonomy along with the appropriate verbs for each

level (corporation n.d.).5

Table 2: Verbs associated with Bloom's taxonomy

Level and meaning Verbs

Knowledge: the remembering of

previously learnt materials, i.e. the

recall of facts

Define, distinguish, identify,

inquire, label, list, match,

memorise, name, read, recall,

recognise, relate, repeat, record,

select

Comprehension: the ability to grasp

the meaning of the knowledge

being learnt.

Associated, describe, differentiate,

discuss, explain, extend, generalise,

give examples, illustrate, infer,

interpret, locate, rearrange,

reorder, restate, rewrite,

summarise, transform, translate

Application: the ability to use

learning materials in new ways

Apply, calculate, choose, classify,

demonstrate, develop, generalise,

illustrate, operate, organise,

practise, restructure, sketch, solve,

transfer, use

Analysis: the ability to break

material down into its parts so that

its organisational structure can be

understood

Analyse, categorise, classify,

compare, contrast, deduce,

describe, detect, diagram,

discriminate, differentiate,

distinguish, experiment, group,

inspect, point out, put into lists,

question, sub-divide, test

Synthesis: the ability to combine

previous experiences with new

material to form a whole new

Combine, compile, create, design,

generate, integrate, modify, plan,

produce, propose, solve

5 See also https://www.brookes.ac.uk/services/ocsld/resources/writing_learning_outcomes.html and http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf

11

structure

Evaluation: the ability to judge the

value of material for a given

purpose

Appraise, assess, choose, compare,

conclude, consider, criticise,

evaluate, judge, measure, rate,

score, select, support, validate,

value

The JISC-funded Cogent project produced a list of verbs that can be used in

learning outcomes, along with a description of each verb.6

The viewpoints principles The JISC-funded viewpoints project produced a set of principles for assessment and feedback, each written on a card.7 Each principle begins with an active verb such as clarify, encourage, provide, deliver to emphasise that an action must be carried out by someone in order to implement that principle. Each card comprises a guiding principle of good practice for assessment and feedback on one side (e.g. help clarify what good performance is (goals, criteria and standards)) and some examples of disciplinary applications of that principle on the other side (e.g. provide opportunities for discussion and reflection about criteria and standards before learners engage in a learning task). The cards are used in curriculum design workshops with academic staff and others involved in designing for learning (such as learning technologists and librarians). Evaluation of the workshops indicated that the resources helped practitioners rethink the ir design process and enabled them to think beyond content to the learner experience, in particular around the themes of ‘assessment and feedback’, ‘developing information skills’, and ‘learner engagement’ (Nicol 2012). The following quotes are illustrative of workshop participants’ views: ‘ideas that represented good practice’, ‘a useful starting point for a discussion about curriculum design’. ‘good use of building on existing research’ and ‘a validated set of pedagogical ideas to support curriculum design’. These were underpinned by the principle that ‘assessment and feedback sho uld help empower and engage learners and provide opportunities for feedback dialogue (peer-learner and/or teacher-learner)’. The first guideline is about clarifying good performance, in terms of determining to what extent do learners have opportunities to engage actively with goals, criteria and standards, before and after the assessment task. The second is concerned with encouraging time and effort on task, and in particular encouraging students to engage in regular study to promote deep learning. The third promotes delivery of high quality (and

6 http://www.pebblepad.co.uk/cogent/Vocab/List 7 http://wiki.ulster.ac.uk/download/attachments/29557521/Assessment&Feedback.pdf ?api=v2

12

learning. The fourth is about providing learners with opportunities to act on feedback, to enable them to close the gap on current and desired performance. The fifth is about encouraging interaction and dialogue with peers and the teacher. The sixth is concerned with developing self-assessment and reflection. The seventh is about giving learners assessment choices, in terms of topics, methods, criteria, weighting and the timing of the assessment. The eight is about encouraging positive motivational beliefs. The ninth is about ensuring that the assessment and feedback processes inform and shape teaching. For each of these there are a set of checklists for the designer to consider. There are also guidelines to help enable learners to develop their information skills8 and ways to foster learner engagement.9

Designing assessment activities The UK’s Quality Assurance Agency (QAA) suggests the following steps for designing assessment activities (QAA 2012):

Decide on the intended learning outcomes. What should the students be able to do on completion of the course, and what underpinning knowledge and understanding will they need in order to do it that they could not do when they started?

Devise the assessment task(s). If you have written precise learning outcomes this should be easy because the assessment should be whether or not they can satisfactorily demonstrate achievement of the outcomes.

Devise the learning activities necessary (including formative assessment tasks) to enable the students to satisfactorily undertake the assessment task(s).

Checklists and rubrics for assessment Table 4 lists a set of assessment checklist questions, to ensure the design of meaningful and measureable assessment of learning against learning outcomes. Table 3: An assessment checklist

Checklist question Are all the learning outcomes assessed? Do the learning outcomes have active verbs, which can be measured?

Are there any elements of diagnostic assessment? Are the assessment elements staggered at an appropriate pace across the course or module?

Is formative feedback provided?

Are there any elements of peer assessment? Have clear assessment criteria being developed mapped to grades?

Is an appropriate mix of assessment methods used? Is the assessment accessible and inclusive?

8 http://wiki.ulster.ac.uk/download/attachments/29557521/InformationSkills.pdf?api=v2 9 http://wiki.ulster.ac.uk/download/attachments/29557521/LearnerEngagement.pdf?api=v2

13

Are the assessment elements clearly described? Are the assessment elements at an appropriate level?

Is the time students spend on each assessment element appropriate? Is feedback clearly linked to the purpose of the assessment and the associated criteria?

Does the assessment process enable learners to demonstrate achievement of all the intended learning outcomes?

A rubric is an assessment tool that clearly indicates marking criteria. It can be used for marking assignments, class participation, or overall grades. There are two types of rubrics: holistic and analytical:

Holistic rubrics group several different assessment criteria and classify them together under grade headings.

Analytic rubrics, on the other hand, separate different assessment criteria and address them comprehensively. The top axis includes values that can be expressed either numerically or by letter grade. The side axis includes the assessment criteria.10

Designing rubrics consists of the following stages:

Decide what criteria or essential elements must be present in the student’s work to ensure that it is high in quality. At this stage, you might even consider selecting samples of exemplary student work that can be shown to students when setting assignments.

Decide how many levels of achievement you will include on the rubric. For each criterion or essential element of quality, develop a clear

description of performance at each achievement level.

Leave space for additional comments and a final grade. Table 4 shows an example of a generic assessment criteria rubric.11 For each grade there is a clear indication of the criterion that needs to be met, from complete attainment of all the learning outcomes to partial or little attainment. Table 4: Mapping grades to assessment criteria

Grade Criteria

A Clear attainment of all learning outcomes, showing complete and comprehensive understanding of the course content, with development of relevant skills and intellectual initiative to an extremely high standard.

B Substantial attainment of most learning outcomes, showing a high level of understanding of the course content, with development of relevant analytical and interpretative skills to a high level

C Sound attainment of some major learning outcomes, with understanding of most of the basic course content and development of

10 Adapted from https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/assessing-student-work/grading-and-feedback/rubrics-useful-assessment-tools 11 Adapted from http://www.gla.ac.uk/media/media_12158_en.pdf

14

relevant skills to a satisfactory level D Some attainment of a range of learning outcomes, showing a basic

understanding of course content with development of relevant skills More specialised rubrics can be developed, for example to evidence cooperative learning, assessment of PowerPoints, podcasts, web pages, e-portfolios, writing, games and simulations, research reports, oral presentations, use of social media, assessing wikis, blogs or the use of Twiter.12 The University of New South Wales (UNSW) has produced a set of guidelines on assessment rubrics.13 They list the following benefits of assessment rubrics:

They provide a framework that clarifies assessment requirements and standards of performance for different grades. In this, they support assessment as learning; students can see what is important and where to focus their learning efforts.

They enable very clear and consistent communication with students about assessment requirements and about how different levels of performance earn different grades. They allow assessors to give very specific feedback to students on their performance.

They encourage students to take responsibility for their performance When used for self-assessment and peer assessment, they can make

students aware of assessment processes and procedures, enhance their meta-cognitive awareness, and improve their capacity to assess their own work

They can result in richer feedback to students, giving them a clearer idea where they sit in terms of an ordered progression towards increased expertise in a learning domain.

By engaging staff teams in rubric-based conversations about quality, help them develop a shared language for talking about learning and assessment.

They can help assessors efficiently and reliably interpret and grade students' work.

They systematically illuminate gaps and weaknesses in students' understanding against particular criteria, helping teachers target areas to address.

The EdTechTeacher website14 lists a useful set of resources for assessing: blogs, wikis, websites/digital portfolios, technology and social media, video, screen casts and digital story telling, podcasts, graphic organizers and coding/gaming. There are a number of tools for creating assessment rubrics, such as the ‘Score Guides’ website, 15 rubistar,16 and rubric builder.17

12 Examples of each of these are available from http://www.uwstout.edu/soe/profdev/rubrics.cfm 13 Adapted from https://teaching.unsw.edu.au/assessment-rubrics 14 http://edtechteacher.org/assessment/ 15 http://digitales.us/evaluating-projects/scoring-guides 16 http://rubistar.4teachers.org/index.php 17 http://landmark-project.com/classweb/tools/rubric_builder.php3

15

Methods of assessment There are numerous methods of assessment, the most traditional being essays and examinations. Technologies can be used to assess students learning in a number of ways; in terms of diagnostic, formative and summative assessment. Diagnostic assessment is used to assess a student’s current level of understanding of a topic, for example assessing a student’s level of mathematical ability at the beginning of a course or their level of language skills. This can then be used to tailor the learning materials appropriately, which might include pointing the student to additional remedial materials. E-assessment, for example through multiple-choice questions is a common means of achieving this. Formative assessment is used to give students appropriate and timely feeding on their learning and understanding of a topic. Getting students to keep a reflective blog about their learning is a useful means of achieving this. Blogs can be used to encourage knowledge sharing, reflection and debate (EDUCAUSE n.d.). The teacher can provide comments on the blog posts, current misunderstandings and direct the student to additional useful materials about the topic. Alternatively, peers can provide feedback on blog posts. Discussion forums can also be used to provide feedback. This can be in the form of a frequently asked questions thread, or a discussion thread on a particular topic, where students are invited to contribute to the discussion; the teacher then responses to these comments. Summative assessment is used to grade students work and to assess to what extend they have achieved the course’s learning outcomes. An alternative to the traditional essay is the use of e-portfolios, where students gather evidence of their achievement of the learning outcomes. An additional benefit of e-portfolios is that they can enable students to reflect on their learning. For example, students might be asked to keep a reflective blog and then to select three key posts that evidence their understanding. To do this the student needs to read through all their posts and make informed choices for which posts they choose. They may also be required to describe why they have chosen those posts. Gray (2008) defines e-portfolios as follows:

An e-portfolio is the product, created by the learner, a collection of digital artefacts articulating experiences, achievements and learning.

She argues that the primary aim of an e-portfolio may be to collect evidence for summative assessment, to demonstrate achievement, record progress and set targets. She suggests that e-portfolios have a number of purposes:

Application – providing evidence in support of an application for a job or for admission to further study

Transition – providing a richer and more immediate picture of learners’ achievements and needs as they progress to a new environment, and supporting them through the process of transition

16

Learning, teaching and assessment – supporting the process of learning through reflection, discussion and formative assessment, and providing evidence for summative assessment

Personal development planning (PDP) and continuing professional development (CPD) – supporting and evidencing the pursuit and achievement of personal or professional competences

Lorenzo and Ittelson (2005) define e-portfolios as follows:

An e-portfolio is a digitized collection of artifeact, including demonstrations, resources, and accomplishments that represent an individual, group, community, organization or institution. This collection can be comprised of text-based, graphic, or multimedia elements archived on a Web site or on other electronic media such as a CD-ROM or DVD.

They argue that e-portfolios can help students become critical thinkers, as well as helping them to develop their writing and multimedia communication skills. E-portfolios can be created using a website, a blog or a specialised e-portfolio tools, such as PebblePad.18 In addition most Virtual Learning Environments (VLEs)/Learning Management Systems (LMSs) have e-portfolio tools. For example, MOODLE has Mahara.19 E-portfolios support lifelong learning and can be used by the student not only in their course, but more broadly, for example sharing the e-portfolio with potential employers. Classroom response systems or clickers are a useful way of eliciting student feedback in the classroom. They can be used in a variety of different ways, such as:20

The teacher poses a multiple-choice question to their students via an overhead or computer projector.

Each student submits an answer to the question using a handheld transmitter (a “clicker”) that beams a radio-frequency signal to a receiver attached to the teacher’s computer.

Software on the teacher’s computer collects the students’ answers and produces a bar chart showing how many students chose each of the answer choices.

The teacher makes “on the fly” instructional choices in response to the bar chart by, for example, leading students in a discussion of the merits of each answer choice or asking students to discuss the question in small groups .

Examples of clicker questions

Recall Questions, where students are to recall facts, concepts, or techniques relevant to class.

18 http://www.pebblepad.co.uk/ 19 https://mahara.org/ 20 Adapted from http://cft.vanderbilt.edu/guides-sub-pages/clickers/

17

Conceptual Understanding Questions, which go beyond recall and assess students’ understanding of important concepts.

Application Questions, which require students to apply their knowledge and understanding to particular situations and contexts.

Critical Thinking Questions, which require students to analyse the relationships among multiple concepts or make evaluations based on particular criteria.

Student Perspective Questions, which ask students to share their opinions, experiences, or demographic information.

Confidence Level Questions, which ask students a content question, then following that by asking students to rate their confidence in their answers.

Monitoring Questions, which are designed to provide instructors with information about how their students are approaching the learning process in their courses.

Classroom Experiments, which can be used to collect data from students for classroom experiments often used in the social sciences.

The freetech4teachers website21 compares nine e-assessment tools, these are: TodaysMeet, Padlet, Socrative, Infuse Learning, Kahoot, Plicjers, 81 Dash, Geddit and Answer Garden.

Conclusion This chapter has provided an overview of the Consider C, namely ways in which learners can be encourages to be reflective and to demonstrate their achievement of learning outcomes. Key research on reflective learning and assessment are described, along with practical examples of creating assessment activities, which are constructively aligned with the intended learning outcomes.

References Anderson, L. W. and D. R. Krathwohl (2001). A taxomony for learning, teaching

and assessment: a revision of Bloom's taxonomy of educational objectives. New York, Longman.

Angelo, T. (1995). Reassessing (and redefining) assessment, AAHE bulletin, 48 (3), 7-9.

Biggs, J. (1999). "What the student does: teaching for enhanced learning." Higher Education Research & Development 18(1): 57–75.

Biggs, J. (n.d.). "Constructive alignment." John Biggs http://www.johnbiggs.com.au/academic/constructive-alignment/.

Bloom, B. S. (1956). Taxonomy of educational objectives, the classification of educational goals: handbook 1 - cognitive domain. New York, McKay.

Boud, D. (2007). Great designs: what should assessment do? REAP international online coference on assessment design for learner responsbility, Glasgow, University of Strathclyde.

21 http://www.freetech4teachers.com/2014/11/nine-popul ar-student-response-tools.html#.VHdOG2SsXC7

18

Boud, D. and N. Falchikov (2007). Rethinking assessment in Higher Education: Learning for the longer term. Abingdon, Routledge.

Boud, D., R. Keogh, et al. (1985). Reflection: turning experience into learning. New York, RoutledgeFalmer.

Brown, S., P. Race, et al. (2005). 500 tips on assessment, 2nd. Edition. London, RoutledgeFalmer.

corporation, T. l. m. (n.d.). "Developong outcomes and objectives." from http://www.thelearningmanager.com/pubdownloads/developing_clear_learning_outcomes_and_objectives.pdf.

Dewey, J. (1916). Experience and Nature. New York, Dover. Dewey, J. (1933). How we think. New York, D.C. Heath and co. EDUCAUSE (n.d.). 7 things you should know about blogs. Gray, L. (2008). Effective practice with eportfolios. Bristol, JISC. Jisc (2006). Jisc roadmap for e-assessment report. Bristol, jisc. Jisc (2007). Effective practice with e-assessment. Bristol, Jisc. Jisc (2010). Effective assessment in a digital age. Bristol, Jisc. Kolb, D. (1984). Experiential learning: experience as the source of learning and

development. Englewood Clifffs, NJ, Prentice Hall. Lewin, K. (1942). Field theory and learning. Field theory in social sciences:

selected theretical papers. D. Cartwright. London, Social Sciences Paperbacks.

Lombard, M. M. (2008). Making the grade: the role of assessment in authentic learning, ELI series. D. Oblinger, EDUCAUSE.

Lorenzo, G. and J. Ittelson (2005). An overview of e-portfolios. Educause Learning Initiative (ELI) Series. D. Oblinger, EDUCAUSE.

Mezirow, J. (1990). How critical reflection triggers transformative learning. Fostering critical reflection in adulthood. San Fransico, Jossey Bass: 1 - 20.

Moon, J. (1999). Reflection in learning and professional development. London, RoutledgeFalmer.

Nicol, D. (2012). Transformational change in teaching and learning: recasting the educational discourse. Evaluation of the viewpoints project at the University of Ulser. Glasgow, University of Strathclyde.

Nicol, D. and S. Draper (2009). A blueprint for transformational organisational change in higher education: REAP as a case study. Educational through technology-enhanced learning. T. Mayes, D. Morrison, H. Mellar, P. Bullen and M. Oliver. York, Higher Education Academy.

Nicol, D. J. and D. Macfarlane-Dick (2006). "Formative assessment and self-regulated learning: A model and seven principles of good feedback practice." Studies in Higher Education 31(2): 199–218.

QAA (2012). Understanding assessment: its role in safeguarding academic standards and quality in higer education: a guide for early career staff. Gloucester, Quality Assurance Agency (QAA).

Race, P., S. Brown, et al. (2005). 500 tips on assessment, 2nd Edition. London, Routledge.

Schön, D. A. (1987). Educating the reflective practioner: towards a new design for teaching and learning within the professions. San Francisco, Joey-Bass Inc.

19

Sherwood, G., D. Freshwater, et al. (2005). The scholarship of reflective practice, position paper, Practice Task Force at University of North Carolina at Chapel Hill School of Nursing. Chapel Hill, University of North Carolina.

Spurlin, J. E. (2006). Technology and learning: defining what you want to assess. ELI Paper 1. D. Oblinger, EDUCAUSE.

Tate, S. and M. Sills (2004). The development of critical reflection in the health professionals London, Higher Education Authority.

Taylor, B. (2000). Reflective practice: a guide for nurses and midvices. Melbourne, Allen and Unwin, Open University Press.

Wills, S., E. Leight, et al. (2010). The power of role-based e-learning. London, Routledge.