“Guidelines for the Development of Learning

Outcomes”

Presentation within the frame of the project

INARM

Veronika Nitsche and Michaela Handke (WUS Austria)

www.wus-austria.org

Learning Outcomes

... are a statement of what a learner is expected to know, understand, or able to do at the end of a learning process.

From the definition of Learning Outcomes we see…

... an emphasis on the learner and an emphasis on the learner’s ability to do something.

Focus on teaching – aims and objectives and use of terms like know, understand, be familiar with.

Focus learning - outcomes: what we want the student to be able to do - use of terms like define, list, name, recall, analyse, calculate, design, etc.

Where LO can be found

Learning Outcomes

Occupational

standards

Assessment criteria

Qualifications

Qualifications framework

Curricula

Where LO can be found… …. on the example of curriculum development at the University of Graz/Austria (I)

• Structure: Bachelor (180 ECTS, 240 ECTS) - >specific occupational field

Master (min. 120 ECTS) -> oriented towards occupation and research Bachelor, Master, PhD should be clearly differentiated -> EUA recommends to use the Dublin

Descriptors

• Change of paradigm: from teaching to learning before: top-down on the basis of the research in

the field now: bottom-up model on the basis of the

potential occupational fields -> Definition of a qualification profile (now first

step in curriculum development)

Curriculum development at the University of Graz/Austria (II)

Qualification profiles:

• Define the qualifications a graduate should have at the end of the studies (professional and social/generic competences);

• This qualification profil is the basis for the definition of the learning outcomes of the study programme and its modules.

Curriculum development at the University of Graz/Austria (III)

Learning outcomes: are defined on the basis of the qualification profile

> Basis for the definition of the type of courses, allocation of ECTS, examination, etc.

Qualification Profiles (IV)

Frameworks for the development of Qualification Profiles:

• European Qualifications Framework

• National Qualifications Framework

• Dublin Descriptors

• Standards by Law: in Austria 2002 University Studies Act (UG 2002)

Writing Learning Outcomes

“Writing Learning Outcomes is a

Process not an Event” (Bloom)

Bloom’s Classification of Cognitive Skills

Category

Definition

Related Behaviors

Knowledge

recalling or remembering something without

necessarily understanding, using, or changing it

define, describe, identify, label, list, match, memorize,

point to, recall, select, state

Comprehension understanding something that has been

communicated without necessarily

alter, account for, annotate, calculate, change, convert,

group, explain, generalize, give examples, infer, interpret,

paraphrase, predict, review, summarize,

Application

using a general concept to solve problems in a

particular situation; using learned material in new

and concrete situations

apply, adopt, collect, construct, demonstrate, discover,

illustrate, interview, make use of, manipulate, relate,

show, solve, use

Analysis

breaking something down into its parts; may focus

on identification of parts or analysis of relationships

between parts, or recognition of organizational

principles

analyze, compare, contrast, diagram, differentiate, dissect,

distinguish, identify, illustrate, infer, outline, point out,

select, separate, sort, subdivide

Synthesis creating something new by putting parts of different

ideas together to make a whole.

blend, build, change, combine, compile, compose,

conceive, create, design, formulate, generate,

hypothesize, plan, predict, produce, reorder, revise, tell,

write

Evaluation

judging the value of material or methods as they

might be applied in a particular situation; judging

with the use of definite criteria

accept, appraise, assess, arbitrate, award, choose,

conclude, criticize, defend, evaluate, grade, judge,

prioritize, recommend, referee, reject, select, support

Some examples ... At the end of the course students will be able to: - Identify and consider ethical implications of scientific

investigations. - Differentiate between civil and criminal law - Select and employ sophisticated techniques for analysing

the efficiencies of energy usage in complex industrial processes.

- Compare and contrast the different electronic business models

- Propose solutions to complex energy management problems both verbally and in writing

- Predict the effect of change of temperature on the position of equilibrium.

Practical advice for writing

learning outcomes (I) • Begin each learning outcome with an action verb.

• Use only one verb per learning outcome.

• Avoid vague terms like know, understand, learn, be familiar with, be exposed to, be acquainted with, and be aware of. These terms are associated with teaching objectives rather than learning outcomes.

• Avoid complicated sentences. If necessary use more one than one sentence to ensure clarity.

• Ensure that the learning outcomes of the module relate to the overall outcomes of the program.

• General recommendation: 5 – 8 learning outcomes per module.

• The learning outcomes must be observable and measurable.

• Ensure that the learning outcomes are capable of being assessed.

Practical advice for writing

learning outcomes (II) • Timescale: ask yourself if it is realistic to achieve the LO within

the time and resources available.

• Assessment: How will you know if the student has achieved these learning outcomes? Very broad LO may be difficult to assess effectively. With very narrow LO, the list of learning outcomes may be too long and detailed.

• Before finalizing the learning outcomes, ask your colleagues and possibly (former) students if the LO make sense to them.

• When writing LO for students at levels beyond first year, try to avoid overloading the list with learning outcomes which are drawn from the bottom of Bloom’s taxonomy.

• It is vital that learning outcomes are clearly written so that they are understood by students, colleagues and external examiners.

ASSESSMENT

-> STEPS involved in linking Learning Outcomes, Teaching and Learning Activities

and Assessment

Clearly define the learning outcomes.

Select teaching and learning methods that are likely to ensure that the learning outcomes are achieved.

Choose a technique or techniques to assess the achievement of the learning outcomes.

Assess the learning outcomes and check to see how well they match with what was intended

Linking Learning Outcomes and Assessment Criteria

Learning outcomes imply the assessment criteria!

Assessment criteria may be developed from the learning outcome or from the assessment task – but in either case they should relate to the learning outcome

Assessment criteria should test, assess or relate to the learning that is mentioned in the learning outcome. e.g if you say students must „write something‟, you should not test them orally.

Benefits and potential problems of Learning Outcomes?

The benefits of Learning Outcomes

Help to explain more clearly to students what is expected of them and thus help to guide them in their studies.

Help teachers to focus more clearly on what exactly they want students to achieve in terms of knowledge and skills.

Help teachers to define the assessment criteria more effectively.

Help to provide guidance to employers about the knowledge and understanding possessed by graduates of programmes.

Help to start discussion on teaching and learning in third level institutions.

Potential problems with Learning Outcomes

Could limit learning if learning outcomes written within a very narrow framework – lack of intellectual challenge to learners.

Learning outcomes should not be reductionist but rather expansive and intended to promote the higher order thinking skills.

Danger of assessment-driven curriculum if learning outcomes too confined.

Could give rise to confusion among students and staff if guidelines not adhered to when drawing up learning outcomes, etc.

Learning Outcomes (summary)

Discussions at European and national level highlight the need for common ground with respect to LO – this does not imply that their need to be a common approach to defining and using LO.

Further reading

Jenny Moon, Exeter University: Linking Levels, Learning Outcomes and Assessment Criteria.

http://www.aic.lv/bolona/Bologna/Bol_semin/Edinburgh/J_Moon_backgrP.pdf

Kennedy, D. et.al.: Writing and Using Learning Outcomes: a Practical Guide. http://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdf

John Biggs: ‘Aligning teaching and assessing to course objectives’. Teaching and Learning in Higher Education: New Trends and Innovations. University of Aveiro, 2003

Using Learning Outcomes. European Qualifications Framework Series: Note 4 (2011)

Further reading

http://www.eua.be/eua-work-and-policy-area/building-the european-higher-education-area/bologna-basics/Bologna-an-overview-of-the-main-elements.aspx

Shared descriptors for Short Cycle, First Cycle, Second Cycle and Third Cycle Awards -> Known as the 'Dublin Descriptors' after the meeting in which they were agreed, in Dublin, March 2004

WP 4

Learning outcomes for MA and BA

4.1 Intended learning outcomes for MA and BA

4.2 Peer-review by EU partners

4.3 Methodology of developing LOs

4.4 Adapted versions of the LOs for MA and BA

Next steps in WP 4?

4.1 Intended learning outcomes for MA and BA: The next step of

the project would be to design and develop intended LOs drawing on the SQFs for informatics and management.

(1) Develop a general set of “competences” (related to occupation) in the field of IT and Management

Timeframe:

- Dev. of set of competences by PCU/WG: submit it until 15/12/2013 (lists of profiles and functions, and related learning outcomes)

- Discussion and decision on the set of competences (1 for Mgmt., 1 IT): -> at the Meeting in Kosice (Feb. 12-15, 2014)

(2) based on the general set of competences (and the methodology developed under 4.3?) to develop a set of learning outcomes -> Meeting in Kiev, Lviv (April 2014)

Next steps in WP 4?

4.2 Peer-review by EU partners (not only of the LO, but also on

the SQF/WP 3)

4.3 Methodology of developing LOs: To ensure wider application of the methodology, guidelines on the LO development will be developed. ->

WHO-> WG (tbc): Jolanta Rodzos, Veronika Nitsche, Svetlana Khapaeva,

Ruben Topchan, Tetyana Daragan, Prof. Telizhenko Alexander (UKR) HOW: presentation and comparison of different approaches and

taxonomies, development of (a) methodologies WHEN -> at the meeting in Kiev, Lviv (April 2014) – tbc. 4.4 Adapted versions of the LOs for MA and BA: PC universities will

adapt the LO to the local needs. The adapted versions will be circulated among the stakeholders for feedback, the LOs will be refined and prepared for publications.

Thank you for your attention!