Learning Goals WorkshopAko Aotearoa Project

July 2010

Based upon the CWSEI Learning Goals Workshop

After this workshop you should be able to …

• Explain the goal limitation set by the course.

• Write goals that target content, skills, or attitudes;

• Use Bloom’s taxonomy to write topic-level learning goals that address all six levels of learning;

• Effectively link assessments to learning goals.

Pre-workshop assignment: the purpose was to provide an opportunity for you to come already thinking about these issues.

Workshop plan of action

1. Introduction 2. Goal limitations from course background3. Course level goals4. Topic level learning goals & Bloom’s taxonomy5. Assessment: building questions that inform both

students and instructors.

Practice and collaborate in working groups

No substitute for discussion with peers &

iteration.

General & small group discussion, and

revisions.

1. Introduction

Definition:– Articulate what students should be able to do after

a course, a module, a lesson, etc. – Reflects what you want students to learn and at what level.

Values of carefully designed goals:– Instructor: to better define, guide, & check your teaching. – Students: Define what they should be learning, and why.– Both: Essential for measuring what students are learning;

Goals help build assessments which provide feedback to students and instructor.

1. Introduction

Wide range of possible goals: – memorizing terminology – complex problem solving skills– transferring ideas to new contexts– thinking like a professional / scientist– many others …..

Goals are NEVER perfect but ALWAYS valuable

Goals at any level should emphasize students

• Teaching is commonly viewed as teacher-centered.

• But … students must do their own learning; we can not do the learning for our students.

• Good teaching involves helping students actively engage with new ideas, skills, and behaviours. Simply presenting new content is NOT “teaching”.

• “We will design more effective courses if we focus on setting goals for what students do, rather than for the faculty member.”

From http://serc.carleton.edu/NAGTWorkshops/coursedesign/tutorial/goals.html

Learning Goals—it’s hard to learn when you don’t know what to pay attention to

http://viscog.beckman.illinois.edu/grafs/demos/15.html

Goal limitations Example: eosc221• Introduction to mineralogy• Context is important for steering goals

– 2nd year students, mostly pursuing a geology degree– Pre-requisite for many subsequent courses

• Well defined requirements for learning

– Student background • Minimal experience in geology • 1st year chemistry

• Assessment- weekly labs, mid term lab exam, final lab exam, final written exam- short answers. Also Clickers, in class exercises

Compare homework- Discuss in your groups in turn-1. How your course context (Homework Q 1.) might influence your learning goals

Activity 1: Goal limitations

Time 5 mins per course Total 10 mins

Workshop plan of action

1. Introduction 2. Goal limitations from course background3. Course level goals4. Topic level learning goals & Bloom’s taxonomy5. Assessment: building questions that inform both

students and instructors.

3. Course-Level learning goals

Characteristics:• Minimal technical terminology (unlike topic level goals)• Should be informed by …

– Pre- and post-requisites– Department goals and resources– Discipline context & constraints (professions etc)

• Should articulate …– Attitudes / beliefs you hope students will gain– Concept areas and skill sets students should acquire– Or, identify what students will be able to do with the new concepts and

skills. Specifics (details) may be implied.

To help focus goals design, 3 domains of learning

CONCEPTS: Eg: analyze, explain, and predict the motion of objects in the world around you.

SKILLS: Cognitive/Process skills: - Reasoning, Problem Solving, Evaluating, Critiquing… - Compare / contrast; reflect; strategize, justify, plan …

Technical skills: - Computer programming, specific software, … - Laboratory skills, wetlab, instruments, tools, … - Library, research, writing, presenting, …

ATTITUDES: Appreciate, Enjoy, Value - e.g. Observe our planet from a more enthusiastic and

scientifically informed perspective.

About nature of learning and doing: - e.g. Believe that learning and doing physics is more about

reasoning and making sense, not memorizing.

(There are many other “models” for learning domains.)

Learning domains

CONCEPTS SKILLSATTITUDES

Eg: Goals for a 1st yr physics course include all 3 aspects of learning:

Lecture-level learning goals Be able to design a fluorescent light bulb.

Explain and justify the requirements on specifications of various basic electronic components.

One Lecture-scale Learning Goal could touch upon 3 course-level goals

Course-level learning goalsRecognize that the behavior of the world around you is not magical and mysterious, but rather can be understood and predicted using certain fundamental principles.

Use knowledge about properties and motion of electric charges to predict and explain various aspects of electricity.

Activity 2: course-level goals

• In your groups- discuss course level goals» Use the characteristic in slide 11 to complement and improve each

others goals.

• Which learning domains are targeted?

• Which goals do your prefer -Why?

Time 5 mins to look over 5 mins to discuss together

Recap: Some aspects of course-level goalsCharacteristics:• Minimal technical terminology (unlike topic level goals).• Should be informed by …

– Pre- and post-requisites– Department goals and resources– Discipline context & constraints (professions etc)

• Articulate …– Attitudes / beliefs you hope students will gain.– Concept areas and skill sets students should acquire

Benefits:• Help structure the course & guide methods of instruction.• Study guide – help students structure their time & effort.• Help instructor’s identify assumptions about prior learning.• Enhances transfer between instructors.

(Instructors retain flexibility at the topic level)

Course context: department, discipline, program

Course-level or topic-level?

Course goals

Topic goals

Pedagogy Active learning Assessment

What can students do?

GeneralOverview

SpecificOutcomes oriented

Bloom's Taxonomy of Learning

1. Factual Knowledge: remember and recall factual information

2. Comprehension: demonstrate understanding of ideas,

concepts

3. Application: apply comprehension to unfamiliar situations

4. Analysis: break down concepts into parts

5. Synthesis: transform, combine ideas to create something

new

6. Evaluation: think critically about and defend a position

4. Topic-level goals

See the Lord and Baviskar paper, and handout #2.

1. Factual Knowledge: remember and recall factual information

2. Comprehension: demonstrate understanding of ideas,

concepts

3. Application: apply comprehension to unfamiliar situations

4. Analysis: break down concepts into parts

5. Synthesis: transform, combine ideas to create something

new

6. Evaluation: think critically about and defend a position

Define, List, State, Label, Name, Describe

Describe, Explain, Summarize, Interpret, Illustrate

Apply, Demonstrate, Use, Compute, Solve, Predict, Construct, Modify

Compare, Contrast, Categorize, Distinguish, Identify, Infer

Develop, Create, Propose, Formulate, Design, Invent

Judge, Appraise, Recommend, Justify, Defend, Criticize, Evaluate Higher level: Require deeper conceptual

understanding

Bloom's Taxonomy of Learning – with verbs

4. Topic-level goals

Practice thinking about Bloom’s taxonomyExamples of topic-level goals:

What Bloom’s level are these? (from eosc221)

1. Describe the rock granite2. Compare and contrast granite with basalt3. Design a methodology that you could use to distinguish between

granite and basalt.4. Criticize your neighbors methodology for distinguishing between

granite and basalt

Are any “levels” better than others? “high level”

“low level”

Developing topic level goals

Handout #2: Useful Tips for writing effective learning goals.

• Topic (or module) level goals define what students can do after learning.

• Each should be consistent with at least one course-level goal.

• Keep in mind learning “domains”:– CONCEPTS – SKILLS– ATTITUDES

Topic-level learning goals (2-3 per class period)

Example:

• Understand how a fluorescent light bulb works. Or …..

• Be able to design a fluorescent light bulb and to explain and justify the requirements on the various basic components.

Rewording- Biology Learning Goal Examples

Course-level learning goal

Topic learning goal

Concepts: Demonstrate how meiosis leads to diversity in the next generation

Skills: Become better problem solvers

Original wording: Understand the rules for inheritance of chromosomes in the process of meiosis.

Human Genetics for non-majors

Predict the probability of generating sperm and egg cells with specific chromosomal makeup. (Blooms level 3).Explain how these cells are produced (Blooms level 2).

Reworded:

Low level goal—explicitly encourages students to memorize the steps from DNA to protein

Describe how the process of extracting information from genetic material is regulated at each step of conversion of DNA to RNA to protein.

CharacteristicsOriginal goal

Increasing Blooms level Introduction to Molecular and Cell Biology

New goal In comparison

Propose two different ways that an abnormal protein could be made in a cell, resulting in disease symptoms.

Higher level goal—encourages student to think about how proteins work, how they are produced, and how they can be altered by mutations in DNA.

Check-list for creating topic-level learning goals (Handout #3) 1. Is goal expressed in terms of what the student will achieve? Does it identify

what students will be able to do after the topic is covered?2. Is the Bloom’s level of the goal aligned with your expectations for students’

learning … Is this what students will be able to do if they “understand” the topic at the level you want? – If you expect reasoning for “why”, does it convey that? – Could you expect a higher level goal?

3. Is it well-defined? Is it clear how you would test achievement? 4. Do chosen verbs have a clear meaning? 5. Is terminology familiar/common? If not, is the terminology a goal?

Not every goal can achieve the following, but it is better if you can: 6. Is it relevant and useful to students? (e.g. connected to their everyday life,

or does it represent a useful application of the ideas).

Topic-level learning goals (2-3 per class period)

Activity 3 – part 1:Your learning goals

• Working with a partner or in a group: Exchange topic-level goals

• Use the checklist to “critique” these goals

• Discuss.Time 5 mins to look over 5 mins to discuss together

Activity 3 – part 2: Refine your learning goal …

• Suggest changes to improve clarity (address checklist items)

• All to discuss one or two of these

Time 5 mins to refine the goal5 mins to discuss together

Break

Now: Testing achievement of learning goals; i.e., assessing progress

Handout #4: Formative Assessment QuestionsWhy:• Formative assessment = closing the feedback loop.

i.e. Test / quiz/ work / etc informs instructor & student about knowledge gaps.

• Know what students think now (before exam), and react to it !

When ….• Pre-test Post-test• Active learning = formative feedback• Evaluations (marked work).

How …. Today’s focus is how to ask question which effectively assess specific learning goals.

1. Factual Knowledge: remember and recall factual information

2. Comprehension: demonstrate understanding of ideas,

concepts

3. Application: apply comprehension to unfamiliar situations

4. Analysis: break down concepts into parts

5. Synthesis: transform, combine ideas to create something

new

6. Evaluation: think critically about and defend a position

Define, List, State, Label, Name, Describe

Describe, Explain, Summarize, Interpret, Illustrate

Apply, Demonstrate, Use, Compute, Solve, Predict, Construct, Modify

Compare, Contrast, Categorize, Distinguish, Identify, Infer

Develop, Create, Propose, Formulate, Design, Invent

Judge, Appraise, Recommend, Justify, Defend, Criticize, Evaluate Higher level: Require deeper conceptual

understanding

Help with writing questionsBloom's Taxonomy of Learning – with verbs

The Montillation of Traxolinewhen assessment goes astray

It is very important that you learn about traxoline. Traxoline is a new

form of zionter. It is montilled in Ceristanna. The Ceristannians gristerlate large quantities of fevon and then bracter it to quasel traxoline. Traxoline may well be one of our most lukized snezlaus in the future, because of our zionter lescelidge.- - - -

Answer these questions in complete sentences. Be sure to use your best handwriting.

1. What is traxoline?2. Where is traxoline montilled?3. How is traxoline quaselled?4. Why is it important to know about traxoline?

[attributed to Judy Lanier]

Are questions

testing anything?

Activity 4: Back to the goals and assesments related to the topic goals you brought in …• Exchange goals and assessments.• Is the Blooms level appropriate, can you raise it ?• Keep in mind:

Student perspective, the learning “domains” are you testing concepts, skills, attitude.

• Avoid rote memorization strategies. Try:– “Troubleshooting” (something changes, what could cause this?)– “redesign …”– “compare and contrast …” scenarios

Implementing assessments (questions) Not just exams …• In class is important – why?

– active learning– Feedback for students AND instructors

• What are some options? (All can be solo / pairs / group work.)– Clickers– Coloured cards to raise (multiple choice questions)– 1 minute papers– Group (team) work with “instant” deliverables to enable compare & discuss– Pre-test quizzes (solo and/or groups)– Pre-test + Post-test to foster curiosity, then wrestle with using new concepts

This is the goal of this semester to develop these assessments

Other ideas for implementation:

• Ask question in exam or homework.

• Ask students to write answer to a question on a sheet of paper and turn it in.

• Ask students to write answer on own, then discuss with others, then write a revised answer.

• Compare question responses before / after a learning activity.

• Ask students to discuss question in groups, write a group answer.

• Ask a multiple choice question and ask students to vote using their hands, colored cards, or clickers.

• Students can answer on their own or discuss in groups.

• Give credit for answering questions but don’t grade.

Research Assistant suggestions1. To use UBC goals and your own suggestions to refine goals and improve their use in the classroom

2. To use UBC assessments and your own suggestions to develop: (1)Formative in class clicker questions- (2)Formative in class 5 min activities (3)A multiple choice test for the end of the semester

Research assissant things to ask

• Are goals presented and clear in the class ?• Can goals be improved upon- higher blooms

level ? More diverse learning domains ?• Is it clear how these goals relate to course

goals ?• Are any of the UBC goals appropriate ?• Are any of the UBC assessments appropriate ?

Recap 1 -

Values of explicit learning goals:

• Students: Define what they should be learning, and why.

• Instructor: to better define, guide, & check your teaching.

• Both: Essential for measuring what students are learning; i.e. goals help build assessments which provide feedback to students and instructor.

Recap 2 -

Building linked course level and topic level goals

• Student (NOT teacher) point of view.

• Cognitive domains (concepts, skills, attitudes).

• Use of Bloom’s taxonomy as a framework.

Assessment is the key – target appropriate levels of learning.• Implementation:

– Formative feedback is key to informing students (and instructors) about the degree of success at meeting goals of the course.

Thank you for your interest

Homework

• Refine your learning goals (course-level and topic-level) developed in the pre-workshop assignment.

• Look up “learning goals xxx” (xxx = your discipline).

A wide range of resources are available from many reputable institutions.

Workshop evaluations: Were these learning goals useful? Achieved?

• Distinguish between course and topic goals.

• Convert a syllabus (list of topics) into learning goals that reflect what students should be able to do;

• Write goals that target content, skills, or attitudes;

• Use Bloom’s taxonomy to write topic-level learning goals that address all six levels of learning;

• Effectively link assessments to learning goals.