Workshop: Formative Assessment

Institute for Excellence in Education Summer Teaching Camp 2013

Edith Gurewitsch Allen, M.D.Mona Mohamed, M.D., Ph.D.

Julianna Jung, M.D.

07/25/2013 Presenters have no relevant disclosures or conflicts of interest

Objectives differentiate formative from summative assessment

demonstrate techniques for formative assessment of learners

develop and solicit formative assessment of:communication skills information literacystrategic planning/critical reasoningattitudes and disposition [professionalism]

X Formative assessment for course or program refinement

What is Assessment?Establishing clear, measurable expected learning

outcomesEnsuring sufficient opportunities to achieve

those outcomesSystematic gathering, analysis and interpretation

of evidence to determine extent to which (how well) learning matched expectations

Using the resulting information to understand and improve learning

General Purpose of Assessment

• Defines what learners will regard as important – Brown 2001– high value placed on marks and grades; – ultimately students focus their efforts on assignments and what

they will be tested on

• Ensures that what is important is learned

ASSESSMENTS MUST ALIGN WITH EDUCATIONAL GOALS

Planning Our TeachingTraditional Paradigm

• What topics should we teach?

• How do we test that the topics were learned?

• What are the criteria for determining sufficiency of learning (what earns an A, B, C, etc)?

New Paradigm

• What do students need to know? – critical understandings,

theories, models; knowledge, skills and attitudes

• How will we assess their learning?

• How do we structure the educational experience to ensure that they learn?

Teacher-Centered Learner-Centered

Characteristics of AssessmentSummative Assessment

• Subjective > Objective

• Personal, Reflective

• Interactive, Social

• Midstream, Guiding

• Prospective: are goals being met?

Formative Assessment

• Objective > Subjective

• Comparative, Relational

• Solitary

• Conclusive, “Add-on”/Post Hoc

• Retrospective: were goals met?

Students Learn Most Effectively When They:

Evidence-based Strategies to Promote Lasting Learning

Understand goals and characteristics of excellent work Relate new learning to prior experiences• Spend significant time studying and practicing, using and

applying new knowledge in some way• Diverse learning styles are respected• Engage in multi-dimensional, real-world tasks, and interact

with others Assessments are learning activities in their own right Reflect on what and how they learned, seeing coherence in

their learning

Universally Valued Learning Goals

• Communication Skills• Information Literacy• Strategic Planning/Problem Solving• Professional Attitudes/Dispositions

In-Class Techniques

Assessing Communication Skills

Drawing Exercise

Communication Skills

ENCODING

• Understand the objective: why are you communicating?

• Understand the audience: who are they? What do they need to know?

• Anticipate reactions or potential sources for confusion; plan the message

• Seek feedback to confirm understanding

DECODING

• Pay attention• Receive information

without first anticipating the response to be given back

• Ask questions to confirm or revise understanding

In-Class Techniques

Information Literacy

“Headbandz”

Information Literacy

• Recognize the need for information• Identify what information is needed• Find the information• Evaluate information critically for credibility

and relevance• Use information to answer question/solve

problem• Use information legally & ethically

In-Class Techniques

Strategic Planning/Critical Reasoning

Think-Aloud Exercise

www.websudoku.com

Metacognition

• Use efficient learning techniques• Discuss and evaluate problem-solving

strategies used• Critically examine the bases for arguments• Correct or revise reasoning• Form efficient plans for completion• Evaluate effectiveness of decisions/actions

In-Class Techniques

Assessing Professional Attitudes/Disposition

Opinion Polling

When a trainee fails to perform a task as expected, it is most often because:

A. s/he doesn’t value the task or see it as personally relevant.

B. s/he does not understand what is being asked of her/him.

C. s/he is overburdened or fails to plan for timely completion.

D. s/he lacks required knowledge or skill to complete the task.

When an educator deviates from a professional standard, it is most often because:

A. s/he opposes or disagrees in general with the standard.

B. s/he is unaware of the standard.C. practical realities make the standard

difficult or impossible to follow. D. s/he deems the standard to be inapplicable

or inappropriate for the particular situation.

How Students Can Contribute to Assessment – Suskie, 2009

• Identify the purpose of the assessment and ensure that they are relevant and worthwhile

• Articulate learning goals, their importance and value

• Design tools (rubrics, surveys, portfolios)• Advise on how to share results with students• Review logistics of assessments for feasibility

and undue burden

Group Activity:Reducing Medication Error

GROUP A: Root Cause Analysis

• 42 y.o. male, limited English, history afib, on verapamil 250 mg three times daily

• Admitted from ER for ORIF wrist fx

• 24 hours later developed atrial fibrillation, which resulted in congestive heart failure

GROUP B: Polypharmacy• Atrovent• Prednisone• Claritin• Xanax• Vasotec• Timoptic• Avandia• Neurontin• Augmentin• Protonix• Bumex

Group Activity:Developing Formative Assessments

What “key” knowledge should your learners demonstrate through this task?

Which “universal” skills (communication, information literacy, strategic planning/critical reasoning, attitude/disposition) will you focus on? To what extent (percentage, points)?

Which criteria will you use to rate the adequacy of their proposal?

What challenges/feedback will you provide to advance their learning beyond the initial effort?