using assessment to improve the quality of education · tips (cont’d) ! give reading assignments...
TRANSCRIPT
![Page 1: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/1.jpg)
Using Assessment to Improve the Quality of Education
August 2007 Jan Arminio
[email protected] Professor, Shippensburg University
and President of CAS
Lafayette College
![Page 2: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/2.jpg)
What is assessment?
![Page 3: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/3.jpg)
What is assessment?
n Systematic collection and interpretation of data
n Locally designed and executed
![Page 4: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/4.jpg)
Why assess?
![Page 5: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/5.jpg)
Why assess?
n To improve education and student learning
n To examine congruence with institution's stated purpose
![Page 6: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/6.jpg)
Also…
n accreditation n performance funding n to address concerns of effectiveness
(access, cost, and decline in student learning)
n indicates seriousness
![Page 7: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/7.jpg)
Plus…
n Ethical practice n Curiosity
![Page 8: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/8.jpg)
What differentiates…
n Sound assessments n Poor assessments
![Page 9: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/9.jpg)
Good Assessments
n Provide useful information n Provide reasonably accurate information n Protect privacy of students n Are systematic and regular n Are effective in terms of time, cost, and
other resources
![Page 10: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/10.jpg)
To examine congruence with institution's stated purpose:
Five Column Model
![Page 11: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/11.jpg)
CAS
n Self assessment
![Page 12: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/12.jpg)
History of CAS n Consortium of professional associations
established in 1979 n First standards published in 1986 n Currently 37 sets of standards and 37
member associations n Purpose is to promulgate standards to
ensure quality in educational programs n Recently published 6th edition of standards
![Page 13: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/13.jpg)
Standards
n What are they? n What role to they serve? n What does compliance to standards
indicate?
![Page 14: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/14.jpg)
How are CAS Standard written?
![Page 15: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/15.jpg)
13 Sections of Standards
n Mission n Program n Leadership n Organization and
Management n Human Resources n Financial Resources n Facilities, Technology,
and Equipment
n Legal Responsibilities n Equity and Access n Campus and External
Relations n Diversity n Ethics n Assessment and
Evaluation
![Page 16: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/16.jpg)
CAS Standards Part 1. MISSION n The purpose of the Campus Activities Program (CAP) must be
to enhance the overall educational experience of students through development of, exposure to, and participation in programs and activities that improve student cooperation and leadership while preparing students to be responsible advocates and citizens and complementing the institution's academic programs.
n These activities could be intellectual, social, recreational, cultural,
multicultural, and spiritual in nature. Programs could pertain to leadership, governance, community service, healthy lifestyles, and organizational development.
![Page 17: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/17.jpg)
CAS Self-Assessment Process
n Establish and Prepare Self-assessment Team n Initiate Self Study n Identify and Summarize Evaluative Evidence n Identify Discrepancies n Determine Appropriate Corrective Action n Recommend Steps for Program Enhancement n Prepare an Action Plan
![Page 18: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/18.jpg)
Establish and Prepare Self-assessment Team n Who should serve? n Who should not serve? n How should the team be trained?
![Page 19: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/19.jpg)
Initiate Self Study
n When should the data be collected? n What data should be collected? n What other information is necessary?
![Page 20: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/20.jpg)
Identify and Summarize Evaluative Evidence n How should the materials be
organized? In this step reviewers ensure that
evidence exists to support criterion measure ratings
![Page 21: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/21.jpg)
Identify Discrepancies
n Is a “good” review one in which no discrepancies are found?
![Page 22: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/22.jpg)
Action Steps
n Determine Appropriate Corrective Action
n Recommend Steps for Program Enhancement
n Prepare an Action Plan
![Page 23: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/23.jpg)
CAS Exercise
![Page 24: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/24.jpg)
Go back to our 5 column model
n Goals
![Page 25: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/25.jpg)
Measuring Learning Outcomes
n The formal education of students consists of the curriculum and the co-curriculum and must promote student learning and development that is purposeful and holistic. Programs must identify relevant and desirable student learning and development outcomes and provide programs and services that encourage the achievement of those outcomes.
![Page 26: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/26.jpg)
Learning and Program Goals:
n Create your own n Learning Reconsidered n Learning Reconsidered II n CAS n General Education Goals
![Page 27: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/27.jpg)
16 CAS Learning Domains n intellectual growth n effective
communication n realistic self-appraisal n enhanced self-esteem n clarified values n career choices n leadership development n healthy behaviors n meaningful
interpersonal relationships
n independence n collaboration n social responsibility n satisfying and
productive lifestyles n appreciation of
diversity n spiritual awareness n achievement of
personal and educational goals.
![Page 28: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/28.jpg)
Bloom’s Cognitive Taxonomy: Bloom Verb * Tasks
1 Knowledge Remember Write, list, label, state, define
2 Comprehension Understand Explain, summarize, paraphrase, illustrate
3 Application Apply Use, compute, demonstrate, apply, solve, construct
4 Analysis Analyze Analyze, categorize, contrast, compare
5 Synthesis Evaluate Create, design, hypothesize, invent
6 Evaluation Create Justify, critique, recommend, judge
* From: Anderson & Krathwohl (2001)
![Page 29: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/29.jpg)
Bloom’s Affective Taxonomy: Domain Verbs
Willingness to Receive acknowledge, ask, attend, be aware, choose, follow, give, hold, identify, listen, locate, receive, reply, select, tolerate, use, view, watch
Responding, Active Participation
agree (to), answer, ask, assist, communicate, consent, conform, contribute, cooperate, discuss, follow-up, greet, help, inquire, participate, pursue, question, react, read, reply, report, request, respond, seek, select, visit, volunteer, write
Valuing adopt, approve, complete, choose, commit, desire, display, endorse, exhibit, explain, express, form, initiate, invite, join, justify, prefer, propose, sanction, select, share,
Organization, Integration of Values
adapt, adhere, alter, categorize, classify, combine, complete, defend, explain, establish, formulate, order, organize, prepare, rank, rate, relate, systemize
Internalization act, advocate, continue, defend, devote, disclose, display, encourage, endure, exemplify, incorporate, influence, justify, practice, question, revise, retain, support, uphold,
![Page 30: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/30.jpg)
Learning Cognitive Domain Example: White Privilege Knowledge Comprehension Application Analysis Evaluation
Define Unearned assets one can count
on
Identify Point out an
example
Dramatize Act out
example during a role play
Compare Differentiate experiences of self and
others
Assesses Revise one’s
worldview based on new
knowledge and
contemplate ways to behave
differently
![Page 31: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/31.jpg)
Learning Affective Domain Example Goal: Become Efficient in Assessment
Willingness To Receive
Responding, Active
Participation
Valuing Integration of Values
Internalization
Attend Participate
Endorse
Systematize
Devote
![Page 32: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/32.jpg)
Bloom’s Cognitive Taxonomy: Goal
Goals Verb (objective) How to Measure
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
![Page 33: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/33.jpg)
Learning Affective Domain
Willingness To Receive
Responding, Active
Participation
Valuing Integration of Values
Internalization
![Page 34: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/34.jpg)
Go back to our 5 column model
n Write Student Learning Outcomes
![Page 35: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/35.jpg)
Processes
n CAS Review n Establish and Prepare Self-
assessment Team ¨ Goals/Priorities
n Initiate Self Study n Identify and Summarize
Evaluative Evidence n Identify Discrepancies n Determine Appropriate Corrective
Action n Recommend Steps for Program
Enhancement n Prepare an Action Plan
n General Assessment n Goals n Select Program and Services
Initiatives n Assessment/Evaluation
n Use Results for
Improvement
![Page 36: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/36.jpg)
Learning Outcomes n How do we measure learning? 1. Identify goals and measurable objectives 2. Identify programs and services that
accomplish associated goals 3. Measure learning through sound data
collection and interpretation 4. Use results to improve learning
opportunities
![Page 37: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/37.jpg)
Processes Measuring Learning
Outcomes 1. Identify goals and measurable
objectives 2. Identify programs and services that
accomplish associated goals 3. Measure learning through sound
data collection and interpretation 4. Use results to improve learning
opportunities
General Assessment n Goals
n Select Program and Services
Initiatives n Assessment/Evaluation
n Use Results for Improvement
![Page 38: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/38.jpg)
Comprehensive Assessment Model
n User Data/Satisfaction n Assessing Student Needs n Campus Environments n Outcomes n Inputs n Motivation
![Page 39: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/39.jpg)
How to measure objectives:
n Surveys n Interviews (including focus groups) n Portfolios n Rubrics (rubric exercise) n Observation
![Page 40: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/40.jpg)
Your assessment plan should use…
n Multiple measures n Indirect and direct measures
![Page 41: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/41.jpg)
Assessment Tips
n Add learning outcomes to program evaluations
n Add assessment expectations to position descriptions
n Ask questions about assessment experience when interviewing new staff
![Page 42: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/42.jpg)
Tips (cont’d)
n Give reading assignments when training and offer pre-tests and post-tests
n Quick, lean, and over the phone or in
combination with on-line surveys
n If using qualitative methodologies, be sure you are knowledgeable and trained
![Page 43: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/43.jpg)
Tips (cont’d)
n Consider longitudinal studies n Do not be afraid to communicate results
and when you do offer executive summaries and access to the full report
n Consider electronic portfolios and automated data collectors
n Compare against national data
![Page 44: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/44.jpg)
Tips (cont’d)
n Search for students who could serve as inexpensive consultants
n Have access to SPSS windows n Systematize the process, for example
include survey to a mailing you already distribute
![Page 45: Using Assessment to Improve the Quality of Education · Tips (cont’d) ! Give reading assignments when training and offer pre-tests and post-tests ! Quick, lean, and over the phone](https://reader033.vdocuments.us/reader033/viewer/2022052022/6037f6d8452ddf4032240c30/html5/thumbnails/45.jpg)
Tips (cont’d)
n Do not confuse assessments with personnel evaluations, however, include willingness to assess
n Do not engage in assessment if you are
not willing to adapt to what the data indicates