2015 fall teaching institute louisiana state university at alexandria august 20, 2015 why do we...
TRANSCRIPT
2015 Fall Teaching InstituteLouisiana State University at Alexandria
August 20, 2015
Why do We Assess? Improving Student Learning through Performance Based Measures
Dr. Kim Chavis, Ph.D.North Carolina A & T State
University
Words to Consider
"Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.” '
Dr. Tom Angelo, Reassessing (and Defining) Assessment. The AAHE Bulletin, 48(2), November 1995, pp.7-9.
THERE IS A DIFFERENCE BETWEEN WHAT IS DESIGNED, TAUGHT, AND LEARNED
Designed curriculum
Official course outlinesCourse catalog
Taught curriculum
Found in syllabi
Learned curriculum
What have they learnedWhat can they doWhat do they care about
NATIONAL MOVE TOWARD ASSESSMENT OF LEARNING
What is assessment?
“the methods that an institution employs to gather evidence and evaluate quality
(Accrediting Commission for Community and Junior Colleges)
What are student learning outcomes?
“statements which define what a student should be able to do after the completion of a course or program”
(http://online.bc.cc.ca.us/courseassessment)
In terms of the learning students are able to demonstrate Rather than
inputs such as number of credit hours or curricular content
There Has Been a Paradigm Shift in the Way That We View Educational Effectiveness
Learning Outcomes:
ASSESSMENT: WHY BOTHER?
Three main purposes for assessment
1. To demonstrate quality and excellence and ensure the same level of quality continues
2. To identify areas needing attention, support and development and decide how to improve those areas
3. To plan changes that will improve policies, procedures, services, curriculum, resources, teaching, campus climate and ultimately improve student learning.
IN THE PRINCIPLES FOR ACCREDITATION
Comprehensive standard 3.3.1
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)
ASSESSMENT STAIRS
WHY BOTHER WITH LEARNING OUTCOMES?
Supporters assert…
They help students learn more effectively Students know what to expect from a
particular course or program Learning outcomes help instructors
Design their materials more effectively Select the appropriate teaching
strategy and technology Design examinations which reflect
what was taught
10
PROGRAMS NEED TO BE ASSESSED
Programs are an aggregation of courses with a common program mission.
The heart of assessment is embedded in the courses.
The program’s product is the graduate.
ESTABLISH CLEAR PROGRAM LEARNING DOCUMENTS
Programs should clearly define
Program Mission
Program Goals and Objectives
Learning Outcomes for each Goal
Assessment Methods
Expected Results
Actual Results
Use of Results to Improve the Program
A Mission…A mission indicates the purpose of the program or its reason for being
A mission states what it hopes to achieve in the future
A mission affirms its principles and beliefs, nature, and values.
A Sample Mission…
The Forestry and Ecology program is designed to educate broad-based, ecologically sensitive resource managers, enabling them to succeed as professional managers and practice wise stewardship of forests or other natural resources for multiple uses. As part of an Historically Black College or University, the program addresses the needs of capable students who are underrepresented in the forestry profession as well as the needs of the minority, forest-landowner community that have historically been under-served by the forestry profession.
A program goal…
A goal is broadly stated
A goal is accomplished via objectives
A goal is not directly measurable
Example: To understand and apply the concepts of bioinformatics in professional life
A program educational objective…
Objectives describe knowledge, skills, abilities, attitudes, or dispositions
An objective is directly measurable
An objective is specific and tied to a series of linked learning outcomes
Educational objectives are sometimes considered synonymous with learning outcomes, but learning outcomes are more detailed, behavioral in nature, and stated using precise verbs.
A program learning outcome…
Describes what…
we want the students in the program to know
we want the students to be able to do
values or attitudes we want to instill in our students
A program learning outcome is capable of being assessed
ProgramSLOs
CourseSLOs Course
SLOs
CourseSLOs
CourseSLOsCourse
SLOs
CourseSLOs
Overlapping Course Student Learning Outcomes Become Program Student Learning Outcomes
COURSE MAPPING
Course mapping helps to develop a sense of how current course offerings in the program cover different learning outcomes
A course map is a table with the learning outcomes on one axis and the courses in the major on the other axis
ADVANTAGES OF CURRICULUM MAPPING IN ASSESSING STUDENT LEARNING OUTCOMESCurriculum Mapping causes…
Thought about LEARNING INPUTS – Things that are in place before learning begins [prerequisites, placement, class size, facilities & equipment.
Thought about LEARNING PROCESSES – Time on task, active learning opportunities.
Thought about LEARNING CONTEXT – Professional v. liberal arts program, professional expectations, attitudes towards learning.
http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm
Simple Course Map
http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm
More Complex Course
Map Indicating Level of
Emphasis on
Learning Outcome
MYTHS ABOUT ASSESSMENT
It is not my job
I already assess; I call it grading
You cannot assess what we do in academia
I do not have the time to do it
I do not have the expertise to do it
Assessment might reveal “bad” news
It is a passing fad
YOU CANNOT ASSESS WHAT YOU HAVE NOT DEFINED: ESTABLISH CLEAR LEARNING OUTCOMES
Are your learning outcomes clear?
Are they shared outcomes?
Can you talk about them easily?
What do the results of assessment mean?
How are we going to use the results to improve student learning?
Closing the Loop
UNTIL THE RESULTS OF ASSESSMENT OF STUDENT LEARNING ARE USED, THEY HAVE NO VALUE
This may take the form of revision of course content of individual courses, revision of curricula, changes in methods of teaching, inclusion of experiential approaches
The changes must in turn be assessedfor effectiveness themselves
Don’t forget – Assessment is
not useful until it is
used!
Learning Outcomes…
Behaviors that demonstrate
• desired knowledge (cognitive domain)
• attitudes or dispositions (affective domain)
• acquired skills (psychomotor domain)
Fundamentally, the key is what will students be able to do as a result of the instruction.
The Cognitive Domain
Knowledge ComprehensionApplication
Synthesis
Analysis Evaluation
Bloom’s Classification
Knowledge…
Recalling or remembering something without necessarily understanding, using, or changing it
Count Define Describe Draw Identify
Label List Match Name Point to
Outline Quote Read Recall Recite
Record Repeat Select State Write
Comprehension…
Understanding something that has been communicated without relating it to anything else
Associate Compute Convert Defend
Discuss Distinguish Estimate Explain
Extend Extrapolate Generalize Infer
Paraphrase Predict Rewrite Summarize
Application…
Using a general concept to solve problems in a particular situation or using learned material in new and concrete situations
Add Apply Calculate Classify
Complete Compute Divide Examine
Graph Interpolate Manipulate Modify
Operate Produce Show Solve
Analysis…Breaking something down into its parts (identification of parts, analysis of relationships among parts, recognition of organizing principles)
Analyze Arrange Breakdown Combine
Design Detect Develop Diagram
Illustrate Outline Point out Relate
Select Separate Subdivide Utilize
Synthesis…
Creating something new by putting parts of different ideas together to make a whole
Categorize Combine Compile Compose
Create Design Devise Generate
Group Integrate Modify Order
Organize Plan Prescribe Propose
Rearrange Reconstruct Reorganize Revise
Evaluation…Judging the value of material or methods as they might be applied in particular situations or judging with the use of definite criteria
Appraise Assess Compare Conclude
Contrast Criticize Determine Grade
Interpret Judge Justify Measure
Rank Rate Support Test
Knowledge Comprehension Application Analysis Synthesis Evaluation
Student remembers or recognizes information as communicated with little personal assimilation
Student grasps the meaning behind the information and interprets or comprehends the information
Student uses information to relate and apply it to a new situation with minimal instructor input
Student discriminates, organizes, and scrutinizes assumptions in an attempt to identify evidence for a conclusion
Student creatively applies knowledge and analysis to integrate concepts or construct an overall theory
Student judges or evaluates information based upon standards and criteria, values, and opinions
Cite LabelListEnumerateIdentifyImitateMatchNameQuoteRecallReproduceStateWrite
ConvertDefineDescribeDiscussEstimateExplainGeneralizeIdentifyIllustrateLocateParaphraseRestateSummarize
ApplyChartComputeDemonstrateDetermineDramatizeEstablishMakeManipulatePrepareProjectSolveUse
AnalyzeCompareContrastCorrelateDiagramDissectDifferentiateDistinguishInferInvestigateLimitOutlineSeparate
AssembleCreateConstructDesignDevelopFormulateGenerateHypothesizeInitiateInventModifyReframeSynthesize
AccessAppraiseConcludeCritiqueDecideDefendDiagnoseEvaluateJudgeJustifyRankRecommendSupport
Cognitive Domain – Learning Outcomes Related to Knowledge
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
Observe Model RecognizeStandards
Correct Apply Coach
Studentstranslate sensory input into physical tasks or activities
Students are able to replicate a fundamental task or skill once shown
Students recognize standards or criteria important to perform a task or skill correctly
Students use standards to evaluate their own performances and make corrections
Students apply this skill or ability to real life situations
Students are able to instruct or train others to perform this skill in other situations
HearIdentifyObserveSeeSmellTasteTouchWatch*
AttemptCopyFollowImitateMimicModelReenactRepeatReproduceShow Try
CheckDetectDiscriminateDifferentiateDistinguishNoticePerceiveRecognizeSelect
AdaptAdjustAlterChangeCorrectCustomizeDevelopImproveManipulateModifyPracticeRevise
BuildComposeConstructCreateDesignOriginateProduce
DemonstrateExhibitIllustrateInstructTeachTrain
Psychomotor Domain – Learning Outcomes Related to Skills
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
Receiving Responding Valuing Organizing Characterizing
Students become aware of an attitude, behavior, or value
Students exhibit a reaction or change as a result of exposure to an attitude, behavior, or value
Students recognize value and display this through involvement or commitment
Students determine a new value or behavior as important or a priority
Students integrate consistent behavior as a naturalized value in spite of discomfort or cost and the value is recognized as part of the person’s character
AcceptAttendDescribeExplainLocateObserveRealizeReceiveRecognize
BehaveComplyCooperateDiscussExamineFollowModelPresentRespondShowStudies
AcceptAdaptBalanceChooseDifferentiateDefendInfluencePreferRecognizeSeekValue
AdaptAdjustAlterChangeCustomizeDevelopImproveManipulateModifyPracticeRevise
AuthenticateCharacterizeDefendDisplayEmbodyHabituateInternalizeProduceRepresentValidateVerify
Affective Domain – Learning Outcomes Related to Attitudes, Behaviors, and Values
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
TO DEVELOP A LEARNING OUTCOME, BUILD IT IN PARTS…
Students will be able to demonstrate
by
learning of…the ability to…competence with…knowledge of…understanding of…
Insert informationand/orskills to belearned
Insertverbfromlist
Insert whatstudent must doto demonstratethe learning
Example:
Students will be able to demonstrate knowledge of plant nutrient requirements by identifying deficiency symptoms and listing actions to alleviate them.
EXAMPLES…
Students will be able to demonstrate knowledge of correct syntax by editing written narratives from other students
Students will be able to demonstrate understanding of human circulatory systems by drawing diagrams illustrating the path that blood follows in the body
Students will be able to demonstrate the ability to use logic by constructing a deductive argument from a real life setting
POOR LEARNING OUTCOMES…
Students will get an introduction to bioinformatics and its practical applications.
Students will demonstrate proficiency by conducting a review of published research.
Students will demonstrate an understanding of the connections between religion and politics.
Students will demonstrate proficiency with conversational Dutch by the end of the semester.
Students will demonstrate an appreciation for music of the Band Era.
LEARNING OUTCOMES ARE NOT THE SAME AS COURSE OBJECTIVES OR GOALS
Outcomes Represent overarching
products of the course Expresses higher level
thinking as observed behaviors, skills, and useable knowledge
Can be assessed as an end product and evaluated against specific criteria
Objectives Present valuable skills, tool,
or content that enable a student to do well in the course
Focus on content that is important in the classroom or what the faculty will do (input into the course)
Can be numerous, specific, and detailed
SAMPLES FOR COMPARISONGoal (the purpose of the course in more general terms)
The goal of the Scientific Writing course is to teach the basic tenets of ethical research, sound scientific writing, presentation of data, and development of proposals in response to funding opportunities.
Objectives (details course contents or activities but not what the student will do to demonstrate learning)Learn how to organize the parts of a scientific paper that is to be submitted as a manuscript for publication.
Learning Outcomes (describes what a student will know or do at the end)Students will demonstrate competency in presenting data by developing graphs for both continuous and non-continuous data.
MINI-QUIZARE THESE GOALS, OBJECTIVES, OR LEARNING OUTCOMES?
This course introduces the concept of human nutrition and its relationship with health interventions, educational training, and policy.
Concepts of nutritional requirements and how they vary with age, gender, and activity will be described.
Students will demonstrate knowledge of nutrition by documenting a nutritional problem, determining a strategy to correct it, and drafting a set of preventative strategies.
GOOD LEARNING OUTCOMES ARE ASSESSABLE!
Define expectations Develop criteria Measure the
outcome
YOU CANNOT ASSESS WHAT YOU HAVE NOT DEFINED:
ESTABLISH CLEAR LEARNING GOALS
Are your learning goals clear? Are they shared goals? Can you talk about them easily?
STUDENT LEARNING OUTCOMES: QUESTIONS TO CONSIDER
How would you ensure students have the opportunity to learn this?
How would you evaluate this in a way that is NOT dependent on a course instructor’s opinion or grade?
How would you compile the results of your evaluation?
How would you share the results? If you were dissatisfied with your
findings, what would you do about it?
47
OUTCOME DEVELOPMENT EXERCISE Use the form and lists of active verbs in
each domain to describe the attitudes, skills, and knowledge that you would like your students to know or do as a result of your course.
Divide them into domains Affective Psychomotor Cognitive
Try to get at least three in each section
DRAFT YOUR OWN LEARNING OUTCOMES
Students will demonstrate…
learning of…the ability to…competence with…knowledge of…understanding of…
Insert informationand/orskills to belearned
Insertverbfromlist
Insert whatstudent must doto demonstratethe learning
of
by
Type of learningKnowledge, skill or attitude
Action or behaviorActive verb
Try to doten of them
WHY AREN’T GRADES ENOUGH?
Grades evaluate a limited set of objectives which may or may not be related to the program objectives
They tend to be inconsistent from section-to-section and from term-to-term to be a valid overall program assessment tool
There is a conflict of interest when the instructor is the only evaluator of whether students have met program objectives
Grades do not reflect long term learning and accomplishments
Many times they do little to capture perceptions, attitudes, and skills
Knowledge
Recalling or remembering something without necessarily understanding, using, or changing it
Good Assessment Tools:
Students will identify, describe, or recount in their own words…-Written tests and essays- Oral tests- Reports- Making lists- Role play focused on accurate information- One minute papers
Understanding something that has been communicated without relating it to anything else
Comprehension
Application…Using a general concept to solve problems in a particular situation or using learned material in new and concrete situations
Good assessment tools:
Students will apply, demonstrate, solve…- Problem scenario- Problem set- Demonstration of skill- Role play- Lab report- Field experience- Internship report
Analysis…Breaking something down into its parts (identification of parts, analysis of relationships among parts, and recognition of organizing principles)
Good assessment tools:
Students will categorize, examine, induce…- Case study- Journal- Field experience/internship reflection- Complex problems- Conduct experiments and test hypotheses- Observations- Examinations of an object, writing, process
Synthesis…
Creating something new by putting parts of different ideas together to make a whole
Good assessment tools:
Students will design, integrate, relate…- Project- Research thesis- Essay- Design an experiment- Annotated bibliography- Create a video or poster- Portfolio
Evaluation…Judging the value of materials or methods as they might be applied in particular situations or judging with the use of definite criteria
Good assessment tools:
Students will conclude, critique, justify…- Debate- Essay- Report- Journal- Write and editorial/position paper- Comment on author’s perspectives- Book review- Prepare a committee/team briefing paper