![Page 1: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/1.jpg)
1
Principles of and Practices in Assessing Student Learning
Presented by Peggy L. Maki Simmons CollegeGraduate School of Library & Information ScienceMay, [email protected]
![Page 2: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/2.jpg)
2
Integrating Teaching, Learning, and AssessmentPedagogy
Curricular design
Instructional design
Educational tools
Educational experiences
Students’ Learning Histories/Styles
![Page 3: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/3.jpg)
3
Integrated Learning….
Cognitive
AffectivePsychomotor
![Page 4: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/4.jpg)
4
Consider the importance of answering the questions under “Questions to Ground Discussion of Teaching, Learning and Assessment.”
![Page 5: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/5.jpg)
5
Major Collaborative Tasks
Articulating learning outcome statements that align with collective educational practices
Mapping outcomes to the curriculum according to a meaningful labeling system that also identifies course-based assessment practices
![Page 6: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/6.jpg)
6
Identifying or designing methods to assess learning—more than one method to capture the dimensions of learning
Developing standards and criteria of judgment to assess student achievement—scoring rubrics—and inter-rater reliability
![Page 7: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/7.jpg)
7
Collecting, scoring, and analyzing
Interpreting results within the context of other relevant data that you identify before you begin to assess
Implementing agreed upon changes to improve or advance learning; then re-entering the assessment cycle
![Page 8: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/8.jpg)
8
The Process: 1. Articulating Learning Outcome Statements
Course/Service/Educational Experience Outcome Statements
Institution-level Outcome Statements
Department- or Program-level Outcome Statements
![Page 9: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/9.jpg)
9
What Is an Outcome Statement?
Describes learning desired within a context
Relies on active verbs (create, compose, calculate)
Emerges from our collective intentions over time
![Page 10: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/10.jpg)
10
Can be mapped to curricular and co-curricular practices (ample, multiple and varied opportunities to learn over time)
Can be assessed quantitatively or qualitatively during students’ undergraduate and graduate careers
Is written for a course, program, or institution
![Page 11: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/11.jpg)
11
Example from ACRL
ONE OUTCOME: Student examines and compares information from various sources in order to evaluate reliability, validity,accuracy, timeliness, and point of view or bias.
![Page 12: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/12.jpg)
12
Ways to Articulate Outcomes
Adapt from professional organizations
Derive from mission of institution/program/department/service
Derive from students’ work that demonstrates interdisciplinary thinking, ways of knowing, or problem solving
![Page 13: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/13.jpg)
13
Derive from faculty to faculty interview process
Derive from exercise focused on listing one or two outcomes “you attend to”
![Page 14: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/14.jpg)
14
Question?
Using the handout, review your outcomes against criteria for outcome statements to determine that your outcomes meet those criteria
![Page 15: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/15.jpg)
15
The Process: 2. Mapping Learning Outcomes
Curricular:• Reveal how we translate outcomes into
educational practices offering students multiple and diverse opportunities to learn
• Help us to identify appropriate times to assess those outcomes
• Identify gaps in learning or opportunities to practice
![Page 16: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/16.jpg)
16
Help students understand our expectations of them
Place ownership of learning on students
Enable them to develop their own maps or learning chronologies
(review sample curricular maps and inventories)
![Page 17: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/17.jpg)
17
Group Work on Mapping
After you have reviewed the two curricular maps and inventories, identify a useful inventory and labeling system you might use to map your outcomes. Your system will enable you to identify assessment methods already in place as well as times to engage in formative and summative assessment.
![Page 18: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/18.jpg)
18
“Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The tasks to which students are asked to respond on an assessment are not arbitrary.“
National Research Council. Knowing what students know: The science and design of educational assessment . Washington, D.C.: National Academy Press, 2001, p. 47.
The Process: 3. Designing or Selecting Assessment Methods
![Page 19: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/19.jpg)
19
Assumptions UnderlyingTeaching
Actual Practices
Assumptions UnderlyingAssessment Tasks
Actual Tasks
![Page 20: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/20.jpg)
20
What Tasks Elicit Learning You Desire?
Tasks that require students to select among possible answers (multiple choice test)?
Tasks that require students to constructanswers (students’ problem-solving and thinking abilities)?
![Page 21: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/21.jpg)
21
When Do You Seek Evidence?
Formative—along the way?For example, to ascertain progress or development
Summative—at the end?For example, to ascertain mastery level of achievement
![Page 22: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/22.jpg)
22
Some Methods That Provide Direct Evidence
Student work samples, such as reports, theses, other written documents or oral presentations
Collections of student work (e.g. Portfolios)
Capstone projects
![Page 23: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/23.jpg)
23
Course-embedded assessment (derive examples; develop agreed upon prompt and ask students to respond to it in class or at a designated time)
Observations of student behavior
Internal juried review of student projects—creations, demonstrations
Simulations—virtual or live
![Page 24: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/24.jpg)
24
External juried review of student projects
Externally reviewed internship
Performance on a case study/problem
Performance on case study accompanied with students’ analysis
![Page 25: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/25.jpg)
25
Representative disciplinary practices
Performance on national licensure examinations or other standardized tests
Locally developed tests
Pre-and post-tests
![Page 26: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/26.jpg)
26
The Process: 4. Developing Criteria and Standards of Judgment
A set of criteria that identifies the expected characteristics of a text and the levels of achievement along those characteristics. Scoring rubrics are criterion-referenced, providing a means to assess the multiple dimensions of student learning.
Are collaboratively designed based on how and what students learn (based on curricular-co-curricular coherence)
![Page 27: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/27.jpg)
27
Are aligned with ways in which students have received feedback (students’ learning histories)
Students use them to develop work and to understand how their work meets standards (can provide a running record of achievement).
![Page 28: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/28.jpg)
28
Raters use them to derive patterns of student achievement to identify strengths and weaknesses
![Page 29: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/29.jpg)
29
Strategies to Develop Scoring Rubrics
Emerging work in professional and disciplinary organizations
Research on learning (from novice to expert)
Student work
Interviews with students
![Page 30: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/30.jpg)
30
Experience observing students’development
![Page 31: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/31.jpg)
31
Pilot-testing the Scoring Rubric
Apply to student work to assure you have identified all the dimensions with no overlapSchedule inter-rater reliability times:
-independent scoring-comparison of scoring-reconciliation of responses-repeat cycle
![Page 32: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/32.jpg)
32
The Process: 5. Collecting and Analyzing Results
Determine how and when to collect student responses
Determine who will analyze results (suitable for aggregation and disaggregation according to question you wish to answer)
![Page 33: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/33.jpg)
33
The Process: 6. Interpreting Results
Seek patterns against criteria and cohorts
Build in program-level discourse
Tell the story that explains the results—triangulate with other data
Determine what you wish to change, revise, or how you want to innovate
![Page 34: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/34.jpg)
34
Implement agreed upon changes
Focus on collective effort—what we do and how we do it
Re-assess to determine efficacy of changes
(See report format to guide annual reports)
The Process: 7. Implementing Changes and Re-Entering the Assessment Cycle
![Page 35: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/35.jpg)
35
Examples of Changes:
Increased attention to weaving experiences across the program to improve student achievement
Changes in advising based on assessment results
Closer monitoring of student achievement—tracking according to demographics or other meaningful criteria
![Page 36: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/36.jpg)
36
Faculty and staff development to learn how to integrate experiences that contribute to improved student learning
Changes in pedagogy and curricular and co-curricular design
Development of modules to assist learning; use of technology; self-paced learning, supplemental learning
![Page 37: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/37.jpg)
37
Mission/Purposes
Learning Outcomes
How well dowe achieve
our outcomes?
Gather Evidence
Interpret Evidence
Enhance teaching/ learning;
inform institutional decision-
making, planning,budgeting
![Page 38: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/38.jpg)
38
“What and how students learn depends to a major extent on how they think they will be assessed.”
John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.
![Page 39: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/39.jpg)
39
List of attachmentsQuestions that ground discussions focused on teaching, learning, and assessmentChecklist for outcome statementsDissemination of learning outcome statementsCurricular-co-curricular mapInventories of educational and assessment practicesIdentification of assessment methodsExamples of scoring rubricsInventory of direct and indirect methodsAnnual report format
![Page 40: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/40.jpg)
40
Integrating Teaching, Learning, and AssessingPedagogy
Curricular design
Instructional design
Educational tools
Educational experiences
Students’ Learning Histories/Styles
![Page 41: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/41.jpg)
41
Integrated Learning….
Cognitive
AffectivePsychomotor
![Page 42: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/42.jpg)
42
The Process: 1. Development of Learning Outcome Statements
Course/Service/Educational Experience Outcome Statements
Institution-level Outcome Statements
Department- or Program-level Outcome Statements
![Page 43: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/43.jpg)
43
What Is an Outcome Statement?
Describes learning desired within a context
Relies on active verbs (create, compose, calculate)
Emerges from our collective intentions over time
![Page 44: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/44.jpg)
44
Can be mapped to curricular and co-curricular practices (ample, multiple and varied opportunities to learn over time)
Can be assessed quantitatively or qualitatively during students’ undergraduate and graduate careers
Is written for a course, program, or institution
![Page 45: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/45.jpg)
45
Distinguishing between Objectives and Outcomes
Objectives state overarching expectations such as—
Students will develop effective oral communication skills.
ORStudents will understand different economic principles.
![Page 46: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/46.jpg)
46
Compare:
Students will write effectively.
Students will compose a range of professional documents designed to solve problems for different audiences and purposes.
to
![Page 47: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/47.jpg)
47
Compare:
Students will writeeffectively.
toStudents will summarize recent articles on economics and identify underlying economic assumptions.
![Page 48: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/48.jpg)
48
Example from ACRL
ONE OUTCOME: Student examines and compares information from various sources in order to evaluate reliability, validity,accuracy, timeliness, and point of view or bias.
![Page 49: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/49.jpg)
49
Ways to Articulate Outcomes
Adapt from professional organizations
Derive from mission of institution/program/department/service
Derive from students’ work that demonstrates interdisciplinary thinking, ways of knowing, or problem solving
![Page 50: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/50.jpg)
50
Derive from faculty to faculty interview process
Derive from exercise focused on listing one or two outcomes “you attend to”
![Page 51: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/51.jpg)
51
Mapping Learning Outcomes
Curricular-Co-curricular maps:• Reveal how we translate outcomes into
educational practices offering students multiple and diverse opportunities to learn
• Help us to identify appropriate times to assess those outcomes
• Identify gaps in learning or opportunities to practice
![Page 52: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/52.jpg)
52
Help students understand our expectations of them
Place ownership of learning on students
Enable them to develop their own maps or learning chronologies
![Page 53: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/53.jpg)
53
“Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The tasks to which students are asked to respond on an assessment are not arbitrary.“
National Research Council. Knowing what students know: The science and design of educational assessment . Washington, D.C.: National Academy Press, 2001, p. 47.
The Process: 2. Design or Selection of Assessment Methods
![Page 54: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/54.jpg)
54
Assumptions UnderlyingTeaching
Actual Practices
Assumptions UnderlyingAssessment Tasks
Actual Tasks
![Page 55: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/55.jpg)
55
What Tasks Elicit Learning You Desire?
Tasks that require students to select among possible answers (multiple choice test)?
Tasks that require students to constructanswers (students’ problem-solving and thinking abilities)?
![Page 56: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/56.jpg)
56
When Do You Seek Evidence?
Formative—along the way?For example, to ascertain progress or development
Summative—at the end?For example, to ascertain mastery level of achievement
![Page 57: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/57.jpg)
57
Some Methods That Provide Direct Evidence
Student work samples, such as lab reports, theses, other written practices
Collections of student work (e.g. Portfolios)
Capstone projects (see AAC& U work on these kinds of projects)
![Page 58: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/58.jpg)
58
Course-embedded assessment (derive examples; develop agreed upon prompt and ask students to respond to it in class or at a designated time)
Observations of student behavior
Internal juried review of student projects—creations, demonstrations
Simulations—virtual or live
![Page 59: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/59.jpg)
59
External juried review of student projects
Externally reviewed internship
Performance on a case study/problem
Performance on case study accompanied with students’ analysis
![Page 60: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/60.jpg)
60
Disciplinary practices
Performance on national licensure examinations or other standardized tests
Locally developed tests
Pre-and post-tests
![Page 61: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/61.jpg)
61
The Process: 3. Development of Criteria and Standards of Judgment
A set of criteria that identifies the expected characteristics of a text and the levels of achievement along those characteristics. Scoring rubrics are criterion-referenced, providing a means to assess the multiple dimensions of student learning.
Are collaboratively designed based on how and what students learn (based on curricular-co-curricular coherence)
![Page 62: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/62.jpg)
62
Are aligned with ways in which students have received feedback (students’ learning histories)
Students use them to develop work and to understand how their work meets standards (can provide a running record of achievement).
![Page 63: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/63.jpg)
63
Raters use them to derive patterns of student achievement to identify strengths and weaknesses
![Page 64: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/64.jpg)
64
Development of Scoring Rubrics
Emerging work in professional and disciplinary organizations
Research on learning (from novice to expert)
Student work
Interviews with students
![Page 65: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/65.jpg)
65
Experience observing students’development
![Page 66: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/66.jpg)
66
Pilot-testing the Scoring Rubric
Apply to student work to assure you have identified all the dimensions with no overlapSchedule inter-rater reliability times:
-independent scoring-comparison of scoring-reconciliation of responses-repeat cycle
![Page 67: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/67.jpg)
67
The Process: 4. Collection and Analysis of Results
Determine how and when to collect student responses
Determine who will analyze results (suitable for aggregation and disaggregation according to question you wish to answer)
![Page 68: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/68.jpg)
68
The Process: 5. Interpretation of Results
Seek patterns against criteria and cohorts
Build in institutional level and program level discourse
Tell the story that explains the results—triangulate with other data
Determine what you wish to change, revise, or how you want to innovate
![Page 69: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/69.jpg)
69
Implement agreed upon changes
Re-assess to determine efficacy of changes
Focus on collective effort—what we do and how we do it
The Process: 6. Implement Changes and Re-Enter the Assessment Cycle
![Page 70: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/70.jpg)
70
Examples of Changes:
Increased attention to weaving experiences across the institution to improve student achievement
Changes in advising based on assessment results
Closer monitoring of student achievement--tracking
![Page 71: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/71.jpg)
71
Faculty and staff development to learn how to integrate experiences that contribute to improved student learning
Changes in pedagogy and curricular and co-curricular design
Development of modules to assist learning; use of technology; self-paced learning, supplemental learning
![Page 72: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/72.jpg)
72
Mission/Purposes
Learning Outcomes
How well dowe achieve
our outcomes?
Gather Evidence
Interpret Evidence
Enhance teaching/ learning;
inform institutional decision-
making, planning,budgeting
![Page 73: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/73.jpg)
73
“What and how students learn depends to a major extent on how they think they will be assessed.”
John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.
![Page 74: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/74.jpg)
74
List of attachmentsChecklist for outcome statementsDissemination of learning outcome statements Questions that ground discussions focused on teaching, learning, and assessmentInventories of educational and assessment practicesCurricular mapsIdentification of assessment methodsExamples of scoring rubricsAnnual report format
![Page 75: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/75.jpg)
75
1. Developing and Mapping Learning Outcome Statements
Course/Service/Educational Experience Outcome Statements
Institution-level Outcome StatementsIncluding GE
Department- or Program-level Outcome Statements
![Page 76: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/76.jpg)
76
What Is an Outcome Statement?
Describes learning desired within a context
Relies on active verbs (create, compose, calculate)
Emerges from our collective intentions over time
![Page 77: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/77.jpg)
77
Can be mapped to curricular and co-curricular practices (ample, multiple and varied opportunities to learn over time)
Can be assessed quantitatively or qualitatively during students’ undergraduate careers
Is written for a course, program, or institution
![Page 78: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/78.jpg)
78
Distinguishing between Objectives and Outcomes
Objectives state overarching expectations such as—
Students will develop effective oral communication skills.
ORStudents will understand different economic principles.
![Page 79: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/79.jpg)
79
Learning Outcome Statementfrom ACRL
ONE OUTCOME: Student examines and compares information from various sources in order to evaluate reliability, validity,accuracy, timeliness, and point of view or bias.
![Page 80: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/80.jpg)
80
Quantitative Literate Graduates according to MAA Should be Able to:
1. Interpret mathematical models such as formulas, graphs, tables, and schematics, and draw inferences from them.
2. Represent mathematical information symbolically, visually, numerically, and verbally.
3. Use arithmetical, algebraic, geometric, and statistical methods to solve problems.
![Page 81: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/81.jpg)
81
Quantitative Literate Graduates according to MAA Should be Able to:
1. Interpret mathematical models such as formulas, graphs, tables, and schematics, and draw inferences from them.
2. Represent mathematical information symbolically, visually, numerically, and verbally.
3. Use arithmetical, algebraic, geometric, and statistical methods to solve problems.
![Page 82: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/82.jpg)
82
4. Estimate and check answers to mathematical problems in order to determine reasonableness, identify alternatives, and select optimal results.
5. Recognize that mathematical and statistical methods have limits. (http://www.ma.org/pubs/books/qrs.html)The Mathematics Association of America (Quantitative Reasoning for College Graduates: A Complement to the Standards, 1996). See also AMATYC draft, 2006.
![Page 83: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/83.jpg)
83
Ethics—Students should be able to…
Identify and analyze real world ethical problems or dilemmas, and identify those affected by the dilemma.
Describe and analyze the complexity and importance of choices that are available to the decision-makers concerned with this dilemma
![Page 84: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/84.jpg)
84
Ways to Articulate Outcomes
Adapt from professional organizations
Derive from mission of institution/program/department/service
Derive from students’ upper level work
![Page 85: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/85.jpg)
85
Derive from ethnographic process
Derive from exercise focused on listing one or two outcomes “you attend to”
![Page 86: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/86.jpg)
86
How well do your current outcome statements address the criteria for well articulated outcome statements (see attachment)
![Page 87: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/87.jpg)
87
Mapping Learning Outcomes
Curricular-Co-curricular maps:• Reveal how we translate outcomes into
educational practices offering students multiple and diverse opportunities to learn
• Help us to identify appropriate times to assess those outcomes
• Identify gaps in learning or opportunities to practice
![Page 88: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/88.jpg)
88
Help students understand our expectations of them
Place ownership of learning on students
Enable them to develop their own maps or learning chronologies
![Page 89: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/89.jpg)
89
Inventories of Practice
Provide in-depth information about how students learn along the continuum of their studies
Identify the range of educational practices and assessment experiences that contribute to learning outcomes (See handouts)
![Page 90: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/90.jpg)
90
How will you use maps and inventories?
Discuss with your team how you will go about the process of developing a curricular or curricular-co-curricular map and how you will label peoples’entries
Discuss with your team how you might use inventories
![Page 91: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/91.jpg)
91
“Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills. The tasks to which students are asked to respond on an assessment are not arbitrary.“
National Research Council. Knowing what students know: The science and design of educational assessment . Washington, D.C.: National Academy Press, 2001, p. 47.
2.Designing or Selecting Assessment Methods
![Page 92: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/92.jpg)
92
Assumptions UnderlyingTeaching
Actual Practices
Assumptions UnderlyingAssessment Tasks
Actual Tasks
![Page 93: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/93.jpg)
93
What Tasks Elicit Learning You Desire?
Tasks that require students to select among possible answers (multiple choice test)?
Tasks that require students to constructanswers (students’ problem-solving and thinking abilities)?
![Page 94: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/94.jpg)
94
Approaches to Learning
Surface Learning
Deep Learning
![Page 95: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/95.jpg)
95
When Do You Seek Evidence?
Formative—along the way?For example, to ascertain progress or development
Summative—at the end?For example, to ascertain mastery level of achievement
![Page 96: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/96.jpg)
96
Direct Methods
Focus on how students represent or demonstrate their learning (meaning making)
Align with students’ learning and assessment experiences
Align with curricular-and co-curricular design verified through mapping
![Page 97: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/97.jpg)
97
Invite collaboration in design (faculty, students, TAs, tutors)
![Page 98: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/98.jpg)
98
Standardized InstrumentsPsychometric approach—values quantitative methods of interpretation
History of validity and reliability
Quick and easy adoption and efficient scoring
One possible source of evidence of learning
![Page 99: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/99.jpg)
99
Do Not Usually Provide
Evidence of strategies, processes, ways of knowing, understanding, and behaving that students draw upon to represent learning
Evidence of complex and diverse ways in which humans construct and generate meaning
Highly useful results that relate to pedagogy, curricular design, sets of educational practices
![Page 100: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/100.jpg)
100
Authentic, Performance-based Methods
Focus on integrated learning
Directly align with students’ learning and previous assessment experiences
Provide opportunity for students to generate responses as opposed to selecting responses
Provide opportunity for students to reflect on their performance
![Page 101: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/101.jpg)
101
Do Not Provide
Immediate reliability and validity (unless there has been a history of use)
Usually do not provide easy scoring unless closed-ended questions are used.
![Page 102: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/102.jpg)
102
Some OptionsE-Portfolios
Capstone project (mid-point and end-point)
Performances, productions, creations
Visual representations (mind mapping, charting, graphing)
![Page 103: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/103.jpg)
103
Disciplinary or professional practices
Agreed upon embedded assignments
Selection of assignments students hand in
Simulations (virtual or live)
![Page 104: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/104.jpg)
104
Team-based or collaborative projects
Internships and service projects
Oral examinations
Critical incidents
![Page 105: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/105.jpg)
105
Externally or internally juried review of student projects
Externally reviewed internship
Performance on a case study/problem
Performance on case study accompanied with students’ analysis
![Page 106: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/106.jpg)
106
Performance on national licensure examinations
Locally developed tests
Standardized tests
Pre-and post-tests
![Page 107: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/107.jpg)
107
Learning Logs or Journals
Magic box—increasingly difficult problems over time
Videotaping
![Page 108: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/108.jpg)
108
Indirect Methods
Surveys (CCSSE, for example)
Interviews
Focus groups
![Page 109: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/109.jpg)
109
Other Sources of Information that Contribute to Your Inference Making
Grades
Participation rates in support services or in the co-curriculum
Course-taking patterns
![Page 110: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/110.jpg)
110
Achievement in Majors
Transcript analyses
Course content analyses
Initial placement
![Page 111: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/111.jpg)
111
Identify Methods to Assess Outcomes
Using the handout, identify both direct and indirect methods you might use to assess several of your outcomes. Determine the kinds of inferences you will be able to make based on each method.
![Page 112: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/112.jpg)
112
3. Developing Standards and Criteria of Judgment
A set of criteria that identifies the expected characteristics of a text and the levels of achievement along those characteristics. Scoring rubrics are criterion-referenced, providing a means to assess the multiple dimensions of student learning.
Are collaboratively designed based on how and what students learn (based on curricular-co-curricular coherence)
![Page 113: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/113.jpg)
113
Are aligned with ways in which students have received feedback (students’ learning histories)
Students use them to develop work and to understand how their work meets standards (can provide a running record of achievement).
![Page 114: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/114.jpg)
114
Raters use them to derive patterns of student achievement to identify strengths and weaknesses
![Page 115: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/115.jpg)
115
4. Collecting and Analyzing Results
Determine how and when to collect student responses
Determine who will analyze results (suitable for aggregation and disaggregation according to question you wish to answer)
![Page 116: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/116.jpg)
116
5. Interpreting Results
Seek patterns against criteria and cohorts
Build in institutional level and program level discourse
Tell the story that explains the results—triangulate with other data
Determine what you wish to change, revise, or how you want to innovate
![Page 117: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/117.jpg)
117
6. Implementing Changes and Re-Entering the Assessment Cycle
Implement agreed upon changes
Re-assess to determine efficacy of changes
Focus on collective effort—what we do and how we do it
![Page 118: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/118.jpg)
118
Examples of Changes:
Increased attention to weaving learning experiences across the institution to improve students’ achievement of outcomes
Changes in advising based on assessment results
Closer monitoring of student achievement--tracking
![Page 119: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/119.jpg)
119
Faculty and staff development to learn how to integrate experiences that contribute to improved student learning
Changes in pedagogy and curricular and co-curricular design
Development of modules to assist learning; use of technology; self-paced learning, supplemental learning
![Page 120: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/120.jpg)
120
Mission/Purposes
Learning Outcomes
How well dowe achieve
our outcomes?
Gather Evidence
Interpret Evidence
Enhance teaching/ learning;
inform institutional decision-
making, planning,budgeting
![Page 121: Principles of and Practices in Assessing Student Learningslis.simmons.edu/coa/docs/Simmons.pdf · Students’ Learning Histories/Styles. 3 Integrated Learning…. Cognitive Psychomotor](https://reader034.vdocuments.us/reader034/viewer/2022051917/60094965270126637f014df2/html5/thumbnails/121.jpg)
121
List of AttachmentsQuestions to Ground Discussion of Teaching, Learning, and AssessmentChecklist for Outcome StatementsDissemination of outcome statements2 curricular mapsInventories of practiceIdentification of assessment methods Four examples of scoring rubricsScoring rubricFormat for annual reporting