ysu office of assessment october 9 & 10, 2012 assessment plans 1
TRANSCRIPT
YSU OFFICE OF ASSESSMENTOCTOBER 9 & 10 , 2012
Assessment Plans1
Goals of Workshop
Current assessment contextAssessment plan evaluation processOverview of new Higher Learning
Commission CriteriaReview key items on the assessment plan
templateWalk through new online reporting form
2
Assessment Context3
Assessment Context
Accreditation Context Last year participating in the Higher Learning
Commission’s (HLC) Academy for the Assessment of Student Learning
New HLC Criteria effective January 1, 2013
Quality of Assessment Processes Strengths
Excellent participation 2011-12 -- LO review (100%), curriculum maps (95%)
Quality/participation in reporting process improving over time – but 100% needed
Challenges Continuous collection of data (no years off!) Stepping back for the big picture—what is the impact on
learning?
4
Academic Plan and Report Quality5
2009-2010 2010-20110%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Exemplary43
Exemplary55
Proficient47
Proficient67
Progressing 10
Progressing 8
Inadequate 3
Inadequate 8No Report 29
No Report 8 93% Participa-
tion
78% Participa-
tion
68% of
high quality
84% of
high qual-ity
Assessment Process
Review Process Focused rubric – emphasis on assessment priorities Team of 2 reviewers:
Assessment Council Member Assessment volunteer – good service opportunity
Final review by Director Feedback via email and/or meeting
Strengths of plan or report Suggestions for next year Revisions, if needed
Quality levels Exemplary, proficient – high quality Progressing – developing expertise Inadequate – request revision
6
Assessment Process, cont.
Assessment Reporting Priorities Focus on use of data Reflect on changes and impact on learning Continuous data collection (i.e. every year!) Streamline reporting – focus on process vitality, not
form
What’s New This Year Online reporting Fewer questions Focused rubrics on priority areas
Future Goals Longer reporting cycle Possible integration with program review
7
Keeping a Student Learning Archive
Accreditation Archives Departments need to keep a student learning archive for
10 years Plan and report submissions kept in OOA for 10 years
8
Archive Examples• Summaries of data on student learning• Representative student work examples at
different performance levels• Student work evaluation criteria, e.g., rubrics• Assessment plans and reports• Newsletters• Website screenshots• Meeting minutes on assessment
HLC New Criteria for Accreditation
New Criteria at: www.higherlearningcommission.org
Guiding Values, includes:1. Focus on student learning4. Culture of continuous improvement5. Evidence-based institutional learning and self-
presentation9. Mission-centered evaluation
The Five Criteria1. Mission2. Integrity: Ethical and Responsible Conduct3. Teaching and Learning: Quality, Resources, and
Support4. Teaching and Learning: Evaluation and
Improvement5. Resources, Planning, and Institutional Effectiveness
9
New HLC Criteria Relevant to Practice
4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning.
1. The institution has clearly stated goals for student learning and effective processes for assessment of student learning and achievement of learning goals.2. The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.3. The institution uses the information gained from assessment to improve student learning.4. The institution’s processes and methodologies to assess student learning reflect good practice, including the substantial participation of faculty and other instructional staff members.
10
New HLC Criteria Relevant to Practice, cont.
3A2: The institution articulates and differentiates learning goals for its undergraduate, graduate, post-baccalaureate, post-graduate, and certificate programs.
3A3: The institution’s program quality and learning goals are consistent across all modes of delivery and all locations (on the main campus, at additional locations, by distance delivery, as dual credit, through contractual or consortial arrangements, or any other modality).
5C2: The institution links its processes for assessment of student learning, evaluation of operations, planning, and budgeting
11
Upcoming Assessment Workshops
Developing an Assessment PlanWednesday, October 10th, 1-2 pm
Completing the Assessment Report Tuesday, October 9th, 1-2 pmWednesday, October 10th, 10-11 am
Note: workshops/forms overlap
12
Assessment Plans Assessment Reports
Plans and methods to cover all SLOs in 3-4 year cycle
Criteria for at least the 1st year
Plans for sharing results with major stakeholders
Data from previous year Two methods and data
summary for two SLOs Analysis of student
learning for strengths and challenges
Action steps based on data Sharing of data and results
Assessment Plans vs. Reports13
Both Plans and Reports:• Engagement of faculty• Impact on learning from previous action steps
EVERYTHING TO KNOW BUT DIDN’T WANT TO ASK
Completing the Assessment Plan Form
14
Assessment Templates
Section 1: Identifying and Contact
Information Section 2: Outside Accreditation Section 3: Assessment and Evaluation of
Student Learning Outcomes Section 4: Use of Data
15
Sections 1 &2: Identifying/Accreditation Information
16
Sections 1 &2: Identifying/Accreditation Information
Degree Program Degree Level Program Tracks
Political Science BA GeneralForeign AffairsPublic Management
Teacher Education BSEd Early ChildhoodMiddle ChildhoodSecondaryCareer/TechnicalMulti-age
Biology BA n/a
Biology BS n/a
Medical Coding Certificate n/a
17
Save time and fill out only online!Note the difference between degree, program,
and track:
Section 3: Assessment and Evaluation of Student Learning Outcomes
18
Section 3: Learning Outcomes
1. List all the student learning outcomes (aka SLOs) your department has determined for this program
Reminder: If you are developing or revising SLOs, they should be short, measurable, and focused on one skill or knowledge set
Example:
19
Good Example Weak Example
Students will explain the difference between major types of rocks.
Students will compare and contrast the characteristics of the major types of rocks in order to differentiate amongst categories.
Tips on Writing Learning Outcomes
Stronger Learning Outcomes:
Include action verbs Reflect higher-order thinking/skills Are measurable Differentiate the unique aspects of
student learning in each degree program
Are simple and clear (and not too generic)
Include a reasonable number of learning outcomes
Articulate the level of learning expected (see Bloom’s taxonomy verb reference sheet at: http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf)
Weaker Learning Outcomes:
Describe program outcomes rather than student learning outcomes
Are too vast/complex/too wordy Include multiple outcomes in one
learning outcome (watch for “and) Are too basic Are not measurable Describe activities or work products
rather than describe what students will learn
Do not differentiate between different levels (associate, bachelor, master, certificate)
Can be reduced in number
20
Section 3: SLO Cycle
2. What is the timeline for assessing all the student learning outcomes over a multi-year cycle?
Typically a 3-4 year cycle If you have a great number of SLOs (e.g., 10+):
If because of accrediting bodies, we can work with you to find best cycle length
We can work with your department to reduce number through consultation
If you have an odd number of SLOs, the last year may have only one SLO
21
B.A. in Physics B.S. in Physics
1. Students will learn to model physical systems and interpret experimental and theoretical results.
2. Students will learn how to measure the physical properties of systems using a variety of test equipment and defend the results of their measurements using the associated accuracy and precision of these measurements.
3. Students will learn to apply the concepts of Classical Physics, Modern Physics, Thermodynamics, and Electrostatics to solve problems and predict numerical results.
1. Students will learn to model physical systems and interpret experimental and theoretical results.
2. Students will learn how to measure the physical properties of systems using a variety of test equipment and defend the results of their measurements using the associated accuracy and precision of these measurements.
3. Students will learn to apply the concepts of Classical Physics, Modern Physics, Thermodynamics, and Electrostatics to solve problems and predict numerical results.
4. In addition to the learning outcomes for the B.A. Program in Physics, students of the B.S. program in Physics will further learn to apply the concepts of Electrodynamics and Quantum Mechanics to solve problems and predict numerical results.
Section 3: When SLOs Overlap22
Section 3: Sample SLO Cycle
Year of Cycle
B.A in Physics B.S. in Physics
Year One
1. Students will learn to model physical systems and interpret experimental and theoretical results.
2. Students will learn how to measure the physical properties of systems using a variety of test equipment and defend the results of their measurements using the associated accuracy and precision of these measurements.
1. Students will learn to model physical systems and interpret experimental and theoretical results.
2. Students will learn how to measure the physical properties of systems using a variety of test equipment and defend the results of their measurements using the associated accuracy and precision of these measurements.
Year Two
1. Students will learn to apply the concepts of Classical Physics, Modern Physics, Thermodynamics, and Electrostatics to solve problems and predict numerical results.
1. Students will learn to apply the concepts of Classical Physics, Modern Physics, Thermodynamics, and Electrostatics to solve problems and predict numerical results.
2. In addition to the learning outcomes for the B.A. Program in Physics, students of the B.S. program in Physics will further learn to apply the concepts of Electrodynamics and Quantum Mechanics to solve problems and predict numerical results.
23
Section 3: Assessment methods
3. What methods will you use to assess student performance? Describe two methods for each student learning outcome.
Methods should include: where it will be administered (e.g., capstone) Performance criteria (Y1) or plan for development (Y2
and on) Same method can span multiple SLOs (e.g., don’t
need 10 methods for 5 SLOs) Map methods to timeline:
Example: LO 1 (writing, Y1): capstone project assessed with rubric (attachment A); research methods final paper assessed with rubric (attachment B)
Must use one direct measure per SLO
24
Assessment Method Definitions25
Term Definition Examples
Direct Measure Direct measures provide for the direct examination or observation of [staff, faculty, student] knowledge or skills against measurable learning outcomes
In-class/embedded assignments, oral presentations, performance appraisals, internship supervisors’ evaluations, behavioral observations, etc.
Indirect Measure Indirect measures of learning that ascertain the opinion or self‐report of the extent or value of learning experiences
Written surveys, exit and other interviews, archival records, focus groups, etc.
Performance Criteria
Specific, measurable statements identifying the performance(s) required to meet the outcome; confirmable through evidence.
Standards, rubrics*, specifications, outcomes, metrics, objectives,
*For an example of rubrics, see the AAC&U’s VALUE Rubric Project: http://www.aacu.org/value/rubrics
Section 4- Use of Data26
Section 4: Sharing Results
4. How will you share the results of the data discussed in section three with your students, your college, and other stakeholders?
Include both internal and external stakeholders
Examples: Students’ review of aggregate data College wide assessment committees Discuss in advisory group meeting Share with foundational subject departments (e.g.,
Engineering Dept. shares findings with Mathematics Dept.)
27
Section 4: SLOs and Curricular Maps
5. How did the assessment activities in 2011-12 (i.e., reviewing learning outcomes and completing curriculum maps) impact your program?No correct answer; just experience of
departmentsQuestions to consider:
Did you streamline learning outcomes? Did they foster faculty discussion? Were gaps in learning or assessment practices
uncovered? Did you find more efficient ways to collect data?
28
Section 4: Impact on Learning
6. In the past several years (e.g., 2008-11), you have analyzed data and identified action steps for learning outcomes. Considering action steps from previous years, what has been an impact on student learning as a result of (one of) those action steps?Refer to past assessment reports (2008-11)Focus on how action step impacted student
learningDo not need specific supporting data, just
professional judgment at this stage
29
Section 4: Engaging Faculty
7. How is your department working to engage all faculty in the assessment process?
All department faculty should be meeting at least once per year to discuss assessment results and decide on action steps
Collective responsibilityNot just one person’s job
30
Section 4 Questions
8. Optional: Is there anything else you would like to share and is there any particular area that you want feedback on that would assist you?
Is there more to “the story” than reflected in the plan?
Something the Office of Assessment or Assessment Council can assist you with?
Examples: Involving students in review of data Increasing faculty participation
31
web.ysu.edu/assessment/templates32
Template
Submission link
web.ysu.edu/assessment/templates
New Online Reporting Form
Online Assessment Plan Submission Form: http://www.jotformpro.com/ysuassessment/2012acadplan
Note: if you have new or revised undergraduate learning outcomes, they should also be sent to Jean Engle at [email protected].
34
TO VIEW ASSESSMENT PLAN OR REPORT FORMS AND SCORING RUBRICS, AS WELL AS
THIS PRESENTATION, VISIT:HTTP: / /WEB.YSU.EDU/ASSESSMENT/TEMPLATES
CONTACT INFO:HILLARY FUHRMAN, X2453
OFFICE OF ASSESSMENT, [email protected]
Thank you for your participation!
35