assessment of academic advising 2011 nacada summer institute charlie nutt the presenters acknowledge...
TRANSCRIPT
Assessmentof
Academic Advising2011 NACADA Summer Institute
Charlie Nutt
The presenters acknowledge and appreciate the contributions of NACADA colleagues Susan
Campbell, Rich Robbins, Mike Kirk-Kuwaye, Lynn Higa, Tom Grites, and Eric White in preparation of
materials for this presentation
NACADA Executive OfficeKansas State University
2323 Anderson Ave, Suite 225Manhattan, KS 66502-2912
Phone: (785) 532-5717 Fax: (785) 532-7732
e-mail: [email protected]
© 2011 National Academic Advising Association
The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association.
The Global Community for Academic Advising
• Review of assessment cycle
• Identifying stakeholders
• Identifying student learning outcomes
• Mapping the students’ learning experiences
• Identifying measures
• Using the CAS standards
• Assessment and research
Overview
The Global Community for Academic Advising
The Assessment Cycle (Maki, 2002, 2004)
Gather
Evidence
Interpret
Evidence
Identify
Outcomes
Implement Change
Mission/PurposesEducationalObjectives
The Global Community for Academic Advising
The Assessment Flowchart (adapted from Darling, 2005)
Student Learning OutcomesCognitive, Psychomotor, AffectiveProcess/Delivery Outcomes
Mapping the ExperienceWhat experiences?When or by when?
Gathering EvidenceWhen gathered?Where gathered?
How often gathered?From whom gathered?
How gathered?Minimum performance criteria for success?
ValuesVision
MissionGoals
Programmatic Outcomes
Sharing and Acting Upon the ResultsInterpret how results inform practice
How and with whom to share interpretationFollow up on implemented changes
Start the process all over again!
The Global Community for Academic Advising
Mapping of Outcomes
The Assessment Matrix/Table
Institutional Mission Statement
Local Mission Statement
Specific Goal or Objective
Specific Process/Delivery Outcome or Student Learning Outcome
Where Outcome Occurs
When or By When Outcome Occurs
Outcome Measure
Minimum Performance Criteria for Success
Data Instruments
Action(s) Based on Outcome Data
The Global Community for Academic Advising
Identifying Key Stakeholders:Who Should Be Involved?
• Colleagues, faculty, administrators, institutional researchers, staff, students, and institutional community
• Decide how the assessment team will interact, overlap, and/or support other institutional efforts
• Encourage stakeholders on and off campus
• Continuous communication and feedback is a must!
The Global Community for Academic Advising
• Building of a shared trust
• Building of a shared motivation
• Building of a shared language
• Building of support for academic advising institutionally-wide
• The result is a shared ownership and belief in the process
Benefits of a Collective and Collaborative Process
The Global Community for Academic Advising
Exercise 1:Identify Your Stakeholders
The Global Community for Academic Advising
Key Terms• Process/Delivery Outcomes – Articulate the expectations
for how academic advising is delivered and what information should be delivered through the academic advising experience
• Student Learning Outcomes – Articulate what students are expected to know, do, and appreciate as a result of involvement in the academic advising experience
• Mapping – The process of determining when, where and how the outcomes for advising will be accomplished over the students’ academic career and beyond
The Global Community for Academic Advising
Process/Delivery Outcomes• Typically what is evaluated via student
satisfaction surveys• e.g., “the advisor provided accurate information”• e.g., “the setting for the advising session was
appropriate”• etc.
• Do not assess student learning, but evaluate what processes occurred during the advising interaction
The Global Community for Academic Advising
Focus onStudent Learning Outcomes
The Global Community for Academic Advising
It’s All About Student LearningIt’s All About Student Learning
Understanding what, how and when students learn through academic advising
The Global Community for Academic Advising
Advising is Teaching
The Global Community for Academic Advising
Advisor as Teacher
• facilitator of communication
• coordinator of learning experiences
• referral agent who connects students with all of the institution’s resources and co-curricular opportunities that can help them be successful
The Global Community for Academic Advising
Teaching
Knowledge of subject matter
Preparation - planning and organization of course material
Advising
Knowledge of institutional policies, procedures, programs, resources
Preparing for advising meetings
The Global Community for Academic Advising
Teaching
Engaging students in participation of their learning
Giving students feedback on their progress
Helping students learn to analyze and problem solve
Advising
Guiding students to be self-directed and autonomous
Working with student to regularly evaluate the student’s goals and progress
Assisting students in decision-making skills
The Global Community for Academic Advising
Teaching
Clarity of presentation of subject matter
Establishment of dialogue with students in the classroom
Demonstration of excellent listening skills
Advising
Sharing information in a clear manner
Leading students to question and interact with the advisor
Listening to what advisees are saying verbally and non-verbally
The Global Community for Academic Advising
Teaching
Assigning out-of class work
Working from a developmental perspective
Providing a learner-centered environment
Advising
Assigning out-of-session tasks
Working from a developmental perspective
Providing a learner-centered environment
The Global Community for Academic Advising
Teaching
Identification and communication of student learning outcomes for the curriculum or course
Use of a course syllabus
Advising
Identification and communication of student learning outcomes for the advising process/interaction
Use of an advising syllabus
The Global Community for Academic Advising
Focus on the Learner/Advisee
As a result of academic advising, what do we want students to demonstrate that they…
• Know (cognitive student learning outcome)
• Are able to do (behavioral student learning outcome)
• Value and appreciate (affective student learning outcome)
The Global Community for Academic Advising
Mapping of Outcomes
The Assessment Matrix/Table
Institutional Mission Statement
Local Mission Statement
Specific Goal or Objective
Specific Process/Delivery Outcome or Student Learning Outcome
Where Outcome Occurs
When or By When Outcome Occurs
Outcome Measure
Minimum Performance Criteria for Success
Data Instruments
Action(s) Based on Outcome Data
The Global Community for Academic Advising
Cognitive SLOs
What do we want students to KNOW as a result of participating in academic advising?
Know general education requirements
Know about academic support services
Know how to use the student information system to register
Know how to use the catalog
Etc…….
The Global Community for Academic Advising
Exercise 2:Identify Three Things You Want Students to Know as a Result of
Academic Advising
The Global Community for Academic Advising
Tips on Developing SLOs
• avoid compound SLOs, e.g., “students understand and value the purpose of a liberal arts education,” “students understand and utilize their degree progress reports”
• as you are developing each SLO, think about where and when the opportunities for student learning may occur for that desired SLO
• as you are developing each SLO, think about how you might measure whether or not the SLO has been achieved
The Global Community for Academic Advising
Behavioral/Psychomotor SLOs
What do we want students toDo as a result of participating in academic advising?
Generate their degree audit
Make advising appointments
Keep advising appointments
Ask for help
Access course descriptions and degree requirements using the online catalog
Etc….
The Global Community for Academic Advising
Exercise 3:Identify Three Things You Want Students to Do as a Result of
Academic Advising
The Global Community for Academic Advising
Affective SLOs
What do we want students to Value or Appreciate
as a result of participating in academic advising?
Value/Appreciate general education
Value/Appreciate the advising relationship
Value/Appreciate the process of learning
Etc….
The Global Community for Academic Advising
Exercise 4:Identify Three Things You Want
Students to Value or Appreciate as a Result of Academic Advising
The Global Community for Academic Advising
Clear and measurable student learning outcomes for academic advising may also be derived from
• the NACADA Concept of Academic Advising
• the NACADA Core Values
• the CAS Standards for Advising
All are included in your Summer Institute Guide
The Global Community for Academic Advising
Using the CAS Standards in Program Assessment
The Global Community for Academic Advising
The Council for the Advancement of Standards in Higher Education (CAS)
has developed roughlythree dozen different
“self assessment” guidesavailable in all areas of
Student Services. They rangefrom assessment of advisingto housing to help desks…
The Global Community for Academic Advising
The CAS Standards for Academic Advising can easily be utilized as outcome statements for your advising program
The Global Community for Academic Advising
It is not necessary to attempt to use all of the CAS Standards as your outcome statements…
…for those you may choose to use, be sure to develop related student learning outcomes as well as process/delivery outcomes
The Global Community for Academic Advising
Mapping the Learning Experience• What should be learned: e.g., student will know the
components of the institution’s General Education requirements
• Where it should be learned: e.g., orientation workshops, advising sessions, personal reading of catalog or curriculum guide
• When or by when it should be learned: e.g., prior to first year (orientation); by end of first semester; by end of first year; before declaring a major, etc.
The Global Community for Academic Advising
Mapping of Outcomes
The Assessment Matrix/Table
Institutional Mission Statement
Local Mission Statement
Specific Goal or Objective
Specific Process/Delivery Outcome or Student Learning Outcome
Where Outcome Occurs
When or By When Outcome Occurs
Outcome Measure
Minimum Performance Criteria for Success
Data Instruments
Action(s) Based on Outcome Data
The Global Community for Academic Advising
Exercise 5:Mapping Your Desired Outcomes:
Identifying Where/When the Process/Delivery Occurs
ORWhere/When the Student Learning
Opportunities Occur
The Global Community for Academic Advising
Once the desired process/delivery and student learning outcomes have been identified, as well as when and where they will occur, the next step is to determine who or what will be measured and how the data will be gathered…
…using multiple measures of varying types
The Global Community for Academic Advising
Mapping of Outcomes
The Assessment Matrix/Table
Institutional Mission Statement
Local Mission Statement
Specific Goal or Objective
Specific Process/Delivery Outcome or Student Learning Outcome
Where Outcome Occurs
When or By When Outcome Occurs
Outcome Measure
Minimum Performance Criteria for Success
Data Instruments
Action(s) Based on Outcome Data
The Global Community for Academic Advising
Types of Measurement and Data
• Qualitative
• Quantitative
• Direct
• Indirect
• Multiple measures!!!
The Global Community for Academic Advising
Measures can (and should) include existing institutional data
• Information from Institutional Research, Admissions, Registrar, etc. can provide tracking data, GPAs, retention rates, and other information you can utilize as assessment data
– this can be a source of some of the multiple measures utilized (in addition to formal instruments, satisfaction inventories, and others)
The Global Community for Academic Advising
For both process/delivery and student learning outcomes, you need to identify the minimum criteria for success of the outcome measure, e. g.,• number of students exhibiting a specific learning performance• percentage of students exhibiting a specific learning performance• advisor rating of student performance• student rating of specific aspect of advising process• advisor rating of specific aspect of advising process• etc.
The Global Community for Academic Advising
Exercise 6:Identifying Multiple Measures
Select one process/delivery or one student learning outcome, and identify three ways to measure whether or not that specific outcome has been met
The Global Community for Academic Advising
So I Have The Data – Now What?
• Sharing and Acting Upon Results– Interpret results regarding how they inform the
advising process/delivery, student learning, and decision-making
– Determine with whom and how the results are reported
– Decide how you will implement changes based on the results
– Start assessment cycle again…
The Global Community for Academic Advising
Determine How and With Whom Results Are Shared
• Administration: President, Provost, various committees
– via annual report, strategic plan, white paper, Web sites, etc.
• Faculty: all faculty, curricular committees, faculty advisors
– via performance reviews, annual reports, strategic plans, Web sites, etc.
• Students: all students, student advisees, student senate, student groups
– via newsletters, annual reports, Web sites, etc.
• Budgeting entities
– via annual reports, budget requests, Web sites, etc.
• Accreditors
– via self-studies, accreditation reports, Web sites, etc.
The Global Community for Academic Advising
• Revise pedagogy or curriculum
• Develop/revise advisor training programs
• Design more effective programming– advising, orientation, mentoring, etc.
• Increase out-of-class learning opportunities
• Shape institutional decision making– planning, resource allocation
• Other…
Interpret How Results Will Inform Decision Making
The Global Community for Academic Advising
Decide How You WillFollow-up on Implemented
Changes• Timetable to implement changes
– implement all or specific components on a schedule
• Assessment of implemented changes– repeat assessment cycle again
• Continuous assessment– assessment is on-going
The Global Community for Academic Advising
Final Exercise:
Complete One of Your Outcome Matrices
The Global Community for Academic Advising
THANK YOU!
2011 NACADA Summer Institute
Assessment of Academic AdvisingDr. Rich Robbins
Bucknell University
The Global Community for Academic Advising