ncsd denver presentation
DESCRIPTION
TRANSCRIPT
NCSD National Conference Denver, CO Oct. 22-24, 2009
Elise Davis-McFarland, Vice President
for Student Services
Sharon Kearns, M.Ed., Director of Testing
We Are All Accountable: Student Learning Outcomes for Student Services
Student Affairs professionals have the responsibility for ensuring that
institutions of higher education become true learning communities committed
to providing transformative educational experiences for all students.
Learning Reconsidered – ACPA, NASPA
The institution provides student support programs, services, and activities consistent
with its mission that promote student learning and enhance the development of its
students. (Student Support Services)
Norht North Central Association Commission on Accreditation and School Improvement
Learning goals are designed to press all students to excellence and focus on enhancing the intellectual, personal, physical, social, and career development of students.
PS 1 The learning goals are challenging, are appropriate in terms of rigor and diversity, meet student needs, and are consistent with the stated goals of the program.
PS 2 Objectives are: identified for the specific subject areas and the various programs such as the activity programs, and student personnel services;
Different Outcomes
Student service Satisfaction, needs, program quality
Student development Values, attitudes, beliefs, psychosocial
development Learning
Knowledge and skills
Why Student Learning Outcomes?
The business of colleges is teaching and learning. If you’re not a player, you’re just sitting on the sidelines.
.
The actual process ismuchlike….
First ……you must lay the groundwork…
You need a frame of reference
1. Learning Reconsidered - ACPA/NASPA Cognitive complexity Knowledge, acquisition, integration,
application Humanitarianism Civic engagement Interpersonal and intrapersonal competence Practical competence Persistence and academic achievement
You need a frame of reference
2. Principles of Good Practice for Student Affairs- ACPA/NASPA
Active learning Values & ethical standards High expectations for student learning Systematic inquiry Effective resource use Educational partnerships Community
Assessment should show that Student Services
Plays a significant role in student learning
Makes a difference in academic success
Promotes retention and persistence
Enhances cognitive development
Enhances affective development
Brainstorming & Consensus
How about?How about…?
Hmm..
Hmm..
What if we…?
What if a student asks…
What should I be able to do as a result of my interaction with the______ office in Student Services?
What will I learn from meetings with my financial aid officer?
What would you say?What do you expect students to be able to
do/know as a result of their interactions or participation
What should students gain from Student Services?
Essential life skills Personal responsibility Perseverance Decision making/critical thinking Financial responsibility Technology use
What should students gain from Student Services?
Essential social skills Effective communication Interpersonal skills Civic mindedness
Ethical behavior Adherence to legal and ethical
requirements
Davis-McFarland, E.
Planning Pyramid
Student Affairs Divisional Plan
College Strategic Plan
Institutional Mission
Adapted from Bentrim-Tapio, 2008
Mission Pyramid
Unit Mission (EOP, EM, SD)
Division
Mission
Institutional Mission
Mission Statements: Before As part of the Student Services Division of Trident Technical College, the staff members of the Financial Aid/Veterans Assistance Office are committed…….
Trident Technical College Registrar’s office mission is to provide the best possible service…..
The Educational Talent Search program will sustain a supportive and……
The mission of Career and Employment Services is to prepare students for the transition from……
Student Activities’ mission is to complement the academic program
and enhance the students’ college….
What’s missing?
Davis-McFarland, E.
BEFORE Mission Statement Example
Orientation ServicesOur Purpose...
The mission of Orientation Services at Trident Technical College is to provide a welcoming environment in which new students and their families obtain information necessary for a successful transition into college life. Through our orientation program, we encourage campus involvement, academic success, and student retention. Our aim is to integrate new students and families into campus life while easing the transitions and stresses associated with starting college. In order to achieve these objectives, students and families who attend an
orientation will…..View a video to familiarize them with the campus environment and physical
facilitiesExplore Trident Technical College’s Website Talk with an Orientation Leader to
Verify the student’s major Learn about important processes, like registration Be assigned an academic advisor Receive important printed materials about the CollegeFind out about student services and organizations Have questions answeredTake a walking tour of the campus, if desired Register for courses during the registration period in the Orientation Services
Centers if they are non-degree students
AFTER
Orientation Services
In support of the Student Services Divisionmission of encouraging student success, Orientation Services provides acomprehensive orientation program toencourage campus involvement, academicsuccess, and student retention.
A good learning outcome
Adapted from Bentrim-Tapio, 2008
Indicates what the student should know or be able to do
Measurable
Related to experiences you provide
Realistic
Aligns with vision, mission and values
Adapted from Bentrim-Tapio, 2008
Learning Outcome Statements
Specify what students will be able to do, know, and/or demonstrate as a result of participating in or completing an activity/service/program.
Should be expressed as knowledge, skills, attitudes or behaviors.
We always use verbs
Bloom’s Taxonomy Describe Demonstrate Develop Identify
Difficult to Measure Appreciate Aware of Familiar with
The wording….As a
result of
BEHAVIOR• Meeting• Participa
ting• Working
with ACTIVITY• Program• Service• Activity
Who• Students• Athletes• Officers
Active Verb• Demonst
rate• Identify• Explain
EXPECTATION • 50%
improvement• Average
performance• Pre-test post-
test improvement
INTENTION Mock interview• Time
management
Adapted from Bentrim-Tapio
Learning Outcome Examples
As a result of students attending financial aid workshops and receiving financial aid education, the student default rate will not exceed 5%.
As a result of instruction from the test proctor, students will manage their time in order to complete their tests within the allotted time.
As a result of attending officer training sessions and holding office, organization officers will demonstrate improvement in their leadership skills from the beginning of their tenure through completion.
The Assessment Cycle
Groundwork
Consensus
Form SLO Committee Mission
ObjectivesGoals
Outcomes
Revise mission statements
Develop surveys,forms, rubrics, etc.
Collect data(develop schedule)
Analyze & reportfindings
Determine findingsuse to enhance & improve
Satisfaction vs. Learning is much like the relationship between Reliability and Validity.
Reliability means nothing when the problem is validity, satisfaction isn’t too important if students don’t learn.
Satisfaction surveys
Bresciani, 2002
don’t help you understand all of the contributions of your programs & services
don’t tell you how your programs and services contribute to student development & learning
don’t tell you everything you need to know about programs & service improvements
Adapted from Bresciani, 2002
Satisfaction
85% of students will agree or strongly agree that mock interviews helped them feel more confident in their interviewing skills.
Sir, we’ve had a paradigm shift since you left.
Learning
85% of students who attend the How to Interview workshop will score average or above average on a mock interview evaluation matrix.
Bresciani, 2002
Student Services Learning Outcomes
Shows Academics that Student Services makes significant contributions to learning
Forms the basis for collaborations that
Promote measurable student learning Promote student engagement and socio-academic
integration Provides models for indirect learning
Examples
Davis-McFarland, E.
2006-2008 Student Services Program Assessment
Department: Orientation Services Director: Brandon Branham Division: Student Services Dean: John Jamrogowicz
Assessment Summary
Key Findings Plan for Improvement Implementation The assessment provided information on student learning outcomes in the areas of advising session preparation, TTC website knowledge, and information retention. Areas of strength for the department are in advising session preparation. Information retention and website knowledge require follow-up.
The department will work on improving students’ website knowledge and on gathering accurate results for information retention in orientation sessions.
Orientation Leaders are trained to instruct students on the importance of website knowledge.
The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services. The Enrollment Management Unit provides an organizational framework and leadership that facilitates cohesion and coordination between and among its component admissions, financial aid, orientation, registrar, and testing services. We intend to be reliable and effective partners in helping our faculty and fellow staff members accomplish their work with our students from entry to exit. We will accomplish our work in a manner that causes other state and regional two-year colleges to recognize us as an enrollment management unit worthy of benchmarking. Orientation Services provides a comprehensive orientation program that assists students in achieving both their academic and developmental goals through a comprehensive orientation program. We encourage campus involvement, academic success, and student retention. Our aim is to integrate new students into campus life while easing the transitions and stresses associated with starting college.
Davis-McFarland, E.
STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEW Department: Orientation Services
Division: Student Services Year: 2006- 2007
A. Student Learning Outcomes
B. Where Outcomes are Assessed
C. Methods for Outcomes Assessment
D. Expected Level of Program Performance
E. Data Collection F. Results G. Plan For Improvement
What should students who interact with your service area be able to do? (1 outcome per block)
Where do you see evidence that the student can do these things?
How does your program evaluate student/graduate skills/abilities?
What is the expected level of student performance for the program?
How and when will you collect the data needed to evaluate the performance of the program?
What are the results of the evaluation?
How will you use this information to improve the program
Students demonstrate an ability to adequately prepare for an academic advising session.
Created new student matrix to be used by Orientation Leaders for pre/post-test data collection regarding advisor visits.
Matrix administered to students prior to and after individual orientation sessions. (pre/post-test)
> 80% of the students will correctly answer post-test questions. Post-test scores will improve ≥20%.
Review of pre/post-test scores from student matrix administered October 2007 - November 2007.
100% of students correctly answered post-test questions
Actions: 1. During orientation sessions,
emphasis will be placed more heavily upon the importance of preparing adequately for an advising session.
Eliminated student matrix for next cycle. Will use original matrix but will speak directly to Advisors and not to students. Percentages were met or exceeded for Fall 2006. No change is indicated.
Advisor Feedback
Advisee evaluation matrix
≥60% of students have acceptable evaluations
Review of advisee evaluation matrices completed by 10 faculty advisors for Priority Registration for Fall 2006 term.
75% of advisees called or e-mailed to set up an appointment. 69% of advisees brought necessary documents to their advising sessions. 95% of advisees communicated course requirements or preferences. 66% of advisees arrived on time for advising sessions.
Davis-McFarland, E.
PROGRAM VITAL STATISTICS
Indicator Trend Analysis
Orientation Numbers by Calendar Year
Mean of 2274 and standard deviation of 211 Trend analysis indicates a very strong decreasing linear trend in the number of Orientations per year. The reported p-value is 0.003
0
500
1000
1500
2000
2500
3000
2003 2004 2005 2006 2007
25622416
2204 2149 2039
Davis-McFarland, E.
2007-2008 Student Services Program Assessment
Department: Financial Aid Director: Ellen Green Division: Student Services Dean: John Jamrogowicz Number of awards FY 2008: 37,189 Total amount awarded for FY 2008: $43,522,825 Default Rate: 5.5%
Assessment Summary
Key Findings Plan for Improvement Implementation The assessment provided information on student learning outcomes in the areas of student’s financial responsibility and demonstrates an understanding of the importance of academic progress. Areas of strength are in the financial responsibility of students who are affected by Return to Title IV (R2T4). The data indicates the need for additional resources regarding loan eligibility and borrowing responsibilities.
Investigate the hiring of a loan default coordinator to work with data from the National Student Loan Data System (NSLDS) to educate students on their financial rights and responsibilities regarding loan borrowing. Review the data to determine if the expected level of performance should be adjusted in light of the expanded loan limits from the Department of Education.
Requested and received approval for the position and hired a Loan Default Coordinator. The coordinator is using resources from NSLDS, ELM, and national data to provide students with information regarding Financial Aid rights and responsibilities in addition to how to borrow responsibly. The department is offering workshops both in classes and during day/evening hours to explain both the financial responsibilities of students and the importance of academic progress for continued eligibility for financial aid.
The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services.
The Enrollment Management Unit provides an organizational framework and leadership that facilitates cohesion and coordination between and among its component admissions, financial aid, orientation, registrar, and testing services. We intend to be reliable and effective partners in helping our faculty and fellow staff members accomplish their work with our students from entry to exit. We will accomplish our work in a manner that causes other state and regional two-year colleges to recognize us as an enrollment management unit worthy of benchmarking.
Davis-McFarland, E.
STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEW Department: Financial Aid
Division: Student Services Year: 2007 - 2008
A. Student Learning Outcomes
B. Where Outcomes are Assessed
C. Methods for Outcomes Assessment
D. Expected Level of Program Performance
E. Data Collection F. Results G. Plan For Improvement
What should students who interact with your service area be able to do?
Where do you see evidence that the student can do these things?
How does your program evaluate student/graduate skills/abilities?
What is the expected level of student performance for the program?
How and when will you collect the data needed to evaluate the performance of the program?
What are the results of the evaluation?
How will you use this information to improve the program
Students receiving financial aid demonstrate financial responsibility.
Default Rates Review default rates The loan default rate for the college is ≤5%.
Review of Department of Education default rate, March 2007.
5.5% of TTC borrowers are in default.
Actions: 1. More aggressively contact
students regarding their status
2. Utilize national data to track students
3. Update data on website to help with understanding borrowing rights and responsibilities
Loan Exit Counseling
Students successfully complete loan exit counseling.
≥85% of students successfully complete loan exit counseling within 2 months of notification.
Students who received loan exit counseling notifications during the Fall 2007 term completed the counseling by 2/28/08.
3.5% of students completed required counseling within 2 months
Actions: 1. Increase access to national
data. 2. Notify students earlier of exit
requirements 3. Provide direct contact with
students when they apply for graduation
Borrowing Rates Review average student loan debt of the graduating class.
Average loan debt for the graduating class does not exceed $5250. (Note: The $5250 was based on average four-year eligibility for loan)
Review Datatel reports which identify the average loan debt for the Spring 2008 graduates
Average borrowing per graduate is $6898
Actions: 1. Review and adjust average
loan borrowing rates due to changes in maximum limits.
2. Review rules regarding packaging of student loans.
3. Conduct individualized counseling to discourage excessive borrowing
Davis-McFarland, E.
VITAL STATISTICS
Indicator Trend Analysis
Loans: Trend analysis indicates a strong increasing linear trend with a p-value of 0.015. The total loan amount has increased by 86.6% since 2004. Grants: Trend analysis indicates a rebounding cubic trend with a p-value of 0.10. The total grant amount has decreased by 4.7% since 2004. Lottery: Trend analysis indicates a weak decreasing quadratic trend with a p-value of 0.22. The total lottery amount has increased by 5.6% since 2004. Scholarship: Trend analysis indicates a strong increasing linear trend with a p-value of 0.01. The total scholarship amount has increased by 59.4%. Work-Study: Trend analysis indicates a weak increasing linear trend with a p-value of 0.16. The total work-study amount has increased by 14.3%.
$-
$5,000,000
$10,000,000
$15,000,000
$20,000,000
2004 2005 2006 2007 2008Loan $11,662,833 $13,454,167 $19,584,212 $19,153,743 $21,774,304
Grant $13,618,677 $13,467,035 $12,770,029 $12,100,065 $12,977,851
Lottery $6,900,605 $7,426,861 $7,553,574 $8,182,811 $7,292,341
Scholarship $677,768 $654,815 $802,125 $953,854 $1,080,503
Work-Study $348,031 $376,733 $400,213 $449,859 $397,826
Total Financial Aid Disbursed by Award Type(by fiscal year ending June 30)
Source: TTC Office of Finance and Administration: Comprehensive Annual Financial Report June 30, 2008Updated: March 3, 2009