presented by michele j. hansen, ph.d., director of uc assessment november 20, 2008

Presented by Michele J. Hansen, Ph.D., Director of UC Assessment

November 20, 2008

CRG Steering Committee Meeting

Understanding Student Success and Retention

Student Progress and Academic Success Highlights of Research on Impacts of UC

Programs First-Year Seminars Themed Learning Communities Summer Bridge Critical Inquiry

Factors Associated with Academic Success 2008 First-Time, Full-Time Cohort

(Indianapolis Only)

Presentation Overview

One-Year Retention Rates

2005 2006 2007

Columbus59% 58% 47%

Indianapolis 64% 67% 70%

IUPUI64% 66% 68%

Six-Year Graduation Rates

1999 2000 2001

Columbus 20% 21% 21%

Indianapolis 26% 28% 32%

IUPUI 25% 28% 31%

IUPUI Six-Year Graduation Trend

IUPUI (Indianapolis Only) Retention Trend Lines

Source: IMIR UC 10-Year Celebration Report.

FT Beginners GPAs and Retention Rates

Source: University Reporting & Research: official reports are available at http://www.indiana.edu/~urr

Approaches to Assessing First-Year Programs

Use assessment to prove and improve programs: summative and formative evaluation.

Employ multiple measures of student learning and academic success - moving beyond retention.

Use program evaluation methodology: understanding needs, processes, and learning outcomes.

Understand the dialogue between qualitative and quantitative research.

Seek involvement of key stakeholders in assessment planning, implementation, and deployment.

Ensure information used for program improvement. Meta-Assessment

Focus on Accountability

1. Four-Year Graduation Rates2. Degree Completion3. Course Completion4. “Value-Added” Interventions5. Cost-Effectiveness6. Student Learning Outcomes

Complexity of Retention

1. Pre-College Factors and Preparation

2. Background Characteristics3. Students’ Intentions 4. Students’ Expectations5. Students’ Needs6. Students’ Financial Issues 7. Institutional Factors 8. External Environmental Factors

First-Year Seminar

N Retention Rate

Participants 2248 70%

Non-Participants

308 65%

Overall 2556 69%

Fall 2007 Impact of Participation in a First-Year Seminar on One-Year Retention

Hierarchical Logistic Regression: 2007 First-Year Seminar and Retention

Step Variables B Wald Prob. df

1 Course load SAT scoreHigh School GPA Female

0.170.000.76-0.05

26.18 3.0239.27 0.27

.000

.082

.000

.603

4, 2558

2 Seminar

Nagelkerke R2 =.078

0.34 5.40 .020 1, 2557

1. 30% of the non-participants earned a grade point average below a 2.0 compared to 23% of participants.

2. The DWF rate for non-participants (29%) was notably higher compared to participants (23%).

3. The IUPUI fall-to-spring retention rate for non-participants was 83% compared to 87% for participants.

Other Indicators of Academic Success: 2007 First-Year Seminars

N H.S. GPA

SAT Score

% African American

%25+

Participants 2248 3.18 997 8% (176) 2%

Non-Participants

309 3.22 1011 14% (43) 7%

309 Non-Participants

No significant differences in hours expected to work off-campus, studying, or care for dependents!

Participants expect to work significantly more hours on-campus.

Type ofAdmit

First Year Seminar N Retention Rate

Adjusted Rate

Regular Admits

Participants 1567 74% 74%

Non-Participants 227 72% 69%Overall 1794 74%

Conditional Admits


Non-Participants 81 46% 49%Overall 762 58%

Impact of Participation in a 2007 First-Year Seminar: One-Year Retention

N Retention Rate

Seminar 1683 68%

Seminar-TLC 565 76%

No Seminar 308 65%

Fall 2007 Impact of Participation in a First-Year Seminar on One-Year Retention

Students’ Reported Notable Improvements and Course Benefits in the Following Areas…

1. Making connections with other students, peer mentors, faculty, and advisors.

2. Academic advising (e.g., knowledgeable, available when needed assistance).

3. Experiencing environment that promotes and respects diversity.

4. Becoming familiar with campus and academic support resources.

5. Deciding on a major or future career.6. Adjusting to college.

1. Making connections and forming a sense of community.

2. Learning more about IUPUI campus and academic support resources.

3. Exploring majors and career opportunities.

4. Learning about library resources.

Most Valued Aspects of Leaning Communities

1. Assignments that do not contribute to learning process: “busy work.”

2. Unorganized activities in and outside of class.

3. Class activities that are not linked with discipline courses.

4. Class activities that are not related to major or career goals.

Least Valued Aspects of Leaning Communities

Significant differences of FYS students in comparison to others:

Students participating in FYS1. made more class presentations.2. worked with students outside of class more often.3. participated in more community-based projects.4. Included diverse perspectives in class discussions or

writing assignments.FYS students also reported

5. Better quality of academic advising. 6. experiencing a more supportive campus environment. 7. engaging in more active and collaborative learning.

National Survey of Student Engagement

2004 N Adjusted GPA* TLC Participants 255 2.65 Non-Participants 1217 2.56



2007

N Adjusted GPA* TLC Participants Non-Participants

535 1499

2.79 2.55

TLC First Semester G.P.A.

Comparison group – students who

participated in a freshman seminar or learning community.

*G.P.A. adjusted to control for significant

covariates including: course load, gender, ethnicity, SAT scores,

high school percentile ranks,

units of high school math, and first-

generation students.

Bolded items are significant p<.01

TLC Retention

Fall 2004 One Year Retention

# of Students Retention Rate*TLC Participants 287 69%

Non-TLC Participants 1351 68%










Comparison group – students who

participated in a freshman seminar or learning community.

*Bolded items are significant p<.05, even while

controlling for differences in

demographics, enrollment, and

academic preparation.

Type ofAdmit

N Average Retention

Regular


Non-Participants

1338 74%

Conditional


Non-Participants

550 56%

TLC Retention: Full-Time Students


Significant differences between TLC students (80) in comparison to others (280):

Students participating in TLCs more often:

1. Worked on a paper or project that required integrating ideas or information from various sources.

2. Made more class presentations. 3. Included diverse perspectives (different races, religions,

genders, political beliefs, etc.) in class discussions or writing assignments.

4. Put together ideas or concepts from different courses when completing assignments or during class discussions .

5. Worked harder than thought they could to meet an instructor's standards or expectations.

6. Tried to better understand someone else's views by imagining how an issue looks from his or her perspective.

7. Learned something that changed the way you understand an issue or concept.


TLC students also reported more institutional emphasis on:

1. Providing the support needed to help students succeed academically

2. Encouraging contact among students from different economic, social, and racial or ethnic backgrounds

3. Helping students cope with non-academic responsibilities (work, family, etc.)

4. Working effectively with others5. Understanding people of other racial and ethnic

backgrounds


PULS AND CORRESPONDING NSSE ITEMS TLC IUPUI

COMPARISON GROUP (NOT-TLC)

PEER INSTITUTION

Integration & Application of Knowledge Worked on a paper or project that required integrating ideas or information from various sources.

3.45 3.08 3.04

Put together ideas or concepts from different courses when completing assignments or during class discussions.

2.84 2.60 2.55

Understanding Society & Culture Included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments.

3.10 2.69 2.85

Participated in a community-based project (e.g., service learning) as part of a regular course.

2.16 1.75 1.38

Institutional emphasis: Encouraging contact among students from different economic, social, and racial or ethnic backgrounds.

3.05 2.56 2.59

Institutional contribution: Understanding people of other racial and ethnic backgrounds.

2.89 2.56 2.78


PULS AND CORRESPONDING NSSE ITEMS TLC IUPUI COMPARISON GROUP (NOT-TLC)

PEER INSTITUTION

Intellectual Depth, Breadth & Adaptiveness Coursework emphasized: SYNTHESIZING and organizing ideas, information, or experiences into new, more complex interpretations and relationships.

3.14 2.91 2.79

Core Communication & Quantitative Skills Institutional contribution: Writing clearly and effectively

3.19 2.98 2.88

Critical Thinking Institutional contribution: Thinking critically and analytically.

3.30 3.07 3.08

Coursework emphasized: ANALYZING the basic elements of an idea, experience, or theory, such as examining a particular case or situation in depth and considering its components.

3.41 3.02 2.99

Academic Performance of TLC Students Compared to Students in Same School or Discipline Seminar

N A ve ra ge F al l G PA A vg . P re dic te d G PA a D ifferen ce

B US TLC 35 2 .97 2.94 0.01B US Sem inar 2 87 2 .47 2.69 -0.21E DUC TLC 66 2 .98 2.64 0.34E DUC 69 2 .40 2.70 -0.24E NG R TLC 45 2 .49 3.03 -0.58E NG R S em in ar 94 2 .61 2.91 -0.37HE RRON TLC 19 3 .20 2.94 0.26HE RRON Sem inar 1 04 2 .89 2.79 0.09P E& TM TLC 25 3 .01 2.87 0.14P E& TM S eminar 23 2 .33 2.68 -0.38NUR S-B TLC 39 3 .33 2.97 0.37NUR S-S em in ar 23 2 .95 2.84 0.11P SY TLC 18 2 .33 2.74 -0.33P SY 65 2 .31 2.76 -0.48S LA TLC 65 2 .73 2.80 -0.11S LA 66 2 .75 2.84 -0.06S PE A TLC 40 2 .20 2.65 -0.41S PE A 22 2 .40 2.65 -0.18S W K T LC 24 2 .95 2.76 0.24S W K** 18 2 .28 2.58 -0.40UCO L TLC 1 80 2 .80 2.63 0.18UCO L-Sem inar 6 63 2 .43 2.67 -0.23

1. Offered as a two-week program for incoming freshmen held in August before fall classes begin

2. Open to students in specific majors3. Based on student interest in pursuing a

particular major or in exploring various major options

4. Provides a collegiate-level curriculum 5. Creates communities of entering students6. Offered free to participants7. Required for all First Generation

Scholarship award winners beginning fall 2006

Essential Elements Of Summer Bridge

Characteristics of 2005 Cohort

1. 175 Students Participated 2. 70% Women 3. 9% African American4. 53% First-Generation College Student5. 18% First-Generation Scholars6. 22% Admitted Conditionally7. 1016 Average SAT Score8. 69% Average High School Percentile Rank9. 39% Campus Housing10. 18-19 Primary Ages. Average=18.17

Characteristics of Fall 2006 Two-Week Cohort

1. 209 Students Participated 2. 71% Women 3. 8% African American4. 89% First-Generation College Student5. 70% First-Generation Scholars6. 7% Admitted Conditionally7. 1000 Average SAT Score8. 75% Average High School Percentile Rank9. 27% Campus Housing 10. 18-19 Primary Ages. Average=18.75

Characteristics of Fall 2007 Two-Week Cohort

1. 361 Students Participated 2. 72% Women 3. 7% African American4. 91% First-Generation College Student5. 78% First-Generation Scholars6. 10% Admitted Conditionally7. 982 Average SAT Score8. 3.31 Average High School GPA9. 34% Campus Housing 10. 18-19 Primary Ages. Average=18.77

Type ofAdmit

Summer Bridge

N Fall GPA Adjusted Fall GPA

Regular

Participants 120 2.94 2.87

Non-Participants

858 2.65 2.68

Overall 978 2.69

Conditional


Non-Participants

461 2.06 2.06

Overall 494 2.09

Impact of Summer Bridge Participation 2005

Type ofAdmit

N Average Retention

Regular


Non-Participants

936 70%

Overall 1263 (70%)

Conditional

Participants 32 77%

Non-Participants

480 56%

Overall 612 (57%)

Impact of Summer Bridge Participation 2005

Two-Week Summer Bridge Participants Compared to Non-Bridge Participants: Conditional Admits

NAvg. Fall

GPA% below a 2.0 GPA DFW Rate

Fall to Fall Retention

Rate 2005 Bridge 37 2.53 27% 19.10% 78%No-Bridge 648 2.08 41% 38.22% 53%2006 Bridge 15 1.58 53% 47.50% 40%No Bridge 713 2.08 41% 37.99% 53%2007 Bridge 35 2.11 29% 31.46% 69%No Bridge 649 2.15 37% 32.78% 58%

Summer Bridge Participants Compared to Non-Bridge Participants: First-Generation Scholars

NAvg. Fall

GPA% below a 2.0 GPA DFW Rate

Fall to Fall Retention

Rate IU2005 NO Bridge 114 2.77 19% 18.58% 73%2006 Two-Week Bridge 147 2.78 14% 14.62% 72%2006 Weekend Bridge 46 2.78 15% 19.53% 70%No Bridge 50 2.66 20% 23.18% 70%2007 Two-Week Bridge 281 2.96 11% 12.14% 77%2007 Weekend Bridge 62 2.96 15% 14.72% 84%No Bridge 12 2.59 17% 32.87% 75%

2007 Bridge-Themed Learning Community Combination has Positive Effects

Hierarchical Multiple Regression: 2007 Two Week Summer Bridge and GPA

First-Time, Full-Time Eligible for the 21st. Century Scholarship Bridge Participants vs. Cohort Non-

Participants

2005 2006 2007Summer Bridge Participants 2.62 2.18 2.56Cohort Non-Participantsa 2.01 2.11 2.22

2005 2006 2007Summer Bridge Participants 28% 32% 23%Cohort Non-Participantsa 49% 42% 39%

2005 2006 2007Summer Bridge Participants 78% 65% 70%Cohort Non-Participants 52% 53% 56%

Fall GPA

Percent Below 2.0 Fall GPA

One-Year Retention Rate

N Mean Std. Deviation

Social Integration/Sense of Community

916 4.41 .55

Campus Resources 896 4.40 .44College Adjustment 894 4.22 .53Interactions with Instructional Team

899 4.17 .60

College Expectations 886 4.17 .56Class Activities 923 4.16 .68Study Skills 911 4.04 .64Critical Thinking 908 4.01 .65Academic Skills 909 3.92 .64

Top Rated Benefits of Summer Bridge

1. College Adjustment2. Course Activities3. Campus Resources4. Interactions with Instructional Team

Members

adjusted R2 =.215, F (9, 756)=23.97, p<.0001).

Factors that Significantly Predict Overall Satisfaction with Summer Bridge

98% of 2008 students surveyed (n=414) said they would recommend the Summer Bridge program to other first-year students.

2007 = 98%2006 = 99%2005 = 96%, 2004 = 98%

Summer Bridge (Two-Week)Student Questionnaire Results

Summer Bridge Long Term Impacts

Critical Inquiry

N Average Spring GPA

Adjusted Spring GPA


Non-Participants

398 2.02 2.03

Overall 602 2.08

Spring 2008 Impact of Participation in a Critical Inquiry Course for Conditional Admits

Critical Inquiry

N Spring-Fall Retention

Adjusted Spring-Fall Retention


Non-Participants

398 66% 66%

Overall 602 69%

Spring 2008 Impact of Participation in a Critical Inquiry Course for Conditional Admits

Academically underprepared for college-level work First-generation college student Gap between high school and college 30+ hours working per week Part-time enrollment Single parent Financially independent Children at home Lack of institutional and goal commitment

George Kuh, 2006

Factors That Threaten Persistence and Graduation from College

Factors Associated with Success (Source: Gary Pike Presentation 4/19/07)

Gender

First-Generation Student

Institutional Commitment (Intent to Transfer)

Amount of Time Spent Working

Student Groups (predicted GPAs based on SAT/ACT & High School GPA)

Ethnicity (minority status) was not significantly related to student success.

IUPUI Factors Related to Academic Success

Levels of Academic Hope. The process of thinking about one’s goals,

along with the motivation to move toward those goals (agency) and the strategies to achieve those goals (pathways).

Comprehension and completion of assignments during the last year of high school.

Participating in First-Year Seminars, Thematic Learning Communities, and Summer Bridge (early interventions).

Summary of Major Changes

ICHE Accountability and Outcomes Focus: degree completion, on-time graduation, value-added experiences.

More academically prepared students. More International and Out-of-State students. More students entering just out of high school:

18-19 years of age. More students living on-campus. Improved retention and graduation rates.

Retention not likely to exceed 80% in next 5 years.

Factors to Consider

• Many students have not completed a rigorous high school college-preparatory curriculum (23% of 2007 FT, FT students earned below a 2.0 GPA during their first semester).

• Difficult for students to make the necessary connections with other students and faculty.

• First generation students may benefit from support/mentoring to help bolster academic performance.

• Notable percentage of African American students are first generation college students (70%).

• Students seem to have unrealistic expectations about their expected levels of academic performance and time they should be devoting to studying.

• Continue investigations of what interventions produce the best educational outcomes and take into account diverse students’ needs.

2008 IUPUI (Indianapolis Only) First-Year Students Profile

• 2551 first-time, full-time students• Only 7% part-time students• 584 (23%) live in campus housing• 608 (24%) admitted conditionally• 59% female• Only 72 (2%) 25 years of age or older• 97% In-State Students• 56% First Generation• 236 (9%) African American

2008 IUPUI (Indianapolis Only) First-Year Students Profile

• 13.71 Average Course Load • 19.23 Average Age• 3.25 Average High School GPA• 1015 Average SAT score • 298 (12%) were Eligible to receive the 21st

Century Scholarship (234 actually awarded scholarship)

• 2401 (92%) participated in First-Year Seminars• 412 participated in the Full Summer Bridge

Program (62 or 15% conditional admits). • 88 students participated in the Weekend

Summer Bridge Program

Student ProfileFall 2008 First-Time, Full-Time

N Percent

Native American 8 .3%Asian 109 4.3%Black/African American 236 9.3%Hispanic/Latino 84 3.3%Native Hawaiian/Other 5 .2%Non-Resident/International 93 3.6%Other American 20 .8%Caucasian 1963 77.0%

Refused to Answer/Not Applicable 38 1%

Total 2551 100%

Number and Percentage of Students in the First-Time, Full-Time Cohort at IUPUI (Indianapolis Only) who are African American

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008Number 288 254 226 193 197 225 251 216 204 236Percentage 12% 11% 11% 9% 9% 11% 11% 10% 8% 9%

Fall Entry Year

First-Time, Full-Time Campus Housing

Reasons for Attending College

1. Acquire knowledge and skills applicable to a specific job or type of work

2. Fulfill a lifelong goal3. Make more money4. Gain a general education5. Get a better job6. Meet new people7. Prepare for graduate or professional school8. My parents wanted me to go9. Change in financial situation10. Advance in my current job11. Issues related to children or childcare12. Change in marital status

(rank order by mean importance)

Top 10 Reasons for Attending IUPUI

1. Availability of specific academic programs (majors)2. Location3. Opportunity to receive an Indiana University or Purdue University

degree4. Career and job opportunities available in the area after completing

my degree5. Cost6. Ability to work while attending college7. Opportunities associated with the location of IUPUI in

Indianapolis8. The variety of entertainment and social activities in the downtown

area9. IUPUI's reputation10. Admissions requirements

(rank order by mean importance)

60% reported that they applied to a college or university other than IUPUI.

64% reported that IUPUI was their first choice (if applied to other universities).

96% reported that it is important for them to graduate from College (agree or strongly agree)?

82% reported that it is important for them to graduate from IUPUI (agree or strongly agree).

89% reported that they made the right choice in attending IUPUI (agree or strongly agree).

Commitment to IUPUI

Fall 2008 Beginning Freshmen Entering Student Survey Results

Academic Goal Commitment

96% have a good understanding of their academic goals (agree or strongly agree).

82% are certain about their career goals (agree or strongly agree).

81% are certain about their choice of educational program or major (agree or strongly agree).

95% feel confident that they will complete their degrees in a timely manner (agree or strongly agree).

91% plan to earn at least a four-year (bachelor’s degree).


20% reported that they read 10 or more books outside of required reading during their last year prior to attending IUPUI.

85% reported that they have completed a math class recently (within the last year).

54% took at least one AP Course during high school (40% took at least one AP Exam).

Average Units of High Units of High School = 8.19 (Full-Time Students). 71% had 8 or greater math credits.

Academic Preparation


Reading Books Outside of Required Reading in Courses

Number of Books N Percent None 160 12%

1-3 494 38%4-6 248 19%7-9 143 11%

10-12 109 8%13-15 30 2%

16 or more 126 10%Total 1310 100%

Title of Last Math Course

Title of Course N Percent

Basic or Advanced Mathematics 34 2.2Algebra 297 16.2Geometry 66 3.3Trigonometry 75 4.3Pre-Calculus 511 35.9Calculus 260 18.7Differential Equations 3 .7Probability and Statistics 203 12.5Other 113 6.1Total 1562 .

Full-Time Students’ Expectations of Time Use Per Week

54% live with family members (parents, spouse, children, or other relatives).

97% reported that they have access to a personal computer that they can use for school work.

82% reported that they have High Speed internet access (Cable Modem, DSL, Satellite).

Students’ Resources


Living Arrangements

1. Live alone = 9%2. With one or more roommates who

are students attending IUPUI = 32%

3. With others not attending this college = 6%


76% of students attending orientation reported that they applied for financial aid.

75% reported that they have some or major concerns about their ability to finance their college education.

Students’ Finances


www.imir.iupui.edu

http://uc.iupui.edu/staff/assessment/index.asp

presented by michele j. hansen, ph.d., director of uc assessment november 20, 2008

Documents

year programs

year sucess symposium

michele j

crg presentation

year graduation trend

presentation overview

year celebration report

michele focus