1

Tracking College Students Over Time: Tracking College Students Over Time: Using CIRP & CSS Data for Longitudinal Using CIRP & CSS Data for Longitudinal AssessmentAssessment

Victor Sáenz (UCLA) Victor Sáenz (UCLA) John Pryor (UCLA) & John Pryor (UCLA) & Gavin Henning (Univ. of New Hampshire)Gavin Henning (Univ. of New Hampshire)

Wednesday, May 17th, 2006Wednesday, May 17th, 2006AIR ForumAIR ForumChicago, ILChicago, IL

2

Presentation GoalsPresentation Goals

Overview of CIRP Freshman Survey

Introduce the CSS survey as a longitudinal follow-up to the CIRP Freshman Survey

• Discuss how institutions can utilize HERI longitudinal data to assess various aspects of the first-year experience

• Share descriptive findings from the 2005 CSS national aggregate data

• Institutional perspective on using HERI longitudinal data

3

Higher Education Research Institute

CIRPCooperative Institutional

Research Program

FreshmanSurvey YFCY CSS

Faculty Survey

Funded Research

•Atlantic Philanthropies•Templeton Foundation•National Institutes

of Health

4

CIRP, CSS, CIRP, CSS, & the I-E-O Model& the I-E-O Model

InputsCIRP Freshman Survey

(e.g., academic performancein high school, financial concerns

prior to college entry, expectations for college,

degree aspirations,self-concept in high school)

Environments CSS

(e.g., place of residence during college, interactions

with peers and faculty, curricularand co-curricular experiences)

OutcomesCSS

(e.g., satisfaction with college, retention, post-college plans)

5

Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Methodologies

•Descriptive analyses with your campus data

•Comparative analyses

•Measures of association

•Longitudinal analyses

•Factor analyses

•Multivariate analyses

6

Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Purposes

•Self-study reports

•Retention studies

•Recruitment issues

•Examining group differences among students

•Strategic planning

•Presentations to various constituents

•Creation of a student information system

•Measuring student development & institutional impact

7

CIRP Freshman SurveyCIRP Freshman Survey

Largest and longest-running national study of American college students…2006 is 40th Anniversary

Initiated in 1966 at the American Council on Education; Housed at HERI (UCLA) since 1973

Administered annually to over 400,000 incoming freshmen at more than 600 colleges and universities nationwide

Since 1966: 12 million students; 1,800 institutions

8

CIRPCIRPFreshman Survey ItemsFreshman Survey Items

Demographic Characteristics Expectations for College High School Experiences Degree Aspirations & Career Plans College Finances Attitudes, Values, & Life Goals Reasons for Attending College

FRESHMAN TRENDS (1966 – present)

9

CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen

Value: Keeping Up to Date with Political Affairs

0

10

20

30

40

50

60

70

Perc

ent

"very

im

port

ant"

or

"ess

enti

al"

10

CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen

Grade Inflation

0

10

20

30

40

50

Per

cent

A- orhigher

C+ orlower

11

CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen

Contrasting Values

0

20

40

60

80

Per

cent

Develop a meaningful philosophy of life

Be very well off financially

12

CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen

Reason for Attending College:"To Be Able To Make More Money"

0

10

20

30

40

50

60

70

80

1976 1979 1982 1985 1988 1991 1994 1997 2000 2003

Perc

ent

"very

im

port

ant"

Men

Women

13

CIRP Freshman TrendsCIRP Freshman Trends

Socio-historical context

Consistent trends over time

Changing student profile over the last 40 years (e.g, values, reasons for going to college, etc.)

What about students’ college experiences?

14

CSSCSS

HERI follow ups since 1967 Sponsored research

College Student Survey Launched in 1993 Allowed institutions to participate on their own timetable

Most use CSS as an exit survey for seniors

15

CSS Major ThemesCSS Major Themes

Academic and social adjustment Sources of satisfaction and dissatisfaction Academic, residential, and employment

experiences Plans for the next academic year Patterns of behavior Life goals Self-concept and feelings of personal success

16

2005 CSS*

*Data are unweighted.

116 Colleges and Universities 17,929 students

85% took the CIRP Freshman Survey in 2001

62% Female 38% Male

17

Results: 2005 CSS

Higher Satisfaction 4 out of 5 would make the same choice over

again to attend their college Class size (89.3%) Quality of Instruction (88.0%) Major courses (86.5%) Sense of community (76.9%) Ability to find faculty/staff mentor (67.5%)

18

Results: 2005 CSS

Higher Satisfaction Internet access (81.2%) Computer facilities (73.3%) Opportunities for Community Service (59.2%) Leadership opportunities (59.6%)

19

Results: 2005 CSS

Lower Satisfaction Student Housing (54.0%) Campus Health Services (43.7%) Job Placement (36.4%)

20

CSS (by racial group):Overall Satisfaction w/College Experience

Overall Satisfaction w/College Experience

77.1% 76.2% 74.0%68.1% 66.8% 64.8%

0%10%20%30%40%50%60%70%80%90%

100%

White AmericanIndian

Latina/o Other Race Asian/PacificIslander

AfricanAmerican

Perc

ent

dissatisfied/very dissatisfied

satisfied/very satisfied

21

Students felt successful in…

36%

42%

49%

52%

67%

69%

73%

0% 20% 40% 60% 80% 100%

Utilize campus resources

Get to know faculty

Manage time

Develop academic skills

Adjust to academic demads

Understand prof expectations

Develop close friends

22

CIRP vs. CSS(longitudinal results)

How do students change over their years of college?

23

Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

18%

46%

44%

28%

57%

61%

0% 20% 40% 60% 80%

Join Frat/Sor

Change Major

ChangeCareer

FreshmanCSS

24

Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

21%

95%

33%

97%

0% 20% 40% 60% 80% 100% 120%

SeekPersonal

Counseling

Socializeanother race

FreshmanCSS

25

Longitudinal Results: CIRP Freshman Survey & CSS

Change in Students' Self-Concept      

Percent rating themselves "above average" or "highest 10% relative to their peers : At college entry At the end of college Change

Computer skills 30.5 44.7 14.2

Self-understanding 56.5 67.2 10.7

Writing ability 52.7 62.7 10.0

Self-confidence (social) 48.2 57.4 9.2

Public speaking ability 40.2 48.9 8.7

Self-confidence (intellectual) 62.5 67.9 5.4

Leadership ability 64.0 67.1 3.1

Understanding of others 68.6 71.6 3.0

Popularity 39.5 42.0 2.5

Artistic ability 28.9 31.0 2.1

Creativity 55.2 57.0 1.8

Competitiveness 57.6 59.1 1.5

Academic ability 79.1 79.9 0.8

26

Longitudinal Results: CIRP Freshman Survey & CSS

Change in Students' Self-Concept      

Percent rating themselves "above average" or "highest 10% relative to their peers : At college entry At the end of college Change

Drive to achieve 78.5 78.3 -0.2

Emotional health 57.4 56.9 -0.5

Spirituality 44.3 39.9 -4.4

Physical health 59.6 53.7 -5.9

Religiousness 39.3 32.0 -7.3

Mathematical ability 51.7 40.2 -11.5

27

Using CIRP – CSS at University of New Hampshire

28

Tradition of Using CIRP Freshman Survey at UNH

UNH was pilot school in 1966 Data for 16 years going back to 1966 Have administered biennially since 1999 Have used data to better understand

incoming students Haven’t been using data for any type of

evaluation or outcomes assessmento UNH is lacking specific outcomes

29

College Student Survey at UNH

In 2001 UNH began to look more at student outcomeso Focus on learning, not just satisfaction and

general programmatic outcomes such as attendance

Viewed CSS as a tool for that assessment

30

College Student Survey at UNH

Compared CSS to NSSE and chose CIRP/CSSo High response rate for CIRPo Control of pre-college factors with CIRPo Great deal of question overlap between CIRP/CSS and

NSSEo CIRP/CSS was cheaper for UNHo Ability to pre-/post-test with same panel of students, not

just sample from same cohorts Need SSNs

o 30 extra questions on CIRP/CSS to customize for our campus

31

College Student Survey at UNH

CSS administration challengeso Inadequate mail/email addresses in 2001o Tried to have departments administer in capstone

experiences and other activities where seniors were together Only three departments participated

o Lack of intentional purpose for using data once collected UNH academic plan is general not specific enough to

evaluate against We are in the process of identifying institutional student

outcomes

32

College Student Survey at UNH

CSS administration spring 2005o Pilot test with 789 students who took CIRP in

2001 and that provided SSNs for linkageo Able to reach 763 of 789 students from this

sampling frame 97% coverage rate

o 278 of 763 students responded 36% response rate

33

Using CIRP/CSS Data at UNH

Understanding student characteristics Needs assessment Satisfaction survey of various campus services Outcomes assessment Benchmarking Program evaluation Strategic planning

34

Using CIRP/CSS Data at UNH

Understanding student characteristicso How they spent their time, behaviors, values,

etc. and how this has changed over their college career CIRP 2006: 27, 32, 33, 36, 38 CSS 2006: 7, 8, 13, 15, 21

o Educational activities engaged in at college CSS 2006: 7, 9

35

Using CIRP/CSS Data at UNH

Needs assessmento Areas of self-identified weaknesso CIRP item 28, 39

Satisfaction survey of various campus serviceso CSS 2006: 10, 18

36

Using CIRP/CSS Data at UNH

Outcomes assessmento Achievement of UNH student outcomes

Can use extra 30 questions Can investigate relationships between variables to

identify possible predictors for outcomes while controlling for pre-college inputs

o Examining how students’ perceived skill level/traits has changed during college career CIRP 2006: 31/CSS 2006: 23 CSS 2006: 19

37

Using CIRP/CSS Data at UNH

Outcomes assessment (continued)o Understanding the perceived impact of faculty

CSS 2006: 24

38

Using CIRP/CSS Data at UNH

Benchmarking against other institutions on any variety of variables

Program evaluationo Can compare students in certain programs

based on CSS variables, or change between CIRP and CSS on particular variables Academic program Co-curricular program

39

Using CIRP/CSS Data at UNH

Strategic planning Helping to assess selected goals and

strategies of the strategic plan Increase opportunities for student engagement

CSS 2006: 7, 9, 13, 19 Strengthen and facilitate sense of community

CSS 2006: 10, 18, 22, 24, 26

40

Using CIRP/CSS Data at UNH

Strategic planning (continued) Helping to assess selected goals and

strategies of the strategic plan Actively develop multicultural competence and

support diversity Establish and maintain a healthy/safe environment

for students and staff CIRP 2006: 26/CSS 2006: 15 CSS 2006: 26

41

Fostering use of CIRP/CSS at UNH

Commitment to assessment and evidence-based planning and decision-making

Concrete goals, strategies, and outcomes that can be evaluated

Marketing how the data can be used Putting the data into practice for assessment,

planning, and decision-making

42

For More Information:For More Information:

Victor Saenz vsaenz@ucla.edu

John Pryor john.pryor@ucla.edu

Gavin Henning gavin.henning@unh.edu

http://www.gseis.ucla.edu/heri/heri.html

(310) 825-1925

http://www.gseis.ucla.edu/heri/cirp.htm

http://www.gseis.ucla.edu/heri/css.html