NSSE 2013 – Overview of Results

Jerre Cline, Reporting Analyst,Office of Institutional Research

Joshua Powers, Associate Vice Presidentfor Student Success

NSSE overview Engagement research & NSSE as an

independent variable NSSE at ISU

◦ 2013 NSSE Administration◦ ISU Results and in Comparison Context◦ Implications and Discussion

Goals of Session

Product of Indiana University’s Center for Postsecondary Research

in the IU-Bloomington School of Education. Part of a larger survey group that includes

the BCSSE and the FSSE. Measures empirically confirmed “good

practices” in undergraduate education related to achieving positive outcomes.

Not a tool that measures student learning.

What is the NSSE?

StudentSuccess Conference 2013

First administered to colleges and universities in 2000.

Indiana State University began participating in the NSSE in 2003.

Revised Survey in 2013 – NSSE 2.0. 4 million college students and 1500+

institutions of higher education have utilized the NSSE since it’s inception. In 2013, 613 institutions and 364,000

students participated in NSSE 2.0.

NSSE History

StudentSuccess Conference 2013

Amount of time and effort that students put towards their studies.

Effective institutional practices provided to students.

Related to student participation in extracurricular activities.

Affected by instructional strategies.

Student Engagement

StudentSuccess Conference 2013

10 official peers participated in NSSE 2.0. DRU - Basic Carnegie Classification. 613 institutions participated in NSSE 2.0. Accepted by Voluntary System of

Accountability and used in College Portraits; aimed at assisting students and parents in selecting a college.

Institutional Comparisons

StudentSuccess Conference 2013

Devoting substantial time and effort, caring about work quality, feeling the work has significance = engagement (Newmann, 1986).

Collaborative learning and experiential learning + accessible & relevant curriculum increase student engagement (Akey, 2006; Heller et al., 2003).

High quality instruction (rigorous; aligned with content standards; using instructional strategies to meet academic needs of all students – key factor in promoting culture of engagement and achievement in the classroom (Weiss & Paisley, 2004).

Engagement Research Findings

StudentSuccess Conference 2013

NSSE results good proxy measures for growth in important educational outcomes (Pascarella et al., 2010).

No relationship between total NSSE scores & GPA or graduation rates (DiRamio & Shannon, 2010).

NSSE Benchmarks failed to meet validity or reliability standards (Campbell & Cabrera, 2010; Porter, 2009).

NSSE benchmarks valid & reliable indicators for quality of student experience for historically underserved students; engagement had stronger effects on 1st year grades and persistence to the 2nd year for underprepared students (Kuh, et al., 2007, 2008).

NSSE 2.0 developed partly in response to research findings; some indicating students didn’t fully understand questions being asked (ambiguous verbiage; unknown contextual interpretation impact).

NSSE Research

StudentSuccess Conference 2013

NSSE 2013 Administration at ISU

Submitted population

Total respondents

2,789

First-year Senior

1,763

633637

Survey completions

Response Rate: 23% 36%

Message schedule

First-yearInvitationReminder 1Reminder 2Reminder 3Final reminder

31%36%

20%23%

02/27/201303/05/2013

DateCumulative response rate

9%18%27%

7%13%17%02/21/2013

02/05/201302/13/2013

Senior

StudentSuccess Conference 2013

Engagement Indicators: Overview

▲ Your students’ average was significantly higher (p<.05) with an effect size at least .3 in magnitude.

△ Your students’ average was significantly higher (p<.05) with an effect size less than .3 in magnitude.

-- No significant difference.

▽ Your students’ average was significantly lower (p<.05) with an effect size less than .3 in magnitude.

▼ Your students’ average was significantly lower (p<.05) with an effect size at least .3 in magnitude.

First-Year (FY) Students

Theme Engagement Indicator

Higher-Order Learning

Reflective and Integrative Learning

Learning Strategies

Quantitative Reasoning

Collaborative Learning

Discussions with Diverse Others

Student-Faculty Interaction

Effective Teaching Practices

Quality of Interactions

Supportive Environment

△--△

--

△ △▽△▽ ▽----

Campus Environment --

Experiences with Faculty

-- --

Engagement Indicators are summary measures based on sets of NSSE questions examining key dimensions of student engagement. The ten indicators are organized within four themes: Academic Challenge, Learning with Peers, Experiences with Faculty, and Campus Environment. The tables below compare average scores for your students with those in your comparison groups.

Learning with Peers

ISU Peer Group Carnegie Class

--

NSSE 2013▽

----

Your FY students compared with

Your FY students compared with

Your FY students compared with

▽▽--

Academic Challenge

▽--

--▽---- ----

37.133.840.028.0

32.140.4

22.939.1

40.936.1

ISU Results

Seniors

Theme Engagement Indicator

Higher-Order Learning

Reflective and Integrative Learning

Learning Strategies

Quantitative Reasoning

Collaborative Learning

Discussions with Diverse Others

Student-Faculty Interaction

Effective Teaching Practices

Quality of Interactions

Supportive Environment

--▽

▽ --

----

▼▽

--

-- ▽ ▽

▽

Carnegie Class

▽--

NSSE 2013

--

▽Campus Environment

Your seniors compared with

Your seniors compared with

Your seniors compared with

▽

-- ▽--

▽

------

Learning with Peers

--

▽Academic Challenge

--△Experiences

with Faculty

ISU Peer Group

▽

40.736.740.828.8

28.540.5

22.839.2

41.729.9

StudentSuccess Conference 2013

High Impact PracticesFirst-Year Students %

11c. Learning community 10 18 *** -.23 17 *** -.19 15 ** -.14

12. Service-learning 69 52 *** .37 56 *** .28 52 *** .36

11e. Research with faculty 5 5 .01 5 .01 5 .00

Participated in at least one 72 58 *** .29 61 *** .23 58 *** .29

Participated in two or more 10 14 * -.11 14 * -.11 12 -.05

Seniors11c. Learning community 31 25 *** .14 21 *** .23 24 *** .16

12. Service-learning 78 58 *** .43 59 *** .41 60 *** .38

11e. Research with faculty 20 20 .00 15 ** .12 23 -.08

11a. Internship or field exp. 59 46 *** .25 36 *** .46 48 *** .22

11d. Study abroad 10 8 * .08 8 .07 13 -.08

11f. Culminating senior exp. 43 36 *** .14 36 ** .13 45 -.03

Participated in at least one 91 82 *** .26 77 *** .39 84 *** .21

Participated in two or more 68 55 *** .27 48 *** .40 60 *** .17

Note: All results weighted by gender and enrollment status (and by institution size for comparison groups).

Indiana State NSSE 2013ISU Peer Group Carnegie Class

% b

a. Percentage of students who responded "Done or in progress" except for service-learning which is the percentage who responded that at least "Some" courses included a community-based project.b. *p<.05, **p<.01, ***p<.001 (z-test comparing participation rates).c. Cohen's h: The standardized difference between two proportions. Effect size indicates the practical importance of an observed difference. An effect size of .2 is generally considered small, .5 medium, and .8 large.

% bEffect size c

Effect size c

Effect size c% b

First-yearHighest Performing Relative to ISU Peer Group

About how many…courses have included a community-based project (service-learning)?e (HIP)

Inst. emphasizes… Helping you manage your non-academic responsibilities (…)c (SE)

Worked with a faculty member on activities other than coursework (…)b (SF)

Discussions with… People of a race or ethnicity other than your ownb (DD)

Discussed your academic performance with a faculty memberb (SF)

SeniorHighest Performing Relative to ISU Peer Group

About how many…courses have included a community-based project (service-learning)?e (HIP)

Participated in an internship, co-op, field exp., student teach., clinical placemt. (HIP)

Instructors… Provided feedback on a draft or work in progress c (ET)

Used numerical information to examine a real-world problem or issue (…)b (QR)

Completed a culminating senior experience (…) (HIP)

Points of Pride

+18+12+7+6+6

+20+12+7+7+7

Areas of ConcernLowest Performing Relative to ISU Peer Group

Tried to better understand someone else's views by imagining…his or her perspective b (RI)

Learned something that changed the way you understand an issue or concept b (RI)

Connected ideas from your courses to your prior experiences and knowledge b (RI)

Inst. emphasizes… Using learning support services (…)c (SE)

Participated in a learning community or some other formal program where… (HIP)

Lowest Performing Relative to ISU Peer GroupInstructors… Used examples or illustrations to explain difficult points c (ET)

Asked another student to help you understand course material b (CL)

Inst. emphasizes… Using learning support services (…)c (SE)

Prepared for exams by discussing or working through course material w/other students b (CL)

Explained course material to one or more students b (CL)

-5-5-6-8-8

-4-5-6-6-7

StudentSuccess Conference 2013

First-year

Senior

Do We Provide Enough Challenge?Challenging Courses

First-year Senior

To what extent did your students' courses challenge them to do their best work? Response options ranged from 1 = "Not at all" to 7 = "Very much."

52% 44% 44% 39%

47%54% 53% 58%

0%

25%

50%

75%

100%

IndianaState

ISU PeerGroup

IndianaState

ISU PeerGroup

StudentSuccess Conference 2013

Freshmen: Statistically below other institutions in all 9 categories, although effect size largest1 (.19 to .21) for:◦ Been available when needed.◦ Listened closely to your concerns and questions.◦ Informed you of academic support options.

Seniors: Statistically below other institutions in 4/9 categories (and none statistically higher), although effect size largest (.20) for:◦ Been available when needed.

Academic Advising Results

1Defined as .19 or above for both academic advising and writing.

StudentSuccess Conference 2013

Freshmen: Statistically above other institutions in 4/13 categories, although effect size largest (.25) for:◦ Gave feedback to a classmate about a draft or outline

he or she had written. Statistically below other institutions in 2/9 categories, although effect size largest for:

◦ Argued a position using evidence and reasoning. Seniors: Statistically below other institutions in

6/13 categories (and none statistically higher), although effect size largest (-.28) for:◦ Argued a position using evidence and reasoning.

Experience with Writing Results

StudentSuccess Conference 2013

Are there any findings that surprised you? How might we optimally disseminate the

results and engage the campus? What should be priority attention areas?

Discussion & Implications