Slide 1

An Experience-Based View of University Quality in the USA: The National Survey of Student EngagementAlexander C. McCormickCenter for Postsecondary ResearchIndiana University Bloomington, USA

HEPI-HEA ConferenceThe British Academy, London22 May 2012

nsse.iub.eduNational Survey of Student Engagement1Context: Conceptions of quality in U.S. tertiary educationThe National Survey of Student Engagement (NSSE)Low stakes, high yield assessmentStudent engagement trendsControversies and cautionsOverviewnsse.iub.eduNational Survey of Student EngagementContextHigher education in the USALimited understanding of qualityStudent engagement as a new approachnsse.iub.eduNational Survey of Student EngagementHigher Education in the USANo national system of higher educationAutonomy of institutionsDiversity of institutions4,600 degree-granting tertiary institutionsPublic and privateUniversities (undergraduate and postgraduate)Colleges (undergraduate only)4-year bachelors degree2-year associates degree (community colleges)nsse.iub.eduNational Survey of Student Engagement4Higher Education in the USA (2)Diversity of studentsFull- and part-timeTraditional college-ageOlder and returning studentsWidely varying levels of preparation

nsse.iub.eduNational Survey of Student Engagement5What Do We Really Know About University Quality?

nsse.iub.eduNational Survey of Student EngagementQuality Assessment in US Higher EducationAccreditationLimited public understandingLimited disclosureTraditional emphasis on capacity and resourcesLimited attention to teaching & learning (until recently)Newsmagazine rankingsEmphasize reputation, resources, other inputsResearch rankingsBlind to undergraduate education

nsse.iub.eduNational Survey of Student EngagementOrigins of the National Survey of Student Engagement (NSSE)A reaction to the dominant discourse about university quality in the USAShift the focus to teaching and learningPromote evidence-based improvement

nsse.iub.eduNational Survey of Student EngagementThe Big IdeaAsk students about their experienceFocus on behavior: empirically confirmed effective practices

nsse.iub.eduNational Survey of Student EngagementWhy a Student Survey?Direct assessment of learning is problematicMeasurement challengesInferential challengesCost challengesMotivation challengesSurvey approach offers advantagesCost-effective and efficientDiagnostic utility of behavioral dataStudents are the experts on their experience

nsse.iub.eduNational Survey of Student Engagement10NSSEs Key AimsEnrich the impoverished national discourse about university qualityShift the focus to teaching & learning, through the lens of effective practicesProvide diagnostic information that can be used to improve undergraduate educationActionable information based on valid & reliable measuresMeaningful comparisons to other institutionsnsse.iub.eduNational Survey of Student EngagementWhat is Student Engagement?The extent to which students engage in and are exposed to effective educational practicesActivities and practices known to be related to desired outcomes

nsse.iub.eduNational Survey of Student Engagement12OK, but what is Engagement?Challenging academic workActive learning Deep approaches to learningHigh-impact practicesInvolvement with instructors

In a campus context that promotes and supports successnsse.iub.eduNational Survey of Student EngagementConceptual & Empirical FoundationsTime on task (Tyler, 1930s)Quality of effort (Pace, 1960s & 70s)Student involvement (Astin, 1984)Academic & social integration (Tinto, 1975 & 1987)Good practices in undergraduate education (Chickering & Gamson, 1987)College impact (Pascarella, 1985)Student engagement (Kuh, 1991, 2005)nsse.iub.eduNational Survey of Student Engagement14Pace (1970s)Pioneer in this movement of looking at the entire student experience versus just looking at test scores or grades to assess student learning. He asked questions related to students academic and social experiences in college assessed how much effort they were putting forth in their educational experiencesAstin (Hired by Pace at UCLA in 1980s)Promoted his theory of student involvementAmount of learning taking place directly proportional to quantity and quality of energy invested in educational activitiesTinto (1970s & 80s)Retention model greater social and academic integration, both formal and informal processes -> greater satisfaction -> more likely to stayChickering and Gamson (1980s analysis of hundreds of studies over several decades)1) Student-faculty contact, 2) Cooperation among students, 3) Active learning, 4) Prompt feedback, 5) Time on task, 6) High expectations, and 7) Respect for diverse talents and ways of learningKuh (1990s idea of student engagement)What students do -- time and energy devoted to educationally purposeful activitiesWhat institutions do -- using effective educational practices to induce students to do the right things

Why Does Engagement Matter?The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement.

Paraphrased from Pascarella & Terenzini (2005), p. 602

nsse.iub.eduNational Survey of Student EngagementHLC-NCA AQIP session 2007 - www.nsse.iub.eduNASPA Conference 200615How Does it Work?

What Do We Ask?nsse.iub.eduNational Survey of Student EngagementHow NSSE WorksPaper or Web-based surveyFirst- & final-year undergraduatesUniform, centralized survey administrationInstitutions receive detailed reports and an identified student data fileCustomizable comparison groupsResults are confidentialSupported by institution fees

nsse.iub.eduNational Survey of Student EngagementParticipation550 to 770 U.S. & Canadian institutions per yearAbout 1,500 since NSSEs launch in 2000Most repeat every 1-4 yearsnsse.iub.eduNational Survey of Student EngagementSurvey ContentAcademic activities & experiencesAcademic challengeReading and writingActive learningCognitive tasks in coursesEnriching educational experiencesTime useStudy, work, socializingCo-curricular activitiesInstitutional emphasesQuality of campus relationshipsCognitive and noncognitive gainsSatisfactionDemographic & enrollment characteristicsnsse.iub.eduNational Survey of Student EngagementNSSE Foundations: Introduction to NSSENSSE Regional Users Workshop (April 2007)19Sample QuestionsAsked questions in class or contributed to class discussions?Made a class presentation?Prepared two or more drafts of a paper or assignment before turning it in?In your experience at your institution during the current school year, about how often have younsse.iub.eduNational Survey of Student EngagementSample QuestionsPut together ideas or concepts from different courses when completing assignments or during class discussions?Discussed ideas from your readings or classes with faculty members outside of class?In your experience at your institution during the current school year, about how often have younsse.iub.eduNational Survey of Student EngagementSample QuestionsReceived prompt written or oral feedback from faculty on your academic performance?Worked harder than you thought you could to meet an instructors standards or expectations?In your experience at your institution during the current school year, about how often have younsse.iub.eduNational Survey of Student EngagementSample QuestionsMemorising facts, ideas, or methodsAnalysing the basic elements of an ideaSynthesising and organizing ideas, informationMaking judgments about the value of informationApplying theories or conceptsDuring the current school year, how muchhas your coursework emphasized the following mental activities?nsse.iub.eduNational Survey of Student EngagementSample QuestionsExamined the strengths or weaknesses of your own views on a topic or issue?Learned something that changed the way you understand an issue or concept?During the current school year, about how often have you done each of the following?nsse.iub.eduNational Survey of Student EngagementUsing NSSE ResultsHow are we doing?

How can we improve?nsse.iub.eduNational Survey of Student EngagementUsing NSSE Results Diagnostic purpose: analysing the undergraduate experienceIdentify strengths and weaknesses, areas where improvement is neededStimulates conversations about qualityInforms decision-making with actionable information

nsse.iub.eduNational Survey of Student Engagement26Many Ways to Use ResultsPeer comparisonsHow do we measure up to others?Self comparisonAre we improving?Absolute standardHow do we measure up to our ideals?Internal variationWho is least engaged?What to look at and how to look at itnsse.iub.eduNational Survey of Student Engagement27NSSE results as a catalyst for campus discussions about teaching & learningnsse.iub.eduNational Survey of Student EngagementComparing Student andInstructor PerceptionsCourses emphasised memorisation Quite a bit or Very muchFaculty Surveynsse.iub.eduNational Survey of Student Engagement29Comparing Student andInstructor PerceptionsCourses emphasised memorisation Quite a bit or Very muchnsse.iub.eduNational Survey of Student Engagement30Results at institutions with several NSSE administrations show many with positive trendsnsse.iub.eduNational Survey of Student EngagementSummary of Findings41% of institutions demonstrated a pattern of improvement in at least one measure for first-year students28% did so for final-year studentsTrends found across many institutional typesControl (public & private), Size, Research emphasisRatio of positive to negative trends: 7:1nsse.iub.eduNational Survey of Student EngagementOverview of Observed Changes: FYnsse.iub.eduNational Survey of Student EngagementOverview of Observed Changes: Srnsse.iub.eduNational Survey of Student EngagementCautions & ControversiesNSSE is no magic bulletThe risks of high-stakes usesParticipation vs. usensse.iub.eduNational Survey of Student EngagementCautionsNSSE is no magic bullet: Best used in combination with other informationNSSEs research foundation is based on studies of US university studentsMost variation is between students, not institutions

nsse.iub.eduNational Survey of Student EngagementControversiesConfidentiality of results: Institutions control disclosureAttaching high stakes to NSSE is riskyInstitutions will decline to participate ifpublic relations cost > institutional benefitReliability of responses could be compromisedMere participation is not sufficientResults must be analysed and usednsse.iub.eduNational Survey of Student EngagementUnderstanding ImprovementOrganizations that improve create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile.

(Elmore in Fullan, 2001)

nsse.iub.eduNational Survey of Student Engagement38Thank you.nsse.iub.eduNational Survey of Student Engagement