nsse:retrospectiveandprospective george kuh aac&u general education and assessment conference...
TRANSCRIPT
NSSE:NSSE:Retrospective Retrospective
and and ProspectiveProspective
George KuhGeorge KuhAAC&U General Education and AAC&U General Education and
Assessment ConferenceAssessment ConferenceMarch 2, 2007March 2, 2007
National Survey of National Survey of Student EngagementStudent Engagement(pronounced “nessie”)
Community College Community College Survey of Student Survey of Student EngagementEngagement(pronounced “cessie”)
College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development
We value what we measureWe value what we measure
Wise decisions are Wise decisions are needed about needed about whatwhat to to measure in the context measure in the context of campus mission, of campus mission, values, and desired values, and desired outcomes. outcomes.
OverviewOverview
1.1. Why student engagement Why student engagement mattersmatters
2.2. What we’ve learnedWhat we’ve learned
3.3. ConversationConversation
What Really Matters in College: Student Engagement
Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.
Pascarella & Terenzini, Pascarella & Terenzini, How College How College Affects StudentsAffects Students, 2005, p. 602, 2005, p. 602
Foundations of Student EngagementFoundations of Student Engagement
Time on task Time on task (Tyler, 1930s)(Tyler, 1930s)
Quality of effort Quality of effort (Pace, 1960-70s)(Pace, 1960-70s)
Student involvement Student involvement (Astin, 1984)(Astin, 1984)
Social, academic integration Social, academic integration (Tinto,1987, 1993)(Tinto,1987, 1993)
Good practices in Good practices in undergraduate education undergraduate education (Chickering & Gamson, 1987)(Chickering & Gamson, 1987)
College impact College impact (Pascarella, 1985)(Pascarella, 1985)
Student engagement Student engagement (Kuh, 1991, (Kuh, 1991, 2005)2005)
Good Practices in Good Practices in Undergraduate EducationUndergraduate Education
(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005))
Student-faculty contactStudent-faculty contact Active learningActive learning Prompt feedbackPrompt feedback Time on taskTime on task High expectationsHigh expectations Experiences with diversityExperiences with diversity Cooperation among studentsCooperation among students
The Student Engagement TrinityThe Student Engagement Trinity
What students What students dodo -- time and energy -- time and energy devoted to educationally purposeful devoted to educationally purposeful activitiesactivities
What institutions What institutions dodo -- using -- using effective educational practices to effective educational practices to induce students to do the right induce students to do the right thingsthings
Educationally effective institutions Educationally effective institutions channel student energy toward the channel student energy toward the right activitiesright activities
NSSE’s Survey InstrumentThe College Student ReportThe College Student Report
Student Behaviors
Institutional Actions & Requirements
Reactions to College
Student BackgroundInformation
Student Learning &
Development
Effective Educational PracticesEffective Educational Practices
Level of Level of Academic Academic ChallengeChallenge
Active & Active & Collaborative Collaborative
LearningLearning
EnrichingEnrichingEducational Educational ExperiencesExperiences
SupportiveSupportiveCampusCampus
EnvironmentEnvironment
StudentStudentFaculty Faculty
InteractionInteraction
NSSE Project ScopeNSSE Project Scope
1,000,000+ students from 1,100+ 1,000,000+ students from 1,100+ different schools different schools
80% of 4-yr U.S. undergraduate 80% of 4-yr U.S. undergraduate FTE FTE
50 states, Puerto Rico, Canada50 states, Puerto Rico, Canada
70+ consortia70+ consortia
Documenting Documenting Good PracticeGood Practice
Institutional Institutional ImprovementImprovement
Public Public AdvocacyAdvocacy
NSSE Core PurposesNSSE Core Purposes
Customized Institutional ReportCustomized Institutional Report
OverviewOverviewInstitutional dataInstitutional dataMeans and frequenciesMeans and frequencies1st year students and 1st year students and
seniorsseniorsComparisons by peers, Comparisons by peers,
Carnegie, nationalCarnegie, nationalNational benchmarks National benchmarks Data use tipsData use tipsCD with raw data, etc.CD with raw data, etc.And more!And more!
What have weWhat have welearned so far?learned so far?
Student engagement varies Student engagement varies more more withinwithin than between than between institutions.institutions.
Supportive Campus Environment:Supportive Campus Environment: Seniors at Master's InstitutionsSeniors at Master's Institutions
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Master's Institutions
Percentile 10
Percentile 50
Percentile 90
Academic Challenge by Institutional TypeAcademic Challenge by Institutional Type
Seniors
25
30
35
40
45
50
55
60
65
70
75
DocExt
DocInt
MABacLA
BacGen Nation
First-Year Students
25
30
35
40
45
50
55
60
65
70
75
DocExt
DocInt
MA BacLA
Bac Gen
Nation
Ben
chm
ark
Sco
res
Behold the Behold the compensatorycompensatory effects of engagementeffects of engagement
Association of American Colleges and Universities
Narrow Learning is Not EnoughNarrow Learning is Not Enough——The Essential Learning OutcomesThe Essential Learning Outcomes
Knowledge of Human Cultures Knowledge of Human Cultures and the and the Physical & Natural WorldPhysical & Natural World
Intellectual and Practical SkillsIntellectual and Practical Skills
Personal and Social ResponsibilityPersonal and Social Responsibility
Integrative LearningIntegrative Learning
Integrating ideas or information from various sources
Included diverse perspectives in class discussions/writing
Put together ideas from different courses
Discussed ideas with faculty members outside of class
Discussed ideas with others outside of class
Analyzing the basic elements of an idea, experience, or theory
Essential Learning Outcome: Integrative Learning
Synthesizing & organizing ideas, info., or experiences
Making judgments about the value of information
Applying theories to practical problems or in new situations
Examined the strengths and weaknesses of your own views
Tried to better understand someone else's views
Learned something that changed how you understand an issue
NSSE DEEP LEARNING SCALE
Essential Learning Outcome: Intellectual and Practical Skills
NSSE SELF-REPORTED GAINS Acquiring job or work-related knowledge and skills
Writing clearly and effectively
Speaking clearly and effectively
Thinking critically and analytically
Analyzing quantitative problems
Using computing and information technology
Working effectively with others
Solving complex real-world problems
NSSE SELF-REPORTED GAINS Voting in local, state, or national elections
Learning effectively on your own
Understanding yourself
Understanding people of other racial and ethnic backgrounds
Developing a personal code of values and ethics
Contributing to the welfare of your community
Developing a deepened sense of spirituality
Essential Learning Outcome: Personal & Social Responsibilities
Gains in Intellectual and Practical Skills by Deep Learning Quartiles
lowest second third highest
Deep Learning Quartiles
First-Year
Senior
Very much
Quite a bit
Some
Very little
Gains in Personal and Social Responsibilities by Deep Learning Quartiles
lowest second third highest
Deep Learning Quartiles
First-Year
Senior
Very much
Quite a bit
Some
Very little
Effective Educational PracticesEffective Educational Practices
First-Year Seminars and Experiences First-Year Seminars and Experiences Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;
Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based
Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects
Effects of Learning Communities on EngagementEffects of Learning Communities on Engagement
Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.
Y-Y-Standardized Standardized
Effect sizeEffect size
Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.
Y-Y-Standardized Standardized
Effect sizeEffect size
Engagement ActivitiesEngagement Activities
Academic EffortsAcademic Efforts .16.16 ****** .32.32 .12.12 ****** .28.28
Higher Order ThinkingHigher Order Thinking .20.20 ****** .40.40 .15.15 ****** .35.35
Academic IntegrationAcademic Integration .19.19 ****** .39.39 .16.16 ****** .38.38
Active and Collaborative LearningActive and Collaborative Learning .26.26 ****** .53.53 .24.24 ****** .54.54
Interactions with FacultyInteractions with Faculty .30.30 ****** .60.60 .22.22 ****** .51.51
Diversity ExperiencesDiversity Experiences .21.21 ****** .41.41 .16.16 ****** .36.36
Perception of Campus EnvironmentPerception of Campus Environment
Quality of Academic AdvisingQuality of Academic Advising .12.12 ****** .23.23 .08.08 ****** .17.17
Supportive Campus EnvironmentSupportive Campus Environment .19.19 ****** .37.37 .14.14 ****** .32.32
SatisfactionSatisfaction .13.13 ****** .25.25 .10.10 ****** .23.23
Learning OutcomesLearning Outcomes
Gains in Personal and Social Gains in Personal and Social .24.24 ****** .48.48 .18.18 ****** .40.40
Gains in Practical CompetenceGains in Practical Competence .22.22 ****** .45.45 .16.16 ****** .36.36General Education GainsGeneral Education Gains .18.18 ****** .36.36 .11.11 ****** .24.24
First-yearFirst-year SeniorSenior
Diversity Diversity Diversity Diversity
Density Climate In Course- Diversity Density Climate In Course- Diversity
Dependent variable Index Diversity Work Press Index Diversity Work Press
Student Engagement
Academic challenge + + + Higher order thinking + + + + + +Active and Collaborative + + + + + +Diversity-related activities + + + + + + + +Supportive Campus Env.
Supportive Campus Env. + + Interpersonal - + Support for learning + + + + + +Satisfaction - + +Gains-Interpers. Dev.
Gains - Personal/social + + + + + + +Gains-Social Awareness
Gains-Understanding div. + + + + + + + +Gains -Cont. to community + + + + + +Gains-Understand self + + + + + +
First-year students Seniors
Faculty Time Devoted to Research and Faculty Time Devoted to Research and Probability of Students Doing ResearchProbability of Students Doing Research
0
0.13
0.25
0.38
0.50
Pro
bab
ilit
y o
f U
nd
erg
rad
uate
Researc
h
2.02 6.04 10.07 14.09 18.11
Faculty Research (hours)
Faculty View of Importance of Doing Research Faculty View of Importance of Doing Research with Students and Probability of Students Doing with Students and Probability of Students Doing
ResearchResearch
0.13
0.25
0.38
0.50
0
Pro
bab
ilit
y o
f U
nd
erg
rad
uate
Researc
h
1.87 2.25 2.62 2.99 3.37
Importance of Undergraduate Research
Faculty Priorities and Faculty Priorities and Student EngagementStudent Engagement
AVGAVG STUDENTSTUDENT
AVGAVG FACULTYFACULTY AcadAcad emicemic cchallhallengeenge
ActActiveive --ccollabollab
Diversity Diversity eexperiencesxperiences
StuStudentdent --ffacacultyulty
AAccadademicemic challchall engeenge emphemphasisasis
ActActiveive --collabcollab practicespractices
EmphEmph asisasis on diversityon diversity experiencesexperiences
EmphEmph asisasis on on hhigherigher orderorder thinkingthinking
ImpImportanceortance enrichingenriching ededucuc eexpxp erienceseriences
What to Make of This? What to Make of This?
1.1. When faculty members When faculty members emphasize certain educational emphasize certain educational practices, students engage in practices, students engage in them to a greater extent than them to a greater extent than their peers elsewhere.their peers elsewhere.
2.2. Good things go togetherGood things go together
Effective Educational PracticesEffective Educational Practices
First-Year Seminars and Experiences First-Year Seminars and Experiences Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;
Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based
Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects
Effective Educational Practices Increase Effective Educational Practices Increase Odds That Students Will:Odds That Students Will:
Invest time and effort Invest time and effort Interact with faculty and peers Interact with faculty and peers
about substantive mattersabout substantive matters Experience diversityExperience diversity Get more frequent feedbackGet more frequent feedback Discover relevance of their Discover relevance of their
learning through real-world learning through real-world applicationsapplications
Institutional ReflectionInstitutional Reflection
Areas of Areas of EffectiveEffective
EducationalEducationalPracticePractice
Areas of Areas of Question or Question or ImprovementImprovement
PossibilitiesPossibilities
Merge NSSE data Merge NSSE data with school recordswith school records
Identify engagement Identify engagement patterns by student patterns by student characteristicscharacteristics
Predict retention, Predict retention, degree attainment, degree attainment, grades, other grades, other outcomesoutcomes
Track student Track student engagement year to engagement year to yearyear
Compare vs. peer, Compare vs. peer, aspirational, same-aspirational, same-region schools region schools
Program evaluation Program evaluation Accreditation Accreditation Consortium and Consortium and
system data sharingsystem data sharing Student outcomes Student outcomes
researchresearch Public reportingPublic reporting A national reporting A national reporting
template?!?template?!?
NSSE VirtuesNSSE Virtues
Research basedFocus on educationally purposeful
activitiesResults point to areas where Results point to areas where
improvement is desirableimprovement is desirableCompelling face validityEstablished psychometrics Transparent operationsThird party administration
NSSE VirtuesNSSE Virtues
Random samplingTargeted samplingFlexibility: consortium question,
modulesBenchmarks for peer, national Benchmarks for peer, national
and other comparisons and other comparisons Link/integrate with other Link/integrate with other
measures -- BCSSE, FSSE, CLA, measures -- BCSSE, FSSE, CLA, MAPP, portfolios, graduation MAPP, portfolios, graduation rates etc. rates etc.
Other Data SourcesOther Data Sources
• Locally-developed measures • National instruments
– Cooperative Institutional Research Program (CIRP)
– Your First College Year (YFCY)– College Student Experiences Questionnaire
(CSEQ)– Noel Levitz Student Satisfaction Inventory– ETS MAPP and Major Field Tests– ACT Collegiate Assessment of Academic
Proficiency – Collegiate Learning Assessment (CLA)
• Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys…
• Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)
NSSE Accreditation ToolkitNSSE Accreditation Toolkit
• Links Between NSSE & Accreditation
• Mapping NSSE to Accreditation Standards
• Vignettes of Institutional Usage
NSSE Questions
1.l. l. Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment
1.m. Used e-mail to communicate with an instructor
10.g. Using computers in academic work
11.g. Using computing and information technology
SACS Standard
3.4.14 The institution’s use of technology enhances student learning, is appropriate for meeting the objectives of its programs, and ensures that students have access to and training in the use of technology.
NSSE CautionsNSSE Cautions
Only one source of information Corroborate resultsEngagement skeptics Not an outcomes measureDenial in face of less-than-
desirable resultsEschew rankings
DiscussionDiscussion
www.iub.edu/~nsse