nova evaluation report 1996-2002 nova evaluation report 1996-2003

NOVA Evaluation ReportNOVA Evaluation Report1996-20021996-2002

NOVA Evaluation Report1996-2003

Higher Education Faces Higher Education Faces Significant ChallengesSignificant Challenges

Nation at Risk (1983)Benchmarks for Science Literacy, Project

2061 (American Assoc. for the Advancement of Science 1993)

Shaping the Future (NSF 1996) NASA Strategic Plan, Implementation

Plan for Education (1999 – 2003)No Child Left Behind (2002)

Higher Education Faces Higher Education Faces Significant ChallengesSignificant Challenges

Higher education faculty are attempting to improve the effectiveness of undergraduate science and math courses.

In the past the process has been slow and the results limited.

NOVA Professional Development NOVA Professional Development for Higher Education STEM for Higher Education STEM

FacultyFacultyNASA Opportunities for Visionary Academics

(NOVA)A professional development program assisting faculty in developing innovative STEM curricula.

NOVA ObjectivesNOVA Objectives

Disseminate NASA’s pre-service education model.

Promote development of the model.Sustain the change process.Increase collaboration.Stimulate and conduct research.

Phases of the NOVA ProgramPhases of the NOVA Program

Phase I—Professional Development Workshop and Proposal Development

Phase II—Research and disseminationPhase III—NASA Field Center Program,

online course enhancementsLeadership Development ConferenceResearch and evaluation, including on-

site assessment

NOVA EvaluationNOVA Evaluation

Evaluation of NOVA 1996-2003 included extensive and diverse data.

The NOVA Evaluation Report is organized around ten central points or question.

NOVA Evaluation Instrumentation Timeline

1

1.NOVA

Documents

2

2.Pre NOVA Workshop

3

3.During NOVA

Workshop

4

4.Proposal Planning

5

5.Implementation Phase: During the year after submission

6

6.Sustaining

Phase:1 year after

proposal submission

7

7.Institutionalization Phase: 2 Years after

proposal submission

Workshop Agendas

Brochures

Workshop Books

NOVA Mission and Work Plans

NOVA Reports

NOVA Products

Background Survey

Pre-Status Survey

Pre-Effacacy Survey

EDCATS

Pre-barriers Survey

Action Plan

NOVA Proposal

Interim-Status Survey

Site Visit:Faculty Surveys

Course MaterialsFaculty Interviews

Post-EfficacyClass Observations

Administrator InterviewsSummary results

Site Visit:Student Surveys

Student focus groups

Perceptions of teachingAttitudes

toward science Teachingefficacy

Nature of Science and/or Mathematics

Final Participant

Report

Post-Post Status Survey

Post-Post Efficacy

Student Surveys

includes barriers,

internal and external impact

Dissemination Artifacts

Long-Term Status Survey

Student and Graduate Surveys

Dissemination Artifacts

NOVA Proposal Review Ranking

1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students

240 institutions attended workshops from February 1996 to February 2003.

NOVA Workshop Institutional Participants

0

10

20

30

40

50

Workshop 43 42 45 27 47 17 19

1996 1997 1998 1999 2000 2001 2002



In 44 states, Washington D.C., Puerto Rico, and the U.S. Virgin Islands.




308 multidisciplinary NOVA faculty funded from 88 institutions

Fig. 4. NOVA Proposals Submitted and Funded

Nonfunded42%(64)

Funded58%(88)

Funded

Nonfunded




308 multidisciplinary NOVA faculty funded from 88 institutions.

154 courses; over 42,000 students, increasing by 12,000 annually.

Undergraduate Students Enrolled In Or Graduated From NOVA

STEM Courses

0

10000

20000

30000

40000

50000

Students 525 1600 4550 14400 22000 31500 42500

1996 1997 1998 1999 2000 2001 2002




308 multidisciplinary NOVA faculty funded from 88 institutions.

154 courses; over 42,000 students, increasing by 12,000 annually.

Institutions range from Doctoral/Research I to Associate of Arts Community Colleges.

NOVA Institution Classification by Percent - Funded

R-I15%

R-II11%

MA-I40%

MA-II6%

B-I9%

B-II17%

AA2%

2. Dissemination of the NOVA Model2. Dissemination of the NOVA Model

23 Phase I Workshops.Over 150 journal articles, reports, and

conference presentations.Research-supported best practices

shared within and between institutions.Over 100 NOVA-related national,

regional, and local workshops.

A Sample of National ConferencesA Sample of National Conferences

AAPT AERA AIAA ASEE AETS ATE ACS AAAS NCTM NARST

SCST QEM International Conference on

Mathematics/Science Education & Technology

12th International Conference on College Teaching and Learning

Project Kaleidoscope Conference


Non-NOVA faculty in over half of NOVA institutions implemented NOVA-type change.

Clone and NOVA-like spin-offs.Administrators reported faculty

incorporating inquiry-based activities into non-NOVA courses.

Springboard to other grant writing.A third of non-funded Phase I teams made

changes based on NOVA model.


Evolution over 8 years from innovative institution-based courses to a national dissemination and collaboration network.

Personnel, data, and materials from NASA Enterprises incorporated in a wide population of higher education institutions and students.

3. Congruency of the NOVA Model with 3. Congruency of the NOVA Model with Major Reports on Teacher PreparationMajor Reports on Teacher Preparation

Eight major elements aimed at changing how science and math are taught in K-12 classrooms by influencing “how” content is taught at the college level.


1. Continuous professional development. 2. Sustained collaboration among

interdisciplinary teams of faculty and administrators.

3. Courses based on national standards for science, math, and technology.

4. Courses utilize content based on NASA Strategic Enterprises.


5. Program improvement at NOVA institutions through ongoing research.

6. Instructional strategies that are inquiry based and center on student interaction.

7. Extensive use of technology.

8. Collaborations among institutions in the NOVA network.

4. Participant Reactions to the NOVA 4. Participant Reactions to the NOVA Professional Development ModelProfessional Development Model

PositiveUseful in developing innovative

pedagogy.Helped develop skills in technology and

curriculum innovation.Increased confidence and expectations.

Reactions to the NOVA program:


Introduce innovative teaching and/or STEM courses.

State education requirements, content standards.

Help validate already innovative faculty.Recruit other faculty into a network of

innovation.

Reasons for participating in NOVA:


Team approach.Commitment to the project.Belief in the model.Use of NASA resources.Monetary support.

Important factors facilitating change:

5. Impact of the NOVA Model on the 5. Impact of the NOVA Model on the Collaborative Work and Organizational Collaborative Work and Organizational

Climate of STEM FacultiesClimate of STEM Faculties

Increase in type and expertise of research-supported teaching practices

Other funding or grant opportunitiesNew department programsProfessional disseminationBreadth of expertise, experienceFaculty initiative, resolution of problemsCampus climate focused on student learning

6. Effect of the NOVA Model on Higher 6. Effect of the NOVA Model on Higher Education CoursesEducation Courses

Chemistry, physics, physical science

16%

Earth and space science

21%

Methods5%

Engineering4%

Biology, life science, life in

space, environmental

science24%

Nature of science, inquiry

science, integrated

science10%

Mathematics, Science, and Technology

20%

6. Effect of the NOVA Model on Higher 6. Effect of the NOVA Model on Higher Education CoursesEducation Courses

Majority of courses at the freshman or sophomore entry level.

NOVA Phase I Course Levels by Percent

200-Level18%

400-Level7%300-Level

13%

500-Level1%

100-Level61%

7. Impact of the NASA Mission, Data and 7. Impact of the NASA Mission, Data and Information, and Fundamental Questions on Information, and Fundamental Questions on

the NOVA Coursesthe NOVA Courses

Multiple connections to NASA Strategic Enterprises.

Used by NOVA students and carried into their K-12 classrooms.

Shared between colleagues.Disseminated on institutional web sites.Presented at national conferences.

Percent of NOVA Courses Making Connections with each NASA

Enterprise

ESE37%

HEDSE19%

SSE25% BPRE

1%

ATE11%

05101520253035

Number of Institutions

5 4 3 2 1

Number of NASA Strategic Enterprises

Multiple NASA Entrprise Connections

7. Impact of the NASA Mission, Data and 7. Impact of the NASA Mission, Data and Information, and Fundamental Questions on Information, and Fundamental Questions on

the NOVA Coursesthe NOVA Courses

Multiple connections to NASA Strategic Enterprises.

Used by NOVA students and carried into their K-12 classrooms.

Shared between colleagues.Disseminated on institutional web sites.Presented at national conferences.

8. Aspects of NOVA Professional 8. Aspects of NOVA Professional Development Model that are Effective in Development Model that are Effective in

Creating and Sustaining ChangeCreating and Sustaining Change

Continuous faculty interaction.Administrative support.Sense of common purpose and trust.NOVA structural supports made possible

and promoted change processes.Faculty action (practitioner) researchDevelopment and sustaining of teams - a

key factor in accomplishing goals.

9. Impact of the NOVA Model on Classrooms 9. Impact of the NOVA Model on Classrooms and Studentsand Students

Faculty action research found significant improvement in student content knowledge, attitudes, and inquiry skills.

Students had a higher efficacy to teach K-12 science.


Decrease in traditional classroom lecture and memorization approaches

Greater use of an inquiry approach in teaching

Emphasis on interactive, student-centered methods

Use of NOVA courses as models for developing additional courses and for meeting state higher education guidelines


Undergraduate education majors’ science teaching efficacy, content mastery, and teacher performance positively affected.

State standardized achievement test scores in science higher for students of NOVA courses.

Studies on impact of NOVA courses on students:

Students experienced science and math as a process rather than being told about content.

Integration of lecture and lab promoted experiential, inquiry learning.

Increase in students’ STEM content knowledge.

Female and minority interest in NOVA courses increased.

10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students


Higher end-of-course growth and understanding than students in similar “non-NOVA” courses.

Two national studies on student understanding of the nature of science and mathematics:

VNOS-I

05

101520253035

ExperimentalStudents

ComparisonStudents




Increased interest and lowered anxiety about this content in their professional and personal lives.


Results related to students’ ability to succeed in science and math-related tasks and careers.



Increased interest and lowered anxiety about this content in their professional and personal lives.

NOVA’s impact on higher education—


NOVA’s impact on higher education—

Important to professionals and our educated citizenry in a democratic society.


nova evaluation report 1996-2002 nova evaluation report 1996-2003

Documents

multidisciplinary nova

nova model

students240 institutions

nova professional development

nova evaluation report1996

novarelated national

puerto rico

local workshops