ascp workshop 04june04 la076 sdsu evaluation & assessment in the undergraduate curriculum...

ASCP Workshop 04June04 LA076 SDSU

Evaluation & Assessmentin the Undergraduate Curriculum

Assessment as Collaboration

Kris Stewart

NPACI Education Center on Computational Science and Engineering

San Diego State University

http://www.edcenter.sdsu.edu/

Professor, Computer Sciencehttp://www.stewart.cs.sdsu.edu/


What is Computational Science?

Science DisciplinePhysics, Chemistry, Biology, etc.

Computer ScienceHardware/Software Applied Mathematics

Numerical Analysis, Modeling, Simulation

Computational ScienceTeamwork and Collaboration


What is Computational Science?Recognized as separate discipline

E.g. Society for Industrial and AppliedMathematicshttp://www.siam.org/cse/and Osman Yassar,IEEE Computing in Science & Engineeringhttp://www.cps.brockport.edu

Should CSE be a separate major?(Computational Science [no, too general] or

Computational Physics [yes, natural extension])Should it be a separate department?

(How to support interdisciplinarity?)


• Numerical Analyst* led to• Supercomputing and Undergraduate Education

(SUE**) led to• Supercomputing Teacher Enhancement Program

(STEP***) led to• Education Center on Computational Science &

Engineering (EC/CSE)

Kris’ Faculty Background(Kris Stewart, Director, San Diego State University, California State University System)

*MS/CS SDSU 1979, JPL 1981, PhD UNM 1987, SDSU 1986** UCES (DoEnergy 1994) *** Smithsonian Research Collection (1996)


UCES Paradigm(thanks Tom Marchioro and the “crew”, 1994?)

Previous exposure to “assessment”


1998/99 Assessment by LEAD

Background• Workshop in Wisconsin April 1997 to learn

about assessment and make it real• NPACI starts 1 October97• EC/CSE requested assessment for 1998

Assessment as a Collaboration

Preparation for SDSU Campus Visit by Baine & Julie• Discussions at SC98- SuperComputing Fall98 Orlando• Email to SDSU faculty gather attitudes


Grand Challenges for HPC (Stewart & Zaslavsky, SC98)

1. Faculty system of rewards does not encourage teaching innovations

2. Lack of awareness of HPC technologies already used in research or teaching for different fields

3. Faculty & students unaware of benefits and accomplishments of HPC

4. HPC technologies considered too complex/inaccessible for undergraduate instruction

5. Sequential HPC-related curricula is absent 6. Curricula using very large data sets not widely available 7. Adjust to different learning styles when material is complex 8. Variety of platforms/software leads to fragmented curricula 9. School administration/support staff not ready for HPC 10. Specs of computers and networks below user expectations

We had been thinking about this (based on April 97 LEAD Workshop in WI)


Building the Community of Faculty

• These challenges are people-centric, not technology-centric and of interest to the community

• Systemic Change requires understanding the system and working within it

• Empower teachers (find the time), recognition (from chair/dean), support (student assistants)


Interviews on SDSU CampusLEAD applying info from email surveys (Spr 99)

Faculty skepticism:

• Convincing evidence that computer-based tools enhance teaching process?

• Knowledge of modern computational methods and availability?

• Incentive from department and insufficient tech support?

LEAD Interviews with V.P., Deans, Chairs

Faculty Fellows program identified as a target

OUTCOMES:

• our local infrastructure at SDSU took us more seriously

• survey instruments refined

• SWB value recognition


Assessment not just requirement

Rather, found to be• vital tool to assist in clarifying student and faculty

needs• improve prioritization skills• validation of focus on human factors to integrate

HPC (modeling & visualization) into undergrad curriculum


California Education Infrastructure

• K12 Education (standards based, performance based)

• Community Colleges (Freshman/Sophomore)– Vocational– University preparation

• California State University System (23 campuses)• University of California (9 campuses, Merced in

2004)• Independents

(Stanford U., CalTech, U. Southern California)


California Education Infrastructure(testbed for change)

IMPAC http://www.cal-impac.org/Intersegmental Major Preparation Articulated Curriculum ProjectCommunity Colleges and Four Year Universities

Charter Schools (Preuss School) preuss.ucsd.eduDr. Rozeanne Steckler and Dr. Mike Bailey“Fostering Scientific Curiosity in All Children”San Diego Supercomputer Center


Involving University FacultyInfrastructure for Change

• NPACI/SDSU Faculty FellowsLocal Support from College Deans and Department Chairs (participation buy-in and faculty recognition)

• SDSU Academic Advisors (across disciplines)• Professional Meeting (SC2001, SIAM, ACM,

SIGCSE, your suggestions?)


Grand Challenges for HPC (Stewart & Zaslavsky, SC98)

1. Faculty system of rewards does not encourage teaching innovations

2. Lack of awareness of HPC technologies already used in research or teaching for different fields

3. Faculty & students unaware of benefits and accomplishments of HPC

4. HPC technologies considered too complex/inaccessible for undergraduate instruction

5. Sequential HPC-related curricula is absent 6. Curricula using very large data sets not widely available 7. Adjust to different learning styles when material is complex 8. Variety of platforms/software leads to fragmented curricula 9. School administration/support staff not ready for HPC 10. Specs of computers and networks below user expectations


Building the Community of Faculty

• These challenges are people-centric, not technology-centric and of interest to the EPSCOR community

• Systemic Change requires understanding the system and working within it

• Empower teachers (find the time), recognition (from chair/dean), support (student assistants)


Faculty Fellows Fall 01 Synergyamong themselves and with their chairs and deans

People, Time, Support, Recognition …


NSF/EHRNational Science

Foundation/Education and Human Resources

Directorate

www.ehr.nsf.gov/EHR/RED/EVAL/handbook/

handbook.htm

LEADAssessment and

Evaluation1998 Formative for the

EC/CSEhttp://www.cae.wisc.edu/~lead/pages/produts/eot-

paci.pdf


EC/CSE involve CSU

Faculty Workshops

• Introduce Computational Science

• Host Biology Workbench (BioQuest)

Support at NPACI AHM02

Extend Faculty Fellows


Contact Us

Kris Stewart, Directorhttp://www.edcenter.sdsu.edu

[email protected] Sale

[email protected] Kirsten Barber

[email protected]


The Ed Center Project and its Evaluation

• Goal: incorporate computational science techniques in undergraduate curricula

• Differences from STEP: target audience, institutional arrangements

• Methods and Strategies:• Faculty Fellows• Workshops, presentations, in-house projects, trying out new

approaches in own teaching, tools development• Computational Science Olympics (curriculum from the bottom-

up!!)• Surveys, continuous assessment of faculty involvement and

learning outcomes• Evaluated by the LEAD Center in 1998-99


Introducing the EC/CSE

The mission of the Ed Center on Computational Science and Engineering? www.edcenter.sdsu.edu

Who are the people involved? www.edcenter.sdsu.edu/staff

Some of our projects: www.edcenter.sdsu.edu/projects/

Some of our activities: www.edcenter.sdsu.edu/news/

Some resources: www.edcenter.sdsu.edu/repository


Undergraduate Faculty: A Tough Target Group

• Obstacles: lack of time, tenure and review considerations, lack of awareness about available technologies

• Undergraduate faculty: – ¾ have used WWW often or sometimes (1997), but not in the

classroom (only 18% - 1998)– The gap between those NEVER using computers in the classroom,

and those using them OFTEN, is the largest for untenured faculty, increasing towards tenure review

– Only 12% of surveyed faculty saw themselves as having a use for HPC applications in courses (higher for Sciences and Engineering)

– 11% of faculty have students working with computer models OFTEN


Using computers in the classroom versus number of years as a faculty member (1997 Faculty Survey)


Primarily graduate faculty Primarily undergraduate faculty

College of Arts & Letters College of Sciences

1. 1 OFTEN2. 2 SOMETIMES3. 3 RARELY4. 4 NEVER5. 9 DK/REF (Missing values)

Students Using Computers in the Classroom (1997 Faculty Survey)


Strategies for Building Faculty Community

• Reliance on most enthusiastic and technically advanced instructors who are already using computing and modeling in classes

• The Faculty Fellows program:• Stakeholders:

– College Deans - Specific support

– Faculty - Compensation, and acknowledgement, of the value of the faculty members contribution

• Benefits – College

– Department (Faculty Fellows as discipline-specific spokespersons for EC/CSE)

– Faculty (as individuals)

– Ed Center on Computational Science and Engineering

– Building a special infrastructure for curriculum transformation: human, institutional, technical – is a requirement for successful introduction of advanced techniques (since they are more demanding on faculty time and efforts)

The problems and strategies are not that much different from STEP!!


Faculty Fellows during 1998-2000

• Faculty Fellows representing four departments from four colleges: Geological Sciences, Geography, Computer Engineering, Business Information Systems

• Bi-weekly meetings at the Ed Center• Faculty Fellows as “ambassadors” of computational

science• Partnership with LEAD for evaluation during 1998-99• Follow-on Activities (Susan Millar, LEAD)

– CATS (Classroom Assessment Techniques)– FLAG (Field-tested Learning Assessment Guide)– SALG (Student Assessment of Learning Gains)


More Strategies

• Trying the new approaches in our own teaching:– Teaching the Supercomputer class with group-based

problem-solving environments

– Real-time distance teaching with Web-based collaborative software (featured as Microsoft Case Study in Higher Ed.)

– Development of Sociology Workbench, a free on-line survey data analysis system that can be used for evaluation of student outcomes and other surveys


Channels for Influencing Pre-Service Teacher Preparation

• Use of advanced computing modules in general ed courses (e.g. Geol 303 “Natural Hazards”)

• Cooperation with College of Education faculty and students, esp. in Education Technology: focus on experimentation with new technologies in classroom setting

• Computational Science Olympics: supporting the “bottom-up” development of computational science curricula

• Providing on-line assessment technologies:– Sociology Workbench: http://edcenter.sdsu.edu


SWB Convenient Tool to Learn from Student Survey Data

• Online tool for “standard public data sets” or your own data set http://edcenter.sdsu.edu

• Small Sample, therefore only useful as feedback for the instructor

• Can be used with “forms” interface directly into SWB format, as in June ‘99 CSU Faculty Workshop


SWB as Analysis ToolView Student Comments (text)

•


SWB as Analysis ToolIsolate on Specific Survey Response

•


•

SWB as Analysis ToolExplain the Response on Learning with “doing more”


SWB as Analysis ToolExplain learning with “active participation”


Lessons Learned

• Institutional support required for program to be sustainable

• Individual reform-ready faculty is focus for support

• Infrastructure:– Build a Synergistic Environment (across disciplines)

for Faculty

– Continuous monitoring through interviews, surveys, discussions


What’s Next

• The approaches we described proved useful for two target audiences. We believe that the strategies and lessons learned can be extrapolated in a targeted effort to incorporate computational science technologies in pre-service teacher preparation

• This may be a funding opportunity?• We will be happy to contribute our knowledge and

share experiences…


References ASCP Workshop

•User-Friendly Handbook for Project Evaluation: Science, Mathematics,

Engineering and Technology Education, NSF 93-152 www.ehr.nsf.gov/EHR/RED/EVAL/handbook/handbook.htm

•Sid Karin: The Importance of Science Literacy in a Computing World (see

enVision Science Magazine, V.15 No. 2

•K. Stewart & I. Zaslavsky, “Ten Grand Challenges”, Supercomputing ’98 Orlando FL SC98 Conference Paper

•J. Foertsch & B. Alexander, “Integrating HPC into the Undergraduate Curriculum”, report by LEAD Center June 1999 http://www.cae.wisc.edu/~lead/pages/products/eot-paci.pdf


Evaluation & Assessment

• U. Wisconsin LEAD for 1998/99 Ed Center evaluation

• www.cae.wisc.edu/~lead/pages/products/eot-paci.pdf

• Follow-on Activities (Susan Millar, LEAD)CATS (Classroom Assessment Techniques)FLAG (Field-tested Learning Assessment Guide)SALG (Student Assessment Learning Guide)


Evaluation and Assessment of Classroom Practice

Where to start?

• User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education, NSF 93-152 www.ehr.nsf.gov/EHR/RED/EVAL/handbook/handbook.htm

• Student Surveys - Need a compatible tool for instructor to examine results with

• Sociology WorkBench (SWB) developed by team of undergraduate computer science majors employed by the EC/CSE


Information Overload

• How can teachers help out?

• How can technology providers help out?

• Ans: personalize the linkage.


Information Overlab Lab

• Using the WWW extensively in the classroom, implies obligation to discuss Information Overload

• http://www.stewart.cs.sdsu.edu/infolab/index.html

Timeline of Technology

http://www.stewart.cs.sdsu.edu/infolab/timeline_tech.html


Computer Network Security

• Campus-wide “buy-in” recognizing

the use of high-speed networks

the responsibility for “secure network use”

• SDSU Academic Senate adopts interim policy• UCSD Acceptable Use Policy useful start


A Little History - May 10, 1869

• What was the major event?

• Http://memory.loc.gov• Thank you to the US Library of Congress for the

on-line exhibit preserving our “Memory”


Promentory Point, UTAH

•


How does this relate to us today?


Technologies to Motivate Learning

• Mr. Sid uses Wavelets for compression of Railroad Maps

• Sense of History must be tied to personal, student- based, “stories”

• University Faculty need encouragement this will “count” for professional recognition by their peers and administrators (Deans…)


ONLINE Resources for Undergrads

• The Education, Outreach and Training (EOT) of the PACIs http://www.eot.org

• feature several projects

• Ed Center on CSE • Shodor

Undergraduate Computational Physics www.orst.edu

• Minority Institutions, a Directory

• Student Biology Workbench (Bioinformatics)

• Underrepresentation in Computer Science (CDC)


Learning Technologies from the PACIs

• Data Intensive Computing Environment

• Digital Video/eTEACH

• Interaction Environments: Bays to Brains

• Neuroscience: Telescience/Tomography

• NISE College Level One Institute on

• Learning Technology

• Remote Collaborative Learning

•www.eot.org


This is Only the Beginning...

YOU ARE HERE

TIME


What’s Next

• Your comments and thoughts?

• SIAM Computational Science & Engineering Conference – Sept. 2000 – Washington D.C.

• SC2000 – November 2000 – Dallas TX


• NPACI Partnership Report• enVision quarterly science

magazine, especially June2000“Why the Future May Very Well Need Us”

• www.npaci.edu/enVision/director.html

• “Online” biweekly electronic publication, www.npaci.edu/online/

• www.npaci.edu

NPACI Sources of Information


References - EPSCoR - July 2000• This Presentation• http://www.stewart.cs.sdsu.edu/PPT/

epscor_20july00.ppt• Ed Center on Computational Science & Engineering• http://www.edcenter.sdsu.edu/

• Timeline of Technology Development

• http://www.stewart.cs.sdsu.edu/infolab/timeline_tech.html

• Education, Outreach and Training

• http://www.eot.org• Information Overload • http://www.stewart.cs.sdsu.edu/infolab/index.html

ascp workshop 04june04 la076 sdsu evaluation & assessment in the undergraduate curriculum...

Documents