ABSTRACT

A scenario-based assessment tool used in twoenvironmental geoscience in-service programs for middleschool and high school teachers served both to guideinstructional techniques and as a method to evaluate thesuccess of the instructional approach. In each in-serviceprogram the participants were assessed at an initialmeeting to determine their understandings ofenvironmental problems. The assessments included ahypothetical watershed scenario in which participantshad to choose water quality monitoring sites, monitoringparameters, and provide justification of their choices. Theresults of these pre-assessments provided staff with anunderstanding of participant misconceptions that couldbe addressed specifically in program activities. In eachprogram the participants were given the same assessmentafter participating in the workshops and this revealedsignificant shifts in certain areas of their understanding ofwatershed quality concepts and monitoring approaches.

Good assessment scenarios are complex and arebased on accurate information or data that are used to an-swer a question or prompt. They are particularly revealingand valuable because they ask students to make decisionsby applying their understandings using information ordata. Development and use of scenario-based assessmentin this program came about only because of genuine andsustained cooperation between science education facultyand geoscience faculty. This highly beneficial collabora-tion was key to providing quality training to participatingin-service teachers, and development of a synergisticteaching and learning environment.

Keywords: education - assessments; assessments - sce-narios; education - in-service; education - in-quiry learning; education - issues-basedlearning; environmental geoscience

INTRODUCTION

To most of us who teach in the geosciences, assessmentrefers to the tests and assignments we use to assigngrades to students. Well-designed assessments,however, can be much more useful both in terms of the

information they provide about student learning, and astools to improve teaching. In addition to determininghow much improvement in students’ understandings isaccomplished through a class, assessments can be usedto determine the initial understandings of the students,any misconceptions about the topic that exist, and howthe student thinks about a topic. All of these pieces ofinformation can be used to design more effectiveinstruction. Just as a research endeavor begins by firstdetermining the state of previous research on that topic,an initial assessment of students’ understandings isnecessary to provide quality and efficient instruction(Voska and Heikkinen, 2000). However, producing agood instrument to assess students’ understandings ofearth science concepts, problem solving abilities, andthought processes is a challenge for which mostgeoscientists are ill-trained and unprepared.

Much work has taken place at Purdue Universityto incorporate sound educational practices andeducational research into existing and new geoscienceclasses (e.g., Harbor, 2000; Trop, et al., 2000; Krockover,et al. 2001). One of the most productive approaches hasbeen cooperative work between faculty in theDepartment of Earth and Atmospheric Sciences (Schoolof Science), and the Department of Curriculum andInstruction (School of Education). This cooperationbetween science faculty and faculty in science educationseems obvious, and is especially appropriate inenvironmental geoscience classes because of the strong(public) education component in all environmentalissues, and because of the environmental emphasis ofearth science classes taught expressly for educationmajors. This collaboration has also extended to jointprojects to develop geoscience based educationalprograms for in-service and pre-service teachers.

In this paper we describe an assessment used intwo programs for middle school and high schoolteachers held at Indiana University/Purdue Universityat Indianapolis (IUPUI) and at Purdue University’s maincampus in West Lafayette, Indiana. These two programswere similar in content and identical in the way theywere assessed, so results from both are combined in thisdiscussion. In each program, the participants wereassessed at the initial meeting to determine theirunderstandings of environmental problems related to ahypothetical watershed. These assessments werecompared to teacher assessments completed after

64 Journal of Geoscience Education, v. 50, n. 1, January, 2002, p.64-71

ASSESSMENTS AS TEACHING AND RESEARCH TOOLS IN AN

ENVIRONMENTAL PROBLEM-SOLVING PROGRAM FOR IN-SERVICE

TEACHERS

Barbara C. Cooper Department of Earth and Atmospheric Sciences, Purdue University, West

Lafayette, IN 47907-1396, bccooper@purdue.edu

Daniel P. Shepardson Department of Curriculum and Instruction, Purdue University, West Lafayette, IN

47907-1442, dshep@purdue.edu

Jonathan M. Harber Department of Earth and Atmospheric Sciences, Purdue University,West

Lafayette, IN 4790701396, jharbor@purdue.edu

participating in the workshops, to determine how theirunderstandings had changed. We also used theinformation about teacher understanding gained fromthe assessment in modifying workshop activitiesspecifically to address teacher misconceptions.

BACKGROUND

Over the past twenty to thirty years,environmental issues and problems have become moreimportant in geoscience classrooms. To understand andsolve environmental problems people need to have anunderstanding of the geologic processes that take placein the near-surface environment, and thus, there is clearrationale for including geoscience material inenvironmental courses. Geology departments areincluding more environmental classes in their curricula(Ulanski, 1995), and in some cases are creating newmajors with a specific focus on environmentalgeosciences. Purdue University’s environmentalgeoscience major, developed in 1993, includes acapstone investigative class for seniors focused on localenvironmental problem solving (Harbor, 2000). Thiscapstone class adapted for training in-service middleschool and secondary science teachers by augmentingthe inquiry and interdisciplinary science components ofthe class with activities and discussions on curriculumdevelopment and assessment, so that teachers couldeffectively integrate this approach in to their owncurricula and classroom instruction.

There are several reasons why we chose to extendthis program to in-service teachers. One of the oft-statedgoals of environmental education in the United States isto create an environmentally “literate” population.(Mancl, et al. 1999; Roth, 1991). However, in order to dothis, the teachers who have responsibility for educatingour children must not only be trained in pedagogy, butmust be comfortable with the many interdisciplinarysciences that are associated with environmentalproblems and issues. Thus, in addition to addressing theeducational needs of geoscience majors, it is importantfor geoscience departments to provide relevantenvironmental geoscience education for pre- andin-service teachers.

Using local environmental issues as the focus of aninvestigation (Harbor, 2000) is one good way to includepractical experience in an environmental program. Thisapproach can be used both with science majors and withpre- and in-service teachers. Teacher educationprograms that emphasize the practical experience ofperforming a real-life environmental investigation(Robottom, 1990), and that include science contentknowledge, model sound environmental educationapproaches that can be used by the teachers in their ownclassrooms. In this way the in-service teachers whoparticipate are taught not only the science componentsof environmental geology, but also how to incorporate

these components successfully into their own classroomsituations.

Two separate programs were funded to improveenvironmental science skills for in-service teachers (seeacknowledgements). The first program was for one yearand took place during the1999-2000 academic year atIndiana University/Purdue University at Indianapolis(IUPUI) and the latter, now named ENVISION, is amulti-year program at the main Purdue Campus of WestLafayette, Indiana that began in 1999 and will continuethrough 2004. Each program evaluated here consisted ofa short spring meeting, a 2-3 week summer institute, andfollow-up meetings in the fall and spring. Theparticipants met for one weekend in the spring for initialassessment and to start work on field investigations,curricular review and development, and sciencecontent. The summer program involved the participantsin two to three intensive weeks of science investigation,writing and reporting, and curriculum integration. Infollow-up meetings the participants reported back totheir peers about the successes and challenges they hadexperienced in their own classrooms. The participantsincluded science teachers from many parts of theMidwest, including teachers of chemistry, biology, earthscience, general science, environmental science, andecology in middle schools and high schools.

The purpose of this paper is to discuss theassessments used in these two programs for the1999-2000 year. Detail descriptions of the programsthemselves can be found in Shepardson, et al. (in press).

THE ASSESSMENTS

A critical element of the in-service programs, and thefocus of this paper, is assessment. In order to efficientlyprepare instruction for the teachers participating in theworkshop, it was first necessary to assess their level ofunderstanding of key environmental concepts. Ourgoals also included assessing how well the teachingmethods utilized in this instructional environment wereworking, and modeling for the teachers how assessmentcould be integrated into an instructional program. Aparticular challenge in this program was to deviserealistic assessments for the range of activities includedin the institute, including improved content knowledgeand practical research skills.

We developed assessments aimed at elicitingparticipants’ ideas, understandings, and misconceptionsin several areas of environmental geoscience. Severaltypes of assessment were used to determine theteachers’ understanding of underlying science, as wellas to reveal the thinking processes and problem solvingtechniques they used. One of the most useful assessmenttools was a constructed-response assessment in the formof a stream monitoring scenario (Figure 1).

Constructed-response assessments requirestudents to create a product that demonstrates their

Cooper et al. - Assessment as Teaching and Research Tools 65

66 Journal of Geoscience Education, v. 50, n. 1, January, 2002, p.64-71

Figure 1. Stream Monitoring Scenario Assessment Instrument Directions: Complete the scenario to the best

of your ability, based on what you currently know.

Cooper et al. - Assessment as Teaching and Research Tools 67

understandings (McTighe & Ferrara, 1998). Assessmentscenarios are a form of constructed-response in thatstudents respond in writing to the situation described inthe scenario. Our scenario required students to proposea plan for monitoring stream quality, based on thedata/information provided in the scenario.

ASSESSMENT SCENARIOS

What Is an Assessment Scenario? - In an assessmentscenario, the student is given a problem scenario thatcould occur in a real life situation. The informationsupplied can be made as simple or complex as thestudents’ ability level permits. The student is asked toperform tasks and make interpretations using theinformation, and is assessed on the type of processesused in coming up with responses, and on the content ofthe answers.

The scenario chosen for the class of in-serviceteachers concerned a stream flowing through awatershed that included several different types of landuses common in the Midwest (Figure 1). Theinstructional program included a field trip and exercisesaddressing similar situations and dealing with the samediverse group of potential water contaminants as usedin the scenario. In the assessment, the students couldpropose to measure or test the water for any five of a listof parameters, at four locations along the stream. Theywere to choose the parameters and the samplinglocations, and to explain the reasoning behind theirchoices. Herman, Aschbacher, and Winters (1992),Hymes, Chafin, and Gonder (1991), Perrone (1991), andShepardson (2001) have identified criteria for guidingthe development of alternative assessment tasks. Ourassessment scenario was developed in alignment withthese design criteria.

Assessment Scenarios as Diagnostic andSummative Tools - This stream monitoring assessmentwas used for both diagnostic and summative purposes.It was given as a pre-test, allowing its use as a tool fordesigning instruction (diagnostic, based on studentsprior knowledge and conceptions), and as a post-test toassess the (summative) impact of the educationalprogram. Inductive analysis (Patton, 1990) was used toidentify patterns in the teachers’ responses to the streammonitoring scenario. Each assessment was individuallyreviewed by two of the authors who then discussed theirreviews, negotiating codes that reflected specificelements in teacher responses. The assessments werethen coded and patterns in codes were clustered to formcategories that reflected the teachers’ understandings.Thus the codes and categories emerged from the data,rather than from any pre-existing or predeterminedcodes or categories (deductive analysis) (Patton, 1990).

The pre-test information was used to makedecisions about modifying the program curriculum and

instructional activities. For example, it was apparentfrom the pre-test that the students had a poorunderstanding of the need to collect base-line data for astream, and to collect information on the sameparameters upstream and downstream of a possiblesource of pollution. These points were reinforced inseveral ways in the instructional program. If thestudents had demonstrated a good understanding ofthese concepts, they could have been less stronglyemphasized in the course.

Why Assessment Scenarios? - Assessment scenariosreflect the National Research Council’s (NRC) emphasesin assessment, which include:

Assessing what is most valued in scienceAssessing rich, well-structured knowledge andunderstandingAssessing scientific understanding andreasoningAssessing to learn what students understand(NRC, 1996).

Different types of assessments measure differentaspects of what students know and can do (Baxter &Shavelson, 1994; Ruiz-Primo & Shavelson, 1996). Ourassessment scenario emphasizes understanding,research design, and reasoning through a situation tosolve a problem.

Assessment scenarios elicit students’ ideas orunderstandings and engage students in thinking andapplying scientific ideas to problem situations or issuesthat they can see are realistic. They require students tocommunicate their ideas, understandings, and thinking.Such scenarios also require students to make decisionsabout how to complete the task. If students are learningthrough tackling real world situations, scenarios matchor reflect these situations and provide authenticity to theassessment (Schwert, 1999). In cases where the studentsdo not learn through working on real world situations,real world scenarios provide a measure of how well astudent can translate knowledge gained in other ways tothe solution of real problems. Finally, assessmentscenarios elicit a better picture of students’ ideas andunderstandings and provide more reliable evidence oftheir scientific understandings. Assessment scenariosused in a pre-test and post-test fashion provide a pictureof the changes in students’ ideas and understandings,that in turn can allow for changes in the instruction thattakes place.

PRE-TEST RESULTS

In both the summer workshops the pre-test scenariosshowed that:

68 Journal of Geoscience Education, v. 50, n.1, January, 2002, p.64-71

� a few of the participants were confused aboutstream flow directions

� many were unsure what tests to use to determinespecific types of contamination, and

� most were confused about how to attain “base-line”data on the stream

� most had a poorly defined concept of “acceptable”levels of contaminants, thinking all contaminantswere bad.

By evaluating pre-test scenarios, we were able to targetthe instruction toward addressing misconceptions anduncertainties. For example, we added emphasis andexplanation both in field trips and in-class activities ontypes and levels of contamination, and strategies formonitoring to obtain base-line data. By comparingpre-test with post-test results, we could determinewhether that the instruction we provided was effective.

Developing Assessment Scenarios - The main stepsin developing an assessment are:

� Identify the purpose for the assessment scenario.� Identify the key concepts, ideas, and skills to be

learned and assessed.� Obtain data or information from a real situation that

reflects the key concepts, ideas, and skills, or create ahypothetical data set that matches a real worldsituation.

� Write the scenario.� Check the scenario to ensure that it aligns with the

key concepts and instruction.� Develop the scoring system.� Field test the scenario and revise where necessary.

An important, but often overlooked aspect ofgood assessment scenarios is their alignment toinstruction and student experiences. A well-writtenscenario that is poorly aligned with instruction willresult in poor student performance and unreliableinformation about what students know and can do orhave learned. Thus, a first step in writing scenarios isidentifying the key concepts or ideas students are tolearn. For example, in Figure 1, the scenario choseninvolved concepts of source identification andmonitoring, and specific land uses and contaminationproblems investigated in the program. The next step is toidentify real world situations and data to draw from inwriting the scenario. The scenario itself does not have tobe a real world situation, but should be based on realworld situations or scientific principles. Real situationsmay also be enhanced with the incorporation ofhypothetical details and/or data. We chose a scenariodepicting land uses and contamination issues common

in Northern Indiana (See Figure 1), yet it is ahypothetical situation.

Good assessment scenarios are complex andbased on accurate information or data that are used toanswer the question or prompt; they allow students tomake decisions by applying their understandings andusing information or data. If real data are not availablethen the data used in the scenario should mirror the realworld as well as possible. It is not necessary to write alengthy scenario. The prompt facilitates the manner inwhich students complete the task, how they analyze thedata or information, and how they respond. Morecomplex scenarios would incorporate differentperspectives from which students could complete thetask. For example, in our stream monitoring scenario(Figure 1) students complete the task from theperspective of an environmental consulting company.We also ask them to do the testing under constraints oflimited assets, like those they would encounter in a realworld situation where information must be purchasedwith limited funds. A different perspective mightrequire students to respond as an employee or biologistof the pork producer who wants to expand hisoperation. This perspective might result in a differentapproach to completing the assessment scenario:different monitoring sites and different parameters.

The scenario narrative needs to contain adescription of the issues and context, and the data orinformation the students use in completing theassessment. In addition, individuals or groups withdifferent perspectives could be included in the narrativeto increase the complexity of the scenario. Incorporatingdifferent stakeholders or perspectives increases thecomplexity of the assessment, as does an increase in thedata or information available for students to use.

The scenario may be written in an open or closedcase format (Shepardson, 2001). Open case formatsprovide students with less structure and more freedomin completing the assessment. This provides studentswith more opportunities to think about the scenario,providing a more in-depth look at students’understandings (the way they think, rather than the waythey respond to the teacher’s thinking). Closed caseformats provide more structure in terms of howstudents complete the task, reducing the ways in whichthey approach solving the problem in the scenario. Italso reduces the decision-making processes in whichstudents engage, but provides a clear focus forevaluation. Our stream monitoring scenario (Figure 1) isan open format in that students have to identify thestream monitoring sites and the parameters, as well asprovide the rationale for the sites and parameters. Theyhave to make decisions about how to complete thescenario. The scenario could be made more closed byindicating what parameters students are to use, but stillallowing them the opportunity to determine themonitoring sites. Indicating the parameters students

Cooper et al. - Assessment as Teaching and Research Tools 69

must use, however, restricts their thinking and providesa narrower view of their understandings. It also reducesthe decision-making process; students only have todecide where to monitor.

Testing and revising a scenario is essential. Ifpossible, field test the scenario on a similar studentpopulation or have a colleague complete and critique thescenario. Based on the field test results, revise thescenario. In our field testing of this scenario, weeliminated any testing that could not be accomplishedby middle or high school students using commonlyavailable equipment and monitoring techniques. Asimilar scenario could be used for college students that ismore complex and contains parameters that are moredifficult to measure, like one for major cations andanions, which require the use of an ion chromatograph.A simplified version of the scenario could be employedfor younger children, where the sites and parameters totest would be significantly reduced.

Scenario Components - The scenario consists of thenarrative, prompt, directions for responding, andscoring system. The narrative presents the issue, context,data and information, and the perspective(s) from whichstudents respond to the scenario. The narrative containsall of the information students need to complete theassessment. The prompt states the question or problemstudents are to address using the information presentedin the scenario narrative. The directions for respondingindicate the format or product the students are toproduce in order to answer the question or solve theproblem. For example, students might be asked torespond to the prompt by creating a written response orproposal, a newspaper article, or a slide show. Thescoring system describes how student responses will bescored and what criteria will be used to assess studentresponses. Depending on the purpose of the assessmentthe criteria may emerge from student responses or bepredetermined.

In our use of the stream monitoring scenario wewere not looking for pre-determined responses, butwere looking for ideas to emerge from teacher responsesthat would enable us to understand their ideas. In ourscenario (Figure 1), the narrative informs students thatthey will be establishing a stream monitoring programfor the watershed depicted in the diagram, and indicatesthe details to be included in the proposal. It also sets thecontext: the watershed that contains multiple land usesituations. The issue in the scenario is how to monitorthe stream. The options include what parameters tosample and where to sample given the land usesurrounding the stream. Finally, it presents theperspective, taking the role of a consulting company.The prompt, “Your company is preparing a proposal inresponse to the above request,” indicates that studentsare to prepare a written proposal for stream monitoringbased on the stated guidelines. The directions for

responding require a written proposal and theidentification of monitoring sites on the watersheddiagram.

Our purpose in using this scenario was todetermine students’ ideas and understandings ratherthan to score or grade students, however the samescenario could be used as a quantitative or qualitativegrading tool. If one were to use it as a grading tool, thecriteria for assessing student responses might be basedon the following: monitoring parameters selected, therationale for selecting each parameter, monitoring sitesselected, the rationale for selecting each site, theidentification of up-stream and down-stream sites, andthe identification of a base line site.

Other Ways of Using the Assessment Scenario -Although we focused on using the scenario to gain anunderstanding of the teachers’ thought processes, theassessment scenario may be used in other ways. Thescenario may be used as an instructional activity wherestudents work through the scenario, with their finalproduct assessed. The scenario may be used in asummative manner to assess student understandingfollowing instruction. Assessment scenarios requiremore time for students to complete, thus if used inconjunction with traditional assessment questions anditems, the number of traditional questions should bereduced. Assessment scenarios may be administered ondemand or completed over several days, depending onthe product format and the extent of student responses.

Research Conclusions Based on the AssessmentScenarios - Changes between the pre-test and thepost-test responses of each participant were analyzed,and a detailed presentation and discussion of theseresults can be found in Shepardson, et al. (in press). Inbrief, changes occurred in three aspects of theparticipants’ responses: selection of the site formonitoring, parameters to be monitored, and why theychose those parameters and locations. In site selection,there was a 6% increase from pre-test to post-test in thepercentage of participants comparing up-stream todown-stream sites. In general, after the training, theparticipants selected a site upstream and downstream ofthe same land use designation and compared the waterquality using the same parameters. Before the training,fewer of the participants realized the need forestablishing a base-line of water quality and insteadconcentrated their testing on downstream sites to findspecific contaminants they felt would be associated withcertain land uses. The selection of parameters measuredalso changed from the pre-test to the post-test with only59% of the participants choosing to measure the sameparameters in several locations in the pre-test and 69%doing so in the post-test.

In the post-test the participants narrativesprovided not only more reasons for choosing their

70 Journal of Geoscience Education, v. 50, n. 1, January, 2002, p.64-71

parameters, but the reasons given were more in-depth,indicating a growth in understanding of streammonitoring concepts. Another notable change in theparticipants’ reasoning was the concept of “acceptable”levels of contaminants. They respondedoverwhelmingly at the beginning of the program thatany contaminants were “bad” for the stream, but at theend of the program their responses shifted to a conceptbased on overall stream quality. Teacher rationales forselecting macro invertebrates as holistic indicators ofstream quality rose from 23% (pre-test) to 90%(post-test), reflecting increasing understanding thatmacro invertebrates serve as indicators of the synergisticeffects of multiple pollutants.

SUMMARY AND CONCLUSIONS

The scenario assessment developed at PurdueUniversity provided both guidance in refininginstructional techniques and a method to evaluate theeffectiveness of an instructional program. Thisassessment was used for two programs to trainin-service teachers in environmental geoscience. In eachprogram, the participants were assessed at the initialmeeting to determine their understandings ofenvironmental problems related to a hypotheticalwatershed. These assessments were compared withassessments completed after participating in theworkshops to determine how their understandings hadchanged, but were also used to direct the workshopactivities toward addressing participantmisconceptions. During the course of the programs wedirected instruction to correct misconceptions aboutstream flow directions, acceptable levels ofcontamination, and what types of contamination wererelated to what land uses. Our assessments showed allthe participants gained a better understanding ofconcepts and approaches in stream quality monitoring.

Participants also reported significant changes intheir teaching related to the instructional program. Ofthe participants who completed the program, 94%indicate that they considered the program to be highlybeneficial and reported including issues-based inquiryscience in their own classrooms. The success of theprogram is also indicated by the enthusiasm of theparticipants to share their experiences with others bymaking presentations at State and NationalConferences. Thirty participants made presentations atthe 2000 Hoosier Association of Science Teachers, Inc.conference, and 6 made presentations at the 2000National Geological Society of America conference. Wewant to stress that the cooperative work between facultyin the Department of Earth and Atmospheric Sciences(School of Science), and the Department of Curriculumand Instruction (School of Education) was essential tothe development of assessment tools in thisenvironmental geoscience program. Science and

Education faculty provided unique perspectives andskills that enriched the program, and the faculty andstaff learned a lot from the collaboration.

The pre-test / post-test format using an authenticassessment instrument, like the watershed scenario, isan excellent way to guide instructional delivery and torationally assess student learning. We encourage othersto consider integrating scenario-based assessments intotheir teaching both as pre-test diagnostic tools to guideinstructional development, and as part of a pre-test,post-test approach to evaluating student learning.

ACKNOWLEDGEMENTS

Our work with in-service middle school and highschool teachers was funded by the Indiana StateCommission on Higher Education (Grant Number98-10) and the National Science Foundation (GrantsEAR-9809826 and 9819439-ESI). This manuscript wascompleted while Harbor was supported by the NewZealand – United States Educational Foundation as aFulbright Senior Scholar.

REFERENCES

Baxter, G.P., and Shavelson, R.J., 1994, Scienceperformance assessments: Benchmarks andsurrogates, International Journal of EducationalResearch, v. 21, p. 279-298.

Harbor, J., 2000, A Capstone Course in EnvironmentalGeosciences, Journal of Geoscience Education, v. 48,p. 617-23.

Herman, J.L., Aschbacher, P.R., & Winters, L., 1992, Apractical guide to alternative assessment,Alexandria, VA, Association for Supervision andCurriculum Development.

Hymes, D.L., Chafin, A.E., & Gonder, P., 1991, Thechanging face of testing and assessment: Problemsand solutions, Alexandria, VA, Association forSupervision and Curriculum Development.

Krockover, G., Adams, P., Eichinger, D., Nakhleh, M.,and Shepardson, D., 2001, Action-Based ResearchTeams, Journal of College Science Teaching,v. 30, p.313-317.

Mancl, K., Carr, K., and Marrone, M., 1999,Environmental Literacy of Ohio Adults, OhioJournal of Science,v. 99, p.57-61.

McTighe, J. and Ferrara, S., 1998, Assessing learning inthe classroom, Washington, DC, National EducationAssociation.

NRC, 1996, National Science Education Standards,National Committee on Science EducationStandards and Assessment, Washington, DC,National Academy Press.

Cooper et al. - Assessment as Teaching and Research Tools 71

Patton, M.Q., 1990, Qualitative evaluation and researchmethods, Newbury Park, CA, Sage Publications,Inc.

Perrone, V., 1991, Expanding student assessment,Alexandria, VA, Association for Supervision andCurriculum Development.

Robottom, I., 1990, The grip of print and other lessonsfrom distance education, in D.C. Engleson and J.F.Disinger (Eds.), Preparing classroom teachers to beenvironmental educators, Troy, OH, NorthAmerican Association for EnvironmentalEducation.

Roth, Charles E., 1991, Towards Shaping EnvironmentalLiteracy for a Sustainable Future, ASTM StandNews,v. 19, p.42-45.

Ruizo-Primo, M.A., and Shavelson, R.J., 1996, Rhetoricand reality in science performance assessments: Anupdate, Journal of Research in Science Teaching, v.33, p. 1045-1063.

Shepardson, D.P., Harbor, J. M., Cooper, B. C., andMcDonald, J. , 2000, The Impact of a ProfessionalDevelopment Program on Teachers’Understandings about Watersheds, Water Quality,and Stream Monitoring, Journal of EnvironmentalEducation, in press.

Shepardson, D.P., 2001, A professional developmentframework for collaborating with teachers to changeclassroom assessment practice (pp. 9-38), in, D.P.Shepardson (ed.), Assessment in science: A guide toprofessional development and classroom practice,Dordrecht, The Netherlands, Kluwer AcademicPublishers.

Schwert, D.P., Saini-Eidukat, B., Slator, B.M., Tilly, L.J.,1999, Assessment of Learning in a Virtual GeologicWorld, the Geology Explorer, Abstracts withPrograms, Geologic Society of America, v. 31, p. 411.

Trop, Jeffery, Krockover, Gerald, Ridgeway, Kenneth,2000, Integration of Field Observations withLaboratory Modeling for UnderstandingHydrologic Processes in an Undergraduate EarthScience Course, Journal of Geoscience Education, v.48, p. 514-521.

Ulanski, S.L., 1995, Curriculum reform inundergraduate geology programs, Journal ofGeologic Education,v. 43, p. 43-46.

Voska, K. W.; Heikkinen, H. W., 2000, Identifications andAnalysis of Student Conceptions Used to SolveChemical Equilibrium Problems, Journal of Researchin Science Teaching,v. 37, p. 160-176.

“The habit of precipitate explanation leads rapidly on to the development oftentative theories. The explanation offered for a given phenomenon is naturally,under the impulse of self-consistency, offered for like phenomena as they presentthemselves, and there is soon developed a general theory explanatory of a largeclass of phenomena similar to the original one. This general theory may not besupported by any further considerations than those which were involved in the first

hasty inspection.”

T.C. Chamberlin