2011_ioannis lefkos_designing experiments on thermal interactions by secondary-school students in a...

This article was downloaded by: [dyah palupi rohmiati]On: 29 January 2015, At: 06:49Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Research in Science & TechnologicalEducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/crst20

Designing experiments on thermalinteractions by secondary-schoolstudents in a simulated laboratoryenvironmentIoannis Lefkos a , Dimitris Psillos a & Euripides Hatzikraniotis ba Department of Primary Education , Aristotle University ofThessaloniki , Thessaloniki, 54124, Greeceb Physics Department , Aristotle University of Thessaloniki ,Thessaloniki, 54124, GreecePublished online: 12 Aug 2011.

To cite this article: Ioannis Lefkos , Dimitris Psillos & Euripides Hatzikraniotis (2011)Designing experiments on thermal interactions by secondary-school students in a simulatedlaboratory environment, Research in Science & Technological Education, 29:2, 189-204, DOI:10.1080/02635143.2010.533266

To link to this article: http://dx.doi.org/10.1080/02635143.2010.533266

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (theContent) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

Designing experiments on thermal interactions by secondary-school students in a simulated laboratory environment

Ioannis Lefkosa*, Dimitris Psillosa and Euripides Hatzikraniotisb

aDepartment of Primary Education, Aristotle University of Thessaloniki, Thessaloniki,54124, Greece; bPhysics Department, Aristotle University of Thessaloniki, Thessaloniki,54124, Greece

Background and purpose: The aim of this study was to explore the effect ofinvestigative activities with manipulations in a virtual laboratory on studentsability to design experiments.Sample: Fourteen students in a lower secondary school in Greece attended ateaching sequence on thermal phenomena based on the use of information andcommunication technology, and specically of the simulated virtual laboratoryThermoLab.Design and methods: A prepost comparison was applied. Students design ofexperiments was rated in eight dimensions; namely, hypothesis forming and ver-ication, selection of variables, initial conditions, device settings, materials anddevices used, process and phenomena description. A three-level ranking schemewas employed for the evaluation of students answers in each dimension.Results: A Wilcoxon signed-rank test revealed a statistically signicant differ-ence between the students pre- and post-test scores. Additional analysis bycomparing the pre- and post-test scores using the Hake gain showed high gainsin all but one dimension, which suggests that this improvement was almostinclusive.Conclusions: We consider that our ndings support the statement that there wasan improvement in students ability to design experiments.

Keywords: design of experiments; thermal phenomena; virtual laboratory

Introduction

Involving students in laboratory activities in science courses is supported by severalscience education researchers and teachers as contributing not only to the construc-tion of conceptual knowledge but also to the development of a scientic way ofthinking. However, as Lynch (1986) notes: when a group of teachers is noddingapproval about practical work they may have quite different purposes in mind. Aslong as it is not clear what students should do or should learn in the laboratory, itis also not clear what should be evaluated. Hodson (1992) suggests that evaluatinglaboratory work only by assessing students practical skills constitutes a narrow per-spective. His counter-proposal is a holistic approach in which students are doingscience that is much closer to scientic practice, using scientic methodology toinvestigate phenomena and solve problems. One important aspect of doing scienceis the tacit knowledge that scientists have and use all the time, which is distinct

*Corresponding author. Email: [email protected]

Research in Science & Technological EducationAquatic InsectsVol. 29, No. 2, July 2011, 189204

ISSN 0263-5143 print/ISSN 1470-1138 online 2011 Taylor & FrancisDOI: 10.1080/02635143.2010.533266http://www.informaworld.com

Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

from the mere possession of laboratory skills of manipulation and the possession ofcertain conceptual understanding. It is the capacity to use theoretical and proceduralknowledge in a purposeful way to achieve certain goals. Hodson (1992) denes athree-element model for doing science, in which one of the elements is the abilityto plan and design powerful experiments that test or illustrate a theory in an ele-gant way. Planning an experiment is largely a concept-driven activity. It is thethinking part of experimental inquiry, embracing several dimensions as identifyinga particular issue or problem for investigation, formulating a hypothesis and den-ing the dependent and independent variables. Designing an experiment involves t-ting a particular experimental procedure to the proposed investigation. It consistsprimarily of making decisions about treatments, conditions, controls, specic mea-surements to be taken, techniques and instruments to be utilised, and so on. Plan-ning and designing are aspects of doing science that can only be gained byexperience either in traditional or simulated laboratories.

Planning is also one of the three elements proposed for the assessment of labo-ratory work by Doran et al. (1993), based on a larger scheme found in an earlierstudy by Lunetta and Tamir (1979). Planning here is used in a broader perspectivethan Hodsons (1992), and it involves identifying questions to investigate, outlininga strategy for investigation, describing how to measure and/or observe variables,and planning for data recording. The ability to design experiments is considered tobe one of the most important of the skills linked to laboratory investigations(Johnstone and Al-Shuaili 2001). According to Garratt and Tomlinson (2001), thisskill is considered to be of even greater importance than the actual execution of anexperiment, as it is related not only to the content under study but to scienticmethodology as well. In the present work we have adopted the term designing ofexperiments in its broadened view, which is close to Hodsons (1992) planningand designing of experiments.

Among various technologies, computer simulations have gained a lot of interestin science teaching. Research ndings support their impact on conceptual learning(van der Meij and de Jong 2006). Simulations that enable essential functions of lab-oratory experiments to be carried out on a computer are called virtual laboratories(Harms 2000) and our study is related to this kind of simulation. Harms (2000) alsosuggests that virtual laboratories intend to transfer both conceptual and proceduralknowledge, since the preparation, performance and evaluation of laboratory experi-ments demands not only background knowledge but also knowledge referring toactually carrying out the experiment. The wide variety of this sort of laboratoryexperiment can be categorised by taking into account a number of characteristicssuch as the realism of visualisation, the freedom of user manipulations in an experi-mental set-up and execution, whether the simulation can support inquiry learning,or whether it can support the construction of new entities and their interactions. Alearning environment that can support these elements of doing science by openexperimentation and investigation is usually called an open environment. TheThermoLab simulated virtual laboratory used in this study and described laterbelongs to this category.

Discussion among researchers is lively, and mixed results on the use and effectsof virtual laboratories are reported. Some issues under study concerning their effec-tiveness in promoting conceptual understanding are whether virtual laboratoriesshould be used alone, whether they are more or less effective as compared withhands-on work, and whether they should be combined with hands-on experiments

190 I. Lefkos et al.

Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

and in what order (Finkelstein et al. 2005; Zacharia, Olympiou, and Papaevripidou2008). While the recent literature is rich in studies investigating the effects on con-ceptual understanding of students who are engaged in virtual manipulation, only afew deal with the development of students ability to design experiments. For exam-ple, Klahr, Triona, and Williams (2007) investigated the relative effectiveness of thephysical vs. the virtual setting, where seventh- and eighth-grade students assembledand tested mousetrap cars with the goal of designing the one that would go the far-thest. Their ndings suggest that both settings were equally effective in producingsignicant gains in learners knowledge about causal factors and their ability todesign optimal cars. In another study, however, Finkelstein et al. (2005) studied theeffects of substituting a computer simulation for real laboratory equipment in a uni-versity introductory physics course, and they found that students who used the sim-ulated equipment out-performed their counterparts in the coordinated tasks ofassembling a real circuit and describing how it worked. These studies focus on thearea of mechanics and electricity; there is currently no work in the area of thermalphenomena and the development of students designing of experiments in a virtualenvironment.

In this context, the aim of the present study was to explore whether the engage-ment of students in investigative activities in a virtual laboratory can result in thedevelopment of their ability to design experiments in order to solve everyday lifeproblems concerning thermal interactions.

Rationale

Investigative experiments were embedded in an innovative, information and com-munication technology (ICT)-enriched teaching sequence in the area of thermal phe-nomena, which is mainly based on the ThermoLab simulated virtual laboratory.ThermoLab is an open learning environment with the feel and look of a schoollaboratory (Hatzikraniotis et al. 2001) and is suitable for active student engagementand investigative experimentation (Zacharia et al. 2008). Users can quickly and eas-ily set up and execute experiments by direct manipulation of the objects on thecomputer screen and observe the results in multiple representations and graphs oftemperature and heat exchange vs. time (Figure 1). By inter-relating the observa-tions between representations, learners build abstractions fostering deeper under-standing of the domain (Ainsworth and VanLabeke 2004).

The sequence consists of ve modules, namely, Introduction to thermal phe-nomena, Thermal equilibrium, Calorimetry, Propagation of heat and Phasetransformations, covering all the thermal phenomena studied in the Greek curricu-lum. During the introductory module students were engaged in hands-on labora-tory activities with conventional lab instruments and Microcomputer-BasedLaboratories (MBL) sensors, in an attempt to get the feeling of thermal interac-tions. ThermoLab provided the learning environment for the main part of thesequence. Investigations were a crucial element in the sequence and were carriedout on the basis of structured modular worksheets consisting of various stepsbased on a laboratory variation of the familiar predictobserveexplain strategy(White and Gunstone 1992), including prediction, carrying out of the experiment,and results/graph interpretation and conclusion (Bisdikian and Psillos 2002). Ingeneral, each worksheet referred to more than one experiment, each of whichincluded phases like:

Research in Science & Technological Education 191

Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

Phase 1: Students are introduced to the phenomena under study, usually by akind of a qualitative problem to solve.

Phase 2: They express predictions about the evolution of the phenomena andthe values of the quantities. In some cases, they draw the graph corre-sponding to their predictions.

Phase 3: In order to test their predictions, students set up and run an experi-ment, observe the evolution of the phenomena and the real-time graph.At this point, students are sometimes asked to change the values ofthe parameters affecting the experiments, make new predictions on thebasis of their ndings, and run the experiment again.

Phases 45: Students compare their predictions with the experimental results andgraphs of previous phases and are guided to draw conclusions.

During the lab sessions students were engaged in semi-structured discussions on theproblem under study, following the worksheets as mentioned previously. We canillustrate this with the following example students use the appropriate worksheetand are asked to complete the following steps in order to study thermal radiationand whether it is affected by the surface of an object:

(1) During summer your cold drink gets hot quite fast due to thermal radiationof the hot surrounding environment. Do you think that the overall surface ofthe glass is important for this phenomenon?

(2) If you have, in a warm environment, two glasses with different overall sur-faces (smallerbigger) but the same amount of cold water, do you think thatthey will get hot in the same or different times?

Figure 1. A typical screenshot of an experiment in ThermoLab and labels of the mainfeatures of the virtual environment.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

(3) Design, set up and run an experiment to study this phenomenon using thevirtual laboratory.

(4) What are the results of your experiment? Was your prediction correct? Whatare your conclusions about the overall surface of an object and the heattransferred to it by thermal radiation?

As our students worked mainly in a virtual laboratory, we do not refer to the devel-opment of hands-on manipulative skills since most physical manipulations werereplaced by virtual ones on the computer screen (Couture 2004). Students can actwith/on the virtual objects in order to set up the experiment, take measurements andobserve the phenomena. The difference is that all manipulations are accomplishedby the use of the mouse and the cursor on the screen. A student in a physical set-ting uses a real bottle of water to pour 50 g of water into a beaker, while in the vir-tual setting this is accomplished by clicking the mouse. Nevertheless, the decisionto use a certain amount of water (e.g. 50 g) as a starting condition for an experi-ment is the same whether he is working in a physical or a virtual setting. The pro-cess background (de Jong and Van Joolingen 1998; Harms 2000) is the same inboth settings, whereas the kinaesthetic background is different (Figure 2). We areaddressing similar cognitive steps in doing science, but not psychomotor steps.ThermoLab, in our view, provides all the necessary elements for experimentation inthe eld of heat, except for the actual physical manipulation of the objects. Itcould thus foster our students skills in designing experiments.

Research design

Context

A prepost design was applied, involving one typical class in a secondary school innorthern Greece, which agreed to participate and apply this teaching sequence. InGreece, lower secondary schooling for general education, is compulsory for all stu-dents and lasts for three years. All students attending lower secondary school haveto take physics as a separate subject at the second and third year. The subjects ofour study were 14 second-year students in this school (1314 years of age). Theteaching sequence covered topics on thermal phenomena included in the Greek

Figure 2. Virtual and physical manipulations: different kinaesthetic same processbackground.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

compulsory education curriculum. Students were familiar with the use of computersand participated in one familiarisation session on the use of ThermoLab before theteaching sequence. The science teacher was experienced in the use of ICT. Studentsworked in pairs, each pair having their own computer and worksheets. They wereengaged in activities that varied from demonstration and cookbook lab to student-directed inquiry. Thus, during the teaching sequence they were gradually introducedto more inquiry-oriented activities. It should be stated that we deliberately followeda non-treatment approach for the dimensions of experimental design reported in thiswork. In other words students had no separate introduction to experimentation,but were gradually introduced to it through their activities in their worksheets. Suchactivities were based on the affordances of ThermoLab. The dimensions of experi-mental design were implied to the students during laboratory investigations that fol-lowed the structure of the worksheets.

Instruments

The students achievement with respect to their ability in experiment designing/planning was assessed by the prepost method; both pre- and post- assessmentsconsisted of written questionnaires followed by semi-structured interviews. Thepost-test assessment was carried out one week after the completion of the teachingsequence. Pre- and post- tasks were similar. The tasks were in the form of consecu-tive questions about a real-life problem situation: Bill claims that milk gets hot fas-ter than water. What do you think?; How will you get the right answer?; Canyou set up an experiment?; What will you need?; What will you observe?;How will you decide if Bill was right or not?. In fact, we were asking the studentsto express a hypothesis, plan an experiment to verify it, write down his/her answersand justify them during the interviews. Semi-structured individual interviews basedon students answers in the written questionnaires were carried out with all studentsin order to clarify their perceptions on designing an experiment.

Framework of analysis

The students answers were recorded, transcribed and evaluated using a speciallydesigned framework, described later. The assessment frameworks that can be foundin the literature mostly concern the overall performance of students practical work,parts of which address the planning or designing of experiments. For example Jose-phy (1986), in a framework called Oxford Certicate of Educational Achievement(OCEA), suggests that planning includes a number of dimensions such as: (stu-dents should) suggest a testable hypothesis, select appropriate apparatus from arange provided; suggest apparatus appropriate to a task; develop a plan which, ifcarried out, would answer a simple problem. Recently, Anagnos et al. (2007) sug-gested a rubric for the assessment of experiment designing by undergraduates,including dimensions such as: select dependent and independent variable(s) to bemeasured or controlled; describe an appropriate protocol for the experiment; selectappropriate materials for conducting the experiment. Each dimension of the rubricis given a ve-point scale from 0 (low) to 4 (high) for the assessment.

Based on these works we introduced a new framework for evaluating the devel-opment of students skill in designing experiments. We dened eight specicdimensions of experiment designing (Figure 3):


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

Forming a hypothesis: what are the students criteria? Hypothesis verication criteria: what criteria do the students set, in order to

decide if their hypothesis is veried or not Identifying variables: can they dene the dependent and independent variables

involved in the phenomena? Devices, instruments and substances: what do students consider necessary for

conducting the experiment? Device settings: which settings of the devices do students set before starting

the experiment? Initial conditions: which conditions do students set before starting the experi-

ment? Process description: how do students plan to observe and record the data? Phenomena description: how do they describe the phenomena involved in the

experiment?

These dimensions are not in hierarchical order.All dimensions are equally assigned to the same number of levels and scores

(see Appendix), so in this aspect our scheme is closer to the rubric proposed byAnagnos et al. (2007), but given the age and abilities of our students we adopted athree-level scoring scale. As shown in the Appendix, if a students answer in agiven dimension is evaluated at level 1, then he/she gets one point, at level 2 he/she gets two points and at level 3 he/she gets three points.

Millar (1988) claims that the skills and processes used by scientists (and that weare trying to develop in schools) have no special link with science, but are featuresof general cognition (for example observing, classifying or hypothesising) thateveryone uses without special instruction. In Millars view, the real focus of instruc-tion should be to develop those skills with a scientic orientation, assisting thechild to construct for herself a mental representation of the scientic ways of

Figure 3. The eight dimensions used for evaluating students design of experiments.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

working and judging. In this context we have adopted a linking between the levelsof success and the use of scientic criteria in students answers in the dimensionswe considered applicable. So while the general rule for forming the levels of assess-ment is: (1) missing; (2) partially accepted; and (3) complete, in the forminghypotheses and phenomena description dimensions the levels are dened as: (1)insufcient or missing; (2) based on alternative conceptions; and (3) scienticallyacceptable (see Appendix).

A pilot test was conducted during the development phase of the teachingsequence, involving students at a similar school. The students answers were dis-cussed with two experienced researchers in order to assure content validity of theassessment framework.

In the present study the students answers were analysed and categorised by tworesearchers in the eight dimensions listed earlier. These raters marked independentlya selected number of answers after coming to an agreement about the characteristicsof each level in all categories. Cases of disagreement between their ratings wereresolved by discussion. A high level of inter-rater reliability (0.86) was achieved inthe nal scoring.

Results

The results show a considerable change between pre- and post-test, with a meanscore of 1.93 for the pre-test and 2.53 for the post-test. The smaller value of stan-dard deviation in the post-test (0.37) compared with the pre-test (0.48) shows thatstudents scores followed a narrower distribution about the mean value.

Given the size of the sample, the Wilcoxon signed-rank test was chosen for non-parametric statistics. The analysis revealed a statistically signicant difference beyond0.1% level between the students pre- and post- scores (z = 3.22, p (1-tailed) =0.0006, p (2-tailed) = 0.0013). This supports the claim that there was a developmentin our students mean scores after the treatment (at the stated level of statisticalsignicance).

Figure 4. Students answers: pre-comparison (dark bars) and post-comparison (light bars)for each dimension.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

In Figure 4 we present detailed prepost comparisons concerning the eightdimensions of experimentation.1

For the forming hypotheses dimension (Figure 4a), in the pre-test there werefour students who expressed their hypothesis with no justication (level 1), eightwho offered insufcient justication or justication based on alternative conceptions(level 2), and only one student who was able to provide a scientic justication(level 3). A typical answer from an interview, based on an alternative conception is:I think that water gets hot faster than olive-oil because it is thinner. At the post-test, the corresponding distribution was two students at level 1, one at level 2 and11 at level 3. Another typical answer of a fully (scientically) justied hypothesisis: Olive-oil gets hot faster than water. . . it takes half the time. . . water has twicethe heat capacity.

As can be seen in Figure 4a, there is a clear shift in the answers from levels 1and 2 to levels 2 and 3 at the post-test (from an average score of 1.79 in the pre-test to 2.64 in the post-test). This means that while initially most of our students(13/14) expressed their hypotheses without justication or insufciently or based onalternative conceptions, after treatment nearly all of them (11/14) used scientic cri-teria to formulate their hypotheses (see Table 1). Following the same representationfor the other diagrams, we can also comment as follows.

For the verifying hypotheses dimension (Figure 4b), in the pre-test there weresix students who were unable to clearly dene criteria (level 2) and eight studentsable to express one criterion (level 3). A typical answer from an interview for level2 is: if one of the liquids is boiling at a lower temperature, it will boil fasterwhere students set confusing criteria for time or temperature. At the post-test, all ofthe students (14/14) were at level 3. Additionally, a few students (2/14) managed toexpress multiple criteria. A sample elaborated answer from an interview for studentsat level 3 expressing multiple criteria is:

. . .we can observe for some time, we will dene a certain time. We will heat up oliveoil rst, record its temperature, then we will start counting from the beginning whileheating up water. . . We can dene a certain temperature e.g. 40 degrees and use ourwatch. When water or oil reaches this temperature we will stop and then compare thetime needed for the other substance.

As can be seen in Figure 4b, there is a shift in the answers from levels 2 and 3(average score 2.57) to level 3 at the post-test (average score 3.00). This means thatwhile initially only about half of our students (8/14) could clearly dene a

Table 1. Pre-post average test scores and Hake gain.

Dimension Pre-test Post-test Hake gain

A Forming hypotheses 1.79 2.64 0.71B Verifying hypotheses 2.57 3.00 1.00C Identifying variables 1.57 2.29 0.50D Devices/instruments 1.71 2.14 0.33E Device settings 1.29 2.07 0.46F Initial conditions 1.64 2.29 0.47G Describing processes 2.43 2.93 0.88H Describing phenomena 2.43 2.86 0.75


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

verication criterion, after treatment all of our students (14/14) could do that, and afew of them (2/14) could additionally dene a second alternative criterion.

For the identifying variables dimension (Figure 4c), in the pre-test there weresix students who were unable to identify the variables affecting the experiment (level1) and eight students who could partially identify them (level 2). An example answerfor level 1 is: we can use a bowl and heat it on the stove. Then we will do the samefor milk and for level 2: we should use the same amount of water and milk in thetwo bowls, because if the water is more it gets hot harder. . .We will put each one onthe stove for the same time and at the same temperature. At the post-test, the corre-sponding distribution was 10 students at level 2 and 4 at level 3. An elaboratedanswer from the interviews for good identication of variables at level 3 is:

. . .in two beakers from the same material, both of glass for example, we will pour thesame amount of water and olive-oil, so that we can compare. . .We will use two burn-ers and set them to the same heating power. . . At rst they should be in the same tem-perature. . . we will heat them up for the same time.

As can be clearly seen in Figure 4c, there is a shift in the answers from levels 1and 2 at pre-test (6/14 and 8/14, average score 1.57) to levels 2 and 3 (10/14 and4/14) at the post-test (average score 2.29). This means that while initially almosthalf of our students could not identify the variables of the problem under study,after the treatment most of them could adequately identify them and a few couldfully identify the variables.

For the devices and instruments dimension (Figure 4d), in the pre-test therewere six students who could mention only a few of the elements necessary for con-ducting the experiment (level 1), six students who could name most of the elements(level 2) and two students who could list all of the necessary elements (level 3).Quoting from interviews, a typical answer for level 1 is: we will need two bowlsand two bottles. . ., and for level 2: we will need two burners, two bowls; we willalso put a thermometer in each bowl. A typical answer for level 3 is: we will usetwo beakers, milk, water and two burners. We will use a stopwatch and two ther-mometers also. At the post-test, the corresponding distribution was two students atlevel 1, eight at level 2 and four at level 3.

As presented in Figure 4d, in the case of devices and instruments there wasonly a slight improvement. Students answers were mainly categorised at levels 1and 2 in the pre-test (6/14 and 6/14 respectively, average score 1.71), while in thepost-test most of the answers were at levels 2 and 3 (8/14 and 4/14 respectively,average score 2.14). This means that initially the majority of our students couldmention a few or most of the necessary elements for conducting the experiment,while after treatment the majority of our students were able to mention most or allof the elements. For the device settings dimension (Figure 4e), in the pre-test therewere 10 students who could not mention any of the necessary settings (level 1) andfour who could mention some of them (level 2). A typical answer for level 1 is:we will use two bowls and two burners (where students just mention the devicesbut not any settings) and for level 2: we will need 2 burners and two bowls. . .identical bowls. At the post-test, the corresponding distribution was 13 students atlevel 2 and one at level 3. The answer corresponding to level 3 is: in two identicalbeakers we will pour water and olive oil. We will set the burners to the same heat-ing power.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

Consequently, for the device settings dimension the improvement shown byFigure 4e is easily observable. Answers at the pre-test were mainly categorised atlevel 1 (10/14, average score 1.29), while at the post-test the majority of theanswers (13/14) were at level 2 (average score 2.07). This means that initially mostof our students could not mention any of the necessary settings of the devices, butafter treatment the majority of our students could more efciently dene the settingsnecessary for conducting the experiment.

For the initial conditions dimension (Figure 4f), in the pre-test there were sixstudents who could not mention even one of the necessary initial conditions (level1), seven students who could mention some of them (level 2) and one student whocould mention all the necessary conditions (level 3). An answer from the interviewsfor level 2 is: if they are initially at the same temperature as the environment. . .e.g. 25 degrees and for level 3: we will use the same amount of liquid in bothbeakers. . . they should also be at the same initial temperature. At the post-test, thecorresponding distribution was two students at level 1, eight at level 2 and four atlevel 3.

As can be seen in Figure 4f, there is a small improvement for the initial condi-tions dimension. Initially, about half of our students (6/14) could not mention anyof the necessary initial conditions, while after treatment a lot of the students (10/14)could mention many of the necessary conditions and some of them (4/14) couldname them all. The average score shifted from 1.64 in the pre-test to 2.29 in thepost-test.

For the describing processes dimension (Figure 4g), in the pre-test there wereeight students able to partially describe the experimental process (level 2) and sixstudents able to give a clear description of the process (level 3). A typical answerfor level 2 is: we will put it on the stove. . . for as long as it takes, 23 minutes;just to see when it will get hot. . . until the temperature starts rising quickly. At thepost-test, the corresponding distribution was 1 student at level 2 and 13 at level 3.Answers for level 3 were elaborated, such as the following from an interview:

We will use two burners and set them to the same heating power. . . at rst they(the substances) should be at the same temperature. . .we will heat them up for thesame time. . . we will observe the temperature rising on the thermometers. . . we willdene a certain temperature e.g. 40 degrees and wait until one of the liquids getsthere rst.

Although there was quite a good level of success in the describing processesdimension even at the pre-test, in Figure 4g we still can observe some improve-ment. Initially, about half of our students (8/14) were not able to describe the exper-imental process adequately, but the rest of them (6/14) could. On the other hand,after treatment nearly all of our students (13/14) were able to give a clear descrip-tion of the process. The average score shifted from an already high 2.43 in the pre-test to 2.93 in the post-test.

Almost the same situation applies for the describing phenomena dimension(Figure 4h). In the pre-test there were eight students who described the phenomenainvolved in the experiment by expressing alternative conceptions i.e. they dis-played difculty in differentiating between heating and boiling (level 2) and sixstudents who used a scientically acceptable description (level 3). A typical answerfrom an interview for level 2 is: we will observe the temperature at which water


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

will start getting hot. . . I mean boiling. At the post-test, the corresponding distribu-tion was two students at level 2 and 12 at level 3. A typical answer for level 3 is:we will heat them up, until they reach e.g. 60 degrees.

As observed in the corresponding Figure 4h, there was also a slight improve-ment between pre- and post-tests. Students answers were categorised at levels 2and 3 in the pre-test and also in the post-test, but the percentages are different.While initially about half of our students (8/14) described the phenomena involvedin the experiment by expressing alternative conceptions, the other half (6/14) used adescription close to the scientically acceptable. After treatment, almost all of ourstudents (12/14) described the phenomena involved in a scientically acceptableway and only two of them were still not able to distinguish between heating andboiling. The average score shifted from an already high 2.43 in the pre-test to 2.86in the post-test. The results of the average prepost test scores are summarised inTable 1.

Discussion and conclusions

The results were additionally analysed by comparing the pre- and post-test scoresusing the Hake gain (or Hake factor). Hake dened this normalised gain as theimprovement between two tests relative to the possible improvement (Hake 1998):

gH actual gainmax imum possible gain post pre

max score pre 1

Where pre and post are the pre-test and post-test mean scores and max_-score is the maximum score a student can achieve (3 in our case). Hake gain is asignicant measure for the efciency of instruction because it eliminates the inu-ence of the students different starting levels, since it equals the maximum possiblegain students can achieve (Lenaerts, Wieme, and Van Zele 2003). The Hake gainresults are also given in Table 1.

Considering the various dimensions we may note that there were high gains, inHake terms, in almost all dimensions. Those with the highest achievement (highestHake gain) are forming hypotheses, verifying hypotheses, describing processesand describing phenomena. These dimensions had Hake gains higher than 0.7. Itis clear that although our students already had high scores in the pre-tests theyshifted into much higher levels in the post-test. The second group of dimensions,with (relatively) lower Hake gains, was identifying variables, device settings andinitial conditions. The gain in these three dimensions was about 0.50, which isalso considered as high achievement.2 Finally, the dimension with the least gainwas devices and instruments. The gain in this dimension was 0.33.

Within the limitations of our small sample, we consider that our ndings supportthe statement that there was an improvement in students ability to design experi-ments. It is notable that the high gains in all but one dimension show that thisimprovement was almost inclusive. We consider that the potential of ThermoLabhad a multi-level impact on such an improvement of students ability to designexperiments, for three reasons. First, this simulated virtual laboratory environmentallowed us, as researchers, to design, develop and apply appropriate guided investi-gations, in a conceptually rich topic like heat, which were embedded in a coherentteaching sequence. Second, in carrying out their activities students were guided to


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

express hypotheses in problem-based situations, design experiments, and easily testtheir designs through engagement in life-like manipulations of virtual objects. Third,students had the opportunity to experience doing science by being activelyengaged in handling a powerful technology-intensive environment in order to con-struct meanings and develop their experimental skills.

Using ThermoLab students had the advantage of interacting with a user-friendly learning environment, where they could quickly and easily conduct experi-mental investigations, look at their hypotheses and examine variables in terms ofmore rich experimental activities than they would in a physical setting. For exam-ple, the high achievement in the post-test of dimensions such as forming hypothe-ses, verifying hypotheses and describing phenomena, which are highly content-dependent, could have been supported by the conceptual coherency of the teachingsequence as well as students involvement in guided investigative activities usingThermoLab. We think that such results are also indicative of our students devel-opment concerning the concepts related to the topic under study, given the fact thatour teaching sequence did not focus only on experimentation but was also concep-tually rich in the topic of heat.

In the course of carrying out the suggested activities using Nhermolab, stu-dents not only directly manipulate the virtual objects but they also take decisionsabout what to manipulate, why, and how and to reect on the feedback they aregiven in order to construct some meaning out of it. While having a different kinaes-thetic background to physical ones, those virtual manipulations share the same pro-cess background, which is a crucial and important factor in the designing ofexperiments (de Jong and Van Joolingen 1998). Moreover, a simulated virtual labo-ratory like ThermoLab presents a simplied view of reality. This is done in such away that ThermoLab, though open, limits the possible manipulations of the stu-dents and directs their attention to relevant objects and properties. This feature offocusing students attention relieves them of the time-consuming physical manipula-tions of a real lab (Psillos and Niedderer 2002). By removing the focus of theirattention and effort from the manipulation of real objects (Redish, Saul, andSteinberg 1997), they were able to concentrate more on taking decisions about theirinvestigations, that is, on designing their experiments (Doran et al. 1993; Hodson1992) and applying their inquiry tactics (Millar 1990). The environment can beconsidered as providing scaffolding to students to help them carry out such guidedactivities and develop design skills (Finkelstein et al. 2005). For example, the highgain in the post-test for the describing processes and identifying variables dimen-sions could have been supported by the character of the ThermoLab software,which employs an underlying structure in assembling an experimental set-up, pro-viding a kind of guidance to the users actions and consequently scaffolding theprocedure to be followed to set-up and conduct an experiment in order to carry outone of the suggested activities from this sequence. In carrying out the activities,moreover, while interacting with the virtual environment by manipulating the virtualobjects students are in fact manipulating the values of input variables and observingthe results in the values of output variables (de Jong 1991).

Finally, there are some indications that certain design characteristics of theThermoLab software seem to a have a specic effect on students responses. Toillustrate, by analysing the data we observed that the majority of our students inboth the pre-test and the post-test failed to mention the use of a stopwatch as anecessary measuring device (although they do refer to time as a variable and they


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

do consider the phenomena to be time-dependent), resulting in a level 2 rankingin the corresponding dimension. One possible explanation for this omission mightbe that as ThermoLab, for reasons of simplicity, automatically starts the timingwhen the run button is pressed, our students were not trained to use stopwatches(and thus do not consider them). The response of our students, who had limitedpractical experience, could have been affected by this simplied function of thisvirtual object inherent in the design of the ThermoLab. This is an issue whichneeds consideration, since the risk with simulated environments, including virtuallaboratories, is that students, unless appropriately notied, may acquire an over-simplied view of reality and disconnect with the real world (Papadouris andConstantinou 2009).

Certain specic features of the activities concerning the integration of the virtualobjects and their use by students in this teaching sequence might also have had aneffect on the quality of students responses. To illustrate this effect, in the devicesettings dimension, our data showed that the majority of our students did not referto the heating power of the burners as a condition to be set, and this affected theidentication and listing of all necessary conditions in their replies. The reasonbehind such responses could be possibly related to the fact that during the entireteaching sequence only a couple of experimental activities involved the use of burn-ers. The majority of the activities involved interactions between two bodies of dif-ferent temperature, or interactions with the surroundings, without an externalheating source. In other words, our students were not trained to account for that set-ting.

In the context of the long-standing discussion between researchers about themerits of virtual laboratories, Ma, and Nickerson (2006), in a recent review, wonderwhether virtual laboratories are effective when it comes to teaching skills fordesigning experiments and argue for further studies on this matter a plea sup-ported by Wofford (2009) in her extensive review in this journal. In our case, affor-dances of ThermoLab seem to have affected students ability to design anexperiment in the context of the aforementioned teaching sequence as discussedpreviously. Our ndings in the area of heat are in line with some recent ndingsin other areas (Klahr, Triona, and Williams 2007). In this context, we consider thatthe present ndings add to the pedagogical inter alia value more commonlyreported for the use of simulated virtual laboratories and their potential for facilitat-ing students in doing science across conceptually different areas.

Educators can exibly integrate simulated laboratories into their teaching, notonly for their ease of use by their students or the reported conceptual gains but also,following an active engagement strategy, to try to develop their students experi-ment-designing abilities. Further research with larger samples and in other eldswill be useful in order to explore the potential of virtual laboratories, like Thermo-Lab and their characteristics in a variety of topics and contexts.

Notes1. Students responses were transcribed and translated by one of the authors, and cross

checked by the others. Since the conversations were in Greek, in the quotes presentedthe translation retain the students original informal expressions.

2. In his original paper, Hake studied the mean gain for classes of over 6000 undergraduatephysics students in total, and dened a high gain as about 0.48, while a low gain wasabout 0.23 (Hake 1998).


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

ReferencesAinsworth, S., and N. VanLabeke. 2004. Multiple forms of dynamic representation. Learning

and Instruction 14, no. 3: 24155.Anagnos, T., C. Komives, N.J. Mourtos, and K.M. McMullin. 2007. Evaluating student mas-

tery of design of experiment. Paper presented at the 37th ASEE/IEEE, Frontiers in Edu-cation 1013 October, in Milwaukee, WI.

Bisdikian, G., and D. Psillos. 2002. Enhancing the linking of theoretical knowledge to physi-cal phenomena by real-time graphing. In Teaching and learning in the science labora-tory, ed. D. Psillos and H. Niedderer, 193204. Dordrecht, The Netherlands: KluwerAcademic Publishers.

Couture, M. 2004. Realism in the design process and credibility of a simulation-based virtuallaboratory. Journal of Computer Assisted Learning 20, no. 1: 409.

de Jong, T. 1991. Learning and instruction with computer simulations. Education and Com-puting 6, no. 34: 21729.

de Jong, T., and W. Van Joolingen. 1998. Scientic discovery learning with computer simu-lations of conceptual domains. Review of Educational Research 68, no. 2: 179201.

Doran, R.L., J. Boorman, F. Chan, and N. Hejaily. 1993. Alternative assessment of highschool laboratory skills. Journal of Research in Science Teaching 30, no. 9: 112131.

Finkelstein, N.D., W.K. Adams, C.J. Keller, P.B. Kohl, K.K. Perkins, N.S. Podolefsky, S.Reid, and R. LeMaster. 2005. When learning about the real world is better done virtu-ally: A study of substituting computer simulations for laboratory equipment. PhysicalReview Special Topics-Physics Education Research 1, no. 1: 101038.

Garratt, J., and J. Tomlinson. 2001. Experimental design can it be taught or learned?University Chemistry Education 5: 749.

Hake, R. 1998. Interactive-engagement versus traditional methods: A six-thousand-studentsurvey of mechanics test data for introductory physics courses. American Journal ofPhysics 66: 6474.

Harms, U. 2000. Virtual and remote labs in physics education. Paper presented at the secondEuropean Conference on Physics Teaching in Engineering Education, 1417 June, inBudapest.

Hatzikraniotis, E., J. Lefkos, G. Bisdikian, D. Psillos, J. Refanidis, and J. Vlahavas. 2001.An open learning environment for thermal phenomena. Paper presented at the 5th Inter-national Conference on Computer Based Learning in Science, 712 July, in Brno, CzechRepublic.

Hodson, D. 1992. Assessment of: practical work. Science and Education 1, no. 2: 11544.Johnstone, A.H., and A. Al-Shuaili. 2001. Learning in the laboratory: Some thoughts from

the literature. University Chemistry Education 5, no. 1: 4251.Josephy, R. 1986. Assessment of practical and experimental work in physics through OCEA.

Physics Education 21, no. 4: 21420.Klahr, D., L. Triona, and C. Williams. 2007. Hands on what? The relative effectiveness of

physical versus virtual materials in an engineering design project by middle school chil-dren. Journal of Research in Science Teaching 44, no. 1: 183203.

Lenaerts, J., W. Wieme, and E. Van Zele. 2003. Peer instruction: A case study for an intro-ductory magnetism course. European Journal of Physics 24, no. 1: 714.

Lunetta, V.N., and P. Tamir. 1979. Matching lab activities with teaching goals. Science Tea-cher 46, no. 5: 224.

Lynch, P.P. 1986. Laboratory work in schools and universities: Structures and strategies stilllargely unexplored. Journal of Science and Mathematics Education in SE Asia 9: 5160.

Ma, J., and J.V. Nickerson. 2006. Hands-on, simulated, and remote laboratories: A compara-tive literature review. ACM Computing Surveys 38, no. 3: 124.

Millar, R. 1988. What is scientic method, and can it be taught. In Skills and processes inscience education. A critical analysis, ed. J. Wellington, 4762. London: Routledge.

Millar, R. 1990. A means to an end: The role of processes in science education. In Practicalscience: The role and reality of practical work in school science, ed. B. Woolnough, 4352. Milton Keynes, UK: Open University Press.

Papadouris, N., and C.P. Constantinou. 2009. A methodology for integrating computer-basedlearning tools in science curricula. Journal of Curriculum Studies 41, no. 4: 52138.


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

Psillos, D., and H. Niedderer. 2002. Issues and questions regarding the effectiveness of lab-work. In Teaching and learning in the science laboratory, ed. D. Psillos and H. Niedder-er, 2130. Dordrecht, The Netherlands: Kluwer.

Redish, E., J. Saul, and R. Steinberg. 1997. On the effectiveness of active-engagementmicrocomputer-based laboratories. American Journal of Physics 65, no. 1: 4554.

van der Meij, J., and T. de Jong. 2006. Supporting students learning with multiple represen-tations in a dynamic simulation-based learning environment. Learning and Instruction16, no. 3: 199212.

White, R., and R. Gunstone. 1992. Probing understanding. London: The Falmer Press.Wofford, J. 2009. K-16 computationally rich science education: A ten-year review of the

journal of science education and technology (19982008). Journal of Science Educationand Technology 18, no. 1: 2936.

Zacharia, Z., G. Olympiou, and M. Papaevripidou. 2008. Effects of experimenting withphysical and virtual manipulatives on students conceptual understanding in heat andtemperature. Journal of Research in Science Teaching 45, no. 9: 102135.

Appendix. Levels of sucess and scoring for all the dimensions

Score (points)

A. Forming hypothesisLevel 1 Hypothesis is formed without justication/insufciently/missing 1Level 2 Hypothesis is formed based on alternative conceptions 2Level 3 Hypothesis is formed based on scientically accepted criteria 3B. Verifying hypothesisLevel 1 Criteria are not stated 1Level 2 Criteria are stated insufciently/ without justication 2Level 3 One/more criteria are clearly stated 3C. Identifying variablesLevel 1 Dependent and independent variables are not identied 1Level 2 Dependent and independent variables are partially identied 2Level 3 Dependent and independent variables are fully identied 3D. Devices, instrumentsLevel 1 A few of the necessary elements are stated 1Level 2 Most of the necessary elements are stated 2Level 3 All of the necessary elements are stated 3E. Device settingsLevel 1 None of the necessary settings are stated 1Level 2 Some of the necessary settings are stated 2Level 3 All of the necessary settings are stated 3F. Initial conditionsLevel 1 Initial conditions are missing 1Level 2 Initial conditions are only partially stated 2Level 3 Initial conditions are sufciently /completely stated 3G. Describing processLevel 1 Description of data measuring and recording is missing 1Level 2 Partial description of data measuring and recording 2Level 3 Description of data measuring and recording is clear 3H. Describing phenomenaLevel 1 Descriptions of the phenomena are insufcient/missing 1Level 2 Descriptions of the phenomena involve alternative conceptions 2Level 3 Descriptions of the phenomena are scientically accepted 3


Dow

nloa

ded

by [d

yah p

alupi

rohm

iati]

at 06

:49 29

Janu

ary 20

15

2011_ioannis lefkos_designing experiments on thermal interactions by secondary-school students in a...

Documents