the effect of guided notes on student performance on ... · pdf filethe effect of guided notes...

Author: Peterson, Theran, A Title: The Effect of Guided Notes on Student Performance on Project Lead

the Way™ End-of-Course Exams The accompanying research report is submitted to the University of Wisconsin-Stout, Graduate School

in partial completion of the requirements for the

Graduate Degree/ Major: MS Technology Education

Research Advisor: Sylvia Tiala, Ph.D.

Submission Term/Year: Spring, 2013

Number of Pages: 46

Style Manual Used: American Psychological Association, 6th edition

x I understand that this research report must be officially approved by the Graduate School and that an electronic copy of the approved version will be made available through the University Library website

1

x I attest that the research report is my original work (that any copyrightable materials have been used with the permission of the original authors), and as such, it is automatically protected by the laws, rules, and regulations of the U.S. Copyright Office. x My research advisor has approved the content and quality of this paper.

STUDENT:

NAME: Theran Peterson DATE: 25 June 2013

ADVISOR: (Committee Chair ifMS Plan A or EdS Thesis or Field Project/Problem):

NAME ~ DATE: 26June2013

This section for MS Plan A Thesis or EdS Thesis/Field Project papers only Committee members (other than your advisor who is listed in the section above)

1. CMTE MEMBER'S NAME: DATE:



This section to be completed by the Graduate School This final research report has been approved by the Graduate School.

Director, Office of Graduate Studies: DATE:

2

Peterson, Theran, A. The Effect of Guided Notes on Student Performance on Project

Lead the Way™ End-of-Course Exams

Abstract

The purpose of this study was to examine the effects of guided notes on student

performance within the Project Lead the Way - Principles of Engineering course at a high

school in central Wisconsin and ultimately how to increase the overall level of student

performance not only in the subject class but department-wide. More specifically, the

study examined the effect of guided notes on student performance on formal assessments,

on daily assignments and activities, and on the perceptions of students relative to guided

note inclusion.

3

Table of Contents

Page

Abstract…………………………………………………………………...... 2

List of Tables ………………………………………………………………. 5

Chapter I: Introduction……………………………………………………... 6

Statement of the Problem…………………………………………… 8

Research Questions…………………………………………………. 8

Definition of Terms…………………………………………………. 9

Chapter II: Review of Literature…………………………………………… 11

Introduction…………………………………………………………. 11

Background…………………………………………………………. 11

Theory Base…………………………………………………………. 12

Note-taking and Cognition………………………………………….. 14

Note Organization…………………………………………………… 15

Note-taking and Achievement………………………………………. 16

Chapter III: Methodology…………………………………………………... 17

Introduction…………………………………………………………. 17

Research Design…………………………………………………….. 17

Population and Sample……………………………………………… 18

Treatment……………………………………………………………. 18

Instrumentation……………………………………………………… 18

Data Collection Procedure…………………………………………... 19

Data Analysis………………………………………………………... 21

4

Chapter IV: Results………………………………………………………… 22

Introduction………………………………………………………… 22

Research Objectives………………………………………………… 22

Findings…………………………………………………………….. 22

Chapter V: Summary, Conclusions and Recommendations……………….. 36

Introduction………………………………………………………… 36

Findings…………………………………………………………….. 37

Conclusions…………………………………………………………. 38

Recommendations…………………………………………………... 39

References…………………………………………………………………... 40

Appendix A: Sample Guided Notesheet……………………………………. 43

Appendix B: Student Survey……………………………………………….. 46

5

List of Tables

Table 1: Pretest Performance……………………………………………….. 23

Table 2: Posttest Performance and Percent Change………………………… 23

Table 3: Daily Assignment Performance…………………………………… 27

Table 4: Student Performance on End-of-Course Assessment……………... 30

Table 5: Subtest versus Overall Test Performance Correlation…………….. 31

Table 6: Student Survey Results……………………………………………. 33

6

Chapter I: Introduction

Project Lead the Way (PLTW) is a national pre-engineering curriculum initiative

originating in New York in the late 1990s which has seen tremendous growth in the last

decade to the point where it is considered by many to be the preeminent secondary-level

pre-engineering curriculum available, serving over 4000 schools and hundreds of

thousands of students in all fifty states. PLTW offers a number of individual classes

relating to engineering education. The pathway begins with two fundamental courses

that highlight basic theories and concepts common to all branches of engineering,

Introduction to Engineering Design and Principles of Engineering. Beyond these classes,

students have the opportunity to choose from a number of specialty courses including

Digital Electronics, Civil Engineering and Architecture, Computer Integrated

Manufacturing, and Aerospace Engineering. The capstone experience for students is a

course called Engineering Design and Development. In this senior-level class, students

are asked to call on of the knowledge garnered throughout the previous PLTW classes in

an effort to develop a solution to a real-life problem from concept to prototype. PLTW

also offers elementary and middle-school engineering curricula and a pathway in

biomedical technology. PLTW offers full curricula for these classes including

presentations, assignments, activities and assessments, however, ultimately it is up to the

teacher to decides what is taught and how. Teachers, however, are discouraged to stray

too far from the set curriculum as the end-of-course exams are written by PLTW and

merely administered by the teacher. To this end, the exam is a reflection on the teaching

of the teacher relative to the given curriculum. The rigorous curriculum as put forth by

7

PLTW has been recognized by many post-secondary colleges and universities who have

agreed to grant college credit to students that pass the PLTW end-of-course exam.

As mentioned above, PLTW does a very good job providing material support for

their curriculum, one area in particular is the archive of electronic presentations. The

presentations give the teacher the ability to present material on which they themselves

may not be proficient and still allow the students the opportunity to view relevant and

important information. However, that issue is more directed at professional development

and beyond the scope of this research project. Though PLTW does an excellent job

providing presentations for all their classes on virtually all of the content, they do not

provide a tool with which students can use to effectively and efficiently record the given

information.

As this author reviewed the notebooks of students, the haphazard nature of their

study skills and more specifically their note-taking practices was highly dismaying from

an organizational standpoint. Many times there are notes from several classes on

successive pages, reading more like a journal of the day's activities than a collection of

information from any one particular class. There were papers and assignments and

schedules inserted anywhere and everywhere, and the organization in general was

appalling.

Many times, as is the case with PLTW, it is assumed that students enrolled in a

college-preparatory class have the ability to efficiently and effectively take notes on

presented information, however, in many cases, the evidence proves otherwise.

Therefore, there was a need to develop and test organizational instruments that aid in the

students’ note-taking efforts in PLTW classes in the hopes that this information gathering

8

tool would benefit their educational process and lead to improved performance on end-of-

unit and end-of-course exams.

Statement of the Problem

Project Lead the Way has provided a wealth of resources for teachers to present

accurate and relevant information to students. However, these same students have

demonstrated a general inability to efficiently and effectively record this information for

later use. A promising strategy that seeks to address the issue is a set of guided notes that

provide students a framework for recording information from each presentation.

However, the measured benefits of the proposed instructional variation in terms of aiding

and developing student study skills has not been established. Therefore, the purpose of

this research is to determine if the inclusion of the organizational instruments will

improve student performance on end-of-unit and end-of-course exams. More

specifically, the study sought to address the following hypotheses:

1. The inclusion of the organizational instruments will increase the level of student

performance on end-of-unit assessments.


performance on daily assignments and activities.


performance on national end-of-course assessments.

9

Definition of Terms

To address the research questions and illuminate the existing literature, the

following terms will be used.

Study skills are a set of behaviors exhibited by a student which, when performed

correctly, will significantly contribute to academic success. Study skills include, but are

not limited to: recording lecture notes, active listening, homework completion, and

information recall.

Note-taking is an activity within the greater umbrella of study skills which

includes two basic aspects, listening to the lecture, and recording information. However,

effective note-taking involves more than just listening and recording. One must be

engaged and involved in the lecture as an active listener, and in addition, the information

that is recorded must be done so in a manner so as to provide easy and correct

recollection at a later time.

Test performance is simply the scores earned by a student on a graded

assessment, though this performance can include objective performance: multiple choice,

matching, true/false as well as subjective performance as well: short answer, essay,

sketching.

Guided Notes are an organizational tool employed to aid the student during the

note-taking process. Guided notes provide a number of organizational cues for the

student to use while observing a lecture. These cues may include: fill in the blanks,

bulleted lists, sketching areas, and blank pages for example problems.

Project Lead the Way pre-engineering curriculum is the subject organization in

the present study. The focus of PLTW is increasing the number of graduating engineers

10

from American colleges and universities, and their method is providing curriculum at the

secondary level for engineering instruction.

Active Listening is a behavior vital to the success of a student within a lecture-

type setting. The active listener: records important information from the lecture, engages

him or herself in the lecture by asking questions, completes all in-class activities, and

individually reviews the presented information at the conclusion of the lecture as well as

prior to the assessment.

11

Chapter II: Review of Literature


performance within the Principles of Engineering course at a high school in central

Wisconsin and ultimately how to increase the overall level of student performance on

daily assignments in addition to tests and quizzes. More specifically, the study examined

the effect of guided notes on student performance on formal assessments, on daily

assignments and activities, and on the perceptions of students relative to guided note

inclusion.

The following review of literature will highlight the importance of note-taking

strategies and study skills in student success by examining the efficacy of a variety of

advanced organizers available to students. A theory examining the state of student note-

taking skills and a theory supporting the role of the note-taking and study skills will be

presented. The key variables include note-taking and cognition, note organization, and

notes and achievement, that factor into its characterization.

Background

Perhaps one of the most daunting tasks facing students is the ability to effectively

and efficiently take notes in a lecture-type instructional setting (Ryan, 2001). This

sentiment is echoed by a number of researchers (Neef, McCord, Ferreri, 2006, Austin, et

al, 2002, Baker, Lombardi, 1985), in fact, the pervasiveness of this observation suggests

many students lack general note-taking study skills. Despite this fact, efforts on behalf of

educational institutions to address this deficiency among students have largely been

unaddressed. Baker and Lombardi (1985) go on to say “Students must develop their

note-taking skills on their own, deciding for themselves how much and what kinds of

12

information they should include” (p. 28). When left to the student, as little as 50% of key

information is included in their notes (Baker, Lombardi, 1985).

Though taking accurate notes and actively listening to the lecture are expected to

occur simultaneously, Austin et al (2002) find that these activities compete for finite

cognitive resources. When focusing on accurately taking notes, the student sacrifices

actively participating in the lecture. Conversely, when the student chooses to participate,

the information presented is not recorded for future use. A potential solution to this

tradeoff is the use of guided notes by the instructor. Well-designed guided notes will

assist students by providing a general structure to the notes, limited writing requirements

to reduce transcription error and allowing for more opportunities to listen and participate

(Neef, McCord, and Ferreri, 2006). There is significant evidence supporting this

proposition. One study of note found not only did student performance on associated

assessments improve when comparing the practice of presenting with an overhead

accompanied by guided notes as opposed to presenting solely with an overhead, but the

pace at which students recalled information improved as well (Austin, et al, 2002).

Theory Base

The review of literature highlighted a number of benefits as to the inclusion of

guided notes within a lecture-based academic course, and ultimately the reasons why

guided notes are a successful addition when seeking to increase the level of student

performance. Heward (1994) developed a modest list of advantages associated with the

inclusion of guided notes:

Students produce complete and accurate lecture notes

Guided notes increase students’ active engagement with course content

13

Students can more easily identify the most important information

Students are more likely to ask the instructor questions

Students earn higher quiz and exam scores with guided notes

Guided notes can serve as an advance organizer for students

Instructors must prepare the lecture carefully

Instructors are more likely to stay on-task with the lecture’s content and sequence

Guided notes help instructors prioritize and limit lecture content

Guided note content can be easily converted into test/exam questions

Students like guided notes and appreciate instructors who prepare them

Through the review of literature, it was widely discussed that students simply do not

know how to take notes (Baker and Lombardi, 1985, Austin et al, 2002, Ryan, 2001).

Instructors from middle school through post-secondary struggle with this issue, because it

presents a significant obstacle for students that must not only identify and comprehend

presented information but also effectively record it for future use. In some cases

instructors must simply teach students how to study before embarking on course content,

but if done correctly, it will yield significant benefits for student performance in the

future (Montis, 2007, Lazarus, 1996).

Because students have difficulties recording important information from lectures,

educational theory suggests that any effort to organize student note-taking would yield

positive results. Through the review of literature many positive outcomes were observed

within guided note lectures. Daniels asserts that guided notes aid in retention, signal the

main points of a lecture and can increase students’ opportunities to respond. Guided

notes allow students to transition from a simple observer or recorder to an active

14

participant in the presentation because a fraction of the efforts are put toward the task of

recording information (Boch and Piolat, 2005). Students can engage in class discussions,

solve problem examples and ask questions in lieu of writing information for the sake of

reviewing it later, after the lecture is finished. This theoretical framework provides a

strong foundation towards the inclusion of guided notes within a lecture-based

educational setting.

Note-taking and Cognition

The practice of taking notes by students during an academic lecture has a

profound impact on future performance. Research has shown that simply recording

information as opposed to passively observing the lecture will significantly improve

performance (Heward, 1994, Daniels, Boch and Piolat, 2005). Research suggests that

while ideally students will be engaged in higher-order thinking during a lecture, in most

cases, students are recording no more than a list of relevant terms (Kiewra & Fletcher,

1984). To further improve the academic performance of the student, an increase in

cognitive interaction is necessary (Hohn and Gallagher, 1990). However, without

providing the student with some sort of information recording instrument, the situation

will not improve. Significant research has found that some sort of guided note system

will aid in this situation (Neef, McCord and Ferreri, 2006, Hohn and Gallagher, 1990,

Weishaar and Boyle, 1999).

There is significant responsibility on the student, as the consumer of the

information, to effectively use the gathered information. An assumption that can be

made is that regardless of the note-taking strategy available to the student, if it is not

used, its benefits will not be realized. When the student is required to recall information

15

for practical application, it has been found that notes of any sort serve as an “external

storage medium” freeing up internal capacity for cognitive operations (Boch and Piolat,

2005). Notes that follow a logical progression or graphical order as determined by a

relative expert (the teacher) as opposed to a novice (the student) will further reduce

cognitive capacities toward deciphering recorded information.

Note Organization

When left to their own accord, students most often record only key terms from the

lecture without regard for the connections between terms or recording extended phrases

or in-class examples. In addition, there is no guarantee of the accuracy of student

generated notes. Whether through transcription errors, inability to record given

information within the allotted time or errors when reviewing generated notes, there are

many opportunities for inaccuracy. In theory, if errors prove to be a problem, one simple

way to eliminate the issue is simply giving students copies of an instructor’s complete

lecture notes. While transcription errors may be eliminated, Hohn and Gallagher (1990)

suggest that if students are not actively engaged in the lecture process, they will fail to

make cognitive connections necessary for greater comprehension.

Several researchers (Hartley, 1976, Kiewra, 1988, Hohn and Gallagher, 1990)

suggest that “skeleton notes” provide the greatest level of informational recall. Students

using guided notes or skeleton notes are given a “script” of the lecture with several

details omitted. While this solution fails to provide every detail of the lecture topics, it

offers enough of a framework to the point that students are less involved with the process

of taking the notes, and more involved with understanding the notes that are being taken

(Baker and Lombardi, 1985).

16

Note-taking and Achievement

Baker and Lombardi (1985) discovered that without the aid of guided notes,

students included only 50% of targeted main ideas within a lecture. Given this as the

starting point, students performed better on quizzes and exams with the aid of guided

notes than without (Austin et al, 2002, Heward, 1994). Several key indicators also saw

marked improvement when guided notes were employed. Austin et al (2002) cites that

quiz performance improved, more questions were being asked by students, but more

follow-up questions were being asked by the instructor as well. These findings suggest

that while the same material was covered with and without guided notes, the inclusion of

guided notes made the lecture process more efficient, and time was available for extended

or enriched learning.

The greatest obstacle to student achievement is students learning how to learn.

Multiple researchers (Baker and Lombardi, 1985, Ryan, 2001, Boch and Piolat, 2005)

suggest that students simply do not know how to take notes. Regardless of educational

level (middle, secondary or post-secondary) an unwritten expectation on the part of the

instructor is that students have the ability to effectively and efficiently take notes to the

point that students can focus solely on the delivered content, which has been proven to be

most definitely untrue (Ryan, 2001). Due to this unfortunate reality, any help given to

the student will be beneficial.

17

Chapter III: Methodology


performance. It examined the effect of guided notes on student performance on end-of-

unit and end-of-course tests, and daily assignment performance.

Research Design

A quasi-experimental design was used to test the research hypotheses through the

use of an equivalent time-samples design. Students in one section of the Project Lead the

Way class “Principles of Engineering” served as subjects in conducting the experiment.

Since there was only one section of the class available, a concurrent control group was

not possible, student performance on individual units served to provide the measure of

impact of guided notes. Initially, a pre-test of short answer questions was administered,

after which, students were asked to simply record information in their own notebooks

during the first instructional unit. A post-test was then administered at the end of the unit

to determine the change in performance due to note-taking behaviors. During the second

unit the process was similar, but the students had the benefit of using a teacher-generated

set of guided notes for recording information. This alternated pattern continued for each

unique unit presented to the class through the duration of the class. Data was collected to

determine the difference in the change of performance during the units with guided notes

versus units without guided notes. The threats to internal validity included multiple-

treatment interference, in which, students may have changed note-taking habits in

subsequent units after exposure to the framework of the guided notes. This poses an

interesting and additional reference for data collection. That being the improvement of

18

student performance on non-guided note units, simply due to the practice of taking clear

and concise notes.

Population and Sample

The population for this study was high school sophomores, juniors and seniors

that enrolled in the Project Lead the Way pre-engineering curriculum at a high school in

central Wisconsin. The sample for this line of inquiry was the 23 tenth, eleventh and

twelfth graders enrolled in the one section of Principles of Engineering course during the

2011-2012 school year.

Treatment

The treatment for the class involved a set of guided notes for students to reference

and complete while course content was presented. The 36 weeks of instruction was

divided into approximately eight unique units, each of which had an associated electronic

presentation for presenting the main concepts. The guided notes available for student use

were composed of a number of different note-taking cues and methods; including, but not

limited to, fill in the blank answers to posed questions, completing a phrase with the key

word(s), diagram notation, and variable definition. The classroom instruction closely

followed the order of information on the sheet to provide a logical flow of events. In

addition to guided notes, example problems were posed on the notesheets for student

reference while the instructor offered a solution in a large group format.

Instrumentation

The instrumentation for this study was pre-tests administered before an

instructional unit and post-tests administered after a particular instructional unit. These

tests consisted of multiple distinct short answer questions. The pre-test was administered

19

prior to any discussion of the content of the new unit. This pre-test was graded and the

results were recorded, but did not have an effect on student grades, this data provided a

baseline for future reference. Upon completion of the pre-test, content instruction began,

unit assignments and activities occurred and ultimately at the end of the unit, students

were administered the same questions as part of a more comprehensive post-test to

discover the relative improvement in performance. The impact of the treatment was

based on the difference in relative improvement between guided note units and non-

guided note units. Unique units within the Principles of Engineering curriculum include

the following topics: Simple Machines, Energy Sources, Energy Applications, Electricity,

Statics, Material Properties, Material Testing, Machine Control, Fluid Power, Statistics

and Kinematics. Scoring the short answer questions of the test was based on a simple

four point scale, one point representing the recording of known and unknown variables,

one point representing the recording of appropriate formulas, one point representing the

substitution of variables into the formulas including work and one point representing the

final answer.

Data Collection Procedure

During the initial stage of each experimental unit within the curriculum, all

students completed a pre-test. This pre-test consisted of multiple-choice questions that

survey the content of the upcoming unit. In addition, students were made aware of the

fact that their score on the pre-test was graded but did not affect their course grade.

After the pre-test was completed, students received the set of guided notes for use

during the electronic presentation of unit content. The unit content was presented in a

manner consistent to units without the benefit of guided notes. This set of guided notes,

20

completed by the student, was available for use during all activities associated with the

unit.

Upon the completion of the associated activities and assignments for a given unit,

a post-test was then be administered to all students. This post-test was, in part, identical

to the pre-test, and the change in score between pre-test and post-test was recorded in

each experimental unit.

There were a number of units in which guided notes were not used. This practice

allowed for collection of baseline data to help determine the benefit of guided notes. In

the case of the units without guided notes, the pre-test, presentation, activities, post-test

procedure remained the same, however, the only change was students were required to

record information for themselves.

The order in which units were delegated for guided note use were as follows: Unit

1 – no notesheet, Unit 2 – notesheet. The order simply alternated through the progression

of the units. In addition to examining the benefit of guided notes for students, a

secondary analysis was the relative change in the difference between pre-test and post-

test scores in the units without guided notes. A change in scores that may be attributed to

students employing better note-taking practices.

In addition to the quantitative data collected, a survey was administered at the end

of the course (see Appendix B). This survey investigated the students’ perceptions of

guided notes. The survey contained eleven statements and a five point Likert scale to

judge the students agreement with the statement. Questions on the survey investigated

student perceptions on personal engagement during lectures, note-taking behavior and

levels of cognition during lectures.

21

Data Analysis

A simple t-test was used to compare the mean scores from the units delivered with

notes (treatment units) with those of the units delivered without notes (control units). A t-

test was also used to determine if any differences between the pre-test scores and the

post-test scores represent a significant gain. The results of the analysis were used to

address the original research problem.

22

Chapter IV: Results

The purpose of this study was to identify the effects of including guided notes

during classroom presentations on student test performance. The research design utilized

a pretest/posttest model to gather information about the relative effect of using guided

notes during instruction. In this chapter demographic information and data addressing

each research objective will be discussed.

The subjects for this study were high school students enrolled in the Principles of

Engineering class within the Project Lead the Way secondary pre-engineering curriculum

at a high school in central Wisconsin during the 2011-2012 school year. A total of 23

students, 21 male and 2 female participated in the study.

Research Objectives

The study sought to address the following hypotheses:

1. The inclusion of the organizational note taking instruments will increase the level

of student performance on end-of-unit assessments.


of student performance on daily assignments and activities.


of student performance on national end-of-course assessments.

Findings

The first research objective sought to identify the impact of guided notes on

performance on unit tests. To answer this question, the researcher administered a

multiple-choice pretest prior to the formal instruction of units within the Principles of

Engineering curriculum; Mechanisms (with guided notes), Energy

23

Applications/Thermodynamics (no guided notes), Energy Sources/Electricity (with

guided notes) and Statics (no guided notes), Fluid Power (with guided notes), Kinematics

(no guided notes), and Statistics (with guided notes). The pretests were collected, scored

and entered as ungraded activities to provide a baseline. Formal instruction was then

presented, lab activities took place, and daily assignments were given. At the end of the

unit, a comprehensive unit test was administered to the students; this test was composed,

in part, of the same questions that made up the pretests.

Table 1

Pretest performance

Unit Name (guided notes) # of points Average Range Std. Dev.

1.1 Mechanisms (yes) 10 3.8 8 1.7

1.2 Energy Sources (yes) 10 3.5 7 2.3

3.2 Fluid Power (yes) 10 6.0 7 1.9

4.1 Statistics (yes) 13 5.2 9 3.6

1.3 Energy Applications (no) 10 6.5 5 2.1

2.1 Statics (no) 10 4.2 6 1.8

4.2 Kinematics (no) 12 4.8 5 1.9

24

Table 2

Posttest performance and percent change (posttest vs. pretest)

Unit Name (guided notes) # of points Average Range Std. Dev. % change

1.1 Mechanisms (yes) 27 21.5 13 2.7 40%

1.2 Energy Sources (yes) 43 35.3 16 4.9 46%

3.2 Fluid Power (yes) 10 8.6 3 1.3 26%

4.1 Statistics (yes) 13 9.5 9 2.9 34%

1.3 Energy Applications (no) 70 61.0 19 4.9 21%

2.1 Statics (no) 20 13.6 7 2.9 26%

4.2 Kinematics (no) 12 6.8 5 2.0 15%

One point of note is the wide variety of student performance on unit pretests.

While all pretests were similar in design (i.e. ten to fifteen question, multiple-choice),

each was significantly different in inclusive content. Upon review of the pretest

performance, the researcher noticed two curious data points in the Energy

Applications/Thermodynamics and Fluid Power pretests. Upon further investigation, it

was found that a large percentage of students had recently completed a similar unit within

their science class, thus artificially skewing the pretest performance higher and much of

the basic concepts covered in the fluid power pretest (work, force, efficiency) were

already discussed earlier in the class. Data such as this suggests that repeated coverage of

fundamental knowledge across curricular areas is beneficial as it reinforces student

learning.

25

Another point to note is the relative similarity in posttest performance regardless

of the note-taking strategy employed. Despite the variety of note-taking strategies

employed, the test performance indicates that students as a whole do exhibit a moderate

level of knowledge of these fundamental engineering concepts.

The relationship between the pre-test and post-test scores for unit 1.1 Mechanisms

was investigated using Microsoft Excel to determine a Pearson correlation coefficient.

There was a small negative correlation between the pre-test scores for Mechanisms and

post-test scores for Mechanisms [r=-0.156, n=21]. A low level (2.4%) of lower pre-test

scores associated with higher post-test scores.

A paired two sample t-test was conducted using Microsoft Excel to evaluate the impact of

teaching with notes on students’ unit 1.1 Mechanisms pre-test and post-test scores. There

was a statistically significant increase in Mechanism scores from pre-test (M = 3.90, SD

= 1.76) to post-test (M = 8.00, SD = 1.10), t (21) = -8.49, p<.05 (p = .000000046). The

eta squared statistic (78.26) indicates a large effect size.

The relationship between the pre-test and post-test scores for unit 1.2 Energy

Sources was investigated using Microsoft Excel to determine a Pearson correlation

coefficient. There was a small negative correlation between the Energy Sources pretest

scores and Energy Sources post test scores [r=-0.190, n=21]. A low level (3.6%) of

lower pre-test scores associated with higher post-test scores.

A paired two sample t-test was conducted using Microsoft Excel to evaluate the

impact of teaching with notes on students’ unit 1.2 Energy Sources pre-test and post-test

scores. There was a statistically significant increase in Energy Sources scores from pre-

26

test (M = 6.00, SD = 2.28) to post-test (M = 8.14, SD = 1.15), t (21) = -3.58, p<.05 (p =

.0019). The eta squared statistic (39.04) indicates a large effect size.

The relationship between the pre-test and post-test scores for unit 1.3 Energy

Applications was investigated using Microsoft Excel to determine a Pearson correlation

coefficient. There was a small negative correlation between the Energy Applications pre-

test and the Energy Applications post-test [r=-0.377, n=21]. A low level (14.2%) of

lower pre-test scores associated with higher post-test scores.


impact of teaching without notes on students’ unit 1.3 Energy Applications pre-test and

post-test scores. There was a statistically significant increase in Energy Applications

scores from pre-test (M = 6.67, SD = 2.08) to post-test (M = 8.67, SD = 0.66), t (21) = -

3.81, p<.05 (p = .0011). The eta squared statistic (41.93) indicates a large effect size.

The relationship between the pre-test and post-test scores for unit 3.2 Fluid Power


There was a small positive correlation between the Fluid Power pre-test and Fluid Power

post-test scores [r=0.187, n=21]. A low level (3.5%) of lower pre-test scores associated

with higher post-test scores.


impact of teaching with notes on students’ unit 3.2 Fluid Power pre-test and post-test

scores. There was a statistically significant increase in Fluid Power scores from pre-test

(M = 6.00, SD = 1.79) to post-test (M = 8.10, SD = 1.04), t (21) = -5.07, p<.05 (p =

.000059). The eta squared statistic (56.21) indicates a large effect size.

27

The relationship between the pre-test and post-test scores for unit 4.1 Statistics


There was a large positive correlation between the Statistics pre-test and Statistics post-

test scores [r=0.545, n=21]. A low level (29.70%) of lower pre-test scores associated

with higher post-test scores.


impact of teaching with notes on students’ unit 4.1 Statistics pre-test and post-test scores.

There was a statistically significant increase in Statistics scores from pre-test (M = 3.99,

SD = 1.96) to post-test (M = 7.11, SD = 0.15), t (21) = -8.34, p<.05 (p = .000000061).

The eta squared statistic (77.66) indicates a large effect size.

The relationship between the pre-test and post-test scores for unit 4.2 Kinematics


There was a large positive correlation between the Kinematics pre-test and Kinematics

post-test scores [r=0.629, n=21]. A moderate level (39.56%) of lower pre-test scores

associated with higher post-test scores.

A paired two sample t-test was conducted using Microsoft Excel to evaluate the impact of

teaching without notes on students’ unit 4.2 Kinematics pre-test and post-test scores.

There was a statistically significant increase in scores from pre-test (M = 3.97, SD =

1.18) to post-test (M = 6.51, SD = 0.15), t (21) = -9.53, p<.05 (p = .0000000071). The

eta squared statistic (81.95) indicates a large effect size.

The second research objective sought to identify the impact of guided notes on

performance on daily assignments within the unit. Each unit contained a homework

assignment as well as an in-class lab activity and the work was graded based on rubrics as

28

developed by Project Lead the Way. The researcher recorded the grades associated with

daily assignments within each unit.

Table 3

Daily Assignment Performance

Assignment (Guided Notes) Class Average %

1.1 Mechanisms Homework (yes) 81%

1.1 Mechanisms Lab Activity (yes) 71%

1.2 Energy Sources Homework (yes) 80%

1.2 Energy Sources Lab Activity (yes) 80%

3.2 Fluid Power Homework (yes) 79%

3.2 Fluid Power Lab Activity (yes) 91%

4.1 Statistics Homework (yes) 79%

4.1 Statistics Lab Activity (yes) 97%

1.3 Energy Applications Homework (no) 61%

1.3 Energy Applications Lab Activity (no) 62%

2.1 Statics Homework (no) 68%

2.1 Statics Lab Activity (no) 70%

4.2 Kinematics Homework (no) 70%

4.2 Kinematics Lab Activity (no) 97%

One point of note is the approximate 11% relative performance difference

between units with and without guided notes. While each unit presented its own unique

content and associated challenges, a constant element available to the students was the

29

manner in which the information was presented during the initial unit presentation.

Information for all units was presented in an electronic presentation format during a

single class session and the electronic presentation file was available for students to

review beyond the day of the presentation on a shared directory within the school. The

notes generated, whether guided or not, were available for reference by the students

throughout any activities and assignments within the unit.

The relationship between performance on the homework and lab activity within a

particular unit was also examined. An initial homework assignment typically was given

following the unit presentation. This assignment was collected, graded and returned for

student review and was typically followed by a modest activity highlighting the unit

topic. During the simple machines unit, students were asked to calculate the ideal

mechanical advantage of a variety of common tools and mechanisms within the facility

(e.g. ironworker shear, drill press, bolt cutters, bicycle). Within the thermodynamics unit,

the activity focused on calculating the heat loss or gain experienced by the students’

home on a winter day. Student groups tested the electrical properties of examples of

various electrical circuits in the electricity unit. Students built, calculated and tested

pasta bridges within the statics unit. Student groups designed, built and tested hydraulic

power tabletop robotic arms within the fluid power unit, analyzed trajectory motion data

from ping pong ball launchers in the kinematics unit and finally students calculated

properties of similar bags of M&M’s for the purpose of statistical analysis.

The statistical difference between student performance on homework both with

and without guided notes was investigated by performing a two-factor without replication

ANOVA test using Microsoft Excel. Findings suggest there was a statistically significant

30

difference in relative student performance (F(1, 21 = 2.12, p < .002), suggesting that the

student that scores well on homework in a guided note unit will continue to score well on

homework in a non-guided note unit, and conversely, a student that scores lower on

homework in a guided note unit is more likely to score lower on homework in a non-

guided note unit regardless of the inclusion of guided notes. The data analysis indicated

that guided notes did not provide a statistically significant difference in class average

performance on homework in guided note units versus non-guided note units (F(1, 21 =

4.35, p =0.155). However, data analysis suggests that guided notes did appear to

positively impact scores 85% of the time, suggesting that the practical significance of the

inclusion of guided notes is real and measureable in the classroom. Regardless, this

suggests that the observed gains in student performance in other areas (pretest vs. posttest

performance and end-of-course assessment performance) are not realized to the same

extent in daily homework performance.

The third research hypothesis sought to identify the impact of guided notes on

student performance on the Project Lead the Way end-of-course assessment. To answer

this question, the end-of-course assessment was administered to the group, and the class

performance was analyzed relative to performance on individual questions. PLTW

provides a wealth of data mining tools within the assessment tool, each question has been

referenced to the overall course concept map, so the unit in which the question was

generated is provided for the teacher as well.

31

Table 4

Student Performance on End-of-Course Assessment

Unit Name (guided notes) # of questions Average Range Std. Dev.

1.1 Mechanisms (yes) 8 5.7 4 1.3

1.2 Energy Sources (yes) 8 4.1 5 1.6

3.2 Fluid Power (yes) 7 2.6 5 1.2

4.1 Statistics (yes) 3 0.9 2 0.6

1.3 Energy Applications (no) 6 3.8 5 1.3

2.1 Statics (no) 12 7.4 7 1.8

4.2 Kinematics (no) 7 4 5 1.2

The end-of-course assessment was composed of 80 multiple choice questions

administered via online testing software. Students were given a prepared formula sheet

as reference during the test. When analyzing the data above, it should be noted that

though all units covered were represented within the scope of the end-of-course

assessment, the number of questions relating to each unit varied significantly (a low of

three to a high of twelve). While the performance data relative to each particular unit

varies widely, this can, in part, be attributed to the number of questions available. For

example, each question relative to Statistics accounts for 33% of the reported grade for

that unit. Should a student happen to incorrectly answer a single Statistics question, the

resulting data swing is very significant.

Data analysis began by normalizing all unit test scores because the number of

questions was different for each unit. Student scores on all questions from units using

guided notes were summed as were student scores from all units not using guided notes.

32

SPSS PASW Statistics 18 was used to compute a Pearson correlation coefficient between

the six subtests and the composite "with guided note" and "without guided note" scores.

The strength of the relationship was interpreted using Cohen's 1988 conventions.

Table 5

Subtest versus Overall Test Performance Correlation

1.1 1.2 3.2 1.3 2.1 4.2

Yes Pearson Correlation (r ) 0.582 0.808 0.437

Significance (2 tailed) 0.006 0.000 0.048

Percentage of Variance 33.87% 65.29% 19.10%

No Pearson Correlation (r )

0.623 0.712 0.579

Significance (2 tailed)

0.003 0.000 0.006

Percentage of Variance

38.81% 50.69% 33.52%

There was a strong positive relationship between the students' scores on the unit

1.1 - Mechanisms, a unit of instruction provided with notes, and the final composite "with

guided notes" score, as measured by the end-of-course Project Lead the Way - Principles

of Engineering (PLTW-POE) test [r=.582, n=21, p=.006]. Approximately thirty four

percent [33.87%] of the score on the end-of-course PLTW-POE test can be explained by

scores on unit 1.1 questions where students were able to use guided notes.


1.2 - Energy Sources, a unit of instruction provided with notes, and the final composite

"with guided notes" score, as measured by the end-of-course Project Lead the Way -

Principles of Engineering (PLTW-POE) test [r=.808, n=21, p=.000]. Approximately sixty

33

five percent [65.29%] of the score on the end-of-course PLTW-POE test can be explained

by scores on unit 1.2 questions where students were able to use guided notes.


3.2 - Fluid Power, a unit of instruction provided with notes, and the final composite "with

guided notes" score, as measured by the end-of-course Project Lead the Way - Principles

of Engineering (PLTW-POE) test [r=.437, n=21, p=.048]. Approximately nineteen

percent [19.10%] of the score on the end-of-course PLTW-POE test can be explained by

scores on unit 3.2 questions where students were able to use guided notes.


1.3 - Energy Applications, a unit of instruction provided without notes, and the final

composite "without guided notes" score, as measured by the end-of-course Project Lead

the Way - Principles of Engineering (PLTW-POE) test [r=.623, n=21, p=.003].

Approximately thirty nine percent [38.81%] of the score on the end-of-course PLTW-

POE test can be explained by scores on unit 1.3 questions where students were not able to

use guided notes.


2.1 - Statics, a unit of instruction provided without notes, and the final composite

"without guided notes" score, as measured by the end-of-course Project Lead the Way -

Principles of Engineering (PLTW-POE) test [r=.712, n=21, p=.000]. Approximately fifty

one percent [50.69%] of the score on the end-of-course PLTW-POE test can be explained

by scores on unit 2.1 questions where students were not able to use guided notes.


4.2 - Kinematics, a unit of instruction provided without notes, and the final composite

34

"without guided notes" score, as measured by the end-of-course Project Lead the Way -

Principles of Engineering (PLTW-POE) test [r=.579, n=21, p=.006]. Approximately

thirty four percent [33.52%] of the score on the end-of-course PLTW-POE test can be

explained by scores on unit 4.2 questions where students were not able to use guided

notes.

In addition to the data collected, a student survey was administered at the end of

the experimental period. This survey was essential a measure of student perceptions of

guided notes. The students had positive reactions to the use of guided notes (see Table

4). The questions were answered with a five point Likert scale (5 = strongly agree, 4 =

agree, 3 = neutral, 2 = disagree, 1 = strongly disagree).

35

Table 6

Student Survey Results

Question Average Response

1. I was better able to pay attention on days when guided notes were used 4.13

2. There was more time to ask questions on days when guided notes were used 3.78

3. There was more class participation on days when guided notes were used 3.86

4. Guided notes helped me improve my note-taking behavior 4.39

5. I learned more on days when guided notes were used 4.47

6. I retained more of the information on days when guided notes were used 4.21

7. I participated more in class on days when guided notes were used 3.60

8. I spent more time listening and thinking about concepts

on days when guided notes were used 3.86

9. I prefer using guided notes to traditional note-taking 4.34

10. I would recommend that guided notes be used in future classes 4.47

11. Overall, I had a positive reaction to guided notes 4.39

36

Chapter V: Summary, Conclusions and Recommendations

The purpose of this study was to examine the effects of guided notes as an

organizational tool on student performance within secondary pre-engineering courses.

More specifically, the study examined the effect of guided notes on student performance

on end-of-unit tests, and daily assignment on-time completion and performance.

The basic design for this investigation involved a pretest/posttest model to collect

the performance data. The subjects for this study were high school students enrolled in

the Principles of Engineering class within the Project Lead the Way secondary pre-

engineering curriculum at a high school in central Wisconsin. A total of 23 students, 21

male and 2 female participated in the study, which was conducted throughout the 2011-

2012 academic year.

Within the course curriculum, seven units were the subject units for the study.

Students completed a pretest consisting of multiple choice questions prior to any

instruction. Then, in four of the units (Simple Machines, Electricity, Fluid Power and

Kinematics), students were given a set of guided notes to follow, and in the other three

units (Thermodynamics, Statics and Statistics), students were not given any

organizational tool, and the electronic presentation was presented. Two elements of data

were collected throughout the course of each unit: student performance on daily

assignments and student performance on end-of-unit tests. In addition, the performance

data of the students was also recorded on the end-of-course assessment at the end of the

school year. The results were used to identify the impact of guided notes on overall

student performance within the Project Lead the Way - Principles of Engineering class.

37

Findings

The inclusion of guided notes proved to be a moderate benefit to student

performance on end-of-unit exams. While guided notes did not statistically prove to be

of significance in terms of ultimate score on end-of-unit or end-of-course exams, what

was significant was the change in pretest versus posttest performance. While the class

performance on all unit posttests was similar, the pretest performance was markedly

lower in the guided note units, thus attributing to the higher change in performance. The

improvement due to guided notes is consistent with the supporting literature (Austin et al,

2002, Baker and Lombardi, 1985). Furthermore, it was found that performance on daily

assignments improved as well. While the content of each unit was unique, the structure

within the unit was relatively consistent; including homework assignments and at least

one lab activity. The data from the current study supports related literature, which

suggest that a moderate gain can be expected when guided notes are used in lecture and

then as a reference during subsequent assignments (Neef, McCord and Ferreri, 2006).

Student perceptions of guided notes were recorded as well, and as suggested by a

host of previous research, the students overwhelmingly supported the use of guided notes

(Neef, McCord and Ferreri, 2006, Austin et al, 2002, Heward, 1996). While ultimately

the performance on graded assignment and tests prove the efficacy of guided notes,

students identified multiple benefits of their inclusion. As a whole, the class felt guided

notes improved the lecture experience by allowing them more opportunities to listen to

and participate in the lecture. They found they were able to ask more questions, record

and recall information easier, and engage in the experience at a deeper level.

38

Conclusions

Based on the findings of this study, the following conclusions were drawn:

The change in student performance on generic unit pretests versus posttests was

consistently statistically significantly, however, the change proved to be greater

when guided notes were employed as opposed to when students were left to their

own accord in generating lecture notes.

Performance on daily assignments, including homework, lab activities, and

presentations improved with the inclusion of guided notes.

Students overwhelmingly embraced and supported the use of guided notes. A

strong majority found the structure provided by guided notes to be of significant

benefit as a reference in their comprehension of the lectured topic.

In addition to the notes serving as a reference and providing a benefit to

comprehension, students felt the use of guided notes allowed them additional

opportunity to ask questions and engage in discussion to more fully understand

the lectured topic as is evidenced by the student perception survey.

The addition of guided notes provides a framework for logical and timely

progression through content presentation for the teacher. Rather than relying

solely on students to pace the instruction due to note transcription, or dedicating

an inordinate amount of time in a presentation to a particular aspect of a topic,

guided notes created by the teacher provide a clear content outline for student and

teacher alike.

39

Recommendations

Based on the findings and conclusions of this study, the following

recommendations were drawn:

To maximize student performance within a lecture-based academic setting, the

use of guided notes, or “skeleton notes” will best serve students, relative to

student-generated notes or completed teacher-generated lecture notes.

Guided notes require a high degree of correlation to the presentation, therefore, it

is essential that a well-designed presentation be generated first with an eye toward

clarity, detail and a logical progression through the topic, then a set of guided

notes that references the presentation can be easily generated.

Expect additional opportunities for student interaction due to the decreased

transcription demands that guided notes afford. Therefore, additional discussion

and/or example problems can be generated with an expectation of including them

in the allotted lecture time. Correspondingly, allow room in the physical structure

of the guided notes for the recording of this additional material.

Rather than simply providing the guided notes for students, an introduction to

using guided notes may be necessary. Instruct students on the use of guided notes

during the lecture and subsequently in their review throughout the unit. Other

considerations to be made in this regard are using a consistent format between

guided notesheets and observing an appropriate level of student interaction with

the guided notes during the lecture. Too much writing will not allow the student

to realize the full potential of guided notes; too little writing will decrease student

involvement and ultimately student learning.

40

References

Arslan, M. (2006). The influence of teaching note-taking and information mapping on

learning and recalling in science. The Turkish Online Journal of Educational

Technology, 5(2), 56-63.

Austin, J.L. (2002). Effects of guided notes on university students' responding and recall

of information. Journal of Behavioral Education, 11(4), 243-254.

Austin, J. L., Lee, M., & Carr, J. P. (n.d.). The effects of guided notes on undergraduate

students' recording of lecture content. Journal of Instructional Psychology, 31(4),

314-320.

Baker, L, & Lombardi, B.R. (1985). Students' lecture notes and their relation to test

performance. Teaching of Psychology, 12(1), 28-32.

Boch, F., & Piolat, A. (2005, September). Note taking and learning: A summary of

research. The WAC Journal, 16, 101-113.

Boyle, J. R. (2001). Enhancing the note-taking skills of students with mild disabilities.

Intervention in School and Clinic, 36(4), 221-224.

Boyle, J. R. (2010, Spring). Strategic note-taking for middle-school students with

learning disabilities in science class. Learning Disability Quarterly, 33, 93-109.

Hartley, J. (1976). Lecture-handouts and student notetaking. Programmed Learning and

Educational Technology, 13, 58-64.

Heward, W. L. (1994). Fast facts for faculty: Guided notes. Retrieved from

http://ada.osu.edu/resources/fastfacts/Guided_Notes.htm

Hohn, R. L., Gallagher, T., & Byrne, M. (1990). Instructor-supplied notes and higher-

order thinking. Journal of Instructional Psychology, 17(2), 71-74.

41

Kiewra, K. A., DuBois, N. F., Christian, D., & McShane, A. (1988). Providing study

notes: comparison of three types of notes for review. Journal of Educational

Psychology, 80(4), 595-597.

Lazarus, B. D. (1988). Using guided notes to aid learning disabled adolescents in

secondary mainstream settings. The Pointer, 35(1), 32-35.

Lazarus, B. D. (1991). Guided notes, review, and achievement of secondary students with

learning disabilities in mainstream content courses. Education and Treatment of

Children, 14(2) 112-128.

Lazarus, B. D. (1996). Flexible skeletons: Guided notes for adolescents. Teaching

Exceptional Children, 28(3), 36-40.

Montis, K. K. (2007, June 22). Guided notes: An interactive method for success in

secondary and college mathematics classrooms. Focus on Learning Problems in

Mathematics, 29(3), 55-68.

Neef, N.A., McCord, B.E., & Ferreri, S.J. (2006). Effects of guided notes versus

completed notes during lectures on college students' quiz performance. Journal of

Applied Behavior Analysis, 39, 123-130.

Pallant, J. (2001). SPSS survival manual. Philadelphia, PA: Open University Press.

Ryan, M.P. (2001). Conceptual models of lecture learning. Reading Psychology, 22, 289-

312.

Suritsky, S. K., & Hughes, C. A. (1991). Benefits of note-taking: Implications for

secondary and postsecondary students with learning disabilities. Learning

Disability Quarterly, 14, 7-9.

42

The University of Kansas. (n.d.). Guided notes. Retrieved from

http://www.specialconnections.ku.edu/?q=instruction/universal_design_for_learni

ng/teacher_tools/guided_notes

Weishaar, M. K., & Boyle, J. R. (1999). Note-taking strategies for students with

disabilities. The Clearing House, 72(6), 392-395.

43

Appendix A: Sample Guided Notesheet

PLTW-POE Electricity Notesheet

Electrical Properties

Property Unit Symbol

Current – ___________________________________________ Switch off - _____________________ Switch on - ______________________ Think garden hose and spigot example. Current Flow Conventional Current – flows from ___________ to ____________ Electron Flow - flows from ___________ to ____________ Engineers use _______________________________ Voltage - ____________________________________________ Switch off - _____________________ Switch on - ______________________ Think garden hose and spigot example. Resistance - __________________________________________ Greater Resistance = _____________________ Multimeter Measuring voltage – measure across ________________________ Measuring current – circuit must ___________________________ Measuring resistance – measure across ______________________ Note: Power must be off when measuring resistance

44

Ohm’s Law – Relationship between Voltage, Current and Resistance V = ______ X _______ I = _______ / _______ R = _______ / _______ Series Circuit –

_____________________________________________________________

_____________________________________________________________ Series Circuit Diagram Parallel Circuit –

_____________________________________________________________

_____________________________________________________________ Parallel Circuit Diagram

45

Kirchhoff’s Voltage Law (KVL) – The sum of all __________ _______ in a __________ circuit equals the total __________ _________. Kirchhoff’s Current Law (KCL) – The ________ __________ in a __________ circuit equals the _______ of the individual __________ __________. Series Circuit Properties Contains only _______ path for current flow If the path breaks, _________________________________________________________ Current flowing through each component is __________ Total resistance (_____) = __________________________________________________ The total applied voltage (_____) = ___________________________________________ Also known as - ________________________________________ Parallel Circuit Properties Contains ________ _________ _________ path for current flow If one path breaks, ________________________________________________________ The voltage across each parallel circuit is __________ The total resistance (_____) = _______________________________________________ Copy formulas here: The total current (____) = __________________________________________________ Also known as - ________________________________________ Power - ________________________________________________________________ P = _______ X ________

46

Appendix B: Student Survey

1. I was better able to pay attention to the lecture on days when guided notes were used.

Strongly Agree Agree Neutral Disagree Strongly Disagree

2. There was more time to ask questions on days when guided notes were used.


3. In general, there was more class participation on days when guided notes were used.


4. Guided notes helped me improve my note-taking behavior


5. I learned more on days when guided notes were used.


6. I retained more of the information presented in class on days when guided notes were used.


7. I participated more in class on days when guided notes were used.


8. I spent more time listening and thinking about concepts presented in class on days when guided notes were used.


9. I prefer using guided notes to traditional note-taking.


10. I would recommend that guided notes be used in future classes.


11. Overall, I had a positive reaction to guided notes.


the effect of guided notes on student performance on ... · pdf filethe effect of guided notes...

Documents