rdsa ii paper final draft

Running head: THE EFFECTS OF A TIME LIMIT ON EXAM PERFORMANCE 1

The Effects of a Time Limit on Exam Performance

Anthony J. DeFilippo

Providence College

THE EFFECTS OF A TIME LIMIT ON EXAM PERFORMANCE 2

Abstract

The effects of a time limit on exam performance were measured using a Stroop Task

study of 21 students affiliated with the Providence College Psychology Department.

The participants were asked to complete a Stroop Task, which is a competitive attention

task. Participants were randomly assigned to one of three conditions: no time limit,

visual time limit, or an audio time limit to measure the effects of a time limit on test

performance. Each participant’s test was scored based on the percentage of correct

responses they provided. The results suggest no relationship between time limit and

test performance, meaning the presence of a time limit did not affect one’s score on the

Stroop task.


The Effects of a Time Limit on Exam Performance

Examinations are used in order to test students’ retention of the material covered

in a given subject. However, what makes taking an exam so stressful? One of the

reasons why test-taking conditions are so stressful is the time limit that is imposed upon

the test-takers. Students often feel a selective pressure due to the limited time and

must resort to rushing to finish the exam. The time limit should play a minor role in

reflecting test scores because it is useful in measuring one’s ability to think quickly and

make decisions. However, a test with a time limit often turns into a race against the

clock and the focus turns away from what the person knows and towards how quickly

they can finish it. The validity of the test becomes threatened when time is limited too

much, and it becomes less accurate in reflecting what it was intended to measure.

Providing a time limit on standardized tests also reduces the cost of administering the

tests by reducing the time required for proctors to be present (Bridgeman, McBride, &

Monaghan, 2004).

One of the main findings regarding limiting testing time is that students tend to

score higher on examinations when there is no time limit (DeDonno, Rivera-Torres,

Monis, & Fagan, 2014). In order to test this relationship between test performance and

time limit, 118 undergraduate participants were chosen, all of whom were enrolled in a

psychology course at the time. Participants were randomly divided into two groups: one

group had a time limit and the other did not. Both groups were instructed to take the

math and reading comprehension sections of the Scholastic Assessment Test (SAT).

The time limit used was the same as the instructions provided by the SAT. Using

Pillai’s trace, there were no significant differences in scores between the group with the


time limit and the group without for the reading comprehension. However, for the math

section, the no time limit group scored better than the group with the time limit, showing

the effect of the time limit on test performance (DeDonno et al., 2014).

Test-taking anxiety is also a problem a number of students deal with in the

classroom. The anxiety can often result in students not performing to their highest

potential (Huesman & Frisbee, 2000). Test-taking anxiety and other forms of pressure

are often a result of the presence of a time limit and the wording of the testing

directions. The importance of recognizing this effect caused by instructions is that the

way test directions are worded can be changed in order to cause the least amount of

stress possible. Even when a time limit is used, the wording of the instructions can

often alleviate some of the pressure and anxiety, relieving the test-takers and allowing

them to perform their best. Telling test-takers that they have an extended time on a

given examination is often a solution, and can be used to reassure test-takers that they

have more time to complete the examination (Huesman & Frisbee, 2000).

The effects of a time limit on examination performance are also important when it

comes to students with learning disabilities. Students with learning disabilities have a

harder time completing an exam during the allotted time and are often given extended

time, even on standardized tests like the SAT (Alster 1997). As a result of the time limit

extension, students’ with learning disabilities tend to perform better on tests. Their

scores are much better with the time extension and are close to if not the same as

students without learning disabilities taking the same test (Alster 1997).

The purpose of this study was to determine the effects of a time limit on test

performance. The hypothesis for this study was that participants taking a test without a


time limit would score higher than participants taking the same test with a time limit.

The independent variable was the testing environment. The timing methods were oral

and visual methods. The dependent variable was the participant’s performance on the

test, which was measured by the percentage of correct answers the participant chose.

Method

Participants

The participants for the experiment to measure the effects of a time limit on test

performance were 21 students from the psychology department. All 21 of the students

attended Providence College and were currently enrolled in a psychology class. The

participants were not monetarily compensated but received credit for a psychology class

by participating. Participants were treated according to APA ethical guidelines

(American Psychological Association, 2010).

Materials

Participants signed up using Sona Experiment Management System, an Internet

site used for online research studies. The test used was a Stroop Test (Stroop, 1935)

and it was used to assess the overall concentration and performance of the participants.

It consisted of 16 rows of words, all of which were names of colors. However, the words

were displayed in a different color from the color the word was actually describing. The

participants then read each line and wrote the first letter of the color in which the word

was printed under the word using a wax pencil. This measured the reaction time of

participants because it often took participants more time to name the color of ink used

when the word denoted a different color and they were also prone to make more errors.


A computer was used to display the time limit for participants in the visual time limit

condition. A stopwatch was also used in order to keep track of the two minutes for the

test.

Procedure

The participants signed up for the study via SONA. The study consisted of three

groups and had a between subjects design. The participants were randomly placed into

one of the three conditions denoting testing conditions: no time limit, visual time limit,

and auditory time limit. Upon arrival, the participants were greeted by myself or

Michelle Ouellette, seated in the room, and asked to sign a consent form.

Participants in the condition with no time limit were asked to read the directions

carefully for the Stroop Test and were told there was no time limit and to work as quickly

and efficiently as possible. The second part of the directions served as a tutorial for the

participant. It provided simple examples of the test the participant completed. The real

test was more challenging, and this was the test that was scored and analyzed. The

participants were stopped after two minutes of taking the test.

For participants in the visual time limit condition, they read the same two pages

of directions. In addition, they were told they had two minutes to complete the test, and

there was a live clock displayed on the computer that counted down the two minutes.

Participants in the third condition received and read the same instructions as

participants in the other two conditions. They were told they had two minutes to

complete the task. The researcher then notified the participant when 30 seconds had

passed, 60 seconds had passed, 90 seconds had passed, and finally when the full two


minutes had passed. After the 2 minute time limit had expired, the participants were

asked to hand in their answers, were debriefed, and given .5 credits on SONA.

Results

The results from the Stroop Task were measured based on the how many words

each participant provided an answer for, which consisted of identifying the color ink

each word was printed in. The mean number of correct responses of participants who

completed the Stroop task in three different time limit conditions was compared using a

one-way ANOVA (See Table 1 in Appendix A). No significant difference was found,

F(2,18) = 1.017, p = 0.381. The number of correct responses did not differ if there was

a time limit, visual or audio, or not. Participants who completed the task with no time

limit had a mean score of 81.86 (sd = 12.93). Participants who completed the task with

a visual time limit had a mean score of 92.00 (sd = 15.37). Participants who completed

the task with an oral time limit had a mean score of 95.43 (sd = 24.99) (See Figure 1 in

Appendix B).

The mean number of attempted responses of participants who completed the

Stroop task in three different time limit conditions was compared using a one-way

ANOVA (See Table 2 in Appendix A). No significant difference was found, F(2,18) =

0.982, p = 0.394. The number of attempted responses did not differ if there was a time

limit, visual or audio, or not. Participants who completed the task with no time limit had

a mean score of 83.14 (sd = 12.60). Participants who completed the task with a visual

time limit had a mean score of 93.00 (sd = 15.07). Participants who completed the task

with an oral time limit had a mean score of 96.14 (sd = 24.46) (See Figure 2 in

Appendix B).


The mean percentage of correct responses of participants who completed the

Stroop task in three different time limit conditions was compared using a one-way

ANOVA (See Table 3 in Appendix A). No significant difference was found, F(2,18) =

0.425, p = 0.660. The percentage of correct responses did not differ if there was a time

limit, visual or audio, or not. Participants who completed the task with no time limit had

a mean score of 98.38% correct (sd = 1.35). Participants who completed the task with a

visual time limit had a mean score of 98.84% correct (sd = 1.72). Participants who

completed the task with an oral time limit had a mean score of 99.09% correct (sd =

1.27) (See Figure 3 in Appendix B).

An independent-samples t test was calculated comparing the mean percentage

of correct responses of participants who had previous exposure to the Stroop task to the

mean percentage of correct responses of participants who did not have previous

exposure (See Table 4 in Appendix A). No significant difference was found, t(19) =

-1.227, p = 0.235. The mean of participants with previous exposure (m = 98.34, sd =

1.65) was not significantly different from the mean of participants without previous

exposure (m = 99.10, sd = 1.18) (See Figure 4 in Appendix B).

Discussion

The computed F values for the one-way analysis of variance (ANOVA) suggest

that there was not a significant difference between the mean number of correct

responses for participants in the no time limit, visual time limit, and audio time limit

conditions. No significant difference was found for the mean percentage of correct

responses either. After computing these results, we decided to test to see if there was

a speed-accuracy trade off. This means that participants would have worked faster and


attempted to answer more questions when a time limit was present and were not as

concerned about getting every answer correct. A one-way ANOVA was computed for

number of attempted responses in order to test the speed-accuracy trade off, and the

obtained F value suggests no significant difference was present. An independent

samples t-test was also performed comparing the mean percentage of correct

responses for participants who had previous exposure to the Stroop task and those who

did not. However, no significant result was found, suggesting previous exposure had no

effect on the scores of the participants.

One of the main strengths of my study is the use of the 1-way ANOVA as the

inferential statistics method for analyzing the data. Analysis of variance is used for

determining whether significant differences exist in an experiment that has two or more

conditions of the independent variable. There are 3 advantages to using ANOVA, all of

which are strengths of my study. The first is that it will allow me to adequately test my

hypothesis. Also, testing more than two conditions of the independent variable more

accurately describes the type of relationship present. The third strength is that it allows

me to obtain the most information I can from my study. The use of random assignment

of participants to conditions is also a strength because it increases the internal validity.

One of the main weaknesses of this study, however, is the poor external validity,

or the extent to which the results of the survey for the participants taking my survey will

be generally similar to that of other participants in other situation. The poor external

validity of the study is a result of the small number of participants in the study, all 21 of

which attend Providence College, are psychology majors, and receive credit for a

psychology class for participating. There will also be a low power because of the small


number of participants. Another weakness of this study was the use of the Stroop Task

as the test. It was very basic and almost every participant scored close to 100%

correct, showing there was possibly a ceiling effect in the study. One way we could

have improved this study was to include a manipulation check as an additional measure

of the influence of the independent variable. This could have been done by measuring

the level of anxiety, via self-report, that the participants experienced during the study as

a result of a time limit. A manipulation check would have been particularly useful for our

study because of the ceiling effect that occurred as a result of the relative easiness of

the Stroop task. We would expect that participants’ levels of anxiety would be

significantly greater when in a time limit condition as opposed to a no time limit

condition.

The purpose of this study was to determine the effects of a time limit on test

performance. One of the major effects of the pressure from a time limit is test-taking

anxiety. As a result of this anxiety, the test-takers may not perform to the best of their

abilities and their test performance will not be an accurate representation of their

academic abilities. This issue is often debated about with the SAT and how the time

limit for each math, English, and reading comprehension section can thwart a student’s

performance. This is a problem because standardized tests like the SAT are a big

component in the college application process. If a student does poorly on the test

despite their academic abilities, they may not be accepted to a highly selective school of

their choice. There are many components that contribute to one’s intelligence and as

we continue to test students under time limits, it becomes a less accurate reflection of

their actual intellectual capabilities.


References

Alster, E. H. (1997). The effects of extended time on algebra test scores for college

students with and without learning disabilities.The Journal of Learning Disabilities,

30 (2), 222-227.

American Psychological Association. (2010). Ethical principles of psychologists and

code of conduct. Retrieved from http://apa.org/ethics/code/index.aspx

Bridgeman, B., McBride, A., & Monaghan, W. (2004). Testing and time limits. R&D

Connections, 1(1), 1-6.

DeDonno, M. A., Rivera-Torres, K., Monis, A., & Fagan, J. F. (2014). The influence of a

time limit & bilingualism on scholastic school assessment test performance.

North American Journal Of Psychology, 16(2), 211-223.

Huesman, R. L., & Frisbie, D. (2000). The validity of ITBS reading comprehension test

scores for learning disabled and non learning disabled students under extended-

time conditions. Educational Resources Information Center, 1(1), 1-37.

Stroop, J. (1935). Studies of interference in serial verbal reactions. Journal of

Experimental Psychology, 18(6), 643-662.


Appendix A

Table 1: One-way ANOVA Results for Number of Attempted Responses:

df SS MS F Sig.

Between Groups 2 644.095 322.048 0.982 0.394

Within Groups 18 5903.714 327.984

Total 20 6547.810

Table 2: One-way ANOVA for Number of Correct Responses:

df SS MS F Sig.

Between Groups 2 697.238 348.619 1.017 0.381

Within Groups 18 6168.571 342.698

Total 20 6865.810

Table 3: One-way ANOVA Results for Percent Correct:

df SS MS F Sig.

Between Groups 2 1.811 0.906 0.425 0.660

Within Groups 18 38.385 2.132

Total 20 40.196

Table 4: t-test Results Comparing Previous Exposure to Percent Correct:

Previous Exposure N Mean SD t-calc t-crit df p Decision

No 9 98.338 1.6544 4.106 -1.227 19 0.235 Reject

Yes 12 99.096 1.1813


Appendix B

Figure 1: Descriptive Statistics for One-way ANOVA for Number of Correct Responses

Figure 2: Descriptive Statistics for One-way ANOVA for Number of Attempted Responses

0

20

40

60

80

100

120

Mean Number of Correct Responses vs. Condition

No Time Limit

Visual Time Limit

Oral Time Limit

Participant Condition

Mea

n N

um

ber

of C

orre

ct R

esp

onse

s

0

20

40

60

80

100

120

Mean Number of Attempted Responses vs. Condition

No Time Limit

Visual Time Limit

Oral Time Limit

Participant Condition

Mea

n N

um

ber

of A

ttem

pte

d R

esp

onse

s


Figure 3: Descriptive Statistics for One-way ANOVA for Number of Attempted Responses

Figure 4. Descriptive Statistics for t-test for previous exposure

0

20

40

60

80

100

120

Percent of Correct Responses vs. Condition

No Time Limit

Visual Time Limit

Oral Time Limit

Participant ConditionMea

n P

erce

nta

ge o

f Cor

rect

Res

pon

ses

10

20

40

60

80

100

120

Percent of Correct Responses vs. Previous Exposure

No Previous Exposure

Previous Exposure

Exposure to Stroop Task

Mea

n P

erce

nta

ge o

f Cor

rect

Res

pon

ses

rdsa ii paper final draft

Documents