11

Chapter 10:Chapter 10: TestsTests

Mitch CardMitch CardEDLT 520: Thea HandEDLT 520: Thea Hand

March 26, 2007March 26, 2007

22

Chapter 10: TestsChapter 10: Tests

Essentially, this chapter reviews the basics of using Essentially, this chapter reviews the basics of using computers when developing tests computers when developing tests

in a computer test center, not necessarily in a in a computer test center, not necessarily in a classroom,classroom,

The information is at least 5 years old, and The information is at least 5 years old, and Is based on the use of Testing Software Is based on the use of Testing Software

developed by major companies (like for the SAT, developed by major companies (like for the SAT, ACT, etc). ACT, etc).

But, much of this is adaptable today to normal But, much of this is adaptable today to normal people like us.people like us.

Things to be covered will include but not be limited to:Things to be covered will include but not be limited to:

1.1. ConstructionConstruction2.2. AdministeringAdministering3.3. ContentContent4.4. SimulationsSimulations5.5. Test SecurityTest Security

33

DefinedDefined

According to According to WikipediaWikipedia, the word test can be , the word test can be defined …defined …

As a “test and experiment (that) form parts As a “test and experiment (that) form parts of the scientific method, to verify or falsify of the scientific method, to verify or falsify an expectation with an observation”.an expectation with an observation”.

As “a student assessment in education, As “a student assessment in education, (also called an "examination" or "exam") (also called an "examination" or "exam") that serves to assess or measure students' that serves to assess or measure students' performance, knowledge or skills”.performance, knowledge or skills”.

44

Before we begin – Before we begin – a word about NCLB and Testinga word about NCLB and Testing

If you would like to know more If you would like to know more about NCLB and testing check-about NCLB and testing check-out the following:out the following:

U.S. Department of Education siteU.S. Department of Education site

ERICERIC

An interesting site:An interesting site:Fair Test: The National Center for Fair Test: The National Center for Fair and Open TestingFair and Open Testing

55

An excerpt from An excerpt from NCLBNCLB:: On what subjects are students tested and when? (This may be On what subjects are students tested and when? (This may be

accomplished using computers in the next year or two)accomplished using computers in the next year or two)

No Child Left BehindNo Child Left Behind requires that, by the 2005-06 school requires that, by the 2005-06 school year, each state must measure every child's progress in year, each state must measure every child's progress in reading and math in each of grades 3 through 8 and at reading and math in each of grades 3 through 8 and at least once during grades 10 through 12. In the meantime, least once during grades 10 through 12. In the meantime, each state must meet the requirements of the previous each state must meet the requirements of the previous law reauthorizing law reauthorizing ESEAESEA (the (the Improving America's Schools Improving America's Schools Act of 1994Act of 1994) for assessments in reading and math at ) for assessments in reading and math at three grade spans (3-5; 6-9; and 10-12). By school year three grade spans (3-5; 6-9; and 10-12). By school year 2007-2008, states must also have in place science 2007-2008, states must also have in place science assessments to be administered at least once during assessments to be administered at least once during grades 3-5; grades 6-9; and grades 10-12. Further, states grades 3-5; grades 6-9; and grades 10-12. Further, states must ensure that districts administer tests of English must ensure that districts administer tests of English proficiency--to measure oral language, reading and proficiency--to measure oral language, reading and writing skills in English--to all limited English proficient writing skills in English--to all limited English proficient students, as of the 2002-03 school year.students, as of the 2002-03 school year.

Students may still undergo state assessments in other Students may still undergo state assessments in other subject areas (i.e., history, geography and writing skills), subject areas (i.e., history, geography and writing skills), if and when the state requires it. if and when the state requires it. No Child Left BehindNo Child Left Behind, , however, requires assessments only in the areas of however, requires assessments only in the areas of reading/language arts, math and science. reading/language arts, math and science.

66

Computerized Test Computerized Test ConstructionConstruction

Advantages:Advantages:

Tests can be designed to be printed or taken on-line,Tests can be designed to be printed or taken on-line, Questions can be re-ordered for each individual so that Questions can be re-ordered for each individual so that

no two tests have the same order of answers/questions,no two tests have the same order of answers/questions, Test pools (banks) can be generated by more than one Test pools (banks) can be generated by more than one

teacher that can be accessed by many teachers at a time teacher that can be accessed by many teachers at a time - for future reference,- for future reference,

Tests can be generated with little notice,Tests can be generated with little notice, Most textbook publishers now supply teachers with test Most textbook publishers now supply teachers with test

banks on CD that can be generated and placed on the banks on CD that can be generated and placed on the Internet, andInternet, and

The tests can be automatically graded and scores e-The tests can be automatically graded and scores e-mailed to the teacher and the student (immediate mailed to the teacher and the student (immediate feedback).feedback).

77

Computerized Computerized Test Test ConstructionConstruction

DisadvantagesDisadvantages

Test security (cheating) becomes an important Test security (cheating) becomes an important issue,issue,

Teachers may not match appropriate questions Teachers may not match appropriate questions to what has actually been taught,to what has actually been taught,

Not all test questions have been produced with Not all test questions have been produced with the same standards of quality, the same standards of quality,

Most graded tests must be restricted to a Most graded tests must be restricted to a multiple choice, matching or short-answer multiple choice, matching or short-answer format (they can not grade essays).format (they can not grade essays).

88

Computerized Test Computerized Test AdministrationAdministration

Although helpful, there Although helpful, there are problems withare problems with

having enough having enough computers to test a class computers to test a class at a time, andat a time, and

if the test structure is if the test structure is confusing or awkward in confusing or awkward in design, test anxiety could design, test anxiety could easily be increased, easily be increased, causing poorer results.causing poorer results.

99

Factors to consider in Factors to consider in anyany Test Test Preparation…Preparation… What is the purpose of the test, what will it What is the purpose of the test, what will it testtest?? Does the test meet the needs of the test taker?Does the test meet the needs of the test taker? Does the test actually assess the target Does the test actually assess the target

objectives of the curriculum?objectives of the curriculum? Is the test length and time allowed appropriate to Is the test length and time allowed appropriate to

the level and age of the student?the level and age of the student?

1010

Test Creator SoftwareTest Creator Software

Must be Must be useruser friendly. friendly. Allow the test generator as well as the test taker Allow the test generator as well as the test taker

appropriate control.appropriate control. Have a ‘Have a ‘safety netsafety net’ to prevent un-wanted deletion ’ to prevent un-wanted deletion

of answers and data from the program – provide of answers and data from the program – provide affirmation phrases.affirmation phrases.

Label tests to make sure the right students are Label tests to make sure the right students are taking the right subject tests.taking the right subject tests.

And, after completion, all new tests should be And, after completion, all new tests should be tested to find errors, unreasonable questions or tested to find errors, unreasonable questions or that the entire testing process works correctly.that the entire testing process works correctly.

1111

Basic Computerized Test Basic Computerized Test RequirementsRequirements

Instructions must be clear and cover all Instructions must be clear and cover all aspects of using the program and aspects of using the program and technology correctly.technology correctly.

There should be a practice page/question There should be a practice page/question that allows the student to see what will that allows the student to see what will happen while they take the test.happen while they take the test.

Be prepared to have problems withBe prepared to have problems with Equipment failure,Equipment failure, Accidental termination of the test, andAccidental termination of the test, and Loss of answers in the data pool.Loss of answers in the data pool.

1212

SimulationsSimulations Needs a computer to Needs a computer to

complete in the first complete in the first placeplace

Can be flexible enough to Can be flexible enough to test skills - including:test skills - including: Lab procedures,Lab procedures, Performance tasks,Performance tasks, Equipment operation & Equipment operation &

management.management.

1313

Adaptive TestingAdaptive Testing-based on responses of the user.-based on responses of the user.

Usually used after some sort of training or Usually used after some sort of training or instruction.instruction.

Incorrect responses lead the testee to a review of Incorrect responses lead the testee to a review of the material.the material.

Correct responses lead the testee to related, Correct responses lead the testee to related, more in depth, questions.more in depth, questions.

1414

Admissable Probability Admissable Probability TestingTesting

Essentially, the computer testing Essentially, the computer testing software has the capacity to grade software has the capacity to grade answers that are only partially answers that are only partially complete.complete.

I am not sure why this is in here, but I am not sure why this is in here, but these tests are rarely used.these tests are rarely used.

Examples:Examples: Admissable Probability Measures Testing (APM)Admissable Probability Measures Testing (APM) Information Referenced Testing (IRT)Information Referenced Testing (IRT)

1515

To ensure security of To ensure security of tests….tests….

Do not allow users to Do not allow users to have anything to record have anything to record answers on while taking answers on while taking the test,the test,

Re-develop tests after Re-develop tests after they are used (using test they are used (using test banks)banks)

Generate new questions Generate new questions and never use used and never use used questions again - a lot of questions again - a lot of work.work.

Take questions off-line Take questions off-line (prevents hacking)(prevents hacking)

Require passwords to Require passwords to any collected test dataany collected test data

1616

So….So….

Stepping away from the chapterStepping away from the chapter Most new scholastic textbooks come with Most new scholastic textbooks come with

test generating software that includes test test generating software that includes test banks.banks.

Generated tests can be converted to HTML Generated tests can be converted to HTML and given on-line (see and given on-line (see exampleexample – put your e-– put your e-mail address in the space provided and get mail address in the space provided and get your graded test sent to you in your e-mail).your graded test sent to you in your e-mail).

These tests are excellent to get students to These tests are excellent to get students to prepare for class discussions, or written prepare for class discussions, or written tests.tests.

So…

1717

Sources Sources – interestingly, there were no URL’s listed in the references for – interestingly, there were no URL’s listed in the references for

this chapter.this chapter.

Alessi, S.M., and Trollip, S.R. (2001). Multimedia for Alessi, S.M., and Trollip, S.R. (2001). Multimedia for learning: methods and development (3learning: methods and development (3rdrd Ed.) Ma: Ally & Ed.) Ma: Ally & Bacon.Bacon.

U.S. Department of Education (n.d.) Stronger U.S. Department of Education (n.d.) Stronger accountibility: accountibility: Testing-frequently asked questionsTesting-frequently asked questions. . Retrieved 3/11/07, from Retrieved 3/11/07, from http://www.ed.gov/nclb/accountability/ayp/testing-faq.htmlhttp://www.ed.gov/nclb/accountability/ayp/testing-faq.html

Wikipedia (n.d.) Test: Wikipedia (n.d.) Test: from Wikipedia, the free on-line from Wikipedia, the free on-line encyclopediaencyclopedia. Retrieved 3/11/07, from . Retrieved 3/11/07, from http://en.wikipedia.org/wiki/Testhttp://en.wikipedia.org/wiki/Test

Images and cartoons collected through Google ‘Images’ Images and cartoons collected through Google ‘Images’ and the MS Picture Library.and the MS Picture Library.

1818

Your Questions….Your Questions….

1.1. Does it appear to you that on-line testing Does it appear to you that on-line testing could be helpful in your teaching format? could be helpful in your teaching format? Why?Why?

2.2. This chapter is mostly referring to This chapter is mostly referring to computerized tests like the SAT. It has been computerized tests like the SAT. It has been proposed that state tests of all kinds should proposed that state tests of all kinds should be tested on-line. How do you think that will be tested on-line. How do you think that will affect test scores, especially for minority affect test scores, especially for minority groups (ESL, LEP, SpEd)?groups (ESL, LEP, SpEd)?