stages of test construction and quantitative analysis: criteria of a good test

Stages of Test Construction

Prepared by: Von Ameri Escobar

In the preparation and assessment phases the following stages are carried out:

1. PlanningOnce the need for a language test arises, the teacher or test developer determines a few essentials, before the actual writing begins.

• Goal

If the goal is assigning students to a level before a course begins, rather than evaluating them afterwards, your test design will reflect this.

• Format

The test developer will have to consider a few format-related dilemmas, such as paper-based test versus digital one.

• TasksThere are a myriad

of possible test tasks, all of which can be used in different formats for different level.A non-exhaustive list of

commonly used tasks might include the following:

Multiple Choice

General term for any task type where the appropriate answer is picked from a list of alternatives

Single response

A multiple choice task where only one option is possible

Tick the item with the correct spelling

Multiple response

A multiple choice task where more than one option is possible.

Which of the following are causes of global warming?

True/False The student is given a statement and should indicate whether it is correct or not.

Does the journalist agree with the interviewee?

Ranking In this task, students are given list of items which they are asked to rank (i.e. by dragging or assigning a number)

Retrace the chronological order in which these events took place.

Matching A kind of multiple choice question in which the test taker is asked to combine two or more items that belong together.

(Reading task)Connect which statements to paragraph they summarize

Hotspot On a drawing or a picture students are asked to select a specific area.

Indicate on the picture where the Dolphin’s vocal chords used to be:

Open answer Any type of question in which the test taker is asked to produce written or spoken sentences or texts

Text-numeric As an answer to a question, students fill in words, short sentences or numbers.

(listening test) Fill in the omitted dates.

Cloze In a running text one or more words or numbers have been deleted. It is up to the test taker to fill in the gaps.

Fill in the correct prepositions…

C-Cloze A cloze test which includes the first letter of each deleted word.

Fill in the correct words.

Extended response

Students can be asked to reply to an open question or to produce longer

Explain your opinion about Johnson’s behaviour

2. Design

a.Collecting testing material

b.Writing a draft version

c.Evaluating the draft version

d.Rewriting the draft version

3. Development

a.PilotingThe test is

distributed among a group of representatives end users

b. RevisingThe final test be

composed.

QUANTITATIVE ANALYSIS

Criteria of a good test

In creating a valid and reliable language test for general or specific purposes, a quantitative or qualitative test analyses are of utmost importance

A complete formal analysis requires a thorough psychometric knowledge, whereas informal analyses are less rigid and can be undertaken with relative ease.

Prior to the analysis component, there are key concepts which are central to good tests.

1.Relevance2.Representativity3.Authenticity4.Balance

5. ValiditySub classifications of validity:

a.Concurrent validityA test is said to have

concurrent validity if the scores it gives correlate highly with a recognized external criterion which means the same area of knowledge or ability.

b. Construct validity

A test is said to have construct validity if scores can be shown to reflect a theory about the nature of a construct or its relation to other constructs.

Content validity

A test is said to have content validity if the items or tasks of which it is made up constitute a representative sample items or tasks for the area of knowledge or ability to be tested

Convergent validityA test is said to have convergent validity when there is a high correlation between scores achieved in it and those achieved in different test measuring the same construct (irrespective of method).

Criterion-related validity

• A test is said to have criterion-related validity if a relationship can be demonstrated between scores and some external criterion which is believed to be a measure of the same ability

Discriminant validity

A test is said to have discriminant validity if the correlation it has with tests of a different trait is lower than correlation with tests of the same trait, irrespective of testing method.

Face validity

The extent to which a test appears to candidates, or those choosing it on behalf of candidates, to be an acceptable measure of the ability they wish to measure

Predictive validityAn indication of how well a test predicts future performance in a relevant skill.

Factors that influence validity:

1. Appropriateness of test items2. Directions3. Reading vocabulary and sentence structure

4. Difficulty of items

5. Construction of test items

6. Length of the test7. Arrangement of items8. Patterns of answers

6. ReliabilityReliability refers to the consistency and stability with which a test measures performance.

A number of variables influence test reliability:

1. Specificity2. Differentiation

3. Difficulty4. Length5. Time6. Item construction

Over the past years, the focus of test construction has shifted from reliability to validity and more specifically construct validity. Additionally, tests are increasingly considered as part of the educational practice.

The more reliable a test is, the less random error it contains. A test which contains systematic error, e.g. bias against a certain group, may be reliable, but not valid.

Possible reasons for the inconsistency of an individual’s score in a test• Scorer’s inconsistency• Limited sampling of behavior• Changes in the individual himself

Factors affecting reliability, factors which influence the reliability of a test

• Objectivity• Difficulty of the test• Length of the test• Adequacy• Testing condition• Test administration

procedures

END OF SLIDE

Thank you for listeningKomawo! Saranghe! Te amo!