1

Educational Measurement and Evaluation

Subjective Test Subjective test require students to write present an original answer. Include short answer

essay, extended-response essay, problem solving, and performance tasks. subjective tests

are evaluated based on the judgment or opinion of the examiner. Subjective tests are

more challenging and expensive to prepare, administer and evaluate correctly, but they

can be more valid. Tests of this nature are often designed in a manner in which the student

is presented with a number of questions or writing prompts for which he/she will

demonstrate mastery of the learning objective in his/her response to the question. When

composing prompts as test questions, it is crucial that you phrase the prompt clearly and

precisely.

Subjective test include the following items:

Short answer Essays

Short answer Short-answer questions are open-ended questions that require students to create an answer. They are commonly used in examinations to assess the basic knowledge and understanding (low cognitive levels) of a topic before more in-depth assessment questions are asked on the topic.

Structure of Short Answer Questions Short Answer Questions do not have a common structure. Questions may require answers such as complete the sentence, supply the missing word, short descriptive or qualitative answers, diagrams with explanations etc. The answer is usually short, from one word to a few lines. Often students may answer in bullet form. Examples

1. MHz measures the _________________ of the computer.

2. List the different types of test?

3. State third law of motion?

2

Advantages of Short Answer Questions Short Answer Questions are relatively fast to mark and can be marked by different

assessors, as long as the questions are set in such a way that all alternative answers can be considered by the assessors.

Short Answer Questions are also relatively easy to set compared to many assessment methods.

Short Answer Questions can be used as part of a formative and summative assessment, as the structure of short answer questions are very similar to examination questions, students are more familiar with the practice and feel less anxious.

Unlike MCQs, there is no guessing on answers, students must supply an answer.

Disadvantages of Short Answer Questions Short Answer Questions (SAQ) are only suitable for questions that can be answered with

short responses. It is very important that the assessor is very clear on the type of answers expected when setting the questions, because SAQ is an open-ended questions, students are free to answer any way they choose, short-answer questions can lead to difficulties in grading if the question is not worded carefully.

Short Answer Questions are typically used for assessing knowledge only, students may often memorize Short Answer Questions with rote learning. If assessors wish to use Short Answer Questions to assess deeper learning, careful attention (and many practices) on appropriate questions are required.

Accuracy of assessment may be influenced by handwriting/spelling skills

There can be time management issues when answering Short Answer Questions

How to design a good Short Answer Question? Design short answer items which are appropriate assessment of the learning

objective

Make sure the content of the short answer question measures knowledge appropriate to the desired learning goal

Express the questions with clear wordings and language which are appropriate to the student population

Ensure there is only one clearly correct answer in each question

Ensure that the item clearly specifies how the question should be answered (e.g. Student should answer it briefly and concisely using a single word or short phrase? Is the question given a specific number of blanks for students to answer?)

Consider whether the positioning of the item blank promote efficient scoring

Write the instructions clearly so as to specify the desired knowledge and specificity of response

Set the questions explicitly and precisely.

3

Direct questions are better than those which require completing the sentences.

For numerical answers, let the students know if they will receive marks for showing partial work (process based) or only the results (product based), also indicated the importance of the units.

Let the students know what your marking style is like, is bullet point format acceptable, or does it have to be an essay format?

Prepare a structured marking sheet; allocate marks or part-marks for acceptable answer(s).

Be prepared to accept other equally acceptable answers, some of which you may not have predicted.

Concept of Essay Test

The word essay has been derived from a French word ‘essayer’ which means ‘to try’ or

‘to attempt’.

Definition:

“Essay test is a test that requires the student to structure a rather long written response

up to several paragraphs.”

William weirsama

Essay test is a type of test that usually allows greater freedom of response to questions and requires more writing. It is a free answer kind of test. In this case the examinees or the students have the responsibility of thinking out the answers to the questions asked. They have the freedom to express or state the answers in their own words. It is used by teachers to measure achievement, performance etc from classroom instruction. The answers to such questions which the learners are confronted differ in quality and degree of correctness.

Characteristic of Essay Test

These are:

`Students answer small number of questions. Because of time limits, It is not

always possible that all the topics be covered by the tests.

The script are written in the students own style, words and usually in his

own handwriting.

The students are considerably free to organize their own answers. This implies

that answers with varying degrees of accuracy and completeness.

4

Types of Essay Questions

C.C. Weidman (1933, 1941) identified ten types of essay question which he arranged in

order from simple to complex. These questions begin with:

1. what, who, when, which, and where

2. list

3. outline

4. describe

5. contrast

6. compare

7. discuss

8. develop

9. summarize

10. evaluate

Types of Essay test

Essay questions are classified into two types

I. Restricted response

II. Extended response

Extended Response or Free Response

These are responses to essay questions in which the examinee is only restricted by time

as no bound is placed as regards the in-depth, breadth and the organization of the

response. In response to such a question the examinee demonstrates his ability to select

and recall the facts which he thinks are appropriate, organize and present his ideas in a

logical and coherent form. This freedom to decide which facts the thinks is most pertinent

to select his own method of organization and to write as much as seems necessary for a

comprehensive answer tends to reveal the ability to evaluate ideas, relate them,

coherently and to express them briefly. In addition, they expose the individual differences

in attitudes, values and creative ability.

5

This type of essay item is mostly useful in measuring learning outcomes at the higher

cognitive levels of educational objectives such as analysis, synthesis and evaluation levels.

Although, the extended response essay type are also limited by two weaknesses which are:

They are insufficient for measuring knowledge of factual materials because they

call for extensive details in selected content area at a time.

Scoring such responses is usually difficult and unreliable since the examinees

have free will in the array of factual information of varying degree of correctness,

coherence and expression. These limitations are minimized in the Restricted

Response Type.

Examples of Free Response or Extended response questions

Describe the sampling technique used in research studies.

Explain the various ways of preventing accident in a school workshop or

laboratory.

What are the implications of fuel price hike in our economy?

Why should the classroom teacher state his instructional objectives to cover

the three domains of educational objectives.

Restricted Response

In this type, the questions are so structured that the students are limited, the scope of the

response is defined and restricted. The answers given are to some extent controlled. In

this type the examinee is limited to the nature, length or organization of response to be

made. The items are directional questions and are aimed at the desired responses. This

limits the examinee freedom to select, recall, and synthesize all that he knows and to

present them logically as he may wish. This type of essay item is most useful in measuring

learning outcomes at the lower cognitive levels of educational objectives, that is

knowledge, comprehension and application levels.

The restricted nature of the expected response in this type of items makes it more efficient

for measuring knowledge of factual material. It reduces to a reasonable extent the

difficulty of scoring and encourages more reliability in scoring. However, the restriction

6

makes it less effective as a measure of ability to select, organize and integrate ideas and

present them in an original and coherent form which is one of the major advantages of

essay test.

Examples of Restricted Response Essay Questions

Give three advantages and two disadvantages of essay tests.

State four uses of tests in education.

Describe four sources of energy.

Constructing the Essay Questions

An essay test is a useful measurement instrument only to the extent that it is constructed,

administered and scored to ensure a high level of objectivity. For this reason, essay test

items should consist of items that will ensure the same understanding and elicit only the

skill or ability one is interested in measuring from every examinee.

The following points are suggested as guide for construction of good essay

test items:

Essay questions are to be used only when it’s desirable and very adequate for

measuring the learning outcomes for full realization of learner’s achievement. In

other words, they are to be used for questions that call for complex learning

outcomes that be appropriate to the organization, integration and expression of

ideas which would not have been possible without the use of essay test items.

Essay questions should be designed to elicit only the skill which the item was

intended to measure. This can be achieved by expressing clearly and precisely the

question in line with clearly defined instructional objective. In addition, an action

verb like compare, contrast, illustrates, differentiates, criticized and so on could be

used to give the test items more focus.

Phrase each question to clearly indicate the examinees task. An essay question has

to be precise . Ensure that the examinee’s task is clearly indicated by delimiting

the area covered by the item, using descriptive words to give specific direction

towards the desired response. Indicate the score allotted to the test. This

suggestion easily lend itself to restricted response type and care should be taken

7

not to narrow the questions when constructing the extended response type in order

not to reduce its effectiveness as a measuring of the ability to select, organize and

integrate ideas.

Adapt the length and complexity of the answer to the examinees’’ level of maturity.

Indication of approximate time limit for each question. It is necessary to indicate

time allotted to each question to enable the examinee to pace their writing on each

question and to allay any anxiety that might arise.

Avoid the use of optional questions because, optional questions might influence

the validity of test results

It is also not easy to construct essay questions of the same difficulty level. Hence,

making valid comparisons of performance among them especially for norm

reference setting will not be possible.

When to use Essay Questions

Use essay questions in the measurement of complex achievement when its distinctive

feature of freedom of response is required. Learners are free to select, relate and present

ideas in their own words. This freedom enhances the value of essay questions as a measure

of complex achievement but it leads to scoring difficulties that make them insufficient as

measure of factual knowledge.

Essay questions should also be used to measure those learning outcomes that cannot be

measured by objective test items. The specific features of essay questions can be utilized

most fully when their shortcomings are offset by the need for such measurement.

Essay questions should be used when learning outcomes concerned with the abilities to

select, organize, integrate, relate, and evaluate ideas require the freedom of response and

the originality provided by essay questions .More so, when these outcomes are of such

great educational significance that the expenditure of energy in the difficulty and time-

consuming task of evaluating the answers can be easily justified.

Advantages of Essay test

The essay test has got some advantages in the area of measurement and evaluation. Some

of these advantages are:-

8

The essay test permits a freedom of response, which in turn

allows the students to prevent their ideas as detailed as they choose so as to

show how deep knowledge they have in the subject area covered by the question.

The free response allows the student to express himself in his own words making use

of his proficiency in the language being used to his advantage.

Essay tests promotes the development of problem – solving skills. This is because

the student has to think out the answer himself and put it down in organized form.

It helps students to improve their writing skills such as writing speed, eligibility etc

because they write in their handwriting.

It encourages creativity by allowing their own unique way. It discourages guess-

work and encourages good study habits in students.

It measures complex learning outcomes that cannot be measured by other means.

For example, it has the ability to measure learner’s communication skills

It measures the learner’s ability to produce an answer, synthesize and organize ideas

and present them readably in a logical and coherent form.

It also enables the measurement of organizational and different thinking skills by

laying emphasis on the integration and application of thinking and problem solving

skills, creativity and originality.

It is very applicable for measuring learning outcomes at the higher levels of

educational objectives such as application, analysis, synthesis and evaluation of

levels of the cognitive domain.

It is easy and economical to administer. It can be easily and conveniently written on

the chalkboard because of the few items involved. This saves materials and time for

production.

Essay item is easy to construct and does not take much time

It can be used to measure in-depth knowledge especially in a restricted subject

matter area.

It does not encourage guessing and cheating during testing.

9

Disadvantages of the Essay test

Although the advantages already listed for essay test, it does not satisfy some important

qualities of a good measuring instrument.

They include that:

They also have poor psychometric quality or measurement qualities although

popular among classroom teachers especially those who are deficient in the skill

required for item construction.

It is inadequate in sampling subject matter content and course objectives since it

provides limited sampling. The provision of few questions results in the invalid

and narrow coverage of subject matter and instructional objectives.

Fewness of the number of questions often asked encourages permutation of some

content areas and creaming of ideal responses to suspected questions. In this

regard, essay questions discourage the development of good study habit.

In addition to the invalidity of the measurement, evaluating the answers to

carelessly developed questions tends to be confusing and time consuming task.

This results in poor reliability in scoring.

Studies have shown that answers to essay questions are scored differently by

different teachers and that even same teachers score the answers differently at

different times.

Sometimes an essay question implies many skills other than that which the item

was intended to measure. The examinee therefore perceives and reacts to the same

questions differently. The differences in the perception of the questions encourage

bluffing and hides differences in the knowledge of basic factual material and the

learners ability to use and organize such facts.

The essay test item does not readily lend itself to empirical study of item qualities

like difficulty and discrimination based on which improvements on the item could

be made.

Scoring is not reliable because different examiners can grade the score answer

differently. In fact, the same examiner can grade the same question differently at

different times.

10

Grading of essay tests is time-consuming.

Essay questions do not cover the course content and the objectives as

comprehensively as possible.

Good command of language places individual students at an

advantage while poor command places some students at a disadvantage.

How To Make Essay Test Less Subjective

As subjectivity is a major limitation of the essay tests. It can be reduce to the barest

minimum after following these tips.

Avoid open-ended questions.

Let the students’ answer the same questions. Avoid options/choices.

Use students’ numbers instead of their names, to conceal their identity.

Score all the answers to each question for all students at a time.

Do not allow score on one question to influence you while marking the next.

Always rearrange the papers before you mark.

Do not allow your feelings or emotions to influence your marking.

Avoid distractions when marking

Scoring Essay Test

Essay test scoring calls for higher degrees of competence, and ordinarily takes

considerably more time, than the scoring of objective tests. In addition to this, essay test

scoring presents two special problems. The first is that of providing a basis for judgment

that is sufficiently definite, and of sufficiently general validity, to give the scores assigned

by a particular reader some objective meaning. To be useful, his scores should not

represent purely subjective opinions and personal biases that equally competent readers

might or might not share. The second problem is that of discounting irrelevant factors,

such as quality of handwriting, verbal fluency, or gamesmanship, in appealing to the

scorer’s interests and biases. The reader’s scores should reflect unbiased estimates of the

essential achievements of the examinee.

11

One means of improving objectivity and relevancy in scoring essay tests is to prepare an

ideal answer to each essay question and to base the scoring on relations between examinee

answers and the ideal answer. Another is to defer assignment of scores until the examinee

answers have been sorted and resorted into three to nine sets at different levels of quality.

Scoring the test question by question through the entire set of papers, rather than paper

by paper (marking all questions on one paper before considering the next) improves the

accuracy of scoring. If several scorers will be marking the same questions in a set of

papers, it is usually helpful to plan a training and practice session in which the scorers

mark the same papers, compare their marks and strive to reach a common basis for

marking.

The construction and scoring of essay questions are interrelated processes that require

attention if a valid and reliable measure of achievement is to be obtained. In the essay test

the examiner is an active part of the measurement instrument. Therefore, the viabilities

within and between examiners affect the resulting score of examinee. This variability is a

source of error, which affects the reliability of essay test if not adequately controlled.

Hence, for the essay test result to serve useful purpose as valid measurement instrument

conscious effort is made to score the test objectively by using appropriate methods to

minimize the effort of pe0rsonal biases and idiosyncrasies on the resulting scores; and

applying standards to ensure that only relevant factors indicated in the course objectives.

The following are procedures for scoring essay questions objectively to enhance

reliability.

Prepare the marking scheme or ideal answer or outline of expected answer

immediately after constructing the test items and indicate how marks are to be

awarded for each section of the expected response.

Use the scoring method that is most appropriate for the test item. That is, use

either the analytic or global method as appropriate to the requirements of the test

item.

Decide how to handle factors that are irrelevant to the learning outcomes being

measured. These factors may include legibility of handwriting, spelling, sentence

structure, punctuation and neatness. These factors should be controlled when

12

judging the content of the answers. Also decide in advance how to handle the

inclusion of irrelevant materials (uncalled for responses).

Score only one item in all the scripts at a time. This helps to control the “halo” effect

in scoring.

Evaluate the answers to responses anonymously without knowledge of the

examinee whose script you are scoring. This helps in controlling bias in scoring the

essay questions.

Evaluate the marking scheme (scoring key) before actual scoring by scoring a

random sample of examinees actual responses. This provides a general idea of the

quality of the response to be expected and might call for a revision of the scoring

key before commencing actual scoring.

Make comments during the scoring of each essay item. These comments act as

feedback to examinees and a source of remediation to both examinees and

examiners.

Obtain two or more independent ratings if important decisions are to be based on

the results. The result of the different scorers should be compared and rating

moderated to reflect the discrepancies for more reliable results.

13

References

Doher, G. 1991. Do teachers' comments on students' papers help? College Teaching 39:48-54.

Everett, E. 1994. Do the write thing. The Science Teacher 61(7): 35-7.

Mayer, R. E. 1975. Information processing variables in learning to solve problems. Review of

Educational Research 45:525-41.

Moore, R 1992. Writing about biology: How should we mark students' essays? Bioscene: Journal of College Biology Teaching. 18(3): 3-9.

Perkins, D. 1993. Teaching for understanding. American Educator 17(3): 8 and 28-35.

Shavelson, R J. and R Stern. 1981. Research on teachers' pedagogical thoughts, judgments,

decisions, and behavior. Review of Educational Research 51:455-98.

Steele, C. W. 1992. Critique; don't bash. Writing and Learning 3(1): 5 6.

Craig W. Steele is an associate professor in the Department of Biology and Health Services at Edinboro University in Edinboro, Pennsylvania.

http://jan.ucc.nau.edu/~slm/AdjCI/Teaching/Essays.html