admissionsandutme.com · web view2019-02-28 · evaluation in this context according to unesco...

DESIGNING AND DEVELOPING EVALUATION SYSTEM. Nature of programme evaluation: Summary of the evaluation process

Feb., 2019

NIGER DELTA UNIVERSITY

WILBERFORCE ISLAND,

BAYELSA STATE

DESIGNING AND DEVELOPING EVALUATION SYSTEM

Nature of programme evaluation

Summary of the evaluation process

Word counts: 2616 (main body only)

Julius Michael Oghenenyoreme

PG/M.ED/FED/17/026

Nworsu Ogechi Maryjane

PG/M.ED/FED/17/015

FACULTY OF EDUCATION

DEPARTMENT OF VOCATIONAL AND TECHNOLOGY EDUCATION

BED 812: EVALUATION IN BUSINESS EDUCATION

DR. F. C. AKPOTOHWO

FEBRUARY, 2019

1


Feb., 2019

Abstract

Evaluation is a very key element in

educational processes. Without

evaluation, it is difficult for educators

to know the extent of educational

objectives that has been attained.

Therefore, this paper on “designing

and developing evaluation system.

Nature of programme evaluation:

Summary of the evaluation process”

aim to elucidate the nature and nitty-

gritty of the evaluation process.

Keywords: Evaluation, Scope, design, education.

Introduction

During the period of designing

educational structures, curriculum,

determining managerial hierarchical

order and even within the process of

teaching and learning, the designers

are fully aware that along the line,

there may be need to come back to

check either as the process is still

ongoing (formative) or at the end of

the programme (summative) how far

the previous arrangement has met the

targeted objectives. This is because

complexities and numerous

intervening variables which the

designers did not initially put into

consideration may influence the

programme either positively or

negatively. Thus, evaluation has

become every part of educational

endeavour to streamline or to keep

check of predetermined targeted

objective.

Evaluation in this context according to

UNESCO (2016) is “the systematic

and objective assessment of an

ongoing or completed project,

programme or policy, its design,

implementation and results, with the

aim to determine the relevance and

fulfillment of objectives, development

efficiency, effectiveness, impact and

sustainability”. The above definition

mentioning ongoing is indicative of the

fact that evaluation can be carried out

2


Feb., 2019

intermediately; this is known as

formative evaluation. Additionally,

this definition also implies that

evaluation can also be carried out as at

when the programme is completed;

this refers to summative evaluation. In

any case however, both types of

evaluation are important if educational

programmes are to succeed.

In this paper however, the authors

looked into two specific areas of

evaluation – the nature of programme

evaluation and summary of the

evaluation process. Nonetheless, the

next subheading will look into the

concept of evaluation before the two

areas were delved into.

Concept of educational evaluation

There are various concepts in

educational evaluation that guides the

understanding of what evaluation truly

is. For instance, Educational

evaluation is broader in scope and

more objective than educational

measurement. It is also the process of

carefully appraising a

programme/individual to form a

variety of information giving device.

Besides testing and other tools of

measurement, evaluation seeks

additional evidences from various

sources of information supplementing

each other: like interviews,

questionnaires, anecdotal records,

cumulative records, case conferences,

mechanical or electronic recorders,

case studies or projective techniques,

etc. and the selection, through careful

analysis of data, most pertinent to a

wise just and comprehensive

interpretation to make value judgment

of the programme under evaluation.

In addition, evaluation as mentioned

above can also be carried out

intermediately of after the completion

of the programme, depending on the

discretion of the evaluator, the

environment and the nature of the

programme under evaluation. All these

are done with the sole intention of

3


Feb., 2019

checking how far the programme have

achieved its aim.

Educational evaluation is a systematic

process which means it omits casual,

uncontrolled, unsystematic ways and

means of judging programme's

progress. Secondly, educational

evaluation strives to judge

programme's progress towards

objectives in the curriculum. It

necessitates that objectives in the

curriculum i.e. educational objectives

should be previously determined. This

is because it is not possible to judge

the growth and progress of

programmes towards objectives or

values without previously determining

the educational objectives.

Educational evaluation is a

comprehensive process in the sense

that it has a bearing on all the stages of

the process of education right from

beginning with the determination of

objectives through planning of the

curriculum/teaching and learning

process till testing. It is comprehensive

again in the sense that it takes into

consideration all the objectives of the

programme and not only achievement.

Educational evaluation is a continuous

process. It is because programmes’

growth and development occur

throughout the year, therefore the

single act of testing growth and

development at the end of an academic

year is not enough to know the true

and complete account of the changes

taking place in programme. It is,

therefore, necessary to judge their

progress from time to time throughout

the year. It is continuous also in the

sense that it does not end with testing.

It strives to interpret the results

obtained and in the light of

interpretation either continues or

modifies educational objectives,

curriculum, methods of instruction or

tools and techniques of evaluation

themselves and the process goes on.

Definition of educational evaluation

4


Feb., 2019

It is virtually impossible to restrict

evaluation into one universally

accepted definition, however, the

following are widely accepted

definitions within the educational

domain.

For Stake (1967), evaluation is

implicitly “the description and

judgment of an educational

programme which encompasses the

role of an evaluator”. Unfortunately,

the authors found this definition

incomplete since it fails to elucidate

the roles of the evaluator. At best, this

definition only brought more

complexities into the already complex

environment as it leaves the reader

with the impression of going further to

look for what an evaluator is.

Contrastingly, Stufflebeam (1971) and

Alkin (1969) were more meticulous in

their definition when they define

evaluation as the “process of

alienating, obtaining and providing

useful information for judging useful

information so as to determine

alternative course of action”. This

definition reveals that during

evaluation, information about a

programme is collected first, then

studied, before judgments can be

passed.

Scriven (2009) saw evaluation from a

tripartite view: (1) “evaluation is the

process of determining merit, worth, or

significance (abbreviated as m/w/s). In

many but not all contexts these three

properties are roughly the same as,

respectively: quality, value, and

importance. (2) “an evaluation is the

product of that process”. (3) “an

evaluand is whatever is evaluated.

(The term evaluee is often used for

human evaluands.) This third triparte

however refers to evaluation as having

a subject and a predicate.

Educational evaluation is a systematic

process of determining the growth and

progress of programes towards

objectives or values in the curriculum.

5


Feb., 2019

In other words, it is a systematic

process of determining the extent to

which educational objectives are

achieved by pupils.

Nevertheless, the writers see

evaluation as a process that includes

measurement and possibly testing but

it also contains the notion of value

judgment. If a teacher administers a

test to a class and computes the

percentage of correct responses, it is

said that measurement and testing has

taken place. The scores must be

interpreted which may mean

converting them to values like As Bs

Cs and so on or judging them to be

excellent, good, fair or poor. This

process is called evaluation. So we can

say, evaluation is concerned with

making judgments about things. When

we act as evaluators, we attribute

'value' or 'worth to behavior, objects,

processes and programmes.

Nature of programme evaluation

Evaluation is a continuous process of

inquiry concerned with the study,

appraisal, and improvement of all

aspects of the education program. It is

a process carried out cooperatively

by all concerned with the growth and

development of the programme. It is

the process of determining the changes

in the programme during the course of

implementation and the intervening

variables that has influenced it during

the period. It is a systematic attempt

at ascertaining the amount of progress

made in the implementation and the

extent of realization of predetermined

objectives. It is an act of judging the

programme’s improvement in terms of

teaching and learning quality, teacher

quality, students’ achievements,

administrative competence, and

numerous other dimensions. On the

basis of the information about the

growth and development of the

programme, suitable evaluative

procedures and instruments can be

prepared by all concerned to find out

6


Feb., 2019

the effectiveness of the whole

educational program in terms of

meeting the needs of the individual

learners, teachers, stakeholders,

government and the community as a

whole. Evaluation can be formative or

summative (Joseph, 2019).

Formative evaluation

A formative evaluation (sometimes

referred to as internal) is a method for

judging the worth of a program while

the program activities are forming (in

progress). Here, the focus of the

evaluation is on the process. They can

be conducted during any phase of

the programme process. This part of

the evaluation focuses on the process.

Thus, formative evaluations are

basically done on the go. They permit

the designers, learners, instructors, and

managers to monitor how well the

instructional goals and objectives are

being met. Its main purpose is to

identify deficiencies as soon as

possible so that proper learning

interventions can take place that allows

the learners to master the required

skills and knowledge.

Formative evaluation is also useful in

analyzing learning materials, student

learning and achievements, and teacher

effectiveness . . . Formative evaluation

is primarily a building process which

accumulates a series of components of

new materials, skills, and problems

into an ultimate meaningful whole

(Guyot, 1978). In

addition, prototyping is used in

formative evaluations to test a

particular design aspect by using one

or more iterations.

7


Feb., 2019

Summative evaluation

A summative

evaluation (sometimes referred to

as external) is a method of judging

the worth of a program at the end

of the program activities

(summation). The focus is on the

outcome. According to Scriven

(1967), “all assessments can be

summative (i.e., have the potential to

serve a summative function), but only

some have the additional capability of

serving formative functions”. The

various instruments used to collect

the data are questionnaires,

surveys, interviews, observations,

and testing. The model or

methodology used to gather the

data should be a specified step-by-

step procedure. It should be

carefully designed and executed to

ensure the data is accurate and

valid. Questionnaires are the least

expensive procedure for external

evaluations and can be used to

collect large samples of

information. The questionnaires

should be tested before using to

ensure the recipients understand

their operation the way the designer

intended. When designing

questionnaires, keep in mind the

most important feature is the

guidance given for its completion.

All instructions should be clearly

stated . . . let nothing be taken for

granted.

General principles of evaluation

8


Feb., 2019

The greatest benefit can be secured

from an evaluation program by

following certain general principles

played out by Clark (2012):

1. Evaluation should be in terms of

the extent to which the

programme has attained the

objectives of education. These

objectives include not only

mastery of subject matter but

also the growth in physical and

mental health, ability to get

along with others, use of critical

thinking to solve problems,

efficiency in using skills,

competency in the creative arts,

wide interest in many fields of

human endeavor, ability to use

knowledge gain.

2. Objectives should be defined in

terms of subject

matter behavior. This refers to a

behavioral analysis of what

students should be able to do

after successfully completing

units of instructions. Behavioral

analysis is the breaking down of

higher level objectives into

component parts, each of which

must be mastered to eventually

lead to a final behavior expected

of the student.

3. Evaluation is an integral part of

the educative process. The

teacher should guide every

experience in terms of the needs

and interests of the group as

seen in life situations.

Evaluation forms the basis for

decisions as to the nature of the

next experience needed by the

learner. The day-to-day

appraisal helps the teacher in

deciding how to guide on-going

experiences and when to

introduce new ones.

4. The evaluation program should

be cooperative. Learners,

teachers, and parents should

participate in the process

9


Feb., 2019

of evaluating educational

objectives.

5. Records should give a complete

picture of each child. Growth as

a continuous process can be

seen only when adequate

records give a complete picture

of the child as he develops under

the care of the teacher. Records,

however, should not be

considered as ends but as

service tools to help

the teacher understands the

learners to interpret behavior,

and to define immediate and

long-term needs.

6. Evaluation should be

comprehensive. It should take

into account the learner’s

individual character, his

background, and the immediate

environmental factors. Data

should include the individual’s

health physiologic needs,

emotional adjustments, mental

characteristics, talents and

aptitudes, values and attitudes,

social relationship and

competence, ability to function

effectively in his environment

and in the whole realm of his

interest, aspirations, and goals.

Records should show the learner

as a developing personality,

including both positive and

negative aspects of the learner’s

development.

7. Evaluation uses a variety of instr

ument, tools, and techniques.

These instruments should be

valid, reliable, and practical

from the standard points of time,

effort, and facilities of the

school. There are teacher made

and standardize tests, anecdotal

records, rating scales, samples

of students’ work, sociograms,

diaries, and journals. The

teacher should choose the

technique suited to the

individual student concerned

and to the specific purpose for

10


Feb., 2019

which the evaluation is being

made.

8. Objective measurement and

subjective judgment are both

essential in evaluation. Records

should be specific and as, far as

possible objective. Subjective

estimate are made objective

by the inclusion of specific

incidents and illustrations.

9. Diagnosis and remedial work

are phase of the evaluative

process. Test results should be

used for the improvement of

instructions. Results should be

carefully interpreted and

necessary follow-up work

should be done accordingly.

10.Evaluation should be

descriptive. Although the

uses of terms like superior,

good, average, and poor is better

than the use of figures, these

terms still leave much to be

desired from the standpoint of

evaluation. A descriptive

concrete statement about the

child is more meaningful and

significant to teachers, to

parents, and to children than any

blanket judgment that merely

indicates that the child

has passed.

Theoretical backing

Kirkpatrick's Four Level Evaluation

Model (1959)

11


Feb., 2019

Perhaps the best known evaluation

methodology for judging learning

processes is Donald

Kirkpatrick's Four Level

Evaluation Model that was first

published in a series of articles in

1959 in the Journal of American

Society of Training Directors (now

known as T+D Magazine). The

series was later compiled and

published as an article, Techniques

for Evaluating Training Programs ,

in a book Kirkpatrick

edited, Evaluating Training

Programs (1975).

However it was not until his 1994

book was published, Evaluating

Training Programs, that the four

levels became popular. Nowadays,

his four levels remain a cornerstone

in the learning environment. While

most people refer to the four

criteria for evaluating learning

processes as “levels,” Kirkpatrick

never used that term, he normally

called them “steps” (Craig, 1996).

In addition, he did not call it a

model, but used words such as,

“techniques for conducting the

evaluation” (Craig, 1996).

The four steps of evaluation consist

of:

o Step 1: Reaction - How well did

the learners like the learning

process?

o Step 2: Learning - What did

they learn? (the extent to which

the learners gain knowledge and

skills)

o Step 3: Behavior - (What

changes in job performance

resulted from the learning

process? (capability to perform

the newly learned skills while

on the job)

12


Feb., 2019

o Step 4: Results - What are the

tangible results of the learning

process in terms of reduced cost,

improved quality, increased

production, efficiency, etc.?

Kirkpatrick's concept is quite

important as it makes an excellent

planning, evaluating, and troubling-

shooting tool, especially if we

make some slight improvements.

Conclusion

From the above, it can be said that

evaluation is an important aspect of the

educational system. Without

evaluation, growth can be impossible.

This is because it is the level of growth

that determines the next course of

action. Therefore, where there is no

assessment/evaluation, deciding the

next level can be difficult.

Accordingly, this paper has made us to

know that evaluation can be carried

out either as the programme is ongoing

(which is called formative evaluation

in this paper – process focus) or as the

programmes concludes (which is

summative-outcome base). It is

however worthy of note that while the

summative has been widely adjudged

to be very important, the formative

evaluation is also very key to

evaluation. This is because assessment

done at the end of a programme may

omit several points that are invaluable

to the overall growth of the

educational system being evaluated.

13


Feb., 2019

References

Alkin, M. C. (1969). Evaluation theory development. Evaluation Comment, 2, 2–7

Clark, D.R. (2012). Types of evaluations in instructional design. Retrieved from http://www.nwlink.com/~donclark/hrd/isd/types_of_evaluations.html

Craig, R. L. (1996). The ASTD training: Development handbook . New York: McGraw-Hill.

Guyot, W. M. (1978). Summative and formative evaluation. The Journal of Business Education. 54(3),127-129.

Joseph, E. (2019). Meaning and nature of evaluation. Retrieved from https://www.scribd.com/doc/26092878/Meaning-and-Nature-of-Evaluation-Evaluation-Is

Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of American Society of Training Directors . 13(3), 21–26.

Scriven, M. (2009). The nature of evaluation. Retrieved from http://www.rismes.it/pdf/Scriven_domain_evaluation.pdf.

Scriven, M. S. (1967). The methodology of evaluation. In R. E. Stake (ed.) Curriculum evaluation. AERA Monograph Series on Curriculum Evaluation (Vol. 1). Chicago: Rand McNally

Stake, R. E. (1967). The countenance (sic) of educational evaluation. Illinois: Center for Instructional Research and Curriculum Evaluation University of Illinois. Retrieved February 26th, 2019, from https://pdfs.semanticscholar.org/b07e/5b61cde550bfb0b64e895674a236c9003335.pdf

Stuffiebeam, D. L. (1971). The relevance of the CIPP evaluation model for educational accountability. Journal of Research and Development in Education, 117-214.

14

https://pdfs.semanticscholar.org/b07e/5b61cde550bfb0b64e895674a236c9003335.pdf

https://pdfs.semanticscholar.org/b07e/5b61cde550bfb0b64e895674a236c9003335.pdf

http://www.rismes.it/pdf/Scriven_domain_evaluation.pdf

http://www.amazon.com/exec/obidos/ASIN/007013359X/bigdogsbowlofbis

http://www.nwlink.com/~donclark/hrd/isd/types_of_evaluations.html


Feb., 2019

UNESCO, (2016). Designing effective monitoring and evaluation of education systems for 2030: A global synthesiss of policies and practices. UNESCO Education Sector Division for Policies and Lifelong Learning Systems (ED/PLS)/Section of Education Policy (ED/PLS/EDP). Retrieved from http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/ED/pdf/me-report.pdf

15

admissionsandutme.com · web view2019-02-28 · evaluation in this context according to unesco...

Documents