Lessons Learned From the Short Course in the Theory and Practice in Test Construction

Presented by

William M. Kapambwe

on behalf of

Pradhan Basanti R.(Nepal)Jearvy Lanohan (Philippines) Lockias Chitanana (Zimbabwe)

at the

2009 NFP CITO Short Course in the Theory and Practice in Test Construction

CITO

Arnhem, The Netherlands

8th June to 19th June, 2009

Outline

1.0 Introduction: Objectives for the Course

2.0 Specific Lessons from Zambia

3.0 Specific Lessons from the Nepal

4.0 Specific Lessons from the Phillipines

5.0 Specific Lessons from Zimbabwe6.0 Strategies for Disseminating the Lessons

1.0 Introduction: Objectives for the Course

To attend to the participants’ needs in assessment, test

construction and evaluation.

To enhance the particpants’ awareness of the function of tests

and assessments.

To enhance the participants knowledge of the psychometric

requirements for valid tests.

To enhance participants’ competence in developing and

implementing tests.

2.0 Specific Lessons Learned- Zambia• The need to have a slogan “Now You Know”• Criteria for Reviewing Tests-COTAN Criteria for Test Quality• Meaning of competence and competence-based assessment• Assessment for Community Studies• Inspectorate Instrument for Measuring Quality Aspects of the Teaching and Learning Process• Deeper understanding of the IRT use in validating tests,

Incomplete Test Administrations and Item banking evaluating test items

• Continual change of item writers• Analysis of open-ended pre-test questions items• Need to publish Test Development Guidelines• Romiszowsky Taxonomy for further clarifying learning outcomes• Deeper skills in constructing objective and subjectives test items

3.0 Specific Lessons Learned - Nepal

• Test item banking system• Romizoski’s test taxonomy for skill and knowledge test• Continual change of test writers• Indepth knowlegde of Cito’’s evaluation process• Dutch Educational System and Evaluation• Importance of norm, Reliabilty and Validity for good

quality test development• Assessment of teachers’ competence.• Evaluation and test item development in other

countries.• Overview of classical item theory and Item response

theory

4.0 Specific Lessons Learned- Philippines

• Test banking• The use of Romiszowski taxonomy- New teachers

in our school are struggling with Bloom’s taxonomy since they find it rather complicated. Romiszowski might be easier for non-education graduate teachers in our school.

• Evaluation process of the Netherlands.• Deeper understanding of IRT and CTT• The importance of reporting tests results • Understanding that preparing and administering

exams are things one should not be a solitary endeavor.

5.0 Specific Lessons Learned-ZimbabweTest construction and item banking

• I have learnt the following aspects:

1. How to prepare, assemble, and administer a valid and reliable test tests grids together with Blooms and Romiszovski’s taxonomies.

2. How to prepare and assemble a test more effectively using item bank?

3. The general guidelines to be followed when writing both multiple choice and open-ended test items

4. How to design a scoring method for open-ended questions that will maintain objectivity and fairness?

5. Some of the pitfalls in developing a test

• I have also, to some extent appreciated the use of Psychometrics to;– appraise the effectiveness of items and test in general, for example through

reliability estimate, standard error measurements, item difficulty (p-value), item-test correlation, etc

– provide guidance to interpret and use test results to determine when a test item needs to the revised

6.0 Strategies for Disseminating the Lessons Zambia • Hold a two session disseminations workshops.

• Meeting with the Director of Standards and Curriculum.• Propose to Management to consider inviting CITO to conduct

workshops in test development for Exam Specialists.• Propose that the ECZ attaches two officers to acquire IRT skills.

Nepal • Presentation to management board about what I have learnt

from this training.• Share my learning to my colleagues• Plan, pilot test and implement my learning to my trainings

The Philipines • Will conduct a two-part echo-seminar with the Faculty of the

PSHS-SMC• Will conduct training with teachers of our adopted public

schools• Will conduct echo-seminar in the annual Test item writers

training (PSHS System)

Zimbabwe •Hold report back workshop to present the ideas I have learnt from •this training.•I hope to develop a course module on Principles of Test Construction to be done by both pre-service and in-service teachers

at undergraduate levels.