developing evaluation instruments. evaluations formative – how are we doing? summative – how did...
TRANSCRIPT
![Page 1: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/1.jpg)
Developing Evaluation Instruments
![Page 2: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/2.jpg)
Evaluations
• Formative – How are we doing?
• Summative – How did we do?
• Confirmative – How are we still doing?
![Page 3: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/3.jpg)
Formative Evaluation
• Even excellent plans and concepts can be improved– Gauge success by classroom program
presentation– Informs the instructor how well instructional
package is serving the objectives as it progresses
![Page 4: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/4.jpg)
Formative Evaluation
• Program of instruction is evaluated through– Test results– Learner reactions/comments– General observation of learners during class– Subject-matter expert reviews– Colleague suggestions
• Quality control of the development process
![Page 5: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/5.jpg)
Formative Evaluation
• Questions that may be asked– Level of learning acceptable compared to
objectives?– Learners using knowledge to perform skills?– Time element acceptable?– Activities appropriate?– Tests measuring objectives?
![Page 6: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/6.jpg)
Summative Evaluation
• Measures success of major outcomes
• Usually follows all instruction, projects and testing in a program
![Page 7: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/7.jpg)
Summative Evaluation
• Measures– Learning efficiency (material mastered/time)– Program cost (Development and delivery)– Continuing expenses– Student comments/reactions/evaluations– Long-term benefit of program
![Page 8: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/8.jpg)
Confirmative Evaluation
• Although instruction initially effective, some problems appear over time
• Tracks learner experiences over time to judge program’s validity
• Relies upon numerous data-collection instruments (interviews, performance assessments, etc.)
![Page 9: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/9.jpg)
Confirmative Evaluation
• Questions to investigate– Do learners continue to perform correctly over
time?– Do materials still meet original objectives?– How has technology and/or resources, trends,
and attitudes changed since inception? – How to best meet clients’ needs over time?
![Page 10: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/10.jpg)
Confirmative Evaluation
– Changes needed in materials? What are the costs?
– If instruction not working as well as before:• Should instruction continue as is?
• Should it be revised?
• Should it be terminated?
• What might replace it?
![Page 11: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/11.jpg)
Validity
• The test assesses what it is supposed to measure – For example:
• Performance tests assess processes and outcomes relating to skills or competencies
• Course attitude surveys need to measure reactions to the course
![Page 12: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/12.jpg)
Reliability
• The test will produce consistent results whenever used
• More questions relating to specific objectives = more reliable test
• Standardized delivery of test
• Scoring methods – the less subjective, the better
![Page 13: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/13.jpg)
Pretesting
• Judges the learner’s preparation to study the material (course or topic)
• Determines competencies already mastered
![Page 14: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/14.jpg)
Testing for Prerequisites
• Standardized paper-and-pencil test
• Performance observations
• Questionnaire (Looks at learner background, training, and experiences)
• Review of past work experience
• Interview past supervisors/managers
![Page 15: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/15.jpg)
Objective tests
• Multiple choice
• True or False?
• Matching items
![Page 16: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/16.jpg)
Constructed-Response
• Make learner plan answers and express them in their own words
• Short-answer
• Essay questions
• Problem solving
![Page 17: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/17.jpg)
Skills and Behavior
• Actions observed
• Process, product, or both?
• Constraints/limitations?
• Testing conditions simulated or realistic?
![Page 18: Developing Evaluation Instruments. Evaluations Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?](https://reader036.vdocuments.us/reader036/viewer/2022072010/56649ddc5503460f94ad3c56/html5/thumbnails/18.jpg)
The Rubric
• General assessment of overall product
• Certain elements attain points
• More objective tool
From Designing Effective Instruction 4th Ed., Morrison, Ross, & Kemp, 2004.