design implement evaluation
TRANSCRIPT
-
7/28/2019 Design Implement Evaluation
1/27
Designing and Implementing Course Evaluations Survival Training for Subject Matter Experts
-
7/28/2019 Design Implement Evaluation
2/27
-
7/28/2019 Design Implement Evaluation
3/27
` The importance of course evaluations
`
Linking course evaluations to the learningand performance objectives
` The most appropriate level of evaluation
based on time, resources and how the datawill be utilized.
-
7/28/2019 Design Implement Evaluation
4/27
` Why do an evaluation?
x Will the class be taught again?
x Will this class be taught by others?
x Will more classes be taught on this subject?
` Who will see and use the information? (Whocares?)
` Who is going to compile the data and how willit be used?
-
7/28/2019 Design Implement Evaluation
5/27
Level 3: Behavior Are the participantsusing the Information taught in the class?
Level 4: Results What is the impact onthe organization?
Level 1: Reaction Did the participants like thetraining?
Level 2: Learning Can the participantsremember what was taught in the training?
-
7/28/2019 Design Implement Evaluation
6/27
` Each successive levelrepresents a more precise
measure of theeffectiveness of the trainingprogram, but at the sametime requires a morerigorous and time-consuming analysis.
-
7/28/2019 Design Implement Evaluation
7/27
` How well was the training perceived
` Gives immediate feedback
` Easy and economical` Administered at the end of the class` Sometimes known as smile sheets
` Paper or electronic
If they dont like it they probably are not learning
-
7/28/2019 Design Implement Evaluation
8/27
A.D.D.I.E.Analysis Learning needs and
gaps
DesignCourse outline
Develop Course material
Implement Class or CBT madeavailable to learners
Evaluate Feedback
-
7/28/2019 Design Implement Evaluation
9/27
Analyze
Implement Design
Develop
Evaluate
-
7/28/2019 Design Implement Evaluation
10/27
Evaluation SectionBasic course information` Course title, instructors name, date, location` Instructors ability to answer questions` Depth and complexity` Difficulty of exercises` Instructors knowledge` Pace of the training` Usefulness of visual aids
` Length of training session
Comments Section:Provides descriptive data such as` Topics to spend additional time on
` Were assessed needs accomplished?
` Are the participants more confident in their skills?
` Logistics pre, during, post
`
Changes for a more effective course?
Level 1 The Questionnaire
-
7/28/2019 Design Implement Evaluation
11/27
Creating test items might seem easy at first, but a lot of thought must gointo test creation to make it effective. Writing effective questions takestime and practice. Whether your goal is to measure knowledge and skills,survey opinions and attitudes, or enhance a learning experience, poorlyworded questions can adversely affect the quality of the results. Here are a
few tips to keep in mind when you write and review questions:
Goal of Test Items` Test items should:` Measure the objectives taught at the required skill or information level
(match verbs)` Focus on important, not trivial, subject area content` Contain accurate information (including correct spelling)` Be written at an appropriate level of difficulty`
Be clear and concise and as short as possible` Avoid vague pronouns (e.g. it, they)` Avoid information that helps answer other items` Be bias free
-
7/28/2019 Design Implement Evaluation
12/27
Responses should:` Be plausible
` Be grammatically consistent with the stem
` Be of approximately the same length
` Be structurally parallel
` Avoid conflicting alternatives
` Provide only one correct answer
` Avoid All of the above and use of None of the above with caution.Avoid syllogistic reasoning choices (e.g. both a and b are correct)unless absolutely necessary.
` Incorrect answers called distractors should not be too obvious
` Distractors should point out errors in thinking along the levels ofBloom's Taxonomy
http://712educators.about.com/od/testconstruction/p/bloomstaxonomy.htmhttp://712educators.about.com/od/testconstruction/p/bloomstaxonomy.htmhttp://712educators.about.com/od/testconstruction/p/bloomstaxonomy.htm -
7/28/2019 Design Implement Evaluation
13/27
Stems should:` Provide sufficient information
` Avoid extra information
` Be grammatically correct
` Avoid clues that help give away answers
` Avoid negatives, especially double negatives
` Avoid unnecessary modifiers, especially absolutes (e.g. always,never, etc.)
` Avoid second person
` Use questions whenever possible (What, Who, When, Where, Why and
How)
-
7/28/2019 Design Implement Evaluation
14/27
Level 2: Learning Can the participantsremember what was taught in the training?
` A test of knowledge or skill
` Did they retain the information?
` Used to assess the amount of learning that
has occurred.` Often uses tests conducted before training
(pretest) and after training (post test).
` Administered at the end of the class or
several days or weeks following the class
-
7/28/2019 Design Implement Evaluation
15/27
` Paper or electronic
` Multiple choice, short answer, true-false
` Simulation, demonstration or observation
` Case study
` Role plays
` Table top exercises
` Or a game!
-
7/28/2019 Design Implement Evaluation
16/27
SurveyMonkey.combecause knowledge is everything
-
7/28/2019 Design Implement Evaluation
17/27
` Measures the changes that have occurred in learners' behavior dueto the training program.
`
Evaluating at this level attempts to answer the question Are the newly acquired skills, knowledge, or
attitude being used in the everyday environment of the learner?
` May require up-front assessment for benchmark
Level 3: Behavior Are the participantsusing the Information taught in the class?
-
7/28/2019 Design Implement Evaluation
18/27
` Occur several weeks or months after the training
` Observation
` Survey of managers
` Analysis of systems
` May require up-front assessment for benchmark
-
7/28/2019 Design Implement Evaluation
19/27
` Attempts to assess training in terms of business results. Decrease in number of
complaints Decrease in queue time
Increase in sales
` Occur several weeks or
months after the training
` Determines if the time
and $$ for the training
was a good investment
Level 4: Results What is the impact onthe organization?
` Observation
` Survey of managers
` Analysis of systems
` Will require up-front assessmentfor benchmark
-
7/28/2019 Design Implement Evaluation
20/27
Heres just a partial list of several excellentresources to continue your learning journey!Books and Magazines` Evaluating Training Programs: The Four Levels Donald L. Kirkpatick and
James D. Kirkpatrick` Evaluating the Impace of Training Scott B. Parry` Handbook of Training Evaluation and Measurement Methods Jack B. Phillips` Evaluation Basics Donald McCain
` Evaluating E- Learning William Horton` The Accidental Trainer Nanette Miller` The Instant Trainer C. Leslie Charles and Chris Clarke-Epstein` The Ten Minute Trainer Sharon L. Bowman` Training for Dummies Elaine Biech` Training SMEs to Train ASTD InfoLine
-
7/28/2019 Design Implement Evaluation
21/27
Teaching Others What You Know Survival Training for Subject Matter Experts
Next session:
Classroom Delivery Skills by invitation only to those who have participated
in Modules 1- 4 and successfully completed the assessments.
Friday, J une 20, 2008 1:00 p.m. 5:00 p.m.
Fidelity Banks Training Center, I-85 and Clairmont Road, Atlanta
www.astdatlanta.org
-
7/28/2019 Design Implement Evaluation
22/27
CREATE exams in a Web browser
DEPLOY your exam with one mouse click
AUTOMATE registration and exam retakesANALYZE detailed reports and statistics
www.exambuilder.com
Special thanks to ExamBuilder for
providing the on-line assessment.
-
7/28/2019 Design Implement Evaluation
23/27
About ASTD
ASTD (American Society for Training & Development) is the worlds largestassociation dedicated to workplace learning and performance professionals.ASTDs members come from more than 100 countries and connect locally innearly 140 U.S. chapters and 25 Global Networks. Members work inthousands of organizations of all sizes, in government, as independentconsultants, and suppliers.
The Greater Atlanta Chapter of ASTD is one of the five largest chapterswithin the US with nearly 550 members. We are a regional association ofworkplace performance professionals, providing our members with a forum
for the exchange of professional development opportunities and resources asthey strive to improve workplace learning and performance.
www.astdatlanta.org
-
7/28/2019 Design Implement Evaluation
24/27
About Cisco
Cisco (Nasdaq: CSCO) is the worldwide leader in networking that transforms how people connect,communicate and collaborate. Information about Cisco can be found at www.cisco.com. For ongoingnews, please go to newsroom.cisco.com.
With 2.3 million registered users, Cisco WebEx is the global leader in on-demand applications forcollaborative business on the web. WebEx offers a full suite of web collaboration applications forspecific business processes including marketing, sales, training, and support. Companies of every
size gain strategic advantage by using WebEx to replace high-cost/high-touch interactions withefficient web-touch interactions. WebEx applications work across firewalls and across platforms,making them ideal for both internal and external collaboration. Cisco WebEx provides its servicesover the WebEx MediaTone Network, a secure global network specifically designed for on-demand
applications. Please call toll free 877-509-3239 or visit www.webex.com for more information
Special thanks to WebEx for partnering with the ASTD Atlanta Chapter to make this eventpossible.
WebEX team members:Brad KingNisha.AhluwaliaLauren FollettJ ennifer Smith
-
7/28/2019 Design Implement Evaluation
25/27
` SillyMonkey LLC is a boutique training company specializing
in innovative games and activities. We design custom boardgames, card games, electronic games, video games and otherinteractive multimedia training components for instructor-ledsessions, as well as, eLearning, mobile learning, socialnetworks and virtual worlds. We offer high quality trainingservices to a an impressive list of global clients that includesmanufacturing, health care, retail, software, pharmaceuticals,and financial services.
` Our team has over 25 years of experience in traditionaltraining and various training methodologies including ADDIE,constructivist, cognitivism and behaviorism. We use mindmapping, memory techniques, improve and storytelling.
` www.sillymonkeyinternational.com
-
7/28/2019 Design Implement Evaluation
26/27
Brad King, WebEx
Bridget Evans, Randstad
Christie Cole, RBS Lynk
Darcie Drymon, GH& I
Latarsha Horne, Orkin
Deborah Lucas, TurnerBroadcasting
Deborah Thomas - SillyMonkey
Garland Watkins, UPS Contractor
Harvey Menden, Kimberly-Clark
Jeff Collins - Arby's
Laurie Gilmore - Orkin
Jon Wakeman, Atlanta JournalConstitution
Katherine Brune, SunTrust Bank
Kim Beckett, Xinnix
Marcia Breese, CNN
Mary Duncan, Deloitte and
ToucheMatt Fazio, Consultant
Patsy Newton, Cabot Corp.
Stacy Woods, UPS Contractor
Stephanie Britz, Carmax
Tony Foley, Childrens
Healthcare of Atlanta
-
7/28/2019 Design Implement Evaluation
27/27
There is a short assessment to check your knowledge.
If you took the assessment for Modules One, Two or Three go to
Go to: http://testing.exambuilder.com, type in your e-mail address and
password: astdatlanta
First Time Users: Go to:http://testing.exambuilder.com/main/exams/index.cfm?loginaction=5
Follow the directions below to register. You will
receive an e-mail from ExamBuilder within one hour with log-in
information.Type in:Your First Name:Your Last Name:
Your Email:Instructors name: [email protected] creator ID: DADQClick on Register.
http://testing.exambuilder.com/mailto:[email protected]:[email protected]://testing.exambuilder.com/