guide to evaluating learning facilitators · pdf filean mcd is assigned to evaluate a specific...

Guide to EvaluatingLearningFacilitators

© This document is copyrighted by the Coaching Association of Canada (2014) and its licensors. All rights reserved. Printed in Canada.

The National Coaching Certification Program is a collaborative program of the Government of Canada, provincial/territorial governments, national/provincial/territorial sport organizations, and the Coaching Association of Canada.

PARTNERS IN COACH EDUCATION

The programs of this organization are funded in part by the Government of Canada.

Guide to Evaluating Learning Facilitators

Version 2.3, 2016 © Coaching Association of Canada

TABLE OF CONTENTS

Introduction ............................................................................................................................... 1

Principles of LF Evaluation in the NCCP ................................................................................. 2 Purpose of LF Evaluation and Certification .............................................................................. 2 Basis of LF Evaluation ............................................................................................................. 2 Outcomes, Criteria, and Evidence ........................................................................................... 2 LF Training .............................................................................................................................. 2 Evaluation Requirements ......................................................................................................... 3

The Steps in the Evaluation Process ....................................................................................... 4 Summary of Roles and Responsibilities ................................................................................... 4

On-site Evaluation .................................................................................................................... 6 Prebrief .................................................................................................................................... 6 On-site Observation ................................................................................................................. 6 Debrief, including the Action Plan ............................................................................................ 7

Evaluation Matrix .................................................................................................................... 11 Structures and Manages the Training Environment Appropriately .......................................... 12 Facilitates the Achievement of Learning Outcomes ............................................................... 13 Displays Appropriate Communication and Leadership to Enhance Coach Learning .............. 16 Manages Group Tasks to Optimize Coaches’ Learning .......................................................... 18

Annex A: Evaluation Tool ....................................................................................................... 20

The Collection, Use, and Disclosure of Personal Information The Coaching Association of Canada collects your NCCP qualifications and personal information and shares it with all NCCP partners according to the privacy policy detailed at www.coach.ca. By participating in the NCCP you are providing consent for your information to be gathered and shared as detailed in the privacy policy. If you have any questions or would like to abstain from participating in the NCCP please contact [email protected].

http://www.coach.ca/

mailto:[email protected]


Version 2.3, 2016 © Coaching Association of Canada

ACKNOWLEDGEMENTS

Main Writer Doug Krochak

Contributors Barry Bartlett

Kathy Brook

Cyndie Flett

Marc Schryburt

Don Watts

Production Lucie LeBel, MATRA ● gs Translation Services Inc., Louise Wood

Introduction

Version 2.3, 2016 © Coaching Association of Canada Page 1

INTRODUCTION This guide is for all Master Coach Developers (MCDs) responsible for the evaluation of Learning Facilitators (LFs) in the NCCP.

This guide supplements Learning Facilitator Evaluation Tool, which is used in LF evaluation, providing essential information on LF evaluation in general and the use of the Evaluation Tool in particular. The guide has four sections:

Principles of LF Evaluation in the NCCP

The Steps in the LF Evaluation Process

On-site Evaluation

Evaluation Matrix

Note: The Evaluation Tool is reproduced in Annex A of this document.


Page 2 Version 2.3, 2016 © Coaching Association of Canada

PRINCIPLES OF LF EVALUATION IN THE NCCP Purpose of LF Evaluation and Certification Evaluation and certification confirm that LFs are capable of doing what is deemed

important to the training of coaches in a given context.

Certification promotes favourable training behaviours that have a positive impact on LF development.

Certification is about acknowledging that LFs have met expectations by providing evidence of competence in the criteria evaluated. It is not about recognizing the perfect Learning Facilitator.

Basis of LF Evaluation LF evaluation reflects the ethical coaching practices promoted by the Canadian sport

system (NCCP Code of Ethics, Fair Play, and the policies of the Canadian Centre for Ethics in Sport).

The processes and procedures involved in LF evaluation and certification are administratively feasible, professionally acceptable, publicly credible, legally defensible, economically affordable, and reasonably accessible.

Outcomes, Criteria, and Evidence Outcomes, criteria, and evidence are the core of the system for evaluating and certifying

LFs:

• Outcomes are what LFs need to be able to do when they are training coaches in a given context. Outcomes provide the foundation for both training and evaluation.

• Criteria are what will be evaluated within a given outcome. One or more criteria may be associated with one outcome.

• Evidence is what MCDs must see to confirm that LFs have met a given criterion. Evidence helps LFs prepare for evaluation by giving them specific points of reference.

It is possible to recognize and respect individual facilitation styles while validating the presence of critical evidence.

Evidence may be gathered 1) by directly observing the LF conducting the module on which he or she is being evaluated or 2) by watching a video of the LF conducting the module on which he or she is being evaluated.

LFs do not need to provide evidence of all objectives identified during training to be recommended as Certified Learning Facilitators.

LF Training Before they are evaluated, LFs must either receive training or demonstrate relevant

experience.

LF training should prepare LFs to meet evaluation criteria.

Principles of LF Evaluation in the NCCP


LF training is competency-based; in other words, LF training focuses on developing the knowledge, skills, and attitudes that LFs need to provide support to coaches in training workshops.

Evaluation Requirements LFs trained to deliver more than one module need to be evaluated on only one module

for certification as an LF.

For certification purposes, only trained MCDs can determine whether LFs have met a given criterion.

Evaluation should take place after LFs have had a number of opportunities to co-facilitate or independently facilitate the module on which they are being evaluated.



THE STEPS IN THE EVALUATION PROCESS All evaluations must be conducted by Master Coach Developers (MCDs).

The NCCP sets the standards for MCD training. This training should include an opportunity to practise key steps in the evaluation process (prebrief, observation, and debrief) during LF training.

The steps in the evaluation process are as follows:

An MCD is assigned to evaluate a specific LF.

The MCD contacts the LF to arrange a prebrief.

The prebrief takes place.

The observation takes place, and the MCD completes the Evaluation Tool.

The debrief takes place. During the debrief, the MCD and LF develop and sign an Action Plan, and the MCD provides his or her recommendation about certifying the LF.

The MCD forwards the Evaluation Tool and Action Plan to the governing organization.

The governing organization enters the required information in the NCCP Database (the Locker).

Summary of Roles and Responsibilities CAC Governing

Organization Master Coach

Developers Learning

Facilitators Develop Guide

to Evaluating Learning Facilitators

Maintain consistency across the country by auditing the evaluation process

Approve evaluation events in the NCCP Database (the Locker)

Develop a thorough knowledge of Guide to Evaluating Learning Facilitators

Select MCDs Arrange for the training

of MCDs Determine which LFs

to evaluate Assign MCDs to

evaluate LFs Process Action Plans

from completed evaluations

Arrange for training or any other follow-up activities outlined in Action Plans

Mediate any disputes arising from the evaluation or the evaluation process

Record evaluation events in the NCCP Database (the Locker)

Develop a thorough knowledge of Guide to Evaluating Learning Facilitators

Take Evaluating Learning Facilitators training (one of the topics in Core Master Coach Developer training)

Complete at least one co-evaluation with an MCD

Contact the assigned LF to arrange a prebrief

Observe the LF deliver the module

Complete the Evaluation Tool

Prepare for and conduct a debrief; this includes developing an

Develop a thorough knowledge of the Outcomes, Criteria, and Evidence and the evaluation process

Attend Core Training for Learning Facilitators

Deliver a portion of a module (micro-facilitation) at the Workshop-specific Training for Learning Facilitators

Facilitate or co-facilitate several sessions of the module being evaluated

Prepare for the evaluation

Participate in a

The Steps in the Evaluation Process


CAC Governing Organization

Master Coach Developers

Learning Facilitators

Action Plan with the LF and a recommendation about whether to certify the LF

Share the recommendation about certification with the LF

Forward the Action Plan to the governing organization

prebrief Deliver a module

while being observed by the MCD

Participate in a debrief; this includes developing an Action Plan with the MCD

Complete the follow-up activities outlined in the Action Plan



ON-SITE EVALUATION On-site evaluation consists of the following:

Prebrief

On-site observation

Debrief, including an Action Plan

Prebrief The purpose of the prebrief is for the MCD to determine: The LF’s objective for the module

The LF’s plan for implementing the module

The indicators of success the LF will use to measure the effectiveness of the module

The prebrief should last between 15 and 30 minutes.

During the prebrief, the MCD must: Explain the process of the evaluation (emphasizing the observation)

Review the Evaluation Tool; the Outcomes, Criteria, and Evidence; and the Standards of Performance

Answer any questions the LF has

Ensure the LF understands the process, Evaluation Tool, and standards

It is strongly recommended that the MCD confirm the logistics of the formal observation — module title, date, location, and time — during the prebrief.

It is also hoped that the MCD will use the prebrief to develop a positive and supportive relationship with the LF. Here are some questions MCDs can use to help this happen:

What are your goals for the module?

What is your comfort level with this module?

What is your plan for reaching your goals?

What do you see as the biggest challenge in reaching your goals?

What is your plan for introducing your goals?

How will you be able to tell if your goals are being achieved?

What adjustments have you prepared to ensure you meet your goals?

What kinds of adjustments have you made in the past?

On-site Observation The purpose of the on-site observation is to determine LFs’ ability to provide support to coaches in training workshops.

On-site Evaluation


Achieving this depends on LFs’ ability to meet these four criteria:

Structures and manages the training environment appropriately

Facilitates the achievement of learning outcomes

Displays appropriate communication and leadership to enhance coach learning

Manages group tasks to optimize coaches’ learning

During the on-site observation, MCDs

Carefully observe LFs. Here are some tips on how to observe LFs:

• Become thoroughly familiar with the Evaluation Tool so you know exactly what to look for.

• Get as close as possible to the action while not affecting the training or the outcome of the evaluation.

• Look and listen for subtle indicators of the LF’s performance without coaching, directing, or asking questions.

• Put the LF at ease with the evaluation process. If the prebrief took place a day or two before the observation, reintroduce yourself and remind the LF that evaluation should be a positive opportunity for personal growth.

• Remind the LF to introduce the MCD to the group and explain what the MCD is doing.

Use the Evaluation Tool to record and make notes on what they see during the observation. The Evaluation Tool is reproduced in Annex A of this document.

The on-site observation should last at least four hours OR the length of module in question.

MCDs should take some time after the observation and before the debrief to collect their thoughts.

Debrief, including the Action Plan The purposes of the debrief are to:

Give LFs an opportunity to reflect on and talk about their training

Give MCDs an opportunity to provide feedback on what they observed, including criteria met and evidence observed

Give MCDs and LFs an opportunity to develop an Action Plan

Give MCDs an opportunity to develop their recommendation about certifying the LF and share it with the LF

During the debrief, MCDs

Provide positive feedback that focuses on future improvement. Here are some questions to use to accomplish this:

• How do you feel about the module? • What went well? What were the indicators that told you things were going well? • What do you want to improve upon? • What did you learn by doing the module? • What would you do differently next time?



Give LFs an opportunity to explain their plans and the reasoning behind some of their actions during training. These explanations may lead MCDs to change what they recorded in the Evaluation Tool.

Work with LFs to develop an Action Plan.

Develop their recommendation about certifying the LF and share it with the LF

The debrief should last between 30 and 60 minutes.

Action Plan The purpose of the Action Plan is to help LFs establish short-term and long-term goals for their development as LFs. This includes specifying, in order, the steps to take to accomplish these goals. For example, the Action Plan could list steps the LF will take to complete his or her evaluation or to improve in certain areas.

The Action Plan includes the MCD’s assessment of LFs’ ability to meet the four required criteria as:

Exceeds Expectations

Meets Expectations

Needs improvement

Presented below is a template for an Action Plan.

On-site Evaluation


Action Plan

LF’S NAME: DATE: MCD’s NAME:

Outcome: Provides support to coaches in training workshops

Criteria Standard Next Steps Structures and manages the training environment appropriately

Exceeds expectations Meets expectations Needs improvement

Facilitates the achievement of learning outcomes


Displays appropriate communication and leadership to enhance coach learning


Manages group tasks to optimize coaches’ learning


The signatures below signify an acceptance of the Evaluation and the Action Plan.

LF’S Signature:

MCD’s Signature:

MCD’s Recommendation: _____ The Learning Facilitator is recommended as a Certified Learning Facilitator

_____ The Learning Facilitator can be re-evaluated after completing an Action Plan

_____ The Learning Facilitator is not recommended as a Certified Learning Facilitator

Note: All Learning Facilitators will receive an Action Plan. MCD’s Signature: _______________________________________Date: ___________



Recommendation about LF Certification To become certified, a Learning Facilitator must achieve Exceeds Expectations or Meets Expectations on all four criteria. Learning Facilitators whose performance is described as Needs Improvement will complete an Action Plan designed to bring them to the level of Meets Expectations.

There are three choices regarding the Learning Facilitator’s status:

The Learning Facilitator is recommended as a Certified Learning Facilitator

The Learning Facilitator can be re-evaluated after completing an Action Plan

The Learning Facilitator is not recommended as a Certified Learning Facilitator

The final determination of the Learning Facilitator’s status should be based upon the data gathered about each of the four criteria. This data should be considered within the entire context of the training observed, as well as the MCD’s own professional judgment, experience, and common sense. For example:

If the Learning Facilitator achieved the Needs Improvement standard in two of the criteria without having a significant negative effect on the overall training, you could recommend that the Learning Facilitator complete an Action Plan and then be re-evaluated.

Note, however, that MCDs MUST NOT recommend the Learning Facilitator as a Certified Learning Facilitator if they observe ANY of the following behaviours, as they undermine the effectiveness of the NCCP and people’s views of the Program:

Presents a negative image of Canadian sport and models inappropriate values and behaviours

Finishes without addressing all learning outcomes

Lacks knowledge of the module’s content and is unable to fully answer questions relevant to completing tasks

Uses language that is racist, sexist, or demeaning to other coaches

Allows coaches to use language that is racist, sexist, or demeaning to other coaches

Provides feedback that is negative or judgmental or both

Creates and manages groups in a manner that does not reflect their stage of development

No matter what the outcome of the evaluation is, MCDs should finish on a positive note by recognizing LFs’ strengths and the contribution LFs are making to the NCCP.

Evaluation Matrix


EVALUATION MATRIX The Evaluation Tool used in LF evaluation contains four criteria tables. These four tables are reproduced on the following pages as an evaluation matrix, with the Comments column from the Tool replaced by an Example column. The purpose of these examples is to help MCDs identify Evidence during their observations and to ensure consistency in evaluations throughout the NCCP.

The following key is used in the matrix:

E Exceeds Expectations

M Meets Expectations

NI Needs Improvement



Structures and Manages the Training Environment Appropriately Standard Examples

Arrival at the Training Site Arrives well in advance of the start time, arranges the training area to

enhance interaction among coaches, distributes module material, and sets up flip charts and other learning aids

E Arrives in time to perform all planned set-up tasks, deal with unexpected problems, and greet coaches as they arrive

Arrives in time to set the training area up and ensures that all materials are ready for use

M Arrives in time to set up the training area and distribute materials

Arrives too late to set up the training area and ensure that all materials are ready for use

NI Arrives too late to set up the training area and distribute materials before the scheduled start time

Use of Equipment Tests and sets up all equipment (AV, laptops, slides, etc.) before coaches

arrive and uses it well during coach training E Is completely at ease with all equipment and

handles any equipment malfunction with minimal or no impact on coach training

Demonstrates the ability to use AV/computer equipment M Uses equipment well and can handle minor equipment malfunctions

Has problems with AV/computer equipment that interfere with coach training

NI Struggles with any equipment malfunction, needing help from others and interrupting the flow of training

Evaluation Matrix


Facilitates the Achievement of Learning Outcomes Standard Examples

NCCP Model Explains the NCCP model and the competency-based approach to

training and refers coaches to sport-specific examples E Has coaches explain where they are on their

coaching path and what they need to do to become certified

Explains the NCCP model and the competency-based approach to training

M Explains the generic NCCP model to coaches

Does not explain either the NCCP model or the competency-based approach to training

NI Does not address the NCCP model at all

Learning Outcomes Assigns activities and debriefs them in a manner that encourage

coaches to reflect on the learning outcomes and NCCP competencies E Gives coaches opportunities to relate their

learning to NCCP competencies Clearly identifies learning outcomes and the NCCP competencies M Outlines the learning objectives at the start of

each module or section within a module Moves from task to task without clearly explaining the purpose of each

task or relating tasks to one another NI Moves from one workbook activity to the next

without bringing closure to the first activity or providing any bridging to the next activity

Learning Activities Adapts learning activities to coaches’ learning pace and learning stage E Uses activities from the Learning Facilitator

Guide but adapts them to meet coaches’ learning needs

Makes minor adjustments to learning activities in the Learning Facilitator Guide

M Uses activities from the Learning Facilitator Guide but makes minor adjustments to them

Does not adapt learning activities to coaches’ learning pace or stage NI Uses activities from the Learning Facilitator Guide exactly as they are presented in the guide



Standard Examples

Links with Coaches’ Experience Helps coaches discover links between current coaching practices and

desired coaching outcomes E Assigns an activity (e.g., on body language in

communication) that causes coaches to see the links between their behaviour and the coaching outcomes they can achieve

Provides common learning experiences when appropriate and debriefs them

M Draws on Reference Material or other coaches’ experiences to show links between the coaches’ experiences and desired coaching outcomes

Makes limited use of coaches’ experience and learning activity debriefs NI Draws only on Reference Material and does not provide examples relevant to the group

Critical Reflection Debriefs, when appropriate, coaches’ feelings and reactions,

understanding of the process, and ability to direct their own learning E Asks probing questions that lead coaches to

reflect on the task and to apply the information to their own coaching context

Helps coaches identify areas for change or improvement in coaching methods

M Uses learning tasks or asks questions that cause coaches to reflect on their current coaching methods

Makes limited use of questions and debriefing about current coaching practices

NI Asks coaches very few questions about their current coaching methods and fails to generate critical reflection on current methods or possible improvements

Use of NCCP Materials Uses coaches’ experience to enhance the Coach Workbook activities

and create links with the Reference Material E Has coaches reflect on their past experiences

before they complete Workbook tasks Uses the Learning Facilitator Guide, Coach Workbook, and Reference

Material effectively M Regularly refers coaches to specific pages in the

Workbook Does not make effective use of the activities in the Coach Workbook or

the Reference Material NI Uses activities that do not reflect or resemble the

tasks in the Workbook Knowledge of Module Content Directs coaches to other material and resources, including sport-specific

material E Provides examples or information that extends

the content in the Reference Material when coaches ask questions

Uses his or her thorough knowledge of the module’s content to help coaches critically reflect on their current coaching practice

M Provides key information that helps coaches complete their tasks

Lacks knowledge of the module’s content and is unable to fully answer questions relevant to completing tasks

NI Is unable to explain critical information when asked to do so by coaches

Evaluation Matrix


Standard Examples

Coaches’ Engagement in Learning Activities Coaches are stimulated to ask questions, explore new ideas, etc. E Coaches apply generic material from their

training to sport-specific situations Coaches are actively engaged in learning throughout the training session M Coaches work productively, developing

responses that apply content to their own coaching contexts

Coaches are not obviously engaged in the learning process NI Coaches spend long periods of time listening to the LF or another coach talk or working on tasks that are not relevant to the outcome

Timelines Respects recommended timelines and addresses all learning outcomes

within the timelines M Completes all learning outcomes in the

recommended time, i.e., without going over the recommended time or rushing through activities to meet timelines

Does not respect recommended timelines and does not address all learning outcomes

NI Rushes through a number of activities in an attempt to cover all learning outcomes



Displays Appropriate Communication and Leadership to Enhance Coach Learning Standard Examples

Image of Canadian Sport Promotes a positive image of Canadian sport and models NCCP values

and philosophy M Makes strong statements in support of ethical

behaviour in sport Presents a negative image of Canadian sport and models inappropriate

values and behaviours NI Provides examples of cheating or unethical

behaviour as a way to win Communication: Presenting Uses his or her position, voice, and teaching aids in a manner that

captures coaches’ attention, engages coaches, and reinforces learning E Adjusts position and voice as required so all can

see and hear Makes effective use of position, voice, and teaching aids M Facilitates from different locations in the training

area and speaks clearly, so most coaches can see and hear the LF most of the time

Makes poor use of position, voice, or teaching aids NI Positions self poorly and speaks too softly, so coaches struggle to see and hear the LF

Communication: Listening Uses a variety of listening and questioning techniques and adapts them

to suit both individuals and groups E Adapts the types of questions he or she asks to

coaches’ individual learning needs Uses listening and questioning techniques effectively M Has a number of ways of asking questions when

coaches do not understand material Makes limited use of effective listening and questioning techniques NI Always asks the same types of questions when

coaches do not understand material Communication: Non-verbal Uses non-verbal cues to enhance the message being delivered E Uses body language and facial expressions that

gain and hold coaches’ attention Non-verbal cues are consistent with the message being delivered M Uses body language and facial expressions that

match the message being delivered Non-verbal cues are inconsistent with the message being delivered NI Disrespects coaches by showing little interest or

engagement in the training process or training activities

Evaluation Matrix


Standard Examples

Respectful Language Effectively addresses comments from coaches that are racist, sexist, or

demeaning to other coaches E Intervenes immediately if coaches make any

comments that could be considered racist, sexist, or demeaning

Uses language that is respectful and promotes inclusion M Uses language that is accepting and supportive of coaches

Uses language that is racist, sexist, or demeaning to other coaches or allows coaches to use language that is racist, sexist, or demeaning to other coaches

NI Tells stories or jokes that are aimed at specific groups of people

Self-directed Learning Coaches are stimulated to explore, problem-solve, and value learning E Encourages coaches to organize as many

assigned tasks as possible Helps coaches become self-directed learners M Allows groups to organize specific module tasks Does not encourage coaches to become self-directed learners NI Determines the structure of all module tasks Feedback Engages coaches in two-way discussions about coach development E Encourages and models supportive and

constructive two-way discussions about coaches’ actions

Provides feedback that is positive, specific, and informative M Responds to coaches’ actions with supportive and corrective (if necessary) comments

Provides feedback that is negative or judgmental or both NI Makes negative and judgmental comments on coaches’ actions



Manages Group Tasks to Optimize Coaches’ Learning Standard Examples

Explanations of Group Tasks Creates an environment where coaches take responsibility for

completing tasks E Creates situations where coaches have

opportunities to discuss tasks and decide how to complete them

Explains group tasks clearly and concisely and allows for questions of clarification

M Explains a group task and then asks a coach to paraphrase the instructions

Either does not explain group tasks clearly and concisely or does not allow questions of clarification

NI Explains a group task but does not provide any further explanation or allow any discussion of the task

Application of Group-development Theory Adapts the formation and management of groups to the situation E Uses larger groups and riskier activities at the

beginning of training with more experienced groups

Applies group-development theory to the formation and management of groups

M Uses activities that are process-oriented and less risky for coaches (don’t lead to conflict or competition) at the beginning of the training

Creates and manages groups in a manner that does not reflect their stage of development

NI Uses activities that are not appropriate to the group’s stage of development, for example, role-plays and debates early in training

Group Interaction Creates an environment of positive interdependence, where learners

understand they need one another to successfully complete tasks E Reinforces the idea that the group can perform

tasks better than coaches can on their own Creates and modifies groupings to enhance interaction and learning M Groups coaches by characteristics (experience,

age, etc.) that are critical to completing a task Does not use groupings to enhance interaction and learning NI Does not group coaches by characteristics

(experience, age, , etc.) that are critical to completing the task at hand

Evaluation Matrix


Standard Examples

Group Process Uses the group process to help coaches develop interpersonal,

communication, and valuing skills E Ensures that coaches get numerous opportunities

to practise interpersonal communication Intervenes in the group process to ensure that coaches communicate

effectively and show respect for one another M Intervenes when coaches are communicating

ineffectively with one another and provides positive, corrective feedback on how to improve

Does not intervene in the group process to ensure that coaches communicate effectively and show respect for one another

NI Does not intervene when coaches are communicating ineffectively with one another

Leadership within the Group Allows coaches to experience a variety of leadership opportunities within

the group E Fosters confidence in coaches to the point where

they assign tasks to one another without prompting from the LF

Assigns roles, including leadership, within groups M Assigns roles (leader, timekeeper, recorder) to different coaches when assigning a group task

Allows one or two coaches to dominate the leadership role NI Lets leadership in group tasks evolve naturally, leading to one or two coaches dominating group activities



ANNEX A: EVALUATION TOOL

Annex A: Evaluation Tool


Visit coach.ca – Canada’s most dynamic coaching community.

Check your certification, complete online evaluations, access sport nutrition tips, read coach stories and more!

guide to evaluating learning facilitators · pdf filean mcd is assigned to evaluate a specific...

Documents