training evaluation

30
POWER HR FORUM RETURN ON TRAINING INVESTMENT ROTI A White Paper

Upload: dmrpanda9940

Post on 04-Sep-2015

5 views

Category:

Documents


2 download

DESCRIPTION

How to do training evaluation

TRANSCRIPT

  • POWER HR FORUM

    RETURN ON TRAINING INVESTMENT

    ROTI

    A White Paper

  • MESSAGE FROM HONORARY SECREATRY

    Evaluation of Training has always been an important concern in our member

    organizations as training is used by them as a method to develop their human

    resource, build their competence as per dynamic business environment for

    sustainable development and they are spending gregariously on this. Huge

    expenditures are one part for justifying the training during evaluation process but

    most important ones are measuring the learning achieved through training,

    application of those learnings at workplace and how actually they are impacting

    the business outcomes. Ultimately all organizations are spending so much for

    achieving the later results only and to evaluate if their training practices are

    achieving the desired results, they have been looking for some credible available

    model for training evaluation.

    To facilitate the needs of our Human Resource function in our member

    organizations and helping them to evolve as HR Business Partners, Power HR

    Forum has come out with a white paper on Return On Training Evaluation ROTI,

    based on Phillips training evaluation model. This Model illustrated step by step

    how training process need to be designed, what factors need to be considered

    while deciding training objectives at different levels, how to do training evaluation

    at different levels.

    We really hope this white paper, Return On Training Investment will help our HR

    fraternity to carry out their training initiatives in a more impactful manner with

    more holistic approach.

    Subhasis Ghosh

    Executive Director (HR), NTPC &

    Honorary Secretary Power HR Forum

  • Training Evaluation: A Framework

    Contents

    1. Introduction ............................................................................................................................. 1

    2. Rationale to Measure ROTI (Return On Training Investment) ................................................ 1

    3. Stakeholders in the ROTI ......................................................................................................... 2

    4. Levels of Evaluation ................................................................................................................. 2

    5. Evaluation Process ................................................................................................................... 3

    Step 1: Develop Training Programme Objectives ................................................................... 4

    Step 2: Develop Evaluation Plan ............................................................................................. 6

    Step 3: Data Collection .......................................................................................................... 9

    Step 4: Data Analysis ............................................................................................................. 9

    6. The Outcome of Study ........................................................................................................... 11

    7. Undertaking a Training Evaluation Project. 12

    Case I: Training Evaluation is taken as an Afterthought...... 12

    Case II: Training Evaluation as a Planned Initiative.. 13

    Annexure I: Sample Feedback Form I & II14

    Annexure II: Test of Learning17

    Annexure III: Programme Cost Structure.20

  • Training Evaluation A Framework

    1. Introduction

    Capability and therefore Human Resource is aunique source of abundant advantage for any

    organization. Organizations are using training as one of the tools forcapability

    enhancement. Providing appropriate training has become the strategy of the best

    companies to get highest quality output from the employees. It has now become an

    undisputed effective business tool and is linked the business outcomes.

    Worldwide, training and performance improvement programmes have been reported

    ashigh as3.6% of an organizations payroll, 1.3% of revenues and 5.9% of profit (as per ASTD

    2013 research report).

    Aptly, we shall use the term ROTI (Return On Training Investment) here for our framework

    on training evaluation.

    2. Rationale to Measure ROTI (Return On Training Investment)

    The large investment and the need to demonstrateits values, are two of the primary drivers

    placing an emphasis on the evaluation of training programmes. In the past, with

    manufacturing industry in predominant, effectiveness of training could be easily measured

    by production output. Not so much now a days, where training covers domain like

    management and development of organizations, teams, individuals and information. And

    measuring value of training is difficult in case of these softer issues. In addition to these

    trends, following issues are driving the current need to measure the results of training and performance improvement programmes:

    1. To build the credibility of the training function, training managers and training

    processes;

    2. To fulfill clients (those funding the training programmes) requests for evaluation data to

    be sure of the appropriateness of investment;

    3. To justify the training budget and competing for scarce organizational resources

    4. To evaluate the training method used and the use of time for trainer and employees.

    5. To provide evidence to management/stakeholders on success of training programmes.

    6. To identify areas for improvement in total cycle of training process.

    Return On Training Investment-ROTI : A Framework

    1

  • 3. Stakeholders in the ROTI

    As we see above, all of the stakeholders in the training process have different needs and

    they need to be prioritized before one goes for training evaluation. Following stakeholders

    can be associated as per the importance of their needs and requirements:

    1. Training function/HR function

    2. HR Head

    3. Participants (past/present/future)

    4. Reporting Officers

    5. Internal Clients (making investment in training programmes)

    6. Finance Function

    7. Business Operations/Line Functions

    4. Levels of Evaluation

    Developing a credible and comprehensive evaluation process is much like putting together

    the pieces of a puzzle. The first step of the puzzle is the evaluation framework. This

    framework defines the levels at which training prorammes are to be evaluated and how

    data is to be captured. The following framework is based on the work of Donald L. Kirpatrick, with an addition of fifth level by Jack J. Phillips.

    Table 1: Training Evaluation Framework

    Level Measurement Category What does it measures?

    1 Reactions and Satisfaction - Participants reaction to training programmes, - Satisfaction with learning experience, contents, perceived

    value of programme, and services like ambience

    2 Learning - Extent to which participants improve knowledge, increase skills and acquire new behavioral changes

    3

    Application and implementation

    - Specific application and implementation of the learning at work place

    4 Business Impact - Outcomes with variables such as output, quality, time and cost as a result of application of learning

    5 ROTI - Compares the monetary benefits of the business impact to the costs of the programme.

    2

    Return On Training Investment-ROTI : A Framework

  • 5.

    Eval

    uat

    ion

    Pro

    cess

    B

    ase

    d o

    n th

    e a

    bov

    e e

    valu

    atio

    ns le

    vels

    in T

    able

    I, e

    valu

    atio

    n p

    roce

    ss is

    de

    fine

    d g

    raph

    ical

    ly a

    s sh

    own

    in F

    igu

    re 1

    :

    Fig

    1.

    Trai

    nin

    g Ev

    alu

    atio

    n P

    roce

    ss

    De

    velo

    p

    Trai

    nin

    g

    Pro

    gram

    me

    O

    bje

    ctiv

    es

    De

    velo

    p

    Eval

    uat

    ion

    Pla

    n

    Co

    llect

    dat

    a

    in t

    he c

    lass

    roo

    m

    Co

    llect

    dat

    a

    at w

    ork

    plac

    e

    Tan

    gib

    le

    -

    Inta

    ngi

    ble

    Iso

    late

    the

    eff

    ect

    s o

    f

    trai

    nin

    g al

    one

    Co

    nve

    rt

    Bu

    sine

    ss Im

    pac

    t

    Dat

    a to

    m

    one

    tary

    val

    ue

    Cal

    cula

    te

    RO

    TI

    STEP

    1, 2

    Eva

    luat

    ion

    P

    lann

    ing

    STEP

    3 D

    ata

    Co

    lle

    ctio

    n

    Lev

    el 2

    : Lea

    rnin

    g Le

    vel

    4:

    Bus

    ines

    s Im

    pact

    STEP

    4 D

    ata

    An

    alys

    is

    Lev

    el1

    : Rea

    ctio

    n &

    Sati

    sfac

    tio

    n

    Lev

    el 3

    :

    Ap

    pli

    cati

    on

    Lev

    el 5

    : RO

    TI

    Pro

    gram

    me

    C

    ost

    3

    Return On Training Investment-ROTI : A Framework

  • Step 1: Develop Training Programme Objectives

    The first step in training evaluation process involves defining the training programme objectives

    i.e. what intended outcome the programme aimed at. Then the evaluation task become

    checking whether these outcomes are achieved or not. Programme objectives range from a

    basic level of achieving certain feedback ratings, to increased knowledge level to on-job

    application.

    Table-2 in the subsequent pages indicate details of different levels of training objectives

    corresponding to different degree of complexity.

    Table 2: Programme Objectives at Various Levels

    Level What does it measures

    Typical Objectives

    1) Reaction &

    Satisfaction

    Participants

    reaction and satisfaction to

    the programme

    The participants to rate the programme at 8/10 (say). It may

    include the following:

    1. Participants feedback on:

    Faculty delivery

    Course/reference material provided Interaction with faculty

    Administrative arrangements

    Ambience

    Training coordination Fulfillment of learning objectives

    Relevance to workplace requirement

    2. Participants subjective responses on how they plan to put into practice the learning gained at their workplace

    2) Learning Knowledge, skills,

    behavioral changes

    The main objective is to check if participants learning are at a higher level than before attending the training programme.

    These can be specified by:

    Difference between pre and post training test scores

    Simulation/role play performance

    Case Study analysis

    Keeping certain cut-off marks in post test

    4

    Return On Training Investment-ROTI : A Framework

  • Level What does it measures

    Typical Objectives

    3) Application & Implementation

    On job application of

    knowledge / skills from training programme

    At the workplace, the participants are expected to implement the knowledge and skills learnt during the training

    programme to perform their jobs more effectively, as given below:

    Write a business proposal requiring no correction

    Preparing a presentation and deliver to audience

    Shaft alignment of a pump, unaided and within timeframe

    List requirements of spare parts before turbine is overhauled so that it is completed within stipulated time

    Time/activity management with least procrastination

    Demonstrate empathy with co-workers

    Give feedback/coach juniors following correct procedures

    Take up and complete TQM/benchmarking project in stipulated time

    Take up and complete HR Audit project for specific HR domain

    4) Business Impact Outcomes of application of learning at workplace

    Because of the above mentioned on the job application of the knowledge and skills acquired, following business outcomes must be achieved. The given below illustrative examples are linked directly to application objectives, as above.

    Reduced time and rework in preparing the documents

    Decisions are quicker and objectively made as a result of focused presentation

    Reduced dependency on others as a result of skills acquired resulting in manpower and time reduction

    Reduced time between overhauls and less numbers of surprises, hampering progress of work

    Timely completion of task assigned

    Improved interpersonal relations in team resulting into more volume of work output

    5) ROTI Monetary benefits

    Compares the monetary benefits accrued because of the training programme to the cost of programme. The monetary

    benefit is because of time saved, less number of manpower and input resources and increased production/sales.

    For any training programme, its ROTI should be within the range of 130 % to 300 %.

    5

    Return On Training Investment-ROTI : A Framework

  • Step 2: Develop Evaluation Plan

    After determining the objectives the next step is to develop the evaluation plan. This step

    ensures that objectives determined for each evaluation level are addressed appropriately, as

    given in Table 2. Before developing the evaluation plan, the levels to be evaluated need to be

    determined for any programme. This is because as the level of evaluation increases, so does it

    difficulty and expense. It takes time and resources to do a comprehensive ROTI study, so its not

    feasible to do it for every training programme. Table 3 suggests the target coverage for the

    evaluation to be done at every level along with ease and their usefulness.

    Table 3: Evaluation Difficulty and Intended Coverage

    Levels Purpose Ease Usefulness Target

    Coverage Level 1

    Reaction & Satisfaction

    Relatively easy

    Least useful data

    100% (say)

    Level 2

    Learning 80% (say)

    Level 3

    Application 20% (say)

    Level 4

    Business Impact 5% (say)

    Level 5

    ROTI Most difficult Most useful 1% (say)

    The sub steps for this particular step are given below:

    1) Deciding which training programme is to be evaluated and at what level;

    2) Developing training objectives at the desired level; if the evaluation is for higher level,

    then training objectives at lower level need not be developed.

    3) Developing numeric target and baseline wherever applicable and appropriate to the

    level of evaluation and nature of training programme; target for satisfaction level (L1),

    learning (L2), ROTI (L5) are easier to specify numerically whereas for Level 3 and Level

    4, it is difficult.

    4) Establishing data collection method including formats, templates, who would collect,

    time limit and how. This is detailed Table 4:

    Pre-programme communication of the evaluation plan to programme participants help in

    reaffirming the importance of the programme and will encourage participants to help in data

    collection at the appropriate time for evaluation process.

    6

    Return On Training Investment-ROTI : A Framework

  • Table 4: Data Collection Plan

    Levels Data Collection Method Timing of Data

    Collection

    Responsibility

    of Data Collection

    Reaction It requires a one page document having Likert scale for different measures on which feedback is sought. Two different formats are given for illustration in Annexure I.

    Immediately at the end of programme, within the classroom

    Training Manager / Coordinator

    Word of Caution: Data Collection at this level is very easy therefore training managers get tempted to include as many factor as possible. But evaluation at this

    level does not provide us any concrete information. Therefore we need to be cautioned while choosing the factors to be measured and only focus on:- Factors which are actionable and which can aid us to take a business decision.

    Learning This level requires instruments using which

    knowledge/skill gained by participants after training can be assessed. Details of various instruments are shared in Annexure II for more details.

    Pre-Post Test: Once

    at beginning and other at end of training programme, within the classroom Only Post Test: Immediately at the end of programme, within the classroom

    Training

    Manager / Coordinator

    Application & Implementation

    Asking immediate stakeholders of training participants as to how much is the application.

    Data Source are*:

    i. Reporting Officer ii. Participant

    iii. Internal Customers iv. External Customers

    v. Subordinates * Based on priority maximum three stakeholders need to be considered.

    Data collection Method: Interviews using different mediums like

    telephonic, electronic, online, pen & paper formats

    Feedback should collected within 3-6 months of

    programme conductance; outside

    the classroom, at workplace

    Training Manager/Coordinator

    7

    Return On Training Investment-ROTI : A Framework

  • Table 4: Data Collection Plan (contd.)

    Levels Data Collection Method Timing of Data Collection

    Responsibility of Data Collection

    Business Impact

    Asking seniors about the outcome of the implementation of new knowledge/skills

    acquired in training programme.

    Data Sources: -Functional Heads

    -Sectional Heads -Direct Reporting Officer (in case of senior level employees)

    Questions to be asked are- how business objectives as decided earlier have resulted in: -% reduction in mandays / time saved -% increase in customer satisfaction -Reduction in input resources as

    compared to previous year -Achieving higher targeted sales and production goals as compared to last year.

    -% reduction in customer complaints due to improved quality.

    Before end of the year of conductance

    of programme

    Training Manager /

    Coordinator

    ROTI 1. Isolating the effects of training programme

    2. Gathering business impact data and convert it to monetary value

    3. Capturing programme costs

    Detailed ROTI process explained in Step 4

    and Progrmme Cost Structure in Annexure III

    Before end of the year of conductance

    of programme

    Training Manager /

    Coordinator

    8

    Return On Training Investment-ROTI : A Framework

  • Step 3: Data Collection

    This stage of ROTI methodology involves data collection in two time frames:

    i) In the classroom, immediate after the programme:

    Data is collected during the programme with intent to measure participants reaction,

    satisfaction as well as learning. Reaction data can also be useful in predicting application

    as well as forecasting impact data and ROTI. This time frame is applicable for level 1 and

    2 of evaluation only.

    ii) Outside the classroom, at the workplace:

    After the programme, information is gathered regarding the application of the skills and

    knowledge as well as the impact the programme has had on the organizations/business.

    Both hard and soft data are collected using a variety of methods. This time frame is

    applicable for level 3, 4 and 5 of evaluation.

    Step 4: Data Analysis

    This pertains mainly to the fifth level of evaluation in ROTI. The benefits of the programme are

    converted into monetary value at this stage. But before that, contribution made by the training

    programme need to be isolated from the effects of the factors other than training. Following

    sub steps are involved in this stage of data analysis:

    i) Isolate the effects of the training programme:

    Performance factors measured should be directly related to the programme as many

    factors influence the performance of employees other than structured training

    programme. Some common strategies commonly used to address this issue are stated

    as:

    Participants/stakeholders estimate the amount of improvement caused, that

    could be ascribed to training programme alone. This may be 40% -

    100%.Encouragement from seniors, reading from books or online sources,

    learning from peers, are some of the other contributing factors.

    Identify the other factors making impact on performance and estimate or

    calculate the impact by them, leaving the remaining unexplained improvement

    attributed to the programme.

    ii) Convert business impact data to monetary value:

    To calculate ROTI, convert business impact data into monetary values and compare

    those values to programme costs. Depending upon the type of data involved in showing

    9

    Return On Training Investment-ROTI : A Framework

  • business impact, following suggested strategies could be used to convert data into

    monetary values:

    Output data, like additional sales can be converted to profit contribution

    Cost of quality measure, such as customer complaint, can be calculated and

    reported as standard value.

    Employee time saved can be converted to wages saved and benefits, etc.

    The programme benefit data is represented in annualized form which means the

    analysis could have been for two months period and while reporting, it has to be

    converted to 12 months by extrapolation.

    iii) Capture Programme Costs: Calculate all the direct and indirect costs involved in

    the programme. A detailed list of various parameters are at Annexure III.

    iv) Calculate the Return on Investment (ROTI): ROTI can be calculated by

    comparing the net programme benefits and costs. The ratio is usually expressed

    as a percent:

    ROTI (%) = (Net Programme Benefits / Programme Costs) x 100

    Where net programme benefits are programme benefits minus programme

    costs.

    v) Identify intangible measures: Identifying and converting data to monetary value

    is more challenging in case soft data or intangible benefits. Intangible benefits

    include items such as:

    Improved public image

    Increased job satisfaction

    Increased organizational commitment

    Reduced stress

    Improved teamwork

    Improved customer service

    Reduced customer response time

    Enhanced technology leadership

    10

    Return On Training Investment-ROTI : A Framework

  • 6. The outcome of Study

    Reporting the results of a comprehensive measurement and evaluation is the last step in the

    ROTI study and should be systematic and well planned. Table 5 provides a framework of

    contents of reporting outcome of ROTI study:

    Table 5: Suggested Framework for Evaluation Report

    Sl Suggested Contents of Evaluation Report

    Page no

    1 Participating Stakeholder, we are aiming at

    2 Stakeholders need and expectations from training programmes

    3 Level of evaluation decided/desired and the rationale behind for specific training programme

    4 Tangible benefits

    5 Intangible benefits

    6 Lessons learned

    7 Limitations of present study

    11

    Return On Training Investment-ROTI : A Framework

  • 7.

    Un

    der

    taki

    ng

    a Tr

    ain

    ing

    Eva

    luat

    ion

    Pro

    ject

    C

    ase

    I

    Trai

    nin

    g Ev

    alua

    tion

    is t

    aken

    up

    as a

    n A

    fter

    tho

    ught

    (af

    ter

    the

    spec

    ific

    tra

    inin

    g pr

    ogra

    mm

    e ha

    s be

    en c

    ondu

    cted

    )

    Fig

    2.

    Trai

    nin

    g Ev

    alu

    atio

    n f

    or

    pro

    gra

    mm

    es

    alre

    ady

    con

    du

    cte

    d

    Hig

    hlig

    ht

    des

    ire

    d

    de

    pth

    & b

    read

    th

    of

    stu

    dy t

    o t

    ake

    care

    of

    stae

    hold

    ers

    con

    cern

    Sec

    2

    De

    term

    ine

    the

    Stak

    ehol

    ders

    inte

    rest

    ed in

    Trai

    nin

    g Ev

    alu

    atio

    n Se

    c 3

    Esti

    mat

    e t

    he

    cost

    of

    stu

    dy

    & g

    et

    app

    rova

    l

    An

    nexu

    re II

    I, Se

    c A

    Set

    up

    the

    re

    alis

    tic

    ob

    ject

    ives

    of

    the

    pro

    gram

    me

    (Re

    acti

    on/

    Lear

    nin

    g/

    Ap

    plic

    atio

    n/B

    usi

    ness

    Im

    pac

    t/R

    OTI

    )

    Dat

    a C

    olle

    ctio

    n

    Dat

    a A

    nal

    ysis

    Re

    po

    rtin

    g

    the

    ou

    tcom

    e

    of

    Eval

    uat

    ion

    P

    roje

    ct

    12

    Return On Training Investment-ROTI : A Framework

  • Fig

    3.

    Trai

    nin

    g Ev

    alu

    atio

    n f

    or

    pro

    gra

    mm

    es

    to b

    e c

    on

    du

    cte

    d

    Cas

    e II

    Tr

    aini

    ng E

    valu

    atio

    n is

    ta

    ken

    up, a

    s a

    plan

    ned

    init

    iati

    ve a

    nd t

    houg

    ht

    of b

    efo

    re t

    he p

    rogr

    amm

    e is

    con

    ceiv

    ed

    St

    akeh

    olde

    r Id

    enti

    fica

    tio

    n

    Cap

    turi

    ng

    Stak

    ehol

    ders

    co

    nce

    rn(s

    )

    Esta

    blis

    h

    leve

    l of

    eva

    luat

    ion

    Pro

    gram

    me

    De

    sign

    Sett

    ing

    Obj

    ect

    ive

    s:

    -Re

    acti

    on

    -Le

    arni

    ng

    -App

    licat

    ion

    -Bu

    sine

    ss Im

    pac

    t

    -R

    OTI

    Dat

    a co

    llect

    ion

    Dat

    a A

    nal

    ysis

    Re

    po

    rtin

    g

    the

    ou

    tco

    me

    of

    eva

    luat

    ion

    13

    Return On Training Investment-ROTI : A Framework

  • Annexure I

    A. Sample Feedback Form I (used by NTPC)

    PROGRAM TITLE Assessor Building for NTPC Business Excellence Model

    PROGRAM DURATION Aug 5-8, 2014

    Organizer PMI

    VENUE SCOPE Complex, MCM Hall, New Delhi

    Program coordinator DMR PANDA

    Please provide feedback on following parameters on a scale of 1-10, where 1 Not Satisfied and in the increasing order of satisfaction 10- Highly Satisfied.

    1. Please rate the extent to which Overall Program Objectives were achieved

    Programme Objective:

    Describe the internal assessment model for business excellence

    State the coverage of nine criteria of excellence Explain the process of assessment of submitted document from participating

    stations Evaluate and score the application document from any NTPC power station

    Describe the process of generating feedback report for the particular station based on the review of application document and plant visit

    S

    No

    Not Satisfied Highly

    Satisfied

    1 2 3 4 5 6 7 8 9 10 1. Overall Program Objectives

    2. Please rate the extent to which the following aspects of the program are met as per your expectation

    S No Dimensions Not Satisfied Highly Satisfied

    1 2 3 4 5 6 7 8 9 10

    1. Content & Depth of Coverage

    2. Program Duration

    3. Relevance to your role

    4. Program Coordinated

    14

    Return On Training Investment-ROTI : A Framework

  • 3. Please rate the extent to which different faculties met your expectations in terms of subject

    knowledge, structure and delivery

    S No Session Topic Faculty Name Not Satisfied Highly Satisfied

    1 2 3 4 5 6 7 8 9 10

    1 Internal BE Model C V Subrahmanyam

    4. Please rate the extent to which following Administrative Aspects are met as per your expectation

    S No Dimensions Not Satisfied Highly Satisfied

    1 2 3 4 5 6 7 8 9 10

    1. Hospitality Arrangements

    2. Overall Administrative Arrangements

    3. In house residential arrangements

    5. Kindly mention where and how you shall apply the learning at workplace?

    6. Kindly give your comments regarding program and suggestions for improvement of the Program.

    Date ------------------------------ Signature________________

    Name ___________________

    Emp No.________________

    15

    Return On Training Investment-ROTI : A Framework

  • B. Sample Feedback Form II (used by Power HR Forum)

    PROGRAM TITLE CSR & SD

    PROGRAM DURATION September 29, 2014

    VENUE Power HR Forum, PMI NOIDA Please provide feedback on following parameters.

    1. Please rate the extent to which Overall Program Objectives were achieved

    Moderate (less than (

  • Annexure-II

    Test of Learning

    The aim of the Learning Evaluation (Level-2) is to check if the learning of the participants

    has reached to some desired level after attending the training programme. There are two

    options in this regard:

    1) Pre-post test

    2) Post test

    Further, each of these categories may include following methods:

    a) MCQ (multiple choice questions)

    b) True/False type

    c) Subjective assessment

    d) Demonstration through role play / working on models

    e) Analysis of case studies

    f) Simulation

    The job of the training manager at this stage is to follow the learning objectives finalized

    during the step 1. Then based on those objectives, determine the mode he/she going to use

    for assessing the change in learning achieved during training programme as expla ined

    below:

    A) Pre-Post Training Test and Post Test:

    Pre-Post Test is preferred where participant has some background knowledge of the

    training programme contents or participants are already working in the same domain

    and the intent of training programme is to acquire advanced skills. Then, training

    manager can make out clearly improvement in the skill levels of employees.

    Table 1: Comparison of Test Methods

    Parameters Pre-Post Test Post Test

    Level of skill to be achieved Degree of learning enhancement

    Certain level of proficiency has to be

    achieved; say 70 % at end of training.

    Criticality of learning High Lower

    Location of proficiency Within the classroom Within the classroom Extra time and cost More Less

    Seriousness/objectivity/engagement of participants

    High Lower

    17

    Return On Training Investment-ROTI : A Framework

  • B) Choosing among Objective/Subjective/Case Studies/Role Plays

    1. Objective (MCQs/True-False):

    These mode of test should be used when the group size is large (say) more than 15,

    time available is less, criticality of the skills to be acquired is not that important or

    specific and focus is upon acquiring knowledge.

    2. Subjective Questions:

    This mode is preferred when the population size is restricted a certain level of

    knowledge/skill have to be demonstrated. In case,where the criticality of the

    knowledge/skill to be acquired is more important than population size does not

    matter.

    3. Simulations:

    This mode of testing is preferred when the skills acquired during the training

    programme are of critical importance and have to be demonstrated by every

    participant to perform his job at workplace.

    4. Case Studies:

    This approach has to be used when we want the participants to learn how to

    implement the learning from the training programmes to the real life situations.

    5. Role Plays:

    This approach is preferred in behavioral trainings where we want participants to act

    on specific situations and learn, feel and experience it to gain more clarity. This

    approach should be preferred when availability of time is not a restriction.

    Table 2: Usage of Different Instruments for evaluating Learning

    Parameters Objective Subjective Simulations Case Studies

    Role Plays

    Participant Group

    More than 15

    Less than 15 Less than 15 Less than 15

    Less than 15

    Testing Time Least Lesser More More Less

    Evaluation Time

    Least More More More More

    Automation

    Level

    Highest Not possible Not possible Not

    possible

    Not

    possible Criticality of skills acquired

    Moderate Moderate Highest High High

    18

    Return On Training Investment-ROTI : A Framework

  • Usually objective test is most widely used which is a good compromise between

    outcome achieved and input required.

    (C) How to decide the Number and Difficulty Level of Questions in Pre-Post/Post Training Tests

    To get the more accurate picture of the knowledge/skills acquired during the training

    programme, inclination is on to ask more number questions, for a clearer assessment.

    But this has to be restricted to limit ourselves and be more focused. When we are going

    for objective type of questions, one rule of thumb is to ask 16 questions for a day of

    training programme or 4 questions per session. Say a training programme is scheduled

    for 3 days, then the number of questions come down to 45. In case of subjective type of

    questions 6 questions per day should be asked. A combination of objective as well

    subjective can also be used.

    While deciding the level of questions to be asked in the Pre-Post/Post Training Tests, the

    objectives of the training programme as pre-decided need to be considered and

    accordingly decided.

    19

    Return On Training Investment-ROTI : A Framework

  • Annexure III

    Programme Cost Structure

    This stage involves calculating the programme costs incurred and outcomes of the

    training programme. Programme cost can be divided into two heads i .e. Direct Cost and

    Indirect Cost.

    A) Programme Cost:

    i) Direct costs are expenses an organization can easily specify for an object or

    services acquired. Companies describe these as expenses where one gets what

    one pays. An organization can have its own list of direct cost depending on the

    resources available with them. A list specifying indirect costs incurred in a

    training programme is given below:-

    1. Facilitator Fees (if outsourced)

    2. Boarding and Lodging Fees for Faculty

    3. Faulty travel fees

    4. Refreshment expenses for participants

    5. Course Material

    6. Stationary Items

    7. Training Venue

    8. Travelling expenses of participants

    9. Loss of productivity or costs of backfilling positions during training

    which is at least equal to CTC of employee

    10. Cost of Evaluation Study (employee cost and time, travel time, etc.)

    ii) Indirect cost involves costs which are spread out for usage for different

    purposes and cannot be calculated directly for the product/service acquired. A

    list specifying indirect costs incurred in a training programme is given below:-

    1. Salaries of Training Department personnel involved

    2. Infrastructure (training venue, audio-visual aids)

    3. Communication elements, air conditioning, building up keep, etc.

    20

    Return On Training Investment-ROTI : A Framework

  • B) Training Benefits: To calculate the programme benefits, business impact created

    due to training programme like reduction in manpower, enhanced customer

    satisfaction, improved quality needs to be converted into the monetary value for

    the same financial year. Depending upon the type of data involved in showing

    business impact, following suggested strategies could be used to convert data into

    monetary values:

    a. Output data, like additional sales can be converted to profit contribution

    b. Cost of quality measure, such as customer complaint, can be calculated and

    reported as standard value. c. Employee time saved can be converted to wages and benefits, etc.

    21

    Return On Training Investment-ROTI : A Framework