measuring roi enabling l & d to drive value of learning

HR Leading Edge Series Measuring ROI

FURTHER READING


Gray Matters Consulting HSPP Consultants Ltd.2

CONTENTS1. Managing the Learning &

Development Function3

2. Learning Strategy – Future Investment

8

3. Identification of Training Needs 14

4. Evaluation of Learning and Development

23

5. Kirkpatrick’s Model 326. Philips ROI Model 357. CIRO Model 408. Brinkerhoff’s Model 429. About HSPP 46

10. About Gray Matters 4711. Facilitator Profiles 49



Managing the Making learning and talent development strategicUntil the early 2000s, when the company was smaller, with just three stores and one hotel, training and development in the company was fairly ad hoc. In 2001 Garvey Group brought in an Operations & HR Director to the board, who came from an Irish-based multinational company where she had been responsible for leading achievement of Excellence through People (Ireland’s national standard for human resource management, comparable with the British ‘Investors in People’ kitemark).

The Operations & HR Director saw Excellence through People as a positive way of structuring HR systems and practices, and of linking learning and development within the overall business strategy. So she led its implementation in the Garvey Group. The result by the mid-2000s was a very structured approach to training, driven now by the business strategy. A corporate training plan is prepared annually, costed and evaluated through deliverables such as sales and profitability. A modified balanced scorecard (kaplan and Norton 1996) provides a framework for the strategy, starting with the core belief that ‘if you had your people trained, your operations would be right, if this was right, in turn your customers would be happy; in turn your business would be right. So the starting building block was always the people’ (Operations & HR Director).

The independent audits provided by Excellence through People were welcomed as a useful source of feedback and suggestion, that helped the company become more systematic. Prior to that training was more ad hoc, and not systematically aligned with business strategy. In the days when the Garvey Group had just three stores and one hotel all within the same areas, this relative informality was perhaps manageable, but as the Group grew throughout the decade Excellence through People was used as a framework to help grow the group, to facilitate acquisition of other business and integrate them in a seamless manner.

From business strategy to training needs analysisEach year a training plan is prepared for each location. Considerable time is spent identifying employees’ training needs, using the appraisal system to provide opportunity to identify training needed and based on a needs analysis for their job.

function

learning & talentdevelopment

The Garvey Group – Case Study

Company ownership, sector and sizeThe Garvey Group is a family-owned retail and hotel company in Ireland, primarily Munster-based (covering Counties kerry, Cork and Waterford in the South of Ireland). It has 10 stores across this region plus two hotels and a sports and leisure shop in Dingle, Co. kerry.There are 850 employees, rising to around 1000 in the summer months. Most staff are department sales assistants working in the retail shops on the meat or fish counter, for example, or on the tills. In the hotels, most would be employed in the bar or restaurant. Each store and hotel has a manager and assistant manager, as well as at least one and sometimes two training managers. Strategic leadership is provided by a small board of directors, comprising five people: the founder and chairman, one other family member and three non-family members.



Some training needs are technical, for example, specific to those who work in the fish department or meat section of the stores. The company uses a competence based approach, with much emphasis put on behavioural competences applicable to all jobs, such as customer care, working as a team and ‘subtle selling’. These skills underpin the business strategy. For sales assistants in particular, great emphasis is placed on customer service training, being ambassadors for the company and selling skills – telling customers in a friendly way about promotions, so customers might not even notice, ‘subtle selling’ as the company call it. Customer service is seen as so central to the achievement of business strategy that there could be training three times a year, to reinforce and deepen the learning. In general if people cannot complete training in their own store they generally get to attend in another area.

Training providers – external and internalThe main provider of training for Garvey Group is the Musgrave Group (the Irish group that owns many household grocery brands across Ireland, the united kingdom and Spain and provides sales, marketing, IT, finance and logistical expertise to retail companies such as Garvey Group.

Garveys use Musgrave Group’s training department for specialist retail training, for example, in flowers or fish, whilst for specialist hotel training, such as bar and restaurant skills, the Irish Hotels Federation (IHF) is used. For other generic skills, such as customer service or team building, great emphasis has been placed on developing internal training expertise. Each store has at least one and often two trained trainers, even occasionally three. The HR person from each store is one of the trainers; whilst the second one is typically one of the checkout supervisors because of their experience of having many people report to them. Trainers typically originate on the shop floor, starting on the checkout tills. They regularly meet with the company’s Operations & HR Director and are appraised by her. They receive training themselves twice a year training on for, example, evaluation, transfer to work, making training relevant.

This emphasis on investing in people’s skills, competences and behaviours, driven by a clear strategic plan is at the forefront of practice in the retailing sector, to the extent that Garvey Group trainers are often called on to run training for other retailers.

External trainers are used only very selectively. The Musgrave group have a retail diploma accredited by Dublin Institute of Technology and also operate a list of approved trainers across the country, people with retail experience who can deliver practical, hands-on and experiential training directly informed by real scenarios, rather than being generic.

Planning and evaluatingEvery course has objectives, and the Kirkpatrick model is used to identify these and to systematically ask what individuals expect to get from any training event. So rather than being sent on a fish course, for instance, a person will know why they are going and what are the objectives.

After the training, they will be asked what they learned, whether it matched the objectives, what they will do now. Also added in is the question of what they will share. Employees are very much encouraged to summarise key learning from any course in their next week’s meeting at work. Even with on-the-job, ‘sit by Nelly’ training, for example in using the tills, there are training objectives set and a review afterwards, to ensure there are no remaining gaps. The focus is constantly on trying to make the training relevant to the job and relevant to the longer-term business objectives.

Training managers feed back the evaluations to the Operations & HR Director, who communicates back to providers, whether these be internal or external from Musgraves and the IHF.

At a corporate level the ROI (return on investment) measures used for evaluation were primarily sales and profit figures. For example, comparison would be made of fish sales and margins before and two months after a fish course. Generally there was an improvement evident, an uplift in sales figures following training and this data was used to justify training investment to retail managers and the board.



Employee expectationsWhen the company’s focus on training began, initially there was a degree of apprehension from the workforce. Whilst some of these were young college and school students, many were 40 and older. Fears were voiced of ‘Is there a test?’ ‘I’ll never be able for it’. So the initial hurdle was to get people’s buy-in, and encourage them to attend at least the first course. As it happened, once they went on one course, they were generally delighted to be able to go again.

Part of this was achieved through getting their input, for example, asking for their ideas on what is the best way to give customer service. The company culture is open, with an emphasis on personal development and improvement. The team of HR and trainers across the whole group regularly meet together around twice a year to explore how they can we make training better. Participation in training is celebrated and annually people get a printout of their training record. The company deliberately ‘hero’ their staff, for example, publicizing pictures across the stores of a Master Butcher, Master Baker or star of the Baby department, to give people a sense of pride in their job and to acknowledge them as an expert.

All this contributes to a culture where people are positive towards training, feel a sense of achievement and are proud of their training records.

Managing the managersInitially the Operations & HR Director had to work hard to persuade the middle managers (store managers, hotel managers) of the value of releasing people for training, because it is an industry where the wage bill is high and they have very specific operational targets regarding rostered hours.

To release an employee for 8 hours' training means not only do they have to pay the person for these hours although they are not on the shop floor, but they also have to buy in a replacement for the 8 hours to cover the shift. So initially there was a huge debate. However, the managers also had other performance measures, such as mystery shoppers, and they could see that if they did not put investment into the training, they were not getting the same recognition for customer service.

Often, if there was a poor mystery shopper result, the excuse was ‘well, she never went on the training course’. In time managers recognised a direct result

between the training of their employees and achievement on the job. Induction was another point of potential tension between the training strategy and managers. Newcomers had two days invested in their induction, covering the basics such as company culture and the emphasis on customer service, even before they began to learn such skills as how to use a till.

There was a tension when a retail manager could be crying out to have them on the till immediately and trainers saying, wait, you want them with the right attitudes and behaviours, they’ll add so much more value.

Persuading the board of the value of investing in training was not so difficult, because the Operations and HR Director was a member of the board and was explicitly hired into the company with this brief.

Nevertheless, it was important to highlight deliverables and to continuously provide evidence to link the training to profitability or sales, for example, demonstrating improvements that followed training.

Employer brandingGarvey Group are very proud of their training record and their recognition with Excellence through People. They were the second Irish retailer to achieve gold and are the first retailer and (by early 2009) one of only 18 Irish companies to achieve platinum. These achievements are seen as a major contributor to their employer branding, and the logos are proudly and widely displayed. The retail sector does not pay high wages, the work is tough, including regular evening and weekend working, dealing with customers, so to make employment attractive, an employer has to be able to highlight how they are distinctive, For Garvey Group this is the experience and development offered to employees.

Being known to treat employees well and train well has meant the group recruited some excellent employees. Once Excellence through People was achieved it gave the company a huge competitive edge; there was a noticeable increase in the quality of applications and a rise in graduate applicants. Candidates were clearly aware of the



Excellence through People recognition and would mention it as one of their reasons for wanting to work with the company, because it signalled they were serious about training and learning and were a company that could offer career opportunities. Excellence through People put them in a different league.

Talent managementGarvey Group have a segment in their balanced scorecard termed 'talent management', which was introduced because, in previous years of relatively full employment there had been a real difficulty in recruiting talent. For the retail sector this is a challenge because a career in the sector is not seen as offering great potential. Having found it difficult to attract supervisors and good trainee managers they made the decision to home-grow their own.

The Garvey understanding of ‘talent’ is people ‘having capacity to be more than they currently are’. There is also a recognition that talent in this sector is very much related to personality and whether a person can relate to the customers or has people management skills, regardless of the degree they have on paper. To grow their own, investment was made in spotting and developing talent. Managers had, once a month, to identify someone who had the potential to progress. Often they were someone working on the till for a few hours after school or in their college breaks. Talent development consisted of an 18-month trainee manager structured fast track development.

Trainee managers can also be recruited directly to the programme, which involves on-the-job training to learn all aspects of the job as well as external training, devised with Musgraves, to be retail specific. During the 18 months they would be met bi-monthly to agree training objectives and there were monthly milestones. Their line manager was accountable for making sure they progressed and were adding value to the store.

Each ‘talent’ is assigned a mentor, with regular review meetings held. Mentoring, however, has proved something of a challenge, because in most cases the mentor has been the trainee’s direct line manager, with the result that meetings became more performance reviews than true mentoring

sessions. It has proved difficult with the size of the company to give people a different mentor.Mentoring has been more successfully used with the store managers, who each have been mentored by one of the Board members who is not their line manager.

Another unresolved aspect of the company’s talent management is that the system only developed towards the position of trainee manager. There was no similar system for development towards supervisor although until the recent economic downturn there was a need.

With the recession, there is now a flood of external supervisors to choose from so growing internal talent is no longer essential.

Top management developmentThe company to date has not had a systematic or strategic approach to developing Board members. Individuals have pursued development opportunities at their own discretion.

The language of learning, training and talentUntil recent years Garvey Group spoke of training and development in all their strategy documents. However, one night, triggered by an Excellence through People audit recommendation, they rewrote all the policy documents, substituting the word ‘learning’ for ‘training’. The employee reaction was quite unexpected. Staff questioned how learning could be the subject of strategy or be documented and recorded because it was what they did anyway.

Training, by contrast, was seen as involving courses, something special and important, that could be documented and signed off. For senior managers this highlighted the evolution the company had been through, coming from having no formal HR to developing good systems to having Excellence through People to achieving Gold then Platinum. Employees needed more time to adjust to talking of ‘learning’ as being part of what they did each day.

The language of ‘talent’ posed a second source of challenge, with some asking ‘are you just singling out special people?’ The answer was, unfortunately yes, because of the need to develop the internal management team.



Into the future?Overall the company has no doubt that investment in people’s learning and talent development produces results to the bottom line. Also, that strategically managing training and learning is essential to get the best out of people. But what of the future challenges in the face of the current economic downturn? It is tougher to keep the focus on investing and the risk is that training will be less, not only because of the cost of training, but also the operational impacts of covering people taken out for training. In the medium term the company believe that, because staff turnover is low, there is some cushioning from past investment in development, although that could become more of an issue in time. There are strengths in being closely tied to the Musgrave group and accessing highly relevant, industry specific training. They also have a degree of internal sustainability through having a well-developed network of trained internal trainers as well as the established framework derived from Excellence through People of using training plans, setting objectives and review. Maintaining this will be the challenge.

ROI as process ROI measurement is the process of collecting and analysing this performance data, and translating this into a measurement of real financial benefit to the organisation. This benefit is then compared to the cost of creating this benefit through training and measurement. In many cases, ROI measurement can be linked to data collected and analysed for the purpose of Training Needs Analysis (TNA). If detailed TNA studies are done prior to the training, the data from these studies can be compared to the feedback and performance data acquired after the training takes place. In addition, the TNA is likely to highlight the expected benefits and results from the training. In this case, the change in performance may be more accurately determined.

ROI as Perception So, what actually is ROI on training? It can be considered to be a perception on the part of the client of how valuable the training has been in achieving their perceived goals; and these perceptions will vary depending on whom you talk to. For example: The Board may see a big picture of how the training affects the company‟s ability to achieve its corporate goals The finance department may be looking to see how training stacks up financially against other ways to invest the company‟s money, and whether the training, as carried out, was financially more effective than alternative forms of development. The business unit manager may be solely concerned with the impact on performance and productivity in achieving the goals of their department. The training and development manager may be concerned with the impact training programmes are having on the credibility and status as the training function within the company and its ability to secure investment in the future to drive further business performance enhancements With all these potentially different viewpoints, one of the first things you need to consider with your client, is what the client actually considers is a return on investment, and which views of success are critical to the measurement process.



LearningIntroductionLearning drives real business value. A learning strategy is an investment in a future of growth and organizational performance in order to both enable and optimize the workforce during any enterprise transformation initiative.

A learning strategy links learning initiatives to overall, enterprise wide organizational priorities and goals. The underlying assumption that drives an effectivelearning strategy is that learning is the critical enabler ensuring that people are continuously refreshing and enhancing their skills and knowledge. People mustbe enabled and re-enabled with the right skills and knowledge to meet the changing demands of the marketplace, customers and business itself. A learning strategy helps to ensure that a business’ learning initiatives are linked to and support the overall business strategy.Fully realizing the potential of learning by linking it to competitive advantage and business success requires an enterprise to align its learning initiatives to organizational priorities using an effective enterprise learning strategy.

A learning strategy must align with and support an overall business strategy to help ensure that people are enabled to meet the changing demands of business. A learning strategy must align learning investments directly to business results while providing metrics to measure the value of those learning investments. A learning strategy is required to identify and develop the critical people skills and resources to help establish alignment with business priorities.In 2005 IBM conducted a Global Human Capital Study (GHCS), which sought to understand how human resource (HR) leaders in organizations are leveraging their people to improve workforce effectiveness and organizational performance.People strategies, practices and metrics were surveyed in 334 organizations throughout 38 countries. Interviews were conducted globally with 106 top HR leaders to understand their people priorities and trends.This research revealed that companies’ primary business objectives include increasing profitability, improving customer responsiveness and reducing costs. It also concluded that people were considered a critical factor to achieving longterm profitable growth.

futureStrategy

Investment



Selected findings and implications resulting from the GHCS study1 indicate the value of a learning strategy. Following are some of these findings:

• Learning activities represent the highest single human capital cost area; most companies, however, are not evaluating their investment in training like they do for other large corporate investments.

• Larger companies are more likely to have learning strategies that are aligned with the company’s business objectives.

• Investments in training days at the senior level appear to have links to greater profits per employee.

Some of the recommendations based on these findings are as follows:• Develop an outcome-focused perspective on

learning activities, including an evaluation of return on investment (ROI).

• Consider learning investments for high-potential individuals at all levels of the organization.

• Identify how blended learning approaches using classroom and distance learning can be used to provide better access to corporate knowledge.

• Recognize that as organizations grow, the barriers associated with locating and sharing knowledge and expertise grow as well, requiring additional attention and focus.

• Consider the need and the capability of the HR organization to provide knowledge and collaboration support.

IBM and ASTD’s research on the perceptions of learning by C-level decision makers2 also provides insight into the role of learning strategy in leading organizations.

Chief Learning Officers (CLOs) and other C-level executives from 26 companies across 10 industry sectors were asked to assess the strategic valueof learning; how it translates into business results; how it stays in alignment; and how its value is measured.

Strategic value provided by the learning function

Chief Operating officers(COO) – “Learning help us develop bench strength and the competencies needed for the future.”“Learning’s role is to build the platform to enable us to change the business.”

Chief Executive Officers(CEO) – “The learning function’s role is to help our company learn, adopt, adapt and grow.”

CLO – “Learning will engage early to analyze and improve the design of jobs and processes and technology based performance support tools. It can help in achieving all the critical success factors for our company, not just the people factors. Our workforce management process ensures that we have the best people at the optimal price in place”.

The executives in this survey see learning’s strategic value to build the skills necessary to address upcoming challenges facing the organization such as innovation, transformation and globalization.

The learning function must evolve to enable strategic enterprise initiatives in addition to addressing tactical business-unit performance issues. Learning strategies, approaches and infrastructures must become more flexible in order to rapidly respond to both enterprise and business unit level strategic needs.



Federal Government Agency

Challenge

A large government agency had many learning challenges. Its learning activities and delivery were event-driven and primarily delivered and conducted via one on-one interaction. They found that their learning initiatives were too expensive and were not meeting required performance-based metrics, that they did nothave a learning infrastructure that allowed for the possibility of various blended learning approaches for delivery, and that the organizational control of learning was too centralized.

Solution

The agency wanted to change from a centralized level of control to a dispersedregional level of control in order to build and maintain mutually beneficial partnershipsthroughout the agency. IBM developed an enterprise learning strategy and competency development approach to drive customized learning across diverse audience groups, while providing the logistics and delivery for a blended learning solution that included both instructor-led and Web-based training. Additionally, the strategy enabled a technology platform that will provide access to embeddedlearning, virtual classrooms and online collaboration.

Benefits

The agency anticipates a reduction of maintenance costs and previously lost taxrevenue as well as reduced loss of life and property damage. Additionally, theagency expects to decrease the amount of public money it spends.

Health Insurance Firm

Challenge

After performing benchmark analysis against key competitors, a health insurance firm learned that its training investments were not keeping pace with other healthcare companies. It saw it had limited and dispersed training resources while facing an environment of increased competition and customer demand.

Solution

In order to become more competitive, the firm identified several outcomes that were key to its continued success in its marketplace including: the requirement for a dynamic, integrated, easy-to-navigate training system; training that is delivered at the right time, at the right location; and an optimized blend of instructor-led andWeb-based training. The firm wanted to ensure these outcomes would be realized over time by implementing enterprise wide training governance and controls.

Benefits

The learning strategy included a learning portal which minimized licensing and maintenance costs. The strategy also helped ensure that people have the skill sets required to do their jobs while delivering better training more quickly at a lower overall cost. The strategy ensured that proven best practices were used for training methods and procedures.

Examples of developing a successful learning strategy

IBM has helped many organizations develop learning strategies. The breadth of possible outcomes resulting from the development and implementation of learning strategy are described in the examples below.



Telecommunications Company

Challenge

A telecom company realized that the knowledge of its sales agents was not keeping pace with its new products and services. The company did not keep up with its training investments (in dollars and resources) when compared withworld-class training benchmarks and its competitors. Also, several independent,unaccountable training organizations and/or budgets began to appear in individual business units.

Solution

IBM conducted a thorough evaluation of the training operations, content developmentand management, training infrastructure and technology. The team compared the telecom company’s existing practices to world-class training practices within and beyond the telecommunications industry, identified gaps and made recommendations to close those gaps.

The gap analysis revealed an absence of enterprise wide coordination, integration,communication or management of the training function. There also were no standards for course development templates, tools, functionality or processes. Nor were there standards in place for purchasing training from external vendors.

Benefits

The company realized the following benefits as a result of gap analysis recommendations:

• Reduced time-to-productivity for new agents• Improved agent product knowledge• Increased available informal learning and

knowledge• Reduced time to release new products and

processes• Integrated enterprise learning management, content management, and virtualclassroom solution, with tangible payback over two years.

Internet Services Company

Challenge

An Internet services company wanted to tie training to its overall business goals. Faced with an uncertain and changing business environment, it sought an understandingof critical learning initiatives needed to support its business model while continuing to operate through change.

Solution

The company assessed the practices of its learning and development group in order to identify its overall impact on the business and understand how to address the business culture changes it was facing. This assessment resultedin the creation of key learning and development initiatives that covered business benefits and prioritized recommendations, a timeline, risks and success factors.

Some of the recommendations included: the creation of enterprise-level learning governance and a requirement to focus training on staff with the greatest need. Additionally the blend of instructor-led versus Web-based training became a 20 percent to 80 percent mix.

Benefits

Benefits included a new learning strategy aligned with business objectives that provided increased control over budgets and strategic purchases, a 10 percent cut in training spending, and a reduction in travel costs by using virtual meetings and collaboration. Additionally, the strategy established new processes for annual learning planning supported by strategic learning plans and service level agreements (SLAs) with business groups.



The role of learning governance

Often a learning strategy recommends implementing a new, or enhancing an existing, learning governance model.Learning governance consists of planning for learning, allocating investments to learning and managing those investments3. A governance structure supports these actions by defining the rules, processes, metrics and organizational modelsneeded for effective decision making, steering and control of learning to achieve business goals.

IBM’s research on the perceptions of learning by C-level decision makers4 provides insight into the role of learning governance in leading organizations.Following are some of the key findings that have emerged from the research:• Governance of learning is no longer optional.

Learning has become too strategic, critical and costly to manage without an ongoing effort.

• There’s no one-size-fits-all. Successful governance structures balance the efficiencies of central control and the effectiveness of making decisions close to the line of business.

• Governance models evolve. The role of governance evolves through identifiable stages from inception through maturity.

Learning vision is a key element of learning strategy

IBM’s point of view on the future of learning3 can help organizations understand how people will learn and work in the future and how to develop a set of priorities that guide the evolution of learning within their enterprises.

The journey toward the future of learning is different for each organization, driven by different starting points, different objectives and different organizational imperatives. This point of view can be tailored or tuned to an individual organizationto help identify and prioritize appropriate recommendations.

Successful execution of a learning strategy—and the transformation it enables—requires a shift in thinking and different execution across five core dimensions.IBM Learning Strategy solution provides a framework for thinking about learning across these five core dimensions during the development of a learning strategy.5These five dimensions are outlined as follows.

Organizational alignment

Organizations that target learning investments to areas that really impact the business use a rigorous learning strategy development process that identifies business goals and then links these business goals to learning investments.

Design and delivery

Expanding instructional design to include more focus on learning that is embedded in work, informal learning has always played a larger role than most people imagined. Now it’s becoming increasingly important as workers take greaterresponsibility for their own learning.

Technology

Technology helps companies move beyond automation of formal learning to the creation of pervasive learning environments that support vibrant learning cultures. Many of the technologies that do this already exist in an organization, they just may not be thought of as potential learning tools.

Governance and management

Reinventing the role of learning by enabling a governance and management system delivers real business value. New leadership and governance models will drive accountability and accelerate organizations’ migration to the future of learning.

Culture and behavior

Recognizing this is a major change initiative with a need to focus as much on culture as on technology is key to a successful learning strategy. The world’s elite organizations share one thing—corporate cultures that value people and howthey learn.



Aligning learning with business objectives

A business case is a key learning strategy deliverable. The business case explains how the learning strategy will support overall business objectives, and can include recommendations for:

• Transforming a product-based firm into the service based business.

• Keeping up with explosive growth and new markets.

• Decentralizing to better serve the needs of customers.

• Improving response to regulatory demands.

The business case enables executive decision makers to evaluate the worthiness of investments in learning relative to other key business investments. In quantifying a variety of benefits, it can articulate the cost benefits of closing existinglearning gaps as well as the acceleration of future learning initiatives.

Some potential benefits of a business case include:

• Development of concrete, top-priority actionable steps.

• Cost reductions for upgrades, staff and infrastructure.

• Improved employee-to-employee and employee-to-expert collaboration.

• Ensuring that people have the right skill sets required to do their jobs.

• Increased training productivity—delivered faster, better, and ultimately less costly.

• Support of the idea of “Build upon our greatest strength—our people”.

• World-class standards for training methods and procedures.

• Measurement and reporting on the effectiveness of training.

• Elimination of duplicate training activities, job roles and activities.

• Assessment of learning effectiveness through benchmarking.

Conclusion

Consider developing an enterprise learning strategy if you see imminent change such as a merger or acquisition, downsizing or growth. Organizations are recognizing that learning accelerates change and that the right learning strategy can help speed time-to-competency and achieve greater business results.

When organizations ask their workforce to change, with new job roles being created and existing roles being redefined, new skills are required and learning can play a critical role in transforming the organization.



Identification Introduction

Identification of training needs is the stepping stone in the appropriate management training system on which the entire edifice of a training program is built. Like the prevalent management development and training systems, appropriate management training system does not take a generalized view of training needs. It attempts to take a specialized and individualized view of training needs of different categories of managers performing more or less similar jobs in similar organizations. Assessing training needs is treated as important and crucial training function before designing and conducting a program.

Concept

An organization normally employs sufficient number of managerial and non- managerial personnel with required competence to perform given jobs leading to accomplishment of organizational objectives. Effective performance of a job requires a standard of competence in the job holder consisting of vital areas such as job related knowledge, skill and attitude. But due to one or other reasons the existing competence of the job holder may fall short of the standard competence required in a job. The process of assessing and finding the gap between the standard competence required in a job and the existing competence in terms of vital knowledge, skill and attitude in the job holder may be called as the identification of the areas of deficiencies and the resultant inventory of gaps in the job holder in terms of knowledge, skill and attitude may be said as his training needs. This logic highlights two processes: first identification and definition of standards of knowledge, skills and attitude required in a job and second assessing existing level of knowledge, skills and attitude of the job holder. These two processes are of critical importance and should be carried out as accurately as possible because the outcome of the comparison between the two provides the information from which an appropriate training program is developed.

Identification of training needs is the stepping stone in the appropriate management training system on which the entire edifice of a training program is built.

TrainingOf

Needs



Assessment Of Training NeedsAlmost all the books in training suggest formal assessment of training needs as pre – requisite to training programs. ILO’s book has taken a very comprehensive view on needs assessment and has described the process of identifying training needs under two situations – ideal and less than ideal. Needs assessment under ideal situation, consists of steps like to take an inventory of present manpower, to make forecasts of future requirements, to find the people needed and to decide what to do to develop manpower. This process under ‘less than ideal’ situation includes stages such as considering terms of reference, considering the situation within the organization, considering information available in respect of new and existing employees, considering the problems which arise within the organization and considering possible approaches like observations, management requests, interviews, job analysis, questionnaire surveys, performance ratings and tests.United States of America which is considered to be very innovative in the field of training and development, surprisingly lacked sound information concerning U.S. employee training not to talk of the specific areas of needs assessment till the publication of the study report in 1986 conducted by Opinion Research Corporation and sponsored by the American Society for Training and Development.The report has indicated that although the first step approach to training programs is a formal needs assessment, about half the training executives indicated that this step is not taken most or all of the time. The frequency of needs assessments declines with the job level for which the training program is being developed. When needs assessments are done, the most frequent methods include personal interviews (83%), direct observation of the work of the prospective trainee group(80%),examination of performance or productivity measures (75%), questionnaires (66%) and task analysis (64%).Like United states of America, there is a lack of information about training scenario in India in general and assessment of training needs in particular except sporadic writings and studies conducted by some training institutions.In general, the private sector organizations gave more weightage to different methods of training needs identification such as job analysis and job specification, performance appraisal b y others and by the individual himself and growth potential of the individual.,

Realizing the drawbacks in the cooperative training system, National Cooperative Development Corporation, a development financial institution gave a new dimension and direction to cooperative education and training. It established its own training centre called as Topic Training Centre which started functioning following systems approach to training in 1985. It regards identification of training needs as specialized and prime task and follows a well defined system. Due to systematic and accurate assessment of training needs, its training programs are highly specialized, effective and result – oriented.

Approaches To Identify Training NeedsThere are two approaches to identify training needs accurately for different categories of personnel particularly managerial personnel – role analysis and job analysis.Role may be defined as a set of various expected behavior of a person occupying a leadership position in an organization. It is highly personal and dynamic.Job may be defined as a set of tasks assigned to a position. Organization expects at least a fair behavior from the position holder in line with the assigned job and tasks. Job, thus, is static and impersonal.In order to identify training needs, role and job descriptions are prepared along with their specifications following organizational analysis. Individuals capabilities and qualities occupying the position are analyzed in light of role and job specifications to find out performance deficiencies constituting training needs.Thus, systematic and accurate identification of training needs involve three processes –organizational analysis, role and job analysis and individual analysis.Organizational AnalysisIt is the process of studying, collecting information and analyzing the state of affairs of an organization, its working in light of its objectives, performance of different category of personnel ranging from top management to bottom level employees and recording performance problems associated with job category. Since this effort involves huge paper work and resources in terms of cost, time and energy of training specialist and the organization in question, it should be used with discretion and differentially depending on the occasions when training needs have to be identified either at the level of a section or a department or the whole organization.



Need to Conduct Organizational AnalysisMost of the organizations in India do not have systematic approach towards training for their managerial personnel. They have two alternatives for management training: in- service training department and off-the-job.Many organizations maintain training departments which usually cater to the training needs of in-service people specially of lower level employees and supervisory personnel. Middle and top managerial personnel are sent to off-the-job training in some specialized training institutions. Organizational analysis serves useful purposes in both the systems of training.Present and Future Training PlanUnderstanding about working of the organization and discussion with its managerial personnel help the trainer to appreciate the strengths, opportunities, weaknesses and threats of the organization, its development plans, investment programs, technological progress, products or services planned, competitors condition, manpower planning etc. The knowledge of present and future scenario serves as a base to identify different types of training needs and a trainer can thus suggest to the management a training plan covering present and future training requirements of the organization.

Organizational analysis helps a trainer to analyze various training needs and decide about the status of training whether they are displaced, inadequate or partial training needs. Further it also helps to decide who should be trained and what type of training intervention is required. Thus, a total picture of training situation is obtained.

Methods to Conduct Organizational AnalysisOrganizational Analysis is basically a process of studying and collection of information on various aspects related to organization’s functioning and to find its training and development needs. Normally, three methods are used for the analysis such as observation, discussion or interview and reference to documented information. Since the analysis will provide objective and professional advice on human resource development to the top management enabling them to decide the nature and extent of the role of traini8ng in accomplishing organizational objective in light of training investment, it is pertinent that the organization study and analysis should be conducted systematically.

Organizational analysis, thus reveals a variety of training needs at organizational level, at the job level or at the individual employees level from the point of view of its objectives.Harmonizing training needs are perceived by an individual employee with that of organization is very significant and a judicious synthesis between the two is beneficial for organization and employees. The revelation of various types of job – oriented or job- specific training needs raises several issues needing special attention such as visualizing standard level of knowledge, skills, attitudes, and interpersonal skills requirement of each job and existing competence in terms of knowledge, skill and attitude, etc., of an employee against the standard requirement. These issues may be considered by using job analysis technique. Job analysis is used for a variety of purposes, its use is basically for the purpose of identifying job oriented training needs.

Job-Oriented Training Analysis (JOTA)Job analysis is the process of studying, collecting information on and examining various aspects of a job. It produces two important documents: job description and job specification.

Job DescriptionJD is a statement of duties and responsibilities assigned to a position. Hence, all the job of a position are described and further each job is broken down into independent responsibilities, duties or tasks.

Job SpecificationIt is an organized statement of human qualities required to perform a job effectively. Therefore, each task of a job is examined to visualize requirements of human qualities for its effective performance. It is developed by considering the nature and characteristics of tasks involved in a job.

Individual AnalysisAn organization employs various personnel to perform various jobs and tasks considering their competence in terms of knowledge, skill and attitude while selecting them. But due to several reasons individual competency may fall short of the standard competence required to perform the job. In order to ascertain the extent of gap of competence, the existing competence of persons is analyzed by using the technique of individual analysis. This technique denotes recording the existing or actual level of qualities in the man



performing those jobs in terms of knowledge, skill and attitude. It starts with analyzing person’s educational qualification, professional competence, experience, training etc. followed by other techniques.

Prioritizing Training NeedsThe training needs identified through the above stages are prioritized with reference to importance and relevance in consultation with the persons concerned, their superior and top management and a final training needs are prepared.

Identifying Training NeedsIdentification of training needs is the stepping stone in the appropriate management training system on which the entire edifice of a training program is built.Training need are related to performance of the organization as a whole. Training need identification is done to find out whether the organization is meeting its current performance standards and objectives. Training need identification focus is on needs as opposed to desires of the employees for a constructive outcome. A training need can be described as a specific skill which an individual must acquire in order to perform a task efficiently and effectively. It is process of assessing and finding the gap between the standard competence required in a job and the existing competence in terms of vital knowledge, skill and attitude in the job holder.

Importance of TNI• To pinpoint if training will make a difference in

productivity and the bottom line.

• To decide what specific training each employee needs and what will improve his or her job performance.

• To differentiate between the need for training and organizational issues in order to bring a match between individual aspirations and organizational goals.

Purpose of TNI• To identify areas of particular strength or weakness

in the organisation• To develop plan to build on strength and reduce

weaknesses i.e. improve performance.• To enable priorities for action to be established• To plan the most effective development of limited

resources, for instance, to ensure cost effectiveness and value for money.

• To justify investment in training by showing how training will contribute to achieving corporate objectives.

• To provide a basis for integrating training into the business by getting line management involvement and commitment.

Objectives of TNI• To determine whether training is needed.• To determine causes of poor performance.• To determine desired training and development

outcomes.• To provide basis of measurement.• To gain management support.

Methods of TNI1.Stakeholder Analysis(groups, individuals, organizations who have a claim, gain or benefit, or who feel they should have some ownership of a process, program or project)It clearly indicates how important it is to identify all possible stakeholders with an interest in the training process, including the identification and assessment of the training needs. Stakeholder analysis in the context of needs assessment will reveal the importance and possible influence of the stakeholders in TNA Training Needs Analysis, their type of participation, interest and possible impact on them.

2. Selecting and using the research methods to identify training needsIdentifying training needs is a form of research. First, evidence suggests that there is a basic problem which can be addressed through training. It may also be necessary to address the problem with non-training measures. So it is important to identify clearly the training gap. This is found by comparing an existing situation with a future, desirable situation, and then finding out how training can bring us from here to there.

3. Planning identification of organizational needsThere are two steps:�List organizations with a stake in the training.� List questions to ask them, e.g. what are the critical changes affecting the work and operations of the organization? What are the relevant policies within the organization? What are the current strengths and weaknesses of the organization? What opportunities and threats are being presented from the external environment? Etc.



4.Planning identification of job needsThis should be carried out using a participatory methodology, ideally with the trainees themselves, prior to the training, or with other stakeholders who are able to provide good quality information - a source of data or sensory input, organized or arranged into a pattern which can be interpreted. about the professional activities of the target group.

5. Planning identification of individual needsHere, it is important to estimate the training needs of individuals, by preparing a variety of questions to them, such as the following:

• What tasks do you do regularly?• What difficulties do you face when doing these

tasks or your job?• What could help you to do your job better?• What kinds of knowledge do you need to do your

job?• What skills do you need to do your job?• What kinds of attitudes do you need to do your job

effectively?• Which of these KSA do you lack now?• How long have you worked in this job?• What do you like most about your job?• What do you like least?• What would you like to change about your job?• Do you think you are doing a good job?• How do you know if you are doing a good job?

6. Data collectionThis is the point when plans become action. If the planning has been effective, then the data collection should go smoothly, but always expect the unexpected. Flexibility, commitment, energy, organization and a sense of humor will all be needed during this period.

7. Analysis of the data collected in the TNAIt is important that data is sorted out as the TNA survey progresses. This has two advantages. Firstly, it will not be necessary to fight with a huge amount of data at the end of the survey. Secondly, there will be a better understanding of important issues emerging during the survey, which may be explored in more detail or clarified in some way. Identify categories into which data can be inserted.The knowledge skills and attitudes (KSA) identified will form the basis of the curricula to be developed. Once these KSA have been identified, it will be necessary to prioritize which training programs can or should be offered, and when.

It is therefore important to develop a training strategy once the results of the TNA are known. For any training course/programme developed, there should be clear evidence which justifies its provision. The information collected in the survey which is not directly related to KSA will provide this evidence, and can be useful when developing a training strategy. It can also be useful in identifying non-training needs. Training is not the answer to every problem!

8. Presenting the TNA dataThere are different ways in which the data could be presented.• Target Group• Existing KSA – 1,2,3, etc• Future Required KSA - 1,2,3, etc

9. Reporting the dataIt is essential to prepare a report of the initial consolidated results of the TNA. This could be organized under the following headings:� Policy.� Environment.� Client organizational issues.� Tasks and activities (existing and future).� Training provider organizational issues.� Individual needs.� KSA (existing and future/required).� Curricular which are going to be planned (including time frame, and a rough idea of content).

10. Sharing the resultsAll the stakeholders involved in the TNA should have the opportunity to give feedback on the results. This may be done through a workshop, to which key stakeholders are invited. The results may be presented, and then participants should have the chance to discuss the results in detail, either in small groups or in a plenary session. It is not always possible to invite all stakeholders to a workshop; in this case the results of the TNA should be disseminated in some other ways (for example, the written report). Where data is collected from groups or individuals who cannot participate in a workshop to discuss the findings and have no access to written reports, it is important to provide opportunity for their feedback during the data collection process. A meeting could be held, for example, where the researchers present the findings from the field directly to those who contributed their ideas. This improves the chance for feedback and validation of the results. It also emphasizes the importance for the researchers to sort their data as they proceed with the collection.



Levels of TNI

At the individuals level :Identification begins with the job description. This will provide a list of the skills and knowledge required. It can be compared with the actual skills and knowledge that the jobholder processes. managers will also be able to identify training for their subordinates. One of the best ways of achieving this is through the appraisal interview, where agreed training needs can be identified.

At the organizational level: Training needs may be identified through the performance appraisal system. This may provide the key channel for feeding back individual needs. The information should be processed by human resource development department in order to plan the overall needs of the organization.

At occupational level: Training need at this level refers to training that is required to upgrade certain job functions. It concentrates on skills, knowledge and attitudes necessary to carry out various duties related to a particular job function.

Needs at level of TNI

Needs at Organizational level – Where a general weakness in the organization exists, where training in the organization is most needed.

Needs at Occupational level – What is needed in terms of skills, knowledge and attitude to carry out the various responsibilities related to a particular job.

Needs at Individual level – Determining deficiencies in particular skills, knowledge and attitude on the part of the individual.

The Business Needs Drive Training EvaluationDetermining when a training solution is the right solution to a performance problem is at the very heart of what we do as learning professionals. Being able to clearly articulate the link between business needs, performance gaps and the impact of the learning solution is where we prove our value as professionals to our business partners. But how do we do that?

It Starts with the Business NeedOur process starts with the initial request for training, when a perceived deficiency in performance is identified. By analyzing performance deficiencies around training development, we identified

performance gaps of training professionals in the bank related to developing training solutions to solve specific business goals. Dana Gains and James Robinson developed their well-known Performance Relationship Map (PRM) as a tool for doing up-front analysis. We used the PRM to clearly identify the business need, desired state and performance gaps within a large U.S. bank to develop a blended learning solution later called the Instructional Design Foundations course. With this approach, the desired state of performance was first identified. Then the current state of performance was identified and the difference between these two states, known as the performance gap is identified.The cause of these gaps was first thought to be solely due to knowledge, skill or ability deficits when, in fact, there was a number of contributing factors to the performance deficiencies. Since a training solution is best used to support increased capabilities in knowledge, skill and ability, it had to be determined that there truly was an actual gap in required knowledge, skill or ability of a sufficient number of associates to justify the development of the course. This process and the resulting information helped us obtain senior leader support for the training solution and helped to have clear and measurable outcomes to measure our success against.

Linking Needs, Solutions and EvaluationOnce the business need is properly identified, objectives are developed to be observable, measurable and linked to identify business needs through performance objectives. In our case the need was identified as a lack of knowledge and skills around performance needs assessment, instructional design and training evaluation. Participants’ preferences or learning styles were also considered when developing learning objectives and helped us tailor the e-learning approach to fit associates learning needs that meant anytime, any-place learning from their own desks. This also helped facilitate the transfer of the learning to the actual workplace.We used electronic pre- and post-assessments built around the module objectives for both the associate and the associate’s manager. An electronic reaction survey was also used to capture the associate’s reaction to the course immediately upon finishing the learning modules.To determine behavior change on the job and business impact we structured the post assessments to report use of the knowledge and



skills on the job 90 days after completing the training and asked both the associates and their managers about the business impacts they had achieved in their workplace that could be directly attributed to the training.The graphic above illustrates the link between the needs analysis we completed using the PRM and training evaluation plan we implemented using pre and post assessments.As shown on the left, the value proposition of the business problem begins the process of determining the business need, which in our case was that trainers throughout the bank had varying degrees of knowledge and skill in using performance consulting, instructional design and training evaluation. This helped identify the behaviors required to achieve the desired state. The desired state was that trainers should be better able to link training solutions to business needs and identify the behaviors that needed to be developed to address the business need. Training professionals throughout the bank were not consistently using industry best practices to develop solutions that would enable their business partners to increase efficiencies, decrease time to market and reduce costs.To change behavior, new knowledge and skills needed to be learned and shared across the bank. The Modules within the course were built around useable chunks of information and case studies to drive home knowledge and understanding. To maximize their learning, the preferred learning styles of the associates were addressed by making content bank relevant and accessible from their desks, eliminating time away from work.This also proved to be very popular with both the associates and their managers, which were captured during the post training surveys and follow-up interviews. The link between training objectives and business needs must be considered at the beginning, instead of at the end of a program, when it becomes an after-thought. If we don’t link the solutions to business needs, we risk becoming irrelevant and expendable in the eyes of our business partners. The up-front analysis determines whether a training solution is required, what type of performance solution is best and the level of evaluation suited to the solution and the business needs.

Implications for PractitionersAs training professionals, we must maintain a business focus, analyze performance gaps to be sure that training is the best solution and ensure that any training solution is linked to performance analysis and identified business needs.

The business impact shown to have the greatest positive impacts was quality, efficiency, reduced re-work, and associate satisfaction. The ability to capture this data has created a “chain of impact,” as Jack Phillips refers to it, which has enabled us to calculate a Return-on-Investment (ROI).The ROI was achieved after collecting the fully loaded costs and the benefits of the training solution and inserting the numbers in the ROI formula: ROI = Net Program Benefits/Program Costs X 100 = X % .

Determining Business Needs In Instructional System DesignStart With The End In MindBegin with the end in mind — a learning or performance initiative should be a means to an end. Thus, learning and performance initiatives need to show how they will improve the business or organization, such as increase revenue, reduce costs, or speed up a process. For example, training salespeople in order to reduce the percentage who fail to make sales will both increase revenue through more sales and reduce turnover costs.

Successful training and development → More effective and efficient performance → Improved business outcomes.

Since the flow of causality of the above pathway is from left to right, training must be planned in the opposite direction:

Desired business outcome → Required changes in performance to produce business outcome→ Experiences and skills needed to produce the desired performance.

Thus, designing a viable learning program should proceed in a manner similar to this:

Analysis Phase: Determine the business outcome —How does it link to the business unit(s)?

Design Phase: Determine required changes in the learner's performance.

Development Phase: Create the experiences that will change the learner's performance.



The analysis phase is the foundation of a learning or training process. The deliverables of this phase are the building blocks for all subsequent design and development activities. It accomplishes this by discovering: • Desired business need or result• If a learning process will fulfill the desired business

need• Performance requirements that supports desired

business outcome• What must be learned• The standards of performance• How the learning process will occur• Who needs to improve their performanceThis analysis phase is sometimes called a Front-End Analysis in that while you normally perform some type of analysis throughout the entire ADDIE process, this “front end” of the process is where the main problem identification is performed, such as identifying the problem, analyzing the job and selecting the tasks to train (U.S. Army Field Artillery School, 1984).

Business NeedsTo determine the business need, investigate the problem or performance initiative to see how it supports the mission statement, leader's vision, and/or organizational goals. Fixing a problem or making a process better is just as good as a Return on Investment (ROI), if not better. Organizations that focus strictly on ROI are normally focusing on cost-cutting. And you can only cut costs so much before you start stripping out the core parts of a business. A much better approach is to improve a performance or process that supports a key organization goal, vision, or mission.When senior executives were asked the most important training initiatives, 77% cited, “aligning learning strategies with business goals”; 75% cited, “ensuring learning content meets workforce requirements”; and 72% said, “boosting productivity and agility” (Training Magazine, 2004). Thus, senior leaders are not looking at training to be a profit center (that is what other business units are for), rather they are looking at performance improvement initiatives to help grow the organization so that it can reach its goals and perform its mission.

Identifying and Analyzing TNI linkages with Business NeedsIn order to fully understand all your training requirements, you should:• Look at your business objectives - any training that

contributes to these may be a priority.

• Implement a training programme - identify what you want to achieve, and inform staff of the learning and development plans

• Ask for staff input - find out what training they feel they require, and how should this training be delivered

• Carry out a full skills audit - some staff may have skills that you didn't know about, or that they haven't had a chance to use yet

• Once you have found out which skills you are missing, you can identify what training is required to fill that gap. Remember that training should be applicable to the job, relevant to the person carrying out the job and tied to business objectives.

• Once you have identified the training needs of your staff, you will need to find out the type of training that will best suit your needs.

• People learn in different ways, and you need to make sure you tailor the right training course to the right person to produce the desired outcomes.



Case StudyTony Devlin set up Glenovation in 2002 and is the managing director. Glenovationprovides multimedia, web and database development support and helps companies apply for public sector contracts in the UK and Ireland. Tony employs four full-time members of staff to help meet the demands of his growing company.

What He did?Brushed up his own skillsOnce he made the decision to become self-employed the first skill gaps to address was his own. Whilst he was very comfortable in the dot com world he didn't have previous business start-up experience and had an immediate training requirement in this area. He registered on business start-up programme, which was a great benefit and gave him invaluable skills around marketing the company. This really helped to get the business off to a good start.

He was also afraid to go into situations with large groups of people. With the help of the start-up program he was able to start effective networking.

Sought advice from training expertsAs the business began to grow he started to recruit and currently employ four full-time people. He did an audit of collective strengths and skill gaps which included marketing and promotion of the company. In order to fill up the gaps he brought in

three marketing gurus who specialized in training staff to be more market aware and spot customer opportunities from every situation. The staff responded really well to outside support and he noticed an improvement in their skills, which has also helped improve his overall performance. Other benefits included increased staff confidence which ultimately lead to an improvement in his profit margins.“

Devoted time to staff developmentFor a small business he spend a lot of time on staff development. He organize group workshops every three to four months to encourage and develop teamwork and communication skills. This includes how to deal with customers; how to spot business opportunities; developing brand awareness and becoming more efficient in their job roles. The time invested in this really pays off as it increases staff motivation and has demonstrable benefits on the bottom-line.

What he did differentlyCarry out a staff skills audit and design a training programmeHe ensured members of staff are equipped with the correct skills from the outset. As well as providing high quality training for the team it's important to train yourself. Linking up with at least one business mentor is a great way to learn from other people's challenges and be able to take these forward positively in your own business.



Evaluation The value of learning and development for organizations

The most basic reason for providing learning and development is to ensure that an organization's employees are able to carry out their current role. Some training may be mandatory in relation to health and safety or occupation-specific issues but much of it will be discretionary where organizations appreciate the added value that they will gain from having highly skilled and knowledgeable employees.

Organizations which are keen to improve their productivity, efficiency and profitability will look to move beyond mandatory training and look at more diverse learning and development activities which will enable the employees to maximize their potential and provide a valuable resource for the organization. Learning and development can be a source of competitive advantage where employees gain appropriate new knowledge and skills (Towler and Dipboye, 2009) which provides a strong argument for organizations to invest in their employees so that they can reap the benefits and differentiate themselves from their competitors.

However, provision of learning and development opportunities alone do not mean that an organization will be more productive and effective, there are many more steps needed. Firstly the opportunities need to be appropriate in terms of content and the way that it is delivered so that they will add value to the employees and the overall organization. As well as looking at appropriateness from an organizational perspective it is also necessary to consider the fit with teams in the organization and individual employees to ensure that their needs can adequately be met.

In addition the learning and development activities need to be delivered in such a way that practical benefits to the workplace can be observed and to enable the employees to be able to transfer their new knowledge and skills to the benefit of all of the key stakeholders.

Chiaburu and Lindsay (2008:199) comments:Training programmes are effective only to the extent that the skills and behaviors learned and practiced during instruction are actually transferred to the workplace.

DevelopmentOf Learning and



In addition many organizations recognize the value of providing suitable learning and development provision where it supports accreditations such as Investors in People which may be highly desirable. With heightened awareness of corporate and social responsibility, different stakeholders may have an interest in the learning and development opportunities offered to employees.

Prospective employees may also be enticed through the provision of a good range of learning and development opportunities as a way of differentiating employers who offer similar salaries. As well as the sometimes tangible benefits of the training this commitment to employees sends out a message that employees are valued and supported within the organization.

It is increasingly recognized that employees need to be actively aware of an organization's strategies and objectives and the provision of learning and development opportunities can help to cement this where clear links are drawn between the learning intervention and how it is aligned with the overall strategy.

Felstead et al (1997:v) also emphasize “mouldingattitudes and generating enthusiasm for corporate objectives”

What do we mean by evaluation of learning and development?

Training evaluation can be described as a systematic process of collecting and analyzing information for and about a training program which can be used for planning and guiding decision making as well as assessing the relevance, effectiveness, and the impact of various training components (Raab et al., 1991). This demonstrates the wide range of activities that are associated with it as well as the many benefits.

James and Roffe (2000:12) provide a simplified explanation of evaluation: “comparing the actual and real with the predicted or promised” which emphasizes the need to reflect on what was achieved in comparison to what was hoped for. This definition also highlights the potential subjectiveness of evaluation for different individuals are likely to have diverse expectations and their review of the event(s) may also differ depending on a wide range of variables, for example; the learning interventions

may be heavily focused towards one particular learning style. Dawson (1995:3) provides another explanation which suggests that the process should be impartial and devoid of subjective opinion:

It is useful at this point to consider just what evaluation of training really does mean. The evaluation of training is the systematic and impartial collection of data for managers and all other interested parties. This information equips them to draw conclusions about the effectiveness of particular training measures as a way of achieving organizational objectives, implementing policy and promoting organizational learning .

Dawson’s (1995) definition is useful in illustrating the wide range of issues that evaluations are intended to capture as well as the assortment of reasons why evaluations might be necessary. It is therefore clear that organisations will need to have clarity in their rationale for evaluating and they need to ensure that this is shared with all of the relevant stakeholders. Failure to do this may lead to a mismatch in the data provided, thereby providing limited use to the organisation and/or the trainer(s).

The importance of evaluating learning and development activities

Mann (1996:14) states “With the huge investment in developing training strategies, the question is no longer “should we train” but rather “is the training worthwhile and effective?”. This is consistent with a wide range of literature which accepts that training is important and recognizes the evaluation of it to be a key issue so that its ‘worth’ can be proven.

Lewis and Thornhill (1994:25) state “there seems to be widespread agreement with the proposition that evaluation is the least well conducted aspect of all training activities” which suggests that there is a need for organizations to focus on this area of the learning and development process in order for evaluation to reach its potential. This attitude was confirmed in the CIPD (2008) Learning and Development survey whereby only one in five (21%) of the respondents said that they spend most of their time monitoring or evaluating training at the moment. Athanasaou(1998) indicates similar results when research was undertaken in Australia.



The importance of the process can be viewed from the perspectives of different stakeholders:

To the organization

There are a number of reasons why organizations should evaluate their learning and development activities and this includes the following: • To help make decisions about what interventions

should (or should not) be replicated in the future. • To provide evidence of investment in their

[organizations] human capital.• To demonstrate the value that interventions bring

to the organization (Burkett, 2005a).• To enable training departments to justify their

expenditure, particularly where there is an increased focus on spending less whilst generating more outcomes (Goldwasser, 2001).

• As part of business efficiency considerations (The Industrial Society, 1998)

• To reinforce the importance of an evaluation process when trialing new programmes for employees (Russ –Eft and Preskill, 2008).

• To assist in determining who should attend training programmes (Mann, 1996).

• To allow the organization to identify whether there are better ways to achieve things rather than through the provision of formal learning and development interventions i.e. job aids which are more effective (Simpson, 2002)

Mann (1996) identifies beneficial legal implications of having effective evaluation data so that if necessary, organizations can demonstrate the job relatedness of training programmes (the work of these authors is set in the American context).

APSC (2005:2) suggest that evaluation is even more important in the public sector and state: Effective evaluation is part of an accountable, professional and ethical public service. It is fundamental to good governance, good practice and good management.

This suggests that public sector organizations need to pay particular attention to this issue so that they can demonstrate the values and benefits that they accrue from their investment into training and development. In the current economic and political climate where there is a strong focus on public sector expenditure this area of spending is likely to be particularly important. In addition a comprehensive evaluation process may lead to cost savings and support the requirement from Gershon (2004) to reduce public sector expenditure.

To individuals

Evaluation provides individual learners with the opportunity to give feedback to their trainers; perhaps this is most useful when it is gathered early on in the process so that they can benefit from any resultant adjustments.

Individuals in subsequent cohorts can also benefit from the evaluation process if feedback is acted upon for the benefit of the programme. This relies on individual learners taking the time to actively engage in the evaluation process and to provide honest feedback in the most relevant areas. Learners may also be able to benefit from the evaluation process if they are members of particular professional bodies which require reflective evidence of continuing professional development (CPD).

To trainers/facilitators

Evaluation data may be used as a performance indicator which justifies the existence of a training department and/or investment in trainers. Independent trainers may also depend on their feedback to gain future business and/or to engage with potential new clients as an indicator of the quality of their provision and delivery. Trainers or facilitators can also benefit from evaluations so that they can update or amend it accordingly; this may be particularly useful if they are working in specific sectors or types of organization.



Importance of evaluation

In many countries Governments may also play a significant role in the expansion of learning and development activities either through the delivery of programmes or the provision of free or subsidized funding for industries or organizations. Where this is the case providers are likely to have to prove the value that they add in a more transparent way (Massey, 2004).

Purpose of evaluation

APSC (2005) provide a simple overview of the purpose of evaluation and state that it has the following four objectives:

I. Assess if intended learning and development objectives have been met

II. Continuous improvement of learning and development

III. Assess whether resources are used wisely IV. Assess the value for money of the learning and

development

From these four options organizations may decide to address all of them or concentrate on some of them. For example, many organizations do not have the resources to be able to assess the value for money of the learning and development so may focus more on the other three objectives. Other issues that may impact on the extent to which programmes are evaluated are:

• The size of the organization • The availability of expertise to generate evaluation

frameworks and subsequently analyze the data provided by learners

• The objectives of specific training programmes – in some cases the focus may be almost exclusively be on ensuring that the program-specific objectives have been met.

The original four objectives shown above were later expanded by APSC (2005) to include the following purposes:

• To Determine success• Assess if objectives have been achieved • Make improvements • Ensure quality • Ensure accountability • Meet external requirements • Account for activity• Assess value or merit • Assess risk • Justify investments • Facilitate decisions whether to

continue/discontinue the activity • Ensure appropriateness and alignment • Identify strengths and weaknesses

Clegg (1987) believes that a key purpose could be to establish guidelines for future programmes by building upon strengths and identifying areas for development. It could also be argued that the evaluation process could also be used to review diversity related issues as Schmidt (2009) believes that a major challenge for training practitioners is being able to meet the needs of diverse groups.

This section of the review has highlighted the many different reasons why organizations use evaluation as part of their processes and it is likely that organizations will vary significantly in the reasons why they use them but they must have a clear understanding of their purpose so that they can collect the necessary information in an appropriate way. It is important that this purpose is also communicated to all of the relevant stakeholders, particularly the trainers and facilitators so that they can ensure that appropriate information is collated.



Linking learning, development and evaluation to business strategy

Learning and training can provide a pivotal link between an organization's human resource strategy and overall business strategy (Mabey, Salaman, and Storey, 1998) by ensuring that the organization's employees have the relevant skills and knowledge needed to be able to execute the HR strategy. This should also give the HR strategy an improved chance of success if they have a clear understanding of capabilities and whether investment and development is required. Smith and Smith (2007) uncovered evidence that more and more employers are demanding that training is deliberately aligned with the strategic needs of the organization.

It is clear from a range of papers that researchers have identified the importance of linking learning with organizations' overall strategies and business objectives, particularly where their people are one of their sources of competitive advantage. This means that those responsible for learning, development and evaluation have to have a good understanding of the strategy and objectives and discuss the key issues with a range of stakeholders.

The extent to which learning and development can be linked to the strategy depends on how close an alignment exists, for example; whether or not learning is explicitly referred to within the strategic planning document and whether there is ongoing communication and links between the learning and development function and the senior managers (Anderson, 2009b).

Anderson (2009a) believes that the alignment of HRD to organizational strategy remains problematic. This concern was also identified in the earlier findings of CIPD (2007) where only one third of their survey participants felt that learning and development implications are considered when organizational strategy is constructed.

As strategies are updated or refocused it will be necessary to review the learning and development process and therefore an ongoing dialogue is needed between those responsible for learning and development and senior managers. This section suggests that whilst alignment with wider HR and organizational strategies are needed there is insufficient evidence of this in operation.

Models of evaluation

The Kirkpatrick modelIn the 1960’s Donald Kirkpatrick wrote a series of articles on evaluation where he identified four stages (or levels of evaluation). Despite its age, Kirkpatrick’s model continues to be used in contemporary research (Schmidt et al, 2009 and Elliott et al, 2009). Kirkpatrick (1977:9) divided the evaluation process into four segments or stages.

Stage1: ReactionHow do the participants feel about the program they attended? To what extent are they ‘satisfied customers’?

Stage2: LearningTo what extent have the trainees learned the information and skills? To what extent have their attitudes changed?

Stage3: BehaviorTo what extent has their job behavior changed as a result of attending the training program?

Stage4: ResultsTo what extent have results been affected by the training program? (Results would include such factors as profits, return-on-investment, sales, production quality, quantity, schedules being met, costs, safety record, absenteeism, turnover, grievances and morale).

Moving beyond Kirkpatrick

Phillips (2002) cited in Burkett (2005b) suggested the addition of a fifth stage on return on investment (ROI) to measure the monetary costs of providing the intervention with the monetary benefits received. This concept of ROI has become an area of increasing interest and is discussed in more detail later in this review. The CIRO model focuses on three questions as shown below in Figure 2 (cited in Elliott et al., 2009) and the main difference from Kirkpatrick’s model in that it focuses on measurements taken before and after the training has been carried out. Perhaps a key strength of this model is that it takes into account the resources of the organization (within the input measure) and their objectives (within the context measure) thereby responding to some of the criticisms of Kirkpatrick.



CIRO model of evaluation

Context – what needs to be addressed? Input – what is likely to bring about the changes? Reaction – how did the learners react to the training? Outcome – what are immediate, intermediate and ultimate outcomes?

Brinkerhoff (2006:302) gives examples of organizations that deliver exactly the same training but that have very different levels of impact and suggests that this can be explained by: “The answer lies in the way that companies make use of training, the way it is organized, planned, implemented, and above all, followed up and supported”.

The role of training needs analysisIt is clear from all of the literature on learning and development that an effective system has to begin with a comprehensive training needs analysis process so that the organization, departments and individuals can identify their specific needs. Through the careful identification of these needs the organization can identify where gaps exist in skills and knowledge and offer appropriate interventions to try to address these gaps.

The process of training needs analysis can take a variety of forms; at an individual level it may be discussed as part of the performance management process; it may be a discrete activity in its own right or they may be driven from other organizational data. The increasing use of staff surveys is an example of using data to identify specific areas requiring development. Throughout the training needs analysis process attention should also be paid to the linkage with the overall organizational strategy and goals to ensure that there is consistency and that learning interventions have the potential to have an impact. A critical success factor for effective training needs analysis is the need for learners to be honest (Tennant et al., 2002) otherwise the learning and development opportunities offered will not be as effective.

Naturally the training needs analysis process has a close link with the evaluation of learning and development as the objectives of the programmes/interventions should be considered as part of the training needs analysis process.

What should we measure?

It is clear from the literature base that measuring satisfaction with the programme is important; however this should not be the sole focus. Learners may enjoy attending a programme but not actually learn anything/enough to have an impact on their performance in the workplace which minimizes the value of the programme.

To a certain extent many of the evaluation criteria may be quite specific to individual programmes and their contexts as well as the purpose of the evaluation. It is important that the key stakeholders agree on the evaluation criteria to ensure that the data will meet all of their needs. There is a need to include a range of stakeholders in making the decision on what should be evaluated, including business unit managers (Goldwasser, 2001). There may be some core evaluation criteria which should be collected from each program but these could be weighted differently depending on specific programmes.

As well as looking at skills and knowledge it will often be appropriate to measure behaviors. Of further interest is to look at behavior both within participation in the intervention and when they return to the workplace (Mann, 1996).

Results from the survey undertaken by Industrial Society (1998) suggest that management training is the most likely to be evaluated followed by technical and IT training and those focusing on developing interpersonal skills. This may be because these are the most frequently delivered programmes.

It is therefore important to make clear judgments on what should be measured, Ellis (2005) refers to this process as calculating the ROI on ROI. The significant work of Philips recommends that only 5-10% of training interventions should be evaluated at this level which means that organizations need to give careful consideration to which programmes would benefit the most from this in-depth approach.



Return On Investment

There is a growing need for HR professionals to be able to provide performance measures for the activities that they carry out (Mulvaney et al, 2006) and particularly for those where there are direct cost implications, for example; the provision of learning and development interventions. Specifically they need to provide the bottom line impact of training programmes. Expressed in simple terms; ROI allows organisations to identify the impact of a specific investment on the bottom line (Simpson, 2002). Despite the fact that there is widespread appreciation of the value of using ROI, few organisations do so (Hall, 2003).

Rowden (2005:31) provides a particularly useful introduction which emphasises one of the challenges of evaluating learning and development activities:

“The ‘bean-counters’ in the organization are likely to know exactly how much training costs but they may have little idea of its value. HR must be able to supply that information if it is to truly become a strategic part of the organization”.

This infers that HR have significant role to play in changing the perceptions of some about the value of training and ensuring that supporting data (whether qualitative or quantitative) can be provided. Management need to be clear about what they want from their training and development (Rowden, 2005) so that appropriate investment measures can be used though Giangreco (2009) argues that it would be too simplistic to think that simple linear relationships can be drawn from attendance at training and later results. Hick (2000) believes that return on investment is important for six main reasons .

The importance of return-on-investment

• HRD budgets have grown and so has accountability.

• Total quality measurement and continuous process improvement has put a bigger focus on measurement.

• Focus on bottom line due to its outsourcing; restructuring this places a link on training meeting business needs (strategic)

• Economic issues • Must show contribution to business• Increased awareness that training can be

measured.

Organizations and trainers may have reservations about attempting to calculate return on investment (ROI) due to concerns about the cost, time and human resources that would be needed to carry it out with confidence (Burkett, 2005a). It is therefore necessary for them to generate appropriate criteria to determine which programmes will be evaluated at this level. Ellis (2005) provides an interesting quote: “bottom line; it’s better to measure strategically than constantly” which reinforces the need to make decisions about which type and level of programmes need to have their ROI measured.

In response to the costs associated with calculating ROI on programmes (assuming that an organization has a large portfolio of training interventions that they offer) Burkett (2005a:12) identifies ten cost saving approaches to ROI (provided below in Figure 7) which should reduce the costs involved in undertaking the ROI process.

Philips (1996) suggests that only 10% of interventions should be explored at ROI level and Ellis (2005) defines specific criteria to identify the most appropriate interventions to be calculated at the ROI level:

• High level of visibility

• Those with strong management interest

• Those with strong ties to company objectives

A wider advantage of the return on investment approach is that it allows the HR team to speak the same financial language as the management team and increased credibility and input into strategic decisions (Purcell, 2000). As ROI is particularly expensive to carry out and the literature suggests that it should only be used on a small number of projects it may be useful to also use return-on-expectations.



Challenges in undertaking evaluation

Dawson (1995) identified a range of reasons why organizations may not systematically carry out evaluations of their provision, including; cost, lethargy, fear of criticism and difficulty of measurement.

From the earlier work of Clegg (1987:67) we can also add ‘lack of adequate evaluation methodology’ and ‘lack of expertise’ to the list. Measurement will be critically important and James and Roffe(2000:13) identify a key challenge as being potential “opaqueness in the measurement criteria”.

Lee and Pershing (2002:184) undertook research with SMEs that looked specifically at the variables needed to construct a useful reaction-based questionnaire which they termed ‘reactionnaire’; this builds on one of the key challenges perceived in using ‘happy sheets’ by designing them in a much more constructive manner.

A key challenge for both organisations and academics is the extent to which the training alone is responsible for any changes? For example, does on the job activities or less formal support may also have an impact. This should be considered when designing the evaluation process. They also need to consider the type of culture that exists in the organization and any possible outcomes on the success of learning and development interventions and evaluation activities.

Buckley and Caple (1990) warn against the situation where an organization invests more into the evaluation process than the cost of the initial reason why the training was needed. This signifies the need to carefully consider the costs and benefits associated with it.

Dawson (1995:3) comments “using such a document [training evaluation form] has become a ritual, unfortunately rituals do not produce fresh data despite their pungent symbolism” which emphasizes the need to consider more diverse methods of evaluation so that more useful and meaningful data can be produced and analyzed.

A final area for consideration occurs where interventions take place over an extended period of time. This raises a question of whether each element should be evaluated individually or should it take place right at the end (where learners may focus on the most recent interventions).

Maximizing the effectiveness of the evaluation process

APSC (2005) identified through their research a range of conditions that need to be in place to maximisethe effectiveness of interventions. One of the more innovative recommendations is that the evaluation feedback needs to be presented in such a way that it will be engaging to senior managers (as well as involving them in the process where appropriate). It is also important that the learners can see how their feedback has been taken onboard as this may motivate them to undertake detailed future evaluations.

Gianfranco et al (2009) suggest four areas that should be considered when revising evaluation strategies; particularly where a diverse range of provision exists:

1. Size of the company

2. Type of audience

3. Contents of training

4. Legal framework

Learning transfer

One of the key criticisms of Kirkpatrick’s model (and some of the subsequent iterations) is that it fails to address the vital area of transferring knowledge into the workplace. As the key purpose of learning and development activities is to have a positive impact on the workplace it is a surprising omission in many of the models.

The need to be able to transfer learning into the workplace in the form of increased performance is of critical importance to organizations as until this happens the organization does not ‘see’ the benefits of their investment into the training of their employees. This is another complex issue and one that is explored in detail in the extensive literature review compiled by Cheng and Hampson (2008:336) who conclude: “We have reviewed the existing literature on transfer of training and found that there are inconsistent and puzzling findings in the empirical research”.



Evaluation as an ongoing HR prioritySystematic training evaluation is widely understood to be important (i.e. 98% of respondents to the Industrial Society, 2001 research) and with more organizations identifying an increased commitment to it (Industrial Society, 2001) it is a key contemporary issue. There are no easy to cures to this problem and isolating the effect of the training is likely to remain an ongoing issue (Rowden, 2005). It is clear from this review that evaluation should be built into the design process to ensure that it will cover the most appropriate areas and that they will be realistic.

Current economic climate In the current economic climate it is clear that there will be an increased focus on the impact of training on the bottom line. However, it is also apparent that this will be at different levels and that it is neither achievable nor appropriate to calculate the ROI on all interventions. Organizations need to carefully consider where their resources would be best placed to collect ROI data and this should be consistent with their overall evaluation strategy.

The continuum of learning interventions There will always be a long continuum of the level of learning interventions from basic knowledge acquisition to more complex concepts which reinforces the need for a range of tools that can be used to capture evaluation data from different interventions. Evaluation tools will also need to be designed so that they meet the requirements of senior managers as this is not always the case (Short, 2009). From a practical perspective it is clear that the cost of the tools should not exceed their value (Ammons and Niedzielski-Eichner, 1985) so this must be considered in the design phase. Whilst one of the reasons why organizations fail to conduct thorough evaluations is cost there is no evidence in the literature that organizations actually know how much the evaluation process costs; this is an area which could be investigated in more detail.

A supportive culture It is important to create a climate where evaluation is seen as a development tool – not labeling programmes as good or bad (Mann, 1996). Where evaluation of learning is a standing item in relevant agendas (APSC, 2005) and is shared with relevant stakeholders this sends a strong message to managers and employees.

Top managers need to champion evaluation tools and lead by example (Tennant et al., 2002). The learning environment also needs to be conducive to a positive learning experience (Chappell, 2008) as does the organizational structure (Bramley, 1999).

ConclusionEvaluation should become built into the training process; something that takes place after all learning interventions however this does not mean that a one size fits all approach is appropriate. This is also partly a cultural issue – managers and employees need to accept that this is an important stage in the overall learning and development provision. Caution appears to be advised by Giangreco et al. (2009) against over-emphasizing the evaluative aspects in case they impact on the training intervention. This suggests that the evaluation process should be carefully introduced within learning interventions so they do not appear to be the main focus and distract from the content of the programs.



Kirkpatrick’s Four Level Of EvaluationAssessing Training Effectiveness often entails using the four level model developed by Donald Kirkpatrick (1994). According to this model, evaluation should always begin with level one , and then, as time and budget allows, should move sequentially through levels two, three, and four. Information from each prior level serves as a base for the next level's evaluation. Thus, each successive level represents a more precise measure of the effectiveness of the training program, but at the same time requires a more rigorous and time-consuming analysis.

Level 1 Evaluation - Reactions Evaluation at this level measures how participants in a training program react to it. It attempts to answer questions regarding the participants' perceptions -Did they like it? Was the material relevant to their work? This type of evaluation is often called a “smile sheet.” According to Kirkpatrick, every program should at least be evaluated at this level to provide for the improvement of a training program. In addition, the participants' reactions have important consequences for learning (level two). Although a positive reaction does not guarantee learning, a negative reaction almost certainly reduces its possibility.Evaluation tools: Program evaluation sheets • Face-to-face interviews• Participant comments throughout the training• Ability of the course to maintain interest• Amount and appropriateness of interactive

exercises• Ease of navigation in Web-based and computer-

based training • Participants' perceived value and transferability to

the workplace

Level 2 Evaluation – LearningTo assess the amount of learning that has occurred due to a training program, level two evaluations often use tests conducted before training (pretest) and after training (post test). Assessing at this level moves the evaluation beyond learner satisfaction and attempts to assess the extent students have advanced in skills, knowledge, or attitude. Measurement at this level is more difficult and laborious than level one. Methods range from formal to informal testing to team assessment and self-assessment. If possible, participants take the test or assessment before the training (pretest) and after training (post test) to determine the amount of learning that has occurred.

Kirkpatrick

evaluation4-Level



Evaluation tools: Individual pre- and post-training tests for comparisons • Assessment of action based learning such as work-

based projects and role-plays • Observations and feedback by peers, managers and

instructors

Level 3 Evaluation – TransferThis level measures the transfer that has occurred in learners' behavior due to the training program. Evaluating at this level attempts to answer the question - Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner? For many trainers this level represents the assessment of a program's effectiveness. However, measuring at this level difficult as it is often impossible to predict when the change in behavior will occur, and thus requires important decisions in terms of when to evaluate, how often to evaluate, and how to evaluate.

Evaluation tools: Individual pre- and post-training tests or surveys• Face-to-face interviews • Observations and feedback from others • Focus groups to gather information and share

knowledge

Level 4 Evaluation- ResultsLevel four evaluation attempts to assess training in terms of business results. In this case, sales transactions improved steadily after training for sales staff occurred in April 1997. Frequently thought of as the bottom line, this level measures the success of the program in terms that managers and executives can understand - increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment. From a business and organizational perspective, this is the overall reason for a training program, yet level four results are not typically addressed. Determining results in financial terms is difficult to measure, and is hard to link directly with training.

How to Apply this Model

Level 1: ReactionStart by identifying how you'll measure reaction. Consider addressing these questions:• Did the trainees feel that the training was worth

their time?• Did they think that it was successful?• What were the biggest strengths of the training,

and the biggest weaknesses?• Did they like the venue and presentation style?• Did the training session accommodate their

personal learning styles ?Next, identify how you want to measure these reactions. To do this you'll typically use employeesatisfaction surveys or questionnaires; however you can also watch trainees' body language during the training, and get verbal feedback by asking trainees directly about their experience.Once you've gathered this information, look at it carefully. Then, think about what changes you could make, based on your trainees' feedback and suggestions.

Level 2: LearningTo measure learning, start by identifying what you want to evaluate. (These things could be changes in knowledge, skills, or attitudes.)It's often helpful to measure these areas both before and after training. So, before training commences, test your trainees to determine their knowledge, skill levels, and attitudes.Once training is finished, test your trainees a second time to measure what they have learned, or measure learning with interviews or verbal assessments.

Level 3: BehaviorIt can be challenging to measure behavior effectively. This is a longer-term activity that should take place weeks or months after the initial training.Consider these questions:• Did the trainees put any of their learning to use?• Are trainees able to teach their new knowledge,

skills, or attitudes to other people?• Are trainees aware that they've changed their

behavior?One of the best ways to measure behavior is to conduct observations and interviews over time.Also, keep in mind that behavior will only change if conditions are favorable. For instance, effective learning could have taken place in the training session. But, if the overall organizational culture isn't set up for any behavior changes, the trainees might not be able to apply what they've learned.

http://www.mindtools.com/pages/article/newTMM_35.htm



Alternatively, trainees might not receive support, recognition, or reward for their behavior change from their boss. So, over time, they disregard the skills or knowledge that they have learned, and go back to their old behaviors.

Level 4: ResultsOf all the levels, measuring the final results of the training is likely to be the most costly and time consuming. The biggest challenges are identifying which outcomes, benefits, or final results are most closely linked to the training, and coming up with an effective way to measure these outcomes over the long term.Here are some outcomes to consider, depending on the objectives of your training:• Increased employee retention.• Increased production.• Higher morale.• Reduced waste.• Increased sales.• Higher quality ratings.• Increased customer satisfaction.• Fewer staff complaints.

Return (Benefit) factors • Reaction and plan

• What is the immediate reaction as to whether the training/learning has been, or is likely to be, useful

• How that will translate into immediate / short-term identification of how the learning will be applied to the job

• What are the immediate expectations of how the learning will positively affect the individual‟s performance

• Measuring Learning • How has the learning affected the

individual‟s skills and knowledge level required to perform effectively; what is the expected change in performance levels and productivity?

• Has the learning positively or negatively affected the individual‟s attitude to their work?

• Assessing application to the job • How has the training actually changed

the individual‟s previous job performance against measured performance standards and criteria?

• Has the individual developed an ability to perform new tasks,?

• Has the individual been able to save costs which would have been incurred without the training?

• Business results (benefits and cost savings) • What is the actual numerical change in

the measured metrics for the specified data items?

• What are the actual monetary values of the changes achieved?

• What other identifiable cost savings have resulted from the training methodology used, compared to alternative methodologies (e.g. costs saved by using elearning vs. classroom)?



Philips ROI

evaluation5-Level

Philips ROI Level Of EvaluationReturn on investment (ROI) has been used in business as a means of determining the value of an investment in financial terms. Phillips’s framework is comparable to Kirkpatrick’s, but Phillips expanded Kirkpatrick’s four-level framework by adding a fifth level, ROI. Return on investment is calculated in order to show value, in financial terms, of a training investment (J.J. Phillips, 1991). The levels of Phillips’s framework are (1) reaction and planned action; (2) learning; (3) job application; (4) business results; and (5) return on investment.

Level 1, reaction and planned action, is similar to Kirkpatrick’s Level 1 but also includes a plan of what participants intend to apply from the program. Some researchers have argued that ROI is contained in Kirkpatrick’s fourth level, results, and that a fifth level is not needed (Lanigan, 1997). The fifth level adds the cost-benefit analysis that is essential to calculate ROI, requiring that any change in Level 4, results, be converted into monetary value and compared to the costs of the program (J.J. Phillips, 1996a; P.P. Phillips, 2002).

The ROI MethodologyBecause of its appropriateness to the business setting, the evaluation model that emerged from the work of Dr. Donald Kirkpatrick and Dr. Jack Phillips has become the most credible and most widely used training and HRD evaluation methodology in the world. In particular, the Phillips ROI Methodology offers a practical way to forecast the potential payoff—return on investment (ROI)—of a proposed training or human resources development initiative before funds are committed.

The MCPCC ROI Forecasting Tool, based on the Phillips approach, is an easy-to-use decision support tool that allows organizations to quickly develop and examine the business case for investing in workforce development. While the tool itself is a self-contained and self-explanatory ROI calculator, the following is a brief overview of the ROI Methodology underlying the forecasting tool. ROI and cost-benefit analysis are familiar decision-making tools used in business management worldwide. They are used most commonly to analyze planned investment decisions for the acquisition of capital equipment and technology.



At its simplest, ROI is the ratio of the net benefits of an investment compared to its total costs.In order to conduct a training ROI analysis, the Phillips model recommends an 18-step process that systematically progresses through the key evaluation/development steps from needsassessment through program objectives development, evaluation strategy, evaluation instrument design, data collection, analysis, return on investment calculating, and reporting.

The Phillips ModelThe Phillips model measures training outcomes at five levels:Level 1(Reaction, Satisfaction, & Planned Action )Measures participant reaction to and satisfaction with the training program and participant’s plans for action.Level 2(Learning )Measures skills and knowledge gains.Level 3(Application and Implementation )Measures changes in on-the-job application, behavior change, and implementation. Level 4(Business Impact )Measures business impact.Level 5(Return on Investment (ROI)Compares the monetary value of the business outcomes with the costs of the training program.

Chain of Impact Ideally, when a training program is implemented, according to Phillips, it should create a chain of impact at several levels beginning at Satisfaction / Planned Action and ending in ROI.

When business results and ROI are to be measured (Levels 4 & 5), it is also important to evaluate the other levels. A chain of impact should occur through the levels as skills and knowledge learned (Level 2) and are applied on the job (Level 3) to produce business impact (Level 4). If measurements are not taken at each level, it is difficult in the end to conclude that the business results achieved were actually caused by the program. On the other hand, if a negative ROI results from the training investment, the evaluator should be able to identify which link(s) in the chain was (were) broken (e.g., the participants did not learn (Level 2), or they were unable to successfully apply the new learning on the job (Level 3).

From the perspective of the organization, the value of the information gathered increases with movement along the chain of impact from Level 1 (Reaction, Satisfaction, and Planned Action) to Level 5 (ROI).

Descriptions of 5 Levels

Level 1 - Reaction, Satisfaction, and Planned Action: Level 1 measures participants’ satisfaction with a program as well as their plans to use what they have learned. Although most organizationsevaluate at Level 1 exclusively, it should be noted that this level of evaluation does not guarantee that participants have learned new skills or knowledge or will use them on the job.

Level 2 – Learning: Using tests, skill practices, role playing, simulations, group evaluations, and other assessment tools, level 2 evaluations assess how much participants have learned. Again, although it is useful to know that participants have absorbed the new skills and knowledge, a positive outcome here does not mean that participants will use the new learning when they are back on the job.

Level 3 - Behavior, Application and Implementation: Level 3 evaluation assesses whether (and how much) participants applied the new knowledge and skills on the job. The extent to which the new learning is applied in the workplace (or changes behaviors) determines Level 3 success. Here too, it should be remembered, a favorable Level 3 evaluation does not guarantee that business outcomes will be positive.

Level 4 - Business Impact: Level 4 measures the extent to which business measures have improved after training. Typical Level 4 measures are output, quality, costs, and time. It is important to go beyond Level 4 business results, however, because even if the training program results in substantial business improvement, there is still a concern that the program’s costs may outweigh its business benefits.

Level 5 - Return on Investment (ROI): ROI is the ultimate level of evaluation. It compares the monetary benefits from the program with the program costs. Although the ROI can be expressed in several ways, it is usually presented as a percentage or cost/benefit ratio. (Phillips 2005)



The Phillips’ model evolves from, and can be distinguished from, the earlier Kirkpatrick model by the adoption of return on investment to yield additional, critical insight. ROI allows decision makers to compare the ultimate value of a training investment with other potential investment opportunities.

Isolation techniquesAnother key technique introduced by Phillips is ‘Isolation.’ In the past, a major barrier to evaluating the business impact of training was the perception that it is all but impossible to separate training’s impact from the many other potential factors driving bottom line results. An important feature of the Dr. Phillips’ evaluation model, therefore, is the recognition that credibility depends on the extent to which the evaluation accounts for these other factors. The Phillips model uses ten techniques to isolate the business impact of training from other potential sources of business improvement (interest rate changes, competitive environment, marketing programs, employee bonus schemes, seasonal effects that temporarily contribute to business improvements, etc.).

Intangibles A further refinement by Phillips to the Kirkpatrick model is the addition of intangibles. According to the methodology, it is important to recognize that some outcomes cannot be easily measured and converted to monetary values. Attempting to put a dollar value on outcomes such as customer satisfaction, a less stressful work environment, and employee satisfaction can be extremely difficult, and the results may be of questionable value. Trying too hard to attach a business value may call into question the credibility of the entire evaluation effort. As a result, the Phillips methodology recommends that evaluators do not try to convert those “soft” business measures, and instead report them as intangible benefits along with the “hard” business improvement outcomes such as increase in sales, reduced defects, time savings, etc.

The ROI Evaluation Process It worthwhile noting that evaluation may serve a number of key quality control functions in addition to establishing training’s business value. Some of the other objectives of evaluation are to:• improve the quality of learning programs• determine if a program meet its objectives � identify

potential strengths and weaknesses in the learning program �

• develop a cost/benefit analysis of training and an Human Resources Development (HRD) investments �

• support marketing of training or HRD programs � determine a program’s appropriateness for the target audience

• �assist in decision making about program investments and establish funding priorities

Key Stages:Two of the most critical stages in the Phillips ROI model are data collection and data analysis.

Data CollectionA central challenge of the ROI process is data collection—selecting the most appropriate method(s) and the time table for gathering outcomes data at each evaluation level. Depending on the nature of the training program, hard data (representing output, quality, cost, and time) or soft data (customer satisfaction and job satisfaction) may be collected in the course of the evaluation.

The following are some of the key data collection methods employed in the Phillips/Kirkpatrick approaches. • Follow-up surveys and questionnaires to measure

stakeholder satisfaction and reaction. � • On-the-job observation to gauge application and

use. � • Tests and assessments to assess learning. �

Interviews to record participant reaction and program implementation. �

• Focus groups to assess job application of the program. �

• Action plans to encourage application and record participants’ progress and impact realized. �

• Business performance monitoring of performance records or operational data to identify improvement.



Data AnalysisOne of the most critical steps in a training evaluation is Isolation. Many other factors (e.g., marketing programs, employee bonus programs, etc.) can impact business outcomes after a new program is implemented. From the standpoint of credibility, therefore, it is important to use one or more isolation strategies to separate the business impact of the training program from the impact of other factors.

The following are some of the most often used isolation techniques:

Control Groups: The outcomes of pilot group of participants in a program are compared with the outcomes of a control group of participants who are not participating in the program. This is the most effective way to isolate training’s impact.

Trend lines: Trend lines are commonly used to project expected business impact data into the future. Such projections are compared with the actual data after a major corporate program or initiative has been implemented to determine if the training has yielded improvement.

Participants/stakeholders estimates: Participants are asked to estimate the amount of improvement directly related to a training or HR program. Because participants have received the training and are personally involved in applying the new skills and knowledge to the workplace, they are often in the best position to assess training’s relative impact.

Converting Data to Monetary ValuesIn order to calculate the ROI, the data collected in a Level 4 evaluation is converted to monetary values and compared to program costs. To do this, a value must be placed on each unit of improvement (output measure) impacted by the training program. Increases in output measures (increased sales, units produced, etc.) are converted to monetary values based on their unit contribution to profit. Often standard values are available for such measures in most organizations. There are several alternative ways to arrive at monetary value such as converting the value of quality improvements or using participants’ wages in those situations where the training has resulted in saving employee time.

Tabulating Program Costs In order to calculate the Benefit/Cost and the ROI, all of the costs involved in the program must be tabulated. Among the typical cost components of training programs that should be included are the needs assessment, cost to design and develop the program, the cost of all program materials provided to each participant, travel and accommodation, and the fully loaded salaries (i.e., salaries plus benefits) of participants and facilitators.

Calculating the Return on InvestmentThe ROI is calculated using the program benefits and costs. The key metrics are Benefit/Cost ratio (BCR) and ROI. The benefits/cost ratio is the total program benefits divided by the total costs. The BCR formula is

BCR = Total Program Benefits Total Program Costs

The return on investment is the net benefits divided by total program costs, as described earlier.

ROI = Total Program Benefits – Total Program Costs /Total Program Costs x 100%

ExampleA sales training program delivered to 34 retail sales associates results in an increase in sales contributing an increase in profits to the organization of $120,000 per year. The total cost of the program was $90,000.

The benefit cost ratio isBCR = $120,000

$90,000= 1.33:1

In other words, for every $1.00 spent on the training program, $1.33 is returned.

The ROI of the program is:

ROI = $120,000 -$90,000 x 100% $90,000

= $30,000 x 100% $90,000

= 33%

This means that every dollar spent on the training program is returned and an additional $0.33 is returned as profit.



Intangible BenefitsIn addition to monetary benefits, most training programs will have intangible, nonmonetary benefits. Intangibles are those measures that cannot easily be converted to monetary values. According to Phillips, “In some programs, such as interpersonal skills training, team development, leadership, communications training, and management development, the intangible (non monetary) benefits can be more important than tangible (monetary) measures. Consequently, these measures should be monitored and reported as part of the overall evaluation. In practice, every project or program, regardless of its nature, scope, and content will have intangible measures associated with it. The challenge is to efficiently identify and report them.” (Phillips 2003)

Typical intangible variables include items such as• Stress reduction � • Employee engagement �• Grievance reduction �• Improved customer satisfaction � • Complaint reduction � • Conflict reduction/avoidance

Tips to Measuring ROI

Don’t go overboard ROI need only demonstrate value beyond a reasonable doubt. Find out what the executive sponsor identifies as success and stick to that.Shift from a quality to a results mindset When designing L&D programs, it’s all about delivering a quality experience that encourages learning. When evaluating, it’s about results. When calculating ROI, focus less on the quality of the experience and more on the effect of learning.Calculate ROI continuously Always know how the L&D program is performing so adjustments can be made. This not only helps improve the program, but can justify how dollars are being spent at any time.Build a step-by-step case for ROI Analyze organizational needs and develop strategic learning plans, prioritize them and present them with sound justification— based on anticipated ROI – about why senior leaders should support them.Gather data beyond the program delivery and don’t forget data that is already available Evaluation and feedback should come from as many sources as feasibly possible—from participants, their supervisors, peers and senior leaders.ROI isn’t just about money When analyzing results, consider such learning measurements as quality, effectiveness, job impact and business results.Be conservative in ROI calculations To compensate for bias, self-reported ROI should be factored down and follow-up evaluations should be weighed more than evaluations reported immediately after the program.Represent the money outlay as a per participant ratio Personalize it. Show the per participant cost (versus a total cost) to make the investment more palatable (e.g. for this target population we are looking at a $7,000 investment in learning for an employee responsible for, on average, $1million worth of business).Communicate the story behind the numbers This is where using anecdotal information can be helpful in confirming the numbers. It never hurts to highlight data with meaningful examples.If the ROI numbers are low, don’t be discouraged ROI is intended to assess what is working and what should be shelved or revamped.



CIRO Model

evaluation4-Level

CIRO ModelThe CIRO approach to evaluate training impact is and another 4 level approach which is originally developed by Warr, Bird and Racham. It is unique was to classify evaluation process.This approach consists of four level of evaluation, first letter of each evaluation forms the word CIRO.Context evaluationInput evaluationReaction evaluationOutcome evaluation

Context evaluation of the learning eventContext evaluation involves collecting information about performance deficiency, assessing that information to establish training needs and on the basis of those findings, setting of objectives at three levels.Context of the learning event concerns with obtaining and using information about the current operational situation in order to determine training needs and objectives. This evaluation determines if training is needed.

During this process three types of objectives may be evaluated.• Ultimate objectives: The particular deficiency in

organization that program will eliminate.• Intermediate objective: Changes in employees

work behavior necessary for attainment of ultimate objectives.

• Immediate objectives: New knowledge, skills or attitudes that employees must acquire to change their behavior and to reach intermediate objectives.

Input evaluation to the learning eventIt concerns with how well the learning event was planned, managed, designed and delivered. It involves determining how cost efficient, cost effective and feasible and well chosen major inputs are. It involves analyzing the resources available and determining how they can be deployed in order to achieve maximum possibility of desired objectives.

Reaction evaluation to the learning eventReaction evaluation concerns with obtaining and using information about participants reactions to improve the HRD process. The distinguishing feature of this type of evaluation is that it rely on subjective inputs of participants. It can be helpful when collected and used in systematic and objective manner.



Outcome evaluation of the learning eventThis involves assessing what actually happened as a result of learning event.

Outcome should be measured at any or all of the following levels, depending on the object of the evaluation exercise and resources available for the task.

The learner level: This involves establishing changes in learners knowledge, skills and attitudes at the completion of the training. These changes can be determined and compared with levels of knowledge, skills and attitudes identified at the beginning of program.

The workplace level: This involves changes that take place at the workplace level in the learners job behavior. This can be measured by appraisal, observation, discussion with the manager of learner/peers/customers/clients.

The team/department or unit level: This involves identifying changes that take place in team, department or unit as a result of learning event. It is very difficult to evaluate changes at departmental level. Changes at departmental level may include alteration in departmental output, costs, scrap rates, absenteeism, staff turnover or accident frequency. Unit level changes may include enhanced productivity rates, reduced labor costs, reduced absenteeism and staff turnover rates.

The organizational level: This involves identifying changes that take place in the organization as whole after the completion of training program. This outcome is also very difficult to evaluate. The changes which may occur after the introduction of training program may include change in culture of organization, more flexibility, reduced level of conflict, enhanced ability to attract and retain valued workers.

Investment (Cost) factors Some of the factors to take into consideration when calculating the total cost of a training programme or course

• Money spent on the training/learning • Equipment upgrades (e.g. for an

advanced e-learning package requiring additional hardware and software, servers etc, or for additional equipment in a classroom)

• External content development and maintenance resource (for training and testing/certification

• Classroom costs (space costs, equipment rental, catering, materials, promotional items, stationery)

• Cost of infrastructure overheads for e-learning and virtual classroom facilities (a share of the ongoing costs of having access to these facilities)

• Communications costs to reach internal and external audiences • promotional programmes • mailers and flyers • poster design and production • conference calls and meetings • PR

• Costs incurred in the evaluation exercise

• Time and effort used • Who

• Training Project team • Training programme managers • Instructors • Senior management time and

attention • Training administration• Learners/students

• What activities • Training Needs Analysis and

Programme design • Materials development and

testing • Accreditation design and

development • Communications and training • Evaluation and ROI calculation



Brinkerhoff

evaluation6-Level

Brinkerhoff ModelBrinkerhoff (2005) squarely addresses one of the critiques of Kirkpatrick’s model: “Performance results can’t be achieved by training alone; therefore training should not be the object of evaluation” (p. 87). According to Brinkerhoff, this is like saying that the success of a marriage depends on the quality of the wedding ceremony. Multiple variables contribute to the impact of a learning opportunity (be it training, performance support, or another solution) and multiple stakeholders own these results—from senior executives to managers to HR professionals/L&D team members to employees.

Given that the factors that influence performance operate at the level of systems, Brinkerhoff urges that the proper focus of evaluation is also at the system level.

• Evaluation should address the following questions:How well is an organization using learning to improve performance?

• What organizational processes/resources are in place to support performance improvement? What needs to be improved?

• What organizational barriers stand in the way of performance improvement?

The six stages of Brinkerhoff’s evaluation model are: Stage I - Evaluate Needs and Goals Stage II - Evaluate the Design Stage III - Evaluate OperationStage IV - Evaluate Learning Stage V - Evaluate Usage and Endurance of Learning Stage VI - Evaluate Payoff

Stage I - Evaluate Needs and GoalsThe data collected at this level are used to analyze and prioritize the needs, problems, and weaknesses of the organization and establish what training goals are worth pursuing. This analysis also provides crucial information to determine whether training is the solution to the weaknesses identified.There are several situations that may call for a training solution, such as performance deficits, organizational changes, or management decisions. Since Stage I analysis will provide a framework for establishing the value of the training and determine its potential payoff, it is directly linked to Stage VI evaluation, which determines whether or not the training was worthwhile.



Stage II - Evaluate Training DesignThis level assesses the appropriateness of the training design. It focuses on the issues that must be considered after the decision is made to undertake a training activity, but before it is implemented. At this stage, several designs may be proposed, and the strengths and weaknesses of each assessed. The design finally adopted may represent a composite of the best elements of several designs.

Stage III - Evaluate Training ImplementationOnce the training design is deemed appropriate, this stage monitors the training activities and gathers feedback on the reaction and level of satisfaction of the participants. It assesses the discrepancy between what is actually taking place in the training and what was planned or expected. To solve the problems encountered at this level, trainers may need to refer back to the training design (Stage II) and make the necessary adjustments.Some useful techniques to conduct Stage III evaluation include:

Interviewing. Whether the interviews with participants are structured or informal, they are a useful technique because they allow the trainers to ask follow-up questions and obtain more detailed information.

Key participant method. This method involves selecting trainees who, because of their expertise or leadership qualities, are able to provide thoughtful comments and insights.

Observations. One trainer observes another and records participant reactions and behaviors. If an observation form is developed, it will render useful quantitative data on the reaction of the participants.

Trainee rating and reaction forms. Questionnaires and surveys may be administered at regular intervals during the training to gauge the satisfaction level of the participants. But because this is the most commonly used method to evaluate reaction to the training, participants may not pay much attention to the forms and provide superficial comments. Nonetheless, their reaction is important in order to proceed to the next stage.

Stage IV - Evaluate Learning Any training event, regardless of its scope or duration, aims to enhance the skill and knowledge level of the participants. The extent to which this improvement has been achieved is the measure of the effectiveness of the program. Stage IV determines the level of learning and improvement that took place. If sufficient learning occurred, we can expect that it will be applied in the workplace and results will be achieved.

The data gathered at this level are used to revise and refine the activities and strategies that will ensure the desired transfer of learning. Brinkerhoff suggests the following uses for Stage IV evaluation:

Gathering evidence that proves the effect of training—accountability. Trainers need to provide evidence that the skill and knowledge level of the participants has improved.

Determine mastery of training results - This information is useful at three levels: it provides feedback to the participants regarding their achievement, to the trainers regarding trainee performance, and to the supervisors regarding the degree of skill mastery of their staff.

Looking for unintended results- If the unintended results are also undesirable, trainers need to know this information to reinforce those areas in the program that produce desirable results.

Planning for Stage V follow up- Because Stage IV evaluation identifies weaknesses in the achievement of skills and knowledge, it sets the framework for Stage V evaluation, which assesses the application of these skills.

Marketing - By reviewing past learning results of training, particularly results based on "reliable and valid measures of achievement," SO teams and customers can make informed decisions on whether training is the appropriate answer.



Stage V - Evaluate Usage of LearningThis level of evaluation indicates how well the trainees are using on the job the knowledge and skills acquired. "It looks at actual performance, not ability to perform.“

Stage V evaluation usually takes place at the workplace, which "represents the richest source of data." Because transfer of training to the workplace does not take place exactly as planned, evaluators should take into account the numerous steps and changes that occur from the learning results phase (Stage IV) to the eventual application of these results.

The purpose of Stage V evaluation then is to record and analyze these steps and changes. Essentially, it documents when, where, how well, and how often the training is being used; which skills are and are not being used; and how long the effects of training have lasted.

The author once again cautions that it is important to clearly define the explicit purposes and uses of a Stage V evaluation before designing it.

Below are some guidelines:Revising training -This level determines the effective and ineffective ways in which the new knowledge is being applied. It signals ways of improving the program to achieve transfer of skills and knowledge at the level expected. Or it may be decided that other types of targeted interventions at the workplace, such as providing peer support or greater guidance, are all that is necessary. Planning ahead for Stage VI evaluation -The benefits that training brings to the organization cannot be assessed without an accurate understanding of how the new skills are actually being applied. Stage V documents instances of appropriate application of the new skills, which forms the basis for Stage VI inquiry. Documenting and accounting for transfer of training- Provides crucial information to potential participants as to what they can expect from training in terms of actual results in the workplace. By documenting the before and after behaviors in the workplace, an evaluator also develops a database for Stage VI evaluation.

Stage VI - Evaluate Payoff By the time the evaluation process reaches this level, we can assume that the training was successful, the participants are applying what they have learned, and an evaluator has identified and recorded the extent to which changes have taken place in the workplace.

The aim of Stage VI evaluation then is to assess the value that these changes have brought to the organization and whether this value was worth the effort given the time and resources expended.

The sequence of events that follows a training intervention—from acquiring knowledge to changing behavior in the workplace to deriving benefit from the change—is an unpredictable and complicated one. The worth of the training is measured by documenting the benefits, assessing their value, and comparing them to the cost of the training.

Thus, the key question posed by Stage VI inquiry, Was the training worth it? cannot be readily answered without first examining four issues:• The benefits that have resulted from training. • The value of each of the benefits (monetary or

otherwise). • How the value of the program’s benefits compare

to training costs. • The extent to which the initial training need or

problem has been resolved.

Some important guidelines to follow when embarking on a Stage VI evaluation include:

Consider a broad range of training impact variables-This involves documenting benefits that may not be directly linked to the needs but are nonetheless beneficial to the organization.Look for specific training applications-The point here is that a list of specific applications —as opposed to general statements of impact—is of greater use when making decisions about the value of future programs.Consider a wide range of cost factors- The worth of training must take into account the numerous costs associated with this activity.



Refer to specific data from preceding evaluation stages -This guideline refers to the recycling concept. Stage VI calls for attributing a value—monetary or otherwise—to the results. Thus, the data obtained at Stages II and III, the design and implementation levels, are used to estimate the cost of the training. Stage I data, as mentioned above, provide a basis for determining the value.

The crucial point that bears repeating is that Stage VI evaluation should not be undertaken without reference to the data obtained at the previous levels.

The six-stage model represents an exhaustive evaluation exercise that is not always feasible to achieve based on competing deadlines and reduced budgets. Nonetheless, Brinkerhoff’s advice is to consider each of the stages, if only briefly, to guide and bolster the training function and educate the customers as to its benefits.

NotesTraining and development -- or "learning anddevelopment" as many refer to it now -- is oneof the most important aspects of our lives andour work. (Many people view "training" as anactivity that produces the result or outcome of"learning" -- and learning is typically viewed asnew knowledge, skills and competencies orabilities.) In our culture, we highly valuelearning. Yet, despite our having attendedmany years of schooling, many of us have noidea how to carefully design an approach totraining and development.LearningTypically, learning is viewed as enhancingone's knowledge, understanding or skills.Some people see learning as enhancement toone's knowledge, awareness and skills. Someprofessionals view learning as enhancing one'scapacity to perform. Some view learning as away of being that includes strong value onreceiving feedback and increasingunderstanding. It's important to note thatlearning is more than collecting information --more than collecting unreferenced books on ashelf. Depending on the needs of the learner,knowledge is converted to skills, that is, thelearner knows how to apply the knowledge toget something done. Ideally, the skills areapplied to the most appropriate tasks andpractices in the organization, therebyproducing performance -- results needed bythe organization.TrainingThis term is often interpreted as the activity when an expert and learner work together to effectively transfer information from the expert to the learner (to enhance a learner's knowledge, attitudes or skills) so the learner can better perform a current task or job.DevelopmentThis term is often viewed as a broad, ongoing multi-faceted set of activities (training activities among them) to bring someone or an organization up to another threshold of performance. This development often includes a wide variety of methods, e.g., orienting about a role, training in a wide variety of areas, ongoing training on the job, coaching, mentoring and forms of self-development. Some view development as a life-long goal and experience.


Consultants Ltd.

About UsA strategic business unit of HSPP Group, HSPP Consultants is a management consultancy firm that brings progressive and innovative techniques to harness and optimize the potential of human resources. We strive to awaken human and organizational excellence through exclusively customized consulting, organizational development (OD), training and human resource (HR) development services.We have strategic partners across the globe who provide training, facilitation and consulting services in Bangladesh under our banner.

In Bangladesh, HSPP is working with a diverse range of industries including telecommunications, banks, financialservices, logistics and freight forwarding. Within a year of starting its operations, HSPP has become strategic partners of Airtel, bKash, IPDC, Robi, HSBC, DAMCO (APM Global Logistics Ltd.), Siemens, Coca-Cola & Hellenic Intl. We take pride in our effective experiential approaches which use state-of-the art business simulations, fully equipped facilities, multimedia tools and case studies. We are a growth catalyst helping develop and transform employees across all levels. The goal of HSPP is not just to be a vendor for organizations, but to become strategic partners in order to build sustainable, long term relationships.

Our ExpertiseHSPP’s expertise covers the full spectrum of learning and development initiatives, including instructional design, training facilitation, project management, needs assessment, organizational development, learning technology consulting, and more. On-demand consulting services also exist, that combine the top-notch expertise and skills of our consultants with speed of placement and a high degree of flexibility tailored to your specific needs, industry and corporate culture.

At HSPP, we strive to become the flag bearer of human and organizational development work in Bangladesh and channel in expert guidance from the global pool of consultants specialized in providing cutting-edge workshop-based skills training.

We aspire to set new standards in the field of L&D solutions for the FI industry in Bangladesh through a team of career bankers with expertise in private banking, priority banking, retail banking and wealth management via practical advice, analysis, insight and information. With state of the art services in training, coaching, consultancy, certification and behavioural assessment, our work has been widely appreciated.

Training Services Overview: Fully customizable services for all your training needsAt HSPP, it’s all about understanding that coordinating a business-related training session can be challenging to manage. That’s why they have made it a mission to help clients plan, organize and deliver the perfect event. Be it conducting an official meeting or implementing a large-scale training project, HSPP provides everything that is needed to achieve the goals envisioned by client companies. HSPP offers a full-range of training and meeting services that can be used in conjunction with their room rentals or as standalone products. Their experienced account managers will create a customized package that best fits the needs of corporations, enabling them to host a comprehensive and successful training event.


advisory servicesGray Matters comprises consultants who understand the challenges that businesses face in attracting, retaining and motivating people in a competitive environment. We work in partnership with our clients to deliver people solutions based on our knowledge of businesses and expertise in HR leading practices, that would help accelerate growth for the organizations.

Our solutions and approaches unlock the full potential of an organization – and deliver real, measurable, substantial and meaningful business results.

impl

emen

tatio

n fo

cus

Many a times we hear, “Your recommendations are excellent and meets our requirements but the real challenge is in implementation.”

At Gray Matters we endeavour to fulfil this gap and extend support to our clients - from recommendation to implementation. More and more organizations are seeking the help of professional HR service firms to address their HR needs. Our research tells us that it is not ‘cost’ that is driving this demand for external service providers. It is the acute shortage of HR talent that is driving more and more organizations to partner with professional HR service firms like Gray Matters to meet their Strategic HR needs.

Moreover, there is a time lag in identifying the right person and ensuring his joining. Thereafter, he spends time in understanding the industry, job needs and his settling down time in the new role. We start the work immediately.

Employees can leave the organization, without completing the process, which tantamounts to double the cost envisaged -Consultants like Gray Matters, are bound by contract to complete a process / assignment at an agreed upon fee structure.

Gray Matters provides “implementation support” to organizations with the “On-site” suit of products: HR On-site and PMS On-site

Gray Matters also helps organizations in Employer Branding Initiatives and Employee Engagement Initiatives as part of their implementation focus

We help in implementation too…

Our Advisory Solutions

For more information on Gray Matters, people strategy and human resource management and other offerings, visit our website at: www.graymatters.co.in

http://www.graymatters.co.in/


Self Development Skills

Negotiation SkillsStress ManagementAnalytical Thinking & CreativityConflict ManagementWorkplace EtiquetteCustomer OrientationSelling to winRelationship Management

Team Building Outdoor Adventure LearningLeadership DevelopmentPerformance CounselingManagerial Effectiveness

Tactical Workplace Skills

Alignment Skills

Balanced ScorecardCompetency Based InterviewingCompetency ManagementCoaching & MentoringTrain the TrainerEmployer Branding

Reckoning Skills

Communication SkillsPresentation SkillsTime ManagementAssertiveness SkillsTransactional AnalysisAchievement OrientationWinning Ways

outd

oor a

dven

ture

beha

viou

ral

Understanding the significance of behavioural skills, many organizations have started investing in it as a part of their corporate training initiative. Behavioural skills training have witnessed a marked transition in the last decade from being ‘generic’ and ‘good to have’ to being ‘defined’ and ‘pre-requisite’. This shift in perception is not without reason. Organizations across the globe have realized that professionals with technical skills only, partly complement the essentials of being a ‘complete professional’. Research study has revealed, that people skills outweigh technical skills by a factor of 3:1, in deciding the long-term success of professionals.

Our Programs…

We Inspire

We Energize

You Learn

training programs


facilitator profiles

Sanjay was Associate Director, Ernst & Young, heading Eastern India and SAARCcountries before raking up the Entrepreneurial spirit inside him and started GrayMatters Consulting , an organization which offers services ranging from HR Advisory,Behavioural Training, Outdoor Adventure Learning, Education and Research to clients inIndia, Bangladesh, Nepal and the Middle East.Sanjay served in various positions for Ernst & Young, India acquiring vast consultingexperience in India, Bangladesh, Nepal, South East Asia and the Middle East. He wasresponsible for business development, delivery and also client relationshipmanagement during his tenure with E&Y.

Sanjay Roy ChowdhuryManaging DirectorGray Matters Consulting

As part of his consulting exposure, Sanjay has worked for assignments ranging from Organizational Restructuring, Visioning,Talent Management, Performance Management System with special emphasis on Balanced Scorecard, CompetencyAssessment Centres and Compensation and Benefits program.As a Trainer, Sanjay has trained corporate executives at the Middle, Senior and the Top Management cadre in organizationslike Tata Steel, Hindalco, BOC India, Oberoi Grand, Vedic Village, VIP Industries, Getit Infoservices, KDS (Bangladesh), KhimjiJewellers, Gupta Power & Infrastructure, Mercy Hospital, Pepsico, Caterpillar (Bangladesh), Lafarge (Bangladesh) and manyothers in the areas of Leadership, Competency Based Interviewing Techniques, Performance Feedback and Counselling,Dealing with People, Achievement Orientation, Analytical Thinking, Teamwork & Cooperation, Personal Effectiveness andOutdoor Experiential Learning.Sanjay entered the corporate world, armed with a Graduation from Presidency College, a Post- Graduation from CalcuttaUniversity with specialization in Biochemistry a Diploma in Computer Application from IIM-Calcutta and a ManagementEducation from XLRI (the premier business school in Asia for HR education).

Shilpa Chakraborty (Guest Faculty)Head Human ResourcesTIL Limited (Caterpillar dealer)Shilpa has over 17 years of Corporate Human Resources Management experience, withthe last 8 years as Head HR in TIL Limited (Caterpillar dealer) and the previous 8 years inconsulting from Ernst & Young, acquiring vast experience in India, Bangladesh andNepal. Shilpa is a Post Graduate in Business Finance from University of Calcutta. She isan Alumni of the National Institute of Personnel Management and Symbiosis – Punewith Specialization in Human Resource. She is a certified Trainer of the India Society forTraining & Development. She has a certification in Strategy from The StrategyAcademy of India. Shilpa is a certified Assessor for Competency Based Assessment andpossesses an International Certification of Thomas Profiling.

In her current role, Shilpa, looks after the entire gamut of Human Resources for TIL Limited and is also part of the Core HRThink Tank for Caterpillar Worldwide. In her current capacity Shilpa is responsible for India, Nepal and Myanmar. As part ofher current role she has been exposed to the Caterpillar Model of calculating ROI for Learning Interventions which she hasimplemented in TIL Limited. She has been successful in linking the Learning Needs to the Business Needs and also show thelearner’s the linkage of Training and Career growth. Shilpa has also been involved in designing various Talent Managementinitiatives and also delivering Training linked to ROI for Middle and Senior Management of TIL Limited.As part of her consulting exposure, Shilpa has worked for assignments ranging from Organizational Restructuring, Visioning,Talent Management, Performance Management System with special emphasis on Balanced Scorecard and CompetencyAssessment Centres with clients like Emami, Indian Oil Corporation, KDS (Bangladesh), Lafarge (Bangladesh) and many otherorganizations. Shilpa is also a regular speaker in the Corporate and Management Education circuit being invited to variousforums and conferences for Management discourses. Shilpa has been awarded HR leadership Award from RASBIC and WomenAchiever Award from EBA as a part of World HR Congress.


Gray Matters Consulting Pvt. Ltd.Corporate Office: 294, S N Roy Road, New Alipore, Kolkata – 700038, Ph: +91-33-24987647 / 24987648Email: [email protected]: www.graymatters.co.in

HSPP Consultants Ltd.Bangladesh Office: House 1, Road 3/A, Block J, BaridharaDhaka 1229, Tel: +88029852981-2, Mob: +8801716585913 Email: [email protected]: www.hsppconsultants.com

mailto:[email protected]

http://www.graymatters.co.in/

mailto:[email protected]

http://www.hsppgroup.com/