Author: Kiesow, Dale J. Title: Evaluation of an OJT Model in a Nonprofit The accompanying research report is submitted to the University of Wisconsin-Stout, Graduate School in partial

completion of the requirements for the

Graduate Degree/ Major: MS Training and Development

Research Adviser: Jeanette Kersten, EdD

Submission Term/Year: Fall2012

Number of Pages: 56

Style Manual Used: American Psychological Association, 6th edition

I:8J I understand that this research report must be officially approved by the Graduate School and that an electronic copy of the approved version will be made available through the University Library website I:8J I attest that the research report is my original work (that any copyrightable materials have been used with the permission of the original authors), and as such, it is automatically protected by the laws, rules, and regulations of the U.S. Copyright Office. I:8J My research adviser has approved the content and quality of this paper.

STUDENT:

NAME Dale Kiesow DATE: 12/7/2012

ADVISER: (Committee Chair ifMS Plan A or EdS Thesis or Field Project/Problem):

NAME Jeanette Kersten, EdD DATE: 12/17/2012

1

s; fA_i/)_ .... Doto I~ -17 /2

---------- -- =--------------------------------------------------------------------------------------------------------------------------

This section to be completed by the Graduate School This final research report has been approved by the Graduate School.

Director, Office of Graduate Studies: DATE:

2

Kiesow, Dale J. Evaluation of an OJT Model in a Nonprofit

Abstract

A number of training models exist for adaptation to the nonprofit setting, but these

models primarily focus on the leadership development of the volunteers and not on developing

volunteer’s skills necessary to complete their duties. This study was conducted to evaluate the

implementation of an on the job training (OJT) model developed by Rothwell and Kazanas

(2004) in a nonprofit environment. The evaluation method used in this study was the Success

Case Method developed by Brinkerhoff (2003) to measure what the volunteers liked most and

what they liked least about the training structure. A survey questionnaire was distributed to the

volunteers who had participated in training classes during the summer of 2012. This paper

discusses the analysis of results for both the quantitative and qualitative data collected followed

by recommendations to the organization and opportunities for future research.

3

Acknowledgments

I have many people to thank for their role in helping me complete this study. So many

people provided their support in so many different ways. I can say that looking back, I can see

what a network of people I have that encourage, support, and are willing to help me fulfill my

goals.

First and foremost, I would like to thank all the family and friends. Without their

constant love and support, I would not be where I am today. My family and friends have shown

support for me while accomplishing this research paper as well as attending the master program

at the University of Stout. They supported me with words of encouragement and reassurance

that inspired me to work hard and remain determined.

I would like to thank my thesis advisor Jeanette Kersten, EdD. for her wisdom and help

throughout the research process. Had it not been for her feedback and insight while conducting

this study, I would have not finished with a quality research paper. She was always pushing me

to finish and provided me with positive words of encouragement when times were difficult. She

demonstrated patience and was always available when I needed advice or clarification. I thank

her for her hard work and for believing in me. Without her, none of this research paper would

have been possible.

I would to like to thank Sandy Gilbert and the volunteers of Refuge Farms, LLC for

allowing me to conduct this study. Had I not been introduced to the wonderful work that she

does with neglected and unwanted horses, I would not have had this opportunity. She and her

family of volunteers do wonderful things to save the lives of these giant animals and they are one

of the most dedicated groups of people I have ever met. I truly hope that the results of this paper

provided useful data to continue to grow the organization and the volunteer workforce.

I would like to thank a friend and graduate of the training and development program at

Stout, Alvin King. He provided me with his experiences with the program at Stout before I

4

enrolled. He spoke highly about the structure of the program and its many benefits. He

encouraged me to enroll in the program and has provided words of support for my research

project as well as friendship ever since. Had it not been for Alvin’s encouragement, I most likely

would not have entered the training and development program and advanced my education like I

have.

5

Table of Contents

Abstract ............................................................................................................................................2

List of Tables ...................................................................................................................................7

Chapter I: Introduction ....................................................................................................................8

Statement of the Problem .....................................................................................................8

Purpose of the Study ............................................................................................................8

Assumptions of the Study ....................................................................................................9

Definition of Terms............................................................................................................10

Limitations of the Study.....................................................................................................12

Methodology ......................................................................................................................13

Summary ............................................................................................................................13

Chapter II: Literature Review ........................................................................................................14

Training Models for Adoption in the Nonprofit Sector .....................................................15

Faith Based and Community Initiative Model .......................................................15

Flowchart Safety Training Model ..........................................................................17

On the Job Training (OJT) .....................................................................................18

Training Evaluation for Adoption in the Nonprofit Sector ................................................23

Kirkpatrick Four Level Evaluations .......................................................................22

Success Case Method .............................................................................................23

Summary ............................................................................................................................25

Chapter III: Methodology ..............................................................................................................25

Subject Selection and Description .....................................................................................25

Instrumentation ..................................................................................................................25

Data Collection Procedures ................................................................................................25

Data Analysis .....................................................................................................................26

6

Limitations .........................................................................................................................26

Summary ............................................................................................................................27

Chapter IV: Results ........................................................................................................................28

Quantitative Data ..............................................................................................................29

Qualitative Data ................................................................................................................30

Varying Levels of Experience ...............................................................................32

Knowledge Transfer ...............................................................................................32

Positive Reception to Training ...............................................................................33

Motivation to Learn ...............................................................................................33

Continuous Improvement of the Program ..............................................................34

Summary ................................................................................................................35

Chapter V: Discussion ...................................................................................................................36

Limitations ........................................................................................................................36

Conclusions ........................................................................................................................37

Recommendations ..............................................................................................................39

Future Research .................................................................................................................41

Summary ............................................................................................................................43

References ......................................................................................................................................44

Appendix A: Letter of Invitation ..................................................................................................48

Appendix B: Implied Consent to Participate in Approved Research ............................................49

Appendix C: Participant invitation Letter ......................................................................................51

Appendix D: Evaluation Check List ..............................................................................................52

Appendix E: Post Training Survey Measure .................................................................................54

Appendix F: IRB Approval Form ..................................................................................................56

7

List of Tables Table 1: Question 1- Duration of Volunteering at Refuge Farms ................................................30

Table 2: Question 3- Training Time ..............................................................................................31

Table 3: Question 5- Skills Provided .............................................................................................31

8

Chapter I: Introduction

Refuge Farms, LLC is a non-profit organization that specializes in the care,

rehabilitation, and placement of abused, injured, and abandoned horses throughout the American

Midwest. The organization became a non-profit in year 2002 based on the vision and

determination of the Executive Director, Sandy Gilbertson and her mentor Andy Durco. Refuge

Farms, LLC is located in Spring Valley, Wisconsin and is currently home to over a dozen horses

(http://www.refugefarms.org).

Refuge Farms, LLC relies on a volunteer workforce comprised of 48 volunteers, seven

full-time and 41 part-time for the daily operations of the farm that includes horse grooming,

feeding, and ground’s keeping. Volunteers are expected to attend horse/farm safety training prior

to engaging in activities related to the farm or the horses and also participate in multiple

fundraising events and activities (http://www.refugefarms.org). This study is derived from a

previous project conducted between Refuge Farms, LLC and the Spring 2012 Management and

Coordination of Training and Development class at the University of Wisconsin Stout. (see

Appendix D).

Statement of the Problem

The organization in this study is a nonprofit organization that does not have a structured

training program which has caused a gap in volunteers acquiring the necessary skills to perform

their tasks and duties competently and safely, impacting consistency among volunteers,

efficiency, and volunteer retention.

Purpose of the Study

Volunteers play an essential role in the performance and effectiveness of any non-profit

organization and, in the case of this study, can be the only workforce an organization has

available. One of the largest hurdles that organizations relying on volunteers face is the need to

reduce turnover. A review of the research has revealed that almost 35% of all volunteers leave

9

before completing their first year and although studies have shown that people end volunteering

for a variety of reasons, volunteer management and development through training have been

shown to play an important role (Kerka, 2003; Davila, 2008). Small nonprofit organizations tend

to have a shortage of opportunities in the development of a formal training infrastructure. As a

result, the nonprofit training environment has relied on other means of collecting needed

information, advice, and training (Pohjoispuro, 2006).

Refuge Farms, LLC has an opportunity to introduce a formal and structured training

program that will provide volunteers with the ability to development their skills with support

from the organization. Most of the skills required are taught informally using on-the-job (OJT)

style of training, conducted by the Executive Director at unscheduled intervals with limited

evaluation of effectiveness and learning transfer (Brinkerhoff, 2003). Implementation of a

development plan for Refuge Farms, LLC will provide an opportunity for volunteers to acquire

new skills and tasks which may positively impact volunteer turnover, burnout, and personal

fulfillment of the mission and vision of the organization (Kerka, 2003).

A structured training program at Refuge Farms, LLC will also provide volunteers with

the skills needed to perform their tasks safely and reduce liability. Incorporating horse handling

techniques into training may provide an adequate mastery of skills for volunteers for managing

large horses that have a history of abuse or limited human contact.

The assumptions of this study are:

1. Refuge Farms, LLC has a variety of volunteers that have different levels of experience in

horse handling, either as a volunteer or in their personal and professional lives.

2. For consistency, each volunteer regardless of their prior experience with horses is

beginning as a Foal volunteer and will only be granted the level of autonomy,

responsibility, and supervision associated with Foal status.

10

3. This study also assumes that all volunteers with Refuge Farms, LLC have attended the

orientation training before beginning horse handling classes.

4. Due to the previous work the organization has had with University of Wisconsin-Stout, it

is assumed that the needs assessment for this training program has been conducted and

analyzed with the Executive Director of Refuge Farms, LLC providing the information

for this study.

5. It is assumed that this study will be effective and that volunteer participants will be in

favor of the new training structure with the organization benefiting from increased

retention of volunteers.

Definition of Terms

Adult learning principles. Also known as Andragogy, the methods or techniques used

to teach adults (www.dictionary.com)

Alpha. (especially of animals) having the highest rank of its sex in a dominance

hierarchy: the alpha female (www.dictionary.com). According to Refuge Farms, LLC a

volunteer with expert levels of horse handling skills (S. Gilbert, personal communication, July

20, 2012).

American National Standards Institute (ANSI). A U.S. organization that recommends

standards for many products in various industries (www.dictionary.com)

Developing a Curriculum (DACUM). DACUM is an acronym for developing a

curriculum process that provides a picture of what the worker does in terms of duties, tasks,

knowledge, skills, traits and in some cases the tools the worker uses

(http://www.dacum.org/index.asp, 2001, para. 3).

Faith Based Community Initiative Model. Model developed designed for community

groups and faith-based organizations seeking to maximize the skills of their volunteers, expand

11

their services to the community, and enhance their effectiveness (Alex, Ennis, Lorrain, Mcghee,

Peterson, Powers, Thompson, & Warder, 2005, p. vii).

Horse Handling Unit (HUE). One unit is equally to one hour of direct contact

volunteering with the horses a Refuge Farms, LLC (S. Gilbert, personal communication, July 20,

2012).

Foal. Young horse, mule, or related animal, especially one that is not yet one year of age

(www.dictionary.com). The definition of according to Refuge Farms, LLC is, a foal level

volunteer is a new volunteer or a volunteer in the entry level of horse handling requiring a

mentor when beyond the gates and will not handle any of the incoming horses (S. Gilbert,

personal communication, July 20, 2012).

Flowchart Safety Training model. Flowchart to show the relationships among the

component processes for all safety and health training (Jensen, 2005, p. 26).

Impact Model. A projection model that portrays what successful behaviors and results

of success would look like if initiative were really working (Brinkerhoff, 2003, p. 33).

Yearling. An animal in its second year (www.dictionary.com). The definition according

Refuge Farms, LLC is a volunteer with a solid horse handling base that may go beyond the gates

under supervision but will require a mentor when in the pastures and does not handle any of the

incoming horses (S. Gilbert, personal communication, July 20, 2012).

Nonprofit. Not established for the purpose of making a profit, not entered into for

money (dictionary.com).

On the job training. A structured process conducted at the employee’s work area to

provide the employee with the knowledge and skills to perform job tasks (Lawson, 1997, p. 2).

Occupational Information Resource Network (O’Net). Database containing

information on hundreds of standardized and occupation-specific descriptors

(http://www.onetcenter.org/overview.html, para. 1)

12

Occupational Safety and Health Administration (OSHA). The division of the

Department of Labor that sets and enforces occupational health and safety rules

(www.dictionary.com)

Self-efficacy. Determines whether coping behavior will be initiated, how much

effort will be expended, and how long it will be sustained in the face of obstacles

and aversive experiences (Bandura, 1977).

Success Case Method (SCM). A carefully crafted, simple, and proven way of quickly

finding out how well a new organizational initiative is working (Brinkerhoff, 2003, p. viii).

Topical Sequencing. Asking experienced performers to clarify the order in which

functions should be taught based on the novice performer needed to understand (Rothwell and

Kazanas, 2004, p. 78).

Limitations of the Study

1) Lack of qualified volunteers to conduct training with sole responsibility lying with

the Executive Director of the organization

2) No previous training budget information has been established by the organization.

3) Limited number of horse handling classes available per year.

4) Lack of training models specific to non-profit volunteers except for leadership

positions (Cosier & Dalton, 1993; Deslandes & Rogers, 2008).

5) Volunteer engagement in the volunteer training program may be a limitation because

participation in the training program has not been determined to be mandatory by the

organization.

6) Sustainability for the program may not occur due to the organization lacking financial

resources and qualified volunteers to maintain the program.

13

Methodology

The methodology of this study included collecting qualitative data upon conclusion of the

structured training program from a volunteer foal to yearling level. The data will be compared to

pre-training data collected in a focus group conducted by a prior project between Refuge Farms,

LLC and INGMT 750 Organization Development course fall semester 2012 University of

Wisconsin Stout, 2012). Post data collection will involve a survey questionnaire to determine

whether participants were satisfied with the use of the selected model and the introduction of a

development plan to the organization. (Rothwell & Kazansa, 2004; Brinkerhoff, 2003).

Summary

Chapter 1 involved stating the need for Refuge Farms, LLC to implement training

development with definitions to key terms found throughout the study. Chapter 2 will provide a

review of the literature for published training and evaluation models. Chapter 3 will provide a

review of the methodology for collecting data and the data analysis techniques used. Chapter 4

will present the results of this study. Chapter 5 will provide a discussion of the findings and

recommendations for future study and application.

14

Chapter II: Literature Review

Many raining models created for nonprofits have concentrated on leadership

development, and have not provided support for skill training for volunteers. Although several

training models exist for adaption, many of them focus on leadership in the nonprofit sector

(Cosier & Dalton, 1993; Deslandes & Rogers, 2008). As a result 3 training models have been

identified as adaptable to the needs of this study. They include the Faith Based Community

Initiative Model created by Alex et al (2005), a Flowchart of Safety Training Model developed

by Jensen (2005), and the DAPPER model created for OJT development by Rothwell and

Kazanas (2004). Two models for evaluation were also identified and included Kirkpatrick’s

(2006) 4 level evaluation model and Brinkerhoff’s (2003) Success Case Method.

In the non-profit sector, formalized training has been demonstrated as having an effect on

the retention and satisfaction of volunteers as illustrated in terms of support by the organization

(Phillips & Little & Goodine, 2002; Grossman & Furano, 2002). Researchers have found

management practices provided support to volunteers related to refresher courses, training, and

supervision were also associated with variability in commitment and service duration of

volunteers (Canaan & Cascio, 1999). Haski-Leventhal and Bargal (as cited in Hidalgo &

Moreno, 2009, p. 595) reinforced this aspect by demonstrating that the majority of doubt by

volunteers was directly correlated to the lack of instruction and formal training provided to them.

According to Lean (1984) many authors have debated whether nonprofits have differed in

operations from business and public organizations or in size related to complexity in developing

training programs. Many professionals in nonprofits have agreed that training is similar between

profit driven corporations and nonprofit organizations and learning practices have also proved to

be similar (Birdi & Patterson & Wood, 2007). In addition, Levinson (1987) identified 6 key

areas for understanding in developing a training and development program for non-profits.

15

1. Nonprofits are small and contain few employees, have small budgets and a narrow

mission

2. Nonprofits have complex interpersonal environments and one staff member may fill

multiple roles

3. Nonprofits exist in an uncertain and unstable financial environment because they usually

have a budget provided by donations

4. Altruism is the core value in many nonprofits because their main reason for existence is

helping others

5. Nonprofit Mangers often juggle dual personal identity because they are not only seen as

managerial but also as an equal by other volunteers or staff

6. Nonprofit boards, volunteers, and employees have a strong belief in the mission of the

organization and will sometimes ignore efficiency, effectiveness or the level of demand

for their service (p.81)

Training Models for Adoption in the Nonprofit Sector

Faith Based and Community Initiative Model. Training has provided volunteers

direction and the skills needed in order to carry out assigned tasks and responsibilities. The Faith

Based Community Initiative Model has been found to be best implemented by a staff member of

the organization, which has been found successful for nonprofits with a very limited workforce

(Alex et al., 2005). A faith-based and community issued handbook was developed for planning,

recruiting, training, managing, and evaluating volunteer programs (see figure 1).

16

Faith-Based and Community Initiative Model

Figure 1. Faith-Based and Community Initiative Model www.samhsa.gov/fbci/Volunteer_

handbook.pdf- Adapted from Volunteer Handbook, p. viii (Alex et al, 2005)

The training section of this model contained the creation of an orientation program,

assignment of orientation leaders, scheduling orientations, development of materials, conduction

of orientation, and training the volunteers (Alex et al., 2005). The model recommended the

inclusion of an orientation process that allowed new volunteers the opportunity to learn about the

organization (Alex et al., 2005). During this phase, volunteers received information on policies

and procedures, culture and norms, safety and health issues, and rights and responsibilities which

were disbursed via a volunteer handbook, orientation packet, or reference guides (Alex et al.,

2005; Kerka, 2003).

Volunteer training, in this model also addressed risk management by the most likely

causes of accidents because liability reductions were found to be important concerns when

working with volunteers. Norris (2006) reinforced this concept in the following statement:

These volunteers are a valuable asset, and yet they also pose a serious liability exposure.

Volunteers may be given access to the organization's finances, operate their own or the

organization's vehicles in the service of the organization, and/or come into contact with

vulnerable populations, such as children, the elderly, and those with special needs (p.1).

Due to the risk involved with volunteers, the Faith-Based and Community Initiative model was

found to encourage the use of policies and procedures to form the basis for orientation (Alex et

al., 2005).

Recruiting Planning Training Managing Evaluating

17

The authors also determined that the Faith Based and Community Initiative Model

needed training that was structured based on the following elements.

Specific to the requirements of the volunteer position

Geared to the skill level of the volunteer

Progressive

Specific to the needs of both the volunteer and supervisor

Periodically evaluated (Alex et al, 2005, p. 33)

The recommended training evaluation technique for the Faith Based and Community

Initiative Model followed an adapted evaluation technique developed by Kirkpatrick (2006).

The authors Alex et al (2005) recommended that when focusing on the evaluation of the

volunteer program 4 areas needed to be assessed in order ensure that your training is effective,

comprehensive, and received.

Reaction- How does the learning feel about the training?

Learning- What knowledge did the learner gain?

Behaviors- What skills did the learner develop?

Results and effectiveness- What results occurred? Did the learner effectively apply

the new skills to the necessary tasks in the organization? (Alex et al., 2005, p. 38).

Flowchart Safety Training Model. The Flowchart Model for safety training was

designed to be tailored to an organization’s needs but complete enough to include all major

training and development components which provided greater adaptability to the topic of

volunteer training (Jensen, 2005). Using the Occupational Safety and Health Administration

(OSHA) guidelines and published documents from the American National Standards Institute

(ANSI), a comparison was conducted which validated key training components, terminology,

and descriptions within the model (Jensen, 2005). The Flowchart Model by Jenson (2005) was

compromised with four key questions needed to be asked in the development of training; Is

18

training needed? Will pre-training evaluation data be collected? Will training be repeated?

Should the program be improved (p. 26-32)?

Jensen (2005) gathered input from stakeholders and trainees to determine training need

(Jensen, 2005). The creation of learning objectives not only included defining the knowledge

and skills to be obtained once training was completed but also incorporated abilities, attitudes,

and behavior that a trainee would be able to demonstrate (Jensen, 2005). With each defined

learning objective, a corresponding learning activity was developed based on adult learning

principles (Jensen, 2005).

The evaluation stage of the model required the collection of pre-training data. Jensen

(2005) required that an assessment be created in order to evaluate the impact of the training

course on the learner. Common assessment techniques included written or skill tests to obtain

the data needed for determination of the effectiveness of the training and successful replication

of training (Jensen, 2005).

Finally, the training model determined, whether the training program was to be

continued, terminated, or improved upon using the 4 level evaluation technique developed by

Kirkpatrick that also mimicked the previously discussed model’s evaluation technique (Jensen,

2005; Kirkpatrick, 2006). While the Flowchart Model of Safety Training was designed to be

flexible and adaptive to organization needs, the model had been designed for occupational safety

and health training in organizations.

On the Job Training (OJT). On the job training (OJT) has been found to beneficial for

small to medium sized businesses that cannot afford the financial means and staff to warrant a

full training program (Rothwell & Kazanas, 2004; Lawson, 1997). According to Lawson (1997)

structured OJT programs improved team building and helped developed cohesion in small

organizations when workgroups were trained together. In addition, OJT conducted onsite was

found to have greater relevancy for trainees and as a result demonstrated increased competency

19

and mastery of job tasks, demonstrating cost savings for smaller organizations (Lawson, 1997).

Research has shown that the quality of an organization’s OJT program has a direct correlation to

the length of time for new employees to train, turnover, and job satisfaction (Lahti, 2002;

Winkler & Janger, 1998).

Two types of OJT were defined in the OJT model: unplanned and planned OJT (Rothwell

& Kazanas, 2004). Unplanned OJT, consisted of new employees expected to learn by watching

experienced workers perform or by doing the work themselves (Rothwell & Kazanas, 2004, pg.

155). Unplanned OJT historically has not been organized or developed with the job performance

needs of the learner but has been driven by work demands and crises. Planned OJT, the second

type of OJT, has consisted of planned instruction occurring on the job and during work, centered

on what workers needed to know or do to perform competently (Rothwell & Kazanas 2004, pg.

154).

Planned OJT was structured around the needs of the learner, importance of activities, and

forecasted work results or outcomes (Rothwell & Kazanas, 2004). The DAPPER Model

developed by Rothwell and Kazanas (2004) was created as a guide to developing planned OJT.

DAPPER was the acronym made of the first letters in every step of the model:

Discovering needs for planned OJT

Analyzing work and workplace for OJT

Preparing planned OJT

Presenting planned OJT

Evaluating the results of OJT

Reviewing aids to planned OJT (p. xviii)

Discovery needs for planned OJT. Rothwell and Kazanas (2004) developed the

following 17 questions to determine the need for planned OJT.

1. Is the individual new to the organization?

20

2. Is the individual new to the division or department?

3. Is the individual new to the work unit

4. Is the individual new to the job?

5. Does the individual lack knowledge of the essential job functions?

6. Does the individual lack ability to perform essential job functions?

7. Does the individual lack knowledge of when to perform?

8. Does the individual lack knowledge of measureable job performance standards?

9. Does the individual lack knowledge of why the job functions are important?

10. Does the individual lack knowledge of where job functions should be performed?

11. Does the individual have a different learning need from the group?

12. Have job functions changed or are they about to change?

13. Have performance obstacles in the work environment been removed?

14. Is OJT more cost effective than other solutions?

15. Can workplace distractions be minimized?

16. Can workplace health and safety hazards be minimized?

17. Can adequate time, staff and other resources be devoted to OJT? (p. 47-57)

Analyzing work and workplace. The analysis of work followed a 6 step approach

created by Rothwell and Kazanas (2004) of gathering information for creation of job

descriptions. Recommendations for gathering information included the use of the Occupational

Information Resource Network or O’Net to view previously created job descriptions and their

associated job functions (Rothwell & Kazanas, 2004). Work analysis information should also be

gathered through observation, individual interviews, group interviews, or written questionnaires

from those that had performed the job (Rothwell & Kazanas, 2004). Rothwell and Kazanas

(2004) recommended the DACUM model of curriculum development be used for the creation of

21

job descriptions. Halasz and Reid (2003) further reinforced this recommendation by showing

that the DACUM model has been a cost effective tool.

The workplace analysis was designed to obtain information about the environment

needed so that trainees could carry out their specific functions competently. Burke and Baldwin

(1999) showed support for the workplace analysis in a study that found that work environments

affected trainees’ ability to apply their knowledge, skills, and abilities back to their jobs after

training had concluded (as cited by Springs, 1997). Further research by Chiaburu, Van dam, and

Hutchins (2010) demonstrated that a work environment that provided perceived organizational

and supervisory support were positively related to a trainee’s self-efficacy and motivation to

learn.

Preparing the OJT plan. The first step in creating the OJT plan involved breaking down

the task to be trained into steps or subtasks. According to Lawson (1997);

Breaking large tasks into smaller ones prevents the trainee from feeling overwhelmed and

makes it easier for the trainee to master by increasing understanding and proficiency one

step at a time. Subtasks lend themselves to smaller incremental goals, giving the trainee

an opportunity to experience frequent and multiple successes along the way (pg. 29).

Part of effective training according to Lawson (1997) was defining performance standards or

what the learner accomplished after the training. These standards of performance needed to be

observable, measurable, and attainable (Lawson, 1997). An effective tool to use in identifying

appropriate performance standards was developing a job/task analysis chart (Lawson, 1997).

The chart identified 3 components with each containing their own column; the what, how, and

why of each step in a task (Lawson, 1997). The approach used by Rothwell and Kanzanas

(2004) was referred to as “Topical Sequencing.” The authors process involved asking

experienced performers to clarify the order in which functions should be taught based on the

novice performer needed to understand (pg. 78).

22

Creating Instructional Materials. A checklist was created by TRHRD 740 (2012)

graduate class on a previous volunteer development project with Refuge Farms, LLC and has

been adapted for use in this study (University of Wisconsin-Stout). A job training checklist

recommended by Rothwell and Kazanas (2004), listed the essential job functions, standards for

assessing how well the learners performed, and used as a feedback tool to the learners (Rothwell

& Kazanas, 2004). Rothwell & Kazanas, (2004) also proposed the use of a lesson plan that

summarized who will receive instruction, what they would learn, where the lesson would be

conducted, and how learners would be assessed on what they learned. In addition, the lesson

plan provided objectives, needed materials, the way the lesson complemented other training, and

the date the lesson would completed (Rothwell & Kazanas, 2004, pg. 83)

Evaluating the Results. The evaluation technique Rothwell and Kazanas (2004) adopted

in the OJT model was developed by Kirkpatrick (1997) and followed 4 levels of evaluation that

consisted of: (1) evaluating reactions; (2) evaluating learning; (3) evaluating behavioral change;

and (4) evaluating organizational results.

In concluding the literature review of different training models, the OJT model by

Rothwell and Kazanas (2004) was found to be the most suitable because of the evidence that it

was found to be beneficial for small to medium sized businesses that cannot afford the financial

means and staff to warrant a full training program. This model was also the most adaptable to

the organization because the learning practices of the organization at the time of this study

already closely resembled OJT methods.

Training Evaluation for Adoption in the Nonprofit Sector

Kirkpatrick Four Level Evaluations. According to Kirkpatrick (1997) and Lawson

(1997), the use of a questionnaire to gain insight into whether the training was favorable to the

trainee participants was recommended. Rothwell and Kazanas (2004) differed in their

recommendation and instead received feedback about the training by conducting one on one

23

feedback sessions with trainees. The authors opined that OJT provided ample opportunities for

feedback to occur during and after the training because trainings were limited in size and often

occurred in individual or small group settings typically (Rothwell & Kazanas, 2004).

Kirkpatrick’s (2006) evaluation method was designed to indicate whether the desired

knowledge, skills, or attitudes were retained by the learner (p. 42). Kirkpatrick (2006) stated that

evaluation process should not take place until the trainees have had an opportunity to apply what

they have learned in the job setting where Lawson (1997) argued that evaluation should take

place within 24 hours of the training. Typical tools used in the evaluation process included,

interviews, written performance appraisals, or surveys (Rothwell & Kazanas, 2004).

Kirkpatrick’s Model also evaluated results of training on the overall effect on the organization.

Typically, for this evaluation to occur the results of the training needed to be linked to business

goals with an analysis of how the results of what was learned affected the profitability of the

organization (Kirkpatrick, 2006; Lawson 1997).

Success Case Method (SCM). According to Brinkerhoff (2003), the Success Case

Method (SCM) was designed to be a simple way of determining the success of new

organizational initiatives. Brinkerhoff (2003) suggested that this model was suitable for

organizations that do not have sufficient time or money for traditional evaluation methods (p.

viii). Brinkerhoff (2003) developed a 5 step process for planning and implementation of the

success case method.

1. Focusing and planning a success case study

2. Creating an “impact model’ that defines what success should look like

3. Designing and implementing a survey

4. Interviewing and document success cases

5. Communicating findings, conclusions, and recommendations (p. 29)

24

The principle objective of step 1 in Brinkerhoff’s (2003) method was to, “clarify and

understand what the study needs to accomplish” (p. 29). In order for the objective to be fulfilled,

the requisite pieces of the study need to be planned, including stakeholders and participants

(Brinkerhoff, 2003). Step 2 required the creation of an “impact model”, a projection of what

success would look like if the initiative was working correctly (Brinkerhoff, 2003, p. 33). The

“impact model” was used as a base for developing the survey in step 3 to ensure that the correct

information was being collected from the selected participants (Brinkerhoff, 2003). The survey

was designed to identify those cases that were successful according to what had been established

in step 1 of the method. Step 4 participants were interviewed and documentation of those cases

was conducted. According to Brinkerhoff (2003) this was the most time consuming component

steps, but yielded the most valuable information. With step 5 the results and recommendations

were communicated to the stakeholders identified in earlier steps of the method (Brinkerhoff,

2003).

The review of the literature identified that the Success Case Method was the evaluation

technique best suited for this study because of the minimal time commitment as well as low cost.

Brinkerhoff’s (2003) success case method determined if the training structure was working, what

was not, and what could be improved upon from the volunteer participant’s perspective. This

evaluation technique determined whether the OJT model for training was well received.

Summary

Chapter 2 provided a review of the literature within the categories of training and

evaluation models. Chapter 3 will provide an overview of the methodology used to collect data

and the technique used for analysis. Chapter 4 will provide the result of this study. Chapter 5

will provide a discussion of the results and recommendation for future studies.

25

Chapter III: Methodology

The purpose of this study was to develop a training program from previously researched

training models for a nonprofit that has not had the benefit of a structured development plan for

its volunteers and to evaluate success among volunteer participants at Refuge Farms, LLC using

a survey to collect qualitative data. Benefits of development plans in organizations that rely on

volunteer workforces can include: increased volunteer satisfaction, higher retention, and provide

volunteers with the skills to perform their task competently and safely (Phillips & Little &

Goodine, 2002; Grossman & Furano, 2002; Alex et al., 2005).

Subject Selection and Description

The organization supplied a letter of invitation to conduct this study (see appendix A).

The sample size for this study was 8 volunteers of Refuge Farms, LLC participating in the

implementation of the training program (see Appendix C). As stated previously in the limitation

of this study, the organization did not require mandatory involvement in the development plan

which provided variability in the sample size.

Instrumentation

The basis for training content is derived from the foal, yearling, and alpha checklist

created from the 2012 Management and Coordination of Training and Development class at the

University of Wisconsin Stout (see Appendix D). A electronic questionnaire was developed and

distributed at the conclusion of training via email using the Qualtrics™ system at UW-Stout.

Survey questions were designed in reference to length of time as a volunteer, prior experience

with horses, response to training, and recommendations for improvement through open ended

questions with corresponding comment sections (see Appendix E).

Data Collection Procedures

At the beginning of the selected training class for evaluation, the purpose of the study and

utilization of results were explained to participants. It was furthered explained that participation

26

was voluntary and would have no effect on their development as a volunteer in the organization

or their ability to remain for the training session. Volunteers were provided with implied consent

and were assured that all surveys would remain confidential and anonymous only to be viewed

by the researcher with recommendations shared with the Executive Director of the organization

(see Appendix B). The signed consent form was provided as the first question of the electronic

survey and participants needed to answer ‘yes’ in order to continue. To ensure anonymity of the

surveys, participants completed the survey in the privacy of their own homes without the

Executive Director present.

Data analysis

Data analysis involved both quantitative and qualitative analysis. Quantitative analysis

involved reviewing the completed surveys and analyzing frequency of responses. Qualitative

data analysis identified common themes and subthemes from participant’s answers to open ended

questions. The themes were organized into categories and subcategories, providing meaning to

the participant’s feelings in regard to the training.

Limitations

1) Lack of qualified volunteers to conduct training with sole responsibility lying with

the Executive Director of the organization

2) No previous training budget information has been established by the organization.

3) Limited number of horse handling classes available per year.

4) Lack of training models specific to non-profit volunteers except for leadership

positions (Cosier & Dalton, 1993; Deslandes & Rogers, 2008).

5) Volunteer engagement in the volunteer training program may be a limitation because

participation in the training program has not been determined to be mandatory by the

organization.

27

6) Sustainability for the program may not occur due to the organization lacking financial

resources and qualified volunteers to maintain the program.

Summary

Chapter 3 identified the proposed methods for this qualitative study. A description of

subjects, sample size, and recruitment methods were proposed. The framework for data analysis

was discussed and limitations identified. Chapter 4 will provide an overview of results of this

study.

28

Chapter IV: Results

The purpose of this study is to determine if the introduction of a formalized training

program would have a positive impact on the volunteer workforce of Refuge Farms, LLC in

acquiring new skills and abilities with regards to horse handling. Using the Success Case

Method developed by Brinkerhoff (2003), the Refuge Farms, LLC horse handling classes were

evaluated using a 9 question electronic survey developed in Qualtrics™ that included 3

quantitative questions and 6 qualitative questions. A list of the questions from the survey is

below.

1. How long have you been a volunteer at Refuge Farms?

2. List any prior experience in handling horses that you have had prior to taking the horse

handling classes at Refuge Farms.

3. Do you feel the length of training was adequate time for you to practice what the trainer

demonstrated?

4. In which ways do you feel the horse handling classes will have a positive impact on your

development at Refuge Farms?

5. Did the horse handling class or classes provide you with skills that you did not previously

have in terms of horse handling?

6. Please list the skills that you have acquired from the horse handling class or classes that

you have attended?

7. What did you like most about the horse handling classes?

8. Is there anything that you would like to see in future horse handling classes that are held

that was not present in the current classes?

9. Please include any additional comments regarding the horse handling classes.

Due to the limited number of volunteers who participated in the training classes, 8

volunteers were sent the survey. With a 100% response rate, the study concluded with a total

29

sample size of N=8 responses. The limitation of this small sample size is apparent in the analysis

of the qualitative data, where a limited number of themes were identified from participant’s

responses.

Quantitative Data

Quantitative data was collected in order to determine: 1) the length of time volunteers had

been with Refuge Farms, LLC; 2) whether they felt that the time allotted for the training was

enough to practice; and 3) if they had acquired new skills related to horse handling. Quantitative

questions were dispersed at random intervals throughout the survey. Responses are represented

below in tables 1, 2, and 3 with the frequency of responses for each quantitative question in the

survey listed as well as percentages.

Table 1

Question 1- Duration of Volunteering at Refuge Farms

# Answer Response %

1 0-2 months 0 0%

2 3-5 months 4 50%

3 6-12 months 0 0%

4 1 year or more 4 50%

Total 8 100%

The participants were asked to indicate how long they had been volunteering at the

Refuge Farms, LLC. The question had 4 possible answers ranging from 0-2 months, 3-5 months,

6-12 months, and 1 year or more (see table 1). The participants were evenly divided with 50%

(n=4) of the participants volunteering at Refuge Farms, LLC from 3-5 months, and 50% (n=4)

answering they had been a volunteer for 1 year or more.

30

Table 2

Question 3- Training Time

# Answer Response %

1 Yes 7 88%

2 No 1 13%

Total 8 100%

The training model used for Refuge Farms, LLC contained hands on training by the

trainees. The second quantitative question was to determine if the time allotted for the training

sessions was adequate for participants to learn the skill and practice what the instructor

demonstrated. Of the participants, n=7 answered that the time was adequate and n=1 answered

that it was not (see table 2).

Table 3

Question 5- Skills Provided

# Answer Response %

1 Yes 7 87.5%

2 No 1 12.5%

Total 8 100%

The final quantitative question in the survey determined if participants felt that the

training classes provided new skills in horse handling knowledge and techniques. The majority

participants at 87.5% answered that they gained skills. The remaining 12.5% stated they had not

(see table 3).

Qualitative Data

Qualitative data was included in the survey in order to determine 1) what prior

knowledge participants had before taking the horse handling classes; 2) how the offered classes

would have a positive impact on their development as a volunteer; 3) what skills they acquired

and 4) what they would like to see if any future classes are offered. The final question of the

31

survey was open for any additional comments. Five themes were identified as; (1) varying levels

of experience with subthemes of limited experience and horse ownership; (2) knowledge transfer

with subthemes of motor skills and cognitive skills; (3) positive reception to the training model

with subthemes of hands on training and relationship building; (4) motivation to learn with

subthemes of desire to learn and success through others; and (5) continuous improvement of the

program with subthemes of defined expectations of volunteers, certification, and testing. Listed

below are the themes and subthemes identified during data analysis.

Varying levels of experience

o Limited experience

o Horse ownership

Knowledge Transfer

o Motor skills

o Cognitive skills

Positive reception to training

o Hands on training

o Relationship building

Motivation to learn

o Desire to learn

o Success through others

Continuous improvement of the program

o Defined expectations of volunteers

o Clarify progression of the program

o Certification and testing

32

Varying levels of experience. Participant’s prior experiences with horse handling

ranged from 50% stating they had limited experiences to 50% of the volunteers having horse

ownership experience. This resulted in recurrent subthemes highlighting limited experience and

ownership levels. Volunteer responses with limited previous experiences ranged from one

participant answering, “None” while another stated, “Brushing horses.” Other respondents spoke

about helping out family members that owned horses providing examples such as, “Cousins had

horses growing up, maybe rode on them (with adult leading us around the pasture) a handful of

times.” These experiences have given some of the sample contact with horses, but experiences

still lacked horse handling skills that were taught during Refuge Farms, LLC classes. Individuals

that stated they had much more experience listed horse ownership and horse training as their

previous experiences, reinforcing the varying levels among the sample as this comment

indicates.

“Horse ownership started in junior high where I boarded a single horse and participated

in gymkanas. By high school I owned and maintained two adult horses and a foal with

my family on a hobby farm. I continued showing with very accomplished horse handlers

for several years who took me under their wings…”

Knowledge transfer. Of the participants that attended the horse handling classes, 87.5%

responded they had learned skills due to the training classes. These findings showed subthemes

of knowledge transfer that included either motor skills or cognitive skills acquired as a result of

the training classes. Through such self-reporting, a link can be established between knowledge

transfer and the participants in the survey. A large percentage (50%) of the sample listed horse

commands and basic horse care as a skill they acquired as a result of the Refuge Farms, LLC.

This is delineated by the following comment, “Basic rules of how to make a horse do what you

want” and, “Learning the proper language and commands.” Others listed some basic horse care

33

skills such as, “blanketing and feeding” and, “haltering, leading, hoof care, blanketing…

methods of feeding…commands and grooming.”

Motor and cognitive skills were recurring subthemes among participant responses in

terms of knowledge transfer. Participants spoke of increased cognitive skills learned during the

classes. One participant spoke of increased awareness he/she received in dealing with blind

horses after attending the classes,

“I’ve never worked with blind horses before. There are so many additional

considerations that must be taken into account when handling a blind horse rather than a

sighted horse. One must earn their trust. I have also acquired quite a bit of knowledge

when it comes to equine diseases, defects, and remedies.”

The remaining participants (50%) stated that they found an increased confidence level in

dealing with the horses or the herd when asked how it would have a positive impact on their

development. According to one participant, “I feel more confident and relaxed around the herd.

I have the basic skills to take on more responsibility.”

Positive reception to training. The structure of the training included hands on practice

and was a reoccurring subtheme with 50% of the comments stating, “Hands-on Training,”

among comments to question 7in the survey. This was followed by individuals writing that the

relationship building that occurred was the most liked aspect of the training. This was explicated

by, “The friendships formed with other volunteers and the opportunity to help each other. I also

have a stronger relationship with the horses.” Several individuals also included both subthemes

in their responses such as this comment made by one participant, “the camaraderie, working with

the horses, watching the volunteers advance, utilizing the expanding skills with the horses.”

Motivation to learn. All participants of the sample (N=8) stated that the horse handling

training classes increased the motivation to learn. This theme resulted in the following

subthemes: 1) desire to learn and 2) success through others. Participants spoke about their

34

increased desire to learn through comments such as, “[Was] the best summer spent” and

“regardless of what the subjects that were taught, I found myself wanting to learn even more.”

This subtheme was reinforced with the recommendations provided from volunteers on what type

of content they would like to see in future classes such as, “Learning about what diseases and

sickness horses can get and how to treat them,” and “I would like to learn more about the current

nutrition trends. Other areas of learning that were identified were: 1) learning why specific feeds

were chosen for certain horses, and 2) different hay types given to the horses based on nutrition

needed.

One individual found success in the value and expanding knowledge the horse handling

classes provided to volunteers. This highlighted the second subtheme: success through others.

The respondent stated that the importance of the horse handling classes fostered consistency

among volunteers, and identified success from watching others expand their skills. This

volunteer provided the following comment,

“I have owned horses for many years, and currently have several. I enjoyed the horse

handling classes as they reinforced the basics of horse handling, covering methods to be

performed Refuge Farms so all volunteers [perform] acts the same so as to not confuse

the horses. I watched other volunteers expand and grow their horse knowledge and enjoy

their new skills.”

Continuous improvement of the program. Several comments were received on

recommendations from volunteers on how to improve the training classes at Refuge Farms, LLC.

Participants comments about opportunities for improvement for the training program resulted in

the following subthemes: 1) establish clear expectations of volunteers taking the horse handling

classes and 2) clarify the full progression of classes and the connection to volunteer development

and responsibility levels, and 3) consider testing and certification processes for volunteers at

35

Refuge Farms, LLC. One participant had this to say about the confusion and lack of connection

to development,

“It could be more clear what the full progression/skill set- we are told there are so many

credits/class[es], but how many overall are we to get? We are told it’s for new volunteers

or those working for a foal/yearling/etc. designation. Is everyone on this track? Are

some already at some level or another? If we aren’t actively working for a designation

because that’s not important to use personally to achieve levels or credits, are we still

expected to come?”

One comment in particular outlines the incorporation of a certification and testing in the trainings

to create motivation for volunteers. The following comment explicates this subtheme,

“I would like to see us use a certification for those that have successfully completed all 5

classes. You cannot get the certification unless you have completed all 5 classes. Almost

like a punch card process. More advanced classes would be for those with that

certification and not just open to all. Gives something for people to work towards, an

elite status. [I] would provide a test scenario to create performance metrics.”

Other recommendations for improvement included having each volunteer work with one horse to

build a better relationship with them, reduce side talking, and have a bigger area for the training

classes.

Summary

Chapter 4 provided an analysis of the results that were received from the survey that was

distributed to the participants who attended the Refuge Farms, LLC horse handling classes from

July 2012 to September 2012. The results included both quantitative and qualitative data.

Common themes and subthemes were presented from the participant’s comments to the 6

qualitative questions. Chapter 5 will provide a discussion of the results, conclusions, and

recommendations for this study.

36

Chapter V: Discussion

This study consisted of conducting a literature review of training models to determine

which model was most suitable for the creation of a training development program to be used by

Refuge Farms, LLC. The models reviewed included the Faith Based and Community Initiative

Model (Alex et al., 2005), the Flowchart Safety Training Model (Jensen, 2005), and the OJT

Model (Rothwell & Kazanas, 2004). Volunteers that participated in the training program for

summer of 2012 were surveyed using the Success Case Method developed by Brinkerhoff

(2003). Qualitative and quantitative survey data was analyzed. Data was presented graphically

and themes and subthemes were presented and explicated by the samples (N=8) responses to

open ended questions.

Limitations

1) Lack of qualified volunteers to conduct training with sole responsibility lying with the

Executive Director of the organization.

2) No previous training budget information had been established by the organization.

3) Limited number of horse handling classes available per year.

4) Lack of training models specific to non-profit volunteers with the exception for

leadership positions (Cosier & Dalton, 1993; Deslandes & Rogers, 2008).

5) Limited volunteer engagement in the volunteer training program because

participation in the training program had not been determined to be mandatory by the

organization.

6) Sustainability for the program may not occur due to the organization lacking financial

resources and qualified volunteers to maintain the program.

At the conclusion of the study, all assumed limitations were present with the exception of

the final limitation regarding the sustainability of the program. This limitation would need future

research to determine if the program is continued in Spring/Summer 2013. The most

37

pronounced limitation with the largest effect was the lack of volunteer engagement in the

training program. While the Executive Director stated that volunteer classes were needed to

handle horses without supervision, such classes were not determined to be mandatory for

volunteers. As a result, only a small group of the volunteer workforce attended the horse

handling classes between June and August 2012. This may have occurred as a result of the lack

of importance communicated to the volunteer workforce; resulting in the small sample size for

data collection.

A new limitation revealed during the study was the lack of identification and

communication of volunteer expectations and an unclear definition of the training program to the

volunteer workforce by the Executive Director. The analysis of results demonstrates many

participants expressed confusion regarding the expectations of attending the training classes,

connection to volunteer development and responsibility levels, and alternatives if volunteers

were not interested in advancing current skills. Other respondents indicated a preference for the

progression of the classes to be more clearly explained and linked to understanding the

advancement from foal to yearling volunteer responsibility status. As a result, volunteers may

have questioned the benefit of the horse handling classes and the relationship to development and

responsibility levels as a volunteer of Refuge Farms, LLC.

Conclusions

The Faith Based and Community Initiative Model (Alex et al., 2005) and Jensen’s

Flowchart Training Safety Model (2005) reviewed in chapter 2 could neither be supported or

negated when analyzing the results of this study. This is likely a result of the training models not

having previous research conducted in terms of application to training in a nonprofit setting.

This holds true for the evaluation techniques that were also discussed in this study. The study’s

results do not support the use of Kirkpatrick’s 4 level evaluation method (2004) over the Success

Case Method (Brinkerhoff, 2003). Both the different training models and the two evaluation

38

models allow opportunities for further research to be conducted with larger sample for both

training models and evaluation methods

Despite the small sample size (N=8), comments provided by the sample population were

positive regarding the reception of the training structure; explicated by 100% of the participants

listing at least one aspect of the training they enjoyed most. Approximately 50% of respondents

stated they enjoyed the hands-on aspect of the training the most. This confirms that the use of

Rothwell and Kazanas (2004) OJT structure for this study was most applicable over all other

models discussed in chapter 2. This conclusion is further reinforced by participant’s relationship

building listed as an aspect of the training they enjoyed; coinciding with Lawson’s (1997)

literature stating that OJT models improved team building and promoted cohesion in small

organizations.

According to Aswegen et al., (2008) OJT has benefits for all levels of employees in an

organization from front line employees to upper management. The findings of this study are

supported by this literature by one participant stating that while they had not gained any new

skills as a result of the horse handling classes, they still felt the classes were a benefit to them by

being, “remedial and a refresher.” This is an indication that the OJT structure of the classes had

benefit to all participants regardless of the amount of previous experience held; promoting the

use of Rothwell and Kazanas’s (2004) model for this study.

Literature from chapter 2 indicated that because OJT was centered on the trainee and

occurred on the job it contained greater relevancy to the trainee, resulting in increased

competency and mastery of job tasks (Lawson, 1997; Rothwell & Kazanas, 2004). According to

the data collected, the majority of the sample (87.5%) stated they had gained new skills from the

horse handling classes. The skills identified by participants were those that involved

demonstration and hands on practice by the trainees as established by the OJT model. As a

result, a correlation can be made between the use of the OJT model and increased skill levels of

39

the study participants, which supports Lawson’s (1997), and Rothwell and Kazana’s (2004)

statement.

In order to prepare an effective OJT model, performance standards or what the learner

accomplished after training needed to be clearly defined (Lawson, 1997). Performance standards

had been established with the Evaluation Form/Checklist (see appendix D) but results of the

survey showed that there was confusion among the sample with regards to volunteer

expectations and progression of participants through the program. This indicates there was a

lack of communication to volunteers regarding performance standards at the beginning of the

program which may have impacted the overall effectiveness of the program because volunteers

had no standards to self-evaluate their position in the program.

Recommendations

The study’s results show that volunteers recommended an evaluation system to increase

volunteer motivation and engagement in the development and progression of the training

program. Refuge Farms, LLC would benefit from utilizing the Evaluation Form/Checklist (see

appendix D) for this purpose as well as to establish performance standards for each level of the

training program. This would provide objective measurement of performance goals that

volunteers would need to meet in order to successfully advance through the program.

Implementation of the Evaluation Form/Checklist (see appendix D) to observe whether the

trainees could successfully demonstrate performance standards would also provide supporting

data of whether knowledge transfer occurred (Kirkpatrick, 2006; Rothwell & Kazanas, 2004).

Such evaluation would occur at the end of each class, providing documentation of achievement

for trainees to progress to the next volunteer level (i.e. foal to yearling). The use of the

evaluation checklist would allow volunteers to see the value of the classes and document their

progression through the development program in real time by receiving immediate and

measurable feedback and coaching from the facilitator of the training. Such implementation

40

could also be used as the foundation for the development of a certification process for the

volunteer workforce.

To support the mission and objective of the training program, the Evaluation

Form/Checklist could be used. By clearly defining the performance standards and ensuring they

support the mission of Refuge Farms, LLC the development program would act as reinforcement

to volunteers of the mission of the organization. This may increase the volunteer engagement in

the program as well as create a personal attachment by volunteers to the success of the program

because it aligns with the culture of the organization and their development.

The lack of volunteer expectations was identified as an opportunity for improvement by

the respondents of this study. This may be related to the lack of introduction the volunteers at

the start of the program as well as poor communication throughout the program by the executive

director of the organization. Volunteers stated they were uncertain about their individual

progression of the program, place in the volunteer development process, and connection between

development of volunteer skills and responsibility levels. Proper introduction and explanation of

the program to volunteers with volunteer onboarding and orientation would facilitate dialog and

learning while decreasing uncertainty and confusion related to volunteer expectations and the

volunteer development process (Alex et al., 2005). Ongoing communication during trainings

regarding the programs importance, incentives, and certification would also increase motivation

and volunteer engagement for Refuge Farms, LLC.

The results of this study indicate a desire and motivation to learn. Refuge Farms, LLC

would benefit from offering more horse handling classes throughout the calendar year. In order

to overcome inclement weather during the winter months, classes could be held in a heated

indoor space. By designating more highly skilled volunteers as trainers to conduct classes, more

horse handling classes could be offered to the volunteer workforce by the volunteer workforce,

once again increasing the engagement level among volunteers while leveraging the volunteer

41

talent base. In addition, this would decrease the workload of the Executive Director. The

Executive Director would be able to focus on the higher level needs of the Refuge Farms, LLC

organization, such as grant writing to support additional paid volunteer positions, such as

training and fundraising personnel.

Paid training personnel would provide faster development of volunteers due to a greater

number of skills taught in the same period of time. Volunteers would also benefit from this

recommendation as make up sessions could be scheduled for those volunteers that could not

attend the original classes. The added training or mentoring personnel would provide an

opportunity for documentation and analysis of volunteer’s performance metrics. This would

provide opportunities for feedback and coaching; further contributing to the development of the

volunteer to advance from the foal to yearling levels of volunteerism. These recommendations

as well as mandating classes would increase participation in the training program.

Future research

Implementation of the recommendations of this study would contribute to future research

to determine if increased volunteer engagement occurred. Increasing volunteer engagement

would result in a larger sample size. A replication of this study could be conducted with a a

larger representation of the volunteer workforce by Refuge Farms, LLC and would provide

opportunities for continuous improvement of the volunteer training program.

Future research is recommended to establish and determine the attrition rate of

volunteers. This is relevant for additional classes and topic offerings. Research has shown that

implementation of a training program has correlation to the reduction of turnover and burn out

rate of volunteers at nonprofit organizations (Kerka, 2003). A study could be conducted to see if

the implementation of the training classes in this study had positive influence on Refuge Farms,

LLC ability to retain volunteers from the previous year’s classes.

42

Future research on the implementation of the evaluation system to measure the

competency of the trainees after training has concluded would support whether knowledge

transfer had occurred. While the sample in this study self-reported that they had learned new

skills, no evaluation system was used to collect data to support that skills were in fact properly

transferred to the trainees. This study could include the use of the Evaluation Form/Checklist at

the conclusion of each class and analysis of such data could assist the organization to identify

problem areas inhibiting volunteers from learning or skills that need additional training.

Research could also be conducted to measure if the Evaluation Form/Checklist decreased

confusion in terms of volunteer advancement, progression of the program, increased motivation,

and volunteer engagement.

Research could be done to measure the impact of introducing mentors to the volunteer

workforce to enhance the experience of future volunteer training classes. The volunteers that

participated in this study could act as mentors to new volunteers entering the organization and

the training program. For example, participants in this study could be engaged as mentors for

new volunteers who have not been introduced to horse handling techniques at Refuge Farms,

LLC. A study could then be conducted on the new volunteers who were mentored with horse

handling skills to see if mentoring had a positive impact on their development versus solely

attending horse handling classes.

Volunteers acting as mentors could increase volunteer engagement in the training

program as well and allow more volunteers to be trained, reducing the workload on the

Executive Director. In turn, the executive director could engage in grant writing to sustain the

training program and allow for paid training or mentoring staff. A follow up training evaluation

could be conducted to determine the effect of the mentors on volunteer engagement in the

training programs.

43

Summary

Chapter 5 discussed the findings of the evaluation of the horse handling classes against

the review of the literature which included the Faith Based and Community Initiative Model

(Alex et al., 2005), the Flowchart Safety Training Model (Jensen, 2005), and the OJT Model

(Rothwell & Kazanas, 2004). A new limitation revealed during the study was the lack of

identification and communication of volunteer expectations and an unclear definition of the

training program to the volunteer workforce by the organization. The study concluded with

several opportunities for additional research; such as, implementing recommendations to

increase participation, thereby increasing sample size; determining the attrition rate of volunteers

involved in this study; implementation of an evaluation process; and the effect of introducing

paid training staff or mentors to the volunteer workforce. The study’s results will be reviewed

with the executive director of the organization to assist in the development and sustainability of

volunteer development at Refuge Farms, LLC. A special thank you to Sandy Gilbert and the

volunteers of Refuge Farms, LLC for allowing this study to be conducted.

44

References

Adult learning principles. (n.d.). In Dictionary online. Retrieved from

http://www.reference.com/browse/Andragogy

Alex, G., & Ennis, M., & Lorrain, C., & McGhee, J., & Peterson, D., & Powers, D., & Styers,

D., & Thompson, N., & Warder, M., (2005). Successful strategies for recruiting,

training, and utilizing volunteers: A guide for faith and community based service

providers [Adobe PDF]. Retrieved from www.samhsa.gov

Aswegen, S., & Banhegyi, S., & Botha, S., & Crafford, A., & Kleynhans, R., & Markham, L.,

& Meyer, W., & Moerdyk, A,. & O'Neill, C., & Schlechter, A., & Train, B. (2008).

Fresh Perspectives: Industrial Psychology 112. Cape Town: Pearson Education

American National Standards Institute (ANSI). (n.d.). In Dictionary online. Retrieved from

http://dictionary.reference.com/browse/American+National+Standards+Institute

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral Change.

Psychological Review, 84(2), 191-215

Birdi, K. S., & Patterson, M. G., & Wood, S. J. (2007). Learning to perform? A comparison of

learning practices and organizational performance in proft- and non-profit-making sectors

in the UK. International Journal of Training and Development,11(4), 265-281

Brinkerhoff, R. O. (2003). The success case method. San Francisco: Berrett-Koehler.

Chiaburu, D.S., & Van Dam, K., & Hutchins, H.M. (2010). Social support in the workplace and

training transfer: A longitudinal analysis. International Journal of Selection and

Assessment, 18(2), 187-200. Retrieved from

http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?vid=23&hid=8&sid=961

0b867-e7e6-4491-9154-1f9fea5608b2%40sessionmgr10

Cosier, R.A., & Dalton, D. R. (1993). Management training and development in a nonprofit

organization. Public Personnel Management, 22(1), 37-32, Retrieved from

45

http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?vid=2&hid=24&sid=c78

c6ead-24a7-44c8-891c-e557dff9bd47%40sessionmgr11

Cnaan, R., & Cascio, T. (1999). Performance and commitment: Issues in management of

volunteers in human service organization. Journal of Social Service Research, 24(3/4), 1-

37. Retrieved from

http://repository.upenn.edu/cgi/viewcontent.cgi?article=1000&context=spp_papers

DACUM (n.d). What is dacum?. [Online resource for occupational analysis]. Retreived from

http://www.dacum.org/index.asp

Deslandes, M., & Rogers, L. (2008). A volunteer framework. Australian Journal of Adult

Learning, 48, 355-368. Retrieved from http://www.eric.ed.gov/PDFS/EJ809990.pdf

Foal. (n.d.). In Dictionary online. Retrieved from

http://dictionary.reference.com/browse/Foal?s=t&ld=1032

Gasaway, R., (2003). I’m a volunteer you can’t tell me what to do. Fire Engineering, 156(12),

12-14. Retrieved from

http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?sid=9610b867-e7e6-

4491-9154-1f9fea5608b2%40sessionmgr10&vid=9&hid=8

Grossman, J.B., & Furano, K. (2002, July). Making the most of volunteers. P/PV Briefs.

Retrieved from http://www.ppv.org/ppv/publications/assets/152_publication.pdf

Halasz, I. M., & Reid, T. (2003). Overview of dacum job analysis process. NIC Academy

Division. Retrieved from http://static.nicic.gov/Library/010699.pdf

Hidalgo, M. C., & Moreno, P. (2009). Organizational socialization of volunteers: Effect on their

retention to remain. Journal of Community Psychology, 37, 594-601.

doi:10.1002/jcop.20317

Jensen, R., (2005). Safety training: Flowchart model facilitates development of effective courses.

Professional Safety, 50(2), 26-32. Retrieved from

46

http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?sid=9610b867-e7e6-

4491-9154-1f9fea5608b2%40sessionmgr10&vid=13&hid=8

Kerka, S. (2003). Volunteer development. ERIC Clearinghouse on Adult, Career, and

Vocational Education. Retrieved from http://www.ericacve.org/pubs.asp

Kirkpatrick, D. L. (1997). Evaluating training programs: The four levels (2nd ed.). San

Francisco: Berrett-Koehler.

Kirkpatrick, D. L. (2006). Evaluating training programs: The four levels (3rd ed.). San

Francisco: Berrett-Koehler.

Lahti, R., & Darr, E., & Valdis, K. (2002). Developing the productivity of a dynamic workforce:

The impact of informal knowledge transfer. Journal of Organizational Excellence, 21(2),

13-21.

Lawson, K. (1997). Improving on the job training and coaching. Alexandria: American Society

for Training & Development

Lean, E. (1984, July). What’s different about training volunteers?. Training and Development

Journal. Retrieved from

http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?sid=a9e68e53-bebd-

434b-9b4c-abc1b7c68206%40sessionmgr11&vid=7&hid=19

Levinson, D. (1987). Training and development for nonprofts: A new area in which to ply your

talents has opened wide. Training and Development Journal, 41(5), 80-82. Retrieved

from http://ezproxy.lib.uwstout.edu:2170/ehost/pdfviewer/pdfviewer?sid=1937550f-

e8b2-4d1c-9bf8-be709aed5073%40sessionmgr14&vid=32&hid=10

Mare. (n.d.). In Dictionary online. Retrieved from

http://dictionary.reference.com/browse/Mare?s=t

Non profit. (n.d.). In Dictionary online. Retrieved from

http://dictionary.reference.com/browse/non+profit?s=t&ld=1032

47

Norris, T., (2006). Are volunteers increasing your organizations liability risks?. Paper presented

at the ASSE Professional Development Conference and Exposition, Seattle, WA.

Retrieved from www.onepetro.org/mslib/servlet/onepetropreview?id=ASSE-06-537

Occupational Information Resource Network (O’net). (n.d.) In O’Net Resource Center.

Retrieved from http://www.onetcenter.org/overview.html

Occupational Safety & Health Administration. (2012). Injury and illness prevention programs

white paper. Retrieved from

https://www.osha.gov/dsg/InjuryIllnessPreventionProgramsWhitePaper.html

Occupational Safety and Health Administration (OSHA). (n.d.). In Dictionary online. Retrieved

from http://dictionary.reference.com/browse/Osha?s=t

Phillips, S., & Little, B., & Goodine, L. (2002). Recruiting, retaining and rewarding volunteers:

What volunteers have to say. [Adobe PDF Version]. Retrieved from

http://library.imaginecanada.ca/files/nonprofitscan/en/iyv/phillips_sr1_english_web.pdf

Pohjoispuro, J. J., (2006). The nature of nonprofit professional development and training in

montana (Master’s thesis). Retrieved from http://etd.lib.umt.edu/theses/available/etd-

12132006-103044/unrestricted/PohjoispuroJenni.pdf

Rothwell, W. J., & Kazanas, H.C. (2004). Improving on the job training. San Francisco: John

Wiley & Sons Inc.

Self-efficacy. (n.d.). In Business Dictionary. Retrieved from

http://www.businessdictionary.com/definition/self-efficacy.html

Springs, A. (2007). Assessing the impact of the work environment on training transfer: An

investigation of the air force acquisition management course (Master’s thesis). Retrieved

from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA476928

Yearling. (n.d.). In Dictionary online. Retrieved from

http://dictionary.reference.com/browse/Yearling?s=t

48

Appendices

Appendix A: Letter of Invitation

Refuge Farms Horse Rescue & Sanctuary 3035 Highway 29 Spring Valley, WI 54767 WWW.REFUGEFARMS.ORG Dale Kiesow

Univeristy of Wisconsin, Stout July 8, 2012

712 South Broadway Street Menomonie, WI 54751

Dale: This letter is to provide you with the required letter of interest from Refuge Farms, Inc., a horse rescue and sanctuary located in Spring Valley, WI. Refuge Farms is a federally approved 501.c.3 non-profit organization and is most appreciative of your interest in us! It is my understanding that the thesis project you will conduct here at Refuge Farms will be a study of our existing volunteer training program. Your goal is to gather data, formulate a training program for our "foal" volunteers to proceed to a "yearling" status, and to also formulate a plan for training non-horse volunteers. In completing this project, you will need to gather data, conduct a study, and, in general, become greatly involved in our existing process resulting in your presentation and implementation of your new process. Dale, you must know that Refuge Farms is grateful for your interest in us. The entire organization will benefit from your efforts and this new training program. Our volunteers are the backbone of this organization and so an improvement and even the establishment of this program will work toward strengthening and rewarding these people. This will ultimately lead us to completing our Missions even better- that is, to save a few more lives! Thank you, Dale. And yes, Refuge Farms gives its wholehearted consent to this project and our support of it and you! Sincerely, Executive Director

'Twill seek the lost, and I will bring back the strayed and I will bind up the crippled and I will strengthen the weak. ..

I will watch over. I will feed them in justice." Ezekiel 34:16

49

Appendix B: Implied Consent to Participate in Approved Research

UW-Stout Implied Consent Statement for Research Involving Human Subjects

Consent to Participate In UW-Stout Approved Research

Title: Evaluation of OJT Model in a Non Profit

Research Sponsor: Dr. Jeanette Kersten, EdD 715-232-5229 kerstenj@uwstout.edu

Investigator: Dale Kiesow 715-205-0568 kiesowd@my.uwstout.edu

Description: This study is an important project designed to look at training and development at Refuge Farms, LLC. This study is being used for a thesis project in the Masters of Training and Human Development program at the University of Wisconsin Stout. You will be asked to complete a brief survey at the conclusion of training that will collect data on your opinions and views on the structure of the training. Risks and Benefits: Participation in the study comes with minimal risks. You may find it distressful to provide you opinions and views related to your development at the organization. If at any type you feel discomfort discontinue the survey. There are direct benefits to your participation in this study. Your participation will assist the organization in continued volunteer development. Special Population There are no special populations in this study Time Commitment and Payment: The survey will require minimal time commitment and is expected to take 15 minutes to complete. There is no monetary payment that will provided to you for you participation in this study. Confidentiality: Your name will not be included in any documentation or reporting that is created from this study. It is assumed that there will be no to connection between survey results and participants. The results of the survey will only be disclosed to the researcher. You information will be kept complete confidential and stored in secure location for 7 years before being destroyed. Right to Withdraw: Participation in this study is entirely voluntary. You may choose not to participate without any adverse consequences to you. You have the right to stop the survey at any time. However, should you choose to participate and later wish to withdraw from the study, there is no way to identify your anonymous document after it has been turned into the investigator.

50

IRB Approval: This study has been reviewed and approved by The University of Wisconsin-Stout's Institutional Review Board (IRB). The IRB has determined that this study meets the ethical obligations required by federal law and University policies. If you have questions or concerns regarding this study please contact the Investigator or Advisor. If you have any questions, concerns, or reports regarding your rights as a research subject, please contact the IRB Administrator. Investigator: Dale Kiesow 715-205-0568 kiesowd@my.uwstout.edu

IRB Administrator Sue Foxwell, Research Services 152 Vocational Rehabilitation Bldg. UW-Stout Menomonie, WI 54751 715.232.2477 foxwells@uwstout.edu

Advisor: Dr. Jeanette Kersten, EdD 715-232-5229 kerstenj@uwstout.edu

Statement of Consent: This section should include the language, “By completing the following survey you agree to participate in the project entitled, Evaluation of OJT Model in a Non Profit.

51

“I will seek the lost, and I will bring back the strayed,

and I will bind up the crippled, and I will strengthen the weak… I will watch over. I will feed them in justice.”

Ezekiel 34:16

Refuge Farms Horse Rescue & Sanctuary

3035 Highway 29

Spring Valley, WI 54767

WWW.REFUGEFARMS.ORG

Subject: Training Program Announcement Date: August 1, 2012 Dear Volunteers: I am very pleased to announce the introduction of a new training program at Refuge Farms. This new program is an exciting change for the FARM and will provide opportunities for volunteers to attend small group horse education classes and develop skills that will stay with you far into your future with Refuge Farms. The training program is structured around three levels of volunteers. A Foal is considered level one, a Yearling is a level two and an Alpha is considered a level three. With each level of volunteer there will be two requirements that must be fulfilled before a volunteer is allowed to advance. Those requirements are:

Evaluation of Skills Learned: Each level of volunteer has skills associated with it that a volunteer must learn and be able to demonstrate before advancing to the next level. These skills will be learned through horse education classes provided by our Executive Director Sandy Gilbertson and during hands on experience while volunteering at the FARM. Horse Education Units (HUE): HUE’s will be granted for every hour of volunteering that involves direct contact with the horses, and/or every hour of horse education class you attend. Each level of volunteer will have a predetermined number of HUE’s that must be fulfilled prior to taking the evaluation of skills.

With this training program everyone from this point forward, regardless of their years of service to Refuge Farms, will be considered a Foal level volunteer. This will ensure consistency and guarantee that everyone receives the same development experience. Most importantly, it will give us all the ability to grow and development together as a team and family. While change may seem scary and unpredictable, it is necessary in order to make certain we are providing the best care possible to the horses of Refuge Farm. Please look for further announcements and correspondence regarding the new program. If you should have any questions do not hesitate to email me at kiesowd@my.uwstout.edu or call 715-205-0568. I look forward to this journey together. Sincerely, Dale Kiesow UW Stout Graduate Student-MS in Training and Development

Appendix C: Participant invitation Letter

52

Evaluation Form/Check List

Foal Yes No

Able to hook bucket of feed while deflecting horse

Removes bridles from horse and stores them properly

Brushes horses thoroughly

Crush and mix meds in feed but not administer

Halter a band halter

Approaches horse on correct side

Comments:

Mentor Signature: Date:

Yearling Yes No

Recognizes which bridles go with what horse

Recalls from memory how much food goes with which horse

Able to fully hook, feed, and unhook horses with no assistance and in the correct order

Able to blanket and un-blanket a horse without supervision

Ability to lead horse

Walks around horse with proper distance to minimize injury from a kick

Administer crushed and paste meds

Halter a band or rope halter

Blanket a sighted horse with open chest blanket

Trailer an experienced sighted horse

Appendix D: Evaluation Check List

53

Volunteer Signature Date

Mentor Signature: Date

Comments:

Mentor Signature: Date:

Alpha Yes No

Accomplishes all tasks associated with previous levels

Remains calm and collected when dealing with incoming horses or horses with behavioral

problems

Uses to control and handle horses when needed

administer injections - following all safety and disposal guidelines

create a halter from a lead rope or a t-shirt

blanket a sighted or blind horse with open and closed chest blankets

trailer a sighted/blind horse with or without previous trailering experience

Comments:

54

Appendix E: Post Training Survey Measure

Training Survey

1. How long have you been a volunteer at Refuge Farms?

a. 0-2 months

b. 3-5 months

c. 6-12 months

d. 1 year or more

2. Please list prior experience you have had with the handling of horse

3. Was the length of the training adequate time for you to practice the skills demonstrated

by the trainer?

a. Yes

b. No

4. In which ways do you feel the mentor program and trainings will have a positive impact

to your development at Refuge Farms, LLC?

a. Yes

b. No

5. Did the training that was conducted today provide you with skills you did not previously

have in terms of horse handling? If so please list the skills acquired.

55

a. Yes

b. No

6. What did you like most about the training?

7. What did you like least about the training?

8.

7. Is there anything that you would like to see in further trainings that were not present in

this session?

8. Please include any additional comments regarding the training.

56

7/30/2012 Dale Kiesow M.S. Training and Human Resource Development UW-Stout RE: Evaluation of an OJT model in a Non-Profit Dear Dale, The IRB has determined your project, " Evaluation of an OJT model in a Non-Profit” is Exempt from review by the Institutional Review Board for the Protection of Human Subjects. The project is exempt under Category # 1 of the Federal Exempt Guidelines and holds for 5 years. Your project is approved from 7/30/2012, through 7/30/2017. Should you need to make modifications to your protocol or informed consent forms that do not fall within the exemption categories, you will need to reapply to the IRB for review of your modified study. If your project involved administration of a survey, please copy and paste the following message to the top of your survey form before dissemination:

If you are conducting an online survey/interview, please copy and paste the following message to the top of the form: “This research has been reviewed by the UW-Stout IRB as required by the Code of Federal Regulations Title 45 Part 46.”

Informed Consent: All UW-Stout faculty, staff, and students conducting human subjects’ research under an approved “exempt” category are still ethically bound to follow the basic ethical principles of the Belmont Report: 1) respect for persons; 2) beneficence; and 3) justice. These three principles are best reflected in the practice of obtaining informed consent from participants. If you have questions, please contact Research Services at 715-232-1126, or foxwells@uwstout.edu, and your question will be directed to the appropriate person. I wish you well in completing your study. Sincerely,

Susan Foxwell Research Administrator and Human Protections Administrator, UW-Stout Institutional Review Board for the Protection of Human Subjects in Research (IRB)

Appendix F: IRB Approval Form