1  

 

Back on Track: A Process Evaluation of Cub Scout Pack #523

Valerie Smith

Clemson University

April 21, 2015

2  

 

Table of Contents

Description of the Organization…………………………...….……………………….……………………3

Stakeholders……………………….…………....…….….…………………………………………………4

Problem………………………………….….……….…..………………...……………………………..…6

Type and Purpose of Evaluation……………..…..…………………………………………………………7

Guiding Questions……...…………….................……………………………….………………………….7

Literature Review……………….…………………….…………………………………………………….8

Participants………...……………...………...……………………………………………………..………12

Data Collection…………..………………..………………………………………………………………13

Implications…………………..……..………………………………………………………………….….21

References……………..………………………………………………………………………………..…22

Appendix A……………../………..………………...……………………………………………………..24

Appendix B……………....………………..………………………………………………...…………….25

3  

 

Back on Track: A Process Evaluation of Cub Scout Pack #523

Description of the Organization

The Boy Scouts of America (BSA) was founded in 1910 and modeled after British military

officer Robert Baden Powell’s original Boy Scout program (Arneil, 2010). In the beginning, the

BSA was geared toward working with adolescent boys, but in 1930 a separate Cub Scout

curriculum for youth between the ages of seven and ten was added to the program (Boy Scouts

of America, 2015). Since 1910 the BSA has served over 114 million boys and young men, with

a goal that they would grow up to “make moral and ethical choices” based on the Scout Oath and

the Scout Law (See Appendix A.) (Boy Scouts of America, 2015). More than 100 years later, the

BSA is still going strong: in 2010 more than 35,000 scouts and leaders gathered at the National

Scout Jamboree in Fort A.P. Hill, Virginia to celebrate the organization’s centennial anniversary

(Wendell, 2010). Recent efforts to encourage scout loyalty and participation promise continued

success for years to come.

Boys can join Cub Scouts beginning in the first grade and are eligible to earn a new rank

every school year. The Cub Scout ranks are as follows: Tigers (1st grade), Wolves (2nd grade),

Bears (3rd grade), Junior Webelos (4th grade) and Senior Webelos (5th grade). A group of Cub

Scouts of the same rank (e.g. Tigers) is called a den, which is led by a den leader. All of the dens

that are based out of the same facility make up a pack, which is officiated by the Cubmaster.

The governing body of the pack is called the Pack Council and includes the Cubmaster and den

leaders, as well as parents and volunteers who hold offices such as president, vice president,

secretary, etc. After completing the Senior Webelos rank requirements, Cub Scouts are eligible

to advance to the Boy Scout troop, where they can stay through the end of their senior year of

high school.

4  

 

Stakeholders

Cub Scout Pack 523 was established in 1983 at St. Luke United Methodist Church in

Hartsville, South Carolina and has remained an active organization since its inception. The

pack’s longevity and reputation as being extremely active led the pack leaders to adopt the

slogan “Pack 523: The Pack on Track.”

Pack 523 currently has 51 boys enrolled in the Cub Scout program. One of those boys is

my son who has been involved in Cub Scouts since he was a Tiger and joined Pack 523 as a

Wolf. The success of the Cub Scout

program in meeting the BSA vision that

all boys “become responsible,

participating citizen[s] and leader[s]”

(Boy Scouts of America, 2015) is

dependent upon the hard work and the

development of strong relationships

between the pack and a number of

individuals and organizations such as

those described below.

Cub Scouts. Boys involved in Cub Scouts develop self-esteem and personal responsibility

though participation in skill-building activities and community service projects. The Cub Scout

program is participation-based (as opposed to mastery-based) and is meant to be an introduction

to the Boy Scout program.

Parents. Parents of Scouts are given the rank of Akela, which means “good leader” (Boy

Scouts of America, 2015). Parents are encouraged to work with the boys as they practice the

5  

 

skills they are learning in scouts. One of the requirements for boys to complete the next rank in

Cub Scouts is to participate in family-centered activities such as helping with housework,

organizing a family game night, or learning about the family budget. Parents also serve as

volunteers for pack activities and many parents work closely in support of the den leaders.

Den leaders. Den leaders guide the Cub Scouts through the process of earning their rank.

Den leaders generally start with a pack of Tiger Cub Scouts and follow the same group of boys

through each rank. In this way, the leaders have an opportunity to develop relationships with

individual boys over the span of five years and take a great deal of pride and satisfaction in

“sending off” a pack of Senior Webelos into the Boy Scout troop.

Boy Scout Troop #523. While it is possible for a teenager to join scouts for the first time

as a Boy Scout, 95% of current Troop 523 Boy Scouts bridged over from Cub Scout Pack 523

(personal communication, T. Thompson, March 12, 2015). Therefore, the livelihood of the Boy

Scout troop depends upon being able to draw new scouts from the Cub Scout pack.

St. Luke United Methodist Church (SLUMC). SLUMC has been the charter

organization for Pack 523 for more than 30 years. SLUMC also houses a number of other

community-based organizations with which Pack 523 shares resources and materials. Clear

Participating in community service activities such as food drives not only benefits the community, but also the scouts themselves and the reputation of the pack and the BSA.  

6  

 

communication among the church, pack, and additional clubs for space and resources is vital to

the success of the pack.

District and National Councils. The regional and national BSA councils depend upon

the scouts, den leaders, and other directly affiliated individuals involved in the Cub Scout

program to promote scouting in the public eye as valid and valuable in the development of youth.

Negative public perception of local programs could lead to a decrease in participation, funding,

and support for the regional and national BSA organization.

Problem

Pack 523 has experienced a number of changes in

leadership within the last two years including the installation

of a new Cubmaster, four new den leaders, and three new

council officers, all with varying levels of experience with

working in scouting. The addition of so many new leaders in a

relatively short amount of time has prompted a discussion of

whether or not Pack 523 is still “on track.” Initial

conversations with the pack council members resulted in the

identification of three areas for improvement: internal

evaluation, communication, and the degree of collaboration

with other scouting programs.

First, it will be important to determine what type of internal evaluation(s) the pack is

using and to what degree the data collected in used to inform programming decisions. In

organizations such as Cub Scouts that are run solely by volunteers, it can be difficult for leaders

to find the time, training, and financial resources to collect and analyze internal data.

Pack 523 is known in the community as “the pack on track.”

7  

 

Nonetheless, aligning programming to the needs and interests of the participants is important to

the success of any organization.

Next, communication on the micro and macro levels is an area of weakness identified by

the pack council. It is expected that an increase in the effectiveness of communication will lead

to improvements in other areas of the organization including stakeholder engagement, scout

recruitment, volunteer recruitment, funding efforts, and overall visibility of the program.

Finally, the pack council expressed concern that Pack 523 is becoming too isolated from

other local, regional and national Cub Scout packs and activities. As a result, the boys of Pack

523 may be missing out on valuable opportunities to interact with and learn from other scouts

and leaders, while den leaders and volunteers may be missing out on training opportunities that

would help them to become more skillful and efficient leaders.

Type and purpose of evaluation

The purpose of this evaluation is to examine the level of program fidelity and the quality of

the organizational functions of Cub Scout Pack 523. Hill et al (2007) explain that examining

program fidelity is “necessary to provide feedback for continuous quality improvement to all

program implementers” (p. 26). To accomplish this, the pack will participate in a formative

process evaluation. A formative process evaluation method was chosen because, while the pack

council agrees that the overall impact of the program on youth is extremely important, the impact

will be greater if the pack is strengthened and streamlined on an organizational level.

Guiding Questions

The questions on the following page are intended to aid the evaluator in examining the

extent to which Pack 523 is functioning efficiently and as intended by the BSA.

8  

 

• What forms of internal evaluation are currently in place? How is the data gathered from

the evaluation(s) used to inform programming decisions? Do all stakeholders have a

voice in the decision-making process?

• Is the governing body of Pack 523 effectively communicating internally and externally?

• To what degree is Pack 523 effectively collaborating with other packs on a local,

regional, and national level?

Literature Review

A review of published literature aimed to identify the most effective processes for the

development of outstanding scouting programs. Despite the fact that the BSA has had a long

history of youth-focused programming, Polson et al (2013) note that their evaluation of

scouting’s influence on the development of social capital and civic engagement is the first of its

kind. Apart from the work of Polson and his colleagues, other researchers who have described

the BSA on an organizational level are quick to point out its strict adherence to tradition; top-

down, hierarchical structure; and targeting of middle class, Caucasian youth (Chyung et. al.,

2013; Polson et. al, 2013; Arniel, 2010). In fact, one researcher (Arniel, 2010) attributes the

decrease in Cub Scout/Boy Scout membership in the 21st century—dropping from 2.8 million in

2008 to 2.1 million in 2014 (Boy Scouts of America, 2014)--to the BSA’s narrow political,

organizational, and social focus; however, Polson et al (2013) argue that while BSA membership

has been showing downward trends, the number of Boy Scouts receiving the Eagle Scout award

has remained stable over the last 50 years. Based on this information, one can infer that those

boys who do choose to participate in the organization are committed to the program and the

values that scouting represents.

9  

 

This level of commitment from scouts may be more than a coincidence: the BSA has

identified scout loyalty as “the one question [scout leaders] need to know” if they want to ensure

the success of their pack (Boy Scouts of America, 2015). To help packs determine and increase

the level of loyalty of their participants, the BSA has generated a list of seven indicators of

program loyalty. These indicators have been tracked by the BSA on a national scale and suggest

that strategies to increase scout loyalty might be working. Data comparing loyalty scores from

Fall 2013 to Spring 2014 show an increase in loyalty for all stakeholders except Cub Scouts,

whose loyalty rating dropped from approximately 40% to 34.9% within the specified time period

(Boy Scouts of America, 2015). However, one must take into account the low response rate

(10.7%) before jumping to conclusions about the data. As more packs continue to use this

instrument, the level of reliability will increase and scout leaders can gain a clearer picture of the

strategies’ overall effectiveness.

While the body of research discussing the impact of BSA as an organization is growing,

there is still little published information focusing formative and process evaluations of local

scouting programs. Both formative and process evaluations focus on a program’s performance

and are “a means of discovering the extent, fidelity, and quality” of the program’s operation

(Butterfoss, 2006, p. 324). Included in researchers’ list of best practices for youth-serving

programs similar to the BSA is conducting internal evaluation with a variety of instruments,

collaboration with stakeholders and outside agencies, increasing and encouraging youth voice;

and fostering the development of social capital (Chyung et.al, 2013; Polson et al, 2013; Law &

Shek, 2011; Lau et. al., 2003; Karapetrovic & Willborn, 2002).

The benefits of internal evaluations of processes are plentiful and discussed at length in

the research of Chyung et al (2013), Lau et al (2003), and Karapetrovic & Willborn (2002). In

10  

 

each of these articles the researchers point out that internal evaluations are cost effective and

flexible to the needs of the organization. One of the advantages of internal evaluation noted by

researchers is that staff and clients view the results with less skepticism than those gathered from

external evaluations. However, Lau et al (2003) advise that managers work in collaboration with

an experienced evaluator who can model and coach them through the internal evaluation process.

Researchers Chyung et al agree, explaining that the role of an external evaluator is to “assist the

internal evaluation group to build capacity to conduct and use evaluation to serve their own

interests” (p. 89). They go on to report that once trained in internal evaluation techniques, youth

development workers gained an increased sense of professionalism, empowerment (Chyung et.

al 2013; Lau et. al 2003) intrinsic motivation, and efficiency (Karapetrovic & Willborn, 2002).

In order to get a clearer picture of which processes are most effective, it is recommended

that organizations gather data using a variety of evaluation instruments including, but not limited

to, questionnaires, observations, interviews, focus groups and reviews of event/activity logs

(Butterfoss, 2006; Lau et. al, 2003). Each method of data collection has distinct advantages and

provides a different type of feedback for the evaluator. For example, a review of an

organization’s activity logs would provide information about the number of participants for

specific events, while interviews with participants would likely provide contextual information

about why someone did or did not attend those same events (Butterfoss, 2006). In this way

organizations can use data collection methods strategically and in a complementary manner

based on the depth of feedback desired. However, it is important that evaluation does not

interfere with the day-to-day programming; instead, it should be integrated into the

organization’s daily practice so that the organization can “become more responsive to youth

needs” (Lau et al, 2003, p. 49).

11  

 

A well-documented key element in youth serving organizations’ success is collaboration

with stakeholders and outside agencies (Law & Shek, 2011; Saito & Sullivan, 2011; Butterfoss,

2006; Lau et. al. 2003). Effective “cross-sector dialogue” is essential to the success of youth

serving programs because it illuminates “just how much the decisions of one sector could affect

those of another” (Lau et. al, 2003, p. 50). For these dialogues to occur, it is essential that the

organization communicate effectively to and with its stakeholders. To accomplish this,

researchers Saito and Sullivan (2011) suggest that organizations have at their disposal a “system-

level opportunities, resources, and technology” tailored to each specific audience (p.120).

Having their information provided in a variety of formats allows the program managers to

include information that is specific to the interests of all stakeholders. For example, parents may

be more interested in information that explains the long-term benefits of scouting, while the

scouts may be more interested in learning about the unique activities they will participate in as a

Cub Scout.

Once stakeholders are fully on-board and understand their role in the organization, then the

lines of communication will open. One line of communication that is sometimes overlooked--

especially in the case of organizations like the BSA that traditionally have a top-down

organizational structure and who serve younger audiences--is the line between the organization

itself and the youth it serves. Yet, a number of studies and commentaries tout the benefits of

developing practices that encourage youth voice, or input (Saito & Sullivan, 2011; Lau et. al,

2003). Saito and Sullivan (2011) describe youth voice as giving youth “the power to influence

programs and policies by sharing their perspectives, information, and/or making a persuasive

case to adults who truly consider their ideas as they make decisions” (p. 116). Although many

programs that encourage youth to have an active voice in their programming, implementation,

12  

 

and/or assessment are targeted at adolescents for developmental reasons, it is important to

remember that younger children can provide valuable input into the direction that a program

takes. Research indicates that organizational leaders would be wise to consider youth’s input

with regard to programming. For example, one

research team found that high levels of youth

voice in programs resulted in increased

enrollment, access to hard-to-reach populations,

improved programming, and higher levels of

community involvement in the program (Lau et.

al, 2003). What is more, youth voice not only

benefits the organization, but also the youth

themselves by developing their sense of efficacy, agency, and belonging and increasing their

problem-solving, communication, and advocacy skills (Saito & Sullivan, 2011).

Finally, Polson et al (2013) note that participation in organizations such as Cub Scouts

contributes to the development of social capital and increases in civic engagement. Putnam

(2000, 2007 in Polson et al 2013) describes social capital as “social networks, norms of

reciprocity, and bonds of trust that develop between individuals and groups” (p. 761). As youth

interact with peers from diverse backgrounds and other adults, they will have more opportunities

to develop the relationships that lead to positive outcomes like increased social capital and skill

building (Eccles & Gootman, 2002).

Participants

The target population for this evaluation includes the 51 Cub Scouts and their parents, ten

registered den leaders, and 45 Boy Scouts. The demographic makeup of the Pack and Troop 523

Carving is a favorite activity of many Cub Scouts.

13  

 

is 98% Caucasian youth from predominately middle class, suburban families (J. Jordan, personal

communication, April 2, 2015). The Cub Scouts, parents, and den leaders will be accessed

through the monthly pack and leaders’ meetings and bi-weekly den meetings, all held at

SLUMC. The Boy Scouts will be accessed through their bi-weekly meetings held at the troop’s

clubhouse on the campus of SLUMC.

Survey data will be collected from all den leaders, at least 80% of active Cub Scouts and

their parents, and a sample of active Boy Scouts. The sample of Boy Scouts will include the

eleven scouts who have bridged over from Pack 523 within the last three years. These boys have

been targeted as participants in the evaluation because scouts who have recently transitioned into

the troop should be able to provide the most relevant information about their experience in the

pack.

In addition to surveys, the evaluator will speak with at least four focus groups to provide

more in-depth information and context for the operation of Pack 523. The focus group panels

will include Cub Scouts, Boy Scouts, parents, and den leaders. Focus group participants will be

selected on a voluntary basis, with preference given to those who have been involved with Pack

523 for at least two years.

Data Collection

To examine the current level of program quality and fidelity of Cub Scout Pack 523, a

variety of data collection methods will be used including a review of internal records and

activity/event logs, surveys, and focus groups. Pack activities predominately run from September

through May; therefore, data that will be gathered from and utilized solely by members of the

pack council will be analyzed and discussed in July during scouts’ summer vacation. Data that

14  

 

will be collected from scouts and parents will be collected in March and November, according to

BSA recommendations (Boy Scouts of America, 2014).

Each group of participants will provide the evaluator with the following information:

• Cub Scouts: satisfaction with the program, types of activities that they would like to see

more/less of, level of participation in local, regional and national events, and reason(s) for

joining Cub Scouts.

• Boy Scouts: perceived level of preparedness entering the Boy Scout troop, level of

participation in local, regional, and national events, and satisfaction with the Cub Scout

experience.

• Parents: reason(s) for signing their child up for scouts, opinion on the effectiveness the

pack’s communication, preferred method(s) to receive communication, level of

participation in local, regional, and national events, and overall satisfaction with their

child’s experience in Cub Scouts.

• Den leaders: effectiveness of communication within the pack council and with parents,

organizational/managerial concerns, availability of training and/or resources, level of

participation in local, regional, and national events, and overall satisfaction with the

experience of being a den leader.

Regardless of the data collection methods and instruments used, it will be critical that the Pack

Council uses the data to inform decisions about future pack activities and council procedures.

Internal Records Review. The majority of the quantitative data collected for this

evaluation will be collected through an internal records review of the following items:

• frequency of den meetings, pack meetings, campouts, service projects, and other events,

• recruitment, retention, and training of den leaders,

15  

 

• percentage of Cub Scouts advancing to the next rank,

• percentage of Cub Scouts advancing to the Boy Scout troop, and

• Frequency of participation in joint scouting activities.

If not already in use, the evaluator may recommend that data listed above be collected on the

Journey to Excellence (JTE) pack evaluation sheet (see Appendix B), which is found on the BSA

website. Not only does the JTE provide a place for the Cubmaster and leaders record data about

each of the items listed above, it also provides the pack council with levels or benchmarks to

show progress toward improving in each of these areas. For example, the JTE defines a bronze

level pack as one that retains 60% of scouts, while a silver level pack retains 65% of scouts, and

a gold level pack retains at least 75% of scouts.

In addition to the data listed above, the council will create similar data collection

instruments to record participation in all meetings and events, including regional and national

events and trainings. Collecting information about scout and parent participation will inform the

pack council about which events were most interesting to the scouts and/or most worthwhile to

the parents. If the pack can identify and provide more events that had high levels of interest

and/or satisfaction, then overall participation in pack events will likely increase. To maintain

validity of the data collected, it will be important than the once an instrument for collecting

internal data is adopted by the Pack Council, it should be used from year to year as long as the

items measured continue to align with the evaluation needs of the pack. In this way, the council

can avoid the effects of instrumentation errors or ambiguous temporal precedence (Campbell &

Stanley, 1963).

The evaluator will also review the pack’s system for logging and collecting all of its public

relations communications. Maintaining a record of media publications will help the council

16  

 

determine the outlets most frequently utilized along with pinpointing additional media outlets

that are not being utilized effectively. The increased presence of positive public relations in the

community should lead to increases in scout recruitment and feelings of pride and belonging for

pack members and stakeholders. Table 1 below defines which member(s) of the pack

council will be responsible for collecting and reviewing each set of data described above.

Surveys. The primary method of data collection will be the use of internally created,

multiple-choice and short-answer surveys to gather both quantitative and qualitative data from

the pack’s stakeholders. A synopsis of the purpose, audience, and frequency of surveys to be

administered is described in Table 2 on the following page. In addition to possible

instrumentation errors and/or errors due to the inability to establish the correct sequence of

events, the use of surveys for data collection may also be affected by additive or interactive

Table 1. Collection and Review of Internal Data

Collected by Frequency of review Frequency of den meetings Den leaders Annually Frequency of pack meetings, campouts, service projects, and other events

Cubmaster Annually

Frequency of attendance and/or collaboration with other packs on a local, regional, and national level

Cubmaster Annually

Recruitment, retention, and training of den leaders

Council President

Annually

Percentage of Cub Scouts advancing to the next rank

Den leaders Annually

Percentage of Cub Scouts advancing to the Boy Scout troop

Cubmaster Annually

Cub Scout attendance at den meetings Den leaders Monthly Cub Scout attendance at pack meetings, campouts, service projects, and other events

Cubmaster Monthly

Type and frequency of public relations communications

Public relations chair

Bi-monthly

17  

 

influences (Campbell & Stanley, 1963) especially in cases where the same survey may be

administered multiple times over the course of an individual’s participation in Pack 523.

The data collected from surveys will inform the pack council about the program’s overall

effectiveness and the satisfaction of the pack’s stakeholders. To encourage participants to

complete the surveys, each survey will be concise, anonymous, and administered at the

beginning or end of a den meeting. The council may also choose to offer incentives, such as

giving an award to the first den to return all surveys completed.

Table 2. Examples of survey data to be collected.

Topic Audience Frequency Preferred type of communication Parents upon initial enrollment or annual

re-enrollment Frequency accessing den and pack website, Facebook page, or other social media

Parents, Cub Scouts November and March

Level of satisfaction with communication between the den leader/pack and families

Parents, Den leaders November and March

Level of satisfaction with internal communication

Pack council November and March

Level of satisfaction with den & pack activities (annual theme, venue(s), & perceived level of voice in decision-making processes, e.g.).

Cub Scouts, parents, den leaders, Boy Scouts

November and March

Level of satisfaction with den & pack meetings (frequency, duration, date/time, and perceived level of organization, e.g.).

Cub Scouts, parents, den leaders, Boy Scouts

November and March

Interest and/or participation in regional and national scouting events

Cub Scouts, Boy Scouts, parents, Pack Council

Annually

Level of satisfaction with the type & frequency of staff training

Den leaders Annually

Visibility of the pack in the community

Parents, den leaders, pack council

November and March

18  

 

Focus Groups. To gather contextual information about the topics listed in the Survey

section, the evaluator will arrange to speak with focus groups. The evaluator will begin by

speaking with four focus groups: one group of six Cub Scouts, one group of six Boy Scouts, one

group of eight parents, and one group of five den leaders. Participants in the focus groups will

first be selected on a volunteer basis because it is assumed that those who volunteer to participate

on a focus group panel will be open to sharing their opinions and experiences with the evaluator.

If there are spaces available after all volunteers have been included, then the pack council

will generate a list of names of potential panelists who will be invited to participate in the focus

group. Additional focus groups panels will be conducted as necessary until

1.) all eligible participants have participated on a panel, and/or

2.) conversations with focus group participants no longer yield new

information.

Upon completion of the evaluation cycle, the pack council will synthesize all data into one

comprehensive document and identify links between the information gathered by each

instrument. The pack council may use a flowchart similar to the one provided on the BSA

website (See Table 3 on the following page) or they may develop their own instrument for this

purpose.

19  

 

Goal: At least 85% of parents are satisfied with communication between the pack

and families. Evidence:        

Key Issues:      

Actions:        

Table 3. Comprehensive data analysis flowchart. (Adapted from Boy Scouts of America, 2015)

Using the Comprehensive Data Analysis Flowchart, council members will

1. identify a target goal evaluated in the previous cycle,

2. identify the key factors evidenced in each method of data collection that support or

detract from the achievement of this goal,

3. identify the possible underlying factors that detract or enhance progress toward achieving

the target goal, and

4. identify the necessary actions steps to either discontinue or support the pack’s progress

toward achievement of the target goal.

This evaluation sequence will allow the evaluator to gain an overview of the strategies that are

working well for the pack and those that still need improvement. The council will then be able

to refine the organizational practices of the pack so that they are better aligned with the BSA’s

recommended guidelines and are more responsive to the needs and interests of all stakeholders.

20  

 

For example, increased satisfaction with communication between the pack and families will

likely lead to higher levels of participation and retention of scouts as well as increased levels of

efficiency for the den leaders.

Barriers. Given the nature of using tables and checklists to record data from internal

records, the barrier that council members are most likely to encounter is human error in

recording and tallying the information. In the event that the pack’s records are incomplete, the

council will take appropriate action to attempt to recover missing data. Additionally, the pack

council will ensure procedures for future data collection are effectively communicated to all

responsible parties and provide any necessary training for gathering the specified data.

A potential barrier to administering surveys as part of the evaluation process may be the

participants’ level recollection about items such as specific pack events or the frequency with

which they access the pack’s social media. Similarly, although asking parents to respond to a

survey while dropping off or picking their child from meetings does provide the den leaders with

easy and relatively consistent access to the parents, one drawback is that the parents may be

rushed or distracted, causing them to provide responses that are incomplete or otherwise not fully

thought out. Finally, survey data can be somewhat subjective to analyze if the participants do not

provide an explanation for their response to certain questions.

A potential barrier to including focus groups as a part of the evaluation process is

documentation errors by the evaluator. To avoid this type of error, the focus groups will be

audio-recorded whenever possible. Also, similar to gathering survey data, participants in the

focus groups may not be able to recollect specific examples of experiences or events that would

aid in informing future programming decisions. It is expected that the interactive process

inherent to focus group design should help alleviate this issue. Finally, it is possible that some of

21  

 

the participants may volunteer for the focus group because they want to vent about a particular

experience or, conversely, some participants may be hesitant to share negative opinions about the

program because they fear judgment from the evaluator, pack council, or other focus group

participants.

Implications

Upon completion of the evaluations described above, it is expected that the Pack Council

will have a clearer understanding of the strenghts and weaknesses of the pack’s organzational

functions and their level of adherece to the BSA’s Cub Scout program design. In addition, the

pack will have established a comprehensive evaluation system and will be able to implement the

necessary practices to utlize the data collected from these evaluations to inform programming

decisions, effectively communicate with stakeholders, and participate in scouting on a broader

scale. Once these practices have been implemented, then the pack can move into the stage of

continuous process monitoring to ensure that the procedures established in this initial evaluation

are upheld. It is believed that implementing and

consistently monitoring the data gathered from the

evaluation instruments previously described has the

potential to lead to a number of benefits for the pack

including increases in organizational efficinecy,

stakeholder satisfacion and buy-in, particpation/enrollment,

staff knowledge and skills, and overall visibility. In short,

Pack 523 will be better prepared to live up to its reputation

as being “The Pack on Track.”

22  

 

References

Arniel, B. (2010). Gender, diversity, and organizational change: The Boy Scouts

vs. Girl Scouts of America. Perspectives on Politics, 8(1), March 2010.

Boy Scouts of America. (2014). Report to the nation.

http://www.scouting.org/About/AnnualReports/2014.aspx

-----. (2015). Boy Scouts of America. http://www.scouting.org

Butterfoss, F. (2006). Process evaluation for community participation. Annual

Review of Public Health, 27, 323-340

Campbell, D. T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for

research. (pp. 171-246). Boston: Houghton Miffin.

Chyung, S. Y. (Yonnie), Wisniewski, A., Inderbitzen, B. & Campbell, D. (2013). An

improvement and accountability oriented program evaluation: An evaluation of the

Adventure Scouts program. Performance Improvement Quarterly. 26(3): 87-115.

Eccles, J.& Gootman, J. (2002).Features of positive developmental settings. Community

programs to promote youth development. (pp. 87-117). Washington, DC: National

Academy Press

Hill, L. G., Maucione, K. & Hood, B. K. (2007). A focused approach to assessing program

fidelity. Prevention Science. 8, 25-34. Doi 10.1007/s11121-006-051-4

Karapetrovic, S. & Willborn, W. (2002). Self-audit of process performance. International

Journal of Quality & Reliability Management. 19(1): 24-45.

Lau, G., Netherland, N., & Haywood, M. (2003). “Collaborating on evaluation for youth

development. New Directions for Evaluation, 98, 47-59.

Law, B. & Shek, D. (2011). Process evaluation of a positive youth development

23  

 

program: P.A.T.H.S. Research on Social Work Practice. 21(5): 539-548.

Polson, E., Kim, Y., Jang, S., Johnson, B., & Smith, B. (2013). Being prepared and

staying connected: scouting’s influence on social capital and community

involvement. Social Science Quarterly. 94(3): 758-776. Doi: 10.11111/ssqu.12002

Saito, R., & Sullivan, T., (2011). The many faces, features, and outcomes of youth engagement.

Journal of Youth Development. 6(3): 110-125.

Wendell, B. (2010, August). 2010 National Scout Jamboree had highest attendance since 1973.

Bryan on Scouting. Retrieved from:

http://blog.scoutingmagazine.org/2010/08/11/jamboree-attendance/

24  

 

Appendix A. Boy Scout Oath and Law.

Scout Oath:

On my honor I will do my best to do my duty to God and my country and to obey the Scout Law;

to help other people at all times; to keep myself physically strong, mentally awake and morally

straight.

Scout Law:

A Scout is trustworthy, loyal, helpful, friendly, courteous, kind, obedient, cheerful, thrifty, brave,

clean and reverent.

25  

 

Appendix B. BSA Journey to Excellence Pack Evaluation Instrument.

Pack ________ of _________________________ District 2015 Scouting's Journey to Excellence

Item Objective Bronze Level Silver Level Gold Level Bronze

Points Silver Points

Gold Points

Planning and Budget Total Points: 200

#1

Planning and Budget: Have a program plan and budget that is regularly reviewed by the pack committee, and it follows BSA policies relating to fundraising.

Have an annual program plan and budget adopted by the pack committee.

Achieve Bronze, plus pack committee meets at least six times during the year to review program plans and finances.

Achieve Silver, plus pack communicates annual program plan and budget prior to October 31, 2015.

50 100 200

Membership Total Points: 500

#2

Building Cub Scouting: Have an increase in Cub Scout membership or maintain a larger than average pack size.

Conduct a formal recruitment program by October 31 and register new members in the pack.

Achieve Bronze, and either increase youth members by 5% or have at least 40 members.

Achieve Silver, and either increase youth members by 10% or have at least 60 members.

50 100 200

#3 Retention: Retain a significant percentage of youth members.

Reregister 60% of eligible members, or have improvement over the prior year.

Reregister 65% of members, or reregister 60% and have improvement over the prior year.

Reregister 75% of members, or reregister 65% and have improvement over the prior year.

50 100 200

#4 Webelos-to-Scout transition: Have an effective plan to graduate Webelos Scouts into Boy Scout troop(s).

With a troop, hold two joint

activities or 75% of second year Webelos have completed "The Scouting Adventure."

60% of eligible Webelos register with a troop.

80% of eligible Webelos register with a troop and the pack has at least one active den chief.

25 50 100

Program Total Points: 900

#5 Advancement: Achieve a high percentage of Cub Scouts earning rank advancements.

Have 50% of Cub Scouts advance one rank

Have 60% of Cub Scouts advance

Have 75% of Cub Scouts advance 100 200 300

26  

 

#6 Outdoor activities: Conduct outdoor activities and field trips.

Each den has the opportunity to participate in three outdoor activities or field trips during the year.

Each den has the opportunity to participate in four outdoor activities or field trips during the year.

Each den has the opportunity to participate in five outdoor activities or field trips during the year.

50 100 200

#7 Day/resident/family camp: Cub Scouts attend day camp, family camp, and/or resident camp.

33% of Cub Scouts participate in a camping experience or improvement over the prior year.

50%, or 33% and have improvement over the prior year.

75%, or 50% and have improvement over the prior year.

50 100 200

#8 Service projects: Participate in service projects.

Participate in two service projects and enter the hours on the JTE website.

Participate in three service projects and enter the hours on the JTE website.

Achieve Silver, plus at least one of the service projects is conservation-oriented.

25 50 100

#9 Pack and den meetings: Dens and the pack have regular meetings.

Hold eight pack meetings a year. Den or pack meetings have started by October 31.

Achieve Bronze, plus dens meet at least twice a month during the school year.

Achieve Silver, plus earn the Summertime Pack Award.

25 50 100

Volunteer Leadership Total Points: 400

#10 Leadership recruitment: The pack is proactive in recruiting sufficient leaders.

Have a Cubmaster, assistant Cubmaster and a committee with at least three members.

Achieve Bronze, and by May 31, the pack committee identifies pack and den leadership for the next year.

Achieve Silver, plus every den has a registered leader by October 15.

50 100 200

#11 Trained leadership: Have a trained and engaged leaders at all levels.

Cubmaster, den leaders, and assistants have completed an orientation and youth protection training.

Achieve Bronze, plus the Cubmaster and den leaders have completed position- specific training or, if new, will complete within three months of joining.

Achieve Silver, plus two-thirds of registered committee members have completed position-specific training for the pack committee.

50 100 200

����  Bronze: Earn at least 525 points by earning points in at least 7 objectives.

����  Silver: Earn at least 800 points by earning points in at least 8 objectives.

27  

 

����  Gold: Earn at least 1,050 points by earning points in at least 8 objectives.

����  Our pack has completed online rechartering by the deadline in order to maintain continuity of our program.

����  We certify that these requirements have been completed:

Total points earned: ���No. of objectives with points:

Cubmaster ___________________________________________________ Committee chair _______________________________________________ Commissioner _________________________________________________ This form should be turned in to the Scout service center or your unit commissioner with your charter renewal paperwork.

Date _____________________ Date _____________________ Date _____________________

Scouting's Journey to Excellence 2015 Pack Performance Recognition Program

Journey to Excellence uses a balanced approach to measure performance. Below provides specific information to help you understand the criteria and exactly what data will be used to determine the three levels of performance. In planning your strategy, use actual numbers from the previous year to guide your performance improvement goal planning. The pack may qualify by meeting a specific standard or by showing improvement.

Planning and Budget Measures

1

The pack has a program plan and budget that is reviewed at all pack committee meetings, and the pack follows BSA policies relating to fundraising and fiscal management as found on the Unit Money-Earning Application form and any other publication that the council has developed for fundraising and fiscal management. Program plans and budget are reviewed with parents at the start of the program year.

Membership Measures

2

A formal recruitment event is conducted by October 31. At charter renewal time, the pack has an increase in the number of youth members over the number of youth members on the previous year's charter renewal. A membership growth plan template can be found at www.scouting.org/membership.

3 Number of youth members on this year’s charter renewal (C) divided by the number of youth members on last year’s charter renewal (D) plus any additional youth members (E) minus any transfer-outs or age-outs (F). Total = (C) / (D+E-F).

28  

 

4

Hold at least two joint activities with a troop or troops, and have graduating boys register with a troop. "The Scouting Adventure" for second-year Webelos is described in the Webelos Scout Handbook. If the pack has no second-year Webelos Scouts, this requirement is met at the Bronze level.

Program Measures

5

Total number of Cub Scouts advancing at least one rank (Bobcat, Tiger, Wolf, Bear, Webelos, Arrow of Light) since last charter renewal date (A), divided by the number of boys at the start of the year (B1) plus all new boys who joined or transferred into the pack at any time during the current year (B2). Advancement = A / (B1 + B2).

6 The pack has activities and field trips in the outdoors, which could include outdoor pack meetings, hikes, family campouts, parades, outdoor service projects, etc. All dens have the opportunity to participate.

7

Cub Scouts attend an in-council or out-of-council Cub Scout day camp, family camp, and/or Cub Scout resident camp in 2015. STEM programs either as a day camp or resident camp are also included. All levels are total number of different Cub Scouts attending (G) divided by total number of Cub Scouts registered in the pack as of June 30, 2015 (H). Total = G / H.

8

The pack participates in at least two service projects during the year and enters them on the Journey to Excellence website. (See instructions at www.Scouting.org/JTE.) The projects may be completed as joint projects with other organizations. At least one project is conservation-oriented.

9

Have at least eight pack meetings within the past 12 months, with one of those meetings being to review the pack's program plans and asking for parental involvement in the pack. Den meetings start by October 31, 2015, and all dens meet at least twice each month during the program year. Pack earns the Summertime Pack Award.

Volunteer Leadership Measures

10

The pack has a Cubmaster, an assistant, and a committee of at least three members. Ideally, the chartered organization representative should not be dual registered as one of the committee members. The pack identifies persons for next year's leadership for existing dens, including Cubmaster, Den Leaders, and Webelos Den Leaders by May 31, 2015. All dens have a registered leader by October 15.

11 Cubmaster, den leaders and assistants have had an orientation and have completed youth protection training. The Cubmaster and den leaders (paid or multiple registration) have completed position-specific training or, if new, will complete within

29  

 

three months of joining. Two- thirds of active committee members have completed leader-specific training.

Scoring the pack's performance: To determine the pack's performance level, you will use the above information to determine the points earned for each of the 11 criteria and then add those individual point scores to determine a composite score. Count only the highest point total achieved in any one criterion. Bronze level requires earning at least 525 points in at least 7 criteria, Silver level requires earning points in at least 8 criteria and 800 points, and Gold level requires earning points in at least 8 criteria and 1,050 points.

WHO USES THIS FORM? Packs with a charter renewal date in any month from October 2015 to September 2016.

For more resources: www.scouting.org/jte