process evaluation for a school-based physical activity intervention for 6th- and 7th-grade boys:...
TRANSCRIPT
Evaluation and Program Planning 42 (2014) 21–31
Process evaluation for a school-based physical activity intervention for6th- and 7th-grade boys: Reach, dose, and fidelity
Lorraine B. Robbins a,*, Karin Allor Pfeiffer b, Stacey M. Wesolek a, Yun-Jia Lo c
a College of Nursing, Michigan State University, East Lansing, MI 48824, USAb Department of Kinesiology, Michigan State University, East Lansing, MI 48824, USAc School of Natural Resources and Environment, University of Michigan, Ann Arbor, MI 48109, USA
A R T I C L E I N F O
Article history:
Received 9 October 2012
Received in revised form 26 June 2013
Accepted 11 September 2013
Keywords:
Adolescent
Sedentary
Urban
School
Male
Exercise
Evaluation
A B S T R A C T
The purpose was to evaluate the reach, dose, and fidelity of Guys Only Activity for Life (G.O.A.L.), a 7-week
pilot intervention conducted from February to March 2011 to increase 6th and 7th grade boys’
moderate-to-vigorous physical activity (MVPA). One middle school was randomly assigned to the
G.O.A.L. intervention and another from the same urban school district in the Midwestern U.S. to a
comparison condition. Thirty boys, ages 10–14 years, participated in each school. The intervention,
guided by the Health Promotion Model (HPM) and Self-Determination Theory (SDT), consisted of a 90-
min after-school physical activity club 4 days/week and one motivational interviewing session with a
registered (school) nurse. Data were gathered via attendance records, club observations, heart rate
monitors, audio-taping of motivational interviewing sessions, and surveys. On average boys attended
the club 2.11 days/week (SD = .86). A trained independent process evaluator reported that the physical
activity club instructors provided the boys with the opportunity for a mean of 25.8 min/day of MVPA.
Using a four-point Likert scale (1 = disagree a lot; 4 = agree a lot), the process evaluator perceived that
the club was delivered with high fidelity and adherence to the underlying theories (M = 3.48; SD = 0.39).
Sessions with the nurse lasted an average of 13 min, 29 s. All boys attended. Two trained independent
coders indicated that the nurse demonstrated at least beginning proficiency for all tasks associated with
motivational interviewing, with the exception of using sufficient open- as opposed to closed-ended
questions and reflections compared to questions. Fidelity related to session delivery and adherence to
the theories was high (M = 3.83; SD = 0.19). The process evaluation data indicated that strategies are
needed to increase attendance and boys’ MVPA during the club time.
� 2013 Elsevier Ltd. All rights reserved.
Contents lists available at ScienceDirect
Evaluation and Program Planning
jo ur n al ho m ep ag e: www .e ls evier . c om / lo cat e/eva lp r og p lan
1. Introduction
United States Department of Health and Human Services(USDHHS; 2008) recommendations call for youth to attain at least60 min of physical activity every day, with most of the hour beingspent engaging in moderate-to-vigorous physical activity (MVPA).Although boys are more active than girls during each year from thebeginning of elementary through the end of high school, thepercentage of boys achieving the physical activity recommenda-tions decreases sharply from close to 49% among 6- to 11-year-oldsto less than 12% by the time age 12 is reached (Troiano et al., 2008).A recently reported study involving trend analyses over close to a
* Corresponding author at: Michigan State University College of Nursing, Bott
Building for Nursing Education and Research, 1355 Bogue Street, C-245, East
Lansing, MI 48824, USA. Tel.: +1 517 353 3011; fax: +517 355 5002;
mobile: +1 734 604 8584.
E-mail address: [email protected] (L.B. Robbins).
0149-7189/$ – see front matter � 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.evalprogplan.2013.09.002
12-year period (between 1999–2000 and 2009–2010) showedsignificant increases in both body mass index (BMI) of 12- through19-year-old boys and obesity prevalence among boys aged 2through 19 years per 2-year survey cycle (Ogden, Carroll, Kit, &Flegal, 2012). No increase in either BMI or obesity prevalenceoccurred for girls (Ogden et al., 2012).
Although schools are touted as excellent settings for promotingphysical activity (Lavelle, Mackay, & Pell, 2012) to combat theoverweight and obesity problem, conflicting evidence still existson whether or not school-based physical activity interventions areeffective in increasing physical activity (Dobbins, De Corby,Robeson, Husson, & Tirilis, 2009; Metcalf, Henley, & Wilkin,2012) or reducing BMI among children and adolescents (Eather,Morgan, & Lubans, 2013; Lavelle, Mackay, & Pell, 2012). Interven-tions involving physical activity conducted during the school dayappear promising as evidenced by some reported findings ofsignificant post-intervention improvements in BMI among bothchildren (Eather et al., 2013) and adolescent boys (Lubans,Morgan, Aguiar, & Callister, 2011). In many schools in the U.S.,
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–3122
however, conducting interventions to increase physical activityduring the school day is not always possible due to the limited timeallotted for this purpose, mainly resulting from competingdemands. The need for physical activity may be perceived as alow priority compared to the time needed for academic subjects(Annesi, Westcott, Faigenbaum, & Unruh, 2005). According to the2006 School Health Policies and Programs Study conducted by theCenters for Disease Control and Prevention (CDC), only 7.9% of allmiddle schools provided daily physical education (PE) or itsequivalent for the recommended 225 min per week for the entireschool year of typically 36 weeks for students in all grades in theschool (Lee, Burgeson, Fulton, & Spain, 2007). Based on the 2009Youth Risk Behavior Survey, only 33.3% of high school studentsparticipate in daily PE (CDC, 2010). Because opportunities forattaining adequate physical activity during the school day arelimited in many schools, supplementary physical activity pro-grams, such as those conducted after school, are needed to fill thisgap (Annesi et al., 2005).
Many physical activity interventions are complex due to theinclusion of multiple components that target both individuals andsocial and physical environments. An individual-level componentmay involve a strategy for building behavioral skills, such as settingpersonal goals. Components related to social and physicalenvironments may include offering opportunities to have funwith others in group classes and providing physical space orequipment for physical activity, respectively. Intervention com-plexity may result in low levels of implementation related tocertain components and potentially unsuccessful outcomes(Young et al., 2008). The possibility also exists that positiveoutcomes can be achieved from an intervention administered in amanner that was very different from what was planned.Inadequate knowledge of exactly how well each component wasdelivered limits the ability to accurately interpret outcomes ormake valid judgments (Durlak & DuPre, 2008).
In order to address this problem, process evaluation, whichinvolves measuring the extent to which an intervention isdelivered or received as planned, has become an essential partof intervention research (Glasgow & Linnan, 2008). Collectingappropriate process evaluation data is important to avoid a Type IIIerror (Basch, Sliepcevich, Gold, Duncan, & Kolbe, 1985), whichoccurs when researchers erroneously conclude that an interven-tion was not effective when, in fact, the lack of significant research
Table 1Summary of process evaluation methods.
Component Characteristic
addressed
Data sources Specific instrume
Physical
activity (PA) club
Reach � Attendance records � Attendance she
Dose delivered � Lesson observations � Lesson observa
Dose received � Heart rate � Heart rate mon
worn by 4 rand
selected boys e
Fidelity � Survey instruments � Process evalua
14-item survey
Motivational
interviewing (MI)
Reach � Attendance at
MI sessions
� Attendance she
date of each in
Dose delivered
and received
� Duration of
MI sessions
� Start and end t
session recorde
sheet and via a
Fidelity � Audio-taped MI sessions
and MITI Code
� Audio-taped MI sessions
and survey instrument
� Audiotapes/rec
� 13-item
survey comple
findings was due to inferior implementation and not to the designof the intervention itself (Glasgow & Linnan, 2008).
Typically, process evaluation involves measuring interventionreach, dose, and fidelity (Linnan & Steckler, 2002; Young et al.,2008). Reach, the proportion of the intended audience thatparticipates in the intervention or in each intervention component(Glasgow & Linnan, 2008), is usually measured by attendance and,therefore, is a characteristic of the participants (Linnan & Steckler,2002). Dose includes what is delivered and received. The former(dose delivered) refers to what is actually delivered to participantsand reflects the efforts or behaviors of the interventionists toprovide the opportunity or the planned amount of intervention.Dose delivered can be measured by an evaluator using directobservation while completing a tool designed specifically for thispurpose. The latter (dose received) involves the extent ofengagement of the participants in the opportunity provided orthe degree to which they are receptive to the intervention andactually use the resources. Similar to reach, the dose received ischaracteristic of the participants (Linnan & Steckler, 2002). Withregard to physical activity, dose received can be evaluated viaobjective measures, such as monitors worn by the participants.Fidelity, a function of the interventionists, refers to the quality ofintervention delivery or the extent to which the intervention wasimplemented in the manner and spirit in which it was intended(Linnan & Steckler, 2002). Measures of fidelity assess whether theintervention is congruent with the underlying theory (Linnan &Steckler, 2002). A thorough process evaluation can assist withunderstanding positive outcomes or elucidating why negativeresults occurred to help identify fruitful approaches for promotingphysical activity in a specific population. Table 1 summarizes theprocess evaluation methods employed in this pilot intervention.
The purpose of this research study is to evaluate the reach, dose,and fidelity of the Guys Only Activity for Life (G.O.A.L.) intervention,a seven-week pilot program designed primarily to encourage low-active 6th and 7th grade boys to increase their MVPA. The G.O.A.L.intervention was based on the integration of the Health PromotionModel (HPM; Pender, Murdaugh, & Parsons, 2011) and Self-Determination Theory (SDT; Ryan & Deci, 2000). According to theHPM, a health-promoting behavior, such as physical activity, canbe influenced by the following cognitive and affective variables:perceived benefits and enjoyment of physical activity; perceivedbarriers to physical activity; physical activity self-efficacy; and
nts Timing Data collectors
et � Daily � PA club instructors
tion tool � Weekly on randomly
selected day each week
� Undergraduate kinesiology
student serving as process
evaluator
itors
omly
ach week
� Weeks 2 and 6 � Undergraduate kinesiology
student serving as
process evaluator
tor’s � Weekly on randomly
selected day each week
(during each observation
period)
� Undergraduate kinesiology
student serving as process
evaluator
et, including
dividual session
� Beginning of intervention � Nurse
ime of each
d on attendance
udiotapes/recorder
� Beginning of intervention � Nurse
order
ted by coder
� End of intervention
� Post-intervention
� Two coders
� Single coder
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–31 23
interpersonal influences, such as social support, role models, andnorms (Pender et al., 2011). SDT postulates that competence,autonomy, and relatedness promote intrinsic motivation to drivebehavior (Ryan & Deci, 2000).
The G.O.A.L. intervention included two major components,both of which were directed toward positively influencing variousHPM and SDT variables: (1) a 90-min after-school physical activityclub offered 4 days a week (Monday–Thursday); and (2) one 15-min individual-level motivational interviewing session con-ducted during the school day by a registered (school) nurse toencourage the boys to attain adequate physical activity. Thephysical activity club was planned for 90 min to allow sufficienttime for providing opportunities for physical activity andinstructing, as well as conducting managerial activities at thebeginning of the club and offering a healthy snack and beverage atthe end of every session. To prevent lack of transportation fromserving as a barrier to attendance after school and allow adequatetime for homework or other responsibilities, two small schoolbuses were provided to take the boys home at a reasonable hour(e.g., 530 P.M.) after every physical activity club session. Dose,reach, and fidelity of the two intervention components, thephysical activity club and motivational interviewing, are evalu-ated. Careful examination of process evaluation data assists inidentifying issues that may be responsible for certain studyoutcomes and, as a result, informs the planning of a future, large-scale intervention.
2. Methods
2.1. Design
The pilot study involved two conditions. Following baselinedata collection, our statistician used flip-of-a-coin randomizationto assign one middle school to the G.O.A.L. intervention, andanother from the same urban school district in the Midwestern U.S.to the comparison condition. Random assignment at the schoolrather than individual level was used to avoid contamination. Thetwo schools were quite similar in race (>60.0% non-White) andclose to 70.0% of the students in each school were eligible for thefree or reduced price lunch, a measure of socioeconomic status(SES) indicating low gross household income based on U.S.guidelines (Hirschman & Chriqui, 2012; U.S. Department ofAgriculture [USDA], 2012).
Boys in the comparison school received a general health-promoting intervention that involved one 90-min workshop afterschool during the same 7-week period. A similar workshop wasoffered on two different days during the same week in case someboys could not attend on a certain day. Bus transportation wasprovided to take the boys home afterward. Twenty boys attendedon the first day, and 9 others attended on the second day. Theworkshop did not address physical activity, but instead includedvarious hands-on activities related to topics such as personalhygiene, hand-washing, self-esteem, anti-bullying/respect forothers, and socialization (e.g., card game activity). The processevaluation was based only on data obtained regarding the G.O.A.L.intervention. Due to the brevity of the pilot, statistically significantbetween-group differences in MVPA were not expected, and thestudy was not adequately powered to achieve this objective.Similar to pilot work conducted by Peralta, Jones, and Okely (2009),the study was designed to inform the development andimplementation of a larger randomized controlled trial.
2.2. Participants
Prior to recruitment, the school district and UniversityBiomedical and Health Institutional Review Board provided
approval to conduct the pilot study. The objective was to recruit30 boys from each school for participation. In January 2011, 327boys from the two middle schools were invited to participate in thestudy. Researchers and trained research assistants visited boysduring their classes or lunch time at school to distribute envelopescontaining a flyer about the study, a letter to inform parents/guardians, assent and parent/guardian consent forms, and ascreening tool to be used for determining eligibility. Boys wereinformed that the researchers were interested only in boys whomet specific inclusion and exclusion criteria. Inclusion criteriawere as follows: able to read, understand and speak English; ableto participate in the study for seven weeks; and not meetingnational MVPA recommendations (one hour or more per day)during the past seven days and/or not currently participating inschool or community sports (Lubans et al., 2011), organizedphysical activities, or lessons involving MVPA that requireparticipation three or more days per week after school. The singleexclusion criterion included: having a health condition preventingsafe participation in MVPA (see Fig. 1 for Screening Tool).
The boys were told if they were interested in participating toreturn the completed forms in the envelope provided to a boxplaced in the main school office or to the school nurse. A total of 316th and 7th grade boys from the intervention school and 30 fromthe comparison school returned completed forms. One boy in theintervention school had to be excluded due to his inability to read,understand, and speak English (see Fig. 2). Because of the small-scale nature of the pilot work and its evaluative purpose, we didnot expend additional effort to recruit more boys or obtainadditional completed forms, such as meeting with the boys orparents/guardians in other venues, mailing information to theirhomes, calling parents/guardians to inquire if they had receivedthe study information, or providing incentives to the boys forreturning the forms.
Of the 30 6th and 7th grade participants in each school, 27(90.0%) in the intervention school and 24 (80.0%) in the comparisonschool were eligible for the free or reduced price lunch. Themajority of the participants were African American (13 [43.3%]from the intervention and 15 [50.0%] from the comparison school).Fifty percent of the participants in each school were not currentlyenrolled in physical education. Height and weight were measuredusing standardized procedures prior to the start of the interventionin order to calculate BMI (weight [kg]/height [m]2; Peralta et al.,2009). Nineteen (63.3%) boys in the intervention and 16 (53.3%) inthe comparison school were overweight (BMI � 85th percentile) orobese (BMI � 95th percentile). Table 2 includes details regardingparticipant characteristics.
The school nurse who conducted the motivational interviewingsession was a female staff member at the intervention school, andthe physical activity instructors included four males, ages 22–45years, from the local community, all of whom had experienceconducting youth physical activity (e.g., dance) or sports programs.To conduct the physical activities, three of the four instructorswere present at the club each day, and one of the three served asthe leader assuming all club management responsibilities.Schedules were planned in advance based on each instructor’savailability. A male teacher who taught PE in the interventionschool, served as an instructional support person and assisted thethree instructors as needed with the various activities offeredduring the physical activity club time. The PE teacher also assistedwith obtaining and maintaining access to rooms in the school andequipment. A total of four individuals were scheduled each day forthe following four reasons: (1) two or three different physicalactivities were offered at the same time on each club day, and aninstructor was needed for each; (2) one instructor was neededevery day to record events and conduct the overall management ofthe club; (3) attendance by the boys each day after school was
Dear Student: Please have an adult help you answer these questions. Student Name: ___________________________ School: ______________________ Grade: _____ Age: _____ Today‛s Date: ___________
Physical activity can be done in sports, when taking lessons, or while playing with friends. MODERATE physical activity (Example: fast walking)
VIGOROUS physical activity (Example: running)
___________________________________________________________________ 1. Can you read, understand and speak English?
a. YES b. NO
2. Do you have any health issue that stops you from doing activities or exercise? a. YES (please explain) ______________________________________________ b. NO
3. Are you doing school or community sports or other organized physical activities ortaking lessons that involve physical activit y and require participation AFTER SCHOOL3 or more days a week? a. YES b. NO
4. During the past 7 days, how many days did you do moderate to vigorous physical activity for a total of 1 hour or more per day? Do NOT count rest breaks. Please circle ONE answer.
0 days/week 1 day/week2 days/week 3 days/week 4 days/week 5 days/week 6 days/week 7 days/week
5. Are you able to participate in this study for 7 weeks of the school year on days thatyou have school?
a. YES b. NO
Fig. 1. Participation screening tool.
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–3124
uncertain; and (4) because instructor absence or illness was alwaysa possibility, a need existed to ensure adequate supervision.
Two female independent coders (an undergraduate kinesiologystudent and a graduate of the master’s program in kinesiologytrained by the first author) evaluated the motivational interview-ing sessions, and an independent process evaluator (a maleundergraduate kinesiology student trained by the second and thirdauthors) evaluated the club. One of the two female coders alsocompleted a survey to provide additional evaluative informationregarding the motivational interviewing sessions (e.g., quality ofdelivery and adherence to theoretical underpinnings).
2.3. Procedure
Evaluations of reach, dose, and fidelity were conducted asrelated to the two major intervention components: the after-school physical activity club and motivational interviewingdelivered by the school nurse during the school day (see Table1). To ensure a comprehensive evaluation, both quantitative andqualitative approaches were employed.
The physical activity club was evaluated via several methods:daily attendance records, lesson observations, heart rate moni-tors, and a survey. Club activities targeted the constructsassociated with the HPM and SDT simultaneously in order tomaximize the impact on the behavior. Table 3 presents theconstructs and includes the strategies used to address each oneduring the club session.
The club began in mid-February and ended on the last day ofMarch during the year 2011. Ninety-minute sessions were plannedafter school four days a week from Monday through Thursday, withthe exception of school breaks (holidays or half days) or schoolcancellations due to snow. Physical activity club instructors weretrained by the research team to conduct each session by including a5-min warm-up with stretching, 60-min opportunity to engage inphysical activity, and 5-min cool-down. Because time is needed formanagerial and instructional or skill-building activities, theobjective was to engage boys in MVPA for at least 50% of theallotted 60-minute period, similar to what is recommended for PEclasses (USDHHS, 2010). The research team told the instructors toprovide at least a 30-min opportunity for physical activity at amoderate intensity during the first 2 weeks. The instructors weretold that a heart rate of 140, which corresponds to 46% of VO2max,was to be considered the cutpoint for moderate physical activity(Allor & Pivarnik, 2000). During subsequent weeks, instructorswere to tell the boys to try to keep slowly increasing the intensityduring the time that they are presented with physical activityopportunities to the point that boys start to sweat or feel warm andexperience an increase in breathing and heart rate, but can stillcarry on a conversation.
To keep the groups manageable, prevent boredom, and increaseopportunities for personal instruction and engagement in MVPA,instructors were to take turns at the beginning of every club day torandomly divide the boys into two or three fairly equal groups.Group size varied depending on the number of boys and instructors
January 2011: Invitat ion to participate in st udy to
6th and 7th gra de boys in 2 middle schools
(n = 170 in Sc hool 1; n = 157 in School 2)
Returned signed consent and assent forms and
assessed for eligi bili ty
(n = 31 in School 1; n = 30 in School 2) Excluded (n = 1) in Schoo l 1
due to in abil ity to rea d,
understand, or speak Eng lish
Baseline assessment (n = 30 in School 1; n = 30 in School 2)
Randomiza tion by school
Recei ved allo cated in tervention co ndition
(n = 30);
School 1 allocated to interventi on condit ion
(n = 30)
School 2 allocated to comparison co ndition
(n = 30)
Recei ved allo cated comparis on condition
(n = 30 )
Anal yzed (n = 30) Anal yze d (n = 30)
157 boys in School 1and 112 boys in Scho ol 2 too k
packe ts explaini ng the study an d including consent
and assent forms
Fig. 2. Flow diagram of participant recruitment.
Table 2Participant characteristics of boys (N = 60; n = 30 in each group).
Items Intervention
n (%)
Comparison
n (%)
Age
11 10 (33.3) 6 (20.0)
12 12 (40.0) 10 (33.3)
13 8 (26.7) 13 (43.3)
14 0 (0.0) 1 (3.3)
Grade
6th 20 (66.7) 14 (46.7)
7th 10 (33.3) 16 (53.3)
Race
African American 13 (43.3) 15 (50.0)
European American 8 (26.7) 8 (26.7)
More than one race 9 (30.0) 6 (20.0)
Asian American 0 (0.0) 1 (3.3)
Ethnicity
Hispanic or Latino 8 (26.7) 7 (23.3)
Not Hispanic or Latino 22 (73.3) 23 (76.7)
Participation in Free/Reduced Lunch Program
Yes 27 (90.0) 24 (80.0)
No 2 (6.7) 3 (10.0)
Did not answer 1 (3.3) 3 (10.0)
Days per week of PE (current)
0 15 (50.0) 15 (50.0)
1 1 (3.3) 2 (6.7)
3 1 (3.3) 0 (0.0)
5 13 (43.3) 13 (43.3)
Body mass index percentile
<85th (healthy weight) 11 (36.7) 14 (46.7)
�85th and <95th (overweight) 6 (20.0) 7 (23.3)
�95th (obese) 13 (43.3) 9 (30.0)
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–31 25
in attendance. Small groups have been used and are recommendedby other researchers (Annesi et al., 2005; Wilson et al., 2006). Forexample, Annesi et al. (2005) reported a 15:1 participant-to-counselor ratio in their after-school program.
In this study, each group was to begin with a differentinstructor-led MVPA offering for approximately 20 min (or 30 minif two groups). Each group focused on one of the following threetypes of activities: (1) Fun games from PE (taught by instructorwith assistance as needed from the PE teacher); (2) dance/hip hop(taught by one of two hired, experienced dance instructors), and(3) sports skills, mainly involving basketball, but also includingswimming, volleyball, karate, and power walking or running(taught by community-based youth sports coach with assistancefrom another instructor if available). After completion of the first20- or 30-min activity period, each group was to switch to anotherinstructor-led MVPA offering for the same period of time until60 min were achieved. In a prior, small-scale study funded by theNational Heart, Lung, and Blood Institute (Robbins, Pfeiffer, Maier,Lo, & Wesolek, 2012), the researchers found that either assigningparticipants to groups or reconfiguring the membership of thegroups each day was important to prevent cliques from formingwith resultant behavioral issues and to provide opportunities forthe development of new friendships. Rotations to differentinstructors prevented participant favoritism of certain instructors.Small groups were also needed due to inconsistent availability oflarge, open areas for physical activity after school. During the last20 min of each physical activity club session, the instructors serveda healthy beverage and snack (e.g., water or 100% fruit juice,vegetable, fruit, or low-fat yogurt or cheese).
The motivational interviewing sessions were evaluated inseveral ways: attendance records, audio-taping, and a survey. Asan initial step, the first author, who completed prior formaltraining in basic and advanced motivational interviewing, com-prised of didactic presentations, group discussions, and role-playsand conducted by a member of the Motivational InterviewingNetwork of Trainers, trained the school nurse to deliver the single
Table 3Theoretical constructs, definitions, and strategies to target each construct in the club sessions.
Theory/model Construct and definition Strategies to target construct in club sessions
HPM Perceived benefits – positive consequences of physical activity Assist boys to identify reasons to be physically active (e.g., make new friends or
be with friends).
HPM Enjoyment – affect or subjective feeling states related to
physical activity
Play music that appeals to boys and offer ‘‘fun’’ physical activities.
HPM Perceived barriers – obstacles preventing physical activity or
personal costs of undertaking physical activity
Assist boys to identify strategies to overcome any reported barriers to physical
activity.
HPM Physical activity self-efficacy – personal capability to engage in
physical activity
Offer activities to promote skill development and provide encouraging and
positive feedback; include activities that are relatively easy to do and at the low
end of moderate intensity, particularly during the 1st 2 weeks of the club, and
progress cautiously to ensure all boys experience periods of success. Recognize
success in a variety of areas (e.g., leadership, spirit, effort, willingness to assist
others, teamwork).
HPM Social support – instrumental and emotional encouragement Offer sufficient and varied opportunities for physical activity to help boys
identify what they may want to continue to do; offer encouragement.
HPM Role models – vicarious learning through observing others
engaged in physical activity
Demonstrate skills and actively participate in physical activity with boys.
HPM Norms – expectations of significant others Help boys understand that physical activity is ‘‘cool,’’ and is the norm for boys
their age and that everyone expects boys to attain adequate physical activity.
SDT Competence – ability to attain valued outcomes in and have an
effect on the environment
Provide clear instructions with skill-building activities and constructive
feedback.
SDT Autonomy – freedom and desire to have an activity be in
agreement with one’s integrated sense of self
Allow boys to choose another option if they do not want to do a certain activity
(e.g., dance or swimming); give boys some control over the intensity of their
physical activity (e.g., moderate or vigorous); provide opportunities for boys to
vote for the activity they want to do.
SDT Relatedness – desire to feel connected to others Help boys feel valued as members and involved in the program; conduct team-
building activities, especially in the first 2 weeks.
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–3126
motivational interviewing session with each boy. Evidencesupports that a train-the-trainer approach that combines differenttechniques, such as interactive methods and learning materials, isan effective way to disseminate information to health profes-sionals and improve clinical behaviors, such as communicationskills (Pearce et al., 2012).
The main purpose of the motivational interviewing sessionwas to motivate each boy to attain adequate physical activityevery day. Various strategies were used to achieve this objective.At baseline assessment, prior to their motivational interviewingsession, boys responded to several instruments assessing the HPMcognitive and affective variables, as related to physical activity.Psychometric data for the following instruments, all of whichwere employed in this study, can be found elsewhere: PerceivedBenefits Scale (Robbins, Wu, Sikorski, & Morley, 2008), PerceivedBarriers Scale (Robbins, Wu et al., 2008), Perceived PhysicalActivity Self-Efficacy Scale (Wu, Robbins, & Hsieh, 2011), PhysicalActivity Enjoyment Scale (Motl et al., 2001), and Social SupportScale (Robbins, Gretebeck, Kazanis, & Pender, 2006; Robbins,Stommel, & Hamel, 2008). The nurse was provided with a one-page computer printout created by the first author that summa-rized each boy’s key responses (e.g., top benefits of or barriers tophysical activity) and the scale scores (e.g., means) associatedwith the cognitive and affective variables. The nurse used theprintout information gleaned from each boy’s responses to theinstruments as a guide to stimulate the discussion during themotivational interviewing session.
At the time of the motivational interviewing session, the nurseexplored the HPM constructs with each boy. For example, thenurse elicited the boy’s perceived benefits of physical activity andperceived barriers to physical activity, including any lack of socialsupport and access to convenient or safe places for physicalactivity. The nurse explored solutions to overcome the barriers,while assessing and employing strategies to enhance each boy’sphysical activity self-efficacy and discussing what he likes to doand how he can make physical activity fun.
The nurse also enhanced each boy’s perceptions related to theconstructs of SDT. Specifically, the nurse supported that each boyhas many choices for physical activity. The nurse also established a
connection or sense of relatedness with each boy by making thesession enjoyable and using the following motivational interview-ing strategies: listening; encouraging expression of ideas; andexpressing empathy and confidence that each boy can indeedchange, but that what he chooses to do is his decision.
2.3.1. Reach and dose
To assist in evaluating reach related to the physical activityclub, a physical activity club instructor who served as the sessionleader recorded attendance every day. To determine dosedelivered, the process evaluator conducted a lesson observationon a different day every week beginning with the second week. Theprocess evaluator used a stopwatch and started it at the beginningof the club session. At each of the six observations that took place,the process evaluator completed an observation form to indicatethe number of minutes used for the required club components(e.g., warm-up, cool-down, healthy snack). The second authorcreated the form by combining characteristics of the following twoinstruments that are also used for observational purposes: Systemfor Observing Fitness Instruction Time (SOFIT; Rowe, Schuldheisz,& van der Mars, 1997) and Academic Learning Time – PhysicalEducation (ALT-PE; Parker, 1989).
The process evaluator used the observation form to documenttime spent in four areas: (1) instruction (e.g., instructor teaching orproviding information about the physical activity or skill); (2)management (e.g., instructor helping boys stay on task), (3) seatedactivity (e.g., boys sitting), and (4) opportunity to engage in lightphysical activity (e.g., boys up and moving or engaging in physicalactivity without any increase in breathing or sweating) or MVPA(e.g., boys engaging in physical activity at least at the intensity of abrisk walk).
To evaluate the dose received, the club instructors were torandomly select 4 boys to wear Polar RS400 heart rate monitors(Polar Electro Inc., Lake Success, NY) during the physical activityclub from Monday through Thursday during week 2 and another4 boys during week 6. The monitor included a chest strap andwrist watch. Heart rate monitors were used for the following tworeasons: (1) they are a valid estimate of energy expenditure(Allor & Pivarnik, 2000); and (2) boys could see their heart rate
Table 4Means of items associated with process evaluator’s observation of the physical activity club.
Item Mean (SD)
The instructors
1. Used positive praise to reinforce good performance/behavior. (HPM: self-efficacy and social support) 3.33 (0.82)
2. Seemed enthusiastic about the lessons/activities. 3.33 (0.52)
3. Gave clear instructions with demonstrations as needed. (HPM: modeling) 3.00 (0.63)
4. Appeared to be prepared to lead the session. 3.00 (1.41)
5. Motivated each boy to increase his moderate to vigorous physical activity in the physical activity club. (HPM: social support; SDT: motivation) 3.25 (0.50)
6. Emphasized the need to be physically active outside the club. (HPM: norms) 3.25 (0.50)
7. Made the club fun for the boys. (HPM: enjoyment) 3.75 (0.50)
8. Gave boys some choice (e.g., re: physical activities they can do). (SDT: autonomy) 3.50 (1.00)
9. Helped each boy feel connected to others in the club (so he felt a sense of belonging in the group). (HPM: benefits; SDT: relatedness) 3.50 (0.58)
10. Helped each boy increase his skills for doing physical activity or sports. (HPM: barriers; SDT: competence) 3.50 (0.58)
The boys
11. Appeared to like the physical activities conducted. 4.00 (0.00)
12. Appeared to like their instructors. 3.80 (0.45)
13. Actively participated in the activities. 3.60 (0.55)
14. Stayed on task. 3.60 (0.55)
Note. Ratings = 1 (disagree at lot); 2 (disagree a little); 3 (agree a little); 4 (agree a lot).
Table 5Motivational interviewing global dimensions: each coder’s mean adherence scores
(N = 9 audio-taped motivational interviewing sessions).
Global dimensions Coder A Coder B
M (SD) M (SD)
Spirit 4.26 (0.36) 3.77 (0.33)
Evocation 4.22 (0.44) 3.78 (0.44)
Collaboration 4.22 (0.44) 3.44 (0.53)
Autonomy/Support 4.33 (0.50) 4.11 (0.33)
Direction 4.33 (0.50) 4.33 (0.50)
Empathy 4.33 (0.50) 3.89 (0.60)
Note. Ratings = 1 (low adherence) to 5 (high adherence).
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–31 27
values and determine if they were being active enough or not. Itshould be noted that the observation captured whether therewas an opportunity to be involved in MVPA, while the heart ratemonitor was used to assess whether the activity was actuallyMVPA. Other researchers have used heart rate monitors tomeasure physical activity. Specifically, Huang et al. (2012) usedthem to measure the MVPA of 4th and 5th grade studentsparticipating in a ballroom dance classroom program.
With regard to the motivational interviewing delivered by theschool nurse, reach was assessed by documentation of the date andduration of sessions completed by each boy. The nurse usedattendance sheets to document this information. To determinedose delivered and received, the researchers evaluated informationfrom the attendance sheets and also asked the nurse to audio-tapethe sessions using two digital tape recorders.
2.3.2. Fidelity
During each period of observation at the physical activity club,the process evaluator completed a 14-item survey to share hisimpressions of the quality of delivery of the club, including thedegree to which the club reflected the theoretical constructs andthe extent to which instructors used strategies to motivate,encourage, or support the boys to increase their MVPA. The processevaluator completed a 14-item survey during each observationperiod. The survey was designed by the investigators, but adaptedfrom those used in other studies (Haerens, Deforche, Vandelanotte,Maes, & De Bourdeaudhuij, 2007; Mauriello et al., 2006; Young etal., 2008). Items were also reviewed by two university-based,doctorally-prepared faculty members, both having expertise inprocess evaluation. Response choices ranged from 1 (disagree a lot)to 4 (agree a lot). A mean above the four-point Likert scalemidpoint or 2.5 was considered high quality of delivery. Specificitems that the process evaluator answered can be seen in Table 4.
To evaluate the motivational interviewing delivered by theschool nurse, two trained undergraduate students transcribed theaudio-recorded information verbatim, and then read each other’stranscripts while listening to the audiotapes to ensure transcriptaccuracy. Following 13 h of training guided by the first author, tworesearch project staff members (a master’s student in kinesiologyand a graduate of the same program), independently listened to theaudiotapes and read the transcripts in order to evaluate thesessions. Specifically, the staff members used the MotivationalInterviewing Treatment Integrity Code 3.1.1 (MITI; Moyers,Martin, Manuel, Miller, & Ernst, 2010) to determine school nurseadherence to motivational interviewing. Moyers and colleagues
(2005) have reported on the MITI’s reliability, validity, andsensitivity. According to the MITI Code, adherence to the followingfive global dimensions is rated from 1 (low) to 5 (high): evocation,collaboration, autonomy/support, direction, and empathy. Evoca-tion, collaboration, and autonomy/support ratings are thenaveraged to yield a global rating of motivational interviewingspirit. To further evaluate intervention delivery, the MITI Coderecommends the use of behavioral counts to calculate a ratio ofreflections to questions and the percentage of all: (a) statementsthat are motivational interviewing adherent as opposed to non-adherent; (b) questions that are open- as compared to closed-ended, and (c) reflections that are complex as opposed to simple(see Tables 5 and 6). For additional information, readers areencouraged to refer to the comprehensive MITI Code 3.1.1, which iseasily accessible via the web (Moyers et al., 2010).
In addition, to further evaluate the quality of delivery of theaudio-taped motivational interviewing sessions, including degreeof adherence to the underlying theory, one of the two codersresponded to 13 items associated with a four-point Likert scale(1 = disagree a lot; 4 = agree a lot; see Table 7). A mean above thescale midpoint or 2.5 was considered high fidelity.
3. Results
3.1. Reach and dose
A total of 30 boys began the G.O.A.L. intervention. On 4 daysduring the 7-week period that the intervention occurred, boys hadno school for various reasons (e.g., holiday, half day, or inclementweather). Therefore, the physical activity club was conducted on atotal of 24 days over the 7-week period.
Table 6Motivational interviewing behavioral counts: each coder’s mean scores and reliability data (N = 9 audio-taped motivational interviewing sessions).
Behaviors Coder A Coder B ICC a
M (SD) M (SD)
Giving information 5.89 (4.17) 4.33 (3.43) .97 .98
MI adherent statements 9.89 (3.95) 8.00 (3.57) .98 .99
MI non-adherent statements 0.11 (0.33) 0.00 (0.00) – –
Total number of statements 10.00 8.00
Closed questions 22.11 (9.25) 22.56 (10.48) .95 .98
Open questions 16.65 (6.84) 16.78 (6.92) .96 .98
Total questions 38.67 39.34 – –
Simple reflections 6.89 (3.18) 8.33 (3.43) .94 .97
Complex reflections 5.78 (4.41) 6.11 (3.86) .87 .93
Total reflections 12.67 14.56 – –
% %
% MI adherent statements 98.90 100.00% open questions 42.82 42.65% complex reflections 45.62 42.31
Ratio Ratio
Reflection:question ratio 12.67:38.67 or 0.33 14.44:39.34 or 0.37
Note. MI = motivational interviewing. As recommended in the MITI Code and based on information included above in the table, boldface values were calculated and used to
evaluate the quality of MI delivery.
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–3128
Reach was evaluated based on attendance sheets completed byclub instructors. On average, boys attended the club 2.11 days perweek (SD = .86; Median = 2.10) or 441 (61.3%) of the 720 possibleopportunities (24 days times 30 boys). Seven boys attended anaverage of three or more days per week. Twenty boys attended oneto 2.9 days per week, while three averaged fewer than one day perweek. A total of four boys arrived either late or left the club earlyfor various reasons, each on only one club day. Based oninformation acquired from audio-taped motivational interview-ing sessions with the school nurse, the most prominent recurringreason for periodically not attending the physical activity clubwas having a school- or non-school-related commitment orresponsibility at home or a previously scheduled appointment.Being sick was another commonly reported problem. Otherbarriers to attendance reported by fewer numbers of boysincluded: having homework, needing to attend to a family issue(e.g., family member in hospital), not wanting to deal withinappropriate language and gestures made by some boys duringthe club, being suspended from or getting ‘‘in trouble’’ at school, orbeing ‘‘too hungry’’ after school.
Beginning with the second week, to determine dose delivered,the process evaluator conducted a total of six weekly lessonobservations, each on a different day of the week for a total of six
Table 7Means of items associated with coder’s evaluation of the motivational interviewing se
Item
1. The nurse let each boy think of his own choices to get more physical activity. (SD
2. The nurse expressed confidence in each boy’s ability to make changes to get regul
3. The nurse seemed to understand how each boy felt about getting regular physical
4. The nurse did a good job listening to each boy. (SDT: relatedness)
5. The nurse encouraged each boy to talk or ask questions about physical activity. (S
6. The nurse helped each boy think of ways to increase his physical activity. (HPM: s
7. The nurse seemed to like meeting with each boy. (HPM: enjoyment; SDT: relatedn
8. The nurse helped each boy come up with a lot of reasons for doing physical activi
9. The nurse helped each boy come up with ways to solve problems that get in the w
10. The nurse tried to help each boy increase his confidence for doing physical activ
11. The nurse helped each boy see that physical activity can be fun to do. (HPM: enj
12. The nurse motivated each boy to increase his moderate to vigorous physical acti
13. The nurse motivated each boy to get regular moderate to vigorous physical activ
Note. Ratings = 1 (disagree a lot); 2 (disagree a little); 3 (agree a little); 4 (agree a lot).
weeks. Only one of the two or three groups of boys was observed ata time. On average, an observed group included 10 boys. Thephysical activity club typically lasted approximately 87–90 min.Following are the mean times spent in various club components:Warm-up (4.00 min; 4.5%); light physical activity, including cool-down (12.33 min; 13.9%); management (23.67 min; 26.6%);instruction (13.38 min; 15.0%); MVPA (25.79 min; 29.0%); andsnack (10.00 min; 11.2%). No time was spent in seated activity. Themost frequently reported process evaluator comment was that, attimes, some boys had difficulty following an instructor’s quick orcomplex physical movements (mainly related to the danceoffering) and needed a slower pace with greater task simplification.
To evaluate dose received, heart rate monitors were used toassess physical activity intensity, but boys complained that themonitors were uncomfortable or constrictive and did not want towear them longer than an average of 11 min on the first day ofweek 2 and 43 min on the second day of week 6. The instructorsforgot to tell the boys to wear the monitors on the first day of week6. The research team told the instructors to avoid forcing boys towear the monitors. Data from week 2 showed the boys had a heartrate �140 beats per minute for 6 of the 11 min worn (53.84% of thewear time). During week 6, boys attained a similar heart rate for 27of the 43 min worn (57.71% of the wear time).
ssions (N = 9 audio-taped motivational interviewing sessions).
Mean (SD)
T: autonomy) 4.00 (0.00)
ar physical activity. (HPM: self-efficacy; SDT: competence) 4.00 (0.00)
activity. (SDT: relatedness) 4.00 (0.00)
4.00 (0.00)
DT: relatedness) 4.00 (0.00)
ocial support) 3.88 (0.35)
ess) 4.00 (0.00)
ty. (HPM: perceived benefits) 3.44 (0.53)
ay of his physical activity. (HPM: perceived barriers) 4.00 (0.00)
ity. (HPM: self-efficacy) 4.00 (0.00)
oyment) 3.44 (0.73)
vity in the physical activity club. (SDT: motivation) 3.67 (1.00)
ity outside the physical activity club. (SDT: motivation) 3.38 (0.74)
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–31 29
With regard to the motivational interviewing session, the nursereported on the attendance sheet that every boy attended asscheduled. As a result, reach was 100%. Dose was determined basedon session times recorded on attendance sheets and data from 9sessions, two of which involved two boys at their request, audio-taped by the nurse. Three boys did not have parental/guardianpermission to be audio-taped. The nurse indicated that timeconstraints prevented her from audio-taping the 16 remainingboys, each of whom had parental/guardian permission andcompleted individual sessions. The duration of the 28 motivationalinterviewing sessions involving the 30 boys ranged from 6 min, 4 sto 20 min, 39 s; and the average length was 13 min, 29 s(SD = 3 min, 55 s). Only four sessions lasted fewer than 10 min.
3.2. Fidelity
As noted in Table 4, survey results indicated the processevaluator perceived the physical activity club was delivered withhigh fidelity (M = 3.48; SD = 0.39). Ratings from the processevaluator were 3.00 or above for all 14 items (M = 3.79; SD = 0.14).
As indicated in Table 5 and based on the 9 audio-taped sessionsused to evaluate fidelity, eight of the 12 global dimension ratingsreceived from the two coders were above 4.0, indicatingcompetency in motivational interviewing delivery (Moyers etal., 2010). Three were above 3.5 or at the level of beginningproficiency. Only one rating, focusing on collaboration, from onecoder was 3.44 or slightly below beginning proficiency.
According to the MITI Code for the behavior counts (Moyers etal., 2010), the goal is to adhere 100% when statements are made.Table 6 shows that the school nurse achieved this objective, asindicated by the high ratings of 98.9% and 100.00% received fromthe coders. To achieve beginning proficiency, the MITI indicatesthat at least 50% of the total number of questions must be open-ended, and at least 40% of all reflections need to be complex(Moyers et al., 2010). Both coders rated the nurse as being slightlybelow beginning proficiency for the former objective and slightlyabove this level for the latter objective. Based on the MITI Code(Moyers et al., 2010), at least an equal, but preferably greater,number of reflections should be stated, as compared to questions.Beginning proficiency, as indicated by a 1:1 reflection to questionratio, was not reached. Both coders noted that the nurse used morequestions than reflections. The level of clinical significance of theinterrater reliability data reported in Table 6 was excellent, asindicated by reliability coefficients between .75 and 1.00(Cicchetti, 1994; Mitcheson, Bhavsar, & McCambridge, 2009).
The coder’s responses to the 13 survey items evaluating theaudio-taped motivational interviewing sessions involving thenurse resulted in an overall mean score of 3.83 (SD = 0.19). Thescore indicated high fidelity related to session delivery andadherence to the underlying theories (see Table 7).
4. Discussion
The process evaluation yielded several important findingsregarding reach, dose, and fidelity related to the physical activityclub and motivational interviewing session conducted by thenurse. Average attendance in the after school physical activity club,indicating reach, was slightly over 60%, which was comparable torates reported in the literature (Barbeau et al., 2007; Wilson et al.,2011). The finding that the main barrier to attendance was aschool- or non-school-related commitment or an appointmentsuggests that asking boys to attend an after-school program fourdays a week may not be realistic due to the multiple demands fortheir time. Offering the physical activity club three days a weekmay be more reasonable. Although this potential solution makessense and allows more time after school during the week for boys
to meet other obligations, reducing structured opportunities mayrequire that greater effort be directed toward assisting boys tounderstand ways they can integrate MVPA into their day on aregular basis outside the club. In one study, when students wereasked why they did not continue their physical activity after astructured after school program had ended, competing demandsemerged as a major barrier (Wilson et al., 2011).
Reach associated with the motivational interviewing sessionsconducted by the school nurse was excellent with every boyattending his respective session. Given the young age groupinvolved, the majority of sessions lasted an acceptable period oftime of at least 10 min (Miller & Rollnick, 2002; Walters & Baer,2006). However, four sessions were relatively brief, indicating thatthe nurse may need a detailed step-by-step roadmap outlining arepertoire of strategies that can be easily employed as needed toenhance communication with less loquacious boys to ensure anadequate dose.
With regard to dose related to the physical activity club, althoughthe researchers planned for at least 30 min/day of MVPA, lessonobservation data showed the instructors actually offered the boys anaverage of 25.79 min/day. Although the MVPA time is comparable towhat is reported in another recently conducted school-basedintervention (Schuna, Lauersdorf, Behrens, Liguori, & Liebert,2013), the inability to achieve the objective underscores someimportant issues. Although the instructors were carefully trained todeliver a structured program with adequate age-appropriate andenjoyable MVPA offerings, they had difficulty trying to reduce theexcessive instruction and management time needed with boys, themajority of whom were overweight or obese and lacked specific,essential, and basic skills. Because more time had to be expendedtrying to help the boys attain a higher skill level, the instructors wereunable to simply offer continuous opportunities for MVPA.Instructors were trained to be supportive and helpful and to avoidcriticism to try to build the boys’ self-efficacy or confidence. Thesmall group size did not appear to be a problem with this particulargroup of boys and allowed for more personal attention andinstruction than what would have been achievable in a larger group.
Because of boys resisting to wear the heart rate monitors(likely due to discomfort from the chest strap and feelings ofconstriction) and instructors forgetting to tell boys to wear them,adequate data concerning dose received could not be obtained.Perhaps, wearing accelerometers may be a more palatableapproach for obtaining an accurate measurement of the timespent at a particular level of physical activity intensity during anintervention. Use of a different approach is also supported byfindings from another intervention, in which Huang et al., 2012reported interference in the data recorded by heart rate monitorsworn by 4th and 5th graders dancing too close to each other.
Despite the fact that the MVPA dose was lower than theresearchers had planned, the process evaluator perceived that theclub adhered to its theoretical underpinnings and quality ofdelivery was high. Results are comparable to other similarinvestigations that yielded positive program evaluations (Gibsonet al., 2008; Melnyk et al., 2007; Steckler et al., 2003).
With regard to fidelity of the motivation interviewing sessions,the school nurse demonstrated at least beginning proficiency forall tasks, with the exception of using sufficient open- as opposed toclosed-ended questions and reflections compared to questions.The results indicate that these two skills need to be strengthened inan initial training session. The findings also support that a singletraining session is probably insufficient (Miller & Mount, 2001) andpost-training booster sessions, such as those providing feedback onaudio-taped sessions, may be critical for achieving a higher level ofcompetence, particularly in a study of longer duration (Miller,Yahne, Moyers, Martinez, & Pirritano, 2004). Despite this situation,the coder perceived that the motivational interviewing sessions
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–3130
reflected the theory on which they were based and quality ofdelivery was high overall. No general agreement concerning thenumber of audio-taped sessions needed to evaluate fidelity wasnoted in the literature. In a recent review of studies involvingmotivational interviewing for smoking cessation, Lai, Cahill, Qin,and Tang (2010) found that monitoring fidelity was highly variableacross the studies, and only one study employed a validated toolsuch as the MITI Code (Moyers et al., 2010).
The study had both strengths and areas for improvement. Thecomprehensive planning that occurred in advance resulted in arigorously conducted process evaluation. Both quantitative andqualitative data on various elements associated with eachintervention component were obtained from a trained processevaluator and two coders. Generalizability of the findings is limiteddue to the involvement of only two schools with one receiving theintervention, and the inclusion of a small convenience sample ofboys in each school. Another limitation involves the short durationof the pilot. Because they are directly promoting physical activityand encouraging the boys to participate, it is generally not feasibleto completely blind program implementers to group assignment(Lai et al., 2010). To potentially reduce the risk of bias, the specificaims and outcomes of the project were not shared with theprogram implementers.
5. Lessons learned
The investigative team learned valuable lessons for use infuture investigations. First, rotating boys in small groups throughdifferent types of MVPA, even as often as every 20 min, helped tomaintain their interest and keep behavioral problems undercontrol. Second, regularly querying administrators and staff atschool regarding other programs offered in the building isimperative for fostering open communication and promotingcollaboration and cooperation. Although this effort may requireadditional time, it enhances the likelihood of being able to continuethe implementation of the intervention and achieve ultimatesuccess. Lastly, following training, a school nurse can conductrelatively high-quality motivational interviewing sessions withboys that focus on increasing physical activity.
6. Conclusion
Although the process evaluation yielded many positive results,some findings underscored areas that could potentially interferewith the ability to achieve successful intervention outcomes. Withregard to the physical activity club, attendance and the minutes ofMVPA offered by the instructors during the club were both lowerthan what the research team had anticipated. This occurrenceindicated the need for specific strategies to improve physicalactivity club attendance and increase MVPA during the club time.Also, during motivational interviewing training, emphasis neededto be placed on training nurses to increase their use of open- asopposed to closed-ended questions and reflections more thanquestions. However, because reflections do not always elicit adetailed response from this young age group, some reflections mayhave to be followed by open-ended questions in a future study, andnurses must be cognizant of and prepared for this situation. Theprocess evaluation data from this pilot study elucidated whereeffort should be directed to enhance the quality of interventionimplementation in a future, large-scale trial.
Funding
The ‘‘Guys Only Activity for Life (G.O.A.L.)’’ intervention wasfunded by Michigan State University (MSU) Clinical and Transla-tional Sciences Institute and the MSU College of Nursing.
Acknowledgements
The authors appreciate the support and assistance from schooladministrators, nurses, teachers, after-school program coordina-tors, and other staff at the time of the study. We thank Kelly Bourneand Marion Bakhoya for assisting us to evaluate the interventionand also Franklin Privette for validating and interpreting the data.We are grateful to the Michigan State University undergraduateand graduate nursing and kinesiology students for their time andeffort to help us during various phases of the project. Weparticularly acknowledge the following students who wereundergraduates at the time of the study: Chris Winther, MaryO’Connell, Ana Van Riper, Mark Pressler, and Jackie Dunayevich.
References
Allor, K. M., & Pivarnik, J. M. (2000). Use of heart rate cutpoints to assess physicalactivity intensity in sixth-grade girls. Pediatric Exercise Science, 12(3), 284–292.
Annesi, J. J., Westcott, W. L., Faigenbaum, A. D., & Unruh, J. L. (2005). Effects of a 12-weekphysical activity protocol delivered by YMCA after-school counselors (Youth Fit forLife) on fitness and self-efficacy changes in 5–12-year-old boys and girls. ResearchQuarterly for Exercise and Sport, 76(4), 468–476.
Barbeau, P., Johnson, M. H., Howe, C. A., Allison, J., Davis, C. L., Gutin, B., et al. (2007). Tenmonths of exercise improves general and visceral adiposity, bone, and fitness inblack girls. Obesity, 15(8), 2077–2085.
Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D. F., & Kolbe, L. J. (1985). Avoidingtype III errors in health education program evaluations: A case study. HealthEducation Quarterly, 12(4), 315–331.
CDC. (2010). Youth risk behavior surveillance – United States, 2009. SurveillanceSummaries (June 4, 2010). Morbidity and Mortality Weekly Report, 59(SS-5), 1–142.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normedand standardized assessment instruments in psychology. Psychological Assessment,6(4), 284–290.
Dobbins, M., De Corby, K., Robeson, P., Husson, H., & Tirilis, D. (2009). School-basedphysical activity programs for promoting physical activity and fitness in childrenand adolescents aged 6–18. Cochrane Database of Systematic Reviews1 http://dx.doi.org/10.1002/14651858. CD007651.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on theinfluence of implementation on program outcomes and the factors affectingimplementation. American Journal of Community Psychology, 41(3-4), 327–350.
Eather, N., Morgan, P. J., & Lubans, D. R. (2013). Improving the fitness and physicalactivity levels of primary school children: Results of the Fit-4-Fun group random-ized controlled trial. Preventive Medicine, 56(1), 12–19.
Gibson, C. A., Smith, B. K., Dubose, K. D., Greene, J. L., Bailey, B. W., Williams, S. L., et al.(2008). Physical activity across the curriculum: Year one process evaluationresults. International Journal of Behavioral Nutrition and Physical Activity, 5, 36http://dx.doi.org/10.1186/1479-5868-5-36.
Glasgow, R. E., & Linnan, L. A. (2008). Evaluation of theory-based interventions. In K.Glanz, B. K. Rimer, & K. Viswanath (Eds.), Health behavior and health education:Theory, research, and practice (4th ed., pp. 487–508). San Francisco, CA: Jossey-Bass.
Haerens, L., Deforche, B., Vandelanotte, C., Maes, L., & De Bourdeaudhuij, I. (2007).Acceptability, feasibility and effectiveness of a computer-tailored physical activityintervention in adolescents. Patient Education and Counseling, 66(3), 303–310.
Hirschman, J., & Chriqui, J. F. (2012). School food and nutrition policy, monitoring andevaluation in the USA. Public Health Nutrition, 1–7 http://dx.doi.org/10.1017/S1368980012004144.
Huang, S. Y., Hogg, J., Zandieh, S., & Bostwick, S. B. (2012). A ballroom dance classroomprogram promotes moderate to vigorous physical activity in elementary schoolchildren. American Journal of Health Promotion, 26(3), 160–165.
Lavelle, H. V., Mackay, D. F., & Pell, J. P. (2012). Systematic review and meta-analysis ofschool-based interventions to reduce body mass index. Journal of Public Health,34(3), 360–369.
Lai, D. T. C., Cahill, K., Qin, Y., & Tang, J. L. (2010). Motivational interviewing for smokingcessation. Cochrane Database of Systematic Reviews1: http://dx.doi.org/10.1002/14651858 (Art. No.: CD006936).
Lee, S. M., Burgeson, C. R., Fulton, J. E., & Spain, C. G. (2007). Physical education andphysical activity: Results from the School Health Policies and Programs Study2006. Journal of School Health, 77(8), 435–463.
Linnan, L., & Steckler, A. (2002). Process evaluation for public health interventions andresearch. In A. Steckler & L. Linnan (Eds.), Process evaluation for public healthinterventions and research (1st ed., pp. 1–23). San Francisco, CA: Jossey-Bass.
Lubans, D. R., Morgan, P. J., Aguiar, E. J., & Callister, R. (2011). Randomized controlledtrial of the Physical Activity Leaders (PALs) program for adolescent boys fromdisadvantaged secondary schools. Preventive Medicine, 52(3-4), 239–246.
Mauriello, L. M., Driskell, M. M., Sherman, K. J., Johnson, S. S., Prochaska, J. M., &Prochaska, J. O. (2006). Acceptability of a school-based intervention for theprevention of adolescent obesity. Journal of School Nursing, 22(5), 269–277.
Melnyk, B. M., Small, L., Morrison-Beedy, D., Strasser, A., Spath, L., Kreipe, R., et al.(2007). The COPE Healthy Lifestyles TEEN program: Feasibility, preliminary effi-cacy & lessons learned from an after school group intervention with overweightadolescents. Journal of Pediatric Health Care, 21(5), 315–322.
L.B. Robbins et al. / Evaluation and Program Planning 42 (2014) 21–31 31
Metcalf, B., Henley, W., & Wilkin, T. (2012). Effectiveness of intervention on physicalactivity of children: Systematic review and meta-analysis of controlled trials withobjectively measured outcomes. British Medical Journal, 345, e5888 http://dx.doi.org/10.1136/bmj.e5888.
Miller, W. R., & Mount, K. A. (2001). A small study of training in motivationalinterviewing: Does one workshop change clinician and client behavior? Beha-vioural and Cognitive Psychotherapy, 29, 457–471.
Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change(2nd ed.). New York: The Guilford Press.
Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). Arandomized trial of methods to help clinicans learn motivational interviewing.Journal of Consulting and Clinical Psychology, 72, 1050–1062.
Mitcheson, L., Bhavsar, K., & McCambridge, J. (2009). Randomized trial of training andsupervision in motivational interviewing with adolescent drug treatment practi-tioners. Journal of Substance Abuse Treatment, 37(1), 73–78.
Motl, R. W., Dishman, R. K., Saunders, R., Dowda, M., Felton, G., & Pate, R. R. (2001).Measuring enjoyment of physical activity in adolescent girls. American Journal ofPreventive Medicine, 21(2), 110–117.
Moyers, T. B., Martin, T., Manuel, J. K., Hendrickson, S. M., & Miller, W. R. (2005).Assessing competence in the use of motivational interviewing. Journal of SubstanceAbuse Treatment, 28(1), 19–26.
Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. (2010). Revised globalscales: Motivational interviewing treatment integrity (MITI) Retrieved January 12,2012 from http://casaa.unm.edu/download/MITI3_1.pdf.
Ogden, C. L., Carroll, M. D., Kit, B. K., & Flegal, K. M. (2012). Prevalence of obesity andtrends in body mass index among U.S. children and adolescents, 1999–2010.Journal of the American Medical Association, 307(5), 483–490.
Parker, M. (1989). Academic Learning Time-Physical Education (ALT-PE) 1982 revision.In P. W. Darst, D. B. Zakrajsek, & V. H. Mancini (Eds.), Analyzing Physical Educationand Sport Instruction (pp. 195–206). Champaign, IL: Human Kinetics.
Pearce, J., Mann, M. K., Jones, C., van Buschbach, S., Olff, M., & Bisson, J. I. (2012). Themost effective way of delivering a train-the-trainers program: A systematic review.Journal of Continuing Education in the Health Professions, 32(3), 215–226.
Pender, N. J., Murdaugh, C. L., & Parsons, M. A. (2011). Health promotion in nursingpractice (6th ed.). Upper Saddle River, NJ: Pearson Education.
Peralta, L. R., Jones, R. A., & Okely, A. D. (2009). Promoting healthy lifestyles amongadolescent boys: The Fitness Improvement and Lifestyle Awareness Program RCT.Preventive Medicine, 48(6), 537–542.
Robbins, L. B., Gretebeck, K. A., Kazanis, A. S., & Pender, N. J. (2006). Girls on the Moveprogram to increase physical activity participation. Nursing Research, 55(3), 206–216.
Robbins, L. B., Pfeiffer, K. A., Maier, K. S., Lo, Y. J., & Wesolek (Ladrig), S. M. (2012). Pilotintervention to increase physical activity among sedentary urban middle schoolgirls: A two-group pretest–posttest quasi-experimental design. Journal of SchoolNursing, 28(4), 302–315.
Robbins, L. B., Stommel, M., & Hamel, L. (2008). Social support for physical activity ofmiddle school students. Public Health Nursing, 25(5), 451–460.
Robbins, L. B., Wu, T. Y., Sikorski, A., & Morley, B. (2008). Perceived benefits and barriersscales related to adolescent physical activity. Journal of Nursing Measurement,16(2), 98–112.
Rowe, P. J., Schuldheisz, J. M., & van der Mars, H. (1997). Measuring physical activity inphysical education: Validation of the SOFIT direct observation instrument for usewith first to eighth grade students. Pediatric Exercise Science, 9(2), 136–149.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation ofintrinsic motivation, social development, and well-being. American Psychologist,55(1), 68–78.
Schuna, J. M., Lauersdorf, R. L., Behrens, T. K., Liguori, G., & Liebert, M. L. (2013). Anobjective assessment of children’s physical activity during the Keep It Moving!After-school program. Journal of School Health, 83(2), 105–111.
Steckler, A., Ethelbah, B., Martin, C. J., Stewart, D., Pardilla, M., Gittelsohn, J., et al.(2003). Pathways process evaluation results: A school-based prevention trial to
promote healthful diet and physical activity in American Indian third, fourth, andfifth grade students. Preventive Medicine, 37(6 Pt 2), S80–S90.
Troiano, R. P., Berrigan, D., Dodd, K. W., Masse, L. C., Tilert, T., & McDowell, M. (2008).Physical activity in the United States measured by accelerometer. Medicine &Science in Sports & Exercise, 40(1), 181–188.
USDA. (2012). Eligibility manual for school meals. Alexandria, VA: USDA Food andNutrition Service.
USDHHS. (2008). 2008 Physical Activity Guidelines for Americans. Washington, DC:Author Retrieved from http://www.health.gov/paguidelines/pdf/paguide.pdf.
USDHHS. (2010). Strategies to improve the quality of physical education. Washington, DC:Author Retrieved from http://www.cdc.gov/HealthyYouth/physicalactivity/pdf/quality_pe.pdf.
Walters, S. T., & Baer, J. S. (2006). Talking with college students about alcohol: Motivationalstrategies for reducing abuse. New York: The Guilford Press.
Wilson, D. K., Griffin, S., Saunders, R. P., Evans, A., Mixon, G., Wright, M., et al. (2006).Formative evaluation of a motivational intervention for increasing physical activi-ty in underserved youth. Evaluation and Program Planning, 29(3), 260–268.
Wilson, D. K., Van Horn, M. L., Kitzman-Ulrich, H., Saunders, R., Pate, R., Lawman, H. G.,et al. (2011). Results of the Active by Choice Today’’ (ACT) Randomized Trial forincreasing physical activity in low-income and minority adolescents. HealthPsychology, 30(4), 463–471.
Wu, T. Y., Robbins, L. B., & Hsieh, H. F. (2011). Instrument development and validation ofPerceived Physical Activity Self-Efficacy Scale for adolescents. Research & Theory forNursing Practice: An International Journal, 25(1), 39–54.
Young, D. R., Steckler, A., Cohen, S., Pratt, C., Felton, G., Moe, S. G., et al. (2008). Processevaluation results from a school- and community-linked intervention: the Trial ofActivity for Adolescent Girls (TAAG). Health Education Research, 23(6), 976–986.
Lorraine B. Robbins, PhD, RN FNP-BC, is an associate professor in the College ofNursing at Michigan State University. Her research has mainly focused on testing theefficacy of theory-based interventions to increase moderate to vigorous physicalactivity and improve cardiovascular fitness, body composition, and cognitive andaffective responses related to physical activity among pre-adolescents and adolescents.She has expertise in using motivational interviewing and computer-delivered individ-ually tailored feedback messages gleaned from survey responses to promote positivebehavior change. As noted in her publications, she has applied both qualitative andquantitative methods in her work.
Karin Allor Pfeiffer, PhD, FACSM, is an associate professor in the Department ofKinesiology and faculty in the Center for Physical Activity and Health at Michigan StateUniversity. She has been studying physical activity and health-related fitness inchildren and youth for the past 16 years. She has experience working with age groupsranging from preschool through college. Her main areas of expertise are in measure-ment of physical activity and interventions to increase physical activity.
Stacey M. Wesolek, MS, has a master’s degree in kinesiology and is the project managerfor a large-scale, 5-year study to test the efficacy of a multi-component, school-basedintervention designed mainly to increase moderate to vigorous physical activity among5th- thru 8th-grade girls (R01HL109101; Robbins, L. B., PI). Prior to her current position,Ms. Wesolek served as the project manager for several smaller investigations focusing onyouth physical activity. She has acquired extensive expertise as a result of her directinvolvement in these studies, particularly with regard to the measurement of physicalactivity via accelerometer, cardiovascular fitness, and body composition.
Yun-Jia Lo, PhD, is a post-doctoral researcher at the School of Natural Resources andEnvironment at the University of Michigan. She recently received her doctoral degreein Measurement and Quantitative Methods at Michigan State University. She hasfocused on causal inference models and social network analysis. Her work has beenpublished in several journals.