behavior-based safety and working alonehomepages.wmich.edu/~rantz/www/5100/working alone...

This article was downloaded by: [Western Michigan University]On: 10 January 2012, At: 11:35Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Journal of OrganizationalBehavior ManagementPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/worg20

Behavior-Based Safety andWorking AloneRyan Olson a & John Austin aa Western Michigan University, USA

Available online: 12 Oct 2008

To cite this article: Ryan Olson & John Austin (2001): Behavior-Based Safety andWorking Alone, Journal of Organizational Behavior Management, 21:3, 5-43

To link to this article: http://dx.doi.org/10.1300/J075v21n03_02

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden.

The publisher does not give any warranty express or implied or make anyrepresentation that the contents will be complete or accurate or up todate. The accuracy of any instructions, formulae, and drug doses should beindependently verified with primary sources. The publisher shall not be liablefor any loss, actions, claims, proceedings, demand, or costs or damageswhatsoever or howsoever caused arising directly or indirectly in connectionwith or arising out of the use of this material.

http://www.tandfonline.com/loi/worg20http://dx.doi.org/10.1300/J075v21n03_02http://www.tandfonline.com/page/terms-and-conditionshttp://www.tandfonline.com/page/terms-and-conditions

EXPERIMENT

Behavior-Based Safety and Working Alone:The Effects

of a Self-Monitoring Packageon the Safe Performance

of Bus Operators

Ryan OlsonJohn Austin

ABSTRACT. Experimental evaluations of Behavior-Based Safety (BBS)processes applied with lone workers are scarce. Clinical and organiza-tional researchers alike have studied the effectiveness of self-monitoringas a performance improvement strategy, but further work is needed to de-termine the power of such interventions for improving safe behavior and

Ryan Olson and John Austin are affiliated with Western Michigan University.Address correspondence to Ryan Olson, 3308 Miami Avenue, Kalamazoo, MI

49048 (E-mail: [email protected]).The authors would like to thank Adam VanAssche and Lisa Olson for implement-

ing critical aspects of the intervention.They would also like to recognize Alicia Alvero and Scott Traynor for their helpful

input regarding the design of the study.

Journal of Organizational Behavior Management, Vol. 21(3) 2001 2001 by The Haworth Press, Inc. All rights reserved. 5

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

to explore the best practices for using such processes with lone workers.

In the current study, four male bus operators (20.5 years average experi-

ence) self-monitored their safe performance and received feedback

based on self-monitoring data. Dispatch supervisors used radio commu-

nication to prompt participants to complete self-monitoring forms and also

conducted special observations of participants to measure target perfor-

mances. Both operators and supervisors were unaware of experimental

observers who measured the performance of each participant by riding on

busses as passengers. A multiple baseline design across performances was

used to assess the effects of the intervention on four performance targets.

The intervention resulted in a 12.3% increase in safe performance for the

group, with individual increases in performance ranging from 2% to 41%

for specific target performances. The results are discussed in terms of the

value of BBS processes for employees who work alone and the research

needed to determine the components of self-monitoring processes that are

most critical for generating improvements in safe performance. [Article copiesavailable for a fee from The Haworth Document Delivery Service: 1-800-HAWORTH. E-mail ad-dress: Website: © 2001 by The Haworth Press, Inc. All rights reserved.]

KEYWORDS. Self-monitoring, behavior-based safety, safe driving,lone workers, bus transit safety, bus operator performance

Over the past 20 years behavioral research in the field of Behav-ior-Based Safety (BBS) has grown steadily. Some of the first conceptualarticles discussing the application of behavior analysis technology to im-prove occupational safety were published in the late 1970’s (e.g., Smith,Cohen, H., Cohen, A., & Cleavland, 1978). The first experimental applica-tions of behavioral technology applied to occupational safety occurred dur-ing the same time period (Komaki, Barwick, & Scott, 1978; Smith, Anger, &Uslan, 1978; Sulzer-Azaroff, 1978). The central foundation of all BBS re-search since these early applications has been the identification and mea-surement of safe and at-risk behaviors and conditions, and the use ofbehavioral technology to increase the frequency of those safe behaviorsand conditions. The body of research has demonstrated the effectiveness ofmany different intervention packages designed to achieve these effects.

Studies have evaluated experimentally the effectiveness of training(Cohen & Jensen, 1984; Komaki, Heinzmann, & Lawson, 1980; Reber &Wallin, 1984; Reddell, Congleton, Huchingson, & Montgomery, 1992),

6 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

goal setting and/or prompts (Austin, Alvero, & Olson, 1998; Berry,Geller, Calef, R. S., & Calef, R. A, 1992; Engerman, Austin, & Bailey,1997; Fellner & Sulzer-Azaroff, 1986; Ludwig & Geller, 1991, 1997;Phillips, Sutherland, & Makin, 1994: Reber & Wallin, 1984; Reber,Wallin, & Chhokar, 1990; Saarela, 1989), verbal and graphic feedback(Alavosius & Sulzer-Azaroff, 1986, 1990; Babcock, Sulzer-Azaroff, &Sanderson, 1992; Chhokar & Wallin, 1984; DeVries, Burnette, &Redmon, 1991; Fellner & Sulzer-Azaroff, 1984; Komaki, Heinzmann, &Lawson, 1980; Nasanen & Saari, 1987; Phillips, Sutherland, & Makin,1994; Sulzer-Azaroff & de Santamaria, 1980), contingent incentives andreinforcement (Austin, Kessler, Riccobono, & Bailey, 1996; Fox, Hopkins, &Anger, 1987; Komaki, Barwick, & Scott, 1978; McAfee & Winn, 1989;Petersen, 1984), and self-monitoring procedures (McCann & Sulzer-Azaroff, 1996) at increasing safe behaviors and conditions. For a recentlypublished, more thorough, review of BBS in manufacturing settings, seeGrindle, Dickinson, and Boettcher (2000). For a review of the impact ofBBS on injury rates, see Sulzer-Azaroff and Austin (2000).

Studies by Geller and colleagues have clear relevance when discuss-ing driving safety. For example, Ludwig and Geller (2000) described aseries of seven studies designed to improve the safe driving of pizza de-liverers. The interventions they evaluated included public and private feed-back, corporate policy changes, commitment card strategies, participativeand assigned goal setting, competition and rewards, and involving thedeliverers as community intervention agents. Ludwig and Geller (2000)reviewed these seven studies in terms of the multiple intervention level(MIL) hierarchy. The MIL is characterized by a continuum of interven-tion intrusiveness and cost, where the least intrusive and most inexpen-sive interventions tend to reach the most people and the most intrusiveand most costly interventions tend to impact the fewest people. TheMIL is not unlike other discussions of treatment intrusiveness in ap-plied behavior analysis (e.g., Meinhold & Mulick, 1990), but the MILspecifically applies these concepts to organizational behavior. Ludwigand Geller (2000) recommended that least intrusive interventions be ap-plied to create large-scale change and that those individuals who remainunaffected by non-intrusive interventions should be exposed to succes-sively more intrusive interventions.

Although discussed in theory by the MIL, experimental evaluation ofself-monitoring procedures to improve safe behavior is a relatively newdevelopment. The field of BBS is growing and reports of successful com-mercial applications with lone workers have begun to surface (e.g.,Krause, 1997; Pettinger, Click, & Geller, 2000). The research base exam-

Experiment 7

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

ining the best practices for improving the safe performance of lone work-ers is small, however, self-monitoring has been widely used in othercontexts as a behavior change technique.

SELF-MONITORING

Richman, Riordan, Reiss, Pyles, and Bailey (1988) conducted astudy that demonstrated the power and utility of self-monitoring proce-dures for improving organizational performance. Richman et al. (1988)used in-service training, self-monitoring, and self-monitoring plusfeedback to improve the on-schedule and on-task performance of staffat a residential setting for persons with mental disabilities. A multiplebaseline design across groups was used to assess the effects of the dif-ferent intervention phases. Three months prior to the study, participantswere informed that a special project was going to take place wherestaff/client interactions would be observed. Under this guise, experi-mental observers collected data for on-schedule and on-task behaviorfor the duration of the study. After baseline data were collected, anin-service training session was held to review job responsibilities thatincluded topics related to on-schedule and on-task performance. Duringthe self-monitoring phase, staff members carried individual schedulecards during the workday and self-recorded the extent to which theywere on-schedule and on-task during the shift. These self-recorded datawere handed in at the end of each shift. For the self-monitoring plusfeedback component, supervisors provided periodic on-the-spot feed-back regarding target performances while the self-monitoring proce-dure continued as before. Two houses participated in the study and werelabeled A and B. For house A, on-schedule behavior averaged 50%,50%, 80%, and 94% across baseline, in-service, self-monitoring, andself-monitoring plus feedback conditions respectively. For house B,on-schedule behavior averaged 39%, 39%, 75%, and 81% across base-line, in-service, self-monitoring, and self-monitoring plus feedbackconditions respectively. For on-task behavior, baseline for both housescombined was 28%. In-service increased the on-task performance ofstaff in house A to 36% but did not affect the performance of staff inhouse B. For house A, on-task behavior averaged 72% and 88% forself-monitoring and self-monitoring plus feedback respectively. Forhouse B, on-task behavior averaged 77% and 80% for self-monitoringand self-monitoring plus feedback respectively.


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Self-monitoring, as part of intervention packages, has also been usedto improve academic performance (Dean, Malott, & Fulton, 1983;DiGangi, Maag, & Rutherford, 1991; Kneedler & Hallahan, 1981; Lan,1996; Stecker, Whinnery, & Fuchs, 1996), to improve the performanceof teachers (Browder, Liberty, Heller, & D’Huyvetters, 1986), to im-prove the performance of athletes (Kessler, 1985; Srikameswaren,1992; Whelan, Mahoney, & Meyers, 1991), to increase interactions be-tween staff and patients at an institution (Burgio, Whitman, & Reid,1983), and to help individuals stop smoking and reduce their caloric in-take (Moinat & Snortum, 1976). Only some of the research listed abovewas conducted with adults and targeted “workplace” performance.However, self-monitoring procedures are potentially relevant across abroader scope of organizational behavior for people who work aloneand for people who work in groups. This broader scope includes supportfor improving the quality, quantity, or timeliness of the performance ofsalespeople or consultants working outside of the home office with cli-ents. People working in teams could use self-monitoring procedures inconcert with peer feedback to track progress on long-term projects or totarget specific “team relevant” skills. In relation to the topic of the cur-rent paper, self-monitoring procedures could complement or substan-tially improve the current performance management strategies utilizedto support and improve the performance of people operating any num-ber of different vehicles in the general transportation and product deliv-ery industries. For example, the first author has been exploringsupplementary performance measurement systems for student pilotsduring the early phases of flight training. An especially risky phase offlight training involves the first series of solo flights without an instruc-tor on board. As part of the exploratory research mentioned above, volun-teer students have been self-monitoring aspects of landing performance ondual (with an instructor) and on solo (without an instructor) flights. Thisproduces data that would otherwise not be available and could poten-tially improve learning and performance, thereby reducing risk. Stu-dents have reported that the procedure has enhanced the learningprocess. If these self-monitoring procedures were used in combinationwith instructors rating the same performances, such systems couldprompt feedback and coaching for specific critical performances. Downthe road when these students become professional pilots working in thecockpits of planes for major airlines, self-monitoring procedures andpeer feedback and discussion could be used to target critical crew re-source management skills. With these potential applications in mind,we can predict that self-monitoring procedures will probably prove

Experiment 9

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

valuable across a wide variety of contexts. However, as self-monitoringresearch and practices expand in organizational settings, we should alsopredict limits to the effectiveness and relevance of self-monitoring pro-cedures in organizational settings.

Self-monitoring procedures have contributed to performance im-provement across many settings and represent a set of methods that maybe especially relevant for improving learning and performance in work-place environments. However, the question of which components ofself-monitoring procedures are most critical for generating behaviorchange is still being explored. For example, the extent to whichself-monitoring data need to be reliable is not clear. Some research sug-gests that self-monitoring procedures produce performance improve-ment even when the self-recorded data are not accurate (Hayes &Nelson, 1983; McCann & Sulzer-Azaroff, 1996). However, whenself-monitoring data are more reliable, effects seem to be enhanced(Baskett, 1985; Kanfer, 1970; McCann & Sulzer-Azaroff, 1996). Itwould be useful to know whether training participants to reliablyself-monitor is a worthy investment. An additional consideration re-lated to the effectiveness of self-monitoring procedures is identifyingthe behavioral functions of the stimuli generated by such procedures.Some of the potential behavioral functions of stimuli produced byself-monitoring processes include: (a) an antecedent function (i.e., in-formational or task clarification), (b) a consequence function (condi-tioned reinforcement or punishment), (c) a rule generating function(i.e., contingency specifying, function-altering stimuli are evoked), and(d) a conditioned establishing operation function.

When a participant is asked to record aspects of his or her behavior,looking at the form, and filling it out may clarify performance expecta-tions or prompt the most appropriate performance. Based upon subse-quent observations of behavior with respect to such informationalstimuli, we would say that participants “know” or “do not know” thesafe manner in which to behave (Skinner, 1953). If self-monitoringfunctions primarily as information or a prompt it would make sense toask participants to self-monitor at the beginning of the workday or justprior to opportunities to perform. Antecedents without consequencesare likely to have only temporary effects due to habituation or extinc-tion (Daniels, 1989). Therefore, it would also make sense to ensure thatpositive consequences were correlated with the antecedent process andthat the self-monitoring procedure was periodically changed.

Aspects of self-monitoring processes may also function as conse-quences. Scoring oneself high or low may function as analogs to rein-


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

forcement or punishment for the desired performance depending uponthe quality of the most recent relevant performance (Malott, R., Malott,M., & Trojan, E., 1999). Thinking of the potential consequence func-tion of self-monitoring procedures may also explain in part why com-pliance with self-monitoring processes is normally less than perfect.Because performance varies, scoring aspects of one’s own performancemight sometimes reinforce and sometimes punish filling out a self-moni-toring form.

Due to the fact that management systems utilize numerous perfor-mance management strategies, filling out self-monitoring forms mayalso cause participants to generate rules related to those strategies. If anorganization regularly uses aversive consequences to discourage unsafepractices, filling out a self-monitoring form might evoke rules such as,“If I improve this performance, I can avoid punishment from my super-visor (because the performances on this form are what he/she caresabout right now).” Schlinger (1993) proposed that a rule such as thisone might produce behavioral effects because it specifies contingenciesand alters the function of stimuli in the immediate environment. For ex-ample, the rule above specifies a new contingency (i.e., my supervisorwill punish me if I don’t improve these behaviors on the form) andmight alter the function of stimuli in the immediate environment (a pre-viously ineffective stop sign now evokes behavior that results in a com-plete stop).

Another way of accounting for the effects of verbal behavior describ-ing contingencies is the concept of the conditioned establishing opera-tion (CEO). An establishing operation is a stimulus or procedure thathas at least two effects; it (1) momentarily alters the effectiveness of areinforcer or punisher, and (2) momentarily alters the frequency of be-havior that has been correlated with the consequence whose effective-ness has been altered (Michael, 1993). Michael has delineated threetypes of CEOs with specific characteristics, but discussing these typesis beyond the scope of this paper. In most cases, CEOs alter the effec-tiveness of conditioned reinforcers or punishers, and given the fact thatmost organizational performance is maintained and shaped by suchconsequences, we should consider the CEO concept a potentially im-portant motivational variable. When approaching a relevant opportu-nity to perform, a rule statement related to the performance beingself-monitored might be evoked. The covert verbal behavior, or perhapsthe stimulus that evoked the covert behavior, may then function as aCEO that alters the effectiveness of salient consequences. For example,a bus driver may perform rolling stops at stop signs because the brakes

Experiment 11

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

squeal less than when he/she performs a complete stop. Participating ina self-monitoring procedure that targeted complete stopping mightcause the sight of a stop sign and/or evoked rule statements to functionas a CEO that momentarily alters the value of the squealing sound, mak-ing it less aversive (weakening motivation to escape or avoid the squeal-ing). Alternatively, CEOs could momentarily establish the squealingsound as an effective reinforcer, thereby evoking behavior (firm footpressure on the brakes) that had produced that consequence in the past.

It is likely that performance improvement generated by self-monitoringprocedures is caused by a complex set of contingencies and behavioralmechanisms. Considering these mechanisms and explanatory conceptsmay guide future research and help discover the most effective practices.With self-monitoring research in BBS being scarce, the field may requiremore studies that demonstrate the effectiveness of self-monitoring proce-dures to improve safe performance before technical questions can be ad-dressed. Below we review two applications of self-monitoring proceduresto improve safe performance that informed the design of the current study.

BBS APPLICATIONS OF SELF-MONITORING PROCEDURES

Preventing Cumulative Trauma Disorders

McCann and Sulzer-Azaroff (1996) used a behavioral approach toprevent cumulative trauma disorders with employees who spent muchof each workday typing in an office setting. Part of the interventionpackage required typists to self-monitor performance along particularbehavioral dimensions. Participants were divided into two groupswhere one group monitored hand and wrist position and the other moni-tored posture. Each participant was exposed to conditions in the follow-ing sequence: (a) baseline, (b) training and self-monitoring, and(c) feedback, goal setting, and reinforcement. During training, partici-pants were taught discriminations between safe and at-risk performanceand were required to pass a discrimination test with a score above 80%correct. Self-monitoring procedures required participants to estimatethe percentage of time they performed target behaviors safely. Duringthe final phase of intervention participants met prior to each session andwere given both graphic and verbal feedback based on levels of safetyobserved by experimenters on the previous days. The graphic feedbackwas in the form of transparencies that, when laid over the participants’self-monitored data, revealed the accuracy of participants’ reports. Ex-


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

perimenters guided participants as they set goals to ensure that goalswere not set higher than the highest data point from the previous ses-sion. And finally, praise was provided for progress and attainment ofgoals.

The study produced consistent improvements in safe performanceacross all participants with moderate to high improvements during thetraining and self-monitoring phase, and very high improvements duringthe feedback, goal setting, and reinforcement phase. Posture ultimatelyimproved to near perfect levels for all participants in the posture group.Hand and wrist position improved to levels clearly above baseline forall participants in the hand/wrist position group.

Participants were not initially given information about the accuracyof their self-estimations of safe performance. Without accuracy infor-mation participants achieved acceptable levels of agreement betweenself-monitored data for posture and experimenter data for posture.However, self-monitoring data for hand and wrist position did not agreewith experimenter data at this stage. Researchers postulated that the“gross motor” nature of the movements involved with posture made thebehavior easier to self-monitor than the “fine motor” hand and wrist po-sition movements, which resulted in the different agreement levels be-tween posture and hand/wrist position. The goal setting, feedback, andreinforcement phase increased the agreement between self-monitoringdata and experimental data for hand and wrist position. The reinforce-ment component (verbal praise) was contingent upon performance im-provement and accurate self-estimations of performance. The researchersreported that high agreement between typists and experimenters was as-sociated with enhanced performance improvement of safe hand and wristposition.

Improving the Safe Performance of Bus Operators

Krause (1997) reported a Behavioral Science Technology, Inc. (BST)consultation effort with a public transportation system where self-moni-toring procedures were utilized. Thirty drivers and several supervisorsparticipated in the project. Interviews with drivers were utilized to de-velop a checklist that contained 34 performances. Drivers estimatedtheir safe performance on these 34 targets once or twice daily and plot-ted their own data on graphs. Every two weeks a supervisor rode witheach driver and collected data using the same checklist.

When the intervention was initially implemented drivers reportedhigh percent safe scores that did not agree with supervisors’ scores of

Experiment 13

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

driver performance. Supervisors discussed these discrepancies withdrivers and plotted the self-monitoring data and supervisor data to-gether on feedback graphs. Over a period of 20 weeks, supervisor datatrended upward and driver data began to trend downward slightly to al-most match supervisor data. Agreement between employees and supervi-sors appeared to take place over time and Krause (1997) reported a 66%decline in injuries and accidents in the organization over the 20-weektime period. However, the project did not employ an experimental designand did not include any formal assessment of the reliability of either su-pervisor or driver data. Therefore, the degree to which driver’s behavioractually changed because of the intervention could not be evaluated.

In order to evaluate experimentally the degree to which self-monitor-ing procedures can improve the safe performance of lone workers, a dem-onstration study similar in design to the McCann and Sulzer-Azaroff(1996) study is needed. The current study was an attempt to synthesizeaspects of Krause (1997) with McCann and Sulzer-Azaroff (1996) andexperimentally evaluate the effectiveness of self-monitoring proceduresfor improving the safe performance of bus operators.

METHOD

Participants and Setting

A public transportation system serving two midwestern cities with acombined estimated population of 160,000 was the sponsoring organi-zation for the study. In addition to operating and maintaining 17 busroutes, the organization operated rail and other public transportationsystems. Within the bus system an operations supervisor managed theperformance of seven dispatch supervisors, who in turn supervised 65bus and other vehicle operators. A university campus route serving acampus of approximately 26,000 students was the location for the studywhere two to eight busses operated from 7 a.m. to 12 midnight on week-days. The bus route consisted of two directional patterns, each lastingabout 30 minutes, and served all major campus locations includingon-campus housing.

Four experienced drivers who worked a 10-hour shift (about 6:30 a.m.to about 4:30 p.m.) were selected by the operations supervisor to partic-ipate in the study (male, ages approximately 40-50; average experience20.5 years, range: 19-23 years). Organizational leadership was inter-


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

ested in this shift because of its duration and the busy pedestrian andtraffic conditions of the university campus.

Prior to the study, the transit system used five methods to motivatesafe driving after drivers were initially trained upon hire. These meth-ods were (a) a $25 bonus for all drivers who worked an entire quarterwithout having a preventable collision, (b) a 7 step progressive disci-pline program for moving violations and preventable collisions, (c) hir-ing private investigators to monitor drivers receiving serious complaints,(d) yearly safety awards at a banquet, and (e) bi-monthly general perfor-mance evaluations by dispatch supervisors. The operations supervisor re-ported that this general management strategy had produced a plateau in totalcollisions per year that had remained relatively stable over the past five years.

Dependent Variables

Dependent variables were identified through an assessment that in-cluded a review of one year of collision reports from the organization’srecords. Passenger and pedestrian injury reports were also reviewed butwere so infrequent that no patterns could be discerned from them. Giventhe high pedestrian traffic conditions of the campus bus route, it islikely that risk for these kinds of events was high when compared toother routes within the transit system. The degree to which acceptableIOA could be achieved was the final consideration for the selectionof dependent variables. Performances observed were divided intothree categories: (a) loading/unloading passengers, (b) bus in motion,and (c) complete stop. Bus in motion performances related to corneringsafely and maintaining adequate following distance were excludedfrom the study due to ceiling effects.

Loading/unloading performances included bus stopping position, re-maining motionless for two seconds after an unload/load instance, andmirror checking. The assessment discovered that 20% of preventablecollisions had occurred at loading zones and another 12% of prevent-able collisions had occurred at parking lots or driveways. Checking mir-rors was identified as a behavior that may have helped prevent 56% ofthe collisions reviewed in the assessment. The route involved in thestudy passed through six major campus parking lots. Moreover, manyloading zones were located near parking lot exits and various otherthroughways.

Bus stopping position was defined as “bus doors must remain shutuntil the bus is completely stopped, and the bus should be positioned sono cars can pass on the right.” Observers scored this performance by

Experiment 15

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

watching the front doors of the bus as it slowed. If the bus was still mov-ing when the doors separated, or if a car could pass on the right, the per-formance was scored at-risk. Two seconds motionless was defined as“the bus should remain motionless for at least two seconds after the lastloading/unloading passenger either steps behind the yellow line on thebus, steps off the bus to the right, or steps clear of the front left corner ofthe bus.” Observers were instructed to count “one-thousand one,one-thousand two,” to themselves to measure this performance andused a wristwatch to periodically calibrate the pace of their counting. Ifthe observer was able to reach “two” before the bus moved the perfor-mance was scored correct while any movement before the observerreached “two” was scored as at-risk. Mirror checking was defined as“the driver should visually check both side mirrors after loading/un-loading passengers as the bus pulls out of a loading zone.” Observerswere instructed to mark this performance as correct if both mirrors werechecked before or as the bus started moving. Checking mirrors after theback of the bus cleared the original load/unload location was scoredat-risk. From the driver’s right hand side of the bus in the second row offorward facing seats his eyes were visible in the center mirror and headmovement could be viewed. If a driver looked in the general directionof either mirror it was assumed he checked that mirror.

Complete termination of forward motion at stop signals is a legal re-quirement and was considered an important safe performance for thecampus route. Drivers making complete stops have a better opportunityto scan traffic and pedestrian conditions at busy intersections. Therewere over 20 stop signals during each 30-minute loop regardless of thedirection the bus was traveling. Rolling stops and jumping a traffic sig-nal were scored as at-risk. The observation technique that achieved reli-ability for complete stops involved picking out an outside object like apole and watching it as the bus slowed. If the outside object stood still inthe observer’s field of vision the performance was scored as safe.

A percent safe score for each dependent measure was calculated bycounting the number of correct scores and dividing that number by thetotal number of observations for that dependent measure, and then mul-tiplying by 100. An overall percent safe score for each observation ses-sion was also calculated in a similar fashion.

Observers and Observation Procedures

The first author and two undergraduate research assistants worked asexperimental data collectors over the course of the study. Undergradu-


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

ate research assistants were selected on the basis of good performancein an organizational psychology class, interest, and availability. Allthree researchers worked for research credits at Western Michigan Uni-versity. Observers sat at the driver’s right hand side of the bus in the sec-ond row of forward facing seats about 10 feet from the driver’s chair.Bus schedules were marked with color codes to locate participants andeach driver was identified by their color code throughout the study forconfidentiality reasons. As an additional measure of confidentiality colorcodes were changed to participant numbers for this paper. Each driverwas generally observed at least once each day for at least 30 minutes (i.e.,one directional loop of the route). However, observers were required tomonitor at least 10 instances of loading/unloading of passengers per ses-sion, resulting in some sessions longer than 30 minutes. On average, therewere 10 or more load/unload instances and over 20 stops observed eachsession.

Once or twice each week all four participants were observed by twoobservers to assess IOA, which was calculated by dividing the numberof agreements by the number of agreements + disagreements, and thenmultiplying by 100. During reliability sessions, the first author was theprimary observer. To protect the independence of observations, the ob-server sitting on the right hand seat next to the window used a three-ringbinder with the left cover held upright to block the visibility of the datasheet. The observer sitting on the left hand seat covered his/her datasheet with his/her right arm and hands (all observers were right handed).

Methods to Minimize Driver Reactivity to Experimental Observers

Participating drivers were not informed of experimental observers un-til a post-experiment debriefing.1 However, it was odd for passengers toride an entire loop of the route without arriving at a destination and driv-ers occasionally asked questions. Observers, who were earning researchcredits for participation, were instructed to answer such questions by say-ing “I’m collecting a survey for a class.” Surveys on bus ridership hadtaken place recently and this proved to be an effective strategy. To furtherreduce the possibility of untoward interactions with drivers observerswere instructed to wear headphones when collecting data by themselves.

Independent Variables

Project Kick-Off Meeting. After an initial baseline phase interven-tion began with an hour and a half meeting at the transit station hub that

Experiment 17

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

consisted of an introduction to BBS and the rationale for piloting such aprocess at the transit system, an introduction to and rationale for aself-monitoring process, and finally a description of the details of run-ning the project. The meeting was conducted by a doctoral student (male,age 26) not involved in data collection and by the operations supervisor.The student was introduced as an external safety consultant without men-tioning his ties to the university. Participants were informed that their in-put about the project would be solicited at a post-project lunch and thatorganizational leadership had signed an agreement that information ob-tained during the project could not be used for disciplinary purposes.Drivers were also told that one or two additional meetings with the con-sultant would be scheduled over the next few weeks. Immediately afterthe kick-off meeting, the student consultant and the operations supervisormet with dispatch supervisors to introduce them to the project. Supervi-sors were not informed of the presence of experimental observers.

Self-Monitoring. Three different self-monitoring forms were usedover the course of the study and were introduced during meetings at thetransit hub (see Experimental Design section below). Drivers used theseforms twice each day during their 10-hour shift to estimate the percent-age of time they performed each of the target performances safely.Blank squares were provided on the form for writing estimations, whichis one strategy suggested by research to avoid shaping respondent an-swers (Schwarz, 1999). At the drivers’ request the locked drop box forself-monitoring forms was located in the drivers’ lounge at the transitsystem hub. Drivers were also told that they would be prompted twice aday by their dispatch supervisors via radio when it was time to self-moni-tor.

Feedback. The first author generated daily color-coded individualand group graphs based on self-monitoring data from the previous day.A research assistant posted a new set of graphs each evening between 8and 9 p.m. in the drivers’ lounge near the drop box and collected com-pleted self-monitoring forms. Each driver was asked to initial the groupgraph at the conclusion of each shift to demonstrate that the feedbackhad been viewed.

Supervisor Prompts and Observations. Dispatch supervisors were in-structed to prompt participating drivers via radio twice each day to usethe self-monitoring forms and record the date and time of their promptson a chart posted in the dispatch office. In addition to deliveringprompts, supervisors conducted special observations of drivers using adata sheet (identical in format to experimental data sheets) limited to theperformances currently being self-monitored by drivers. Experimental


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

observers arranged to measure performance concurrently with supervi-sor observations. On these occasions, experimental observers boardedthe bus prior to the supervisor visit and left the bus one or two stops afterthe supervisor left the bus. This procedure was added to the design ofthe study as a type of probe, where performance changes generated bythe presence of a supervisor could be measured and compared to datacollected on the same day without supervisor presence. To create thiscomparison, each driver was observed for an additional session on thesame day either before or after the supervisor probe was completed.

Independent Variable Integrity. Three measures of independent vari-able integrity were calculated. Percentage of compliance with theself-monitoring procedure was calculated by counting the actual num-ber of self-monitoring forms completed by each driver, dividing thatnumber by the expected number of completed self-monitoring formsfor each driver (two per day), and then multiplying by 100. Percentageof compliance with feedback procedures was calculated by counting thenumber of days each driver signed the feedback graph, dividing that fig-ure by the number of days the driver was expected to sign the feedbackform, and then multiplying by 100. And finally, the percentage of super-visor compliance with delivering prompts was calculated by countingthe number of prompts recorded on the supervisor form, dividing thatfigure by the number of prompts that were expected to be given, andthen multiplying by 100.

Experimental Design

A multiple baseline design across performances was used to assessthe effects of the intervention. Intervention began after a baseline of 9 to11 sessions for each individual driver (group baseline sessions totaled13 because individual driver baseline sessions were obtained across dif-ferent days). Intervention was first implemented for complete stop per-formance and lasted for eight workdays while baseline conditionscontinued for the remaining dependent variables. Phase two added theperformance of remaining motionless for two seconds after loading/un-loading passengers and lasted for five workdays while baseline condi-tions continued for the remaining dependent variables. The third andfinal phase of intervention introduced checking mirrors and bus stop-ping position. After five more working days using this final form, theroute stopped running for the semester and the study was concluded.

Experiment 19

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

RESULTS

Group Performance

Group performance was calculated by creating a group percent safescore for each day of the experiment for each dependent measure. Forpractical purposes, average improvement percentages for each depend-ent measure were then summed and divided by four to obtain a simpleoverall improvement percentage. Using this method, the group im-proved safe driving by an average of 12.3% over baseline conditions.The dependent variable realizing the largest improvement for the groupwas complete stop, which improved by an average of 21.2% (range:14%-41%). Two seconds motionless after loading/unloading passen-gers improved by an average of 11.8% (range: 3%-19%), mirror checkimproved by an average of 10% (range: 3%-15%), and bus stopping po-sition improved by an average of 6.2% (range: 2%-12%). Figure 1 rep-resents the grouped data (i.e., averaged across the four drivers for eachday of data collection) for each of the four dependent variables in themultiple baseline design. As can be seen during visual inspection ofgroup data, calculations of average “improvement” may not indicateclear effects, especially with regard to the last phase of the intervention.For example, visual inspection does not indicate any clear effect for busstopping position (patterns in intervention data closely resemble pat-terns in baseline data). For mirror check, an up trend in the data suggestsan effect, but more data would be required to draw this conclusion.

Individual Performance

The results of individual participants are presented in order of largestto smallest overall improvement. The word “improvement” is usedthroughout the discussion of individual performance, although it shouldbe understood that small average increases for specific dependent mea-sures do not necessarily indicate clear effects of the intervention. Forexample, average improvement percentages for phase 3 of the interven-tion are based on only 3 to 4 data points and should therefore be inter-preted conservatively. Overall improvement percentages for eachparticipant are necessarily influenced by this characteristic of the exper-iment, and should also be interpreted conservatively. Several alterna-tives for computing overall improvement were considered, but forpractical purposes, the same method used to calculate overall group im-provement was applied to individual participants. Percentages related


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Experiment 21

COMPLETE STOP

86.5%

68%46.8%

Perc

entS

afe

100

80

60

40

20

01 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days

LOAD/UNLOAD 2 SECONDS MOTIONLESS

60%

45.3% 57.1%

Perc

entS

afe

100

80

60

40

20

01 3 5 7 9 11 13 15 17 19 21

Days

23 25 27 29

LOAD/UNLOAD MIRROR CHECK99%

68.3%58.3%Perc

entS

afe

100

80

60

40

20

01 3 5 7 9 11 13 15 17

Days

19 21 23 25 27 29

LOAD/UNLOAD STOPPING POSITION

72.8% 79%

95.5%

Perc

entS

afe

100

80

60

40

20

01 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days

FIGURE 1. Group results in multiple baseline design format. Closed circle datapoints are experimenter data averaged for each day of the experiment andopen circle data points are self-monitoring data averaged for each day of theexperiment.

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

to individual performance have been rounded to the nearest whole num-ber to make them easier to read.

Participant 1 improved by an average of 14% over baseline levels.The largest average improvement was for two seconds motionless witha 19% improvement (baseline, 43% safe; intervention, 62% safe). Busstopping position improved 8% (baseline, 70% safe; intervention, 78%safe), mirror check improved 15% (baseline, 73% safe; intervention,88% safe), and stopping improved 14% (baseline, 63% safe; interven-tion, 77% safe). A supervisor probe on the first day of phase two of theintervention created systematic effects on the performance of partici-pant 1. Complete stop and two seconds motionless, which were beingself-monitored, improved to over 20% above the levels measured on thesame day without supervisor presence. Mirror check and bus stoppingposition, which were still under baseline conditions, did not change inthe presence of the supervisor. For a graphic display of these data seeFigure 2.

Participant 2 improved by an average of 13% over baseline condi-tions. His largest average improvement was for complete stop with a41% improvement (baseline, 51% safe; intervention, 92% safe). Thisimprovement stands out as the most clear and dramatic effect of the in-tervention procedures. Bus stopping position improved 3% (baseline,49% safe; intervention, 52% safe), two seconds motionless improved3% (baseline, 28% safe; intervention, 31% safe), and mirror check alsoimproved 3% (baseline, 38% safe; intervention, 41% safe). For agraphic display of these data see Figure 3.

Participant 3 improved by an average of 12% over baseline condi-tions. His largest average improvement was for mirror check with a15% improvement (baseline, 65% safe; intervention, 80% safe). Busstopping position improved 12% (baseline, 81% safe; intervention,93% safe), two seconds motionless improved 12% (baseline, 47% safe;intervention, 59% safe), and complete stop improved 9% (baseline,38% safe; intervention, 47% safe). For a graphic display of these datasee Figure 4.

Participant 4 improved by an average of 10% over baseline condi-tions. His largest average improvement was for complete stop with a19% improvement (baseline, 38% safe; intervention, 57% safe). Busstopping position improved 2% (baseline, 94% safe; intervention, 96%safe), two seconds motionless improved 5% (baseline, 66% safe; inter-vention, 71% safe), and mirror check improved 15% (baseline, 58%safe; intervention, 73% safe). For a graphic display of these data seeFigure 5.


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Experiment 23

COMPLETE STOP

77%63%

100

89

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Sessions

Perc

entS

afe

100

80

60

40

20

0

Perc

entS

afe

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Sessions

LOAD/UNLOAD 2 SECONDS MOTIONLESS 100

83

62%43%

Sessions

242322212019181716151413121110987654321

Perc

entS

afe

0

20

40

60

80

100 LOAD/UNLOAD MIRROR CHECK

88%73%

242322212019181716151413121110987654321

Sessions

0

20

40

60

80

100

Perc

entS

afe


70% 78%

FIGURE 2. Participant 1 results in multiple baseline design format. Closed cir-cle data points are experimenter data, open circle data points are self-monitor-ing data, closed triangles are experimenter data during supervisor probes, andopen triangles are supervisor data.

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12


1 2 3 4 5 6 7 8 9 10 11 12 13

Sessions

14 15 16 17 18 19 20 21 22 23 24 25

100

Perc

entS

afe

80

60

40

20

0

92%

51%

100

100

COMPLETE STOP

1 2

Sessions

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Perc

entS

afe

0

20

40

60

80

100


28% 31%

57

55

2524232221201918171615

Sessions

1413121110987654321

0

20

40

60

80

Perc

entS

afe

100

38%93

64

41%

LOAD/UNLOAD MIRROR CHECK

100

80

60

40

20

0

Perc

entS

afe

Sessions

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

49% 52%

100

91



Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Experiment 25

1

Perc

entS

afe

2

100

3

80

4 5

60

6 7 8

40

9 10 11

Sessions

20

0

12

COMPLETE STOP

7667

38% 47%

13 14 15 16 17 18 19 20 21 22 23 24

Perc

entS

afe

20

0

40

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Sessions

16 17 18 19 20 21 22 23 24


86

70

47% 59%

60

80

100

1 2 3 4 5 6 7

0

20

40

60

Perc

entS

afe

8Sessions

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

65% 80%

100

86


81% 93%

100

75

Perc

entS

afe

0

1 2 3 4 5 6

20

40

7Sessions

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

60

80

100

80

100



Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12


Perc

entS

afe

100

80

60

40

20

0

COMPLETE STOP100

80

1

38%

57%

2 3

Sessions

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

26252423222120

Sessions

191817161514131211109876

0

20

40

60

Perc

entS

afe

54321

80

100

66%

71%


2625242322212019181716151413

Sessions

121110987654321

Perc

entS

afe

0

20

40

60

80

100

58%73%


Perc

entS

afe

0

20

40

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Sessions

17 18 19 20 21 22 23 24 25 26

80

100

94%

96%



Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Results of Self-Monitoring Estimations

It should be noted that drivers estimated their performance for an en-tire day with two self-observations, and experimenters only sampledtheir behavior between 30 minutes to 60 minutes each day. Therefore,the comparison between experimenter and self-monitoring data is notan exact comparison. Drivers’ self-estimations are plotted as open cir-cles on individual results figures.

For participant 1, the average of percent safe estimations across allintervention phases was 72% (complete stop, 79%; two seconds mo-tionless, 67%; mirror check, 94%; bus stopping position, 81%). His ac-tual overall percent safe score, as calculated from experimental observa-tions, was 73% (complete stop, 77%; two seconds motionless, 62%;mirror check, 88%; bus stopping position, 78%). The largest discrep-ancy between his self-monitoring data and experimenter data occurredfor mirror check, with a difference of 6%. The smallest discrepancy oc-curred for complete stop, with a difference of 2%.

For participant 2, the average of percent safe estimations across allintervention phases was 98% (complete stop, 99%; two seconds mo-tionless, 100%; mirror check, 100%; bus stopping position, 81%). Hisactual overall percent safe score as calculated from experimental obser-vations was 53% (complete stop, 92%; two seconds motionless, 31%;mirror check, 41%; bus stopping position, 52%). The largest discrepancybetween his self-monitoring data and experimenter data occurred for twoseconds motionless, where the difference was 69%. The smallest discrep-ancy occurred for complete stop, where the difference was 7%.

For participant 3, the average of percent safe estimations across allintervention phases was 78% (complete stop, 85%; two seconds mo-tionless, 99.9%; mirror check, 100%; bus stopping position, 100%). Hisactual overall percent safe score as calculated from experimental obser-vations was 65% (complete stop, 47%; two seconds motionless, 59%;mirror check, 80%; bus stopping position, 93%). The largest discrep-ancy between his self-monitoring data and experimenter data occurredfor two seconds motionless, where the difference was 40.9%. Thesmallest discrepancy occurred for bus stopping position, with a differ-ence of 7%.

For participant 4, the average of percent safe estimations across allintervention phases was 74% (complete stop, 82%; two seconds mo-tionless, 18%; mirror check, 100%; bus stopping position, 100%). Hisactual overall percent safe score as calculated from experimental obser-vations was 71% (complete stop, 57%; two seconds motionless, 71%;

Experiment 27

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

mirror check, 73%; bus stopping position, 96%). The largest discrep-ancy between his self-monitoring data and experimenter data occurredfor two seconds motionless, where the difference was 53%. The smallestdiscrepancy was for bus stopping position, where the difference was 4%.

Independent Variable Integrity

Group compliance with the rule to fill out two estimations of safeperformance each day was 76.5%. During phases one, two, and three ofthe intervention, compliance was 91.5%, 72.5%, and 60.5% respec-tively. Group compliance with the rule to sign the feedback graph at theend of each shift was 58.8%. During phases one, two, and three of theintervention, compliance was 43.3%, 52%, and 85.5% respectively.Drivers received 68.3% of the supervisor prompts via radio that wereplanned. During phases one, two, and three of the intervention, supervi-sor compliance with the prompting procedure was 66%, 81.5%, and57.5% respectively. Individual participants received at least one prompton 88.3% of the days during the project, and received two daily promptson 48.3% of the days during the project. Independent variable integrityfor individual participants is summarized in Table 1.

Reliability

A total of 99 experimental observations of driver performance tookplace over the course of the study. Two independent observers collecteddata simultaneously for 30 sessions (30.3% of total sessions). The aver-age agreement percentage was 89.8% (range: 70-100). IOA scores werecalculated for each dependent variable for every IOA session. Agree-ment scores under 80 percent were limited to 11 out of 120 total IOAcalculations. Table 2 shows ranges of IOA scores for each dependentvariable over the course of the study.

Debriefing

At the conclusion of the study the participants met with the opera-tions supervisor and student consultant for lunch and debriefing. A sur-vey was administered to the drivers to investigate issues related to thestudy and solicit their opinions about the process, and afterwards, par-ticipants were informed about experimental observers and were eachprovided with a coded summary of self-monitoring results and the aver-age percent improvement for each individual as observed by experi-


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

mental observers. Participant responses to this information were positive.After discussing all questions that were raised during debriefing the oper-ations supervisor left the room while consent was obtained for the use ofdata.

Survey results showed that participants believed their self-monitor-ing estimations were accurate to slightly high. Participants also identi-fied which performances had actually changed the most and which hadchanged the least over the course of the study. Drivers were given op-portunities in the survey to describe why they thought their perfor-mance had improved or stayed the same. Comments on this topic wereinformative and will be presented when relevant in the discussion sec-tion. Participants rank-ordered aspects of the project from most to leastuseful in the following order: (1) being able to share opinions about theproject, (2) talking with co-workers about safety and aspects of theroute, (3) meetings to discuss the project, (4) using self-monitoringforms, (5) graphs of safe performance, (6) process not attached to disci-

Experiment 29

TABLE 1. Independent Variable Integrity

Participant and Variable Phase One Phase Two Phase Three All Phases

Participant 1

Self-Monitoring 100.0 100.0 33.0 82.0

Feedback 50.0 33.0 67.0 50.0

Supervisor Prompts 80.0 66.7 66.7 73.0

Overall IV Integrity 76.7 66.6 55.6 68.3

Participant 2

Self-Monitoring 100.0 69.0 67.0 83.0

Feedback 83.0 75.0 100.0 85.0



Participant 3

Self-Monitoring 83.0 63.0 67.0 73.0

Feedback 40.0 75.0 75.0 62.0



Participant 4

Self-Monitoring 83.0 38.0 75.0 68.0

Feedback 0.0 25.0 100.0 38.0



Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

pline in any way, (7) supervisors observed the same behaviors we did,and (8) more frequent contact from supervisors. All four participantsrecommended extending the use of customized self-monitoring pro-cesses to other parts of the organization for both new and experienceddrivers. They also responded favorably to having the union participatein choosing target behaviors.

DISCUSSION

The results of the study suggest that a self-monitoring package canchange the safe performance of bus operators. Furthermore, the studyrepresents a rare empirical evaluation of lone worker performance.However, because of the small number of participants and short dura-tion of the study, it cannot be concluded that changes in safe behaviorled to an important decline in collisions. All four participants were “col-lision free” for five weeks, but the transit system as a whole had threeseparate months without collisions in 1997.

The overall effects of the intervention were small to moderate (12.3%overall average improvement; individual performance improvement onspecific targets range: 2% to 41%). This may have been due in part tothe lack of participant involvement in activities such as performancetarget selection and design of the process (i.e., “buy-in” activities). Thefact that participants were aware of the short-term nature of the projectmay have also contributed to this effect and, for some participants, mayhave been the reason for low treatment compliance. Perhaps some didnot take the procedures “seriously” because the process was presentedas temporary rather than permanent. Only future research can answerthese questions conclusively. However, the results of this study do sug-


TABLE 2. Inter-Observer Agreement Percentages for Each Dependent Vari-able

Dependent Variables Average % IOA Range % IOA Sessions < 80%

Bus Stopping Position 93.2 70.0-100 2

2 Seconds Motionless 90.0 73.0-100 0

Mirror Check 84.1 70.0-100 8

Complete Stop 91.8 77.2-100 1

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

gest that there are limits to the effectiveness of self-monitoring inter-ventions (i.e., not all behaviors improved; certain antecedent conditionsor additional variables may be necessary to ensure that all behaviors im-prove). Indeed, it may be very difficult to ensure that self-monitoringpackages produce substantial behavior change.

The Cumulative Benefits of Small Effects

In the current study the intervention phases were relatively short,with the entire intervention lasting only three weeks. Without any op-portunity to significantly involve participants or to allow participants tobecome familiar with the new process, a 12.3% improvement in overallsafe performance was achieved. It is possible that larger effects wouldoccur under more supportive circumstances. This fact notwithstanding,we should consider the possible practical importance of the level of be-havior change observed in the current study. Mawhinney (1999) notedthat it is important to consider how small to moderate improvementswould impact an organization over time. To investigate this issue wewill consider the potential impact of the changes made by participant 3,whose overall average improvement of 12% was not clearly visible to usin graphic form until after mean lines were added (see Figure 4). As sug-gested by Mawhinney (1999), we agree that “cumulatively large benefitscan result from incrementally small intervention effects” (p. 83).

On the campus route there were usually about 10 instances of load-ing/unloading passengers every 30 minutes. During a ten-hour shiftwith a regular flow of passengers, each driver could stop to unload orload 200 times each day. During baseline conditions, participant 3checked both side mirrors 65% of the time when loading/unloading pas-sengers. This would represent 130 safe mirror checks out of 200 oppor-tunities each day. During brief intervention conditions, he checked bothside mirrors 80% of the time. This would represent 160 safe mirrorchecks out of 200 opportunities each day. During one month perform-ing at baseline levels participant 3 would achieve 2080 safe mirrorchecks out of 3200 opportunities whereas one month of interventionlevel performance would achieve 2560 safe mirror checks out of 3200.So a 15% average improvement on checking mirrors could result in asmany as 480 fewer at-risk load/unload instances each month. If the re-maining 64 drivers working in the transit system were also participatingand improved to similar levels (assuming similar passenger rates), thetransit system could realize 31,200 fewer at-risk behaviors each monthand 374,400 fewer at-risk behaviors each year. Managers and practitio-

Experiment 31

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

ners applying BBS with lone workers should find moderate effect sizespromising, especially when maintained over longer periods of time.

Behavioral Functions of the Self-Monitoring Package

The effect size and variability produced in this study are similar to ef-fects and variability generated with antecedent interventions targetingsafety (e.g., Austin, Alvero, & Olson, 1998; Engerman, Austin, &Bailey, 1997; Ludwig & Geller, 1997; Streff, Kalsher, & Geller, 1993).Although the performance targets of the above studies are not identicalto the targets in the current study, all seem to involve a common safetydilemma where immediate and probable consequences support riskyperformance, while delayed or improbable consequences fail to supportsafe performance. Given the similar behavioral underpinnings of perfor-mance targets, it is interesting to note that the self-monitoring package gen-erated effects similar in magnitude to purely antecedent strategies. Incontrast to antecedent strategies, most safety studies that use pro-grammed consequences have demonstrated much larger changes in be-havior (e.g., Austin, Kessler, Riccobono, & Bailey, 1996; Sulzer-Azaroff & de Santamaria, 1980). These results, combined with the ef-fect size and variability issues discussed above, leads us to believe thatour self-monitoring procedure might have served an important anteced-ent function.

One may also argue, in line with Hayes and Nelson (1983), that thewhole self-monitoring package (the instructions, the sheets, theprompts, and posted feedback) made more effective the natural conse-quences of the particular performances we measured. That is, monitor-ing complete stops, for example, could have made the potentialconsequences of behaving unsafely (e.g., colliding with student pedes-trians or other vehicles) more salient. One driver’s answer to a surveyquestion highlights this issue. When explaining why some of his behav-ior did not change very much, participant 2 circled the statement “acci-dents/collisions just don’t happen often enough to warrant any extraeffort to prevent them.” Alternatively, participant 1 reported the follow-ing with regard to the self-monitoring package, “It caused me to con-sider the effects on others (students) of my errant behavior (rollingstops).” Aversive outcomes like collisions, as horrific as they may be,tend to be too improbable to motivate safe behavior. In addition, the saf-est way of doing things often requires the person to endure immediateaversive conditions (taking longer to complete a task, wearing uncom-fortable personal protective equipment, etc.). Reports such as those


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

from participant 1 suggest that the self-monitoring package, in somecases, overcame the contingencies favoring risk-taking by making theconsequences for unsafe acts more salient. From a molecular analyticperspective, the self-monitoring package may have generated CEOsthat altered the effectiveness of direct-acting reinforcers or punishers(Malott, R., Malott, M., & Trojan, E., 1999; Michael, 1993).

Discussion of Individual Performance

It was hoped that very consistent effects would be observed across par-ticipants, or at least systematic improvements related to the degree towhich participants complied with intervention procedures. However,each participant’s largest improvement was not necessarily the most ac-curately self-estimated performance. Among individual participantsthere were very small to very large improvements for specific target per-formances. Understanding these individual differences in performancerequires a consideration of the accuracy of each participant’s self-moni-toring estimations, the extent of exposure to the independent variables(i.e., independent variable integrity) for each participant, the self-reportdata obtained from each participant, and anecdotal information obtainedby experimental observers. Because aggregate data can obscure interven-tion effects and relationships between variables, we chose to providemore detailed analyses of the data for participants 1 and 4.

Participant 1 Performance

Participant 1 realized the greatest average improvement (14%) andthe most consistent improvements of any participant. He was also themost accurate self-estimator of safe performance. Upon visual inspec-tion of his data, it is clear that his estimations closely tracked his actualperformance (see Figure 1). The data from participant 1 support thefindings of McCann and Sulzer-Azaroff (1996), suggesting that thegreatest improvements in safe performance occur when participants aremost accurate in their self-estimations.

Overall independent variable integrity for participant 1 was 68.3%(Phase one, 76.7%; Phase two, 66.6%; Phase three, 55.6%). The declinein integrity percentages was largely the result of decreased participationin self-monitoring procedures (he was 33% compliant during phasethree). This may explain the sharp drop in his performance on bus stop-ping position during the last two days of intervention (see Figure 1).

Experiment 33

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

Anecdotally, participant 1 appeared to be deliberate and conscien-tious and seemed to take great pride in his profession. It is possible thatcertain “personality characteristics” (i.e., verbal responses to questionson valid psychometric instruments) could predict the effectiveness of orcompliance with self-monitoring procedures in some cases. Participant1 also responded very systematically to supervisor observations, duringwhich his performance on the variables being self-monitored was about20% higher than his performance on the same day without supervisorpresence. During supervisor observations, dependent variables thatwere not being self-monitored remained at baseline levels. This effectsuggests relatively low reactivity to experimental observers as com-pared to reactivity to supervisor presence.

Participant 4 Performance

Participant 4 achieved the smallest overall improvement for thegroup (10%) and also had the lowest overall independent variable integ-rity of all participants (59.7%). The clearest effects for this participantoccurred during the first phase of intervention. During baseline condi-tions he seemed to come to a complete stop only when he was forced todo so by traffic conditions. His typical pattern of performance was toroll slowly through stop signs. This distinctive performance duringbaseline made behavior changes observed on the first day of interven-tion very dramatic. The deterioration of this improvement in perfor-mance was also distinctive as it gradually returned to baseline levelsover the six sessions of phase one (see Figure 5). Contributing to this ef-fect may have been the fact that participant 4 did not ever sign the feed-back graph during phase one of the intervention, suggesting he did notview the graphs, which may have eliminated a consequence componentfrom the self-monitoring package.

Another clear effect achieved during the first phase of the study forthis participant occurred during the supervisor probe. He scored 30%higher on complete stops when the supervisor was present than he didwhen the same performance was measured on the same day without thepresence of a supervisor. In addition, the baseline dependent measuresall showed slightly lower performance with the supervisor present thanthey did without the presence of the supervisor, showing that the partici-pant was reactive only to the performance being self-monitored.

At the onset of phase two, participant 4’s performance dropped to 20%and 16% safe on 2 seconds motionless and complete stop respectively. Atthat time we hypothesized that this might represent counter-controlling


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

behavior in response to the intervention procedures (Ludwig & Geller,1999; Skinner, 1953). However, this low performance did not continuebeyond the first day of phase two of the intervention.

We selected participant 4’s performance to analyze because it demon-strates the fact that grouping data across behaviors within a single partici-pant can conceal true effects; just as effects can be concealed by groupingthe performance data of several individual participants (as in group re-search). To clarify, we consider his performance relative to the accuracyof his estimations in more detail. Although his estimations were the leastaccurate and his overall performance changed the least of all participants,the accuracy of participant 4’s self-estimations did not seem to systemati-cally vary with his performance improvement. The smallest discrepancybetween his self-estimations and experimenter data occurred for busstopping position with an average difference of 4%. However, this per-formance improved by only 2% over baseline levels (see Figure 5). Incontrast, his estimations differed from experimenter data by 25% forcomplete stop, where he realized his greatest improvement (19% overbaseline levels). Another large discrepancy occurred for two seconds mo-tionless, which improved by only 5%, where his self-estimations were53% lower than experimenter data. If we were to look only at his overallbehavior change and his overall estimations (i.e., data grouped across be-haviors), the data would suggest a clear relationship between his estima-tions and the resulting behavior change (i.e., he had the smallest overalleffect size and reported the least accurate data), especially in light of par-ticipant 1’s results discussed above (i.e., who reported the most accuratedata and had the largest overall effect size). However, as one can see fromthe more detailed analysis that considers each behavior singly, the rela-tionship between estimations and behavior change is far from clear. It islogical that accuracy of self-estimations would affect the degree to whichbehavior changes. However, the degree of agreement between self-moni-toring and experimenter data in our study did not predict the degree of im-provement for each participant on particular performance targets.Accuracy is only estimated by assessing IOA (Johnston & Pennypacker,1993). Therefore, with regard to the importance of the accuracy ofself-monitoring data, we have only scratched the surface of the topic.

FUTURE RESEARCH

There are many unanswered questions regarding applications ofself-monitoring procedures to improve the performance of lone work-

Experiment 35

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

ers. The most important question in terms of safety is the effectivenessof these kinds of procedures at reducing accidents and/or injuries.Krause (1997) reported a 66% reduction in accidents and injuries in hisapplication with bus drivers, but methodological issues prevent draw-ing firm conclusions about the degree of behavior change generated bythe self-monitoring procedure. While we were able to successfullybuild upon the consultation effort reported by Krause (1997) by experi-mentally evaluating behavior change, we were not able to answer ques-tions about accident/injury reduction. In order for researchers to drawconclusions about accident/injury reduction, it may be necessary topartner with companies or consulting firms implementing behav-ior-based safety processes with lone workers on a large scale. Academicparties could ensure reliable assessment of behavior change and compa-nies or consultation firms could ensure a large-scale implementationwith a long duration that could impact accident/injury rates. An addi-tional question that still remains unanswered is the extent to which ac-curacy of self-monitoring influences its effectiveness (McCann &Sulzer-Azaroff, 1996). Such research could begin by training partici-pants to accuracy at the beginning of a study and then assessing drift andconcurrent performance levels with confederate observers or someother unobtrusive measurement system over time. Some electronicforms of driving performance measurement are now becoming avail-able that might be of use in such research. The issues discussed aboveare excellent research questions, however, both require that behaviorchange be produced reliably by a self-monitoring procedure. It is hopedthat the successes and failures of the current study will inform the devel-opment of self-monitoring procedures that can reliably produce sub-stantial changes in safe behavior.

Several methodological improvements are needed in order for exten-sions and/or replications of this work to (a) more clearly assess behaviorchange and (b) generate greater behavior change. First, baseline and in-tervention phases of longer duration would allow researchers to discernbehavioral effects more reliably. This is especially relevant for the thirdphase of the intervention that targeted mirror check and bus stoppingposition, which lasted for only a few sessions. Even if methodologicalimprovements indicate no effect for any one dependent measure, thisenables researchers to draw stronger conclusions about the “bound-aries” of the effectiveness of an intervention. In other words, research-ers could then explore questions about why some performancesimproved while others did not. Research of this kind would be costly interms of labor and time, but would enable researchers to draw stronger


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

conclusions about behavioral changes. Second, potentially more pow-erful intervention variables should be tested. An obvious choice wouldbe the addition of valuable consequences that were dependent uponsome aspect of driver performance (e.g., participation or performancelevels as assessed by observers). In addition to adding more powerfulconsequences to such interventions, we feel that there are at least twoless costly or less intrusive conditions that might add power to the inter-vention: these are (a) participants selecting performance targets of per-sonal value or interest and (b) evaluating an implementation that wasperceived as permanent rather than temporary. We discuss these two is-sues, as well as research that might reveal the behavioral functions ofstimuli generated by self-monitoring procedures, below.

A key component missing from the current study was the absence ofemployee participation in the design stages of the project and other ac-tivities said to generate “buy in.” Both Krause (1997) and McSween(1995) heavily promote employee participation in BBS processes. As-pects of such employee involvement may function as motivational vari-ables where the value of consequences related to safety improvement isincreased and behaviors correlated with those improvements are morefrequently evoked. When behavior is analyzed on a molecular scale(i.e., behavior is analyzed in terms of its immediate antecedents andconsequences), Michael’s (1993) taxonomy of CEOs may be relevant.

Participant comments on the debriefing survey suggest that employ-ees may make the greatest improvements when they “value” the targetperformance. Technically, values can be defined as a set or constella-tion of conditioned reinforcers (Malott, R. Malott, M., & Trojan, 1999).Participant 2 improved complete stops by 41% and his survey com-ments regarding the self-monitoring process emphasized this specifictarget performance. He wrote, “Complete stops are important. A lot canhappen in a short amount of time at an intersection. Really have to stopcompletely to see the whole picture.” These results and self-report com-ments concur with McSween (1995), who suggested that learning expe-riences prior to the onset of a BBS initiative may be importantstrategically and that individuals’ values should be incorporated intoperformance improvement initiatives. Future research should examinemore closely this potential relationship between employee “buy-in” ac-tivities and the effectiveness of and compliance with treatment proto-cols.

Whether self-monitoring procedures with lone workers tend to func-tion primarily as antecedents, as does a prompt for safety belt use, is aninteresting research question. Researchers have suggested that self-moni-

Experiment 37

Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

toring is effective as a result of an individual applying consequences con-tingent upon his or her own behavior (Kanfer, 1970). Future researchcould explore this question by requiring the self-monitoring to take placeeither just before (antecedent function), or just after (consequence func-tion) a work shift and/or by measuring the use of self-delivered conse-quences through talk-aloud procedures. If self-monitoring tends tofunction primarily as an antecedent, practitioners should emphasize the ad-dition of programmed reinforcement to ensure prolonged effectiveness.Future researchers should also consider the clinical literature on self-moni-toring. For example, some clinical research suggests that the power ofself-monitoring procedures is enhanced when participants monitor the fre-quency of undesired, rather than desired, performance (Kopp, 1988).

DISCUSSION AND CONCLUSION

Strengths of the current study include measures of independent vari-able integrity, collection of self-report measures at the conclusion of thestudy, and supervisor probes. In some cases IV integrity measures pro-vided insight into unusual patterns in the data. The survey instrumentgave participants a chance to express their opinions about aspects of theprocess, and gave us a chance to collect information about covert be-havior that may have impacted their performance. Supervisor probes af-fected performance systematically across all participants and mayrepresent one method for assessing participants’ understanding of thetarget performances. For some participants the probes demonstratedthat they understood and were capable of performing the target behav-iors at high percent safe levels. For participant 2, however, the probeshowed that he may not have understood or discriminated certain targetbehaviors. His performance improved when the supervisor was present,but only to 55% and 64% for 2 seconds motionless and mirror check, re-spectively. In general, the probes demonstrated that supervisor presencewas a more powerful intervention than self-monitoring, and that partici-pants improved only the behaviors that supervisors were observing.

Weaknesses of the current study include the relatively short durationof the intervention, the absence of meaningful outcome measures (alsodue to the short duration), the small number of participants, the lack ofemployee “buy-in,” and the small to moderate effect size of the interven-tion. The study was cut short because the particular bus route terminatedfor summer break, so we could not determine whether performancechanges maintained, improved, or deteriorated over time. As mentioned


Dow

nloa

ded

by [

Wes

tern

Mic

higa

n U

nive

rsity

] at

11:

35 1

0 Ja

nuar

y 20

12

previously, future researchers should consider methods that provide moretime for the stabilization of performance under each experimental condition.In addition to concerns about the ability to assess behavior change, a reduc-tion in collisions or injuries is not possible unless more participants are in-volved over long periods of time. With regard t

behavior-based safety and working alonehomepages.wmich.edu/~rantz/www/5100/working alone...

Documents