SPECIAL SECTION: SUPERVISION PRACTICES

Towards a Competency-Based, Ethical, and Socially ValidApproach to the Supervision of Applied BehaviorAnalytic Trainees

Laura B. Turner1 & Aaron J. Fischer2 & James K. Luiselli3

# Association for Behavior Analysis International 2016

Abstract Competency-based supervision of trainees has re-cently come to the forefront of behavior analytic practice;however, there are minimal data to support the effectivenessof various supervision practices on trainee outcomes.Accordingly, this paper is intended to spark further discussionand research activity regarding the supervision of those seek-ing to become Board Certified Behavior Analysts (BCBA).We present a practice model and considerations for supervis-ing applied behavior analytic trainees consistent with theBehavior Analyst Certification Board (BACB) SupervisorTraining Curriculum Outline (Behavior Analyst CertificationBoard, 2012b), the Professional and Ethical Compliance Codefor Behavior Analysts (Behavior Analyst Certification Board,2014), and extant literature from behavior analysis and relatedfields. Inherent to the current model is a focus on bi-directional feedback and collaboration between the supervisorand trainee to frequently evaluate the acceptability of the pro-cedures, process, outcomes, and effectiveness of supervision.We present a Supervision Monitoring and Evaluation Formconsistent with the current model and discuss the assumed

importance of objective and subjective self-assessment of su-pervisor competence to the ultimate advancement of the prac-tice of applied behavior analysis.

Keywords Applied behavior analysis . Competency-basedsupervision . Social validity . Supervision evaluation form .

Trainee

Supervision is a crucial activity of most Board CertifiedBehavior Analysts (BCBA) and is especially important giventhe growing number of applied behavior analytic trainees andthe expanding demand for applied behavior analysis (ABA)services worldwide. It is generally assumed that quality super-vision will contribute to the development of effective andethical practitioners, protection of clients, and advances inthe field. Initial research also suggests that the intensity ofsupervision provided by applied behavior analysts is positive-ly related to clinically significant improvements in client per-formance (Eikeseth, Hayward, Gale, Gitlesen, & Eldevek,2009). However, until recently, there were minimal standardson how to provide a valuable supervision experience fortrainees. In 2011, the Behavior Analyst Certification Board(BACB) initiated a Supervision Task Force to develop defin-itive supervision guidelines and training standards for the field(Behavior Analyst Certification Board, 2011). Resulting fromthis task force were minimal competencies required of thoseproviding supervision to individuals pursuing a BACB certi-fication or practicing as Board Certified Assistant BehaviorAnalysts (BCaBA) or Registered Behavior Technicians (RBT;Behavior Analyst Certification Board, 2012b). Relevant is-sues were designing a supervision contract, implementing be-havioral skills training (BST), and delivering frequent perfor-mance feedback.

Electronic supplementary material The online version of this article(doi:10.1007/s40617-016-0121-4) contains supplementary material,which is available to authorized users.

* Laura B. Turnerlbturner@usj.edu

1 University of Saint Joseph, 1678 Asylum Avenue, WestHartford, CT 06117, USA

2 University of Utah, 1721 Campus Center Drive, Salt LakeCity, UT 84112, USA

3 Clinical Solutions, Inc. and North East Educational andDevelopmental Support Center, 1120 Main Street,Tewksbury, MA 01940, USA

DOI 10.1007/s40617-016-0121-4

Published online: 28March 2016

Behav Analysis Practice (2016) 9:287–298

Underlying this competency-based approach is the as-sumption that, rather than Bosmosis^ (Falender &Shafranske, 2012, p. 133), supervision is an independent areaof practice and requires acquisition, refinement, and mainte-nance of skills; although strong clinical and research skills areessential to being a competent supervisor, these skills alone donot automatically equate to strong supervisory skills.Attention is now being given to the development of supervisorcompetencies and identification of effective components ofsupervision, a process mirrored by several other helping pro-fessions, such as social work (Kraemer-Tebes et al., 2011),psychology (American Psychological Association, 2015;Falender et al., 2004), and medicine (Frank et al., 2010).

The purpose of this paper is to present a practice modelthat integrates evidence-based training and supervisionpractices with the complex ethical and practical consider-ations regarding supervision. Because there are currentlylimited data on the effectiveness of supervision and specif-ic supervision practices in developing well-trainedBCBAs, BCaBAs, and RBTs, the current model is guidedby the supervisory experiences of the authors, extant liter-ature from behavior analysis and related fields, the BACBSupervision Training Curriculum Outline (BehaviorAnalyst Certification Board, 2012b), and the newly devel-oped Professional and Ethical Compliance Code forBehavior Analysts (Behavior Analyst Certification Board,2014), which now more comprehensively reflects the pro-fessional responsibilities of supervisors. Some of these rec-ommendations may be time-consuming and go beyond therequirements of the BACB; however, these points of con-sideration on the application of existing supervisory andethical guidelines may enhance supervisory practice.Note that we have adopted the term Btrainee^ throughoutto refer to BCBA and BCaBA candidates, as well as prac-ticing BCaBAs and RBTs as they require ongoing super-vision and cannot practice independently.

Within this framework, we review the many competen-cies required of supervisors including fluency with the eth-ical guidelines regarding supervision, effective trainingtechniques across basic and complex skill areas (e.g., clin-ical decision-making and professionalism), and perfor-mance feedback methods to support acquisition and main-tenance of skills. Our premise is that the implementation ofan evidence-based approach within a positive, collabora-tive, and individualized environment fosters a well-rounded and effective trainee. We also provide examplesand considerations for how to incorporate these competen-cy areas into a practice repertoire in which supervisor be-havior, in addition to trainee behavior, is frequently evalu-ated and modified based on trainee feedback and progress.Focus is placed on the use of social validity measures to aidin the ongoing adjustment of supervisory behavior and theongoing self-assessment of the supervisor.

Practice Model for Behavior Analytic Supervision

Set the Occasion for Collaborative and EthicalSupervision

A strong and collaborative supervisory relationship is widelyacknowledged as contributing to the quality and effectivenessof clinical supervision across several other disciplines (e.g.,Bernard & Goodyear, 2014; Bordin, 1983; Falender et al.,2004). Although notoriously complex and difficult to objec-tively define and measure, the supervisory relationship is typ-ically defined in two parts. In behavior analytic terms, the firstpart can be described as the degree to which a supervisor and atrainee are mutual discriminative stimuli for generalized rein-forcement (i.e., Brapport^ as discussed by Carr and colleagues,1997, p. 111). Carr and colleagues (1997; Magito,McLaughlin, & Carr, 2005) further discuss rapport as a settingevent and demonstrate that the development of rapport is a keyaspect to approaching skill development. Although their dis-cussion is specific to individuals with developmental disabil-ities, the basic behavioral principles behind their observationshave clear implications for the supervisor-trainee relationship.

As an example, the relative rapport between a supervisorand trainee may impact the trainee’s approach behavior to-wards the supervisor. If a supervisor does not function as acue for reinforcement (or the supervisor functions as a cue forpunishment), it is likely that the trainee will avoid the super-visor and instead seek out others, including non-behavior an-alysts, for guidance and feedback. While the specific behav-iors required to develop rapport within the context of the su-pervisory relationship have not yet been systematically iden-tified, a few face valid examples that may assist supervisors inestablishing themselves as reinforcers include being ap-proachable (e.g., visibility and nonverbal behaviors) and re-sponsive (e.g., to emails), using more positive statements rel-ative to negative (e.g., 4:1 ratio; Daniels & Bailey, 2014), anddemonstrating undivided attention towards the trainee duringmeetings. Importantly, these behaviors certainly do not ensurethat the supervisor will produce a competent trainee, and othervariables are also likely to impact the degree to which thetrainee approaches the supervisor with relevant issues andquestions (e.g., supervisor competency).

The second component of the supervisory relationship canbe described as the degree to which a supervisor and traineegenerate mutually agreed upon expectations and goals for su-pervision and engage in behavior consistent with those expec-tations and goals. The supervision contract, a requirement ofBACB approved supervision (Behavior Analyst CertificationBoard, 2015), can be used to guide the development of thesupervisory relationship and the terms of supervision (seeGuideline 5.05; Behavior Analyst Certification Board,2014). The contract fosters an initial and collaborative discus-sion between a supervisor and trainee with regard to the nature

288 Behav Analysis Practice (2016) 9:287–298

of the professional relationship and the expectations and re-sponsibilities of each member. Supervisors can use this dis-cussion to model shared expectations and commitment to thesupervision process.

Self-report data from related fields suggests that a poorsupervisory relationship is related to a trainee’s decreasedwillingness to disclose relevant clinical information (Mehr,Ladany, & Caskie, 2015) and follow through with a supervi-sor’s recommendations (Ladany, Hill, Corbett, & Nutt, 1996),as well as increased likelihood of burnout (Gibson, Grey, &Hastings, 2009). However, to what extent the quality of thesupervisory relationship influences the outcomes of supervi-sion is unknown and should be a focus of future research (seean extended discussion below). Until then, we hypothesizethat the development of a technically skilled, ethical, and pro-fessional behavior analyst may potentially suffer without thepresence of a strong supervisory relationship. Thus, an impor-tant first, and ongoing, step in the supervision process may beto build a strong supervisory relationship. As part of, or inaddition to the supervision contract, we suggest supervisorsparticipate in the following discussions with trainees to furtherestablish mutual expectations for the supervision experienceand ensure consideration of relevant ethical guidelines in aneffort to avoid potential harm to trainees and the professionalsupervisory relationship.

Define the Relationship The first step in any professionalrelationship is to define basic and fundamental expectationsbetween participating individuals. While there are clear exam-ples of inappropriate supervisory relationships that often donot necessitate a lengthy initial discussion (e.g., sexualrelationships; see Guideline 1.07; Behavior AnalystCertification Board, 2014), subtle aspects of the supervisoryrelationship may be unclear to the trainee and should be clar-ified. The supervisory relationship can be confusing as thesupervisor may serve as a teacher, mentor, evaluator, and fa-cilitator of self-assessment and growth, all of which are over-lapping roles and imply a slightly different style of profession-al interaction and power differential. For example, a supervi-sor is commonly defined as someone who oversees certainactivities and holds evaluative power over the trainee, whilea mentor is often described as a trusted advisor and confidant.Further, the everyday practice of behavior analysis can bestressful, often physically and mentally exhausting, makinginitial discussion of a supportive relationship beneficial.

While all these roles are supervisory in nature and holdingthese roles concurrently with a trainee does not necessarilyconstitute a multiple relationship as defined by the BACB(Behavior Analyst Certification Board, 2014), supervisorsmay find themselves differentially allocating their time acrossseveral of these roles depending on the current context ofsupervision and the trainee’s experiences. It may be beneficialto discuss these potentially changing roles at the outset of

supervision and to engage in self-assessment and monitoring,as well as peer consultation, to ensure the role fits the currentneeds of the trainee while still within the parameters of aprofessional relationship.

Further, trainees will already have negative and/or positiveexperiences working with people in these various roles andmay have preconceived expectations about the nature of thecurrent supervisory relationship. For example, trainees withpast supervisors who did not encourage collaboration maybe reticent to present contrasting ideas or raise concerns withtheir new supervisors. It may be fruitful for supervisors to asktrainees about their definition of supervision and their previ-ous experiences being supervised (whether or not in behavioranalysis) and to discuss how their definition and experiencesalign with those of the supervisor and the most recent versionof the BACB Experience Standards (i.e., Behavior AnalystCertification Board, 2015). Also important is the verbal rec-ognition of and sensitivity to the fact that the supervisoryrelationship can be impacted by beliefs, values, and interper-sonal biases shaped by previous experiences. Discussing suchissues may reduce the likelihood of dissimilar views and ex-pectations regarding the supervision process.

Paying for supervision further complicates the role of thesupervisor and trainee. Many individuals seeking a BACBcredential do not have access to local or Bin-house^ BCBAswho are approved to provide supervision. This challenge re-quires those seeking supervision to identify approved BCBAsupervisors in their community or even across great distances(through videoconferencing). When seeking supervisionthrough this format, supervisors typically charge an hourlyrate, which is determined by the supervisor and payment forsupervision services must be discussed with the trainee at theonset of the supervisory relationship. Since trainees pay forservices in this arrangement, the supervisor and trainee mustconsider any potential negative impact payment may have onthe professional supervisory relationship.

Performance Expectations It is important to clearly define,outline, and agree upon the performance expectations andresponsibilities of both the trainee and supervisor. Dual re-sponsibilities foster a give-and-take relationship and allowfor shared accountability. To ensure a clear understanding ofthe expectations of the trainee and supervisor according to theBACB, we recommend reviewing the BACB ExperienceStandards (Behavior Analyst Certification Board, 2015) atthe outset of supervision, even if the trainee has already ac-crued supervision hours with another supervisor. It is alsoimportant to identify any idiosyncratic expectations of thetrainee and/or supervisor that, if not discussed, may placestress on the supervisory relationship. Such expectationsmay be that both parties must contact each other at least24 h in advance to reschedule or that both parties will respondto email correspondence within 48 h.

289Behav Analysis Practice (2016) 9:287–298

The degree to which either a trainee or supervisor can con-sistently complete the agreed-upon responsibilities is oftenimpacted by an individual’s time and competing commit-ments. Conducting an analysis of time for both parties is onemethod for outlining reasonable responsibilities. With regardto the supervisor’s time, consider a supervisor supervising onetrainee who is accruing 30 h/week of Supervised IndependentFieldwork. Per the Experience Standards (Behavior AnalystCertification Board, 2015), this equates to 3 h of supervisionfor every 2-week supervisory period (i.e., 5 % of total accruedhours); however, 3 h of supervision almost always requiresmore than 3 h of time. Often overlooked in the initial calcula-tion of time is (a) travel time to and from the trainee’s work-place for direct observations (i.e., if different from the super-visor’s workplace and if audio-video recording or videocon-ferencing is not viable), (b) travel time to and from supervisionmeetings (i.e., if videoconferencing is not an option), (c) timeto search for appropriate articles and resources for the trainee,and (d) time to read (or re-read) those articles. Additional timeis required in responding to trainee emails, reviewing traineepermanent products, giving detailed feedback during the su-pervision meeting, completing performance evaluations,documenting various aspects of the supervision process(e.g., remediation), and seeking peer feedback, and/or consul-tation with regard to the supervisory behaviors of the supervi-sor. Taken together, these activities can quickly increase those3 h every 2 weeks to 6 h or more every 2 weeks! We encour-age supervisors and trainees to complete this activity togetherto establish appropriate workloads and expectations for bothparties (see Guideline 5.02; Behavior Analyst CertificationBoard, 2014).

Evaluation of Performance Expectations One of the mainresponsibilities of supervisors is to consistently evaluate anddocument trainee performance. This matter is often an area oftrainee distress and an area without strict guidelines, so it iscrucial to gain initial agreement about the behaviors to beevaluated (e.g., technical, professional, and ethical), the eval-uation process, and the consequences of poor performanceevaluations. Agreeing on the terms of a poor evaluation ofthe trainee beforehand may make execution of the set conse-quences more straightforward and perhaps have fewer nega-tive side effects. Include in the guidelines clear, objective andmeasureable circumstances for all consequences. For exam-ple, at what point will a remedial plan be developed (e.g., after3 consistent supervisory periods of unsatisfactory perfor-mance), what will the remedial plan consist of (e.g., an addi-tional individual meeting with the supervisor, extra readings),and what is the consequence of performing poorly during, ornot completing, the remediation (e.g., supervisor terminatingthe relationship)? Lastly, if the relationship is ultimately ter-minated, it will be important to pre-determine what, if any,hours the supervisor will approve.

Supervisors are highly encouraged to seek feedback fromtrainees concerning their performance as a supervisor andtheir ability to follow through with the agreed upon expecta-tions. We recommend communicating the value of this ap-proach to the trainee at the outset of supervision to furthercreate a sense of shared accountability with respect toobtaining the goals of the supervisory relationship.Obtaining candid information from trainees can be difficult(Mehr, Ladany, & Caskie, 2010), and supervisors must con-sider the inherent power differential; however, a strong super-visory relationship between the supervisor and trainee and anexpectation of the reciprocal nature of feedback is hypothe-sized to counteract this limitation.

Confidentiality The BACB Professional and EthicalCompliance Code (Behavior Analyst Certification Board,2014) states that behavior analysts have a primary obligationto protect the confidentiality of the individuals they supervise(Guideline 2.06). Confidential information in a supervisoryrelationship predominantly includes performance evaluations,such as written reviews or ratings on evaluation forms. It maybe beneficial to initially communicate to trainees that this in-formation will be kept out of public sight and only discussedwith others that have a clear, professional reason for beingconcerned with such matters, such as a trainee’s on-sight su-pervisor. It may also be beneficial to let trainees know thatindividual corrective feedback will be given in private and notduring group supervision sessions.

Given the sometimes close, professional relationships thatcan develop through supervision, the trainee may disclosepersonal information germane to his or her behavior analyticwork. As an example, a trainee may acknowledge an interper-sonal conflict with a professional colleague. Direct conversa-tion about the confidentiality of such disclosures may increasethe likelihood that a trainee approaches their supervisor con-fidently and without fear of disclosure so that the issue may bediscussed and an appropriate solution determined.

Remote Supervision With the advent of contemporary vid-eoconferencing technology, behavior analytic supervision canoccur remotely by interacting through live two-way audio-video communication across the Internet or cellular networks.Supervision through videoconferencing allows supervisors toengage in supervision sessions with trainees without having tobe in the same physical location, which results in reducedtravel time, increased savings associated with travel costs,and the potential to provide supervision for individuals in ruralor hard to reach areas who may not have access to locallyqualified behavior analysts.

Despite these benefits, supervision conducted through vid-eoconferencing also poses many potential challenges. Thesechallenges include breaches of client confidentiality, technicaldifficulties, and lack of physical presence of a supervisor

290 Behav Analysis Practice (2016) 9:287–298

during high-risk situations (Florell, 2016). Supervision con-ducted via videoconferencing must ensure client confidential-ity by facilitating sessions through HIPAA and FERPA com-pliant videoconferencing software (e.g., Adobe Connect,Vidyo, VSee); however, due to perpetual changes to video-conferencing software, supervisors should thoroughly scruti-nize potential software before using it during remote supervi-sion. Readers are also encouraged to review Cavalari, Gillis,Kruser, and Romanczyk (2015) for an extended discussionregarding digital sharing and storage of audio-visualrecordings.

Regarding technical difficulties, supervisors and traineesshould be familiar with the videoconferencing software thatis used, ensure that they have the required hardware for high-quality videoconferencing (e.g., 720p or higher resolutionwebcam, high bandwidth Internet connection, computer/lap-top/tablet with fast processing speed), and become familiarwith the evidence-based practice guidelines for telementalhealth services (cf. Grady et al., 2011). To guarantee supportfor trainees during high-risk situations, the supervisor shoulddevelop a plan with the trainee so that an on-site supervisor isavailable or the supervisor can be reached by phone (Abbasset al., 2011; Panos, Panos, Cox, Roby, & Matheson, 2002).

Termination of the Relationship Just as expected with cli-ents, the parameters of planned or unplanned termination ofthe supervisory relationship should be discussed at the outsetof supervision (Guideline 2.15; Behavior AnalystCertification Board, 2014). The BACB encourages traineesto accrue experience hours with different supervisors.Accordingly, an initial conversation about the number of su-pervisors trainees have already worked with and the numberof experience hours trainees have accrued is warranted to de-termine a mutually agreed upon duration for the current su-pervisory relationship. Additionally, if the trainee is transfer-ring to another supervisor, we encourage the previous super-visor to speak with the new supervisor about the strengths andweaknesses of the trainee and current goals. This dialogueshould be initiated with approval from the trainee. In the eventof an unplanned termination such as the supervisor changingemployers, development of a contingency plan will ensure thetrainee continues to receive supervision.

Baseline Assessment of Trainee Skills

Although there are no standard procedures to initially assesstrainee skills, we can incorporate strategies from the field ofbehavior analysis to guide our evaluation. We suggest the fol-lowing methods as strategies to evaluate trainee baseline per-formance across various skills: behavioral observation, infor-mal assessment, review of course syllabi, discussion with pre-vious supervisors, and review of the BACB Fourth EditionTask List (Behavior Analyst Certification Board, 2012a).

Using these indirect and direct methods, the supervisor andtrainee are able to collaboratively develop goals that are objec-tive, measureable, and obtainable (Cooper, Heron, & Heward,2007; Daniels & Bailey, 2014). We will discuss each of thesestrategies and their contribution to assessing trainee skills.

Behavioral observation, the basis of all behavior analyticwork, requires a behavior analyst to operationally define be-haviors of interest and subsequently record the occurrence ofthose behaviors. In the context of supervision, a supervisorshould observe and record specific trainee behavior such asthe extent to which an intervention or behavioral support planis implemented with integrity (Gresham, Gansle, & Noell,1993), verbal behavior of a trainee as they interact with staff,colleagues, clients, or parents, and reliability of trainee behav-ioral observation through assessment of inter-observer agree-ment (Cooper et al., 2007).

Informal (formative) assessment, although not behavioral,is useful to gather information from a trainee. This type ofassessment is frequently used in education (Dunn &Mulvenon, 2009) and is achieved through conversations withtrainees about their previous experiences, completion ofchecklists, and review of writing samples or portfolios. Suchassessment should combine records review, semi-structuredinterview, and discussion about the areas of practice as a be-havior analyst. The informal assessment should be used togather information about past experience but also specificareas of strength and weakness. Additionally, as an ongoingskills assessment, supervisors should ask questions of thetrainees to evaluate their knowledge of behavioral procedures(e.g., BWhat were the four test conditions of the Iwata, Dorsey,Slifer, Bauman, and Richman (1994) functional analysis[FA]?^). Depending on trainees’ understanding of the FA lit-erature and their prior implementation history, supervisorsshould ask follow-up questions (e.g., BWhat are pros and consregarding the manymodifications to the original—Iwata et al.,1994—FA?^). These types of questions allow supervisors togauge trainees’ skills and trainees’ responses provide a base-line assessment of their ability to think critically and discussbehavior analytic theory broadly or more acutely.

Review of course syllabi with the trainee is quite usefulwhen assessing skills. In a collaborative format within super-vision sessions, the supervisor and trainee may benefit fromdiscussing the topics covered in the trainee’s BACB-approvedcoursework. Independent of the supervision session, the super-visor is encouraged to review the syllabi to ensure that relevantreadings have been assigned—this action provides the oppor-tunity to introduce readings to the trainee that might not havebeen assigned. The benefit of assigning readings within thesupervision context is, as a supervisor, you can work individ-ually with your trainee and critically analyze and discuss thebehavior analytic literature. Subsequently, reviewing articles isassumed to teach critical thinking that is essential for appliedwork and preparation for the upcoming BACB exam.

291Behav Analysis Practice (2016) 9:287–298

Discussion with previous supervisors is a valuable way toassess a trainee’s skill set; however, the availability and ease ofthese options is variable depending on the practicum site. Forexample, if trainees receive their experience hours at a largehuman service provider with multiple schools within the or-ganization, trainees might change schools or classrooms. Inthis situation, the supervisors might change as well, leavingthe opportunity for discourse between the prior and currentsupervisor.

Finally, it is recommended that the supervisor andtrainee jointly review the BACB Fourth Edition Task List(Behavior Analyst Certification Board, 2012a) and use thedocument as a checklist of areas the trainee has alreadymastered and those that are still in need of instruction andpractice.

Methods to Teach and Promote Skills

Acknowledging that trainees have varying levels of skills,supervisors can expect at some point that they will need todirectly and systematically teach new skills. Importantly, thesupervisor must attempt to match teaching techniques andperformance expectations to the trainee’s current perfor-mance. For example, a novice trainee might not be fluent withall the different approaches to conducting preference assess-ments. In this case, the supervisor could indirectly discussmethods and provide relevant literature for the trainee to readabout the topic and improve his or her knowledge. For thetrainee who is familiar with the literature on preference assess-ments but has not actually conducted such an assessment witha client, the supervisor could model correct procedures. Lastly,for the trainee who has mastered designing and implementingpreference assessments, the supervisor could speak with thetrainee about possible modifications to implementing prefer-ence assessments and/or ask that the trainee uses behavioralapproaches to teach the skill to another staff member or parent.

In each of the cases described above, the supervisor en-gages in different behaviors and those behaviors are appropri-ate to the trainee’s skill set. Taking the time to comprehensive-ly evaluate the current skill level of a trainee and set appropri-ate behavioral goals and expectations may allow for a morepositive and successful supervisory experience for the trainee.As well, this strategy allows supervisors to provide ampleamounts of positive feedback to trainees. If supervisors areunaware of trainees’ current skill set, supervisors might pro-vide a type and/or level of supervision that is inappropriate,and run the risk of confusing or overwhelming trainees, whichin turn may potentially harm a client (see Guideline 5.03;Behavior Analyst Certification Board, 2014).

Within this individualized framework, it is expected thatsupervisors use behavioral skills training (BST) to developnovel skills among trainees (Behavior Analyst CertificationBoard, 2012b). BST is ubiquitous in the behavior analytic

literature (Rosales, Stone, & Rehfeldt, 2009) and shown tobe effective for skill acquisition in typically developing chil-dren (Himle, Miltenberger, Flessner, & Gatheridge, 2004;Jones, Kazdin, & Haney, 1981; Yeaton & Bailey, 1978), chil-dren with autism (Gunby & Rapp, 2014), teachers and para-professionals (Lavie & Sturmey, 2002; Sarokoff & Sturmey,2004), and parents (Himle & Wright, 2014; Seiverling,Williams, Sturmey, & Hart, 2012; Shayne & Miltenberger,2013). In the context of supervision, BST involves providinga rationale to the trainee about the skills to be acquired, sys-tematic instruction of skills to the trainee, modeling the previ-ously taught skills to the trainee, rehearsal of the skills by thetrainee (i.e., either in vivo or role-play), and supervisor posi-tive and corrective feedback to the trainee about his or herperformance of the skills (Behavior Analyst CertificationBoard, 2012b; Parsons, Rollyson, & Reid, 2012). Despitethe effectiveness of BST, to our knowledge, there are no be-havioral studies that have evaluated BST to teach skills (i.e.,especially relatively complex skills) to individuals pursuing aBACB certification.

It is also expected that supervisors model technical, profes-sional, and ethical behavior (Behavior Analyst CertificationBoard, 2012b). The behavior of the supervisor is at the foun-dation of competency-based supervision. Ideally, a traineelooks up to the supervisor to learn how to behave in a varietyof situations. As a supervisor, modeling technical behavior isimportant for many skills, such as the language we use todiscuss behavior analysis and for the work we conduct withindividuals. A supervisor should demonstrate mastery of theformal terminology of ABAwhile being competent with col-loquial speech when conversing with a layperson. Supervisorsshould teach their trainees both the formal and informal waysof discussing terminology, while explaining the limitations ofusing jargon with their constituents.

In addition to the technical language, supervisors shouldalso model technical competency when working with individ-uals. Throughout the supervision process, the supervisormight need to Bjump in^ with the trainee and demonstratehow to implement a certain procedure with fidelity. As such,the supervisor must stay current on their clinical practice.Similar to professional athletes, a technically sound supervisorshould be able to model technical skills and Bmake it lookeasy.^ Not only should supervisors model technical skills,they should also model professional and ethical behavior.

Professional and ethical behavior is guided in part by theBACB Professional and Ethical Compliance Code (BehaviorAnalyst Certification Board, 2014). The BACB ethical code,like other ethical codes (e.g., American PsychologicalAssociation), allows for interpretation and professional judg-ment. As a supervisor, it is important not to engage in egre-gious behavior including sexual relationships with a trainee(or a client/family member), accept gifts, fabricate data, orplagiarize. The previously described behaviors are clearer in

292 Behav Analysis Practice (2016) 9:287–298

the ethical code; however, there are other more subtle profes-sional and ethical behaviors that face fewer punishers for thesupervisor, but that have the potential to negatively impact thedevelopment of a trainee. Common examples of those behav-iors include, answering phone calls, emails, and text messagesduring supervision (i.e., not providing undivided attention),being tardy and unprepared for meetings, as well as not mak-ing time available outside of supervision for emergencies,staying relevant with the literature, protecting the confidenti-ality of supervision, and modeling effective professional be-havior, interpersonal skills and collaboration with clients, con-stituents and other service providers.

Developing Case Conceptualization, Problem Solving,and Decision-Making Repertoires

Using effective teaching practices, supervisors should estab-lish fluency in trainees’ ability to behaviorally conceptualize acase, as this is a critical component of ABA (Behavior AnalystCertification Board, 2012b). The ability to effectively concep-tualize a case is influenced by mastery of behavior analytictheory and content, experience applying theory in practice,and the ability/willingness to engage in critical thinking. Fortrainees, all of the previously described components are de-veloping with varying levels across areas. The integration ofthese skills requires practice and the process of mastering caseconceptualization is thought to be enhanced by supervisionthat includes active and critical conversations by supervisors.As mentioned earlier, engaging trainees in critical conversa-tion is a great way for trainees to gain competency in behavioranalysis; likewise, it also allows trainees to carefully consideraspects of an individual’s behavior including, but not limitedto, motivating operations, discriminative stimuli, conse-quences, and hypothesized function. As the supervision pro-cess progresses, the trainee should be more autonomous whenconceptualizing cases and the role of the supervisor should beto confirm the trainee’s conclusion, while providing feedbackas necessary.

Supervisors should also focus on developing problem solv-ing and decision-making abilities among trainees (BehaviorAnalyst Certification Board, 2012b). The practice of ABA isrooted in a problem-solving framework. Behavior analystsdeal with a variety of problems and their ability to effectivelymanage those problems largely dictates their professional suc-cess. Nezu, Nezu, and D’Zurilla (2013) defined problem solv-ing as B…a self-directed process by which individuals attemptto identify, discover, and/or develop adaptive coping solutionsfor problems^ (p. 8). Problem solving involves identificationof a problem, the formation of potential alternatives to solvethe problem, a decision to choose one solution over others,and evaluation of the chosen solution. In supervision, the su-pervisor is responsible for fostering problem-solving skills inthe trainee.

Like any behavior, problem solving can be taught usingBST. The supervisor should discuss and model each of thesteps of problem solving with the trainee (as needed), andparticular focus should be on the decision-making step. Inour experience, this step is often difficult for trainees to makebecause they are concerned that they will make the Bwrong^choice and subsequently Bmess something up.^ Althoughmaking a contraindicated choice is a concern, the supervisor’sduty is to ensure that the trainee’s choice does not cause harm.Ultimately, the supervisor should shape independent problemsolving as competency in problem solving and decision-making is a critical component to the independent practiceof a behavior analyst.

Considerations in the Delivery of Performance Feedback

It is widely acknowledged that performance feedback is key tothe development, maintenance, and enhancement of all skills(e.g., Daniels & Bailey, 2014; Mortenson & Witt, 1998). Theeffectiveness of frequent, genuine, positive, corrective, andspecific feedback based on direct observations of behavior iscommonly accepted. As such, the delivery of timely and ef-fective feedback (i.e., feedback that improves behavior) isrequired of all behavior analytic supervisors (BehaviorAnalyst Certification Board, 2012b; Guideline 5.06;Behavior Analyst Certification Board, 2014). Feedback canbe given in a variety of modalities (e.g., verbal, written, graph-ic, informal) and has been shown to be most effective whencombined with review of data, goal setting, and modificationsto antecedents and consequences (Alvero, Bucklin, & Austin,2001; Balacazar, Hopkins, & Suarez, 1985; DiGennaro,Martens, & Kleinmann, 2007; Sanetti, Luiselli, & Handler,2007). Due to the importance of feedback to the supervisionprocess, there are several variables to be considered, includingbut not limited to the individualization of feedback, ratio ofpositive to corrective feedback, use of rationales, delivery ofcorrective feedback regarding interpersonal or subjective be-havior, documentation of feedback, and a trainee’s currentskill level.

The BACB Supervision Experience Form includes a sam-ple written evaluation of trainee performance across broaddomains such as timeliness, professionalism, and acquisitionof target behavior analytic skills. However, the form may bemodified to better fit the individual needs of the trainee (butsee the BACB Experience Standards [Behavior AnalystCertification Board, 2015] for the required components of thisform). For example, items can be added (e.g., trainee-specificgoals) and removed (e.g., items that are consistently rated assatisfactory) or the rating scale can be adjusted to better depictthe breadth of trainee performance in any one area (e.g., 5-point rating scale). Individualizing the form collaborativelywith the trainee at the outset of, and throughout, the supervi-sion process may increase the relevancy of the feedback for

293Behav Analysis Practice (2016) 9:287–298

the trainee, demonstrate the supervisor’s commitment to su-pervision, and allow the supervisor to focus feedback on thetrainee’s current goals.

Feedback provided with a high positive to negative ratio isrecommended within the performance management literature(e.g., Daniels & Bailey, 2014). Maintaining a supervisor’svalue as a positive reinforcer may be enhanced by such apractice. We also suggest that positive feedback may be en-hanced by a rationale (i.e., a core component of both BST andcorrective feedback, Behavior Analyst Certification Board,2012b) that intimately links trainees’ behavior to their short-and/or long-term goals. For example, BVery impressive per-formance today with Johnny. Even though it was a stressfulsituation, you quickly jumped in to help the other staff imple-ment the reactive procedures of his behavior plan. The greatthing was that all the other staff saw you keep your cool andimplement the intervention as written, which is important asyou’re trying to establish yourself as a role model in the class-room. The more you can get in there and model, just like youdid today, the more others will start to look to you as a leader.^

Equally as necessary as positive feedback is the delivery ofcorrective feedback, which requires empathy, detailed reviewof incorrect and correct performance, modeling, and practice(Behavior Analyst Certification Board, 2012b). When givingcorrective feedback, it is often recommended to remain objec-tive (e.g., do not correct when in a bad mood; Daniels &Bailey, 2014), do so privately, and keep in mind the trainee’scontext prior to giving feedback (i.e., empathy). As a point ofemphasis, supervisors are encouraged to be mindful ofBfavorites^ (i.e., potential side effect of mutual conditionedreinforcers) due to personal characteristics as these biaseshave the potential to overshadow difficulties of or mistakesmade by the trainee.

While provision of corrective feedback for common behav-ior analytic activities such as graphing or implementing apreference assessment is relatively straightforward, deliveringcorrective feedback for personal or more subjective perfor-mance areas that are negatively impacting a trainee’s workcan be significantly more challenging. These areas could bea trainee’s poor social interaction abilities, insensitivity toothers, and inflexibility. Self-report data collected from re-searchers in related fields suggests that supervisors often with-hold corrective feedback regarding negative reactions to sub-jective aspects of a trainee’s performance or give higher rat-ings in fear of harming the supervisory relationship (Hoffman,Hill, Holmes, & Freitas, 2005; Gonsalvez & Freestone, 2007).

As supervisors, we are the gatekeepers of our field; failingto raise important feedback can have negative implications forclients and the advancement of the profession. When supervi-sors are faced with having to provide relatively more subjec-tive or difficult feedback, they are encouraged to seek peerconsultation and discuss potential approaches and practicedelivering corrective feedback prior to meeting with the

trainee. We recommend objectively documenting all relevantvariables to the delivery of corrective feedback includingdates/times, details of events, feedback-specific areas of diffi-culty, and objective responses and reactions from the trainee.

Lastly, tailoring the parameters in which supervisors pro-vide feedback should be considered as trainees progressthrough different skill areas. Novice trainees, or more ad-vanced trainees learning a new skill, often require a higherrate of performance feedback. As trainees begin to developskills in each area, the ultimate goal is to have them accuratelyand reliably evaluate their own ability to demonstrate eachskill. Asking trainees questions such as, BWhat did you docorrectly?^ or BWhat would you do differently next time^may help them reflect and think critically about their perfor-mance. One method for aiding this process is to have traineesview audio-visual recordings of themselves performing activ-ities such as leading a parent or staff training andimplementing conditions of a functional analysis. Followingobservation of videos, the supervisor and trainee are able todiscuss performance objectively. Another method that maypromote this process is to have trainees evaluate their writtenwork (e.g., FBA or BIP) relative to a rubric and compare theirresults to those of the supervisor.

Evaluation of the Process and Outcomes of Supervision

Behavior analytic supervisors should systematically evaluatethe outcomes of their own supervision practices (Guideline5.07, Behavior Analyst Certification Board, 2014, 2012b)using both objective and subjective measures. Direct and re-peated measurement of trainees’ behavior is required to ob-jec t ively evaluate the outcomes of supervis ion.Commensurate with the dimensions of ABA (Baer, Wolf, &Risley, 1968), the supervisor should define the skills to beacquired, collect data before, during and after skills training,and use these data to provide feedback to the trainee. Forexample, using a procedural fidelity form, supervisors canmonitor and graph trainees’ performance implementing skillacquisition programs, behavior reduction plans, and behavior-al skills training. Direct measures of clearly defined, profes-sional behaviors (e.g., timeliness) can also assist in deliveringfeedback and evaluating outcomes. Additionally, behavior an-alysts are encouraged to publish the results in peer-reviewedjournals to demonstrate the effectiveness of behavioral ap-proaches within the context of supervision and the relevantskills.

The review of permanent products also serves to objective-ly evaluate a trainee’s performance. Permanent products arethe tangible results of behavior and are commonly used as adata recording method. They can be collected on academictasks (e.g., completion of mathematics problems), problembehavior (e.g., count of holes in a wall), and social skills(count of phone numbers obtained) to name a few. In the case

294 Behav Analysis Practice (2016) 9:287–298

of supervision, materials developed by the trainee are the mostobvious permanent products. Those materials include, for ex-ample, written behavior intervention plans, data sheets, skill-acquisition plans, assessment reports, graphs, and relevantgoals and objectives in a student’s IEP. Moreover, there areother permanent products that should be reviewed such asnotes/letters to parents and email correspondence withteachers/staff. When utilizing a rubric and clear criteria, allof these documents allow a supervisor to objectively evaluatea trainee’s growing competency regarding the quality of writ-ten materials that are critical for constituents that behavioranalysts serve.

Complimentary to objective measures, social validity mea-sures (Wolf, 1978) can be used to evaluate socially significantchanges in the behavior of a trainee’s clients and satisfactionwith the performance of the trainee by parents and other pro-fessionals. Social validity measures can also be used to assessthe trainee’s satisfaction with the goals, procedures, and out-comes of the supervision process. Besides the weekly (or bi-weekly) trainee rating forms we described earlier, supervisorsare encouraged to obtain this type of feedback from theirtrainees. Although not mandated by the BACB, and whilesuch measures may not be appropriate means to evaluate theeffectiveness of the supervision process with regard to skilldevelopment and client and staff performance outcomes, theycan establish more acceptable and mutually agreed-upon su-pervision practices and serve as a tool for monitoring thesupervisor-trainee relationship. The objective is having thetrainee identify the methods that are most acceptable, andallow supervisors to modify their approach during thetrainee’s experience (if needed) and enhance the supervisionprocess overall. The supervisor may disagree with the feed-back provided by the trainee; however, this activity opens adialogue about what is and is not working and ideally, ensuresthat the supervisor and trainee speak openly to each other.Then, a plan can be created to alter the behavior of the traineeand/or the supervisor. To reiterate, the foundation for this typeof feedback is grounded in the performance feedback litera-ture and has clear applications for BCBA supervision.

To assist with this type of evaluation, we developed aSupervision Monitoring and Evaluation Form as a projectduring training at a doctoral internship program in clinicalpsychology. The internship site had a strong behavioral orien-tation and employed/trained behavior analysts. Duringmonth-ly meetings with the Director of Training, the internship classdiscussed strengths and weaknesses of supervisors,commented about their previous experiences in supervision,and read literature on competency-based supervision frommultiple fields. After aggregating all of the information, thegroup developed the Supervision Monitoring and EvaluationForm (see appendix).

After developing the form, the clinical interns piloted im-plementation with their supervisors several times throughout

the year. This implementation not only gave the trainees (i.e.,behavioral psychology interns) the opportunity to Btest-drive^the measure, but also allowed them to provide feedback totheir supervisors. Supervisors, in fact, were uniformly positiveabout the process of evaluation and readily embraced the feed-back from their trainees.We suspect that this positive outcomewas due, in part, to having supervisors participate in develop-ing the Supervision Monitoring and Evaluation Form at theearliest stages and being receptive to having interns objective-ly evaluate supervision practices. Again, it is the reciprocitybetween trainees and supervisors that is hypothesized to en-gender success and mutual satisfaction.

Continuing Education in Supervision

The above sections highlight the many responsibilities andconsiderations that are present to behavior analytic supervi-sors and the degree to which supervision of those pursuing aBACB certification can be complex, time consuming, andoften requires skills beyond the research and practice skillstaught in the formal education of behavior analysts. In recog-nition of this fact and in addition to maintaining competencyin behavior analysis, the BACB now requires behavior ana-lysts to receive ongoing education in supervision (BehaviorAnalyst Certification Board, 2015), such as effective supervi-sion practices, ethical considerations regarding supervision,and the standards of BACB supervision.

While supervisors can obtain formal continuing educationpertaining to supervision through conference presentations,workshops, and online seminars, supervisors should also seekout their own supervision (Behavior Analyst CertificationBoard, 2012b). The current model highlights the importanceof reciprocal feedback and collaboration between the traineeand supervisor, but it is also important to develop this type ofrelationship among supervisors to encourage mentorship andconsultation in an effort to maintain and enhance supervisorcompetence. In addition to consultation, to the extent possible,it is important to have other competent supervisors observeyour supervision activities and provide feedback based on theobjective criteria we outlined previously.

Social validity measures may further be used as a self-monitoring tool by providing a means for supervisors to con-tinually evaluate their performance as a supervisor, whichprovides an excellent model of continual self-assessment fortrainees. Placing focus on self-assessment will enable super-visors to be aware of their strengths and weaknesses in super-vision practices as well as clinical and research activities.Using this self-assessment approach, competency-based su-pervision brings accountability to the practice of behavioranalysis and sets the stage for quality supervision. A first nextstep is to conduct an evaluation of the impact of self-monitoring and evaluation on supervisor behavior and the

295Behav Analysis Practice (2016) 9:287–298

supervisory relationship, and in turn the impact on trainee skilldevelopment.

Concluding Remarks: Initiating a Research Agendain Behavior Analytic Supervision

Competency-based models allow for systematic measurementof both supervisor and trainee behavior throughout the entiresupervision process, which makes these approaches appropri-ate for applied research. As there is a dearth of research in thisarea, the specific direction of and topics that future researchmay take are vast; however, there we present a few consider-ations for research in this area. First, it will be essential toobjectively define and task analyze core competencies oftrainees and supervisors, especially those falling within therealm of professional and ethical behaviors (e.g., specificskills denoting professionalism, display of empathic behav-ior), so that we can make objective conclusions about an in-dividual’s competency. It will also be necessary to identify thevalidity and value of various outcome measures such as socialvalidity, BACB exam pass rate, trainee observable behavior incontrived and naturalistic settings, and client behavior change.

Within this framework, future research should focus on theidentification of effective methods for training generalizableand maintainable skills to trainees. While the training of basicprocedural skills (e.g., implementing functional analysis con-ditions) may be a first step, methods for training the complexskills required of behavior analysts are needed (e.g.,interviewing skills, problem solving, responding to ethicaldilemmas). For example, what are the considerations forimplementing BST for these skills? How many exemplarsare needed to obtain competency and generalization? Whatare effective methods of prompting (e.g., video modeling)and teaching for generalization?

Further, there is a tentative hypothesis that the quality of thesupervisory relationship as defined by the presence of rapportand mutually agreed upon expectations will impact the super-vision experience, notwithstanding the need for more thor-ough empirical validation. For example, does the supervisoryrelationship actually lead to quicker acquisition of skills and/or a higher number of skill competencies obtained by thetrainee? In order to answer these and many other questions,we must first objectively define the supervisory relationship.While we presented an initial definition of the relationshipwithin this paper, objective indicators of such a relationshipmust be described, measured, and incorporated into supervi-sion practices. Thus, attempts to measure Bgood^ rapport havefocused on the presence of smiling and laughing and the ab-sence of problem behavior (Magito et al., 2005) but othermeasures appear relevant. Trust, for example, can be observedas someone consistently following through on verbal state-ments. BLikeability,^ another researched construct, translatesto a person who has a positive attitude, appears happy, speaks

in a pleasant tone, and helps others achieve their goals(Sanders, 2006). Given the focus of the supervisory relation-ship in all other models of supervision across disciplines, itwould be advantageous for behavior analysts to systematicallyevaluate the impact of the relationship on trainee outcomesand disseminate to the larger supervision community. Takentogether, comprehensive considerations of the supervisoryguidelines, ethical code and establishing a research agendain behavior analytic supervision will ideally lead to an effec-tive, ethical, and professional group of rapidly growing prac-ticing behavior analysts.

Compliance with Ethical Standards This article does not contain anystudies with human participants or animals performed by any of the au-thors.

Conflict of Interest The authors declare that they have no conflict ofinterest.

Funding Funding was not obtained for the creation of this manuscript.

References

Abbass, A., Arthey, S., Elliott, J., Fedak, T., Nowoweiski, D., Markovski,J., et al. (2011). Web-conference supervision for advanced psycho-therapy training: a practical guide. Psychotherapy, 48(2), 109–118.

Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective reviewof the effectiveness and essential characteristics of performancefeedback in organizational settings. Journal of OrganizationalBehavior Management, 21(1), 3–29. doi:10.1300/J075v21n01_02.

American Psychological Association. (2015). Guidelines for clinical su-pervision in health service psychology. American Psychologist,70(1), 33–46. doi:10.1037/a0038112.

Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimen-sions of applied behavior analysis. Journal of Applied BehaviorAnalysis, 1(1), 91–97. doi:10.1901/jaba.1968.1-91.

Balacazar, F. E., Hopkins, B. L., & Suarez, Y. (1985). A critical objectiveview of performance feedback. Journal of Organizational BehaviorManagement, 7(3–4), 65–89. doi:10.1300/J075v07n03_05.

Behavior Analyst Certification Board. (2011). Online newsletter,September issue. Littleton, CO: Behavior Analyst CertificationBoard.

Behavior Analyst Certification Board. (2012a). Coursework require-ments for BACB credentials: fourth edition task list. Littleton, CO:Behavior Analyst Certification Board.

Behavior Analyst Certification Board. (2012b). Supervisor training cur-riculum outline. Littleton, CO: Behavior Analyst CertificationBoard.

Behavior Analyst Certification Board. (2014). Professional and ethicalcompliance code for behavior analysts. Littleton, CO: BehaviorAnalyst Certification Board.

Behavior Analyst Certification Board. (2015). Experience standards.Littleton, CO: Behavior Analyst Certification Board.

Bernard, J. M., & Goodyear, R. K. (2014). Fundamentals of clinicalsupervision (5th ed.). Boston, MA: Pearson.

Bordin, E. S. (1983). Aworking alliance based model of supervision. TheCounseling Psychologist , 11(1) , 35–42. doi:10.1177/0011000083111007.

296 Behav Analysis Practice (2016) 9:287–298

Carr, E. G., Levin, L., McConnachie, G., Carlson, J. I., Kemp, D. C., &Smith, C. E. (1997). Communication-based intervention for prob-lem behavior: a user’s guide for producing positive change.Baltimore, Maryland: Paul H. Brookes Publishing Co., Inc.

Cavalari, R. N. S., Gillis, J. M., Kruser, N., & Romanczyk, R. G. (2015).Digital communication and records in service provision and super-vision: regulation and practice. Behavior Analysis in Practice, 8(2),176–189. doi:10.1007/s40617-014-0030-3.

Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavioranalysis. Upper Saddle River: Pearson Education.

Daniels, A. C., & Bailey, J. S. (2014). Performance management: chang-ing behavior that drives organizational effectiveness (5th ed.).Atlanta: Aubrey Daniels International, Inc.

DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A com-parison of performance feedback procedures on teachers’ treatmentimplementation integrity and students’ inappropriate behavior inspecial education classrooms. Journal of Applied BehaviorAnalysis, 40(3), 447–461. doi:10.1901/jaba.2007.40-447.

Dunn, K. E., & Mulvenon, S. W. (2009). A critical review of research onformative assessment: the limited scientific evidence of the impactof formative assessment in education. Practical Assessment,Research & Evaluation, 14(7), 1–11.

Eikeseth, S., Hayward, D., Gale, C., Gitlesen, J., & Eldevek, S. (2009).Intensity of supervision and outcome for preschool aged childrenreceiving early and intensive behavioral interventions: a preliminarystudy. Research in Autism Spectrum Disorders, 3(1), 67–73. doi:10.1016/j.rasd.2008.04.003.

Falender, C. A., Erickson Cornish, J. A., Goodyear, R., Hatcher, R.,Kaslow, N. J., Leventhal, G., et al. (2004). Defining competenciesin psychology supervision: a consensus statement. Journal ofClinical Psychology, 60(7), 771–785. doi:10.1002/jclp.20013.

Falender, C. A., & Shafranske, E. P. (2012). The importance ofcompetency-based clinical supervision and training in the twenty-first century: why bother? Journal of Contemporary Psychotherapy,42(3), 129–137. doi:10.1007/s10879-011-9198-9.

Florell, D. (2016).Web-based training and supervision. In J. K. Luiselli &A. J. Fischer (Eds.), Computer-assisted and web-based innovationsin psychology, special education, and health. New York: AcademicPress/Elsevier.

Frank, F. R., Snell, L. S., tenCate, O., Holmboe, E. S., Carraccio, C.,Swing, S. R., et al. (2010). Competency-based medical education:theory to practice. Medical Teacher, 32(8), 638–645. doi:10.3109/0142159X.2010.501190.

Gibson, J. A., Grey, I. M., & Hastings, R. P. (2009). Supervisor support asa predictor of burnout and therapeutic self-efficacy in therapistsworking in ABA schools. Journal of Autism and DevelopmentalDisorders, 39, 1024–1030. doi:10.1007/s10803-009-0709-4.

Gonsalvez, C., & Freestone, J. (2007). Field supervisors’ assessments oftrainee performance: are they reliable and valid? AustralianPsychologist, 42(1), 23–32. doi:10.1111/j.17429544.2011.00033.x.

Grady, B., Myers, K. M., Nelson, E., Belz, N., Bennett, L., Carnahan, L.,et al. (2011). Evidence-based practice for telemental health.Telemedicine and e-Health, 17(2), 131–148.

Gresham, F.M., Gansle, K. A., &Noell, G. H. (1993). Treatment integrityin applied behavior analysis with children. Journal of AppliedBehavior Analysis, 26(2), 257–263. doi:10.1901/jaba.1993.26-257.

Gunby, K. V., & Rapp, J. T. (2014). The use of behavioral skills trainingand in situ feedback to protect children with autism from abductionlures. Journal of Applied Behavior Analysis, 47(4), 856–860. doi:10.1002/jaba.173.

Himle, M. B., Miltenberger, R. G., Flessner, C., & Gatheridge, B.(2004). Teaching safety skills to children to prevent gun play.Journal of Applied Behavior Analysis, 37(1), 1–10. doi:10.1901/jaba.2004.37-1.

Himle, M. B., & Wright, K. A. (2014). Behavioral skills training toimprove installation and use of child passenger safety restraints.

Journal of Applied Behavior Analysis, 47(3), 549–559. doi:10.1002/jaba.143.

Hoffman, M. A., Hill, C. E., Holmes, S. E., & Freitas, G. F. (2005).Supervisor perspective on the process and outcome of giving easy,difficult, or no feedback to trainees. Journal of CounselingPsychology, 52(1), 3–13. doi:10.1037/0022-0167.52.1.3.

Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G.S. (1994). Toward a functional analysis of self-injury. Journal ofApplied Behavior Analysis, 27(2), 197–209. doi:10.1901/jaba.1994.27-197.

Jones, R. T., Kazdin, A. E., & Haney, J. L. (1981). Social validation andtraining of emergency fire safety skills for potential injury preven-tion and life saving. Journal of Applied Behavior Analysis, 14(3),249–260. doi:10.1901/jaba.1981.14-249.

Kraemer-Tebes, J., Matlin, S. L., Migdole, S. J., Farkas, M. S., Money, R.W., Shulman, L., et al. (2011). Providing competency training toclinical supervisors through an interactional supervision approach.Research on Social Work Practice, 21(2), 190–199. doi:10.1177/1049731510385827.

Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature,extent, and importance of what psychotherapy trainees do not dis-close to their supervisors. Journal of Counseling Psychology, 43(1),10–24. doi:10.1037/0022-0167.43.1.10.

Lavie, T., & Sturmey, P. (2002). Training staff to conduct a paired-stimulus preference assessment. Journal of Applied BehaviorAnalysis, 35(2), 209–211. doi:10.1901/jaba.2002.35-209.

Magito McLaughlin, D., & Carr, E.G. (2005). Quality of rapport as aset t ing event for problem behavior : assessment andintervention. Journal of Positive Behavior Interventions, 7(2), 68-91.

Mehr, K. E., Ladany, N., & Caskie, G. I. L. (2010). Trainee nondisclosurein supervision: what are they not telling you? Counselling andPsychotherapy Research, 10(2), 103–113. doi:10.1080/1473314100371230d.

Mehr, K. E., Ladany, N., & Caskie, G. I. L. (2015). Factors influencingtrainee willingness to disclose in supervision. Training andEducation in Professional Psychology, 9(1), 44–51. doi:10.1037/tep0000028.

Mortenson, B. P., & Witt, J. C. (1998). The use of weekly perfor-mance feedback to increase teacher implementation of a pre-referral academic intervention. School Psychology Review,27(4), 613–627.

Nezu, A. M., Nezu, C. M., & D’Zurilla, T. J. (2013). Problem-solving therapy: a treatment manual. New York: SpringerPublishing.

Panos, P. T., Panos, A., Cox, S. E., Roby, J. L., & Matheson, K. W.(2002). Ethical issues concerning the use of videoconferencing tosupervise international social work field practicum students.International Social Work, 38(3), 421–437.

Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-basedstaff training: a guide for practitioners. Behavior Analysis inPractice, 5(2), 2–11.

Rosales, R., Stone, K., & Rehfeldt, R. A. (2009). The effects of thebehavioral skills training on implementation of the picture exchangecommunication system. Journal of Applied Behavior Analysis,42(3), 541–549. doi:10.1901/jaba.2009.42-541.

Sanders, T. (2006). The likeability factor. New York: Crown Business.Sanetti, L. M. H., Luiselli, J. K., & Handler, M. W. (2007). Effects

of verbal and graphic performance feedback on behavior sup-port plan implementation in a public elementary school.Behavior Modification, 31(4), 454–465. doi:10.1177/0145445506297583.

Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skillstraining on staff implementation of discrete-trial teaching. Journal ofApplied Behavior Analysis, 37(4), 535–538. doi:10.1901/jaba.2004.37-535.

297Behav Analysis Practice (2016) 9:287–298

Seiverling, L., Williams, K., Sturmey, P., & Hart, S. (2012). Effects ofbehavioral skills training on parental treatment of children’s foodselectivity. Journal of Applied Behavior Analysis, 45(1), 197–203.doi:10.1901/jaba.2012.45-197.

Shayne, R., &Miltenberger, R. G. (2013). Evaluation of behavioral skillstraining for teaching functional assessment and treatment selectionskills to parents. Behavioral Interventions, 28(1), 4–21. doi:10.1002/bin.1350.

Wolf, M. M. (1978). Social validity: the case for subjective measurementor how applied behavior analysis is finding its heart. Journal ofApplied Behavior Analysis, 11(2), 203–214. doi:10.1901/jaba.1978.11-203.

Yeaton, W. H., & Bailey, J. S. (1978). Teaching pedestrian safety skills toyoung children: an analysis and one-year follow up. Journal ofApplied Behavior Analysis, 11(3), 315–329. doi:10.1901/jaba.1978.11-315.

298 Behav Analysis Practice (2016) 9:287–298