Running head: AUDIO FEEDBACK

© 2019, Sheryl A Cifrino

Simmons University

The College of Natural, Behavioral, and Health

Sciences

EXPLORING THE EFFECTS OF AUDIO FEEDBACK ON THE

COMMUNICATION KNOWLEDGE AND MOTIVATIONAL INTERVIEWING

SKILLS OF UNDERGRADUATE NURSING STUDENTS ADDRESSING

BEHAVIORAL HEALTH ISSUES WITH PATIENTS

by

SHERYL A. CIFRINO

Submitted in partial fulfillment of the

requirements for the degree of

Doctorate of

Philosophy

02.27.2019

Committee Members:

Donna Glynn PhD., RN, ANP-BC

Susan Duty Sc.D., RN, ANP-BC

Coleen Toronto PhD., RN, CNE

AUDIO FEEDBACK ii

AUDIO FEEDBACK iii

Abstract

Background

The enhancement of healthcare providers’ communication with emphasis on patient

centered care and the promotion of patient autonomy is an essential element for

promoting positive patient centered outcomes. The communication technique of

motivational interviewing (MI) may be useful to engage behavioral health patients in a

collaborative patient centered rapport. Research supports the value of audio feedback to

student learning, and development of academic skills. There is a dearth of literature that

explores the use of audio feedback as an effective method for development of

motivational interviewing communication knowledge and skills with behavioral health

patients in baccalaureate nursing students.

Purpose

To explore the possible differences in the effect of audio feedback compared to the

written feedback on the development of motivational interviewing knowledge and skills

using a standardized patient case among undergraduate baccalaureate nursing students

addressing behavioral health issues.

Methods

Forty undergraduate baccalaureate nursing students were recruited to participate from one

college and randomly assigned to a control group receiving written feedback using the

BECCI index tool and an experimental group receiving audio feedback. Both groups

engaged in an online education program module describing use and techniques of MI.

Each group completed a pre-post test on motivational interviewing knowledge prior to

and after the assigned feedback intervention. Both groups engaged in practice of

AUDIO FEEDBACK iv

motivational interviewing skills with a standardized audio taped scripted patient prior to

and post feedback. Participants received audio feedback or written feedback after the first

practice interview. The student MI interviews were rated by research assistants blinded to

intervention group using the BECCI index tool for quantifying motivational interviewing

skills.

Results

Each group demonstrated increased knowledge Motivational Interviewing Knowledge

and Attitudes Test (MIKAT) scores and Behavioral Change Counseling Index (BECCI)

skills scores across the study. No statistically significant differences were observed in

mean MIKAT or BECCI scores between treatment groups. A significant improvement in

participant knowledge (MIKAT score) and skills (BECCI scores) increased within groups

post intervention (p< 0.05).

Conclusion

Audio feedback and written feedback are comparable ways to deliver formative

assessment to help develop motivational interviewing knowledge and skills in

undergraduate baccalaureate nursing students. These venues for feedback can offer ways

to provide learners with insight on their performance during standardized patient

encounters allowing for self-directed integration into their knowledge about motivational

interviewing techniques.

AUDIO FEEDBACK v

Acknowledgements

There are a number of talented people I wish to acknowledge for their guidance and

support throughout this experience. First and foremost, my committee chair Dr. Donna

Glynn whose support and guidance has been invaluable. Dr. Susan Duty for sharing her

expertise in research and guidance through this process. Dr. Coleen Toronto for sharing

her perspectives and knowledge with me without reservation or judgement. My

classmates Tracy, Judi, and Donna whose consistent support has been treasured during

my dissertation process.

AUDIO FEEDBACK vi

Dedication

This dissertation is dedicated to my husband David and Teresa Damien whose support

throughout this process has been unfaltering and continuous.

AUDIO FEEDBACK vii

Table of Contents

Chapter 1: Introduction……………………………………………………………. 1

Research Question…………………………………………………………………. 6

Definition of Terms………………………………………………………………... 6

Conceptual Terms………………………………………………………... 6

Operational Terms……………………………………………………….. 7

Significance………………………………………………………………….…….. 7

Theoretical Framework……………………………………………………………. 9

Chapter 2: Literature Review……...……………………………………………..... 12

Communication in Healthcare…………………………………….……... 12

Barriers to Effective Communication………………………….………… 13

Recommendation for Decreasing Communication Barriers……………... 14

Motivational Interviewing……………………………………………….. 15

Education and Feedback……………………………………….………… 21

Audio Feedback………………………………………………………….. 22

Audio Versus Written Feedback…………………………...…………….. 24

Summary of the Literature……………………………………………….. 26

Chapter 3: Methods………………………………………………………………... 28

Design……………………………………………………………...…….. 28

Setting…………………………………………………………...……….. 29

Sample…………………………………………………………………… 30

Recruitment……………………………………………………………… 30

Data Collection Tools.………………………………………………….... 31

AUDIO FEEDBACK viii

Knowledge and attitudes test..………..………………………. 31

Skills index tool ……………………………………………… 33

Procedures………………………………………………………………... 34

Randomization…………………………………………….... 35

Intervention………………………………………………………………. 35

Baseline surveys……………………………………………… 36

Internet based, self-paced, educational training in

motivational interviewing……………………………………..

36

Interview sessions...…………………………………………... 37

Experimental groups...……………………………………….. 38

Treatment fidelity…………………………………………….. 39

Post intervention surveys……………………………………... 39

Data Analysis…………………………………………………………….. 40

Human Participants Protection…………………………………………... 40

Chapter 4: Results………………..……………………………………………....... 42

Participants…………………………..…………………………………... 42

Intervention Fidelity………………………………...…………………… 43

Knowledge……………………………………………………………….. 43

Between group analyses……………………………………... 44

Within group analyses…………………………………….…. 44

Skills Development………………………………………………………. 44

Between group analyses……………………………………... 45

Within group analyses…………………………………….…. 45

AUDIO FEEDBACK ix

Practitioner speaking time proportions……………………… 45

Summary…………………………………………………………………. 45

Chapter 5: Discussion……………………...………………………………………. 47

Instrument Reliability……………………………………………………. 50

Strengths and Limitations….…………………………………………….. 52

Future Educational Research and Recommendations …..……………….. 52

Conclusion ………………………………………………………………. 53

References………………………………………………………….……………… 57

Appendix A: Motivational Interviewing Knowledge and Attitudes Test MIKAT1.. 72

Appendix B: Behavioral Change Counseling Index (BECCI)…………………….. 75

Appendix C: Permission to use the Behavioral Change Counseling Index

(BECCI)……………………………………………………………………………

76

Appendix D: Demographic Form……………………………….…………………. 77

Appendix E: Informed Consent Form.…………………………………….………. 78

Appendix F: IRB Approval Letters……………………………………………....... 81

Appendix G: Results Tables...……………………………………………………... 83

AUDIO FEEDBACK x

List of Figures

Figure 1. CONSORT Flow diagram of phrases of randomized trial research

groups……………………………………………………………………………….

29

Figure 2. Flowchart of research procedures……………………………………….. 36

AUDIO FEEDBACK 1

Exploring the Effects of Audio Feedback on the Communication Knowledge and

Motivational Interviewing Skills of Undergraduate Nursing Students Addressing Behavioral

Health Issues with Patients

Chapter 1

Introduction

Mental health and substance use conditions contribute to a growing burden of disease on

a global scale. In the United States, analysis of available data from 2013 reports the mounting

misuse of opioids and subsequent health costs have increased by 20 billion dollars per year since

2007 (National Institute on Drug Abuse, 2017). The Substance Abuse and Mental Health

Services Administration (SAMHSA) report patient emergency room visits for prescription drug

abuse and illicit drug abuse numbered over 350 in 100,000 encounters (SAMHSA, 2013). The

use of opioid prescription medications has steadily increased in the past twenty years. There was

a 2.8-fold increase in the total number of deaths from opioid drugs in the United States from

2002 to 2015 (NIDA, 2017). In the United States, reported opioid related overdose deaths

continue to rise (Seth, Scholl, Rudd & Bacon, 2018). As a result of this growing behavioral

health crisis hundreds of local, state, and federal initiatives have been executed (SAMHSA,

2017).

Behavioral health represents mental and emotional well-being and/or activities that affect

individual wellness. Behavioral health conditions span problems resulting from unhealthy stress

or subclinical states to the diagnosed and treatable conditions of serious mental illness, and

substance use disorders (SAMHSA, 2017). Baccalaureate prepared nurse graduates must be

competent to deliver holistic communication with these patients within every healthcare setting.

AUDIO FEEDBACK 2

The American Association of Colleges of Nursing (AACN) put forth The Essentials of

Baccalaureate Education for Professional Nursing Practice (2008) that articulate that a graduate

should be prepared to utilize interprofessional and intraprofessional communication skills that

enable the delivery of collaborative evidence-based patient centered care. The baccalaureate

prepared nurse role requires patient centered approaches to care based on communication that

respects patient values, preferences, and educates patients and caregivers in matters of health,

wellness, management and prevention of disease conditions (AACN, 2008). There is a noted

connection between behavioral health issues and modifiable lifestyle risk factors such as

smoking, alcohol use and drug use that impact chronic health conditions such as cardiovascular

disease, obesity, and asthma (Centers for Disease Control and Prevention, 2012; National

Institute of Mental Health, 2018).

Research notes that training healthcare providers in the communication technique of

motivational interviewing may be useful to engage behavioral health patients in a collaborative

patient centered rapport (Drevenhon, Bengston, Nyberg, Kjellgren, 2015; Lundahl, Kunz,

Brownell, Tollefoson & Burke, 2010; Magill, Colby, Orchowski, Murphy, Hoadley, Brazil &

Barnett, 2017). Motivational interviewing (MI) is a conversational approach to counseling that is

collaborative in nature and utilized to support person-centered motivation and commitment to

assist a person to resolve ambivalence and pose a pathway to alter a behavioral condition. This

well-developed patient centered communication method can encourage positive change in

patients with behavioral health conditions. The goal of enhancing communication with emphasis

on patient centered care and patient autonomy is an essential element for encouraging optimal

patient centered outcomes (Carr, 2017; Miller, 1983; Miller & Rollnick, 1991; Miller &

Rollnick, 2013). Research stipulates that the use of motivational interviewing techniques by

AUDIO FEEDBACK 3

healthcare providers affords patients with chronic health conditions the opportunity to explore

values, beliefs, goals, and clarifications for improving health status (Lee, Choi, Yum, Doris &

Chair, 2015; Lundahl et al., 2010; Mehta, Cameron & Battistella, 2014; White Gazewood &

Mounsey, 2007). The literature highlights educational intervention for development of

motivational interviewing knowledge and skills in nurse practitioner students, medical/surgical

nurses, psychologists, community workers, nurses, primary care nurses, Human

Immunodeficiency Virus/Auto Immune Deficiency Syndrome (HIV/AIDS) counselors, veteran

peer specialists, dental hygiene students, certified therapeutic recreational therapists, and

medical students (Edwards, Stapleton, Williams & Ball, 2015; Evangeli, Engelbrecht, Swartz,

Turner, Forsberg & Soka, 2009; Mills et al., 2017; Nesbitt, Murray & Mensink, 2013; Piatt &

Chiasson, 2016; Ostlund, Wadensten, Haggstrom, & Kristofferzon, 2013; Stoffers & Hatler,

2017; Tsai et al., 2017). Identified is a gap in the literature that addresses educating

undergraduate nursing students about motivational interviewing skills for addressing behavioral

health issues with patients.

The literature outlines education strategies used to train healthcare workers in

motivational interviewing techniques and exploring the use of motivational interviewing to

impact specific health behaviors in a variety of populations. Strategies include virtual and face to

face training workshops which measure specific clinician response to training in attainment of

MI knowledge and skills (Amodeo, Lundren, Beltrame, Chassler, Cohen & D’Ippolito, 2013;

Cucciare, Ketroser, Wilbourne, Midboe, Cronkite, Berg-Smith & Chardos, 2012; Kennedy

Apodaca, Trowbridge, Hafeman, Roderick & Modrcin, 2016; Noordman, van der Weijden, van

Dulmen, 2014; Ostlund, Wadensten, Kristofferzon & Haggstrom, 2013). In specific populations,

exploration of the effect of motivational interviewing interventions on targeted health behaviors

AUDIO FEEDBACK 4

reveals measurable outcomes in terms of reportable reduction in a health behavior or compliance

with treatment regimens (Bertrand, Roy, Vaillancout, Vandermeerschen, Berbiche & Boivin,

2015; Chang, Comptom, Almeter & Fox, 2015; Magill, Colby, Murphy, Hoadley, Brazil &

Barnett, 2017 & Russell, et al., 2011; Stoffers & Hatler, 2017).

The training with healthcare providers engages them as learners in a cycle of introduction

to MI knowledge and skills using educational interventions coupled with formative written

feedback mechanisms to measure varying levels of comprehension, skill attainment, and or

experiences in utilization of techniques in practice sessions (Amodeo et al., 2013; Cucciare, et

al., 2012; Kennedy et al., 2016; Ostlund et al., 2015 & Schumacher, Williams, Burke, Epler,

Simon & Coffey, 2018).

The process of learning involves the learner receiving feedback to provide information

for the learner to adjust their performance of skill or increase their knowledge and

comprehension (Forrest, 2005; Gould & Day, 2013). In higher education, formative assessment

and feedback should allow the learner to participate in the act of self-regulation of goal

achievement (Nicol &Macfarlane-Dick; Gould & Day, 2013). The use of active learning

strategies to enhance learner processing in development of knowledge and skills is linked to the

exploring influence of using audio feedback as a formative assessment. In a hallmark study of

student attitudes regarding the use of audio feedback Merry and Osmond (2008) establish that

this modality of feedback is perceived by the students to have value for comprehending the

assessments and integrating them in a meaningful way into their learning process.

A recent qualitative review of the literature by Killingback, Ahmed and Williams (2019)

highlights the views of students in tertiary education (post-secondary education) regarding the

use of audio, video, podcast and screen cast venues of feedback. The analysis revealed five

AUDIO FEEDBACK 5

themes related to these alternative feedback types; belonging, increased comprehension from

nonverbal characteristics of communication, person/individualized, technical and practical parts,

and circumstances and situational. The overall student perceptions of use of alternative forms of

feedback was positive (Killingback, Ahmed & Williams, 2019). The use of recorded audio

feedback has been utilized with research done to investigate the effectiveness of this formative

assessment with learners in the fields of education, early childhood studies, diagnostic imaging

students, online learners in an asynchronous teaching strategies course, medical students,

community pre licensure nursing students in the United Kingdom, a cohort of master’s level

nursing students in the clinical setting, and junior- level nursing students enrolled in a medical

surgical nursing course or a research course (Bourgault, Mundy & Joshua, 2013; Carruthers,

McCarron, Bolan, Devine, McMahon-Beattie & Burns, 2015; Cavanaugh & Song, 2014; Gould

& Day, 2013; Green, 2015; Harrison, Molyneux, Blackwell & Wass 2014; Ice, Curtis, Phillips &

Wells, 2007; Race & Williams, 2018). One example of this is highlighted by Race and Williams

(2018) who conducted a descriptive correlational study to explore student perceptions of and

levels of satisfaction with digital auditory feedback in clinical nursing and research theory

courses. Students reported perceptions of audio feedback as useful in giving detailed feedback

for performance improvement in terms of increasing confidence in clinical coursework and

lowering levels of frustration in finishing medical surgical course work. There is a paucity in the

literature delineating the use of audio feedback in educating baccalaureate level nursing students.

The purpose of this quantitative study was to:

explore possible differences in the effect of audio feedback compared to the written

feedback on the development of motivational interviewing knowledge and skills using a

AUDIO FEEDBACK 6

standardized patient case among undergraduate baccalaureate nursing students addressing

behavioral health issues.

Research Question

The following question guided the study: What is the effect of audio feedback compared

to the written feedback on the development of motivational interviewing knowledge and skills

using a standardized patient case among undergraduate baccalaureate nursing students

addressing behavioral health issues?

Definition of Terms

Conceptual Terms

For the purpose of this study the following terms are defined.

1. Audio Feedback: a type of feedback delivery of information that confers assessment

of work in education that is delivered by either taped or digital venue. (Merry &

Orsmond, 2008; Nicol & Macfarlane-Dick, 2006; Price et al., 2010)

2. Behavioral Health: A term that represents mental and emotional well-being and/or

activities that affect individual wellness. Behavioral health conditions span problems

resulting from unhealthy stress or subclinical states to the diagnosed and treatable

conditions of serious mental illness, and substance use disorders (SAMHSA, 2017).

3. Motivational Interviewing Knowledge & Skills: A conversational approach to

counseling that is collaborative in nature and is utilized to support a person-centered

motivation and commitment to alter a behavioral condition (Miller, 1983; Miller &

Rollnick, 1991; Miller& Rollnick 2013).

4. Formative Assessment: An assessment that provides feedback to the learner to use

for reference in addressing areas of improvement (Iwasiw & Goldenberg. 2015).

AUDIO FEEDBACK 7

5. Healthcare Provider: Person or business that provides health care services to

consumers (Business Dictionary, 2019).

Operational Terms

For the purpose of this study the following terms are defined.

1. MI Communication Techniques: skills measured by the Behavior Change

Counseling Index (BECCI) a validated tool by to assist trainers and researchers to

evaluate change in behavior prior to, or post training for assessment of application of

motivational interviewing skills post training initiatives training (Lane, 2002).

2. Motivational Interviewing Knowledge and Attitudes Test (MIKAT): a validated

tool test for evaluating motivational interviewing knowledge consistent with

principles of MI.

3. MP3 Audio files: an audio recording of interview sessions used to evaluate

motivational interviewing skills.

4. Standardized Patient Case: a scripted standardized patient behavioral health

presentation based on scripted audio file of an interview developed for training

purposes in consultation with a content expert.

Significance

In current healthcare systems, there is a need to train baccalaureate prepared nursing

students in communication techniques that prepare them with the knowledge and skills necessary

for dealing with patients who present with behavioral health issues in healthcare settings

(SAMHSA, 2017). The research supports the use of motivational interviewing with patients and

recognizes it may positively impact the clinician/patient relationship and the behavioral health

outcomes of patients (Amodeo et al., 2013; Chang et al., 2015; Drevenhon et al., 2015 & Magill

AUDIO FEEDBACK 8

et al., 2017; Piatt & Chiasson, 2017; Shannon, Donovan-Hall & Bruton, 2017; Stoffers & Hatler,

2017).

Educational interventions for development of motivational interviewing knowledge and

skills in research support focus on student centered approaches that allow a progressive process

of learning and enhancement (Amodeo et al., 2013; Kennedy et al., 2016; Noordman et al., 2014;

Ostlund et al., 2013). The scaffolding of knowledge is done by utilizing presented information

and creating the context in which the information is broken down to allow growth in knowledge,

understanding and attitudes that allow application of skills in practice at the level of expertise or

competency standard (Cunningham & Duffy, 1996; Iwasiw & Goldenberg, 2015). The primary

intent of formative assessment and feedback is to endow the learner with the ability to improve

as a self-regulated learner. There is evidence that students can be provided the opportunity in

learning to construct their understanding and apply what is learned ((Nicol &Macfarlane-Dick;

Gould & Day, 2013). Research on best practices for feedback suggests that for feedback to be

valuable it must be delivered in a form that is easily comprehended and readily useful for the

learner (Hennessey & Forrester, 2014; Nicol & Macfarlane-Dick, 2006).

Audio feedback is a type of feedback delivery of information that confers assessment of

work in education that is delivered by either taped or digital venue. (Merry & Orsmond, 2008;

Nicol & Macfarlane-Dick, 2006; Price et al., 2010) The current available body of evidence

supports the value of audio feedback as students’ report perceiving audio feedback as

convenient, personalized in nature, contributory to learning, and beneficial in development of

academic skills (Carruthers et. al., 2015; Lunt & Curran 2010; Munro & Hollingworth, 2014;

Race & Williams, 2018). There is a dearth of literature that explores audio feedback as an

effective method for development of motivational interviewing communication knowledge and

AUDIO FEEDBACK 9

skills with behavioral health patients in baccalaureate nursing students. Ostlund, Wadensten,

Kristofferzon and Haggstrom (2013) explored experiences of Swedish nurses trained in

motivational interviewing and recognized the inherent potential for educating nursing health

providers. Exploring the effect of audio feedback in relation to development of motivational

interviewing knowledge and skills in baccalaureate nursing student education may have the

potential to positively influence the behavioral outcomes of patients. This research aligns with

the AACN Essentials of Baccalaureate Education for Professional Nursing Practice (2008) that

express the necessity for nursing education to prepare nurses to practice evidenced based

collaborative care through the use of communication techniques that foster enhancement of

dialogue in patient centered healthcare (AACN, 2008).

The goal of this study is to contribute to the body of knowledge of the effectiveness of

audio feedback on the development of motivational interviewing knowledge and skills

development in undergraduate nursing students addressing behavioral health issues with patients.

The study evaluated the effects of audio feedback in development of beginning levels of

motivational interviewing (MI) knowledge and skills in baccalaureate level nursing students

addressing behavioral health issues using written feedback as a comparison.

Theoretical Framework

The overall approach to this research was based on the theoretical framework of

Constructivism. Constructivism articulates that individuals construct their understanding and

knowledge of the world through experiencing things and reflecting on those experiences. The

constructivist perspective on learning is one that is built on the premise that integration of new

knowledge is combined with what is already known. (Cunningham & Duffy, 1996). The overall

intention is to enable the learner to self-direct the construction of knowledge regarding the

AUDIO FEEDBACK 10

organization, depth, and structure. The individual builds on existing understanding and

application during a process which synthesizes newly presented information. The scaffolding of

knowledge is done by utilizing presented information and creating the context in which the

information is broken down to allow growth in the knowledge, understanding, and attitudes that

allow application of skills in practice at the level of expertise or competency standard

(Cunningham & Duffy, 1996; Iwasiw & Goldenberg, 2015). Learning is an active process that is

built from and shaped by experiences. Learner-centered teaching encourages ownership for the

student in the learning process and essentially develops skills for engagement in life-long

learning (Cunningham & Duffy, 1996 & Honbein, 1996). To engage in learner centered teaching

the nursing faculty structure learning opportunities that allow students to explore communication

situations presented in the context of patient/ nurse interaction using motivational interviewing

skills. The faculty facilitates students to integrate the motivational interviewing skills and this

allows the learner to build on their prior knowledge and experience.

The concept of active learning and constructivist theoretical ideas relate to ability of

teacher/student interaction to influence the learner. In a classroom simulation or clinical setting,

the learning experience transforms the practice of using communication techniques to impact

patient centered care into something the students integrate into their clinical application. The

assumption that simulation takes acquired knowledge and integrates the understanding into

concrete operations during active learning is one idea that comes to mind. Noting that the

theoretical premises put forth by active learning experiences encompass the learning style of the

learner, interaction between the teacher and student, the idea of situated learning enhancing the

learning experience both cognitively, and through promotion of deeper learning. Constructivism

comes into the picture regarding the way the individual processes the knowledge and learning

AUDIO FEEDBACK 11

experience regarding personal understanding. The theoretical assumptions are that learning is an

active process, knowledge is constructed from (and shaped by) experience and learning is a

personal interpretation of the world (Honebein, 1996). The components of this theoretical

framework aligned with the purpose of this study that explored the impact of audio feedback on

the acquisition of motivational interviewing knowledge and skill in undergraduate baccalaureate

nursing students. The use of active learning strategies to assist the student to process the

knowledge and skills for motivational interviewing is linked to the aim of exploring the influence

of utilizing audio feedback as a formative assessment. This research explored use of audio

feedback as compared to written feedback for influence on development of a beginning level of

motivational interviewing knowledge and skills using a standardized patient case among

undergraduate baccalaureate nursing students addressing behavioral health issues.

AUDIO FEEDBACK 12

Chapter 2

Literature Review

An electronic literature search was conducted in the Cumulative Index for Nursing and

Allied Health Literature (CINAHL), ERIC, PubMed and available EBSCO electronic databases

between 2010 and 2018. This range of years allow capture of studies done with audio feedback

since the hallmark study on student attitudes and audio feedback by Merry and Orsmond in

2008.These databases allow access to educational research in health profession fields, sociology,

psychology, higher education and communication. The keywords used in the search were

motivational interviewing, motivational interviewing knowledge and skills, behavioral health,

baccalaureate nursing students, health professions, students, audio feedback, communication

barriers, communication, healthcare, and education. The search was limited to peer reviewed

research studies inclusive of thesis works. Excluded were meta-analysis literature reviews and

articles not available in English. Boolean operators used were AND, OR, and NOT. An

ancestry approach was also used to identify articles. Approximately 114 articles were identified

and reviewed revealing 51 articles for final inclusion.

Communication in Healthcare

The delivery of high-quality healthcare that is safe is dependent on effective

communication between all members involved in caring for patients (Institute for Healthcare

Communication, 2018; World Health Organization, 2010). The Joint Commission (2016) notes

that failure in communication in healthcare is the third leading cause of sentinel events.

Healthcare providers that engage in care of the behavioral health client utilizing effective

communication skills are noted as making a positive impact in lifestyle behavior changes

AUDIO FEEDBACK 13

associated with Diabetes, Hypertension, and Mental Illness (Blixen, Kanuch, Perzynski, Thomas,

Dawson & Sajatovic, 2016;Drevenhorn, Bengston, Nyberg & Kjellgren, 2014).

Barriers to Effective Communication

The current literature examines multiple aspects of communication in patient doctor

relationships related to patient complaints, barriers and facilitators related to health seeking for

unemployed persons with mental health conditions, perception of information by patients

diagnosed with Depression, and exploration of barriers to effective communication, teamwork in

healthcare (Graham, Hasking, Clarke & Meadows, 2015; Kee, Khoo, Lim & Koh, 2017; Staiger,

Waldmann, Rűsch & Krumm, 2017; Thomson, Outram, Gilligan & Levett-Jones, 2015; Weller

Boyd & Cumin, 2014). Poor communication is noted to be an integral part of dissatisfaction

reported in the provider/patient relationship. Among the highlighted barriers to establishing a

productive rapport with providers reported by patients are lack of trust, respect, empathy, active

engagement in listening, poor quality of information, and not feeling supported by clinicians

during clinical encounters (Gilburt, Rose & Slade, 2008; Kee et al., 2017; Staiger et al., 2017;

Weller et al., 2014). Health literacy is noted to influence the client understanding of information

provided in relation to behavioral health issues (Grahm et al., 2015; Staiger et al., 2017).

Grahm et al. (2015) explored the receipt and perception of behavioral health information

among those diagnosed with Depression in Australia and identified that the three predictors of

educational level, perception of need for mental health services and receipt of mental health

services were significantly related to receipt of information and perception of mental illness

information as helpful. Those with a higher level of education are more likely to receive mental

health information and perceive it as beneficial. This relationship to patient perception is also

reflected in the findings by Staiger et al. (2017) who notes that those unemployed patients

AUDIO FEEDBACK 14

diagnosed with mental illness report perceptions of being treated different in social settings and

by healthcare workers relative to their diagnosis of mental illness. These perceptions patients

report view help seeking as associated with negative traits such as helplessness and weakness

(Staiger et al., 2017).

The amount and quality of information are fundamental elements of communication

between healthcare provider and client. Poor quality or lack of complete and factual information

contributes to client unmet needs about knowledge of care in those with mental illness and other

comorbid conditions such as diabetes (Blixen et al., 2016; Kee et al., 2017). A recent study by

Hayes, Wolfe, Labbé, Peterson and Murray (2017) examined the primary health providers’ role

in the treatment of patients affected by obesity identify challenges associated with

communication among primary team members and patients. The lack of collaborative integrated

care among providers, acceptance of obesity as a chronic condition, and clearly defined roles of

providers impacts care and adds to an awareness of feeling unimportant for both providers and

patients. There is an identified need for a client sense of autonomy, and perception of a trusting

relationship between client and provider. This expertise is expressed in demonstration of clinical

knowledge communicated during patient/provider encounters (Hayes, et al., 2017; Weller et al.,

2014).

Recommendations for Decreasing Communication Barriers

The research reveals a consistent emphasis that it is important for patients to be partners

in making decisions and be active in their behavioral healthcare (Grahm et al., 2015; Weller et

al., 2014). The desire for patients to seek information may be associated with a desire to reduce

anxiety, tension and stress experienced dealing with mental health issues (Blixen et al., 2016;

Staiger et al., 2017). Patient acquisition of knowledge, increased awareness of mental health

AUDIO FEEDBACK 15

conditions, perceived social support and desire to change increased motivation to seek help

(Staiger et al., 2017). Recognition by providers that patients’ educational needs for information

are always changing and not static is important. Awareness that communication is influenced by

patient perception of negative attributes associated with help seeking, communication ability,

emotional state, perceived provider lack of empathy or respect, and role expectation are elements

that impact the interaction between provider and patient (Kee, et al., 2017; Staiger et al., 2017).

The literature suggests a need for integration of team based person centered care that is

holistic in nature (Blixen, 2016;Hayes, et al., 2017). Healthcare service providers should

acknowledge that integrative practice for psychiatric and medical conditions together promote

involvement and communication in a holistic manner. Encouraging self-management practices can

lead to decrease isolation of clientele as experience of stress impacts their health state. Providing

social support by talking to family and friends is one avenue of approach (Blixen et al., 2016) The

sharing of all information across inter-professional care teams coupled with training for

development of clinician self-awareness of communication strategies assist in teaching providers

how to construct therapeutic relationships with patients (Gilburt et al., 2008; Hayes et al., 2017).

Research suggests use of the communication strategy of motivational interviewing is deemed an

effective, patient centered, therapeutic communication approach for guiding patients (Kazemi,

Levine Dmochowski, Nile & Sun, 2013; Miller, 1983; Miller & Rollnick, 1991; Miller& Rollnick

2013; Shannon, Donovan-Hall & Bruton, 2017).

Motivational Interviewing

Motivational interviewing (MI) is a conversational approach to counseling that is

collaborative in nature and is utilized to support a person-centered motivation and commitment to

alter a behavioral condition (Miller, 1983; Miller & Rollnick, 1991; Miller& Rollnick 2013).

AUDIO FEEDBACK 16

MI has its origins in the experience of treating individuals with alcohol problems as described by

Miller in 1983 and has evolved as a patient centered counseling style for evoking behavior

change through collaborative partnership through guided communication (Miller, 1983; Miller &

Rollnick, 1991; Miller& Rollnick 2013; Petroliene, 2013). This communication style has evolved

to a counseling approach for use with individuals to engage, clarify strengths and desires, and

possibly evoke individual motivation for change while promoting autonomy (Lundahl, Kunz,

Brownell, Tollefson & Burk, 2010; Miller & Rollnick, 1991; Miller & Rollnick, 2013).

The overall approach to using MI is constructed from a philosophical standpoint that

employs a wide collection of techniques to assist people in approaching change and explore

uncertainty about behavioral change. The main philosophical orientation articulates a person-

centered approach with the recognition that each individual is unique in their level of readiness

to change. MI uses a facilitated style of communication to engage individuals in the process of

gaining clarification on their strengths and aspirations and inducing personal motivations for

change while still maintaining a sense of autonomy (Miller & Rollnick, 1991; Miller & Rollnick,

2013).

The concepts of MI include two dynamic perspectives. The spirit of MI and the four

processes of MI. The spirit of MI is comprised of the four vital aspects of partnership,

acceptance, compassion and evocation. For each of these aspects there is an experiential and

behavioral element. This is highlighted in the example that one can have the experience of

acceptance but without the behavioral expression of acceptance towards another it is not useful.

In brief, the components of MI can be defined simply by looking at them as habits of spirit.

Partnership is defined as the respect for another in an interpersonal exchange that is

accomplished through the collaboration of working with the person and accompanying them on

AUDIO FEEDBACK 17

their journey toward change. Acceptance is the attitude and acknowledgement founded in the

dispositions of absolute worth, autonomy, accurate empathy and affirmation. This involves

viewing another person’s absolute worth as valuable and respect for who they are as an

individual while taking an active interest in the person and recognizing their right to self-

determination, displaying empathy for their perspective, and supporting their strengths and

efforts. Compassion is defined as the disposition of having good intention and fostering the

welfare of others. Evocation is defined as the disposition of recognizing that an individual has

strengths and resources and acting on these can induce possible motivation to change (Miller &

Rollnick, 1991; Miller & Rollnick, 2013).

The four processes involved in MI are engaging, focusing, evoking and planning. They

are sequential and foundational in nature and are all reliant on each other. The process of

engaging is defined as establishment of a connected working relationship founded on trust and

mutual respect. The process of focusing is the second tier and founded on engagement which

leads to emphasis on a particular direction for change while engaging in conversation. It assists

in clarifying the direction in which the individual has intention to move toward. The third tier of

the process is evoking and this is defined as having the individual voice the arguments for

change and capturing their ideas about how to accomplish the desired change. The planning

process is defined as the engagement in conversation about actions and plans that promote an

individual’s autonomy and support a continuing strength for their commitment toward change

(Miller & Rollnick, 1991; Miller & Rollnick, 2013).

The overall framework outlines the method used to encounter individuals in a person-

centered communication encounter that is designed to strengthen individual sense of motivation

and commitment toward a specific goal by arousing and exploring the individual’s reasons for

AUDIO FEEDBACK 18

change within the context of a therapeutic relationship that is based in acceptance and

compassion. The ability to develop communication within this framework is directly related to

the exploration of using audio feedback to strengthen MI skills in baccalaureate nursing students

so that they may be prepared to encounter behavioral health patients in all healthcare settings to

promote optimal levels of wellness and health.

At present motivational interviewing is considered an effective, patient centered,

therapeutic communication approach for guiding patients to engage in a behavioral health change

process (Kazemi, Levine, Dmochowski, Nile & Sun, 2013; Miller & Rollnick, 1991; Miller &

Rollnick, 2013). Motivational interviewing is a technique that is currently being utilized to

impact health behaviors of alcohol and other substance use disorders, adherence to treatment

regimens with a variety of health conditions and training for health providers (Chang, Compton,

Almeter & Fox, 2015; Hirdle & Vaughan, 2016; Russell, Cronk, Herron, Knowles, Matteson,

Peace & Ponferrada, 2011). The use of MI to impact the way healthcare providers interact with

patients who present with behavioral health issues is recognized as showing promise to improve

adherence to treatment regimens and impact health behaviors of specific populations such as

freshmen college student engagement in high-risk drinking and illicit drug use (Kazemi et al,

2013; Magill, Colby, Orchowski, Murphy, Hoadley, Brazil & Barnett, 2017; Russell et. al,

2011).

Current educational research exploring health providers being trained in motivational

interviewing include randomized study protocol for trial testing three strategies for use on

inpatient medical units, exploration of the influence of evidence based methods on addiction

treatment outcomes, exploration of required supervision to attain proficiency, and examination of

the effect of training on primary care staff are articulated in recent literature (Cucciare, Ketroser,

AUDIO FEEDBACK 19

Wilbourne, Midboe, Cronkite, Berg-Smith & Chardos, 2012; Martino, Zimbrean, Forray,

Kaufman, Desan, Olmstead, Gueorguieva, Howell, McCaherty & Yonkers, 2015; Miller &

Moyers, 2014 & Schumacher et al., 2018). Shaping health professional education to include MI

skills has shown promise in nurse practitioner students, psychologists, community workers,

nurses, primary care nurses, Human Immunodeficiency Virus/Auto Immune Deficiency

Syndrome (HIV/AIDS) counselors, primary care staff, and medical students (Cucciare, et al.,

2012; Edwards, Stapleton, Williams & Ball, 2015; Evangeli, Engelbrecht, Swartz, Turner,

Forsberg & Soka, 2009; Nesbitt, Murray & Mensink, 2013; Ostlund et al., 2013; Stoffers &

Hatler, 2017).

Current strategies used by health profession educators, researchers, and professional

development departments are outlined in the literature to investigate the use of motivational

interviewing techniques for training healthcare providers, counselors, and students include

development of virtual and face to face training workshops, creation of checklists and vignettes

for initiating a response from the clinician, investigation of organizational factors related to

provision of provider training, measurement of clinical competence, effects of video feedback on

skill and competence and qualitative inquiry into provider experience of using motivational

interviewing (Amodeo et al., 2013; Cucciare et al., 2012; Kennedy et al., 2016; Noordman et al.,

2014; & Ostlund et al., 2013). The research also includes studies that utilize motivational

interviewing to target specific health behaviors among freshmen college students and blackouts

from alcohol, adherence to opioids among older adults with chronic pain, adherence to dialysis

and treatment regimen for end stage renal disease patients, and reducing injection high risk

behaviors among people who inject drugs (Bertrand, Roy, Vaillancout, Vandermeerschen,

Berbiche & Boivin, 2015; Chang, Comptom, Almeter & Fox, 2015; Magill, Colby, Murphy,

AUDIO FEEDBACK 20

Hoadley, Brazil & Barnett, 2017 & Russell, et al., 2011). The majority of these studies

conducted follow up within short periods after training or MI utilization. The findings suggest

that MI influenced patient centered care positively, and in some cases, significantly impacted the

clinician/patient relationship and the behavioral health outcomes of patients (Amodeo et al.,

2013; Chang et al., 2015; Drevenhon et al., 2015 & Magill et al., 2017).

One recent study explored the impact of educational training in MI knowledge and skills

for a blended group of healthcare providers. Edwards, Stapleton, Williams and Ball (2015)

conducted a study with 163 health providers including psychologists, counselors, nurses, and

community workers. The research study examined a brief MI training and explored the effects on

enhancing provider levels of confidence in and use of MI knowledge and skills for directing

eating and exercise behavior change. The researchers utilized the Motivational Interviewing

Knowledge and Attitudes Test to measure knowledge, The MI Confidence Scale to assess level

of confidence, and the Behavior Change Counseling Index to assess skills in MI. The results

displayed that the trained group showed significant increases from pre-training to post-training in

knowledge, skills, and confidence levels (Edwards et al., 2015). The findings support similar

research conclusions endorsing the use of intervention in motivational interviewing knowledge

and skills training for health providers providing care to behavioral health patients (Chang et al.,

2015; Cucciare, Ketroser, Drevenhon et al., 2015 & Magill et al., 2017; Stoffers & Hatler, 2017).

Education and Feedback

In education, active learning occurs as the learner participates in the process rather than

remaining a passive observer. Learners who are active acquire skills through engaging in hands

on experiences and learn by doing. An active learner retains information longer than a passive

student. The cycle of learning involves the learner receiving feedback to provide information for

AUDIO FEEDBACK 21

the learner to adjust their performance of skill or increase their knowledge and comprehension

(Forrest, 2005; Gould & Day, 2013). Feedback is conceptually defined as the delivery of

information that is delivered by a representative regarding an aspect of comprehension or

performance. The representative can be a teacher, peer, book, parent, self, or even experience

that delivers the information. (Hattie & Timperley, 2007). Feedback is formative assessment that

provides the learner with information related to performance objectives and benchmarks and

expected standards. In higher education formative assessment and feedback should allow the

learner to participate in the act of self-regulation of goal achievement (Gould & Day, 2013; Nicol

&Macfarlane-Dick).

Self-regulation includes the learner monitoring their own thinking, learning and

performance and they need to be able to comprehend and engage with the feedback (Merry &

Orsmond, 2008; Price, Handley, Millar & O’Donovan, 2010). The research on best practices for

feedback iterate that for feedback to be valuable it must be delivered in a form that is easy to

comprehend and readily useful for the learner (Hennessey & Forrester, 2014; Nicol &

Macfarlane-Dick, 2006). The principles of good feedback are articulated in the seven factors

outlined by Nicol and Macfarlane- Dick (2006) and include providing information to learners

that encourage dialogue with educators, delivery of criteria related observations to students,

facilitate reflective learning, provide opportunities for achievement in the learning process, and

postulate information for educators that can be used to inform teaching. The current research on

audio feedback notes these factors align with best practices for conducting research using

feedback (Green, 2015; Gould & Day, 2013; McCarthy, 2015).

AUDIO FEEDBACK 22

Audio Feedback

Audio feedback is defined as information conferring assessment of work in education that

is delivered by either taped or digital venue. Audio feedback is also referred to as auditory

commentary (Merry & Orsmond, 2008; Nicol & Macfarlane-Dick, 2006; Price et al., 2010). In a

hallmark study of student attitudes regarding the use of audio feedback Merry and Osmond

(2008) establish that this modality of feedback is perceived by the students to have value for

comprehending the assessments and integrating them in a meaningful way into their learning

process. This finding is reiterated in research that explores the use of audio feedback with

students (Lunt & Curran, 2010; Munro & Hollingworth, 2014).

Audio feedback is utilized in research to investigate the effectiveness of the modality

with learners in the fields of education and early childhood studies, among diagnostic imaging

students, online learners in an asynchronous teaching strategies course, medical students,

community pre licensure nursing students in the United Kingdom, a cohort of master’s level

nursing students in the clinical setting in the United States, post graduate computed education

students in Australia, and junior- level nursing students enrolled in a medical surgical nursing

course or a research course (Bourgault et al., 2013; Carruthers, McCarron, Bolan, Devine,

McMahon-Beattie & Burns, 2015; Cavanaugh & Song, 2014; Gould & Day, 2013; Green, 2015;

Harrison, Molyneux, Blackwell & Wass 2014; Ice, Curtis, Phillips & Wells, 2007; Race &

Williams, 2018).

The practice of delivery of audio feedback is related to the context of the assigned work

that the feedback is being directed toward and the nature of feedback delivery mode (Hennessy

& Forrester, 2014; Price et al., 2010). Audio feedback delivered with clarity of purpose and a

clear formative intent to develop student knowledge and ability to apply the feedback in

AUDIO FEEDBACK 23

subsequent work is valued by learners (Carruthers, McCarron, Bolan, Devine, McMahone-

Beattie & Burns, 2015; Hennessey & Forrester, 2014; Parkes & Fletcher, 2017; Price et al.,

2010). Cuthrell, Fogarty, Smith and Ledford (2013) reported similar findings for use of peer

audio feedback delivery process with undergraduate and graduate college students. They note

that students value the feedback for increasing knowledge and completing assignments.

Research supports the descriptive points made by Merry and Orsmond (2008) that

students report perceiving audio feedback as convenient, personalized in nature, contributory to

learning, and beneficial in development of academic skills (Carruthers et. al., 2015; Lunt &

Curran 2010; Munro & Hollingworth, 2014; Race & Williams, 2018). The qualitative data

collected reflects commonalities in findings that specifically point toward audio feedback as a

complimentary alternative to attain deeper understanding of concepts that are presented in higher

educational settings in both virtual and classroom learning environments (Carruthers et al., 2015;

Ice et al., 2007; Rasi & Vuojärvi, 2017). Quantification of student perception done in a

classroom setting in the United Kingdom by Gould and Day (2013) noted that a majority of

students (n=49) 92% report audio feedback as instrumental in their learning experience This

mixed methods study included student expression of the use of audio feedback technique as

being “easy to understand what the (lecturers) meant” and like a “mini tutorial” (p. 561). A

recent study by Parkes and Fletcher reported on a three-year longitudinal study exploring the

experiences of postgraduate level students provided with audio feedback in a computer education

distance-learning program. The 225 students surveyed perceived audio feedback as clear and

easy to comprehend with 92% of students noting audio feedback was higher quality than

previously received written feedback. There were no statistically significant differences found

between male and female attitudes towards audio feedback (Parkes & Fletcher, 2017). These

AUDIO FEEDBACK 24

student perceptions reflect the positive learner sentiment expressed across different learner

groups and environmental settings (Ice et al., 2007; Harrison et al, 2015; Munro & Hollingworth,

2014; Parkes & Fletcher, 2017).

Audio Versus Written Feedback

Comparison between utilization of audio feedback versus written feedback has been the

focus of inquiry in the research done with science laboratory first year university students,

pharmacy students, nursing students and clinical assignments, tertiary digital media students in

assessment of video, audio and written venues for feedback, and with students in virtual online

learning environments (Bourgault, et al., 2013; Cavanaugh & Song 2014; Lunt & Curran, 2010;

McCarthy, 2015 & Morris & Chikwa, 2016; Nemec & Dintzner, 2016). The research exploring

the formats of audio feedback and written feedback seeks to clarify student perceptions of these

forms of feedback in different branches of education among a variety of settings (Ice et al., 2007;

Lunt & Curran, 2010, & Merry & Orsmond, 2008). The technology utilized with delivering

audio feedback in research consists of software packages, MP3 files and unspecified audio files

loaded into virtual learning environments. Research explored student preference in feedback

venues along with ability to assist their learning in terms of interpretation and effectiveness.

Qualitative findings reveal common themes across the research that include perceiving audio

feedback as more personalized than written feedback, increased availability of detail in audio

feedback, and increased sense of comprehension of feedback for utility in learning (Ice, Curtis,

Phillips & Wells, 2007; Lunt & Curran, 2010; Nemec & Dintzer, 2016; Sipple, 2007).

The student perspective on preference for feedback is noted to be associated with their

preferences for how they integrated feedback to address correcting assignments. Handwritten

feedback is noted as a way for students to locate their need corrections and then proceed to revise

AUDIO FEEDBACK 25

their written work (Cavanaugh & Song, 2014;Sipple, 2007 ). Recently Morris and Chikwa

(2016) investigated audio and written feedback, learner preference and possible impact on

student academic performance. The study reports that type of feedback did not impact student

academic performance in later assignments. McCarthy (2015) examined audio feedback use in

summative assessment for student preference when compared to video feedback and written

feedback. Findings reveal video feedback is preferred over audio and written forms of feedback.

It is noteworthy that in this study written feedback is attached to fifty percent of their grade and

students report perceiving written feedback as the legitimate source of feedback and these factors

may have impacted student perception (McCarthy, 2015).

Overall the research articulates that comparison of audio feedback versus written

feedback is not significantly associated with any one learning environment as both face to face

and online students expressed broad positive appreciation for both types of feedback and

preference for feedback method may be context specific (Cavanaugh & Song, 2014; Morris &

Chikwa, 2016). Bourgalt et al. (2013) noted no statistical significance for preference of audio

versus written feedback on clinical assignments with nursing students when correlated with

learning style preferences. This was also indicated in the commentary students provided for use

of audio feedback in the online environment where the student preference iterated relationship

with preference for written feedback on areas of assignments where finite details impact

corrections (Cavanaugh & Song, 2014). Similar findings are reported by Morris and Chikwa

(2016) noting that even though students expressed positive general feelings in relation to audio

feedback the preferred method for future feedback was written and this was due to their

preference for being able to connect specific comments to specific areas in their assignments.

The review of literature on comparing feedback methods reveals that possible connections to

AUDIO FEEDBACK 26

explore are characteristics of the learner such as first-generation college, high risk, culturally

diverse, English second language, and preferred learning style of the student along with

identifying the nature of the learning task and type of feedback provided (Cavanaugh & Song,

2014; Morris & Chikwa, 2016; Race & Williams, 2018).

Summary of the Literature

Learning experiences that provide students with feedback can enhance understanding of

the content and influence the development of knowledge and skills through self-regulated

learning (Forrest, 2005; Gould & Day, 2013). Audio feedback that is delivered with a clarity of

purpose and a clear formative intent to develop student knowledge and ability to apply the

feedback in subsequent work is a learner valued instructional strategy (Carruthers, McCarron,

Bolan, Devine, McMahone-Beattie & Burns, 2015; Hennessey & Forrester, 2014; Price et al.,

2010). Communication is a cornerstone of provider/patient relationship development and self-

awareness of communication strategies can facilitate providers to build therapeutic relationships

in behavioral health patient encounters (Barr, Bonasia, Verma, Dannenberg, Yi, Andrews,

Durand, 2018; Blixen et al, 2016; Gilburt et al., 2008; Hayes, et. al, 2017). A recent study

highlights the impact of audio feedback in the clinical arena. Barr et al., (2018) utilized a cross-

sectional survey to explore the prevalence of the use of audio recordings of clinic visits, attitudes

of public and clinicians, and existence of policies regarding the use of audio feedback. The

sample consisted of 456 clinicians and 524 public respondents. The results revealed that 28.3

percent of clinicians and 18.7 percent of the public have utilized audio recording as a way of

obtaining both provider and personal client feedback. Qualitative findings revealed that use of

audio recordings is viewed as favorable ways of obtaining feedback for both clinical outcomes

and client behavioral and health related outcomes (Barr et al., 2018).

AUDIO FEEDBACK 27

MI is a significant communication component part of patient centered care delivery that

can be utilized to impact patient outcomes for a multitude of behavioral health conditions (Chang

et al., 2015; Drevenhon et al., 2015 & Magill et al., 2017). Further research is needed to

investigate the use of instructional strategy of audio feedback to teach baccalaureate level

nursing students about motivational interviewing knowledge and skills.

The changing and evolving healthcare environment requires nurses and healthcare

providers to be prepared to deliver client centered interventions for addressing behavioral health

concerns during patient encounters. The use of motivational interviewing knowledge and skills is

an educational component recognized as a necessary training requirement for healthcare

providers to address patient health behaviors. Research supports the need for educational venues

to include training in motivational interviewing knowledge and skills in healthcare provider

training. Nursing students as future frontline providers in healthcare need education in this

patient centered communication technique that can influence patient care outcomes. Instructional

strategies such as audio feedback may provide new venues for educational interventions to equip

nurses as care providers to deal effectively with behavioral health issues in a complex changing

healthcare environment.

AUDIO FEEDBACK 28

Chapter 3

Methods

Design

An experimental 2 group pretest/posttest randomized control group, single blinded study

design was used to determine if audio feedback was superior to written feedback in the

development of beginning levels of motivational interviewing knowledge and skills among

undergraduate baccalaureate level nursing students in the context of a behavioral health scenario.

The randomized control group design was used as it reduces internal threats to validity and the

influence of alternative hypotheses. The outcome measures included a 19-item knowledge test

(MIKAT) score and a calculated observational skills assessment score (BECCI) as well as a

research assistant estimate of the percent of time the participant spent talking to the simulated

client during the motivational interview. The research assistant responsible for ascertaining

outcomes was blinded to treatment group status. Figure 1 presents a Consolidated Standards of

Reporting Trials (CONSORT) flow diagram of the progress through the phases of this research

with the parallel randomized trial of two groups (Boutron, Altman, Moher, Schulz, & Ravaud,

2017).

AUDIO FEEDBACK 29

Figure 1. CONSORT Flow diagram of phases of randomized trial research groups

Setting

The study was conducted in a small private liberal arts college located in the Northeastern

United States. The college has a school of nursing with a traditional undergraduate baccalaureate

nursing program comprised of 120 students direct from high school and the nontraditional

program is comprised of 60 students with a prior baccalaureate degree. The nursing students are

mainly English speaking, Caucasian and female. Approval to conduct the study was obtained

from the Institutional Review Board (IRB) of the college.

AUDIO FEEDBACK 30

Sample

A convenience sample of nursing students from a baccalaureate nursing program in a

single private college were invited to participate. The traditional undergraduate baccalaureate

nursing programs is comprised of students direct from high school and the nontraditional

program is comprised of those with a prior baccalaureate degree. The potential eligible sample

was 180 nursing students. Participants met inclusion criteria if they were traditional and second-

degree undergraduate nursing students in the psychiatric, community health, and senior

practicum nursing courses who completed their ‘Adult I Medical Surgical Nursing’ coursework.

Students were excluded if they were graduate students, post-licensure students, or had not

completed their ‘Adult I Medical Surgical Nursing’ having not received foundational education

on health provider communication. Using published descriptive statistics from a prior study that

used the MIKAT tool (Dear 2014), an online power and sample size calculator (Brant, 2017) was

used to determine that 35 participants were needed for each treatment group to achieve 80%

power with an alpha of 0.05 to observe a difference of 2.83 points (SD 3.1) in MIKAT score

across the educational intervention.

Recruitment

Participants were recruited through direct email invitation, IRB approved flyers and

brochures posted in multiple locations throughout the college and via the learning management

system (Blackboard) with faculty permission. In addition, the principal investigator set up a table

with balloons and invitation to participate flyers in common areas on campus at three different

times. Snowball sampling was also utilized as a technique to recruit participants.

AUDIO FEEDBACK 31

Data Collection Tools

Knowledge was measured with the Motivational Interviewing Knowledge Attitudes Test

score and skills were measured by a calculated score on the Behavioral Change Counseling

Index Tool. Demographic variables were collected using a short survey (Appendix D) and

included age, gender, race, and whether or not students were in the accelerated second-degree

program or traditional undergraduate program.

Knowledge and attitudes test. The pre and post-test used in the study was the MIKAT

developed by Leffingwell (2006) as a relatively simple test of motivational interviewing

knowledge and attitudes consistent with principles of MI. For this study, the control and

experimental groups had a single opportunity to complete both the pretest and post-test in paper

form. The pretest served as a method to measure baseline knowledge of motivational

interviewing. The post- test served to evaluate knowledge after completing the intervention. The

tool was validated in pretest/posttest study design done with child and family home-based care

providers (case workers, social workers, team leaders of clinical practice) and youth service as

novice trainees participating in a training workshop for motivational interviewing. The test is

comprised of 15 items in total; 14 true or false items about addiction myths and motivational

interviewing attitudes and assumptions, and a single item counseling behaviors checklist which

is a ‘select all that apply’ single item describing behaviors recommended for an effective

motivational interviewing approach. Scoring requires calculation of a summary score by tallying

the number of correct true and false items with a possible range of 0-14 points and a separate

summed score for the number of correct selections on the counseling behaviors checklist item

with a possible range of 0-5 points. A score closer to 14 on the true and false questions indicates

ability to identify addiction myths and attitudes and assumptions consistent with motivational

AUDIO FEEDBACK 32

interviewing. The checklist scores closer to five is indicative of ability to identify behaviors

consistent with a motivational interviewing approach. The two summary scores are added

together for a total score which can range from 0 -19 percentage points. Changes in the

‘summary score’ from pretest to posttest (posttest minus pretest) were recorded and the

difference in mean change within and between experimental groups were analyzed for statistical

significance (Leffingwell, 2006).

The MIKAT (Appendix A) tool is reported as valid and reliable among correctional

juvenile facility staff (Doran, Hohman & Koutsenok, 2011). Internal consistency of the tool has

been reported only once in the literature disclosing a Cronbach’s a of 0.84 among correctional

facility staff (Doran, Hohman & Koutsenok, 2013). The MIKAT is sensitive to the effects of

training producing significant improvement in knowledge scores consistently among correctional

facility workers, rehabilitation focused employment case managers, healthcare workers, and staff

of child/youth family services populations (Dear, 2014; Doran et al., 2011; Edwards et al., 2015

& Manthey, 2013). Simon and Ward (2014) utilizing a slightly altered version of MIKAT with

academic advisors to promote use of MI knowledge and skills note the data met distributional

assumptions prior to conducting independent t-tests. In single group pretest/posttest research

significantly higher scores on the post test MIKAT summary score demonstrate beneficial

outcomes after motivational interviewing trainings (Doran et al., 2013; Doran et al., 2011;

Edwards et al., 2015; Manthey, 2013; Simon & Ward, 2014). Dear (2014) reported significant

increases in mean scores from pre-training to post-training among staff of child/youth family

services noting an eighty-five percent power, and observing a medium effect size (d= .50).

Leffingwell (2006) provides permission for use of the tool in research and education (Appendix

A).

AUDIO FEEDBACK 33

Skills index tool. The Behavior Change Counseling Index (BECCI) (Appendix B) is a tool

that is an adaptation of motivational interviewing skills assessment suitable for brief encounters in

healthcare settings developed by Lane (2002). The tool assists trainers and researchers to evaluate

change in skill application post training. Permission was granted by the author for use (Appendix

C). The tool has demonstrated acceptable levels of validity, reliability, and responsiveness among

healthcare practitioners (Lane, Huws-Thomas, Hood, Rollnick, Edwards & Robling 2005). The

internal reliability of the BECCI in prior research is reported as a = 0.71 a pre-level training

assessment at baseline and a = 0.63 post training among a healthcare practitioner population (Lane

et al., 2005). A recent research study conducted in a randomized control trial of behavior change

counseling done with medical students report an internal reliability of the BECCI as assessed by

Cronbach’s alpha as 0.82 at baseline and 0.77 post training assessment (Spollen, Thrush, Mui,

Woods Tariq & Hicks, 2010). The tool has been utilized to assess behavior change counseling

skills post training. The mean total scores are calculated across all items. The tool is presented in

prior research with simulated consultations to train practicing health providers (Edwards et al.,

2015; Lane et al., 2005). The tool and coding manual are available online and for use without

permission from www.motivationalinterviewing.org.

The tool consists of 11 items to code application of motivational interviewing skills. The

11 items are scored 0-4 on a five-point Likert scale ranging from 0 = “not at all” to 4 = “a great

extent”. Total scores can range from 0 to 44 and are calculated as the summary scores across all

items. Higher scores are indicative of greater ratings of skills There is one additional item that is

scored separately and captures the proportion of time the practitioner speaks during the interview

process and is categorized as more than half, about half, or less than half the time (Lane, 2002;

Lane et al., 2005).

AUDIO FEEDBACK 34

Although there is controversy in the literature that address the idea of using Likert type

responses in research as ordinal versus continuous (interval) data, Bishop & Herron (2015) note

that Carifio and Perla (2007; 2008) strongly endorse treating Likert type responses as interval

data. They posit that all true scales should include multiple questions on a specific topic and the

summative score reflects the measurement and is the proper item of analysis versus individual

items. This view is supported by Willits, Thoedori and Luloff, (2016) who note that Likert type

items can be treated interval data representing numerical ratings and can be combined into

composite scores. Therefore, summary scores will be treated as interval level data in this study.

Research assistants were trained and utilized practice audio tapes prior to conducting

analysis and coding of participant audio files. For this study the process entailed training sessions

with a nurse consultation expert in motivational interviewing. Following training the two raters

participating at different times in this study utilized the BECCI tool to rate the audiotapes. The

interrater reliability was calculated by examining the agreement between the raters use of the tool

for scoring on two audio recordings. The percentage of agreement on ratings achieved were 73

and 84 percent. As recommended by Hallgren (2012) for ordinal or interval data using

percentage of agreement is acceptable.

In summary, the tools used to collect data in this study have been found to be sensitive to

the effects of training programs. In this study knowledge was assessed using the MIKAT tool and

skills were assessed using the BECCI tool.

Procedures

The outcomes of the study were to evaluate changes in knowledge as measured by the MIKAT

score and changes in skills as measured by the (BECCI) score across the educational initiative

and between the groups and within groups.

AUDIO FEEDBACK 35

Randomization. After informed consent was obtained, a stratified random sample of the

two defined groups of nursing students; traditional undergraduate and second-degree nursing

students was utilized. Randomization of each stratum to the experimental and control groups

controlled for the potentially confounding effect of student type in the relationship between

audio feedback and motivational interviewing skills. An online ‘Research Randomizer’ was

used to create the stratified random assignment to the control and experimental groups (Urbaniak

& Plous, 2013).

Both randomized groups received the same educational instruction to assess the effect of

whether audio or written feedback during an educational intervention would improve knowledge

and skills. The experimental group received audio feedback via an emailed mp3 file and the

control group received written feedback in the form of a BECCI tool document.

Intervention

The study was conducted in a multistep process. Figure 2 displays the flowchart of the

procedures that took place in the study after informed consent was obtained and participants

were randomly assigned to treatment group. Both groups received the exact same education, the

only difference in procedures was the use of audio feedback versus written feedback as a

formative assessment of learning during the motivational interviewing educational initiative.

AUDIO FEEDBACK 36

Figure 2. Flowchart of Research Procedure

Baseline surveys. The students in both groups took a pre-intervention MIKAT to

establish a baseline evaluation of participant knowledge of MI and to ensure no baseline

differences in knowledge between experimental groups. Once this baseline was established

participants were invited to complete the education.

Internet based, self-paced educational training in motivational interviewing.

Following the pretest participants were asked to complete an internet-based training in MI. The

study participants independently completed the following learning activity and achieved the

certificate before moving on to the interview session. Students in both groups completed a four-

hour self-paced learning program entitled ‘A Tour of Motivational Interviewing’. The course

introduced the learner to the use of essential motivational interviewing skills through five

individual modules. This course was developed by the University of Missouri Kansas City

School of Nursing and Health Studies Mid-America Addiction Technology Transfer Center

Network (ATTC) and funded by the Substance Abuse and Mental Health Services

Administration (SAMHSA). The content of the course is evidenced based and validated by the

findings from the National Institute on Drug Abuse Clinical Trials Network ‘The Science of

AUDIO FEEDBACK 37

Treatment: Dissemination of Research-based Drug Addiction Treatment’ publication as an

effective way to train providers in motivational interviewing (National Institute on Drug Abuse,

2009). The free course was offered on a web-based platform from HealtheKnowledge.org where

the participants registered and completed the modules at a pace that allows engagement with

demonstration, descriptions and learning activities that provide an introduction to motivational

interviewing (ATTC, 2018). The course composed of five modules allowed the participants to

repeatedly review the modules and take the tests at a self- pace and then obtain a certificate of

completion after passing each module assessment test. The participant emailed the certificate of

completion to the research assistant prior to participation in the first interview session.

Interview sessions. After verifying completion of the online modules each participant

conducted the first of two interview sessions utilizing MI skills. Participants practiced one at a

time engaging as the nurse for 8 to 16 minutes in the interview session, using a scripted

standardized patient presentation developed by the researcher in consultation with faculty expert

who is a licensed psychiatric nurse practitioner with over 20 years of experience in using

motivational interviewing. The standardized patient used in all interviews was based on a

scripted audio file Moyer, Martin, Catley, Harris and Ahluwalia (2003) developed for training

purposes and approved by the content expert. All interview sessions were audiotaped. The MP3

audio files were uploaded to a password protected computer file and stored in a Google drive

account created specifically for this purpose. The research assistants were the only ones with

access to this account. The audiotapes were identified by a unique study ID and the research

assistant was blinded as to whether the audiotaped interview was from a participant in the

experimental or control group. The research assistant rated each individual interview tape using

the BECCI tool by checking off each successfully completed skill. Following the first practice

AUDIO FEEDBACK 38

sessions, formative assessments were provided to each group. The control group received written

feedback using the BECCI tool of observed ratings checked off for motivational interviewing

skills while the experimental group received audio feedback in MP3 format describing their

performance related to the criteria on the BECCI tool of observed ratings for motivational

interviewing skills. Both forms of feedback were sent to participants by email. After receiving

this formative feedback, both groups were presented with another opportunity to apply the

principles of MI.

The second interview session was conducted exactly the same as the first session using

the same scripted standardized patient. These were also uploaded to the same password protected

Google account for rating by the research assistant. This final assessment of behaviors/skills

formed the BECCI tool post interview outcome measure. The idea was that the formative

feedback was used to inform an application of skills change during the second motivational

interview session.

Participants received formative assessments either through audio feedback or written

feedback. Audio feedback is defined as information conferring assessment of work in education

that is delivered by either taped or digital venue. Audio feedback is also referred to as auditory

commentary (Merry & Orsmond, 2008; Nicol & Macfarlane-Dick, 2006; Price et al., 2010).

Written feedback is a description of performance as it relates to criteria of a specific tool or set of

procedural steps and measures.

Experimental groups. Participants were randomized to receive one of two types of

formative assessment. The experimental group received audio feedback and the control group

received written feedback. The students in the experimental audio feedback group received their

formative assessment of performance after completion of the first interview session with the

AUDIO FEEDBACK 39

standardized patient by email with an attached MP3 audio recording of spoken feedback given in

a segment ranging 2 to 3 minutes. The students in the control written feedback group received

their formative assessment of performance after the completion of the first interview session with

the standardized patient by email with an attached copy of the BECCI tool with the written

ratings. The feedback was based on demonstrated behaviors observed when the evaluator,

blinded to treatment group, listened to the auditory recording of interview one. The BECCI tool

was utilized to outline the delivery of feedback on MI behaviors exhibited by the participant

during the first practice session. Examples of behaviors assessed included but were not limited

to demonstrating use of open-ended questions during motivational interviewing and use of

positive reflective statements about the challenges to behavior change the patient faces.

Treatment fidelity. The fidelity of the feedback treatment was confirmed by

97.5 % of the participants acknowledging through self-report of having reviewed the feedback

sent through email. After completion of the formal study protocol some of the participants

informally offered anecdotal commentary, however no a priori plan was developed to

systematically gather and assess qualitative feedback in this study so no analysis of these data

was conducted.

Post intervention surveys. The MIKAT post-test was given at the completion of the

second interview session. This allowed for evaluation of knowledge about motivational

interviewing and provided a score to be utilized in comparison with the baseline pretest.

The total time for completion of the study ranged from three to four months. All participants

were allotted five weeks to complete the internet-based education and the interview sessions

were subsequently scheduled at a convenient time for the participants. Adult learning principles

AUDIO FEEDBACK 40

as well as recognition for competing demands of a nursing program were taken into

consideration in the allotted time for participants to complete the study.

Data Analysis

Analysis of the data was performed using the program Statistical Package for Social

Sciences (SPSS) version 23 (IBM, 2015). The demographic characteristics, MIKAT and BECCI

scores were described with means and standard deviations for continuous variables and relative

frequencies for categorical variables. Fisher’s Exact tests were conducted to check for

associations between group assignments and categorical demographic characteristics to assess

for potential confounders. Distribution of MIKAT and BECCI Scores were evaluated with

histograms, parametric and nonparametric tests to assess assumptions of normality and

homogeneity of variance. Based on the assumptions of normality tests, inferential analyses

included an independent sample t-test or a Mann Whitney U test performed for between group

differences for each of the tools to evaluate the experimental condition of whether audio versus

written feedback differentially influenced change in MI knowledge and skills. Paired t-tests were

conducted on each of the tools for assessment of within groups mean change of total summary

scores to evaluate effectiveness of the educational modules.

Human Participants Protection

Approval to conduct the study was obtained from the IRB of the college. An explanation

of the study was provided to the participants and those who agreed to take part in the study were

asked to sign an informed consent (Appendix E). Participants were informed that there were no

foreseeable risks associated with participating in the study and that all data collected was

confidential and de-identified. Participants were also informed that those who wish to withdraw

from the study could do so at any time without any impact or risks to grades and academic

AUDIO FEEDBACK 41

standing. Data were maintained in password protected drives and/or locked offices of the

researcher. This researcher prevented coercion by not recruiting students in my classes so

participation could not influence course grades. Participants did receive a ten-dollar gift card at

the conclusion of the study and two participants were randomly selected and each received a

fifty-dollar gift card.

AUDIO FEEDBACK 42

Chapter 4

Results

The following chapter presents the results obtained from the statistical analyses

conducted in this study. The purpose of this study was to evaluate the effect of audio feedback

compared to the written feedback on the development of motivational interviewing knowledge

and skills using a standardized patient case among undergraduate baccalaureate nursing students

addressing behavioral health issues. Relative frequencies of demographics of the sample are

presented in tables (See Appendix G) for the experimental and the control groups. Results from

inferential statistical analyses of changes in outcome measures are described and displayed in

tables (See Appendix G).

Participants

A convenience sample of 40 undergraduate nursing students from the nursing

baccalaureate programs of a suburban college participated in the study. Participants were

randomly assigned and equally distributed between groups. The experimental and control groups

each had 20 participants. Refer to Figure 1 on page 29 CONSORT template diagram.

All students had completed the medical surgical nursing adult I course of the nursing

program curriculum. Thirty-four participants were from the traditional undergraduate program

and an additional six participants were from undergraduate accelerated program. As expected,

participants were mainly White females with median age of 21 years. Refer to Table 4.1 for

more details. No differences in demographics were observed between the groups - see Table 4.2

for between group tests that assessed the potential confounding. (See Appendix G).

AUDIO FEEDBACK 43

Intervention Fidelity

All of the forty students completed the online training. Completion time for finishing the

online training modules ranged from two weeks to two months. The loss to follow up occurred at

the point immediately following signing the informed consent. Forty-nine participants consented

to participate in the study and nine students declined to participate in the module training

resulting in their attrition. All forty students who completed the modules responded to the

treatment fidelity question and 97.5 % reported opening the feedback. This research did not

collect learner reactions to the online educational modules.

Knowledge

The MIKAT scores ranged from 6 to 16, with a mean of 10.65 and a median of 10.0

however there were significant differences at baseline between treatment groups (See Table 4.3

Appendix G). The mean pre-MIKAT scores were significantly higher in the control (written

feedback) group 11.4 (2.037) compared to the experimental (audio feedback) group 9.90 (2.292:

M = 1.5, t (37.382) = 2.188, p = .035.

Because of the between group difference in scores at baseline, only the mean of individual

pre/post differences were compared between treatment groups to assess effectiveness of the

intervention.

MIKAT post scores ranged from 8 to 19 with no significant difference between groups

after educational initiative. A summary of the post MIKAT scores for the groups is also shown in

Table 4.3. The mean and median of the experimental group were M = 12.2 and Mdn = 12.5

respectfully. The corresponding scores for the control group were M = 12.8 and Mdn = 12. The

assumption of normality test was not met between the groups and a Mann Whitney U test was

AUDIO FEEDBACK 44

conducted. The p-value for the Mann Whitney, p = .640 indicated no significant difference in the

median post MIKAT scores between the experimental and control groups.

Between group analyses. The mean of individual pre-post differences in MIKAT scores

were the primary outcome for testing whether audio versus written formative feedback

influences MI knowledge and skills. The mean pre-post difference MIKAT scores were higher in

the experiment group M = 2.30 (2.296) compared to the control group M = 1.40 (2.722), p = .266

however this difference did not reach statistical significance.

Within group analyses. All students regardless of group demonstrated significantly

greater MIKAT mean summary scores after the education (post=12.5 ± 2.298 compared to pre-

10.7 ± 2.271) with a mean (SD) difference of 1.85 (2.53): paired t-test revealed t (39) = 4.63, p <

.0001. The calculated effect size was medium (d = 0.73). Table 4.5 (Appendix G).

Skills Development

The BECCI scores ranged from 9 to 44 with a mean of 23.83 and a median of 24.5 with

no significant differences at baseline between treatment groups, (See Table 4.4 Appendix G).

The mean pre-BECCI scores were higher in the control group M = 24.1 (8.252) compared to the

experimental group M = 23.6 (10.329).

The BECCI post scores ranged from 10 to 44 with no significant difference between groups

after educational initiative. A summary of the post BECCI scores for the groups is also shown in

Table 4.4 The mean and median of the experimental group were M = 27.7 and Mdn = 30

respectfully. The corresponding scores for the control group were M = 31.9 and Mdn = 34.5. The

assumption of normality test was not met between the groups and a Mann Whitney U test was

conducted. The p-value for the Mann Whitney, p = .114 indicated no significant difference in the

median post BECCI scores between the experimental and control groups.

AUDIO FEEDBACK 45

Between group analyses. The mean of individual pre-post differences in BECCI scores

were the primary outcome for testing whether audio versus written formative feedback

influences MI knowledge and skills. The mean pre-post difference BECCI scores were higher in

the control group M = 7.80 (11.687) compared to the experimental group M = 4.15 (11.431) ,

however this difference did not reach statistical significance.

Within group analyses. All students regardless of group demonstrated significantly

greater BECCI mean summary scores after the education (post =29.80 ± 8.742 compared to pre-

23.83 ± 9.232) with a mean (SD) difference of 5.98 (11.56): paired t-test revealed - t (39) =

3.269, p < .002. The calculated effect size was medium (d = 0.52). Table 4.5 (Appendix G).

Practitioner speaking time proportions. Speaking time of the participants conducting

the MI was categorized into three groups and displayed by relative frequency in Table 4.6 (See

Appendix G). The majority spoke about 50% of the time with smaller portions monopolizing the

conversation and even fewer listening more than speaking. The speaking time reduced from pre

to post interviews for speaking less than half the time (pre: 17.5%, versus post: 15.0%, and

speaking more than half the time (pre: 32.2%, versus post: 22.5%). For practitioner speaking

about half the time these figures show a slight increase (pre: 50.0%, versus post 62.5%).

Summary

The data in this study were analyzed using descriptive statistics and inferential analysis.

There was a statistically significant differences at baseline on the MIKAT between the written

feedback group and the audio feedback group therefore individual pre/post difference scores

were compared between groups. There were no statistically significant differences in MIKAT or

BECCI scores after the intervention. There were statistically significant improvements in

knowledge and skills within each group for knowledge and skills reflecting the effectiveness of

AUDIO FEEDBACK 46

the educational modules with no additional influence of the type of audio feedback. The

calculated pre/post within group effect sizes are moderate for the knowledge test and moderate

for the skills assessment. The study interventions were successfully implemented and completed

with forty participants within three to four months.

AUDIO FEEDBACK 47

Chapter 5

Discussion

To effectively prepare students for a career in nursing, health professions education

programs should offer both formative assessments and feedback to the learner which allows for

introspection and development of communication knowledge and skills. The primary aim of this

study was to explore possible differences in the development of MI knowledge and skills among

undergraduate nursing students using audio feedback compared to written feedback. The study

did demonstrate significant increases in MI knowledge and skills within groups after the

educational modules but no additional significant benefit was observed from the type of

feedback provided on the development of communication knowledge and motivational

interviewing skills of undergraduate nursing students addressing behavioral health issues during

standardized patient encounters. Plausible reasons for this include potential flaws in the study

namely difficulty with recruitment with resulting low sample size and failure to meet power

required to detect difference or it may be that there was truly no difference in using a subtle

feedback intervention with one strong educational endeavor. A helpful addition to data collection

would have been to collect perceptions about feedback and have participants rate the value of the

type of feedback they received. The significant results within groups for change in mean scores

for knowledge and skills do provide evidence of the effectiveness of the online modules and

cannot be separated from the potential utility of including audio feedback and written feedback

as a component part of formative assessment in the development of communication knowledge

and motivational interviewing skills in undergraduate nursing students. There is a noted value of

allowing the learner to reflect on formative assessments in education. Maybe without either of

the forms of feedback the education would not have been as effective within groups.

AUDIO FEEDBACK 48

There was a statistically significant difference found between the written feedback group

and the audio feedback group mean pretest scores on the baseline MIKAT knowledge

assessment, demonstrating a p = .035. The finding suggests that at baseline participants in the

written feedback group demonstrated a positive trend for correctly answering the questions posed

on the knowledge test on motivational interviewing prior to any intervention. As a potential

threat to internal validity this was avoided by analyzing the change in scores which adjusted for

differences in baseline knowledge and strengthened the precision of the estimates.

The efforts to ensure rigorous methodology and recruitment efforts in this study did not

yield the desired sample size. The recruitment for the study expanded over a period of eleven

months and encountered barriers included competing academic required work, student

perception of increased effort that participation in the study would require and lack of interest in

participating in an online educational intervention not related to academic grade attainment. The

study demonstrated no statistically significant differences between the audio feedback

(experimental) group and the written feedback (control) group. Both treatment groups did

demonstrate improvement in knowledge and skills scores and the within groups achieved

significant improvement in knowledge and skills. The effect size for knowledge and skills within

groups was medium from pre to post intervention. There are no studies that reported utilization

or effectiveness of the online modules and these results are a contribution to the validity of the

modules to effect change in knowledge and skills and could open new areas of future research.

Although this study did not detect significant differences between experimental groups the

current literature highlights that the use of audio feedback delivered with a clear formative intent

to develop student knowledge and ability to apply the feedback in subsequent work is valued by

the learner as an instructional strategy (Carruthers, McCarron, Bolan, Devine, McMahone-

AUDIO FEEDBACK 49

Beattie & Burns, 2015; Hennessey & Forrester, 2014; Price et al., 2010). The current study did

not collect data on the learners’ perceived value and this is an area to include in future research.

Current literature notes that audio feedback done with single group pre-post training

provides learners with opportunity to enhance development in knowledge and skills (Harrison et

al, 2015; Ice et al., 2007; Munro & Hollingworth, 2014; Parkes & Fletcher, 2017 Race &

Williams, 2018). This study done with two groups may be one possible reason for the present

study outcomes. A recent study done by Morris and Chikwa (2016) investigated audio and

written feedback, learner preference and possible impact on student academic performance. The

study reported that type of feedback did not impact student academic performance in later

assignments. The study design did include randomization of participants and feedback on two

assignments but was not powered sufficiently to detect differences (Morris & Chikwa, 2016).

This highlights the need for comparison of feedback to be done over a longer academic timeline

and not just one or two assignments and the need to address recruitment issues in educational

research. A future study design should employ randomization of types of feedback and the use of

feedback throughout an academic semester over multiple assignments. The data collection

should also include quantitative value assessment for type of feedback coupled with qualitative

data, and analysis of impact on grades.

A recent study done by Race and Williams (2018) explored student perceptions of and

levels of satisfaction with digital auditory feedback in a clinical nursing and research theory

course. Students reported perceptions of audio feedback as useful in giving detailed feedback for

performance improvement in terms of increasing confidence in clinical coursework and lowering

levels of aggravation in finishing medical surgical course work. In the current study, an

experimental design was utilized to attempt to expand the literature on this topic by quantifying

AUDIO FEEDBACK 50

the relationship between audio feedback and knowledge and skill development and exploring

cause and effect relationships. Further research on larger and more diverse groups with the

inclusion of more sensitive outcome measures such as student reactions and perceptions are

needed to further explore the potential impact of audio feedback on undergraduate nursing

student educational development.

Instrument Reliability

The Cronbach’s alpha of 0.94 calculated for BECCI tool is consistent with previous

findings for graduate health providers and extends the reliability to a population of baccalaureate

nursing students. The close means in the MIKAT post-test scores and the BECCI post scores

support the use of both audio feedback and the written feedback for use in provision of formative

assessment in the development of motivational knowledge skills in undergraduate nursing

students. The educational modules were effective in achieving increases in knowledge and skills

within groups where both audio and written feedback seem to produce similar results. However,

the study would need to be repeated with a larger population to ensure that the null findings

between groups were related to the intervention and not the loss of power that resulted from poor

recruitment.

This study utilized clear formative feedback based on a reliable instrument that evaluated

the observable motivational interviewing skills prior to and after the feedback intervention. The

intervention rigor was maintained utilizing the language in the BECCI tool for the written

feedback and the audio MP3 feedback provided. Treatment fidelity was confirmed through self-

report of participants with 97.5% compliance in reviewing feedback sent through email.

The MIKAT tool reliability in this study, with a sample of undergraduate nursing

students in a baccalaureate program, demonstrated much lower reliability (Cronbach’s a of 0.48)

AUDIO FEEDBACK 51

than previous research describing a Cronbach’s a of 0.84 among correctional facility staff

(Doran, Hohman & Koutsenok, 2011). Plausible reasons for the lower reliability in this score

may be that the desired sample size for power was not reached or that the tool was not validated

first among nursing students. Failure to assess reliability among nursing students may have led to

imprecision which makes it difficult to detect differences between groups even if they do exist.

Future research should explore ways to improve the performance of this tool among nursing

students.

The MIKAT tool is noted to have sensitivity to effects of training producing significant

improvement in scores consistently among correctional facility worker, rehabilitation focused

employment case manager, healthcare worker, and staff of child/youth family services

populations (Dear, 2014; Doran et al., 2011; Edwards et al., 2015 & Manthey, 2013). Dear

(2014) reported significant increases in mean scores from pre-training to post-training among

staff of child/youth family services noting an eighty-five percent power, and observing a medium

effect size (d = .50). In this study among undergraduate nursing students the within groups

paired differences did demonstrate increase in knowledge and skills with a medium effect size (d

= .73). A difference in the mean scores on the MIKAT and between pre- and post-intervention

for the audio feedback group and written feedback group were M =2.30, SD 2.299 and M =1.40,

SD 2.722 respectively. These results are close to the mean difference observed in Dear (2014)

who demonstrated MIKAT pre and post differences of 2.83 with a standard deviation of 3.06.

The present study results support that motivational interviewing knowledge and skill

development can be delivered through educational interventions such as training and engagement

in practicing the principles of MI (Chang et al., 2015; Magill et al., 2017; Stoffers & Hatler,

2017).

AUDIO FEEDBACK 52

Strengths and Limitations

A limitation of this research study is the small sample size and recruitment from only one

institution which limits generalizability. The recruitment efforts for this study extended over

eleven months and possibly provided opportunity for participants to share aspects of the study

protocol. Another limitation was the poor performance of the MIKAT tool in this population of

undergraduate nursing students. The strengths of this study include the rigor of the experimental

design methodology and reliability of the BECCI tool for measurement of motivational

interviewing skills. The stratified randomization process assisted in controlling for most threats

to internal validity including selection bias, information bias and the influence of potential

confounding variables.

Future Educational Research and Recommendations

Findings from this research endeavor add to the body of knowledge on use of audio

feedback and written feedback for use in development of MI knowledge and skills although

limited in generalizability. Audio feedback is a venue valued by the learner and is demonstrating

an impact on development of communication knowledge and motivational interviewing skills

across many healthcare student and providers areas. The research methodology employed in this

study is strong and provides opportunity for future research on audio feedback versus written

feedback use. Recommendations for future research inquiry include:

1. Conducting research with a sample size recruited from participants in different settings in

the northeast employing randomization and methods used in this study to increase

generalizability of findings.

AUDIO FEEDBACK 53

2. Employing use of taped video feedback using simulation standardized patients in

comparison with narrative written feedback to explore development of MI knowledge and

skills in nursing student population.

3. Conducting study with a larger sample size across multiple baccalaureate nursing

programs for assessment of audio feedback with increased narrative of ways to improve (in

place of just instrument ratings) versus written feedback employing randomization and

collecting qualitative data on student perceptions and outcome measures of ratings for

increased sense of belonging related to the different types of feedback.

Conclusion

In this research study, findings revealed that audio feedback and written feedback are

comparable ways to deliver formative assessment to develop communication knowledge and

motivational interviewing skills in undergraduate baccalaureate nursing students within the

context of addressing behavioral health issues. These venues for feedback can offer ways to

provide learners with insight on their performance during standardized patient encounters

allowing for self-directed integration into their knowledge base. Providing clear and concise

feedback on performance can enhance the application of motivational interviewing knowledge to

opportunities to communicate more effectively with patients presenting with behavioral health

problems. This aligns with the overall framework of constructivism. As constructivism

articulates that individuals construct their understanding and knowledge of the world through

experiencing things and integrating new knowledge based on their experiences. The

constructivist perspective on learning is one that is built on the premise that integration of new

knowledge is combined with what is already known. The scaffolding of knowledge is done by

utilizing presented information and creating the context in which the information is broken down

AUDIO FEEDBACK 54

to allow growth in the knowledge base, understanding, and attitudes that allow application of

skills in practice at the level of expertise or competency standard (Cunningham & Duffy, 1996;

Iwasiw & Goldenberg, 2015). In this study the students are given opportunity through education

about MI to build upon their foundational knowledge of communication and integrate their

understanding of MI principles into their understanding. Integration of knowledge allowed

opportunity for application of MI skills in the interview sessions.

This study conducted with undergraduate nursing students adds to the body of research on

use of audio feedback and written feedback to investigate the effectiveness of this formative

assessment with learners in the fields of education, early childhood studies, diagnostic imaging

students, online learners in an asynchronous teaching strategies course, medical students,

community pre licensure nursing students in the United Kingdom, a cohort of master’s level

nursing students in the clinical setting, and junior- level nursing students enrolled in a medical

surgical nursing course or a research course (Bourgault, Mundy & Joshua, 2013; Carruthers,

McCarron, Bolan, Devine, McMahon-Beattie & Burns, 2015; Cavanaugh & Song, 2014; Gould

& Day, 2013; Green, 2015; Harrison, Molyneux, Blackwell & Wass 2014; Ice, Curtis, Phillips &

Wells, 2007; Race & Williams, 2018). The current study supports the use of either audio

feedback and or written feedback coupled with education about MI principles with

undergraduate baccalaureate nursing students’ development of communication knowledge and

motivational interviewing skills by demonstrating a strong within groups increase in overall

scores from pre to post interventions. As a venue for development of communication knowledge

and motivational interviewing skills the study design provided an opportunity for learners to

practice the foundations of this communication technique. The patient centered care initiative to

include patients as partners in their health care aligns with the premises of MI knowledge and

AUDIO FEEDBACK 55

skill development. MI literature concludes that use of open-ended questions, affirming

statements, reflective listening and summarization statements assists healthcare professionals in

creating a rapport with clients that is collaborative and supportive in nature (Miller, 1983; Miller

& Rollnick, 1991; Miller& Rollnick 2013).

The literature available on the effects of audio feedback has mainly been conducted

qualitatively describing common themes like the perception that audio feedback as more

personalized than written feedback, that detail in audio feedback is increased, and in the sense

that feedback is more easily comprehended and useful for learning (Ice, Curtis, Phillips & Wells,

2007; Lunt & Curran, 2010; Nemec & Dintzer, 2016; Sipple, 2007).The current study

methodology presents the first attempt to conduct quantitative comparison between the use of

MP3 audio feedback versus the written feedback technique. Although both methods of feedback

were associated with a post intervention increase in participant knowledge and skill scores, the

effect of the online educational modules cannot be teased apart from the potential effect of the

intervention especially in light of small sample sizes reducing the power to detect subtle

differences. Although these findings suggest development of communication and knowledge

may be affected by feedback, there remains a dearth in the literature that provides rigorous

inquiry into quantitative assessment among large sample sizes of different health professions

population.

Effective communication is foundational to successful interaction between healthcare

professional and clients. The availability for health professionals to engage in practice of

motivational interviewing skills provides opportunity to enhance communication knowledge that

targets client self- directed participation in patient centered care. Behavioral health patient

populations are rising component of the healthcare arena and engaging them with empathy,

AUDIO FEEDBACK 56

openness, and sensitivity embodied in MI communication techniques can only assist in

attainment of optimal patient centered outcomes. Both audio and written feedback coupled with

education provides a technique to engage health professions learners in the process for

development of communication knowledge and motivational interviewing skills.

AUDIO FEEDBACK 57

References

Addiction Technology Transfer Center (ATTC) Network (2018). Healthknolwedge.org

Retrieved from: https://attcnetwork.org/projects/blendinginitiative.aspx

Amodeo, M., Lundgren, L., Fernanda Beltrame, C., Chassler, D., Cohen, A., & D'Ippolito, M.

(2013). Facilitating factors in implementing four evidence-based practices: Reports from

addiction treatment staff. Substance Use & Misuse, 48(8), 600-611.

http://dx.doi.org/10.3109/10826084.2013.794838

Arthur, D. (1999). Assessing nursing students' basic communication and interviewing skills: the

development and testing of a rating scale. Journal of Advanced Nursing, 29(3), 658-665.

http://dx.doi.org/10.1046/j.1365-2648.1999.00934.x

Barr, P. J., Bonasia, K., Verma, K., Dannenberg, M. D., Yi, C., Andrews, E., … Durand, M.-A.

(2018). Audio-/Videorecording Clinic Visits for Patient’s Personal Use in the United

States: Cross-Sectional Survey. Journal of Medical Internet Research, 20(9),

66.Retrieved from: https://doi-org.odin.curry.edu/10.2196/11308

Bartlett, R., Brown, L., Shattell, M., Wright, T., & Lewallen, L. (2013). Harm reduction:

compassionate care of persons with addictions. Medsurg Nursing, 22(6), 349–358.

Bertrand, K., Roy, É., Vaillancourt, É., Vandermeerschen, J., Berbiche, D., & Boivin, J. (2015).

Randomized controlled trial of motivational interviewing for reducing injection risk

behaviours among people who inject drugs. Addiction, 110(5), 832-841.

http://dx.doi.org/10.1111/add.12867

Bishop, P. A. & Herron, R. L. (2015). Use and misuse of likert item responses and other ordinal

measures. International Journal of Exercise Science, 8 (3), 297-302.

AUDIO FEEDBACK 58

Blixen, C. E., Kanuch, S. Perzynski, A. T., Thomas, C., Dawson, N.V. & Sajatovic, M. (2016).

Barriers to self-management of serious mental illness and diabetes. American Journal of

Health Behavior, 40 (2), 194-204, doi: http://dx.doi.org/10.5993/AJHB.40.2.4

Bourgault, A., Mundy, C., & Joshua, T. (2013). Comparison of audio vs. written feedback on

clinical assignments of nursing students. Nursing Education Perspectives, 34(1), 43-46.

http://dx.doi.org/10.5480/1536-5026-34.1.43

Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P. (2017).CONSORT Statement for

Randomized Trials of Nonpharmacologic Treatments: A 2017 Update and a CONSORT

Extension for Nonpharmacologic Trial Abstracts. Annals of Internal Medicine,167(1),

40–7.

Brant, R. (2017). Power/sample size calculator/The University of British Columbia Department

of Statistics Retrieved from: https://www.stat.ubc.ca/~rollin/stats/ssize/b2.html

Caranaugh, A., & Song, L. (2014). Audio feedback versus written feedback: Instructors' and

students' perspectives. MERLOT Journal of Online Learning and Teaching, 10(1), 122-137.

Carr, D. D. (2017). Motivational interviewing supports patient centered-care and

communication. Journal of the New York State Nurses Association, 45 (1), 39-43. Retrieved

from: https://www.nysna.org/sites/default/files/attach/1142/2017/08/2017-08-

01nysnaJournalv45n1.pdf

Carruthers, C., McCarron, B., Bolan, P., Devine, A., McMahon-Beattie, U., & Burns, A. (2014).

‘I like the sound of that’ – an evaluation of providing audio feedback via the virtual learning

environment for summative assessment. Assessment & Evaluation in Higher Education,

40(3), 352-370. Retrieved from: http://dx.doi.org/10.1080/02602938.2014.917145

AUDIO FEEDBACK 59

Centers for Disease Control and Prevention (2012). Mental health and chronic diseases. Issue

Brief No.2 October. Retrieved from:

file:///C:/Users/sacif/Documents/1%201%20spring%2017%20lit%20review/issue-brief-no-

2-mental-health-and-chronic-disease.pdf

Chang, Y., Compton, P., Almeter, P., & Fox, C. (2014). The effect of motivational interviewing

on prescription opioid adherence among older adults with chronic pain. Perspectives in

Psychiatric Care, 51(3), 211-219.

Cucciare, M.A., Ketroser, N., Wilbourne, P., Midboe, A. M., Cronkite R. & Chardos, J. (2012).

Teaching motivational interviewing to primary care staff in the veterans’ health

administration. Journal of General Internal Medicine, 27 (8), 953-961.

Cunningham, D., & Duffy, T. (1996). Constructivism: Implications for the design and delivery of

instruction. Handbook of Research for Educational Communications and Technology (pp.

170-198. Retrieved from http://www.aect.org/edtech/edpdf/07.pdf).

Cuthrell, K., Fogarty, E., Smith, J., & Ledford, C. (2013). Implications of using peer audio

feedback for the college learner: Enhancing instruction. Delta Kappa Gamma Bulletin,

79(4), 13-21.

Dear, E. J. P. (2014). Evaluating motivational interviewing measures of knowledge and skill

using training outcome data (Unpublished master's thesis). University of

Canterbury, Canterbury, New Zealand

Dixon, S. (2015). The pastoral potential of audio feedback: a review of the literature. Pastoral

Care in Education, 33(2), 96-104. Retrieved from:

http://dx.doi.org/10.1080/02643944.2015.1035317

AUDIO FEEDBACK 60

Doran, N., Hohman, M., & Koutsenok, I. (2013). Motivational interviewing training in juvenile

corrections: A comparison of outside experts and internal trainers. Legal and

Criminological Psychology,18,262-273.

Doran, N., Hohman, M., & Koutsenok, I. (2011). Linking basic and advances motivational

interviewing training outcomes for juvenile correctional staff in California. Journal of

Psychoactive Drugs, 43, 19–26.

Drevenhorn, E., Bengtson, A., Nyberg, P., & Kjellgren, K. (2015). Assessment of hypertensive

patients’ self-care agency after counseling training of nurses. Journal of The American

Association of Nurse Practitioners, 27(11), 624-630. Retrieved from:

http://dx.doi.org/10.1002/2327-6924.12222

Edwards, E., Stapleton, P., Williams, K., & Ball, L. (2015). Building skills, knowledge and

confidence in eating and exercise behavior change: Brief motivational interviewing training

for healthcare providers. Patient Education and Counseling, 98(5), 674-676. Retrieved

from: http://dx.doi.org/10.1016/j.pec.2015.02.006

Forest, S. (2005). Learning Styles. In L. Caputi & L. Engelmann, Teaching Nursing: The Art

and Science (pp. 389-406). Glen Ellyn: Illinois: College of DuPage Press.

Gilburt, H., Rose D. & Slade, M. (2008). The importance of relationships in mental healthcare: A

qualitative study of service users’ experiences of psychiatric hospital admission in the

UK. Biomed Central (BMC) Health Services Research, 8 (92), 1-12.

Gould, J., & Day, P. (2013). Hearing you loud and clear: student perspectives of audio feedback

in higher education. Assessment & Evaluation in Higher Education, 38(5), 554-566.

AUDIO FEEDBACK 61

Graham, A. L., Hasking, P., Clarke, D. & Meadows, G. (2015). How people with depression

receive and perceive mental illness information: Findings from the Australian National

Survey of Mental Health and Wellbeing. Community Mental Health Journal, 51, 994-

1001.

Green, D. (2015). AUDIO FEEDBACK: An evaluation of its effectiveness amongst diagnostic

imaging students. Synergy: Imaging & Therapy Practice, 11-16.

Hallgren K. A. (2012). Computing Inter-Rater Reliability for Observational Data: An Overview

and Tutorial. Tutorials in quantitative methods for psychology, 8(1), 23-34.

Harrison, C., Molyneux, A., Blackwell, S., & Wass, V. (2014). How we give personalized audio

feedback after summative OSCEs. Medical Teacher, 37(4), 323-326. Retrieved from:

http://dx.doi.org/10.3109/0142159x.2014.932901

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research,

77(1), 81-112.

Hayes, S. Wolf, C. Labbé, S., Peterson, E. & Murray S. (2017). Primary healthcare provider’s

roles and responsibilities: A qualitative exploration of ‘who does what’ in the treatment

and management of persons affected by obesity. Journal of Communication in

Healthcare, 10(1), 47-54.

Healthcare provider. BusinessDictionary.com. Retrieved January 23, 2019, from

BusinessDictionary.com website: http://www.businessdictionary.com/definition/health-care-

provider.html

Hennessy, C., & Forrester, G. (2013). Developing a framework for effective audio feedback: a

case study. Assessment & Evaluation in Higher Education, 39(7), 777-789. Retrieved from:

http://dx.doi.org/10.1080/02602938.2013.870530

AUDIO FEEDBACK 62

Hirdle, J., & Vaughn, T. (2016). Exploring the impact of motivational interviewing training for

qualified health visitors. Community Practitioner; London, 89(7), 38-42.

Holleran Steiker, L. (2009). Motivational interviewing and addiction: An interview with Melinda

Hohman. Journal of Social Work Practice in Addictions, 9, 236-239.

Honebein, P. (1996). Seven goals for the design of constructivist learning environments. In B.

Wilson, Constructivist Learning Environments: Case Studies in Instructional Design (pp.

11-24). Englewood Cliffs, CA: Educational Technology Publications.

Howard, L., & Williams, B. (2016). A focused ethnography of baccalaureate nursing students

who are using motivational interviewing. Journal of Nursing Scholarship, 48(5), 472-481.

http://dx.doi.org/10.1111/jnu.12224

IBM Corp. Released 2015. IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM

Corp.

Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007). Using asynchronous audio feedback to

enhance teaching presence and students' sense of community. Journal of Asynchronous

Learning Networks, 11(2), 3-25.

Institute for Healthcare Communication (2018). Impact of communication in healthcare.

Retrieved from: https://healthcarecomm.org/about-us/impact-of-communication-in-

healthcare/

Iwasiw, C., & Goldenberg, D. (2015). Curriculum development in nursing education (3rd ed.).

Burlington, MA: Jones & Bartlett.

Joy, J., Bell, S., & Leukefeld, C. (2015). Beyond SBIRT: Incorporating recovering volunteers

after the intervention. Counselor, 16(6), 44-49.

AUDIO FEEDBACK 63

Kazemi, D., Levine, M., Dmochowski, J., Nies, M., & Sun, L. (2013). Effects of motivational

interviewing intervention on blackouts among college freshmen. Journal of Nursing

Scholarship, Retrieved from: n/a-n/a. http://dx.doi.org/10.1111/jnu.12022

Kee, J. W. Y., Khoo, H. S., Lim, I. & Koh, M. Y. H. (2017). Communication skills in patient-

doctor interactions: Learning from patient complaints. Health Professions Education.

Retrieved from: http://dx.doi.org/10.1016/j.hpe.2017.03.006

Kennedy, D., Apodaca, T., Trowbridge, K., Hafeman, C., Roderick, E., & Modrcin, A. (2016).

Learning motivational interviewing: A pathway to caring and mindful patient encounters.

Journal of Pediatric Nursing, 31(5), 505-510. Retrieved from:

http://dx.doi.org/10.1016/j.pedn.2015.12.011

Killingback, C., Ahmed, O. & Williams, J. (2019). ‘It was all in your voice’- Tertiary student

perceptions of alternative feedback modes (audio, video, podcast, and screencast): A

qualitative literature review. Nurse Education Today, 72, 32-39. Retrieved from:

https://doi.org/10.1016/j.nedt.2018.10.012

Lane, C., Huws-Thomas, M., Hood, K., Rollnick, S. Edwards, K. & Robling, M. (2005).

Measuring adaptations of motivational interviewing: The development and validation of the

behavior change counseling index (BECCI). Patient Education and Counseling, 56 (2), 166-

173.

Lee, W. W., Choi, K. C., Yum, R. W., Doris, S. F., & Chair, S. Y. (2015). Effectiveness of

motivational interviewing on lifestyle modification and health outcomes of clients at risk or

diagnosed with cardiovascular diseases: A systematic review. International Journal of

Nursing Studies, 53, 331-341. doi:10.1016/j.ijnurstu.2015.09.010

AUDIO FEEDBACK 64

Leffingwell, T. R. (2006). Motivational Interviewing Knowledge and Attitudes Test (MIKAT)

for evaluation of training outcomes. MINUET, 13, 10-11, Retrieved from:

http://psychology.okstate.edu/faculty/leffingwell/bcl/resources/reprints/mikat_mintbull.pdf.

Lundahl, B. W.,Kunz, C., Brownell, C., Tollefson, D. & Burke, B. L,. (2010). A meta-analysis of

motivational interviewing: Twenty-five years of empirical studies. Research on Social Work

Practice, 20, 137-160. doi:10.1177/1049731509347850.

Lunt, T., & Curran, J. (2010). ‘Are you listening please?’ The advantages of electronic audio

feedback compared to written feedback. Assessment & Evaluation in Higher Education,

35(7), 759-769. Retrieved from: http://dx.doi.org/10.1080/02602930902977772

Magill, M., Colby, S., Orchowski, L., Murphy, J., Hoadley, A., Brazil, L., & Barnett, N. (2017).

How does brief motivational intervention change heavy drinking and harm among underage

young adult drinkers? Journal of Consulting and Clinical Psychology. Retrieved from:

http://dx.doi.org/10.1037/ccp0000200

Martino, S., Carroll, K., Nich, C., & Rounsaville, B. (2006). A randomized controlled pilot study

of motivational interviewing for patients with psychotic and drug use disorders. Addiction,

101(10), 1479-1492. Retrieved from: http://dx.doi.org/10.1111/j.1360-0443.2006.01554.x

Martino, S., Zimbrean, P., Forray, A., Kaufman, J., Desan, P., & Olmstead, T. et al. (2015). See

One, Do One, Order One: a study protocol for cluster randomized controlled trial testing

three strategies for implementing motivational interviewing on medical inpatient units.

Implementation Science, 10(1). Retrieved from: http://dx.doi.org/10.1186/s13012-015-0327-

9

AUDIO FEEDBACK 65

Mathey, T. (2013). A pilot study of introductory motivational interviewing training for supported

employment case managers. International Journal of Psychosocial Rehabilitation, 18(1),

133-138.

McCarthy, J. (2015). Evaluating written, audio and video feedback in higher education

summative assessment tasks. Issues in Educational Research, 25(2), 153-169.

Memec, E. C. & Dintzner, M. (2016). Comparison of audio versus written feedback on writing

assignments. Currents in Pharmacy Teaching and Learning, 8, 155-159.

Merry, S., & Orsmond, P. (2008). Students’ Attitudes to and Usage of Academic Feedback

Provided Via Audio Files. Bioscience Education, 11(1), 1-11.

Miller, W. (2015). Celebrating Carl Rogers: Motivational interviewing and the person-centered

approach. Motivational Interviewing: Training, Research, Implementation, Practice, 1(3), 4.

Retrieved from: http://dx.doi.org/10.5195/mitrip.2014.54.

Miller, W. (1983). Motivational interviewing with problem drinkers. Behavioral Psychotherapy,

11, 147-172.

Miller, W., & Moyers, T. (2014). The forest and the trees: relational and specific factors in

addiction treatment. Addiction, 110(3), 401-413. Retrieved from:

http://dx.doi.org/10.1111/add.12693

Miller, W., & Rollnick, S. (2013). Motivational interviewing (1st ed.). New York: Guildford

Press.

Miller, W., & Rollnick, S. (1991). Motivational Interviewing: preparing people to change

addictive behaviour. New York: Guilford Press.

AUDIO FEEDBACK 66

Mills, A., Kerschbaum, W. E., Richards, P. S. Czarnecki, G. A., Kinney, J. S. & Gwozdek, A. E.

(2017). Dental hygiene students’ percetions of importance and confidence in applying

motivaitonal interviewing during patient care. The Journal of Dental Hygiene, 91 (1), 15-23.

Morris, C., & Chikwa, G. (2016). Audio versus written feedback: Exploring learners’ preference

and the impact of feedback format on students’ academic performance. Active Learning in

Higher Education, 17(2), 125-137. Retrieved from:

http://dx.doi.org/10.1177/1469787416637482

Moyers, T., Martin, T., Catley, D., Harris, K. J., & Ahluwalia, J. S. (2003). Assessing the

integrity of motivational interviewing interventions: Reliability of the Motivational

Interviewing Skills Code. Behavioural and Cognitive Psychotherapy, 31(2), 177–184.

Munro, W., & Hollingworth, L. (2014). Audio feedback to physiotherapy students for viva voce:

how effective is ‘the living voice’? Assessment & Evaluation in Higher Education, 39(7),

865-878.

National Institute on Drug Abuse (2009). The Science of Treatment: Dissemination of Research-

based Drug Addiction Treatment Findings. Retrieved from:

http://www.attcnetwork.org/projects/blendinginitiative.aspx

National Institute on Drug Abuse (2017). Advancing addiction science: Trends and statistics.

Retrieved from: https://www.drugabuse.gov/related-topics/trends-statistics#supplemental-

references-for-economic-costs

National Institute of Mental Health (2018). Chronic illness and mental health. Retrieved from:

https://www.nimh.nih.gov/health/publications/chronic-illness-mental-health/index.shtml

AUDIO FEEDBACK 67

Nesbitt, B., Murray, D., & Mensink, A. (2013). Teaching motivational interviewing to nurse

practitioner students: A pilot study. Journal of The American Association of Nurse

Practitioners, 26(3), 131-135. Retrieved from: http://dx.doi.org/10.1002/2327-6924.12041

Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a

model and seven principles of good feedback practice. Studies in Higher Education, 31(2),

199-218. Retrieved from: http://dx.doi.org/10.1080/03075070600572090

Noordman, J., van der Weijden, T., & van Dulmen, S. (2014). Effects of video-feedback on the

communication, clinical competence and motivational interviewing skills of practice nurses:

a pretest posttest control group study. Journal of Advanced Nursing, 70(10), 2272-2283.

Retrieved from: http://dx.doi.org/10.1111/jan.12376

Östlund, A., Wadensten, B., Häggström, E., & Kristofferzon, M. (2013). District nurses’ and

registered nurses’ training in and use of motivational interviewing in primary care settings.

Journal of Clinical Nursing, 23(15-16), 2284-2294. Retrieved from:

http://dx.doi.org/10.1111/jocn.12509

Parkes, M. & Fletcher, P. (2017). A longitudinal, quantitative study of student attitudes towards

audio feedback for assessment. Assessment & Evaluation in Higher Education, 42 (7),

1046-1053. Retrieved from: https://doi.org/10.1080/02602938.2016.1224810

Petrova, T., Kavookjian, J., Madson, M., Dagley, J., Shannon, D., & McDonough, S. (2015).

Motivational interviewing skills in healthcare encounters (MISHCE): Development and

psychometric testing of an assessment tool. Research in Social and Administrative

Pharmacy, 11(5), 696-707. Retrieved from:

http://dx.doi.org/10.1016/j.sapharm.2014.12.001

AUDIO FEEDBACK 68

Piatt, J. A. & Chiasson, L. (2016). Impact of motivational interviewing on readiness for change

and self-care behaviors of an assisted living resident with type I diabetes. Therapeutic

Recreation Journal, 3, 228-241. Retrieved from: http://dx.doi.org/10.18666/TRJ-2016-V50-

I3-6540

Price, M., Handley, K., Millar, J., & O'Donovan, B. (2010). Feedback: all that effort, but what is

the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289. Retrieved from:

http://dx.doi.org/10.1080/02602930903541007

Race, D. A. & Williams, M. G. (2018). Digital audio feedback techonolgy: Nursing students’

perspectives. Computers, Informatics, Nursing, 36 (2), 63-67.

Rasi, P. & Vuojärvi, H. (2017). Toward personal and emotional connectivity in mobile higher

education through asynchronous formative audio feedback. British Journal of Educational

Technology, 00, 1-13. doi:10.1111/bjet.12587

Rockinson-Szapkiw, A. (2012). Should online doctoral instructors adopt audio feedback as an

instructional strategy? Preliminary evidence. International Journal of Doctoral Studies, 7,

243-258.

Schumacher, J. A., Williams, D. C., Burke, R.S., Epler, A. J., Simon, P. & Coffey, S. F. (2018).

Competency-based supervision in motivational interviewing for advanced psychology

trainees: Targeting an a priori benchmark. Training and Education in Professional

Psychology. Advance online publication. Retrieved from:

http://dx.doi.org/10.1037/tep0000177

Sellman, D. (2011). The 10 most important things known about addiction. FOCUS, 9(1), 99-106.

Retrieved from: http://dx.doi.org/10.1176/foc.9.1.foc99

AUDIO FEEDBACK 69

Seth, P., Scholl, L., Rudd, & Bacon, S. (2018). Overdose dealths involving opioids, cocaine, and

psychostimulants-United States, 2015-2016. Morbidity and Mortality Weekly Report

(MMWR), 67, 349-358. Retrieved from: http://dx.doi.org/10.15585/mmwr.mm6712a1

Shannon, R., Donovan-Hall, M. & Bruton, A. (2017). Motivational interviewing in respiratory

therapy: What do clinicians need to make it part of routine care? A qualitative study. Plos

One, 12 (10), 1-15. Retrieved from: https://doi.org/10.1371/journal.pone.0187335

Simon, P., & Ward, N. L. (2014). An evaluation of training for lay providers in the use of

Motivational Interviewing to promote academic achievement among urban youth. Advances

in School Mental Health Promotion, 7(4), 255–270. Retrieved from:

http://doi.org/10.1080/1754730X.2014.949062

Sipple, S. (2007). Ideas in practice: Developmental writers' attitudes toward audio and written

feedback. Journal of Developmental Education, 30(3), 22-31.

Spollen, J. J., Thrush, C. R., Mui, D., Woods, M. B., Tariq, S. G. & Hicks, E. (2010). A

randomized controlled trial of behavior change counseling education for medical students.

Medical Teacher, 32, (4), e170-e177. doi: 10.3109/01431590903514614

Staiger, T., Waldmann, T., Rűsch, N. & Krumm, S. (2017). Barriers and facilitators of help-

seeking among unemployed persons with mental health problems: A qualitative study.

Biomed Central (BMC) Health Services Research, 17 (39), 1-9.

Stoffers, P. J. & Hatler, C. (2017). Increasing nurse confidence in patient teaching using

motivational interviewing. Journal for Nurses in Professional Development, 33 (4), 189-

195.

AUDIO FEEDBACK 70

Substance Abuse and Mental Health Services Administration (SAMHSA). (2013). Drug abuse

warning network, 2011: National estimates of drug-related emergency department visits.

Retrieved from:

https://www.samhsa.gov/data/sites/default/files/DAWN2k11ED/DAWN2k11ED/DAWN

2k11ED.pdf

Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Behavioral

health treatments and services. Retrieved from: https://www.samhsa.gov/treatment

Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Leading

Change 2.0: Advancing the behavioral health of the nation 2015-2018.Retrieved from:

https://www.samhsa.gov/workforce/development-competencies-capacities

The Joint Commission (2016). Sentinel event type by year for 2015. Joint Commission

Perspectives (April). Retrieved from: http://info.jcrinc.com/rs/494-MTZ-

066/images/Sentinel39.pdf?ref=EMHAPWeekly&mkt_tok=eyJpIjoiWWpaa05qazBZakk

yTVdNNSIsInQiOiJPdVdqZ0kzTzd1ZnM3VDdDeGhoR2t4Z3Izcm5IdzFcL3djd1hwOE

VENHJxZHh5OE1WRmI5OFgxNVErZHFkS2tCMk5ySFdjMXltaWs4WjZcL2EzU2NP

ZDI0ME9paTNKUGN4bG90K21KeG5tcWc4PSJ9

Tsai, J., Klee, A., Shea, N., Lawless, M., Payne, K. A., Goggin, E., . . . Smith, K. (2017).

Training peer specialists with mental illness in motivational interviewing: A pilot study.

Psychiatric Rehabilitation Journal, 40 (4), 354-360. Retrieved from:

http://dx.doi.org/10.1037/prj0000226

Urbaniak, G. C., & Plous, S. (2013). Research Randomizer (Version 4.0) [Computer software].

Retrieved from: http://www.randomizer.org/

AUDIO FEEDBACK 71

Weller, J., Boyd, M. & Cumin, D. (2014). Teams, tribes and patient safety: Overcoming barriers

to effective communication in healthcare. Postgraduate Medical Journal, 90, 149-154.

doi: 10.1136/postgradmedj-2012-131168

Willits, F. K., Theodori, G.L. & Luloff, A. E. (2016). Another look at likert scales. Journal of

Rural Social Sciences, 31 (3), 126-139.

World Health Organization. (2010). Framework for action on interprofessional education and

collaborative practice. Geneva: Health Professions Networks Nursing and Midwifery

office. Humans Resources for Health. Retrieved from:

http://apps.who.int/iris/bitstream/10665/70185/1/WHO_HRH_HPN_10.3_eng.pdf

AUDIO FEEDBACK 72

Appendix A: Motivational Interviewing Knowledge and Attitudes Test MIKAT1

AUDIO FEEDBACK 73

AUDIO FEEDBACK 74

__________________________

1From “Motivational Interviewing Knowledge and Attitudes Test for Evaluating Training

Outcomes” in MINUET by T. R. Leffingwell (2006) p. 10–11. Reprinted with permission.

AUDIO FEEDBACK 75

Appendix B: Behavioral Change Counseling Index (BECCI)

AUDIO FEEDBACK 76

Appendix C: Permission to use the Behavioral Change Counseling Index (BECCI)

Claire Lane <clairelane1978@gmail.com>

Reply| Mon 4/17, 12:05 PM Cifrino, Sheryl

Dear Sheryl, Thanks for your enquiry. You don't need my permission to use the BECCI - it is in the public domain and free for anyone to use so please go ahead. I hope that helps. Best regards, Claire

AUDIO FEEDBACK 77

Appendix D: Demographic Form

Please fill in or check the appropriate response for each question.

1. Age in years ____________

2. Gender Male Female

3. What is your race (select all that apply)?

American Indian or Alaska

Asian

Black or African American

Native Hawaiian or Other Pacific Islander

White

Prefer not to answer

4. Please indicate which undergraduate baccalaureate nursing program you are currently

attending:

Accelerated Nursing Program.

Traditional Nursing Program.

Thank you for taking the time to complete this questionnaire. Your contribution to this research

effort is greatly appreciated.

AUDIO FEEDBACK 78

Appendix E: Informed Consent

AUDIO FEEDBACK 79

AUDIO FEEDBACK 80

AUDIO FEEDBACK 81

Appendix F: IRB Approval Letters

AUDIO FEEDBACK 82

Researcher’s Name: Supervising Faculty Sponsor: Dr. Donna Glynn, Regis College Ph.D. Student: Sheryl Cifrino, Professor of Nursing, Curry College Project Title: Exploring the Effects of Audio Feedback on the Communication Knowledge and Motivational Interviewing Skills of Undergraduate Nursing Students Addressing Behavioral Issues Date of Submission: Continuance Submission: October 29, 2018 Original Submission: October 16, 2017 FOR USE BY THE IRB:

Exempt Research: Expedited Research: X Signature of IRB member: J. Balboni, Ph.D., Chair

Date of IRB Review: 10/29/2018

Category of IRB Review: Initial _______ Continuing ___X____

Expedited review ___X_ Full review ______

The Curry College IRB recommends: Approval: Thank you for submitting the annual report, the Simmons IRB documentation, and the additional information via email, regarding the progress of this study. As described in the materials, the continuation of this study has been approved through an Expedited Review process, in accordance with OHRP guidelines, 45 CFR 46. 108 (b).

AUDIO FEEDBACK 83

Appendix G: Results Tables

Table 4.1 Demographics of the Sample

Variables Control Grp.

Experimental

Grp. Total

Gender %

Male

Female

Total

6 (30%)

14 (70%)

20

3 (15%)

17 (85%)

20

9 (23%)

31 (77%)

40 (100%)

Age mean(sd) years

18-26 (%)

27-54 (%)

Total

24.3 (9.5)

17

3

20

21.4 (2.7)

19

1

20

36 (90%)

4 (10%)

40 (100%)

Race %

White

Black/African American

Total

19

1

20

19

1

20

38 (95%)

2 (5%)

40 (100%)

Nsg Program %

Traditional

NonTraditional

Total

17

3

20

17

3

20

34 (85%)

6 (15%)

40 (100%)

Table 4.2 Fisher's Exact Tests

Variables pvalue

Gender 0.45

Program 1.00

Race 1.00

Age 0.61

AUDIO FEEDBACK 84

Table 4.3 Mean (SD) of MIKAT scores between groups

N = 20 in each group (Knowledge)

Experimental Control

M (SD) Mdn M (SD) Mdn MWU t df

p

value

Pre MIKAT Scores 9.9 (2.3) 11.4 (2.0) 2.188 37.382 *0.04a

Post MIKATScores 12.2 (2.5) 12.5 12.8 (2.0) 12 182.5 0.64b

Pre-Post Difference

MIKAT Scoresc

2.3 (2.3) 1.4 (2.7) -1.130 38 0.27a

* p < 0.05

Abbreviations: M= mean, SD= standard deviation, N=sample size, Mdn= median,

MWU = Mann Whitney U test statistic, t= t statistic

Legend: a= independent t-test p-value, b= Mann Whitney U test p-value, c = difference score is

the mean of individual pre-post differences

AUDIO FEEDBACK 85

Table 4.4 Mean (SD) of BECCI scores between groups

N = 20 in each group (Skills)

Experimental Control

M (SD) Mdn M (SD) Mdn MWU t df

p

value

Pre BECCI Scores 23.6 (10.3) 24.1 (8.3) 0.186 38 0.85a

Post BECCI

Scores 27.7 (9.0) 30 31.9 (8.2) 34.5 141 0.11b

Pre-Post Difference BECCI

Scoresc

4.2 (11.4) 7.8 (11.7) 0.999 38 0.32a

Abbreviations: M= mean, SD= standard deviation, N=sample size, Mdn= median,

MWU = Mann Whitney U test statistic, t= t statistic

Legend: a= independent t-test p-value, b= Mann Whitney U test p-value, c = difference score is the

mean of individual pre-post differences

AUDIO FEEDBACK 86

Table 4.5 Results for Total MIKAT Pre-Post Test Scores & Total BECCI Pre-Post Test

Scores within groups

N = 40 Paired T-test

M SD

SE

Mean t df p value

Effect

Size

Pair 1

Total MIKAT

pretest 10.7 2.3 0.35904

Total MIKAT

post test 12.5 2.3 0.36339

Pre-Post

Difference

MIKAT Scoresc

1.9 2.5 0.3996 4.63 39 *0.00a 0.73d

Pair 2

Total BECCI

pretest 23.8 9.2 1.45967

Total BECCI post

test 29.8 8.7 1.38221

Pre-Post

Difference

BECCI Scoresc

5.9 11. 6 1.82767 3.269 39 *0.00a 0.52d

*Denotes within group change had a p < 0 .05

Abbreviations: M= mean, SD= standard deviation, SE = standard error, N= sample size, test statistic,

t= t statistic df= degrees of freedom

Legend: a= paired samples t-test p-value, c = difference score is the mean of group pre-post differences,

d= Cohen’s D is a within group effect size estimate.

AUDIO FEEDBACK 87

Table 4.6 Practitioner Speaking Time BECCI pre/post N= 20 in each group

Variables

Control

Group

Experimental

Group Percent

Practitioner Speaks: Pre

More than half the time 17.5% 15.0% 32.50%

About half the time 27.5% 22.5% 50.0%

Less than half the time 5.0% 12.5% 17.50%

Total 20 20 100%

Practitioner Speaks: Post

More than half the time 17.5% 5.0% 22.50%

About half the time 32.5% 30.0% 62.50%

Less than half the time 0% 15.0% 15.0%

Total 20 20 100%