ww2.lipscomb.edu - prog… · web viewmission statement and program goals. the mission for the...

Graduate Studies in Counseling

Program Evaluation Report

2014-2015 Academic Year

Graduate Department of Psychology, Counseling, and Family Science

Lipscomb University

One University Park DriveNashville, TN 37024-3951

1-800-333-4358 x5906615-966-5906

1

Table of ContentsDescription of Program, Faculty and Curriculum 6

MISSION STATEMENT AND PROGRAM GOALS 6

Program Goals 6

Program Objectives 6

Program Description 7

PROGRAM FACULTY 9

CURRICULUM 9

Courses 10

Systematic Plan for Program Evaluation 10

ACCREDITATION STANDARDS 10

PHILOSOPHY OF PROGRAM EVALUATION 11

EIGHT-POINT STUDENT, FACULTY AND PROGRAM EVALUATION PLAN 15

Application and Interview 15

New Student Orientation 15

First Semester 15

Pre-Practicum Semesters 16

Application to Candidacy 16

Clinical Semesters 16

Final Semester 16

Alumni 16

MEETING PROGRAM GOALS AND OBJECTIVES THROUGH THE EIGHT POINT ASSESSMENT PLAN 16

Program Goals 17

Program Objectives 17

PROGRAM EVALUATION CALENDAR 19

PROGRAM EVALUATION RESPONSIBILITY ASSIGNMENTS 20

Internal Evaluation of Students 22

DESCRIPTION OF DATA COLLECTION PROCEDURE PRE-ADMISSION 22

Review of Applicants 22

DESCRIPTION OF DATA COLLECTION PROCEDURE 24

Skills Course Evaluation 24

Progress Review Committee (PRC) Evaluation 24

Grades for Each Course 25

2

Student Candidacy Review 25

Clinical End of Semester Reviews 26

Final Student Review 27

Remediation and Gatekeeping for the Profession 27

INTERNAL PROGRAM EVALUATION OF STUDENTS REVIEW 28

Strengths 28

Areas to Improve 29

Internal Program Evaluation of Faculty and Program 31


New Student Orientation Feedback 31

Course Evaluations 32

Mid-Program Evaluation 33

Practicum Orientation Evaluation 34

Group Supervisor Evaluation 35

Student Site and Site Supervisor Evaluations 37

Exit Interview 38

Final Program Evaluation 38

INTERNAL PROGRAM EVALUATION OF FACULTY AND PROGRAM REVIEW 39

Strengths 39

Areas to Improve Error! Bookmark not defined.

External Evaluation of Program and Students 42


Site Supervisor Mid-Semester and End of Semester Evaluations 42

Site Supervisor Program Evaluations 43

CPCE Exam 44

Alumni Survey 45

Employer Survey 46

EXTERNAL EVALUATION OF PROGRAM AND STUDENTS REVIEW 46

Strengths 46

Areas to Improve 47

Program Modifications based on Review Data 48

OVERALL PROGRAM STRENGTHS 48


Internal Evaluation of Program and Faculty 48

3

External Evaluation of the Program and Students 49

Strengths Indicated from All Internal and External Evaluations 49

OVERALL PROGRAM AREAS TO IMPROVE 50


Internal Evaluation of Program and Faculty 50

External Evaluation of Program and Students 51

Areas to Improve Indicated by Internal and External Evaluations 51

Questions to Consider Error! Bookmark not defined.

PLAN TO ADDRESS CHALLENGES 51

CONCLUSION 53

4

Table of FiguresFigure 1: Graduate Studies in Counseling within the College of Arts and Sciences 8

Figure 2: Diagram of Program Philosophy 12

Figure 3: Eight-Point Student, Faculty, and Program Evaluation 14

Figure 4: Internal and External Program Evaluation Assessments 15

Figure 5: Matching Assessments with Program Goals and Objectives 17

Figure 6: Program Evaluation Calendar 19

Figure 7: Assessment of Students and Philosophy of Program Evaluation 24

Figure 8: External Evaluations and Philosophy of Program Evaluation 42

Figure 9: Philosophy of Program Assessment with Strengths and Areas to Improve 52

5

Description of Program, Faculty and Curriculum

Mission Statement and Program Goals The mission for the Lipscomb University Graduate Studies in Counseling program is to provide an educational environment characterized by academic excellence and Christian faith, where counseling students are prepared to become competent, confidant, committed and caring professional mental health counselors. See Figure 1.

Figure 1: The Four C's of our Mission

Program GoalsThe Graduate Studies in Counseling program promotes the following goals:

1. To attract and develop a high quality, diverse student body. 2. To increase student knowledge in the field of professional counseling.3. To facilitate competence in the practice of professional counseling.4. To serve and improve the quality of life in the local, state, national, and international community.5. To encourage an understanding and commitment to the scientist practitioner model.6. To facilitate an understanding of the individual characteristics that each student brings to the counseling

process.

Program ObjectivesThe Graduate Studies in Counseling program promotes the following objectives:

1. To gain knowledge of the counseling profession and other related helping professionals.2. To gain knowledge of and become accountable to the ACA Professional Code of Ethics.3. To gain knowledge and competence in serving and advocating for culturally diverse populations, including

assessment, treatment planning, treatment, and outcome evaluation.4. To gain knowledge of eight core knowledge areas: professional orientation and ethical practice, social and

cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation.

5. To gain knowledge of a wellness model of mental health.6. To gain knowledge in the foundations of research and inquiry.

6

Through academic excellence and Christian

faith, counseling students are prepared to become:

CaringCommitted to the Profession

ConfidentCompetent

Program DescriptionThe Graduate Studies in Counseling program is housed in the College of Arts and Sciences. The department of Psychology, Counseling, and Family Science is part of the School of Communications and Social Sciences within that college. Our department includes programs in Undergraduate Psychology and Family Science, Graduate Studies in Psychology, Graduate Studies in Counseling and Marriage and Family. See Figure 2.

7

Figure 2: Graduate Studies in Counseling within the College of Arts and Sciences

8

LIFE ProgramSchool of Natural

Sciences and Mathematics

Lipscomb Family Therapy

Center

Master of Science in

Marriage and Family Therapy

University Counseling

Center


Clinical Mental Health


Psychology

Graduate Programs

Family Science

Psychology

Undergraduate

Department of Psychology and

Counseling

School of Communicatio

n and Social Sciences

School of Humanities

School of Fine and Performing

Arts

COLLEGE OF ARTS AND SCIENCES

Program Faculty

FacultyOur departmental faculty consists of full-time and part-time professors representing the areas of professional counseling, marriage and family therapy, and the fields of clinical, experimental, neuroscience, developmental and cognitive psychology. We supplement this group with excellent part-time faculty members who are experts in their fields. For further information about each faculty member, visit our website.

Core FacultyJake Morris, B.A., M.Ed., Ph.D. (University of Mississippi), LPC, Director of Graduate Studies in Psychology and

Counseling, Professor of Psychology and Counseling

Melanie H. Morris. B.S. (Lipscomb University), M.S., Ph.D. (University of Mississippi), Licensed Psychologist, HSP, Clinical Director of Graduate Studies in Professional Counseling, Assistant Professor of Psychology and Counseling

Shanna D. Ray, B.S. (Lipscomb University), M.S. (Middle Tennessee State University), M.S., Ph.D. (Vanderbilt University), Department Chair of Graduate Studies in Psychology and Counseling, Thesis Coordinator, Associate Professor of Psychology and Counseling

Paul E. Turner, A.A. (Faulkner University), B.S. (Auburn University), M.A. (Southern Christian University), M.S. (University of Western Kentucky), M.A. (University of Southern Mississippi), Ph.D. (University of Southern Mississippi), Professor of Psychology and Counseling

J. Dale Alden III, B.S. (Freed Hardeman University), M.S. (Abilene Christian University), Ph.D. (Virginia Polytechnic University), Internship/Residency, (Vanderbilt University School of Medicine), Licensed Psychologist, HSP, Assistant Professor of Psychology and Counseling

Douglas Ribeiro, B.S., M.S. (Freed Hardeman University), Ph.D. (Georgia State University), Assistant Professor of Psychology and Counseling

Denis’ Thomas, B.A. (Abilene Christian University), M.Ed. (Middle Tennessee State University), Ph.D. (University of Tennessee),NCC, Coordinator of Play Therapy Specialization, Assistant Professor of Psychology and Counseling

Scott, Frank,

Curriculum The Master of Science in Clinical Mental Health Counseling offers a 61-hour curriculum designed to meet the academic and graduate clinical experiences required for licensure as a Professional Counselor with Mental Health Service Provider designation (LPC-MHSP) in the State of Tennessee (TN). This credential allows students to provide counseling services in a variety of settings as outlined by the State Board of Certification for Professional Counselors and Marital and Family Therapists and Clinical Pastoral Therapists. For detailed information about this licensure, see the rules of the State Board of Certification for Professional Counselors and Marital and Family Therapists and Clinical Pastoral Therapists. Our program’s emphasis on faith integration is especially helpful for those who wish to practice in faith-related settings (although students are not limited to only these settings). Graduation from the Counseling Program requires the following completed steps:

● Cumulative GPA of 3.0 or higher

● Successful completion of Plan of Study (POS)

● Successful completion of the Progress Review process

9

● Approval for Candidacy

● Successful completion of Clinical Experiences

● Successful completion of Coursework including Capstone and comprehensive exam

● Completion of Application for Graduation and completing all required paperwork

● Recommendation of the Graduate Studies in Counseling faculty

Courses

Core Courses (46 Hours)

COUN 5103 Theories of Counseling and Personality

COUN 5113 Introduction to Clinical Mental Health

COUN 5203 Introduction to Psychopathology and Adaptive Behavior

COUN 5303 Lifespan Development

COUN 5603 Research Methods and Statistics

COUN 5703 Ethics and Professional Issues in Counseling

COUN 5803 Integration of Psychology and Christianity Seminar

COUN 5903 Introduction to Counseling Skills

COUN 6053 Multicultural Issues in Counseling

COUN 6103 Group Dynamics and Group Counseling

COUN 6253 Career Counseling

COUN 6303 Substance Abuse Counseling

COUN 6603 Advanced Psychopathology

COUN 6653 Assessment Techniques

COUN 6703 Effective Counseling and Treatment Planning

COUN 6951 Capstone Seminar in Counseling

10

Clinical Experience Courses (9 Hours)

COUN 6803 Practicum

COUN 6853 Internship I

COUN 6903 Internship II

COUN 6910 Internship Continuation (if needed)

Elective Courses (select 6 Hours)

COUN 5003 Survey of Psychology (for students who did not major in psychology)

COUN 6003 Grief and Crisis Counseling

COUN 6153 Marriage and Family Counseling

COUN 6203 Child and Adolescent Counseling

COUN 6353 Addictive Disorders

COUN 6403 Psychopharmacology

COUN 6453 Gerontology

COUN 6503 Human Sexuality

PSG 6553 Psychology of Religion and Spirituality

COUN 675V Psychological/Counseling Topics

COUN 675V Introduction to Play Therapy

COUN 675V Advanced Play Therapy and Expressive Arts

PSG 6933 Thesis I

PSG 6943 Thesis II

Systematic Plan for Program EvaluationAccreditation Standards The primary accrediting body for the counseling programs is the Council for Accreditation of Counseling and Related Educational Programs (CACREP). The 2009 CACREP standards and policies are guidelines for the plan for program evaluation for the Graduate Studies in Counseling program at Lipscomb University. The following sections are excerpted from these standards, which can be found in their entirety here, and guided our comprehensive program evaluation plan, including internal evaluation of students, internal evaluation of the program and faculty, and external evaluation of students and program.

Section I: THE LEARNING ENVIRONMENT : STRUCTURE AND EVALUATION

THE ACADEMIC UNIT (CACREP, 2009, p. 5).

P. The program faculty conducts a systematic developmental assessment of each student’s progress throughout the program, including consideration of the student’s academic performance, professional development, and personal development. Consistent with established institutional due process policy and the American Counseling Association’s (ACA) code of ethics and other relevant codes of ethics and standards of practice, if evaluations indicate that a student is not appropriate for the program, faculty members help facilitate the student’s transition out of the program and, if possible, into a more appropriate area of study.

EVALUATION (CACREP, 2009, p. 8)

11

AA. Program faculty members engage in continuous systematic program evaluation indicating how the mission, objectives, and student learning outcomes are measured and met. The plan includes the following:

1. A review by program faculty of programs, curricular offerings, and characteristics of program applicants.2. Formal follow-up studies of program graduates to assess graduate perceptions and evaluations of major

aspects of the program.3. Formal studies of site supervisors and program graduate employers that assess their perceptions and

evaluations of major aspects of the program.4. Assessment of student learning and performance on professional identity, professional practice, and

program area standards.5. Evidence of the use of findings to inform program modifications.6. Distribution of an official report that documents outcomes of the systematic program evaluation, with

descriptions of any program modifications, to students currently in the program, program faculty, institutional administrators, personnel in cooperating agencies (e.g., employers, site supervisors), and the public.

BB. Students have regular and systematic opportunities to formally evaluate faculty who provide curricular experiences and supervisors of clinical experiences.

CC. Annual results of student course evaluations are provided to faculty.

DD. Written faculty evaluation procedures are presented to program faculty and supervisors at the beginning of each evaluation period and whenever changes are made in the procedures.

Philosophy of Program Evaluation The philosophical approach that guided our plan for program evaluation was to integrate three areas to create a comprehensive plan to uphold 1) the standards of the profession, 2) Lipscomb University’s ideals, and 3) student growth. The mission for the Lipscomb University Graduate Studies in Counseling is to provide an educational environment characterized by academic excellence and Christian faith. Counseling students are prepared to become competent, confident, committed and caring (the 4 C’s) professional mental health counselors in eight core areas of knowledge: professional orientation and ethical practice, social and cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation. This takes place both inside and outside of the classroom and is characterized by academic, professional, and personal growth. See Figure 3.

12

Figure 3: Diagram of Program Philosophy

Student Learning Objectives The Clinical Mental Health Counseling Program at Lipscomb University used our program philosophy to develop 15 student learning outcomes.

Educational Environment of Academic Excellence and Christian Faith

A.1.Students are prepared to become competent professional mental health counselors.

A.2. Students are prepared to become confidant professional mental health counselors.

A.3.Students are prepared to become committed professional mental health counselors.

A.4.Students are prepared to become caring professional mental health counselors.

Core Areas of Knowledge

B.1.Students gain academic and practitioner knowledge in professional orientation and ethical practice.

B.2. Students gain academic and practitioner knowledge in social and cultural diversity.

B.3.Students gain academic and practitioner knowledge in human growth and development.

B.4.Students gain academic and practitioner knowledge in career development.

13

Academic Excellence & Christian Faithprofessional mental health counselors who

are:competentconfidentcommittedcaring

Growthacademic

professionalpersonal

8 Core Areas of Knowledgeprofessional orientation and

ethical practicesocial and cultural diversityhuman growth and developmentcareer developmenthelping relationshipsgroup workassessment and testingresearch and program evaluation

B.5.Students gain academic and practitioner knowledge in helping relationships.

B.6.Students gain academic and practitioner knowledge in group work.

B.7.Students gain academic and practitioner knowledge in assessment.

B.8.Students gain academic and practitioner knowledge in research and program evaluation.

Student Growth

C.1.Students grow in academic knowledge of the counseling profession.

C.2.Students develop a professional counseling identity through involvement in local, state and national professional

organizations.

C.3.Students experience personal growth including awareness of personal issues, potential biases, identification of a

personal counseling theory, and strengths.

In order to monitor our effectiveness as a training program, we have implemented a comprehensive plan to provide continuous and systematic data that we review, consider, and evaluate to determine our areas of strength and address areas that could be improved. We believe this on-going process is critical in improving our program to meet the needs of a changing culture and profession.

The program is based on data collected at eight key points, structured around a student’s progress through the program: the application stage, orientation to the program, first semester, each pre-practicum semester, application to candidacy, each clinical semester, the last semester, and as alumni. Our system for program evaluation contains 26 different assessments, around 90 assessments during a typical student’s program cycle, and thousands of data points. We sought data from faculty, students, clinical site supervisors, alumni, employers of graduates, and an outside standardized test. See Figure 4.

14

Systematic Student, Faculty and Program Evaluation

Figure 4: Eight-Point Student, Faculty, and Program Evaluation

15

Internal Eval of Faculty/ Program

Course EvaluationsInternational Service

Trip Culture and Values Review

Internal Eval of Students

GradesProgress Review

Pre-Practicum Semesters


Course Evaluations


Skills EvaluationGradesProgress Review

Semester 1


Orientation Evaluation

Orientation


Stakeholder Program Evaluation

Annual Faculty Evaluations


Evaluation of Application to Determine Admission

Applicant Interview

Application/ Interview

External Eval of Program

Alumni SurveyEmployer Survey

Alumni

External Eval of Program

National CPCE Exam


Exit InterviewCourse EvalFinal Program Evaluation


Final Student ReviewGrades

Final Semester

External Site Supervisor Eval of Program

Mid-term Eval of Students

End of Semester Eval of Students


Group Supervisor EvalSite Supervisor EvalSite EvalCourse Eval


End of Semester EvalGrades

Clinical Semesters


Mid-Program EvaluationPracticum Orientation

Evaluation

Internal Evalof Students

Admission to Candidacy Review

Application to

Candidacy

Eight-Point Student, Faculty and Program Evaluation Plan The program evaluation plan for the Graduate Studies in Counseling Program revolved around three areas: eight evaluations of students within the program, twelve evaluations of the program within the program, and six evaluations of students and the program from external sources. See Figure 5. Some assessments were used multiple times during the program cycle. See Figure 4.

Figure 5: Internal and External Program Evaluation Assessments

Application and InterviewThe first point of data collection for evaluating the program comes from information provided by potential students from applications to the program as well as assessment data from the interview process. This includes self-reported information on ethnicity, faith and gender, undergraduate GPA, and GRE scores. Also, during the summer semester we conduct the annual stakeholder review of the program and each faculty member is evaluated. Since this conceptually gives us a starting point for the next academic year, we have placed those here.

New Student OrientationFor each new cohort admitted to the program, we hold an orientation early in the first semester. During the orientation, we distribute the counseling handbook and other materials, give an overview of the curriculum, and outline the clinical experiences and expectations. Students are asked to provide feedback on the orientation to help us make it more beneficial as the final agenda item for the orientation meeting.

First SemesterDuring the first semester, we conduct three internal evaluations of students and one student evaluation of faculty and facilities. The first semester in the program provides an important time for faculty to begin evaluating student skills, character, and grades and address any areas of deficiency that are identified. Since all first semester students are required to take the Introduction to Counseling Skills course, we conduct a skills evaluation for students in that course. The Progress Review Committee meets at the conclusion of each semester to review student dispositions, grades and

16

Clinical Site Supervisor Mid-Term Evaluations

Clinical Site Supervisor End of Semester

Evaluations

Site Supervisor Program Evaluations

Alumni Surveys

Employer Surveys

CPCE Exam

External Evaluation

of Program and Students

Stakeholder Program Review

Annual Faculty Evaluations

New Student Orientation Evaluation

Course Evaluations

Service Trip Culture and Values Review

Mid-Program Evaluation

Practicum Orientation Evaluation

Group Supervisor Evaluation

Student Site Supervisor Evaluation

Site Evaluation

Exit Interview

Final Program Evaluation

Internal Program Evaluation

of Program

Application Review

Applicant Interview

Skills Course Evaluation

Progress Review Committee Evaluation

Grades for each course

Student Candidacy Review

Clinical End of Semester Reviews

Final Student Review

Internal Evaluation

of Students

any faculty concerns for each pre-candidacy student. Students provide feedback about faculty and facilities through course evaluations.

Pre-Practicum SemestersFor each semester between the first semester and Practicum, students are evaluated through grades. As with every semester, any concerns with student behavior, grades, or dispositions are discussed by the Progress Review Committee. Many students opt to go on an international service trip before clinical work begins, so we have added that opportunity for feedback here although students may opt to go during their clinical semesters as well. Students provide feedback through course evaluations.

Application to CandidacyAll students formally apply for candidacy. The candidacy application form verifies that students have: rectified any undergraduate deficiencies; completed at least 12 hours of graduate work, including a minimum of four core classes; maintained a 3.0 GPA; received successful reviews by the departmental Performance Review Committee; and earned successful evaluations on the Professional Performance Review forms during the first two semesters. Students also complete a mid-program evaluation at the practicum orientation meeting, which falls at about this time in the program cycle.

Clinical SemestersStudents spend at least three semesters (one calendar year) registered for a clinical experience (Practicum, Internship 1, or Internship 2). These semesters provide opportunities to evaluate real-world student counseling skills, as well as academic, professional and personal growth. During these semesters, we require site supervisor feedback on the students and our program. To ensure high quality site placements, students evaluate and give feedback on placement sites and site supervisors.

Final SemesterBefore students graduate from the Graduate Studies in Counseling program at Lipscomb University, we conduct a Final Student Review to ensure that all students meet graduation requirements. Students complete the CPCE exam, participate in an Exit Interview, and complete the Final Program Evaluation form to give summative feedback on the program and their experience.

Alumni Finally, we assess data from those who have graduated from our program and those who employ graduates from our program. Specifically, we want to assess graduates professionally in the areas of competence, confidence, commitment to the profession and caring in their work as professional mental health counselors. We contact employers of our alumni to gather feedback on our graduates.

Meeting Program Goals and Objectives Through the Eight Point Assessment Plan Using the eight-point program evaluation plan and the three categories of evaluations (internal student, internal program and faculty, and external student and program) as a template, we aligned our program goals and objectives to the 26 different assessments we conduct during the program cycle. See Figure 6.

17

Meeting Program Goals and Objectives Program Goals Program Objectives

1 2 3 4 5 6 1 2 3 4 5 6

Internal Evaluation of Students

Application Review X X

Applicant Interview X X

Skills Course Evaluation X X X X X X

Progress Review Committee Evaluation X X X X X X

Grades (every semester) X X X X X X X X X X

Student Candidacy Review X X X X X X

Clinical End of Semester Reviews X X X X X X X X X X

Final Student Review X X X

Internal Program Evaluation

Stakeholder Program Evaluation X X X X X X X X X X X X

Annual Faculty Evaluation X X X X

New Student Orientation Evaluation X X X

Course Evaluations X X X

Service Trip Feedback X X X X X X X

Mid-Program Evaluation X X X X X X X

Practicum Orientation Evaluation X X

Group Supervisor Evaluation X X X X X

Student Site Supervisor Evaluation

Site Evaluation

Exit Interview X X X X X

Final Program Evaluation X X X X X X X

External Program and Student Evaluations

Clinical Site Supervisor Mid-Term Evals X X X X X X X X X X

Clinical Site Supervisor End Semester Evals X X X X X X X X X X

Site Supervisor Program Evaluations X

18

Alumni Surveys X X X

Employer Surveys X X X

CPCE Exam X X X X X X

Figure 6: Matching Assessments with Program Goals and Objectives

Program GoalsThe Graduate Studies in Counseling program promotes the following goals:

1. To attract and develop a high quality, diverse student body. 2. To increase student knowledge in the field of professional counseling.3. To facilitate competence in the practice of professional counseling.4. To serve and improve the quality of life in the local, state, national, and international community.5. To encourage an understanding and commitment to the scientist practitioner model.6. To facilitate an understanding of the individual characteristics that each student brings to the counseling process.

Program ObjectivesThe Graduate Studies in Counseling program promotes the following objectives:

1. To gain knowledge of the counseling profession and other related helping professionals.2. To gain knowledge of and become accountable to the ACA Professional Code of Ethics.3. To gain knowledge and competence in serving and advocating for culturally diverse populations, including assessment,

treatment planning, treatment, and outcome evaluation.4. To gain knowledge of eight core knowledge areas: professional orientation and ethical practice, social and cultural

diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation.

5. To gain knowledge of a wellness model of mental health.6. To gain knowledge in the foundations of research and inquiry.

To aid in our implementation of the program, we have created a calendar of tasks for each semester and a chart to delineate responsibilities. See Figure 7 and the following chart.

19

Program Evaluation Calendar

Figure 7: Program Evaluation Calendar

20

EndEVAL OF STUDENTSGradesProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview

INTERNAL PROGRAM EVALSAnnual Faculty EvaluationFaculty Supervisor Course

EvaluationsGroup Supervisor Course

EvaluationsStudent Mid-Program

EvaluationStudent Evaluation of Site

SupervisorStudent Evaluation of Site

EXTERNAL EVALSSite Supervisor EvaluationsEmployer SurveyCPCE Exam

MidEVAL OF STUDENTSReview of ApplicationsAdmission to Candidacy Review

INTERNAL PROGRAM EVALSAnnual Program Evaluation

Report meetingService Trip Feedback

EXTERNAL EVALSSite Supervisor Mid-Semester

Student Evaluations

BeginningINTERNAL PROGRAM EVALSPracticum Orientation

Evaluation

Summer Semester

EndEVAL OF STUDENTSGradesSkills Course EvalProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview

INTERNAL PROGRAM EVALSFaculty Supervisor Course





EXTERNAL EVALSSite Supervisor EvaluationsAlumni SurveyCPCE Exam

MidEVAL OF STUDENTSAdmission to Candidacy Review

INTERNAL PROGRAM EVALSService Trip Feedback


Student EvaluationsSite Supervisor Program

Evaluation

BeginningINTERNAL PROGRAM EVALSNew Student Orientation

EvaluationPracticum Orientation

Evaluation

Spring Semester

EndEVAL OF STUDENTSGradesSkills Course EvalProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview

INTERNAL PROGRAM EVALSFaculty Supervisor Course





EXTERNAL EVALSSite Supervisor EvaluationsCPCE Exam

MidEVAL OF STUDENTSReview of ApplicationsAdmission to Candidacy Review


Student Evaluations

BeginningINTERNAL PROGRAM EVALSNew Student Orientation

EvaluationPracticum Orientation

Evaluation

Fall Semester

Program Evaluation Responsibility Assignments Beginning of Semesters

WITHIN PROGRAM EVALUATIONNew Student Orientation Evaluation Clinical DirectorPracticum Orientation Evaluation Clinical DirectorStudent Mid-Program Evaluations Director of Program Evaluation

Mid-SemestersEVALUATION OF STUDENTSReview of Applications Department ChairAdmission to Candidacy Review Progress Review Committee

WITHIN PROGRAM EVALUATIONStakeholder Program Evaluation Report Meeting Director of Program EvaluationAnnual Faculty Evaluation Department ChairService Trip Feedback Service Trip Faculty

EXTERNAL EVALUATIONSSite Supervisor Mid- Semester Student Evaluations Clinical FacultySite Supervisor Program Evaluation Clinical Director

End of SemestersEVALUATION OF STUDENTSGrades All facultySkills Course Evaluation Skills Course FacultyProgress Review Committee Assessment Progress Review Committee Clinical Evaluation Practicum, Internship 1, and Internship 2 FacultyFinal Semester Student Review Department ChairFinal Semester Exit Interview Capstone Course Faculty

WITHIN PROGRAM EVALUATIONFaculty Course Evaluations Director of Institutional ResearchGroup Supervisor Evaluations Director of Institutional ResearchStudent Final Program Evaluation Director of Program EvaluationStudent Evaluation of Site Supervisor Clinical FacultyStudent Evaluation of Site Clinical FacultyFinal Exit Interview Capstone Faculty

EXTERNAL EVALUATIONSite Supervisor Final Evaluations Clinical FacultyEmployer Program Evaluation Clinical DirectorAlumni Program Evaluation Director of Program EvaluationCPCE Exam Capstone Faculty

21

22

Internal

Evaluation

of

Students

Internal Evaluation of StudentsDescription of Data Collection Procedure Pre-Admission Our first review of students begins before admission to the program, during the application process. Applicants to the Graduate Studies in Counseling program submit the following application materials to be considered:

1. Application Form2. Application Fee3. Goals Statement4. Graduate Record Examination (GRE) Standardized exam score5. GPA6. Test of English as a Foreign Language (TOEFL) for international students7. Three letters of reference8. Official Transcript(s) from all schools attended

After submitting all application materials, admission materials are reviewed by members of the Graduate Studies in Counseling Admissions Committee, which consists of the Director of the Graduate Program, the Clinical Director of the Graduate Program, and two additional counseling faculty appointed by the Director.

Review of ApplicantsOur first program goal is, “To attract and develop a high quality, diverse student body.” To improve upon the quality of students, in the spring of 2014 we implemented an evaluation rubric to help in reviewing admission materials. This provided a baseline assessment of the quality and diversity of the students that applied to the program. We examined the variables of age, undergraduate GPA, GRE scores, race and ethnicity, faith or religious preference, and gender to assess quality and diversity. Thirty-six students submitted all materials necessary to apply for the Graduate Studies in Counseling Program for the fall 2015 semester. The mean age of applicant was X years with an average undergraduate GPA of 3.44 and an average GRE score of 151 in the verbal section and 146 in the quantitative section. Applicants self-identified data related to race/ethnicity and gender, but responses to both were optional. For the fall 2015 cohort, X% of applicants were female and X% male. Of the thirty-six applicants, X% chose not to disclose their ethnicity. Another X% identify as African American or black and X% identified themselves as Caucasian. Rubrics were also used to provide a fair and consistent evaluation of the applicants. Although scores were very consistent across reviewers and differences in scores were small, below are the top three categories (1-3) and the bottom three categories (14-16) across interviewees:

Caring

Cultural

Sensitivity

Competence

Commit

ment

Ability to be Reflective

Professio

nal

Presentation

Confi

dence

Ability to think Critically

Dependability

Relates to Others

Attending Skills

Ability to Listen

Verbal

Expression

Group B

ehaviors

Enthusiasm

Un

derstanding the Profession

4.71 4.68 4.56 4.56 4.48 4.48 4.46 4.44 4.43 4.43 4.41 4.38 4.37 4.32 4.32 4.27

23

To help us capture a snapshot of our typical student, we examined the variables of age, undergraduate GPA, GRE scores, race and ethnicity, faith or religious preference, and gender to assess quality and diversity. Twenty-five new students enrolled in classes during the fall and spring semesters of the 2014-2015 academic year. The mean age of accepted students was 27 years with an average undergraduate GPA of 3.54 and an average GRE score of 151 Verbal and 148.2 Quantitative. Applicants self-identified data related to race and ethnicity, faith or religious preference, and gender, but it is noteworthy that not all chose to provide that information on their application, especially in the category of religious preference (22 out of 25 chose not to provide, equaling 88%).

Figure 9: Gender Distribution of New Students

24

Figure : Ethnic Distribution of New Students

Description of Data Collection Procedure As part of our internal evaluation of counseling students admitted to the program, faculty reviewed students’ academic, professional and personal growth throughout the program. Specifically, we identified six separate types of faculty-conducted evaluations, and a minimum of one evaluation per student every semester they were enrolled in the program. See Figure 11. This systematic evaluation helped us identify problematic behaviors and concerns earlier in the program and more accurately evaluate student learning outcomes.

8 Core Knowledge Areas Mission Statement Growth

Ethics

Diversity

Development

Career

Helping

Group

Assessment

Research

Competent

Confident

Commitment

Caring

Academic

Professional

Personal

Skills Course Evaluation

X X X X X X X X X X

PRC Evaluation

X X X X X X X X

GradesX X X X X X X X X X

Candidacy Review

X X X X X X X

Clinical Review

X X X X X X X X X X X X X X

Final Student Review

X X X X X X X X X X

Figure 11: Assessments of Students and Student Learning Outcomes

Skills Course EvaluationThe Introduction to Counseling Skills course has been identified as an important course to begin evaluating counseling behaviors. It meets two of our program goals: to facilitate competence in the practice of professional counseling, and to facilitate an understanding of the individual characteristics that each student brings to the counseling process. Because the Skills course is required at the beginning of the program, it gives us an early evaluation of personal and professional

25

Figure : Age Distribution of New Students

behaviors in the first semester. This assessment improves gatekeeping for the profession at the beginning of the program. The faculty members of the course complete assessments rating counseling students from 1 (low) to 5 (high) on 28 questions. Questions are grouped in the following domains with the following average scores: classroom conduct (5.0), caring (4.83), professional responsibility (4.87), competence (4.78), and confidence and maturity (4.8). These scores represent improvements across all domains. Unfortunately, reporting for the skills course evaluation is somewhat incomplete. Three sections were provided in the 2014-15’ school year, but only one instructor (n=13) reported complete results. Another instructor was only able to report final course grades, which, it should be noted, were all A’s. Detailed scores were collected for the third course, but the report for these was apparently lost. Moving forward, skills course instructors should be encouraged to follow a uniform process for data collection and storage.

Progress Review Committee (PRC) EvaluationThe Progress Review Committee meets each semester to discuss student progress. Faculty for each course that a student enrolled until accepted for candidacy are required to complete the Professional Performance Review form assessing the following areas: open to new ideas, flexible, cooperates with others, accepts and uses feedback, aware of impact on others, effectively deals with conflict, accepts personal responsibility, expresses feelings effectively and appropriately, attends to ethical and legal considerations, and takes initiative, and is motivated. Faculty consider not only academic performance, but also professional, intrapersonal, and interpersonal skills and behavior. Informally, students are evaluated on the appropriateness of skills through interactions with faculty, staff, and other students.

After both the first and second semesters (plus additional semesters until students are accepted for candidacy), the professional performance review committee meets to review the Progress Review forms for each student. If a faculty member has concerns about the appropriateness of a student’s progress, course performance, or behavior, then the student receives a Notice of Concern letter. If students do not receive written notification, they are considered in good standing.

Grades for Each CourseOur second program goal is, “To increase student knowledge in the field of professional counseling,” and one way we address that goal is through academic knowledge gained through courses. All coursework in the university is assigned a letter grade (no plus or minus grades) as an assessment of student knowledge of course material. Students earn grades on an “A” (excellent), “B” (good), “C” (marginal) and “F” (failing) scale. An “I” (incomplete) may be given under special circumstances, but is computed as an “F” until the course is completed. Each letter is assigned a quality point used to determine Grade Point Average (GPA). At the beginning of each course, students receive a syllabus detailing expectations for the course and how grades will be assigned.

The curriculum for the program requires students to complete 46 hours in the following core courses: Theories of Counseling and Personality,Introduction to Clinical Mental Health,Introduction to Psychopathology and Adaptive Behavior, Lifespan Development,Research Methods and Statistics,Ethics and Professional Issues in Counseling,Integration of Psychology and Christianity Seminar, Introduction to Counseling Skills,Multicultural Issues in Counseling,Group Dynamics and Group Counseling,Career Counseling,Substance Abuse Counseling,Advanced PsychopathologyAssessment Techniques,Effective Counseling and Treatment Planning, andCapstone Seminar in Counseling. The program also requires students to complete a minimum of 9 hours clinical experience (Practicum, Internship 1, and Internship 2), and 6 hours of electives.

Grades are systematically reviewed through the Progress Review Committee Evaluations, Student Candidacy Review, and Final Student Review. Advisors review student grades each semester as needed, especially for each student with a performance improvement plan.

26

Student Candidacy ReviewAfter the student receives approval from the Progress Review Committee to proceed into the second semester of coursework and completes at least 12 hours of coursework, the student may apply for “Candidacy” which allows the student to continue progress in the program and pursue the clinical experience. For admission to Candidacy, the student satisfy the following requirements:

1. Complete all required undergraduate deficiencies if admitted on condition, including successful completion of a PIP leading to “Graduate Student” status.

2. Complete at least twelve hours of graduate work (in no less than two semesters). The twelve hours must include four of the following: COUN 5903 Introduction to Counseling Skills, COUN 5113 Introduction to Clinical Mental Health Counseling, COUN 5203 Introduction to Psychopathology and Adaptive Behavior, COUN 5103 Theories of Counseling and Personality, COUN 5703 Ethics and Professional Issues in Counseling, and COUN 6603 Advanced Psychopathology.

3. Maintain a 3.00 GPA on all courses taken toward the requirements for the degree with no incomplete grades.

4. Receive successful reviews by PRC and successful evaluations on Professional Performance Review forms during the first and second semesters.

5. File an “Application for Candidacy” form with Graduate Studies in Counseling which will initiate a formal review by the PRC. Applications are due each semester on the Monday after graduation by 4:00 p.m. to the departmental administrative assistant.

Clinical End of Semester ReviewsThe Lipscomb University Faculty Group Supervisor assesses each student enrolled in a Practicum or Internship clinical experience. At the end of the course, students are evaluated in the following domains: assessment and diagnostics, therapeutic intervention, supervision, ethical and professional behavior, strengths and opportunities for improvement. The evaluation form also documents that students meet the clinical hourly and direct service requirements.

Hourly Requirements. Practicum students are required to complete a minimum of 150 hours, with at least 40 hours in direct service. Internship students are required to complete a total (over 2 or more semesters) of 600 hours, with at least 240 in direct service. Our students exceeded the minimum hourly requirements (by 29.38% in Practicum), accumulated experience with groups each semester, and received an average of 43.02 hours of direct supervision each semester.

Clinical Hours Required

Hours

Average

Total

Required

Direct Hours

Average

Direct

Group Site Supervision

Faculty Supervision

Practicum

n=14

150 194.07 60

62.44 11.81 12.57 26.96

Last year 213 73.66 21.43 18.73 25.58

Difference -18.93 -11.22 -9.62 -6.16 1.38

Internship 1 & 2

n=26

300 386.1 120

149.25 49.21 15.21 26.93

Last year 322.9 133 33 23.7 25.59

Difference 63.2 16.25 16.21 -8.49 1.34

27

Evaluation Categories. The evaluation form for the clinical experiences rated students in four domains: assessment and diagnostics, therapeutic intervention, supervision, ethical and professional behavior. The site supervisors completed a nearly identical evaluation for the students, which will be reviewed in that section. The scale ranked students as excellent (4), competent (3), needs work (2) or very poor (1). Although, as expected, beginning Practicum students scored lower in all categories except supervision, the rank order of the categories was same. This means that Lipscomb students demonstrated strong characteristics in supervision, along with high ethical and professional behavior, but scored the weakest in assessment and diagnostics, regardless of which clinical experience they were enrolled.

Evaluation Categories for Practicum, Internship 1, and Internship 2 (n=40) All Clinical

Students

Practicum Internship

Supervision 3.76 3.86 3.65

Ethical and Professional Behavior 3.54 3.53 3.55

Therapeutic Interventions 3.34 3.19 3.48

Assessment and Diagnostics 3.17 3.05 3.29

Evaluation Questions. When reviewing students, it helped to examine the specific questions that students, overall, scored highest and lowest on to determine strengths and areas to improve. Students averaged a score of 3.75 or higher on 9 questions. The five lowest scoring questions had an average score below 3.15.

Questions with mean scores of 3.8 or higher (n=40) Mean Score

Ethical and Professional Behavior

Is tactful 3.80

Is cooperative with supervisor and peers 3.83

Shows maturity as expected of a student at the end of graduate training 3.84

Supervision

Openly receives suggestions and criticisms without defensiveness 3.90

Willing to carry a sufficient case load 3.84

Participates constructively in group supervision 3.90

Provides useful feedback to peers in group supervision 3.98

Therapeutic Interventions

Demonstrates appropriate interpersonal skills with clients 3.76

Recognizes and encourages clients’ strengths 3.77

Assessment and Diagnostics

28

none

Questions with lowest mean scores (n=40) Mean Score


none

Supervision

none


Demonstrates competence with marriage & family therapy skills 3.13

Implementation of crisis protocol when appropriate 3.12


Demonstrates competency in diagnosis and use of DSM 3.14

Shows skill in developing and implementing treatment plans 3.08

Demonstrates thorough clinical interviewing skills 3.11

The data indicates that students 1) demonstrate strong professional and ethical behaviors, especially with tact, cooperation, and maturity; 2) students participate well and give and receive feedback in supervision; and 3) they have good interpersonal skills with clients and recognize their client’s strengths. The areas that need work, though still rated competent, were 1) marriage and family interventions; 2) implementation of crisis protocol when appropriate; 3) competency using DSM for diagnosis; 4) developing and implementing treatment plans; and 5) clinical interviewing skills.

Final Student ReviewAs a final review to ensure that all academic requirements are met in order for the degree to be conferred, students complete the Petition to Walk in the semester in which all coursework will be completed for graduation. The Graduate Program Director reviewed and signed the form as verification. This final review was a checklist of requirements.

Remediation and Gatekeeping for the ProfessionIn student discipline and gatekeeping for the profession, we strive to come alongside students and use remediation to facilitate growth – academically, personally, and professionally. We believe many deficiencies can be successfully addressed by the student and faculty member within the context of a course or other experience (e.g., clinical experiences) without recourse to more formal procedures. Whenever it is not appropriate to only address concerns about a student within the context of a course or other experience—either because they are too serious or extend beyond a single course or experience—those concerns are brought to the attention of the Performance Review Committee (PRC) by completing a Professional Performance Review (PRR) form. This form outlines deficiencies in the student’s professional, intrapersonal, and interpersonal functioning needing to be addressed, and they are handled in the same manner as progress or annual reviews.

29

In the event that a student fails to meet program expectations, the student may be placed on probation and provided with a Performance Improvement Plan (PIP) to aid the student in overcoming performance deficiencies and return to good standing in the program. The plan is normally developed by the Progress Review Committee (except for PIPs related to program admission decisions, which are handled by the Graduate Studies in Counseling Admissions Committee) and outlines required activities and deadlines that must be achieved by the student, as well as consequences for failure to complete the PIP. Failure to successfully complete directives prescribed in the PIP could result in temporary or permanent dismissal from the program. If the PRC determines that a student’s performance deficiencies are of such a nature and severity that neither the student nor the program will benefit from a PIP, then a student may be immediately dismissed temporarily or permanently from the program.

A Performance Improvement Plan included 1) a detailed description of the performance/behavioral deficiencies, 2) a detailed explanation of the level of progress or performance required to remediate each deficiency—with reference to supporting documentation from written standards in the program (e.g., syllabi, practicum evaluation criteria), 3) the required actions to be taken and the documentation necessary to substantiate successful completion of the action steps, 4) the specific timeframe for completion of the plan, and 5) the consequences for failure to complete the plan. The plan is signed and dated by the student, the advisor, and Graduate Studies in Counseling Director, with a copy given to the student and another placed in the student’s file. The advisor and student discuss and establish mechanisms to aid the student in completing the PIP in the allotted time, and the advisor monitors the student’s plan progress and reports this progress to the PRC.

Based on a review of the student’s progress, the Progress Review Committee determines whether the student meets the terms of the Performance Improvement Plan. If the student demonstrates compliance with the terms of the PIP, the student is considered in good standing and no longer on probation. If the student fails to complete the PIP in accordance with its terms, the student was is subject to sanctions up to and including temporary or permanent dismissal from the program. The PRC communicates its determinations in writing to the student, with a copy placed in the student’s file. Final decisions regarding dismissal due to performance/behavioral deficiencies, when necessary, are made by the Graduate Studies in Counseling Director, upon recommendation by the Performance Review Committee.

Internal Program Evaluation of Students Review Through six evaluation opportunities (once admitted to the program) and a minimum of 33 faculty evaluations per student (for a typical 7 semester program cycle), faculty assessed students early and consistently throughout the program. This enabled us to identify developing patterns in students systematically and from a variety of faculty viewpoints. We were able to 1) evaluate students’ academic, professional and personal growth 2) assess the eight core knowledge areas, and 3) determine how well our students demonstrated caring, competence, confidence, and commitment to the profession.

Strengths

● Quality Students – We accept a diverse group of students related to ethnicity and age. We have many

indications that Lipscomb counseling students are achieving academic excellence, dedicated to personal growth, and are developing well professionally. Most of our students receive excellent grades, recommendations to continue progressing, and good to excellent faculty clinical evaluations. Most students progress through the program without any need for a program improvement plan or other remediation. Our students are open to criticism and constructively work with others to improve.

● Ethical and Professional Behavior – Students consistently demonstrate strong professional and ethical behaviors,

especially with tact, cooperation, and, this year, maturity. They also score high on valuing cross-cultural

30

sensitivity and respecting alternative worldviews, along with effective communication skills.

● Therapeutic Interventions – Students have good interpersonal skills with clients and recognize their client’s

strengths. They also score high at maintaining expected caseloads at their sites.

● Feedback in Supervision – Students continue to participate well in supervision, and they averaged high scores in

appropriately giving and receiving feedback.

● Experience Conducting Groups – Students are spending more hours facilitating group work in internship.

● Above and Beyond Minimum Requirements – Students are dedicated to learning both in and out of the

classroom. Practicum students on average exceed the number of required hours by 29.38%.

Improvements from Last Year

● Faculty Supervision – For the second straight year, students, on average, received more than the accreditation

standard minimum of 24 hours per semester of faculty supervision.

● Experience Conducting Groups – Despite decreases in the practicum group, internship students are spending

more than 49 hours, on average, conducting groups.

● Assessment, Diagnosis and Clinical Interventions – Although students continue to score lower overall in these

categories, the data indicates that our students continue to score competently in these domains.

Areas to Improve

● Implementation of Crisis Protocol when Appropriate –students scored the third lowest in this category.

● Reviewing Research – Although students read and consult current research for classes, the data indicates that

skill may not be fully used in their practitioner work.

● Defending Clinical Treatment Procedures – students create and follow treatment plans, but may lack the ability

to articulate the rationale and purpose.

31

32

Internal

Program Evaluation

of

Faculty and Program

Internal Program Evaluation of Faculty and ProgramDescription of Data Collection Procedure We believe that our students are our best sources to determine how well we are doing at providing an educational environment characterized by academic excellence and Christian faith where counseling students are prepared to become competent, confident, committed and caring professional mental health counselors in eight core areas of knowledge: professional orientation and ethical practice, social and cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation. They also are most familiar with their own academic, professional, and personal growth. Therefore we designed 10 assessments during the program to solicit feedback from students. In addition, we also conducted two additional internal evaluations: the annual program evaluation review, which is presented to faculty during the summer and presents the results of this report, and annual faculty evaluations.

Annual Program EvaluationDuring the summer, the director of program assessment presents the comprehensive results of program evaluation to the faculty and other stakeholders. This presentation outlines the results and conclusions that are summarized in this report. Faculty discusses the strengths, areas to improve, and strategies to address these, which are summarized in the section, “Program Modifications Based on Review Data” below.

Annual Faculty EvaluationsEach faculty member receives a formal evaluation during August from the department chair. During this meeting, faculty goals in teaching, intellectual pursuits, and service are reviewed for the past year and upcoming year. For tenure and promotion, teaching is evaluated in the areas of: course design and management, collegiality and teamwork, and continued academician. Intellectual pursuits are evaluated by scholarly and intellectual activities. Service evaluation criteria include: academic support service, institutional service, church related service, community engagement, professional and discipline-specific service.

New Student Orientation FeedbackBefore the fall and spring semesters, students attend a mandatory program orientation meeting for new students. This meeting introduces students to faculty, presents an overview of the program, and gives students an informational packet, including a copy of the program handbook. At the conclusion of the orientation, students evaluate the orientation. The survey asks the following questions, which students rated from Strongly Disagree to Strongly Agree (a scale of 1 to 5):

Please rate your agreement with the following statements (n=48)

The orientation provided valuable information regarding the program and my professional options.

4.46

The orientation was well organized. 4.46

The orientation helped me feel informed about my choice of academic studies. 4.30

The individual providing the orientation was warm and enthusiastic. 4.63

The orientation provided me many opportunities to ask questions and get concerns resolved.

4.31

33

The orientation provided useful reference materials. 4.52

Overall, the orientation met my expectations. 4.42

We also asked the students for comments (to explain any of their answers above), strengths of the orientation, and areas to improve.

Comments:

“This was a fun night! The food was great, the atmosphere was welcoming and fun, and the organization of information into 4 C's was helpful and engaging. It was a great night!”

“I really appreciated the folders with so much information I will need throughout my education, all in one place. And I felt relieved to meet with my advisor so early on and know where to land when I have a concern.”

“I didn't really feel there was much of a question portion, mainly because it was later at night and people were tired.”

“The information provided will greatly assist in my planning while in the program and as I move into a career.”

Strengths:

● Formal and informal face time with students (e.g. the panel, the tours).

● Plenty of time for Q & A with faculty and staff.

● The interview time.

Areas to Improve:

● “Shorten the schedule by reducing waiting times for interviews and/or cutting the campus tour.”

● “More structure in the student panel discussion; it dragged on in many places.”

The new student orientation, according to student feedback, provided valuable information and included ample face time between prospects, current students, and faculty. Last year’s proposal to implement more Q&A was well received. However, for the second year in a row, attendees report that condensing it and adding more structure would be welcome improvements.

Course EvaluationsFor every course that students enroll, they have the opportunity to complete a course evaluation. This evaluation process is a university-wide process and is conducted at the university, not departmental, level. Email reminders encourage students to complete course evaluations and offer incentives. The evaluation asks students questions in the following domains: course as a whole, teacher effectiveness, interpersonal relationships, evaluation methods, classroom facilities, and additional course questions. Students rate on a 5-point Likert scale from 1 (poor) to 5 (excellent) with 3 as average. Data is tabulated for the department and individual faculty data is sent to the department chair for review.

In responding to course evaluations for all COUN courses (except clinical courses, which are described in the Group Supervisor section), students rated courses, facilities and faculty on a Likert scale from 1 to 5. Course evaluations had

34

about a 60% return rate. Students reported that they put a high level of effort into making their courses a meaningful experience (4.16) and spent about 3.46 hours a week on homework per course, increases of .16 and .15 respectively over the previous year. They rated the work required as above average (3.59), an increase of .08 over last year. Students indicated that, on average, before courses began, their interest level was a 3.8, but after taking the course, their interest level rose to a 4.52 Therefore, we know our students are motivated, spending about 10.38 hours a week outside of classes on homework if enrolled in three courses, and are becoming more interested in the subject matter as they learn more.

The charts below outline how students rated courses and faculty.

Course Evaluations (n=301) COURSE Scale 1-5

Requirements 4.45

How good was the course as a whole 4.5

How well was it organized 4.42

Tests & other assignments cover the material from the class 4.37

Beneficial choices of texts, other materials, and assignments 4.36

How challenging was it intellectually 4.28

Evaluations require critical thinking 4.43

Effective testing & assignments 4.31

Conducive university facilities to learning 4.26

Computer equipment to learning 4.3

Course Evaluations (n=301) PROFESSOR Scale 1-5

Patient and respectful 4.63

Relate this subject to the broader issues of life 4.67

Enthusiastic about the subject 4.66

Keeping appointments and designated office hours 4.62

Approachable 4.46

Encouraging 4.43

Effective interpersonal relationships 4.53

35

Direct and encourage open classroom discussion 4.64

Effectiveness of teacher 4.56

Fairly deal with students 4.71

Clearly communicate information and expectations 4.59

Stimulate critical thinking in writing, discussions, and testing 4.27

Returning graded material within a reasonable time 4.35

Stimulate interest and hold attention 4.31

Providing clear comments for improvement 4.27

Students, overall, scored course requirements, the course as a whole, and the critical thinking stimulated by the course well. Though still well above average, the lowest scoring areas were the university facilities and the intellectual challenge of the courses. In evaluating the professor, students believe their instructors to be patient and respectful, good at relating the subject to the broader issue of life, and enthusiastic about the subject. They are keeping appointments and designated office hours and are approachable, which all scored 4.45 or higher. For the second year in a row, the lowest scoring area was on providing clear comments for improvement (4.27). Other lower scoring areas were stimulating interest and holding attention, returning graded material within a reasonable time, and stimulating critical thinking. Clearly communicating information and expectations was one area of major improvement over last year, with ratings increasing by .25.

International Service Trip EvaluationStudents have the opportunity, but are not required, to travel internationally for academic and service learning trips. During the spring semester, students travel to Kenya for two weeks while enrolled in Multicultural Issues in Counseling and Integration of Psychology and Christianity Seminar (or a Special Topics course). Some years, students travel to Moldova. And during the summer, students may travel to Ghana for courses in Loss, Trauma and Resilience and Research Practicum.

Students taking courses that were part of an international service trip completed the university-wide course evaluation that all courses include. In addition, as part of a larger research project, students traveling to Kenya for study also had the opportunity to voluntarily participate in quantitative and qualitative research assessing reflective thinking, spirituality, and multicultural practices. Students completed three pre- and post-test questionnaires (Level of Reflective Thinking Survey, Daily Spiritual Experiences Scale, and the Multicultural Counseling Inventory) and submitted five photographs with a narrative explanation on the integration of faith and counseling. The results of this research are being submitted for publication and conference presentations, and thus are not included here.

Mid-Program EvaluationStudents evaluate their experience with the program at the mid-point in the program. They rate how well they think the program has prepared them academically in the eight core areas, how the program facilitates their growth academically, professionally, and personally, how they believe professional activities enhance their counselor effectiveness, and how they rate themselves on empathy, comfort with strong emotional expression, and their counseling skills. They also complete five open-ended questions:

36

1. At this point in the program, in what professional organization(s) and activities are you involved?2. What has been the most important element of this program (e.g., courses, assignments, activities, personal

work, other) 3. What has detracted from your growth and learning in the program?4. What would make the program better for upcoming students?5. What are you wiling to contribute to make the program better?

14 students completed the Mid-Program Evaluation. Although the sample size was small, it was interesting to note several results.

● On a scale from 1 (very poor) to 5 (exceptional) the three content areas that students felt the least prepared for

academically were: Career Development (3), Counseling Skills (3), and Research (3.4).

● On a scale from 1 (very poor) to 5 (exceptional) the three content areas that students felt the most prepared for

academically were: Emotional Expression (4.67), Professional Orientation and Ethical Practice (4.5), and Social and Cultural Diversity (4.33).

● Students report personal growth, (4.33), academic growth (4.2), and professional growth (4.33) and they believe

that professional activities enhance their effectiveness as a counselor (4.2).

● Some of the most important elements in the program that students identified included courses, practice/ role

play/ skills videos, assignments and activities, and the multicultural emphasis.

● The two biggest distractions to learning were 1) balancing school, work, and personal life, and 2) lack of

intellectual and academic rigor/expectations.

● Although there were no clear themes in what would improve the program, a few of the responses included:

more attention to burnout, inviting professional guest speakers to discuss the profession, and raising the academic standards.

Practicum Orientation EvaluationStudents are required to attend an orientation meeting before beginning Practicum. The orientation provides information for how to dress, behave and compile final notebooks. Students receive clinical handbooks, and requirements are reviewed.

Students enroll to begin Practicum in all three semesters – fall, spring, and summer. A practicum orientation is conducted before each semester, and students are asked to provide feedback. This is done both to assess the orientation and to model giving and receiving feedback, which students are expected to do in all clinical experiences. Eighteen respondents answered four Likert-scale (ranging from 0 to 100) questions asking:

1) How helpful was the information you received?2) How was the content and the amount of information (verbally and in your packets)?3) How would you rate the orientation overall?4) How well did the orientation meet your expectations?

Overall, students evaluated the fall and spring practicum orientations as helpful with an average score of 81.44 out of 100, with all answers falling between 64 and 100. Students also found the orientation to be helpful as it pertained to the information given (average score 76.28) and the verbal and written contents of the orientation (average score 78.61). Students reported that the orientation exceeded their expectations (average score 76.22).

Next, students are asked to complete the following open-ended questions and add any additional comments:

37

5) Please explain your answer on Question #4.6) Please list a few things that were especially helpful. 7) Please list a few things that would have improved the orientation.8) What suggestions do you have for changes to improve the orientation for future students? 9) What was the most memorable thing from the orientation?

Open-ended questions elicited the following comments. When asked about what was especially helpful, students mentioned the panel of students five times. Other items found to be helpful were the information regarding privacy/security/encryption and review of paperwork and documentation procedures. Five students mentioned the training videos as helpful tools, and students found that having time to meet with the faculty supervisors and supervision group members as helpful as well.

Students indicated that the orientation could be improved by being longer, supplemented with an additional meeting earlier in the semester, and/or being held at a time other than during finals week (4 mentions). Other improvements include more Q&A time (2 mentions), more information for setting up the iPods and using Time2Track (4), and providing more information leading up to the orientation (1 mention).

Finally, students found the Peer Panel and videos to be the most memorable portion of the orientation with 2 mentions each. Unfortunately, eight students did not respond to this question, and one student commented s/he would have left it blank but felt that it was important to emphasize how doing the orientation at the end of the semester was inconvenient and distracted from the helpfulness of the meeting.

Group Supervisor EvaluationTo more effectively evaluate faculty group supervision, we modified the course evaluation given by the university to more accurately reflect a practicum/ internship course. The new evaluation was implemented in the fall of 2013. Students rate on a 5-point Likert scale from 1 (poor) to 5 (excellent) with 3 as average. Over the fall, spring, and summer terms 72 students responded to the course evaluation, about a 50% return rate. Students indicated that they put a high level of effort into making it a meaningful experience (4.44), and reported spending an average of 3.73 hours per week on homework.

Course Evaluations (n=72) COURSE Scale 1-5

How good was the course as a whole 4.01

How challenging was it intellectually 3.95

Requirements 3.73

Work required 4.16

How well was it organized 3.47

Beneficial choices of texts, other materials, and assignments 3.76

Choice of locations for the course 3.36

38

Course Evaluations (n=72) FACULTY GROUP SUPERVISOR Scale 1-5

Stimulate critical thinking in writing, discussions, and testing 4.23

Direct and encourage open classroom discussion 4.21

Effectiveness: content application to professional practice 4.5

Relate this subject to the broader issues of life 4.29

Effectiveness of teacher 4.23

Enthusiastic about the subject 4.17

Effectiveness: open discussion 4.46

Effective interpersonal relationships 4.29

Patient and respectful 4.26

Effectiveness of faculty supervisor 4.3

Effectiveness: group facilitation 4.37

Encouraging of students 4.31

Stimulate interest and hold attention 3.93

Approachability 4.23

Constructive feedback 4.16

Encouraging 4.4

Patient-respectful 4.26

Approachable 4.26

Handling of inappropriate student behavior 4.45

Effectiveness of interpersonal relationships 4.26

Keeping appointments and designated office hours 4.33

Effectiveness of evaluation methods 3.96

Fairly deal with students 4.46

Clearly communicate information and expectations 3.51

Receptive to feedback 4.21

39

Time management 3.83

Providing clear comments for improvement 4.22

Effectiveness of interpersonal relationships with site supervisor 4.16

Returning graded material within a reasonable time 3.97

In addition, we began requiring clinical students to complete Arcinue’s Group Supervision Scale and Efstation, Patton, & Kardash’s Supervisory Working Alliance Supervisee Form at the conclusion of each clinical course. The data included (X number of) respondents in groups led by 4 faculty group supervisors, reporting at the end of two semesters. A more in depth analysis will be conducted as part of a larger research project, but as a preliminary understanding, the overall scores, averaging all faculty led groups together is included below:

Group Supervision Scale n=67 Scale 1-5

Group Safety 1-5 4.61

Skill Development 1-5 4.54

Group Management 4-20 17.85

Supervisory Working Alliance

Client Focus 1-7 6.28

Rapport 1-7 6.26

Based on the responses across the course evaluation, Group Supervision Scale and Supervisory Working Alliance students have a strong sense of group safety, agree their supervisors improve their skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at applying course content to professional practice, facilitating open discussions, encouraging students, dealing fairly with students, and handling inappropriate behavior.

However, almost every area scored lower than the previous year, some by a significant margin. First, students reported much lower clarity of communication of information and expectations, a 1.1-point decrease from last year. The second glaring area of need is stimulating interest and holding attention, which decreased from 4.81 to 3.93 this year.

Student Site EvaluationsEach semester that students enroll in clinical courses, they are asked to evaluate their practicum or internship site. They respond to three questions about the site to assess their difficulty acquiring hours, direct hours, and benefits of site activities. Rating their sites on a scale of 1 (strongly disagree) to 5 (strongly agree). According to the RedCap report on student site evaluations, only six new responses have been recorded since the Spring 2014 semester began. This may suggest a failure in the current data collection strategy, thus meriting a proposal for new data collection methods. Nonetheless, the average recorded responses for 2014-2015 were:

40

Evaluation of the Site n=75 Mean Last Year Mean This Year

I had no difficulty achieving my required total site hours. 4.25 3.67

There was no difficulty getting clients for my face-to-face minimum. 4 4.67

The activities completing my site hours were beneficial. 4.57 4.0

Overall, students had positive experiences with their sites, and they agreed that the activities completing their site hours were beneficial. For the second year in a row, students increasingly struggled to achieve their total hours, though they reported more success achieving the face-to-face minimum requirement.

Student Site Supervisor EvaluationsIn addition to evaluating clinical sites, students also provide feedback about their experiences with their site supervisors. Students answer six questions about their site supervisor to assess the following: goal-achievement, case supervision, weekly supervision, regularly scheduled supervision, supervisor availability, and in-vivo supervision. Questions are scored on a 5-point Likert scale.

Unfortunately, a failure in collecting data for this category seems to have occurred for 2014-15’ reporting. RedCap has only recorded one student response for the year, and these responses are recorded below. For future reporting, the 2013-14’ results should be retained for comparison with next year’s data. A new strategy for data collection is recommended to prevent future failures.

Evaluation of the Site Supervisor n=75 Mean

Last Year

Mean

This Year

The agreed-upon initial goals were achieved during my time there. 4.39 5

I received meaningful case supervision from my site supervisor. 4.45 5

Supervision occurred at least weekly. 4.48 5

Supervision occurred at a regularly scheduled time. 4.33 5

My site supervisor was available when needed. 4.53 4

My site supervisor provided in-vivo supervision. 36% 100%

In addition, we ask three open-ended questions to allow students to provide additional feedback. Two of the questions are: Please elaborate on your overall rating, and what special preparation is needed for students at this site? The student response recorded this year mentioned that the site’s culture pushed students toward excellence while also creating a nurturing environment for students; the student cited time management as the primary skill students should be prepared to use at this site.

The third open-ended question was: What type of students would not benefit well from this experience? Like the last two years, the student responded that a student who required a lot of direction, lacked initiative and motivation, and

41

was not a self-starter would not benefit from this experience.

Exit InterviewBy the time a student completes the program, multiple evaluations a semester have occurred. The Final Student Review is a short, informal assessment conducted by the department chair (who is also the Capstone professor) in the form of a Closing Conversation. The questions to the students generally include:

1. How would you rate the quality of instruction that you received in this program?

2. How would you rate the availability and helpfulness of the faculty in this program?

3. How would you rate the availability and helpfulness of the support staff?

4. How well do you believe you are prepared by the program to function as a licensed professional counselor?

5. Would you recommend this program to a friend, family member or colleague?

Final Program EvaluationThe Final Program Evaluation is identical to the mid-program evaluation. Students rate how well they think the program has prepared them academically in the eight core areas, how the program facilitates their growth academically, professionally, and personally, how they believe professional activities enhance their counselor effectiveness, and how they rate themselves on empathy, comfort with strong emotional expression, and their counseling skills. They also complete five open-ended questions.

11 students completed the Final Program Evaluation. Each was completing the 61-hour program with an average of X hours (range from Y-Z) in an average of X semesters (ranging from Y-Z semesters). While last year, all respondents completed the program in 7 to 9 semesters, this year, X completed the program in Y semesters and about X% completed the program in X semesters or less. Although the sample size was small, it was interesting to note several results:

● On a scale from 1 (low) to 10 (high) the two content areas that students felt the least prepared for academically

were: Career Development (4.5) and Assessment (6.8). These were the same bottom two last year and the year before that. Both Career and Assessment dropped for the second consecutive year.

● On a scale from 1 (low) to 10 (high) the two content areas that students felt the most prepared for academically

were: Professional Orientation and Ethical Practice (10), and Helping Relationships (10).

● The program highly promoted student growth, with personal, academic, and professional growth each receiving

“exceptional” ratings.

● 10 of the 11 participating graduating students declined to rate their counseling skills at this final point in the

program.

● The Practicum and Internship experiences and other courses were the most often listed elements most

important in the program. Multiple respondents also emphasized experiential components in class, such as mock sessions.

● The most suggested learning detractor was online class. The degree plan was also mentioned, with one student

complaining that had s/he taken Research Methods before practicum and internship, s/he would have been better prepared.

42

● Things that would make the program better included more emphasis and preparation for the NCE, less busy

work for the online classes, better job placement assistance, and more opportunities for scholarships and grants, i.e. helping students minimize debt.

Internal Program Evaluation of Faculty and Program Review Students have opportunities to let us know what they think throughout the program. They provide feedback after the New Student Orientation, through in course evaluations, from international service trips, at Mid-Program, after the Practicum Orientation, for each Lipscomb clinical group supervisor, about their Practicum and Internship sites, about site supervisors, during the Exit Interview, and at the end of the program. In addition to the ten internal program evaluations from students, faculty receives feedback on teaching, scholarly activity, and service during an annual review. We summarize all the evaluation information in the Stakeholder Program Review (discussed at the end of the report).

The following is a summary of these internal reviews of our program strengths and challenges.

Strengths

● Timely Completion – Most students (65%) complete the program in seven or eight semesters.

● Courses – Students, overall, scored course requirements, the course as a whole, and the organization of the

course highly.

● Professors – In evaluating the course professor, students rated their instructors patient and respectful, good at

relating the subject to the broader issue of life, and enthusiastic about the subject. They kept appointments and designated office hours and were approachable.

● Faculty Group Supervisors –Students have a strong sense of group safety, agree their supervisors improve their

skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at stimulating critical thinking, encouraging open discussions, applying the content to professional practice and relating the subject to broader issues of life. The supervisors are effective teachers, enthusiastic about their supervision and effective at open discussion.

● Orientations – The orientations for new students and Practicum continue to provide valuable information and

are helpful to students. The new student orientation, according to student feedback, provided valuable information and was well organized. It was perceived to be fun, friendly, warm and welcoming. At the Practicum orientation, students found the student panel, privacy and security information, and review of paperwork and documentation to be especially helpful.

● Knowledge – The content areas that students report feeling the most prepared for academically include:

Professional Orientation and Ethical Practice, Group, and Helping Relationships.

● Student Growth – Students perceive growth that continues throughout the program. Students reported high

levels of personal growth, academic growth, and professional growth at mid-program and even higher levels at the end of the program.

● Clinical Experiences – The Practicum and Internship experiences were the most often listed elements most

important in the program. Overall, students had positive experiences with their sites and site supervisors, especially when it came to supervisor availability and meeting weekly for supervision.

43

Improvements from Last Year

● Mentoring – We incorporated students further along in the program at the orientations to provide their

experience and answer questions. Students responded very positively.

● Orientations – While both the Program Orientation and the Practicum Orientations are perceived as too dense

last year, this year, students reported that they were informative, better organized, and more fun.

● Program Organization – The program is perceived as more organized now.

● Professional Involvement – Last year, students who were least involved in professional organizations saw less

value of professional activities in enhancing their effectiveness as a counselor. We now require membership in the American Counseling Association and encourage state and local professional involvement, and students report a higher value of professional activities.

Areas to Improve

● Knowledge – Career and Assessment continue to be the lowest scoring content areas, and both had lower scores

than last year.

● Facilities and Locations – Students report a strong dislike for the classrooms and furniture in Ward. They also

reported mixed responses to the location of clinical supervision groups (inconsistency in scoring, which may reflect the different locations). Students also repeatedly mentioned parking challenges.

● Library – Students want more library resources, better electronic journal access through databases, and

individual and group graduate study places.

● Grading and Feedback – Returning graded material within a reasonable time and providing clear comments for

improvement consistently has lower scores. Students also indicated that professors are not always clearly communicating information and expectations.

● Integrating Technology – Another low-scoring question was about the importance of computer equipment to

learning. Professors may need to address integrating more relevant technology.

● Advising – Students mentioned that they would like better advising. While not a theme across assessments, this

was mentioned a few times.

● Program Benefits –Students indicated disapproval of the discrepancy in benefits between the mental health

counseling and marriage and family therapy programs.

● Student Challenges – Program faculty should be aware that students report struggling to balance school, work,

and personal life, and they feel the financial pressure of graduate school.

44

45

External

Evaluation

of

Program and Students

External Evaluation of Program and Students Description of Data Collection Procedure

8 Core Knowledge Areas Mission Statement Growth

Ethics

Diversity

Development

Career

Helping

Group

Assessment

Research

Competent

Confident

Commitment

Caring

Academic

Professional

Personal

Site Supervisor Mid-Semester Eval

X X X X X X X X X X X X X

Site Supervisor End Semester Eval

X X X X X X X X X X X X X

Site Supervisor Program Eval

X X X X

CPCE Exam X X X X X X X X

Alumni Survey X X X X

Employer Survey X X X X

Evaluation Categories for Practicum, Internship 1, and Internship 2 (n=78)

All Clinical

Students

Last Year All Clinical Students

Practicum Internship

Supervision 3.66 3.86 3.50 3.74

Ethical and Professional Behavior 3.55 3.79 3.43 3.60

Therapeutic Interventions 3.41 3.65 3.28 3.47

Assessment and Diagnostics 3.38 3.66 3.29 3.42

Evaluation Questions. When reviewing students, it helped to examine the specific questions that site supervisors ranked students, overall, highest and lowest on to determine strengths and areas to improve. Students averaged a score of 3.68 or higher on 10 questions, all within the categories of supervision and ethical and profession behavior. The eight lowest

46

scoring questions had an average score below 3.3, and were mostly in the categories of assessment and therapeutic interventions.

Questions with mean scores of 3.68 or higher (n=78) Mean Score

Supervision

Participates constructively in group supervision 3.71

Openly receives suggestions and criticisms without defensiveness 3.72

Prompt and prepared for supervision meetings 3.70

Uses feedback to modify subsequent counseling behavior 3.69

Self-examines 3.72


Is enthusiastic 3.74

Is able to work independently 3.69

Prompt and dressed appropriately for client appointments 3.73

Is dependable 3.68

Is tactful 3.76


none


none

Questions with lowest mean scores (n=78) Mean Score

Supervision

none


Consults and integrates research to develop a foundation for clinical work 3.26


Willing to take appropriate risks in assessment & treatment 3.22

Demonstrates competence with child interventions 3.14

47

Demonstrates competence with group interventions 3.26

Implements crisis protocol appropriately when needed 3.14

Demonstrates understanding of site crisis protocol 3.29


Makes accurate diagnoses within 2-3 sessions 3.26

Competent use of instruments 3.25

The data indicates that students 1) demonstrate strong supervisee skills, especially with participation in supervision, receiving suggestions and criticism, and being prompt and prepared; and 2) have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently. Areas that need work, though still rated competent, included 1) implementation of crisis protocol; 2) demonstrating competence with child interventions; 3) and taking appropriate risks in assessment and treatment.

Site Supervisor Program EvaluationsLicensed practitioners working in the counseling profession who supervise our students have a unique and valuable perspective on the program. They witness the academic knowledge and skills that students have developed, as well as the growth of students during clinical experiences. Therefore, we sought their evaluation of our program as well.

We asked site supervisors to complete a web-based survey. From that survey we learned that respondents had supervised an average of X Lipscomb counseling students since the counseling program inception in 2007, with a range from Y-Z students. The data revealed that X (Y%) have been supervising Lipscomb students for one to two years, and X (Y%) have been supervising Lipscomb students for three or more years.

Preparedness of Students. We asked site supervisors on a scale from 0 to 100 (0 = not prepared, 50 = adequate, and 100 = extremely prepared) how prepared Lipscomb students seemed to be compared to other university counseling students. They rated us above adequate at 66. Last year was a nearly identical rating at 65.5 (scores ranged from 22-96). Next, we asked what suggestions the respondents might have to help our students become more prepared. While there were not clear themes, respondents suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Other suggestions included more practical/ experiential/ role play experience and students demonstrating more flexible availability at their site.

Cooperation and Communication Between Lipscomb and Site. The program was rated much higher, however, when we asked about the cooperation between the Lipscomb program faculty and site supervisors (0 = poor, 50 = okay, 100 = excellent), which resulted in an average of 83.9. The average rating of respondents for communication between the Lipscomb program faculty and site supervisors was rated at 80.6. We asked for suggestions that could enhance communication. The largest category of those answers, 46% of responses, suggested nothing or keeping things the same. Interestingly, one said, “Sometime all the emails are overwhelming,” while another said, “Continue to send out the newsletter with opportunities for training for supervisors.”

Employability of Students. Finally, we wanted to know how confident site supervisors would be about hiring a Lipscomb graduate (0 = no, 50 = somewhat, 100 = absolutely). The average score indicated confidence in hiring Lipscomb students, at 75.5. Next, we asked why or why not. A few of the comments included:

48

● The students we have had have been great. We have carefully screened them for our program. While I think of

Lipscomb as a conservative university, the students we have had have been open to a variety of views and have worked with clients with a variety of backgrounds with openness and care.

● Students are professional and eager to learn. They appear to have a solid understanding of ethics and the

importance of building their therapeutic orientation.

● They seem to have the academic background and are more than willing to get the experience needed to work in

the field.

● We have hired some of your students.

Survey Question n=13 Average Score Last Year

Average Score

Compared to other counseling students from other universities at the start of practicum, how prepared do

Lipscomb Counseling students seem to you?

65.5 66

How would you rate the cooperation between the Lipscomb program faculty and you?

83.9 83.3

How would you rate the communication between the Lipscomb program faculty and you?

80.6 81.5

Given your impression of the Lipscomb University counselor training program, how confident would you feel hiring

Lipscomb graduates for your agency?

75.5 81.6

CPCE ExamEvery student takes the Counselor Preparation Comprehensive Examination (CPCE) during the Capstone class in the last semester of the program. The CPCE is designed to assess counseling students’ knowledge in eight core knowledge areas. Although the categories have slightly different titles, they closely correspond to the eight Core Areas of Knowledge identified by our accrediting body. The CPCE categories are: professional orientation and ethics, social and cultural foundations, human growth and development, career and life development, helping relationships, group work, appraisal, and research and program evaluation.

Our sample sizes were small (X in the fall and Y in the spring) and thus subject to extreme scores significantly influencing the mean. However, when comparing each group’s mean for each content area to the national mean (n=602), we were able to determine the trends which students demonstrated strength and areas needing improvement. Students, on average, scored above the national mean in the areas of group work, social and cultural foundations, helping relationships, research and program evaluation, and professional orientation and ethics. Students scored, on average below the national mean in appraisal, human growth and development, and career and life development.

Core Area LU Mean National Mean Difference

Group work 13.04 10.58 2.46

Social and cultural foundations 10.835 8.57 2.265

49

Helping relationships 12.54 10.32 2.22

Research and program evaluation 10.365 9.6 0.765

Professional orientation and ethics 12.46 11.97 0.49

Appraisal 8.71 10.1 -1.39

Human growth and development 10.885 12.37 -1.485

Career and life development 7.565 10.57 -3.005

Alumni SurveyIn order to gauge how effective the counseling program is at equipping current professionals who graduate from the program, as well as to collect data regarding licensure status and further education, we solicit alumni feedback. A link for an online survey is sent to all alumni of the counseling program for whom the university has contact email addresses. We seek to determine how alumni rate themselves on the qualities of caring, competence, commitment to the profession, and confidence in professional abilities. Further, we want to know what alumni consider the most beneficial aspect of the program. In addition, we ask alumni to list all licenses and credentials and if they have taken and passed the three exams required for licensure in Tennessee: the National Counselor’s Exam, the National Clinical Mental Health Counseling Exam, and the Tennessee Jurisprudence exam.

Respondents to the survey included 66 graduates of the graduate counseling program, both the professional counseling (the old program) and the clinical mental health counseling (the new program). From those respondents, 13 took the NCE, 5 took the NCMHCE, and 8 took the Tennessee Jurisprudence Exam, the three exams required for licensure in the state of Tennessee. All passed the NCE and NCMHCE, and five of eight passed the Tennessee Jurisprudence Exam.

We asked alumni to rate how instrumental the counseling program at Lipscomb was in developing the characteristics of caring, professional competence, commitment to the profession, and confidence in professional abilities. They evaluated each question from 1 (no influence) to 10 (very influential). In descending order, alumni rated competence (7.37) first, then caring (7.27), commitment to the profession (7.07), and confidence (6.27).

Alumni reported to us what they perceived as the most beneficial aspects of the program. These answers were grouped into three themes in order of significance: caring faculty and staff; courses, especially practicum and internship; and preparation for the National Counselor’s Exam and licensure. Some examples of the responses included:

Caring Faculty and Staff:

“The faculty's commitment and concern in my growth and development as a counselor.”

“Some of the professors who were so great at teaching and were passionate about the field.”

“…Teachers & staff members who were genuinely interested in a student's personal achievement & therefore made themselves readily available to help students succeed in assignments & classes when the student was struggling.”

“The availability of teachers and their willingness to connect personally with students and help however they could.”

50

Courses:

“The internship and practicum were by far the most beneficial, assuming you have great supervisors.”

Preparation for NCE Exam and Licensure:

“Good preparation for NCE and NCMHCE. Diagnosis covered well, ethics covered well…”

Employer SurveyOne of the best sources of feedback for how Lipscomb counseling graduates are equipped comes from those who employ them. We polled employers to learn how they view our graduates. The sample size was extremely small (n=5), but we did gather some valuable data.

The employers had hired from 0 to 6 graduates, with an average of 3.4. We asked them, “Compared to other counseling graduates from other universities, how prepared for the profession do Lipscomb Counseling graduates seem to you?” On a scale from 1 to 100, with 50 being equally prepared, the mean was 79 (range from 60 to 90). All respondents were moderately or extremely confident about hiring a Lipscomb counseling graduate.

Employers rated their Lipscomb counseling graduate employees on the four C’s from our mission statement Again, the sample size was small this year and last year, but employers rated us higher in all four categories:

This Year Last Year

● Caring 94.2 (90.33)

● Competence 85.4 (80.66)

● Commitment to the Profession 83.75 (78.22)

● Confidence 85.4 (74.55)

To summarize, employers rated this program’s graduates as well above average in preparedness, and all expressed confidence in hiring a Lipscomb graduate. They scored alumni very high in caring and high in competence, professional commitment and confidence.

Some of the comments included:

● At the beginning of the program, provide students with a clear picture of the licensing process and post-master's

job market including income potential, etc.

● Additional exposure to psychological assessment would be beneficial, including personality assessment (PAI,

MMPI) and cognitive assessment (WAIS, WASI).

● Students can benefit by knowing various intervention strategies for dealing with Anxiety, Depression, Addiction

etc.

External Evaluation of Program and Students Review

Strengths

● Knowledge Areas – Students scored well on CPCE in the areas of Group Work, Social and Cultural Foundations,

and Helping Relationships.

51

● Students as Supervisees – students demonstrate strong supervisee skills, especially with participation in

supervision, receiving suggestions and criticism, and being prompt and prepared; and students have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently.

● Preparation for the Profession – Alumni indicated that practicum and internship were helpful and that they felt

prepared for the National Counselor’s Exam and licensure; in fact most who attempted, passed the three exams required for licensure in Tennessee. Alumni and employers rated the competence of Lipscomb graduates just below their caring, and employers indicated that Lipscomb students are more prepared than students from other universities.

● Employability – Site supervisors report that they are highly confident about hiring Lipscomb counseling

graduates, and employers report they are moderately to extremely confident about hiring a Lipscomb counseling graduate.

● Cooperation and Communication – Lipscomb faculty supervisors cooperate and communicate well with clinical

site supervisors.

Areas of Improvement

● Professional Identity – Employers indicated an increase in Lipscomb alumni involvement in professional

activities. By requiring students to maintain membership in the American Counseling Association and encouraging involvement in other professional activities, the community is noticing more commitment to the profession.

● Assessment and Diagnosis – While still rating these categories lowest, site supervisors did not score Lipscomb

students as weak as last year overall in the assessment and diagnostics category.

● Student Preparedness – We did receive the level of feedback this year about students need for more experience

through role-plays and personal counseling experience. We now incorporate mock counseling sessions in the Introduction to Skills course and highly encourage students to seek their own personal counseling.

Areas to Improve

● Integrating Research – Site supervisors are not seeing strong evidence that our students are grounded in the

current research.

● Assessments and Interventions – Site supervisors reported students lower in taking appropriate risks in

treatment and making accurate diagnoses within 2-3 sessions. They also suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Employers also noted that assessment and intervention strategies could be improved.

● Core Knowledge Areas – CPCE scores indicate that students score lower than the national average in career and

life development, human growth and development, and appraisal.

● Student Confidence – Alumni and employers rate confidence lower or lowest of the 4 C’s.

52

Program Modifications Based on Review DataDuring a typical program cycle for each student (2 years, 7 semesters, and 3 semesters of clinical experience), we have implemented nearly 90 assessments of students, the program, and the faculty. The Eight-Point Student, Faculty, and Program Evaluation Plan provides a broad approach for assessment, including internal assessments of students, internal assessments of the program and faculty, and external assessments of the students and program. We have a strong vision combining our mission statement, eight core areas of knowledge, and student growth. The curriculum more than adequately meets accreditation standards, reflects our uniquely Lipscomb University mission, and offers a range of electives to help students gain specialized knowledge and experience, including international service trips and a play therapy specialization. By proactively seeking data from students, faculty, site supervisors, employers, alumni and others, we have a solid sense of the strengths of this counseling program and areas to improve.

Overall Program Strengths


● Quality Students – We accept a diverse group of students related to ethnicity and age. We have many

indications that Lipscomb counseling students are achieving academic excellence, dedicated to personal growth, and are developing well professionally. Most of our students receive excellent grades, recommendations to continue progressing, and good to excellent faculty clinical evaluations. Most students progress through the program without any need for a program improvement plan or other remediation. Our applicants to the program present themselves in alignment with our program philosophy: high in caring, competence, and commitment to the profession. The evaluation from the Introduction to Counseling Skills class echoes this, since students scored highest in caring and professional responsibility.

● Ethical and Professional Behavior – Students consistently demonstrate strong professional and ethical behaviors,

especially with tact, cooperation, and dependability. They also score high on valuing cross-cultural sensitivity and respecting alternative worldviews, along with effective communication skills.

● Therapeutic Interventions – Students have good interpersonal skills with clients and recognize their client’s

strengths. They also score high at maintaining expected caseloads at their sites.

● Feedback in Supervision – Students continue to participate well in supervision, and they averaged high scores in

appropriately giving and receiving feedback.

● Experience Conducting Groups – Students are spending more hours facilitating group work across all clinical

experiences.

● Above and Beyond Minimum Requirements – Students are dedicated to learning both in and out of the

classroom. Practicum students on average exceed the number of required hours by 142%.

Internal Evaluation of Program and Faculty

● Timely Completion – Most students (65%) complete the program in seven or eight semesters.

● Courses – Students, overall, scored course requirements, the course as a whole, and the organization of the

course highly.

53

● Professors – In evaluating the course professor, students rated their instructors patient and respectful, good at

relating the subject to the broader issue of life, and enthusiastic about the subject. They kept appointments and designated office hours and were approachable.

● Faculty Group Supervisors –Students have a strong sense of group safety, agree their supervisors improve their

skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at stimulating critical thinking, encouraging open discussions, applying the content to professional practice and relating the subject to broader issues of life. The supervisors are effective teachers, enthusiastic about their supervision and effective at open discussion.

● Orientations – The orientations for new students and Practicum continue to provide valuable information and

are helpful to students. The new student orientation, according to student feedback, provided valuable information and was well organized. It was perceived to be fun, friendly, warm and welcoming. At the Practicum orientation, students found the student panel, privacy and security information, and review of paperwork and documentation to be especially helpful.

● Knowledge – The content areas that students report feeling the most prepared for academically include:

Professional Orientation and Ethical Practice, Group, and Helping Relationships.

● Student Growth – Students perceive growth that continues throughout the program. Students reported high

levels of personal growth, academic growth, and professional growth at mid-program and even higher levels at the end of the program.

● Clinical Experiences – The Practicum and Internship experiences were the most often listed elements most

important in the program. Overall, students had positive experiences with their sites and site supervisors, especially when it came to supervisor availability and meeting weekly for supervision.

External Evaluation of the Program and Students

● Knowledge Areas – Students scored well on CPCE in the areas of Group Work, Social and Cultural Foundations,

and Helping Relationships.

● Students as Supervisees – students demonstrate strong supervisee skills, especially with participation in

supervision, receiving suggestions and criticism, and being prompt and prepared; and students have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently.

● Preparation for the Profession – Alumni indicated that practicum and internship were helpful and that they felt

prepared for the National Counselor’s Exam and licensure; in fact most who attempted, passed the three exams required for licensure in Tennessee. Alumni and employers rated the competence of Lipscomb graduates just below their caring, and employers indicated that Lipscomb students are more prepared than students from other universities.

● Employability – Site supervisors report that they are highly confident about hiring Lipscomb counseling

graduates, and employers report they are moderately to extremely confident about hiring a Lipscomb counseling graduate.

54

● Cooperation and Communication – Lipscomb faculty supervisors cooperate and communicate well with clinical

site supervisors.

Strengths Indicated from All Internal and External Evaluations1. Students – The students are clearly a strength of the program. We are recruiting and retaining diverse

and quality students who demonstrate strong professional and ethical behavior, are motivated to learn and do more than minimum requirements, and both give and receive feedback in supervision. They strongly rate personal, academic and professional growth at mid-program, and growth continues throughout the program. Most receive excellent grades and graduate with a minimum of 61 hours in 8 semesters or less. Most importantly, alumni pursue licensure, pass the required exams for licensure, and are viewed as employable and better prepared than other universities.

2. Cultural Diversity – The program values and admits diverse students, using a broad definition including (among other areas) ethnicity, age, and gender. Students score high on valuing cross-cultural sensitivity and respecting alternative worldviews at their clinical sites and indicated that Social and Cultural Diversity was on of the areas in which they felt most prepared. They also scored well in this area of the CPCE exam.

3. Clinical Experience – According to employers, alumni, current students, and site supervisors, students gain valuable experience in Practicum and Internship, including individual, group, and professional training experience.

4. Courses – Students are prepared academically for the profession. They rate courses highly as good overall and well-organized, and they pass national exams for state licensure at very high rates.

5. Faculty and staff – Relationships with program employees (full-time faculty, adjunct faculty, and office staff —many listed by name) are repeatedly listed as important and helpful. Professors are patient and respectful, good at relating the subject to the broader issue of life, and enthusiastic about the subject. Faculty group supervisors create group safety, improve supervisors’ skills, have a client focus, and maintain a strong rapport with their supervisees, and they cultivate cooperation and communication with site supervisors.

6. Core Knowledge Areas – Students continue to feel most prepared in the content areas of Professional Orientation and Ethical Practice, Group, and Helping Relationships.

7. Caring and Competence – Students and alumni of the program are, generally, viewed as demonstrating increasingly high levels of caring and counseling competence.

Overall Program Areas to Improve


● Confidence and Maturity – first semester students continue to score lower in this category.

● Reviewing Research – Although students read and consult current research for classes, the data indicates that

skill may not be fully used in their practitioner work.

● Defending Clinical Treatment Procedures – students create and follow treatment plans, but may lack the ability

to articulate the rationale and purpose.

55

Internal Evaluation of Program and Faculty

● Knowledge – Career and Assessment continue to be the lowest scoring content areas, and both had lower scores

than last year.

● Facilities and Locations – Students report a strong dislike for the classrooms and furniture in Ward. They also

reported mixed responses to the location of clinical supervision groups (inconsistency in scoring, which may reflect the different locations). Students also repeatedly mentioned parking challenges.

● Library – Students want more library resources, better electronic journal access through databases, and

individual and group graduate study places.

● Grading and Feedback – Returning graded material within a reasonable time and providing clear comments for

improvement consistently has lower scores. Students also indicated that professors are not always clearly communicating information and expectations.

● Integrating Technology – Another low-scoring question was about the importance of computer equipment to

learning. Professors may need to address integrating more relevant technology.

● Advising – Students mentioned that they would like better advising. While not a theme across assessments, this

was mentioned a few times.

● Program Benefits –Students indicated disapproval of the discrepancy in benefits between the mental health

counseling and marriage and family therapy programs.

● Student Challenges – Program faculty should be aware that students report struggling to balance school, work,

and personal life, and they feel the financial pressure of graduate school.

External Evaluation of Program and Students

● Integrating Research – Site supervisors are not seeing strong evidence that our students are grounded in the

current research.

● Assessments and Interventions – Site supervisors reported students lower in taking appropriate risks in

treatment and making accurate diagnoses within 2-3 sessions. They also suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Employers also noted that assessment and intervention strategies could be improved.

● Core Knowledge Areas – CPCE scores indicate that students score lower than the national average in career and

life development, human growth and development, and appraisal.

● Student Confidence – Alumni and employers rate confidence lower or lowest of the 4 C’s.

Areas to Improve Indicated by All Evaluations1. Incorporating Research – Students do not demonstrate high skill in incorporating research into their

clinical work and supervision.

2. Assessment, Diagnosis and Treatment Interventions – Students score lowest in these areas. They also score themselves lower in Assessment. Faculty and site supervisors reported that assessment and diagnosis was the area that needed the most improvement.

56

3. Student Confidence – Of the 4 C’s this was consistently lower. Beginning with the skills evaluation and continuing through alumni and employer evaluations, confidence, though adequate, is lowest ranking.

4. Core Knowledge Areas – Career was consistently indicated as the weakest knowledge area. Assessment and Testing also a low scoring knowledge area according to students and supervisors. Both areas slipped in student ratings compared to last year.

5. Facilities – Students reported on course evaluations and program evaluations that the classrooms in Ward were uncomfortable and unsatisfactory. They also indicated that conducting research for classes was difficult with current library resources, specifically access to online journals. Finally, parking was considered challenging.

6. Timely Grading and Feedback – Professors and group supervisors are not always timely in grading or clear in providing helpful feedback for improvement.

Plan to Address Challenges Taking what we have learned from all 26 different types of assessments and superimposing that information on the visual representation of the philosophy of assessment, we can easily our strengths and areas to improve. See Figure 9.

Figure 13: Philosophy of Program Assessment with Strengths and Areas to Improve

The Director of Program Assessment presented a synopsis of the results of all program assessments at the annual Stakeholder Program Review meeting comprised of core faculty, staff, students, site supervisors and stakeholders. (X

57

Academic Excellence & Christian Faithprofessional mental health counselors who

are:caring competent committedconfident

Growthacademic

professionalpersonal

8 Core Areas of Knowledgeprofessional orientation and

ethical practicegroup work helping relationshipssocial and cultural diversityhuman growth and developmentresearch and program evaluationassessment and testingcareer development

Purple indicates program strength

Red indicates area to improve

number of people) people attended. After presenting the results, the group discussed ways to address the challenges and suggestions for improvement. Based on the discussion, the following recommendations were suggested.

1. Incorporating Research – Every core course syllabi will be revised to add current research as required reading.

2. Assessment, Diagnosis and Treatment Interventions –Improve confidence through practice diagnosing; improve clinical interviewing skills; and practice intake interviews and reviewing paperwork.

3. Student Confidence – Faculty will strive to incorporate more classroom demonstrations. The program will subscribe to Alexander Street Press Videos so students have access to examples.

4. Core Knowledge Areas – The Career course should be revised to add a “This Is Your Career” component, exploring students’ own career concerns and options as a relevant way to learn the content. Business skills for private practice should also be included.

5. Facilities – The building is scheduled for remodeling this year. We will add Alexander Street Press video library to enhance current library resources.

6. Timely Grading and Feedback – Faculty will work on reducing turnaround time for assignments.7. Data Input Improvements – Continue to work with computer students to create a database that will manage

assessments, reduce paperwork, and improve data collection.

8. Advising – All students are required to meet with their advisor the first semester. Students will be encouraged to meet any other time they have registration or career questions.

9. Discouragement about Student Loans and Finances – Intentionally build confidence about skills and diversity of job options; offer specialized training and workshops to improve marketability.

Conclusion Lipscomb University’s Graduate Studies in Counseling Program is proud of the highly competent and caring students graduating from our program and seeking licensure. They are perceived as qualified, above average when compared to other university counseling students, and employable. Our faculty and staff are an important part of the success of our program. The academic component is strong, as evidenced by students’ performance on external standardized tests. Students who complete the requirements for a degree from this program are well prepared as professional counselors.

This report has been a description of how we conceptualize our program, gather and analyze data, and learn from the information gathered. We value feedback from students, site supervisors, employers, alumni, and stakeholders, and we have made and will continue to make adaptations to improve our program. Our program not only prepares students for a career in the counseling profession, but also to change lives, beginning with their own.

58

Some study data were collected and managed using REDCap electronic data capture tools hosted at Lipscomb University.1 REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.

1Paul A. Harris, Robert Taylor, Robert Thielke, Jonathon Payne, Nathaniel Gonzalez, Jose G. Conde, Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform. 2009 Apr;42(2):377-81.

59

ww2.lipscomb.edu - prog… · web viewmission statement and program goals. the mission for the...

Documents