T223 Gearing up for AccreditationVIRGINIA L. SNYDER, PhD, PA-C and MICHELLE OSTMOE, BS

Objectives …share our experience and lessons learned in preparation for accreditation

…identify program-developed self-assessment tools

…discuss the benefits of utilizing PAEA services geared toward application review and a mock site visit

Gear up!

General Tips… no tricks!▪Guided by the Standards, but also by your Mission and Vision ▪Develop a solid infrastructure ▪Recognize from the get-go that this is a “work in progress” ▪Understand the lay of the land:

▪ Application with all of its appendices (numbered) vs. Self Study Report (Appendix 13 with its own lettered appendices) vs. Required List of Materials for Site Visit

▪Speak directly to the Standards- fully address all ANNOTATIONS ▪ Integrate courses taught outside program into the assessment

process, i.e. Anatomy and MMI (Infectious Disease of Human Beings)

General Tips▪Be okay with redundancy ▪ Refer to appendices or on-site materials ▪ Example: B2s

B2.14 The program curriculum must include instruction about PA licensure, credentialing and laws and regulations regarding clinical practice. Provide Narrative describing how the program demonstrates and reviews compliance with B2.14

This curriculum is taught in: PA616/PA618/PA716 Professional Seminar I/III/IV, PA717/PA718 Professional Seminar and Capstone I/II.

Curriculum mapping demonstrates learning outcomes in this instructional area in the above courses and will be available to site visitors. Course directors are responsible for learning outcomes within courses. Oversight by the Academic Director on an ongoing basis reviews compliance of overall curricular content.

General Tips▪ ‘Syllabi, Instructional Objectives, Learning Outcomes’ (ARC-PA document) ▪ Evidence-based accreditation:

▪ Refer to the ARC-PA Accreditation Manual for examples of what data sets will demonstrate compliance with the standards

▪ http://www.arc-pa.org/acc_standards/Accreditation_Manual.html

Standard Evidence Suggestions/Performance Indicators

B2.07 The program curriculum must include instruction in technical skills and procedures based on current professional practice.

- List of technical skills and procedures taught. - Curriculum outlines, course syllabi, handouts,

lecture notes, data reporting hands-on practicum time allotted to specific activities.

- Student evaluation mechanisms that relate to the development, application and evaluation of these skills.

General Tips▪Collect proper and adequate data ▪Ensure valid and reliable measurement tools ▪Be reasonable in approaches to analysis:

▪ Trends, cause & effect, comparisons, e.g. internal and external (national data per PAEA Annual Report)

▪ Quantitative vs. qualitative analysis

Trends Year to Year Analysis

Cause & Effect Comparisons

Notable trends upward/downwardNominal Group Technique

Internal, e.g. similar health care programsAffinity Diagrams

Note modifications made along the continuum

Fish Bone Analysis External, e.g. comparison to national data per PAEA Annual

Report

Tool: Fishbone Analysis re: Improved Pharmacology Instruction

Tool: Policy Template

UW-Madison PA Program Policy

Policy Title: ______________________________________________________ Origin Date: _______Last Revised: ________Proposed Review Date: ________ Responsibility (Person/Title responsible): ______________________________ Approving Committee: _____________________________________________ Category: ________________________________________________________ ARC-PA Associated Standard: ________________________________________ Purpose:

Background:

Policy Statement:

Approved by (committee, date):

N:\ADMIN\paprog\OPERATIONS\POLICIES\

Tool: Policy Tracking Template

A Standards (Administration)A2.03 Principal faculty must be sufficient in number to meet the academic needs of enrolled students. ▪ FTE determination and comparison to national data

Tool: State of Program (SOP) Reports

A Standards (Administration)A2.04 Principal faculty and the program director should have academic appointments and privileges comparable to other faculty with similar academic responsibilities in the institution. ▪ Based on the institution and policies ▪ Addressed by printing off all faculty appointments in Department and

showing comparison; compared to other health care professional programs

Policies Governing Faculty Appointments University of Wisconsin School of Medicine and Public Health

August 1996 with updates as of September 2006

A Standards (Administration)A2.10 The program director must supervise the medical director, principal and instructional faculty and staff in all activities that directly relate to the PA program. ▪ Organization chart, reporting structure

A2.16/B3.06 All instructional faculty serving as supervised clinical practice experience preceptors must hold a valid license that allows them to practice at the clinical site. ▪ Policy/process for verification  

Policy Title: Verification of Preceptor Board Certification and Licensure Policy Origin Date: 11/25/13 Last Reviewed: 11/24/14 Proposed Review Date: Annually (or as needed) Responsibility (Person/Title responsible): Clinical Coordinator Approving Committee: Faculty Committee Category: Clinical Year ARC-PA Associated Standard: A2.16, B3.06 (4th edition)

A Standards (Administration)A3.14 The program must define, publish and make readily available to enrolled and prospective students general program information to include:

a) the program’s ARC-PA accreditation status, In order to be in compliance with A3.14a, the program must publish the appropriate wording below for its accreditation status, exactly as it is presented:

Accreditation Continued Statement The Accreditation Review Commission on Education for the Physician Assistant (ARC-

PA) has granted Accreditation-Continued status to the [Enter program name] sponsored by [Enter institution.] Accreditation-Continued is an accreditation status granted when

a currently accredited program is in compliance with the ARC-PA Standards…

b) the success of the program in achieving its goals, In order to be in compliance with A3.14b, the program must first define and then publish its goals on the web site. The expectation is that these will be published and readily available on the program’s web site.

Notes to Programs

A3.14 Cont’d. The program must define, publish and make readily available to enrolled and prospective students general program information to include:

c) first time PANCE rates for the five most recent graduating classes, In order to be in compliance with this standard A3.14c,

▪ The NCCPA PANCE Pass Rate Summary Report of the most recent five-year first time graduate performance, unedited as provided by the NCCPA must be published on the program website.

▪ The report should reflect the most recent graduating class that has taken the PANCE.

▪ The expectation is that this document will be easily recognizable from the home page of the program web site, in a category related to graduate outcomes, not hidden within other categories. If the link connects to a separate web page in the site, the PANCE report must be readily evident.

Compliance judged by review of program web page.

This is a good example of paying attention to the ANNOTATION and making sure all points are addressed.

Goals and Outcomes…

B Standards (Curriculum)B1.03 The curriculum must be of sufficient breadth and depth to prepare the student for the clinical practice of medicine. and B1.07 The curriculum must include instruction related to the development of problem solving and medical decision-making skills. and B1.04 The curriculum design must reflect sequencing that enables students to develop the competencies necessary for current and evolving clinical practice.

ANNOTATION: The concept of sequencing refers to the coordination and integration of content both horizontally and vertically across the curriculum. It does not mandate that content be delivered in separate courses with traditional discipline names. Appropriate sequencing involves considering overall program design and integration of content. Content and course sequencing are expected to build upon previously achieved student learning.

How did we address this?

Bloom’s Taxonomy http://smg.photobucket.com

theenglish planner.wordpress.com

L1 L2 L3

Tool: Curriculum Mapping

Tool: Curriculum Mapping

Analysis: B1.03 (breadth & depth), B1.04 (sequencing), and B1.07 (development of problem solving and medical decision-making skills)

Analysis: B1.03 (breadth & depth)

Learning Outcomes - % Mapped to Competencies

B1.08 and Appendix 16 of Application

B1.08 The curriculum must include instruction to prepare students to work collaboratively in interprofessional patient centered teams.

• ANNOTATION: Such instruction includes content on the roles and responsibilities of various health care professionals, emphasizing the team approach to patient centered care beyond the traditional physician-PA team approach. It assists students in learning the principles of interprofessional practice and includes opportunities for students to apply these principles in interprofessional teams within the curriculum.

Appendix 16 All expected student competencies/learning objectives for the following areas:

• physical diagnosis (B2.05) • pharmacology (B2.02d) • interprofessional patient centered teams (B1.08) • family medicine (supervised clinical practical experience) (B3.07a)

“If course syllabi are submitted as part of the evidence…”

Tool: Appendix 16 - Annotated document outlining longitudinal curriculum from multiple courses and activities

Also included: ▪ Typical learning objectives that illustrated

the 7 specific skills

▪ Integrative Cases

▪ Cases of Patients

▪ MEDiC Clinic

▪ Interprofessional Panels

▪ Leadership Conference for students

B3 Standards – Supervised Clinical PracticeB3.02-B3.07 ▪Formal processes ▪ Identify any “worry areas”, e.g. do you have a unique approach to

fulfilling clinical rotations (B3.07)? ▪Data becomes more critical (rotations, preceptors, evaluations,

licensure, etc.), e.g. preceptor database, patient encounter logging ▪Emphasis on assessment, outcomes, comparisons (C3, C4) ▪Using small team – analyze, reanalyze, look at question in every

way possibleTool: Patient Encounter Logging System

Tool: Patient Encounter Log

AnalysisAvg. number of Patient Encounters

50

93

135

178

220

Peds

209216

189

Class of 2012 Class of 2013 Class of 2014

Peds Encounters by Individual Students

0

150

300

450

600

0 10 20 30 40

Tool: Ongoing Evaluation of Curriculum

C Standards (Evaluation)

C1 Ongoing Program Assessment – Appendix 13 C2 Self Study Report (SSR) – Appendix 13 C3 Student Evaluation

C3.01 Complete ARC-PA TEMPLATE for Appendix 14 and Provide Narrative describing how and when program conducts frequent, objective and documented evaluations of students related to learning outcomes for both didactic and supervised clinical education components.

Appendix 14

C Standards (Evaluation)

C3.02 The program must document student demonstration of defined professional behaviors. ▪Policy ‘Nonacademic Conduct/Misconduct Policy’ and

‘Academic and Professionalism Evaluation Policy’ ▪Faculty/Staff Meetings – ‘Student Progress and/or Concerns’ ▪Promotions and Progress Committee

Tool: Behavioral Indicators of Professionalism

Behavioral Indicators of Professionalism

▪ Built into the ‘Academic and Professionalism Evaluation Policy’

▪ Discussed at New Student Orientation ▪ Attached as an appendix in each

course syllabus ▪ Employs self and faculty evaluation ▪ Program expectation: ‘Meets

Expectations’ ▪ Professionalism evaluated at mid-

semester (didactic) and End of Rotations (clinical)

C Standards (Evaluation)C3.04 The program must conduct and document a summative evaluation of each student within the final four months of the program to verify that each student is prepared to enter clinical practice.

ANNOTATION: Evaluation products designed primarily for individual student self-assessment, such as PACKRAT are not to be used by programs to fulfill the summative evaluation of students within the final four months of the program. The ARC-PA expects that a program demonstrating compliance with the standards will incorporate evaluation instrument/s that correlates with the didactic and clinical components of the program’s curriculum and that measures if the learner has the knowledge, interpersonal skills, patient care skills and professionalism required to enter clinical practice.

Provide Narrative describing the program’s summative evaluation process, indicating its timing during the program and what occurs if a student does not meet defined performance expectations.

C Standards (Evaluation)C3.04 Summative Evaluation Policy

▪When summative evaluation is administered ▪ Defined minimal performance ▪ Four components - as described in ARC-PA ANNOTATION – and examples of

assessment methods used for each component (e.g. written exam, practical exam, OSCEs, Behavioral Indicators) ▪ Medical Knowledge ▪ Interpersonal Skills ▪ Patient Care Skills ▪ Professionalism

▪ “What if” the student does not meet the minimal performance expectation? ▪ Documentation of successful completion of summative evaluation in student

file

Summative Exam mapped to PANCE

C Standards (Evaluation)C3.05 The program must document equivalency of student evaluation methods and outcomes when instruction is: a) conducted at geographically separate locations

b) provided by different pedagogical and instructional methods or techniques for some students.

Provide Narrative as to how program demonstrates compliance.

Analysis: ▪ State of Program Reports, e.g.

course grades, PACKRAT, PANCE

▪ Placement and discipline ▪ Comparisons to other cohorts

Tool:

C4 Clinical Site EvaluationC4.01 The program must define, maintain and document effective processes for the initial and ongoing evaluation of all sites and preceptors used for supervised clinical practice experiences to ensure that sites and preceptors meet program defined expectations for learning outcomes and performance evaluation measures. Provide Narrative describing program’s process for the initial review of all sites and for the ongoing review of all sites. ▪ Policies:

▪ Evaluation of New Preceptorship Sites and Preceptors ▪ Ongoing Evaluation of Preceptorship Sites and Preceptors

▪ Site Intake Form, Mid-point and Final Site Critiques, Patient Encounter Logging

C4 Clinical Site EvaluationProvide Narrative detailing the number, type and frequency of actions that have been taken to improve or dismiss sites over the past two clinical rotation years.

Tool: Student/ Preceptor Issue Tracking Guide

C Standards (Evaluation)SSR – Appendix 13A through 13N ▪Appendix 13F – Preceptor Evaluations

▪ Match template

▪Appendix 13I – Sufficiency and Effectiveness of Faculty and Staff ▪Appendix 13L – Faculty Evaluation of the Curriculum and

Administrative Aspects of the Program ▪Appendix 13M – Modifications as a Result of Self Assessment

▪ Tied to Standards; but also ‘big picture’ (tied to mission) ▪Appendix 13N – Strengths and Areas in Need of Improvement

▪ Somewhat balanced

Continuous Performance Management System (CPMS)

Tool: CPMS Organizational Chart

CPMS Chart

Tool: CPMS Report for Documentation (w/ Analysis)

CPMS Report

PAEA Evaluation ServicesReview of Application ▪ Timing – prior to mock site visit; remember busy schedules ▪ Adequate time to incorporate recommendations ▪ Consistency in writing style (e.g. “written by committee”), succinctness,

polished product

Mock Site Visit: PAEA ▪ Invaluable! ▪ Assisted in anticipating areas of ARC-PA focus, e.g. remediation policy,

academic appointments ▪ Emphasized importance of institutional oversight ▪ Suggestions for additional data and analyses that could be added to

files, e.g. data triangulation ▪ “Walk through steps of student engagement of program from initial

interaction throughout program (and beyond).”

Lessons Learned▪ Have clear program expectations, goals ▪ Read things literally! ▪ Importance of assessment measures and what to choose to best

demonstrate outcomes ▪ Importance of curriculum mapping and continuous updates ▪ Identify Team Leaders, e.g. writing team, graphics team ▪ Revise evaluation forms (e.g. preceptor evaluations) to best match the

criteria in Application/Appendix charts ▪ Modifications- ongoing documentation of changes ▪ Communication! ▪ Organization is key! Presentation of materials – binders, rolling file w/

index

How do you Gear UP?

References▪ Accreditation Manual: Accreditation Standards for Physician Assistant Education.

4th edition. Accreditation Review Commission in Education for the Physician Assistant. Web site: http://www.arc-pa.org/acc_standards/Accreditation_Manual.html. Published 2010. Latest revision, April 2015. Accessed June 1, 2015.

▪ Accreditation Resources. Web site: http://www.arcpa.org/acc_programs/acc_resources.html. Updated May 27, 2015. Accessed June 1, 2015.

▪ Competencies for the Physician Assistant. American Academy of Physician Assistants. Web site: https://www.aapa.org. Adopted 2005. Reaffirmed 2010. Amended 2013.

▪ Payne D, Snyder V. Improving Curricula by Labeling Lecture Objectives with Physician Assistant Competency and Bloom’s Taxonomy Codes. Workshop presented at: PAEA Annual Education Forum; November 4-8, 2009; Portland, Oregon.

▪ Essary AC, Statler PM. Using a Curriculum Map to Link the Competencies for the PA Profession With Assessment Tools in PA Education. J Physician Assistant Educ. 2007; 18(1):22-28.