afmc.ca t… · web view · 2016-03-24association of faculties of medicine of canada. 2733...

GUIDE TO THEMEDICAL SCHOOL SELF-STUDY

FOR MEDICAL EDUCATION PROGRAMSLEADING TO THE M.D. DEGREE

FOR SITE VISITS IN THE 2017-2018 ACADEMIC YEAR(Published March 2016)

For further information, contact:CACMS Secretariat

Committee on Accreditation of Canadian Medical SchoolsAssociation of Faculties of Medicine of Canada

2733 Lancaster Road, Suite 100Ottawa, Ontario, Canada K1B 0A9

Phone: 613-730-0687 Ext 225 Fax: [email protected]

Visit the CACMS website at:https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms

Guide to the Medical School Self-StudyFor medical education programs leading to the M.D. Degree

©Copyright March 2016 by the Committee on Accreditation of Canadian Medical Schools. All rights reserved. All material subject to this copyright may be reproduced, with citation, for the noncommercial purpose of scientific or educational advancement.

i

https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms

mailto:[email protected]

TABLE OF CONTENTS

OVERVIEW OF THE ACCREDITATION PROCESS................................................................1A. Purposes of Accreditation and Self-Study..................................................................1B. Accreditation Standards.............................................................................................1

GENERAL STEPS IN THE ACCREDITATION PROCESS.......................................................2A. Completion of the DCI and Compilation of Other Documents....................................3B. Self-Study Analysis and Final Report Development....................................................4C. The Accreditation Visit and Preparation of the Visit Report........................................4D. Accreditation Decisions and Follow-Up......................................................................5

TYPICAL SCHEDULE FOR A CACMS FULL ACCREDITATION REVIEW..............................5A. Assistance from the CACMS Secretariat.....................................................................7

COMPLETING THE DATA COLLECTION INSTRUMENT (DCI).............................................8A. Supporting Documentation........................................................................................8B. Date Range..................................................................................................................

8C. Updates .................................................................................................................9

CONDUCTING THE MEDICAL SCHOOL SELF-STUDY.........................................................9A. The Self-Study Task Force..........................................................................................9B. Subcommittees of the Task Force..............................................................................9C. Preparation of the Final Medical School Self-Study Report.......................................10

EVALUATION OF ELEMENTS..............................................................................................11A. Instructions..................................................................................................................

11B. Comment Field - Evidence to Support the Rating.....................................................11C. Rating the Elements.................................................................................................13D. Comment Field - Recommendations to Address Identified Problems.......................13

COMPONENTS OF THE SELF-STUDY REPORT.................................................................14

ii

OVERVIEW OF THE ACCREDITATION PROCESS A. PURPOSES OF ACCREDITATION AND SELF-STUDY

Obtaining accreditation from the Committee on Accreditation of Canadian Medical Schools (CACMS) and the Liaison Committee on Medical Education (LCME) ensures that medical education programs are in compliance with defined standards. The accreditation process has two general and related aims: to promote medical school self-evaluation and improvement, and to determine whether a medical education program meets prescribed standards.

As a process of evaluation, accreditation seeks to answer three general questions:

1. Has the medical school clearly established its mission and goals for the educational program?

2. Are the program's curriculum and resources organized to meet its mission and goals?

3. What is the evidence that the program is currently achieving its mission and goals and is likely to continue to meet them in the future?

The medical school self-study process and the resulting findings are central to these aims. In the process of conducting its self-study, a medical school brings together representatives of the medical school administration, faculty, student body, and other constituencies to: 1) collect and review data about the medical school and its educational program, 2) identify areas that require improvement, and 3) define strategies to ensure that any problems are addressed effectively.

The report resulting from the self-study process provides an evaluation of the quality and effectiveness of the medical education program and the adequacy of resources to support it. The usefulness of the self-study as a guide for planning and change is enhanced when participation is broad and representative, when the results and conclusions are widely disseminated, and when the participants have engaged in a thoughtful process of analysis and reflection. Because of the time and resources required to conduct a self-study, schools should give careful thought to other purposes that may be served by the process. For example, the self-study might serve as a vehicle to familiarize a new dean, dean’s staff member or department chair with the environment and operation of the school; to initiate a curriculum review; and/or to provide the academic community at large with an opportunity to reaffirm the school’s educational mission and goals or set new strategic directions for the medical education program. A self-study process that serves multiple purposes and involves multiple constituencies is more likely to have a productive outcome related to medical school improvement than one that is conducted solely to satisfy accreditation requirements.

B. ACCREDITATION STANDARDS

The self-study is directly linked to the standards for accreditation. The standards for accreditation of Canadian medical schools are contained in the CACMS publication CACMS Accreditation Standards and Elements (S&E).

Medical schools with accreditation visits during the 2017-2018 academic year will use the corresponding version of S&E. These standards and elements have been widely reviewed and endorsed by the medical education community, including the organizations that sponsor the CACMS.

Medical schools are expected to achieve compliance with each of the 12 standards. Guide to the Medical School Self-Study (AY 2017-2018)

Page 1

Compliance with a standard will be based on satisfactory performance in the elements associated with the standard. See “Accreditation Decisions and Follow-Up” below.

GENERAL STEPS IN THE ACCREDITATION PROCESS Information provided by the medical school is considered by the medical school, the site visit team and lastly the CACMS in the context of accreditation standards. The general steps in the process are as follows:

1. Completion of the data collection instrument (DCI) and compilation of supporting documents.

2. Analysis of data from the Independent Student Analysis (ISA), the most recent graduation questionnaires, and the DCI, narrative responses and appendices, by a medical school self-study subcommittees and task force, development of self-study reports for each standard, and compilation of these updated reports into a final medical school self-study report.

3. Visit by an ad hoc site visit team and preparation of the site visit report for review by the CACMS.

4. Action on accreditation by the CACMS and LCME.

Each of the steps is summarized below and in the accompanying schedule, which shows the usual timetable for completion of each step.

Medical School Site Visit PersonnelThe Dean must designate a core team of faculty and staff to manage the aspects of the site visit preparation process. The faculty accreditation lead manages the data collection and self-study processes; the site visit coordinator typically manages site visit logistics, and may assist with data collection. It is critical that both positions be staffed by individuals who have a deep understanding of the program and who will be able to work well with key individuals within and external to the medical school. Designated personnel will need the authority and experience to gather accurate information and garner widespread participation among faculty, staff, and students. Please refer to the full position descriptions below before making these designations. Schools must complete the site visit personnel designation form no later than six weeks following publication of the DCI for their respective site visit year. This will ensure that the appropriate school personnel receive updates and event notifications from the CACMS Secretariat.

Faculty Accreditation LeadThe faculty accreditation lead should be a senior faculty member, who may also hold a position of vice, associate or assistant dean or other leadership position in the medical school, who is knowledgeable about the medical school and its educational program. This individual should be able to locate medical school or university policies and information sources, explain medical school conventions, and ensure participation by members of the senior administration, faculty, and student body. Ideally, the faculty accreditation lead will be familiar with CACMS site visit processes, and will have served on a site visit team as the designated faculty fellow for his or her school.

The school must ensure that the faculty accreditation lead has appropriate administrative support, and release time from other duties in order to accomplish the responsibilities associated with this role. The faculty accreditation lead will be required to:

Guide to the Medical School Self-Study (AY 2017-2018)Page 2

• Answer questions during DCI preparation• Assign specific questions/sections of the DCI to individuals with the appropriate

knowledge• Ensure factual accuracy and typographical/grammatical clarity in the DCI• Ensure that each aspect of multi-part DCI questions are fully-addressed • Synthesize all narrative DCI responses into a cohesive, factually and stylistically-

consistent document that accurately reflects the medical school • Coordinate the activities of self-study subcommittees• Staff the self-study task force• Develop the site visit schedule in collaboration with the site visit team secretary• Serve as the school’s primary point of contact for the CACMS Secretariat and site

visit team secretary

Site Visit CoordinatorThe site visit coordinator should be an experienced, senior administrative staff member who will manage the logistics of the site visit and other administrative functions such as formatting and submitting the site visit package. The site visit coordinator will normally make hotel reservations for the team, coordinate ground transportation for the visit, and schedule the necessary faculty and staff identified for sessions during the site visit.

A. COMPLETION OF THE DCI AND COMPILATION OF OTHER DOCUMENTS

The questions in the DCI are directly linked to specific elements. The questions should be answered and the relevant documents compiled by the persons most knowledgeable about each of the topics. Care should be taken to ensure that the data and terminology are current, accurate, and consistent across the DCI (e.g., consistent abbreviations, consistent names and abbreviations for committees). The faculty accreditation lead who oversees the accreditation process at the school should ensure that the completed DCI undergoes a comprehensive review to identify any inaccuracies, missing items, or inconsistencies in reported information. The absence of a document, data, and/or information specifically requested in the DCI will be taken to mean that the document, data, and/or information do not exist.

Data from the independent student analysis, the most recent graduation questionnaire, and internal sources should be reviewed by the relevant self-study subcommittees and utilized in the development of the individual subcommittee reports and the final medical school self-study report.

While the DCI is being completed, medical students will carry out their own review of the educational program, student services, and other areas of relevance to students. While the administration may provide logistical support and assistance in analyzing the data, planning for the student survey and the interpretation of the results is a student responsibility. Students should be directed to the CACMS publication: Guide to the Independent Student Analysis (AY 2017-2018).


B. SELF-STUDY ANALYSIS AND FINAL REPORT DEVELOPMENT

The medical school self-study task force and its subcommittees are responsible for conducting the self-study. The project as a whole should be guided by the faculty accreditation lead. Each subcommittee should review and analyze the ISA, the most recent graduation questionnaires, and the DCI data, narrative responses and appendices for the elements to which they have been assigned. In the majority of cases, the relevant documentation to evaluate the element has been requested in the DCI itself. The subcommittees carry out the first evaluation and rating of elements, provide evidence to support the rating, and develop recommendations with timelines to address areas of unsatisfactory performance. This process is described in more detail later in the guide. Subsequently, the task force analyzes the subcommittee reports, along with the supporting documentation and any new information, and revises as appropriate the individual subcommittee element ratings. The task force updates the evidence to support the rating and the final recommendations to address elements where performance is unsatisfactory and elements requiring monitoring. The task force creates the final self-study report that includes all of the element evaluation forms and the taskforce’s reflection on areas requiring improvement identified during the self study in relation to areas requiring improvement identified at the last full site visit and the intervening period. The components of the self-study report are described in detail later in this guide.

The medical school self-study report (in Word format), the ISA, the most recent graduation questionnaires and the completed DCI (in Word format) with its appendices need to be submitted to the visiting team three months prior to the visit as described on the CACMS website.

C. THE ACCREDITATION VISIT AND PREPARATION OF THE VISIT REPORT

A full visit typically begins on Sunday evening with an entrance conference with the dean and concludes early Wednesday afternoon at exit conferences with the dean and with the university president or delegate. Schools with distributed campuses may have an additional day added to the visit. If, during the visit, the dean has concerns regarding the conduct of the visit, he or she should contact the CACMS Secretariat immediately. Prior to the visit, the visiting team will review the materials submitted by the school in detail. At the time of the visit, the school will provide copies of these documents, as well as the individual self-study subcommittee reports, to the visiting team in print and electronic formats.

During the visit, the team will develop a list of its findings that relate to specific elements. These summary findings will be reported orally to the dean and the university official on the final day of the visit and a written copy of the team findings related to the elements will be provided to the dean. These initial findings may be revised during the process of review of the visit report. The visiting team makes neither recommendations nor decisions regarding the medical school’s accreditation status; the determination of accreditation status is the purview of the CACMS and the LCME.

After the visit, a draft report is prepared by the visiting team according to the format specified in the Site Visit Report Guide. The report evaluates the information in the DCI, the self-study report, the independent student analysis, as well as information obtained by the team during the visit, and presents the team's findings from the visit. The visit report will include the team’s judgment about findings related to elements, which will be categorized as: 1) satisfactory, 2) satisfactory with a need for monitoring, and 3) unsatisfactory.

A draft version of the report is sent by the team secretary to the CACMS Secretariat office for a preliminary review to verify that the report is complete and adequately documents the team’s findings. It is then returned to the team secretary who, in consultation with the team, Guide to the Medical School Self-Study (AY 2017-2018)

Page 4

will produce the draft that will be forwarded to the dean for review. The dean has 10 business days to respond to the draft report in writing (in hard copy and/or electronic format) with areas he or she believes contains errors of fact or concerns about the “tone” of the report. Information provided as part of the dean’s response must be referenced to information contained in the Data Collection Instrument or provided to the team during the visit and must refer to the time of the visit. Events occurring or actions taken by the school after the visit will not be considered in mitigation of the findings of unsatisfactory or satisfactory with a need for monitoring identified in the visit report. The dean’s comments about the site visit report are sent to the team secretary with a copy to the CACMS Secretariat. The team secretary will submit the final report to the CACMS Secretariat with a copy to the dean. Following receipt of the final report, if the dean has remaining concerns about the process of the visit, errors of fact or the tone of the report, he or she may write a letter to the CACMS Secretariat detailing these concerns within 10 business days. The information referenced must have been contained in the Data Collection Instrument or provided to the visiting team at the time of the visit. No new information, regarding events or actions taken by the school after the visit may be provided in the dean’s letter to the CACMS Secretariat and no attachments to the letter will be accepted. The dean’s letter will be provided to the CACMS when the visit report is reviewed by the committee.

D. ACCREDITATION DECISIONS AND FOLLOW-UP

The site visit report is reviewed by the CACMS at its next regular meeting (in September, January, or May), at which time a formulated decision about the program’s accreditation status and follow-up is made. Subsequently this formulated decision will be reviewed by the LCME at its next regularly scheduled meeting (October, February or June) and a final CACMS/LCME decision on the accreditation status and follow-up will be determined. Accreditation may be granted or renewed for a period of eight years, however the program may be given an indeterminate or shortened term. As a condition for granting or renewing accreditation, the CACMS may 1) require that the dean submit one or more written status reports; 2) schedule a limited visit; 3) direct its Secretariat to conduct a visit for consultation or fact-finding; or 4) order another full visit before the completion of the eight-year term. If major problems have been identified, the CACMS may continue accreditation with no term specified pending the results of a follow-up visit or to place the program on warning status or on probation status. The CACMS/LCME may withdraw accreditation if such problems are not corrected within a reasonable period of time or if problems are identified during a visit that indicate that the program is not adequately preparing medical students to enter the next phase of training or that the program is not sustainable for any reason.

TYPICAL SCHEDULE FOR A CACMS FULL ACCREDITATION REVIEW

Months -/+ Visit Activities

-18 CACMS Secretariat establishes site visit dates with the medical school dean.

A committee of students responsible for the Independent Student Analysis (ISA) is formed and begins developing a few additional questions for the ISA survey of the student body.

-15 ISA survey is distributed to the student body. Note that data from the ISA survey are needed for completion of the DCI, so the survey should be timed accordingly.


-15 The CACMS Secretariat publishes the DCI on the CACMS webpage.

Dean designates the school’s core visit personnel and notifies the CACMS using the form provided by the CACMS Secretariat

The faculty accreditation lead initiates data collection activities.-15/-12 School appoints members of the medical school self-study task force. The task

force establishes its objectives, scope of study, and methods of data collection, and establishes various subcommittees.

The students charged with conducting the ISA provide survey data to the faculty accreditation lead and begin independent analysis of the data.

Various individuals or groups begin responding to questions in the DCI.

-12/-6 Students provide the final ISA to the faculty accreditation lead. Faculty accreditation lead distributes the ISA report and completed DCI sections to the self-study task force and appropriate subcommittees. Subcommittees review and analyze the relevant sections and prepare reports that are forwarded to the task force.

If not begun already, action should be taken to correct issues identified by the various subcommittees.

-4/-3 The CACMS Secretariat sends the faculty accreditation lead instructions for the visit and a final list of visiting team members is sent to the dean.

The faculty accreditation lead reviews the DCI, self-study summary report, and other required documents and makes any required updates/corrections.

-3 The final accreditation package, consisting of the DCI and supporting documentation, the ISA report, and the Medical School Self-study Report is submitted according to the instructions available on the CACMS webpage, https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms/cacms-publications

-3/2.5 Shortly after receiving the school’s accreditation materials, the secretary of the visiting team will contact the faculty accreditation lead to begin work on the visit schedule and will contact the staff visit coordinator to discuss logistical planning.

The faculty accreditation lead drafts a visit schedule based on the sample visit template in this document and sends it to the team secretary for review.

Based on initial review of the accreditation package, the team secretary may request additional information/materials and/or that additional sessions with specific faculty or staff be added to the schedule.

-2 If necessary, corrections and/or updates to the DCI are bundled and sent to the visiting team secretary following the procedures outlined on the CACMS webpage


https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms/cacms-publications

https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms/cacms-publications

The team secretary and school finalize the visit schedule.-1 If necessary, a final set of bundled corrections and/or updates to the DCI are

bundled and sent to the visiting team secretary following the procedures outlined on the CACMS webpage

0 Team visits the school.

The exit conference is conducted.

The faculty accreditation lead submits one bundled update to the CACMS Secretariat containing any supplementary material provided to the team before or during the visit, and any corrections or updates provided to the team after the initial submission (at -3 months). This includes updates/corrections made at the time of the visit. These are submitted electronically on a USB memory stick sent to the CACMS Secretariat by mail.

+1/+2 The team secretary sends a first draft of the report to the CACMS Secretariat for review; Secretariat feedback is incorporated as seen fit by the team into a second draft, which is sent to the dean for review.

+1/2 (+10 days) The dean provides feedback; feedback is incorporated into the final report at the discretion of the team secretary and chair.

+2/3 The report is finalized. The team secretary sends the final report to the Dean and to the CACMS Secretariat. The final report is circulated by the Secretariat to CAMCS members for review prior to the next CACMS meeting.

+3/4 The CACMS determines an accreditation decision at its next regularly scheduled meeting (January/May/September).

CACMS decision is reviewed by LCME at its next meeting. Final joint decision is rendered.

Within 30 days of LCME meeting

The university president/chief executive and medical school dean are sent copies of the final report and are notified, in writing, of the final decision regarding accreditation status and any required follow-up.

A. ASSISTANCE FROM THE CACMS SECRETARIAT

Schools are encouraged to contact the CACMS Secretariat at any time via email or telephone, and to attend the preparation sessions available to schools with upcoming visits. These include an annual Visit Preparation session at the Canadian Conference on Medical Education (CCME). These sessions provide general information about accreditation and the self-study process and give participants an opportunity to discuss specific issues with members of the Secretariat. Designated school personnel will automatically receive invitations to these events.

COMPLETING THE DATA COLLECTION INSTRUMENT (DCI) The DCI is organized according to the 12 CACMS accreditation standards:


Standard 1: Mission, Planning, Organization, and IntegrityStandard 2: Leadership and AdministrationStandard 3: Academic and Learning EnvironmentsStandard 4: Faculty Preparation, Productivity, Participation, and PoliciesStandard 5: Educational Resources and InfrastructureStandard 6: Competencies, Curricular Objectives, and Curricular DesignStandard 7: Curricular ContentStandard 8: Curricular Management, Evaluation, and EnhancementStandard 9: Teaching, Supervision, Assessment, and Student and Patient SafetyStandard 10: Medical Student Selection, Assignment, and ProgressStandard 11: Medical Student Academic Support, Career Advising, and Educational

RecordsStandard 12: Medical Student Health Services, Personal Counseling, and Financial Aid

Services

The DCI for a given year is normally available from the CACMS Secretariat at least 15 months prior to the visit. The faculty accreditation lead should distribute sections of the DCI (by standard, element, or even question) to those individuals best able to provide accurate and current information. A first draft of the DCI should be completed and returned to the faculty accreditation lead within two or three months. The faculty accreditation lead will then review the DCI responses to ensure the information is complete, and accurate.

Some tables in the DCI (e.g., DCI financial) will be completed by the AFMC and provided to the medical school for verification. Subsequently the data will be provided to the CACMS Secretariat who will insert the tables into the Draft Site Visit Report and provide it to the team secretary approximately three months prior to the onsite visit.

A. SUPPORTING DOCUMENTATION

Provide the DCI with all of the appendices to the visiting team on a USB drive organized according to the instructions from the CACMS Secretariat.

B. DATE RANGE

Provide data for all of the requested academic years. While the self-study should consistently focus on data from a specific period of time i.e., the most recently completed academic year before the site visit, the DCI should be completed with all requested historical data. The time period covered by the data should be clearly indicated.

Because the DCI will likely have been prepared nine months or more before the visit, certain quantitative information must be updated prior to submission. The visiting team will need current student enrollment data, updates on changes in the educational program, and any other significant new information. These updates should be made before the DCI is finalized and submitted (i.e., three months before the


scheduled visit). The AFMC will update the financial information, student enrollment, faculty and resident numbers with the most recent information provided by the medical school. Schools are responsible for updating the responses to other questions, as needed.

C. UPDATES

Updates or corrections made to the DCI after the accreditation package has been submitted (3 months before the visit) should be bundled and sent to the visiting team so that they may be incorporated into the visit schedule and priorities. It is recommended that updates be consolidated and sent to the team in batches not more than twice prior to the visit (e.g., -2 and -1 month). Note that this does not include supplemental material requested by the team.

CONDUCTING THE MEDICAL SCHOOL SELF-STUDY A. THE SELF-STUDY TASK FORCE

The self-study process requires the time and effort of the medical school’s educational leadership, faculty members, students, administrative support staff and others associated with the medical school, its clinical affiliates, and, if relevant, its parent university.

The ultimate responsibility for conducting the self-study and preparing the final self-study report rests with the self-study task force, as supported by the faculty accreditation lead. This group determines the objectives of the self-study, sets the timetable for the completion of all related activities, and finalizes the medical school self-study report.

The self-study task force should be broadly representative of the constituencies of the medical school. It should, therefore, include some combination of the following: medical school senior and administrative leaders (academic, fiscal, managerial), department chairs and heads of sections, junior and senior faculty members, medical students, medical school graduates, faculty members and/or administrators of the general university, representatives of clinical affiliates, and trustees (regents) of the medical school/university. Additionally, the task force could include graduate students in the basic biomedical sciences, residents involved in medical student education, and community physicians. Although the general guidelines about the composition of the task force should be followed, each school must make its own decisions about membership based on its specific environment and circumstances. The self-study task force might be chaired by the dean or by a vice dean, senior associate dean, department chair, or senior faculty member. The medical school should provide administrative staff assistance to facilitate the timely completion of task force work.

B. SUBCOMMITTEES OF THE TASK FORCE

Various subcommittees should be appointed to evaluate the 12 standards. Each standard should be addressed by a subcommittee, however one subcommittee may be given responsibility for several standards. For example, there could be a subcommittee that has responsibility for the standards related to medical students (standards 10, 11, and 12). Schools may wish to create additional subcommittees to review specific topics, either to undertake a more detailed review or to accommodate unique medical


school needs. For example, a school with distributed campuses may want to create a separate subcommittee to review each campus, or a school with a particularly strong research mission may want to create a subcommittee to review the relationship of that mission to the medical education program.

Each subcommittee should have appropriate membership, including vice, associate, or assistant deans, faculty members, and, where appropriate, students. It is helpful to have one or more members of the task force serve on each subcommittee in order to provide continuity and to facilitate communication. Each subcommittee should review the relevant portions of the DCI and evaluate the elements as described later in this guide. Subcommittees may wish to collect other data germane to their area(s) of responsibility.

As described previously, students will manage an independent review of the medical education program, following the guidelines described in the document entitled Guide to the Independent Student Analysis. The subcommittees responsible for standards dealing with medical student services, the educational program, and facilities should refer to the results from the independent student analysis during their deliberations. These components of the Independent Student Analysis are explicitly referenced by element in the DCI.

The subcommittees systematically evaluate whether the specific requirements of each element are being met by the medical school. The subcommittees will likely need two or three months to complete their data gathering, review and analyses. Each subcommittee will evaluate and rate the elements of their assigned standards using the element evaluation forms provided in the individual MSS – Element Evaluation Forms for each standard. See the detailed description below of the information that must be provided to explain the rating of the element (evidence that supports the rating) and recommendations to address unsatisfactory elements and those requiring monitoring. The subcommittee reports consist of the element summary table for each assigned standard followed by the individual evaluation pages for all the elements in numeric order. The reports should be forwarded to the task force chair or the faculty accreditation lead. There should be separate reports for each of the twelve standards.

C. PREPARATION OF THE FINAL MEDICAL SCHOOL SELF-STUDY REPORT

It is the responsibility of the task force to analyze all of the subcommittee evaluations of the elements including those elements that were rated as satisfactory. In conducting its review of the subcommittee reports along with the supporting documentation (e.g., ISA, DCI), and any new information, the task force revises as appropriate, the individual subcommittee element ratings. The task force updates the evidence to support the rating and the final recommendations to address elements where performance is unsatisfactory and elements requiring monitoring. The task force makes the final determination of the element ratings and recommendations to address elements that are rated as unsatisfactory or require monitoring.

The task force subsequently completes the color-coded element summary table for all standards (MSS Element Rating Summary Table). Following the summary table, the evaluation forms for each standard of all elements are included in numeric order. See the detailed description below of the information that must be provided to explain the rating of the element (evidence that supports the rating) and recommendations to address unsatisfactory elements and those requiring monitoring.

Members of the subcommittees and the self-study task force may find it helpful to refer to the CACMS Site Visit Report Guide, the publication used by visiting team members when compiling the visit report (CACMS website).


The final MSS report is submitted as part of the accreditation package three months prior to the visit. Print copies of the individual subcommittee reports must be provided to the visiting team in the team’s workroom during the visit, but should not be submitted with the accreditation package.

EVALUATION OF ELEMENTS A. INSTRUCTIONS

The element evaluation forms are linked directly to specific CACMS accreditation standards and elements as contained in the corresponding academic year version of the CACMS Standards and Elements document. The standard appears at the beginning of each section followed by the elements that comprise the standard. Under each element, a series of statements describe the requirements for the element. Each subcommittee should review and analyze the DCI data, narrative responses and appendices for the elements to which they have been assigned. The relevant questions from the AFMC Graduation Questionnaire (AFMC GQ) and the ISA pertaining to a specific element are provided in the DCI for that element. Subcommittee members should also read the ISA Summary to identify concerns of the students that are relevant to the elements to which they have been assigned. If the school operates one or more geographically distributed campuses, an analysis of the circumstances at these sites needs to be considered when relevant in the evaluation of whether the requirements are being met.

B. COMMENT FIELD - EVIDENCE TO SUPPORT THE RATING

A comment field follows the element rating wherein an explanation for the rating is given. A comment must be provided for each requirement statement. For each requirement statement the self-study subcommittee and subsequently the task force should refer to specific parts of the DCI (tables or narrative responses/documents (by Appendix and page number) or other information collected by the school (referenced to an Appendix) as evidence to support the rating of the element. The explanation for some elements may be a statement of fact(s) e.g., element 1.6 Eligibility Requirements, and element 6.8 Duration of the Program. In most other cases the evidence may include a reference to the DCI (tables or narrative responses/documents (by Appendix and page number), or other information collected by the school. In general, the comment field for each element should be approximately one half-page in length. The explanation may also make note of any circumstances specific to the medical school that was considered in determining the rating. In the event that a consensus cannot be reached on the element rating, differing viewpoints should be included.

i. Elements rated as Satisfactory

The comment field should provide evidence that the school is meeting each requirement statement as described above. The subcommittee and later the taskforce will refer to specific parts of the DCI (tables or narrative responses/documents (by Appendix and page number) or other information collected by the school as evidence to support the rating of the element.

ii. Elements rated as Satisfactory with a need for Monitoring

For elements rated as Satisfactory with a need for Monitoring (SM), the comment field will show that the school has only recently met one or more requirements of the element and has about 6 months of preliminary data showing that the system or process is effective. For some elements e.g., 3.3 Diversity, pipeline programs and partnerships, no preliminary data Guide to the Medical School Self-Study (AY 2017-2018)

Page 11

may be available to determine effectiveness in which case the steps taken by the school (policies and practices) if recent may be sufficient to rate the requirement as satisfactory with a need for monitoring. The element may also be assigned satisfactory with a need for monitoring if all of the requirement statements are being met however known circumstances exist that may cause the school not to meet the requirements. In both cases, the comment field will also provide evidence that the other element requirements are being met by the school. The self-study subcommittee should copy the requirement statement on the element evaluation form that has only recently been met and paste it into the comment field. Then this statement should be revised to describe the situation at the school. For example to address the problem of students not receiving formal mid-point formative feedback as required by element 9.7 Timely Formative Assessment and Feedback, the DCI showed that 12 months ago the medical school developed a policy that requires faculty to provide formal feedback at least at the mid-point of each required learning experience and approximately every six weeks for the clinical skills course that is one year long and for the longitudinal integrated clerkship they provide. The medical school now has administrative data showing that feedback has been provided to students for the last 6 months. In this case, the subcommittee or task force would copy and revise requirement 9.7 c, and add the underlined sentence at the end of the statement as illustrated below.

The medical school recently implemented a new system to ensure that: a) Formal feedback occurs at least at the mid-point of each required learning

experience and

b) Formal feedback occurs approximately every six weeks for the clinical skills course that is one year long and for the longitudinal integrated clerkship.

The medical school has administrative data showing that the system has been effective for the last 6 months. (do not include the underlining in the comment).

If the element was rated as Noncompliance or Compliance with a need for Monitoring at the time of the last full site visit or the intervening period, a note to this effect should be added at the end of the comment box.

iii. Elements rated as Unsatisfactory

The comment field will show that the school is not meeting one or more of the requirement statements. There should be a comment for each requirement statement i.e., those that the school is meeting as well as those that are not being met. For requirement statements that are not being met by the school, the self-study should copy the statement(s) under the element (on the MSS Element Evaluation Form) that is not currently being met by the school and paste it into the comment field. Then the statement should be revised from positive to negative and modified to describe the circumstances at the school. For example, for element 4.4 Feedback to Faculty, in the case where the self-study subcommittee found that not all faculty members receive feedback as required by the element (i.e., faculty in the departments of internal medicine and surgery do not receive regularly scheduled and timely feedback) the subcommittee would cut and paste statement 4.4 a, from the element evaluation page and paste it into the comment field for Element 4. Then the sentence would be revised to reflect the specific situation at the medical school. The statement might read: “Faculty members in the departments of internal medicine and surgery do not receive regularly scheduled and timely feedback from departmental and/or medical education program or university leaders on his or her academic performance and progress toward promotion and, when


applicable, tenure at each campus (underlining provided to indicate the change in wording).” If the element was rated as Noncompliance or Compliance with a need for Monitoring at the time of the last full visit or the intervening period, a note to this effect should be added at the end of the comment box.

C. RATING THE ELEMENTS

Based on the evaluation of the elements described above, the self-study subcommittee and later the taskforce rates the element as one of the following:

1. Satisfactory (S): All the requirement statements are being met by the school;

2. Satisfactory with a need for Monitoring (SM)*:a) the medical school currently meets the requirement statements e.g., has the

required policy, process, resource, or system in place, however there is insufficient outcome data demonstrating that it is effective, OR

b) the medical school currently meets the requirement statements of the element, however known circumstances exist that may cause the medical school not to meet the requirements of the element;

* for some but not all elements

3. Unsatisfactory (US):One or more of the requirement statements are not being met by the school

D. COMMENT FIELD - RECOMMENDATIONS TO ADDRESS IDENTIFIED PROBLEMS

If the rating of the element was unsatisfactory, a recommendation to address the identified issue and the timeline to achieve a satisfactory rating should be provided in the appropriate box on the element evaluation form.

If the rating is satisfactory with a need for monitoring as noted in 3.a) in the previous paragraph, provide a description of, and the date when, the process or policy was put in place, a description of preliminary data showing that the requirements are met, and a description of the outcome data that will be used to demonstrate effectiveness of the process. If the reason for the rating of satisfactory with a need for monitoring was 3.b) noted in the previous paragraph, describe the existing circumstances and recommendations to address any challenge to meet the requirement of the element.


COMPONENTS OF THE SELF-STUDY REPORT 1. TITLE PAGE

2. TABLE OF CONTENTS

3. FINAL COLOUR-CODED MSS ELEMENT RATING SUMMARY TABLE

4. INTRODUCTION

a. Prior accreditation historyInsert the table provided by the CACMS Secretariat listing the standards and the status of the standards described above, the type and timing of follow-up and the status of the medical education program since the time of the last full visit. The data presented in the table has been reformatted into the language of the new elements. In one page or less, briefly summarize the steps taken to address areas of noncompliance with accreditation standards and areas in compliance with a need for monitoring identified at the time of the previous full visit.

b. Description of the self-studyIn one page or less, provide a brief overview of how the medical school self-study was conducted, and the level of participation by the various members of the academic community, including students. In the appendix section, insert a table listing the membership (names and positions) of the self-study task force and its subcommittees.

5. EVALUATION OF ELEMENTS

Insert the updated element rating table and the element evaluation forms for each of the 12 standards in numeric order.

6. TASK FORCE SUMMARY STATEMENT

Limit this component of the report to no more than two pages.

Briefly summarize the elements rated as unsatisfactory and those requiring monitoring. Were areas of non-compliance and compliance requiring monitoring identified at the last full visit, identified as unsatisfactory elements or elements requiring monitoring during the self-study? If so, what factors contribute to persistent problems in these areas and what strategies will the medical school use to address them?

Note any challenges that may be contributing to unsatisfactory performance in more than one element within or across standards. How were these challenges considered by the task force in developing the final recommendations to address unsatisfactory elements and those requiring monitoring?

7. APPENDIX TO THE MSS

List members (names and positions) of the self-study task force and its subcommittees.
