the new blueprint: challenging the comfort zone
DESCRIPTION
Blueprint project update - 2014 Annual MeetingTRANSCRIPT
1
BLUEPRINT PROJECT
September 14, 2014
Beyond the Point-in-time Assessments for High-Stakes Decision Making
MCC Annual General Meeting
2
Plenary Objectives
• Outline the progress of the Blueprint project since AGM 2013 – Claire Touchie
• Review assessment and validity frameworks for high-stakes decision making – Dame Lesley Southgate
• Propose models for combining formative and summative assessments for high-stakes decision making – André De Champlain
The New BlueprintChallenging the Comfort Zone
4
Purpose of the Blueprinting
• … is to assure the public that physicians licensed to practice medicine have the required knowledge, skills and attitudes for safe and effective patient care.
• Only those who meet this standard are qualified to enter professional practice
5
• ensure that critical core competencies, knowledge, skills and behaviors required of a physician entering supervised and unsupervised practice are being appropriately assessed
The process will
• ensure that MCC assessments continue to fulfill all the requirements and standards for credentialing examinations
• provide a clearly documented and deliberate process to
• update exam specifications • respond to ongoing developments in the profession
Project Objectives
Stakeholder Consultations Overview
7
Recommended Common Blueprint
Dimensions of Care
Health Promotion and
Illness Prevention
Acute Chronic Psychosocial Aspects
Physicia
n Activitie
s
Assessment/Diagnosis
Management
Communication
Professional Behaviors
8
External ConsultationsGovernance Board July 5, 2013
UGME Meeting June 20, 2013
CaRMS
August 22, 2013
Annual General Meeting
Sept. 15-17, 2013
AFMC – CACMS September 23, 2013(11h45 – 15 mins)
AFMC – Council of Deans October 30, 2013(in am – 15 mins)
Collège des médecins du Québec
(CMQ)
October 31st, 2013
RCPSCAssessment Committee
November 7, 2013
UBC
November 12, 2013
Student & Residents
(2 meetings)
November 18, 2013 (Meeting 1 of 2)
November 25, 2013 (Meeting 2 of 2)
FMRAC
November 26, 2013
(9:00 a.m. – 2:00 p.m.)
CFPC
November 27th, 2013(a.m.)
RCPSCEducation Committee
November 27, 20131:00 – 2:00 p.m.
PGME
November 27th, 2013(7 – 9 p.m.)
IMG Symposium January 14, 2014
9
Initial Reactions
What have we heard?
• Overall support for the Blueprint dimensions and definitions as proposed
• Importance of language: – supervised means different things to different folks– Residents do certain activities unsupervised
• Need to make explicit certain terms:– Teamwork, patient safety, inter- and intra-professionalism, leadership– Concerns about the term for and placement of the dimension
“Psychosocial Aspects”
• Relevance of the D2 (and part II) when certification is the final step to licensure
• All CanMEDS roles can be mapped to each dimension• Focused on challenges and opportunities
10
MCC Common BlueprintIncorporating Consultation Feedback
12
Recommended Common Blueprint
Dimensions of Care
Health Promotion and
Illness Prevention
Acute Chronic Psychosocial Aspects
Physicia
n Activitie
s
Assessment/Diagnosis
Management
Communication
Professional Behaviors
Definitions
Dimensions of Care Focus of care for the patient, family, community and/or population.
Health Promotion and Illness Prevention
The process of enabling people to increase control over their health and its determinants, and thereby improve their health. Illness prevention covers measures not only to prevent the occurrence of illness such as risk factor reduction but also to arrest its progress and reduce its consequences once established. This includes but is not limited to screening, periodic health exam, health maintenance, patient education and advocacy, and community and population health.
AcuteBrief episode of illness, within the time span defined by initial presentation through to transition of care. This dimension includes but is not limited to urgent, emergent, and life-threatening conditions, new conditions, and exacerbation of underlying conditions.
Chronic Illness of long duration that includes but is not limited to illnesses with slow progression.
Psychosocial AspectsPresentations influenced by the social and psychological determinants of health and how these can impact on wellbeing or illness. The determinants include but are not limited to life challenges, income, culture, and the impact of the patient’s social and physical environment.
DefinitionsPhysician Activities Reflects the scope of practice and behaviors of a physician practicing in
Canada
Assessment/ Diagnosis
Exploration of illness and disease through gathering, interpreting and synthesizing relevant information that includes but is not limited to history taking, physical examination and investigation.
ManagementProcess that includes but is not limited to generating, planning, and organizing safe and effective care in collaboration with patients, families, communities, populations, and other professionals (e.g., finding common ground, agreeing on problems and goals of care, time and resource management, roles to arrive at mutual decisions for treatment, working in teams).
Communication
Interactions with patients, families, caregivers, other professionals, communities and populations. Elements include but are not limited to relationship development, intraprofessional and interprofessional collaborative care, education, verbal communication (e.g.: using the patient-centered interview and active listening), non-verbal and written communication, obtaining informed consent, and disclosure of patient safety incidents.
Professional Behaviors
Attitudes, knowledge, and skills based on clinical and/or medical administrative competence, communication ,ethics, as well as societal and legal duties . The wise application of these behaviors demonstrates a commitment to excellence, respect, integrity, empathy, accountability and altruism within the Canadian health-care system. Professional behaviors also include but are not limited to self- awareness, reflection, life-long learning, leadership, scholarly habits and physician health for sustainable practice.
15
Dimensions of Care
Health Promotion and Illness Prevention
Acute Chronic Psychosocial Aspects
Row Percent
Physician
Activities
Assessment/Diagnosis 30±5
Management 20±5
Communication 30±5
Professional Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
Assessment leading up to Decision 1: Entry into Residency
16
Dimensions of Care
Health Promotion and Illness Prevention
Acute Chronic Psychosocial Aspects
Row Percent
Physician
Activities
Assessment/Diagnosis 25±5
Management 35±5
Communication 20±5
Professional Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
Assessment leading up toDecision 2: Entry into Independent Practice
TransitionsMCCQE Part I and Part II
18
Classification of MCC Content
3 days and 26 physicians
• 3514 Multiple-choice items
• 1321 Clinical Decision Making items
• 103 OSCE cases
19
Gap Analysis Results
MCCQE Part IDIMENSIONS OF CARE
Health promotion and Illness Prevention
Acute Chronic Psychosocial Aspects
TOTAL
PHYSICIAN ACTIVTIES
Assessment/ Diagnosis
151 1654 715 33 2555
3.1% 34.2% 14.8% 0.7% 52.9%
Management 235 1068 467 27 1797
4.9% 22.1% 9.7% 0.6% 37.2%
Communication 34 53 22 28 137
0.7% 1.1% 0.5% 0.6% 2.8%
Professional Behaviours
60 82 130 57 329
1.2% 1.7% 2.7% 1.2% 6.8%
TOTAL480 2857 1335 145 4834
9.9% 59.1% 27.6% 3.0% 100%
Gap Analysis Results
MCCQE Part II
20
DIMENSIONS OF CARE
Health promotion and Illness Prevention
Acute Chronic Psychosocial Aspects
TOTAL
PHYSICIAN ACTIVTIES
Assessment/ Diagnosis
0.2% 37.2% 8.4% 2.5% 48.3%
Management 2.8% 7% 2.3% 0.6% 12.7%
Communication 6.5% 23% 2.9% 2.0% 34.4%
Professional Behaviours
0.6% 3.2% 0.3% 0.5% 4.5%
TOTAL10.1% 70.4% 13.9% 5.6%
21
Transition of Exams
• MCC Part I– MCQs and CDM items– Spring 2017
• MCC Part II– New station types being piloted– Focus on communication and professional
behaviors within a clinical context– Still will expect components of history-taking and
physical examinations– Fall 2018
Our Work Going ForwardAssessment Evolution – Challenging the Comfort Zone
23
To meet overall Blueprint
• Will need additional assessments– Opportunities– Innovations– Collaborations
Consultation Input - Opportunities
Harmonize with surgical foundations exam
A UG ITER
Influence accreditation standards wrt assessment expectations
Self assessment
Standardize mini-CEX across schools
Encourage discussions with both colleges and related initiatives
Common technical skills assessment tools
Standardize ITER/FITER to include as assessment tool
E-portfolio rollout
Standardize faculty of medicine OSCEs
Harmonize with specialty examinations at the Royal College
0 5 10 15 20 25 30 35 40 45 50
0
1
1
2
5
10
15
19
23
28
45
24
25
Assessment Evolution - Opportunities
26
UGME Retreat
4 priority areas to be explored:• Assessment Design & Creation – Content• Assessment Design & Creation – Design• Item Banks• E-portfolios
27
Moving forward
• Meeting with “opportunity leads” to move forward with the UGME recommended opportunities
• Environmental scans– UGME assessments– Assessment Evolution meeting/FMEC-PG
recommendation #6 meeting
Tomorrow’s workshops
29
Workshop 1
An Assessment E-Portfolio across the Continuum: A Discussion of the “What”
• Discuss the types of assessments to be
included in an e-portfolio for the two MCC decision points
• Propose mechanisms for this to be feasible and acceptable for schools, candidates and regulators
30
Workshop 2
Feeding Forward: From A Student-Centered to Patient-Centered Approach
• Define the meaning and understanding of
“feeding forward” • Discuss opportunities and barriers to sharing
of assessment information within and between stakeholder organizations for the purpose of quality improvement and learning
31
Blueprint Project Team
The Blueprint Project Team
Dr. Claire Touchie Chief Medical Education Advisor – Project co-lead
Cindy Streefkerk Consultant – Project co-lead
Tanya Rivard Senior Test Development Officer and Special Projects
Anna Di Medio Project Administrator
Dr. Sydney Smee Manager, Strategic Initiatives and Clinical Skills Assessment
Jessica Hertzog-Grenier Director of Communications
Ingrid de Vries Associate Director – Evaluation Bureau
Becca Carroll Manager - Computer-Based Testing
Dr. Andrea Gotzmann Research Psychometrician
Dr. Marguerite Roy Medical Education Researcher
Sasha Papayanis Business Analyst – Information Technology
Alex Clay Project Coordination Manager
Thank You!