Practical Tips for identifying Unperceived needs

Sam J Daniel, MD, FRCSC Director Pediatric Otolaryngology,

McGill University

Disclosures

• Speaker Abbott • Advisory panel Merck

Objectives At the end of this workshop participants will be able to:

• Describe the differences between perceived and

unperceived needs • Utilize strategies to identify practice / performance

gaps of physicians • Explain the barriers and challenges to identifying and

integrating unperceived needs within CPD planning processes

Learning need

The gap in • knowledge, • skill, • attitude, • practice between what currently exists and what is desired.

Needs assessment

• The tool that we use to identify and measure the gap.

• should be conducted to identify perceived and unperceived educational needs of the target audience.

• Intended to identify an absence or lack of necessary knowledge or skill that creates a variance between current practice and best practice activities.

• The needs assessment process should be used to identify the content and format of the educational event.

Perceived Needs

• I know what I want and/or need to know

Unperceived needs

• I don’t know

what I don’t know and/or need to know

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we don’t know. But there are also unknown unknowns. These are things we don’t know we don’t know. Donald Rumsfeld, US Secretary of Defence, 2002

Foot in the Mouth Award Winner 2003

Applies to any field

Johari Window

Known to Self Unknown to Self

Known to Others Open Arena Blind Spots

Unknown to Others Facade

Unknown (Hidden depths)

Perceived needs

Determining perceived needs

• Surveys • Solicited topics- open-ended questionnaires • Interviews • Focus group • Planning committee • Evaluations from previous CPD events • Requests from the target audience

Unperceived needs

Determining unperceived needs

• Can be very difficult. • Unperceived needs are outside the awareness

of the learner. • if someone is not aware of a need to learn

something new it is unlikely that they will. • Also referred to as “objective needs”.

Determining unperceived needs

• Assessment activities provide physicians with tools and a process that generates data and provides feedback to facilitate an assessment of multiple aspects of a competence and performance in relation to external standards.

• This allows us to identify previously unperceived needs.

Physician self-assessment Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006 Sep 6;296(9):1094-102.

The ability of physicians to accurately evaluate their own knowledge, skills and performance without external measures is limited.

Physician self-assessment

Perhaps of greatest concern are the findings that those who perform the least well by external assessment also self-assess less well.

Determining unperceived needs

• Knowledge test • Simulation • Chart audit • Critical incident • Expert advisory group • Patient feed-back • Quality assurance data from hospitals • Combo

Knowledge test:

• Learners are asked to respond to questions that test their knowledge.

• These can be multiple choice quizzes, a true/false quiz, a presentation of cases with questions or any variation thereof.

• An Audience Response System can be used in large group settings.

Interactive case:

• Presenting a case with numerous opportunities for participants to reflect on their own practice vs. what is recommended.

Disorienting case:

• Presenting a “disorienting dilemma” to the audience in which something that is commonly believed to be true is proven to be false (or vice a versa).

Performance assessment

• Can occur in a simulated environment or in the actual practice environment of an

• individual (personal performance, assessment), • group of physicians or inter-professional health

team (collective performance assessment).

Broad range of situations

• clinical, • education, • research, • administrative practice.

Simulation:

• Simulation activities reflect a broad range of fidelity (from standardized patients and joint models to programmed mannequins) and can be designed to be administered through the web or face to face.

• Simulations lead to effective learning by providing feedback during the learning experience and by enabling physicians to identify unperceived needs.

Virtual Patients

“Specific type of computer program that simulates real-life clinical scenarios; learners emulate the roles of health care providers to obtain a history, conduct a physical exam, and make diagnostic and therapeutic decisions.”

Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Medical education. 2009;43(4):303-11.

Virtual Patients

• Variety of VPs – Linear vs. branched – Individual vs. group – Synchronous vs. asynchronous Huwendiek S, De Leng BA, Zary N, et al. Towards a typology of virtual patients. Medical Teacher. 2009;31(8):743-748.

SimuCase Virtual Patient

Assessment (examples) CanMEDS role Individual Team System/ resources

Medical Expert Knowledge, pt assessment, mgmt, problem-solving…

Problem-solving, task prioritization…

Resources for assessment & mgmt & support…

Communicator Share mental model, listen, assertiveness, elicit info…

Synthesis into shared care plan…

Systems for communication

Collaborator Own role, delegation, leadership, conflict resolution…

Role adaptivity, synthesis, situational awareness, conflict resolution

Policies on roles/ responsibilities, support for collaboration…

Health Advocate Recognition & system activation…

System response mechanisms, barriers to access care…

Manager Own role, leadership, priority-setting, resource allocation…

Synthesize care plan, resource optimization…

Support & barriers to team-based care…

Scholar Insight into limitations, peer feedback…

Insight into limitations…

Professional Ethical practice, interprofessional attitudes…

Chart audit

• Systematic examination of patient charts looking for patterns of care that can be appropriate or inappropriate.

• Expensive and complex to perform properly.

Critical incident:

• Information using this technique can be gathered by using a hypothetical case situation.

• Can take place as a review of clinical records after an important event such as a missed diagnosis has occurred.

Duplicate prescription/ health care diary:

• Prescriptions are filled out in duplicate.

• Patterns in prescribing and their effectiveness can be determined over time.

• Can be asked to keep a practice diary for a few weeks before a program and after a program related to specific topics.

Patient feedback:

• Patients are asked to fill in surveys or respond to interviews about specific aspects of their care.

• Forums / discussion boards Rate MD etc.

Quality improvement structures

• Mortality & Morbidity Rounds: – Learning from group discussions – Learning from ‘others’ mistakes

• Quality improvement committees

• Incident reports

Expert Advisory group:

• Experts often know a great deal about the unperceived needs of other health professionals because they receive patients as a result of referrals.

• The referrals can give a lot of information on unperceived learning needs of the target audience.

• Experts are asked many questions by HP which can point to learning needs quite effectively.

Data from other sources • Self-assessment programs. • CPD reporting tools – Mainport. • National examination boards. • CMPA. • Litigation expert consultants. • Quality assurance/audit data. • Provincial databases. • National benchmarks.

Data from other sources

• Epidemiological data. • Re-credential review. • Statistics Infection control data. • Surgical procedures statistics. • Professional society requirements. • News media.

Published literature

• Journal articles/literature citations • Random controlled trials, cohort studies • Clinical guidelines • Developments in scientific research

Inferred needs

• Development of new technology • New methods of diagnosis or treatment • Availability of new medications • New therapeutic indications • Experts’ input re advances in medical knowledge • Acquisition of new facilities or equipment • Legislative, regulatory or organizational changes

with direct effect on patient care

Challenges to identifying unperceived needs within CPD planning processes

Challenges to integrating unperceived needs within CPD planning processes

Tough questions to be asked by the CPD planning committee

• How important is the need among the target audience?

• How many different assessment sources have validated this need?

• How significantly will the unfulfilled need or knowledge gap hinder health care delivery?

• How directly is the need related to actual healthcare provider performance?

Tough questions to be asked by the CPD planning committee

• How likely is it that a CPD activity will change behaviour? • How likely is it that a CPD activity will improve performance? • How likely is it that the CPD activity will translate into

improved healthcare outcomes? • Are sufficient resources available to effectively address this

need? • How receptive will the target audience be to a session on

this particular topic?

Conclusion

• Unperceived needs should be a priority for every CPD planner.

• Multiple strategies exist to identify practice / performance gaps of physicians.

Sam.daniel@mcgill.ca