combining breadth with depth: validation of … · combining breadth with depth: validation of...
TRANSCRIPT
Combining Breadth with Depth:
Validation of Competencies in ClinicalResearch and Medicines Development
Honorio Silva, MD
Joint Task Force for Clinical Trials Competence, USA.International Federation of Associations of Pharmaceutical Physicians and
Pharmaceutical Medicine (IFAPP) The Netherlands
Rutgers University, School of Health Related Professions and Rutgers School of Business
EMBL Conference: Life-long learning in the biomedical sciences
Heidelberg, July 5-7, 2016
Learning Objectives:
Following completion of this session attendees will be able to:
• Explain the evolution of clinical research and medicines development from activity to profession and the need to define (and validate) professional competencies
• Explain the rationale for developing and implementing the Joint Task Force (JTF) framework of Core Competencies in Clinical Research and the process and outcomes of its initial validation.
• Explain the ongoing process for validation of competencies in pharmaceutical medicine and medicines development
• Describe the regional differences in the perception of competence and relevance of competency domains.
Activity
• Informal training: coaching, tutoring
• Short term courses: how to do it!
• Professional bodies (training, support)
Discipline
• Academic involvement: Standards and competencies
• Formal Curriculum: short and long term programs
• National accreditation and certification
Profession
• International Standards/Harmonization of Training/mutual recognition
• International Certification/Specialization
• Maintenance through CPD
Evolution of Education and Training in Clinical Research and Medicines Development
Honorio Silva MD, 2010
Drug
Target ID
Discovery
Pre-Clinical
Phase I
Trials
Phase II
Trials
Phase III
Trials
Phase IV
Post-Marketing
BiochemistMicrobiologistGeneticistsMolecular BiologistPharmacologistOrganic ChemistMedicinal ChemistStatisticianMD/PhD
BioinformaticsPatent attorneyLawyersProject ManagerPharmacologistImmunologistMD SpecialistQA/QCInformaticsStatisticianToxicologistChemistBiotechnologist
BiotechnologistRegulatory AffairsCRCs, PIs (MDs, PhDs)CRO, CRA, NursesMedical WritersMedical AffairsProject ManagersQA/QC ManagersData ManagersStatisticianSite IRBsSite Pharmacists
ManufacturerMSLsMarketingMedical WritersCommunication Spec.CRCs, PIsCRO, CRAsProject ManagersData ManagersStatisticianSite IRBsMedical AffairsRegulatory Affairs
Year 0
Professionals involved in Medicines Development
GROWING PROBLEM! - A MISMATCH
TEACHING OUTPUT,
QUALIFICATIONSHEALTHCARE NEEDS,INDUSTRY EXPECTATIONS
P.Stonier, ICPM 2016
H. Silva, 2014
Needs for integration in education and training
THE LINK BETWEEN COMPETENT PROFESSIONALS AND JOB PROFILES
COMPETENCIES
P. Stonier, ICPM 2016
Competency vs. Competence
Competencies encompass knowledge, skills, attitudes, and behaviors necessary for a particular set of tasks or objectives in a specific /function
•Are usually organized in clusters (domains)
•Can be measured (assessed)
•Can be improved
Professional Competence is defined as “the array of abilities across multiple domains or aspects of professional performance in a certain context…is multidimensional and dynamic. It changes with time, experience and setting”.
A competent professional is one possessing the required abilities in all domains in a certain context at a defined stage of education or practice.
(Frank J. Medical Teacher, 2010)
8
Joint Task Force for Clinical Trial Competency
Harmonized competencies can be the basis for various ends
9
Education
Job
Descriptions/
Comp Profiles
Site
Qualification
Investigator
Selection
Accreditation
standards
Training
Requirements
Streamlining educational
requirements
Standardizing job
descriptions
Defining criteria for site
selection and qualification
Standardizing and
streamlining training
requirements
Defining standards for
accreditation of educational
programs
Defining criteria for
investigator selection
PharmaTrain: Vision 2014
Harmonized Core Competencies for the Clinical Research Professional: The Joint Task ForceSonstein et al. Clinical Researcher, 2014
A Comprehensive Process to define and validate competencies (IFAPP and JTF)
1-Establish an efficient, effective and representative Project Working Group and define objectives and stakeholders
2- Thorough bibliographic review , consultations and identification of key domains
3- Initial drafting of competencies, qualification and ranking: first level of competencies (cognitive)
4- Confirmation of second level of competencies (knowledge, skills and behaviors)
5-Expert review and discussion with stakeholders
6- Pilot Test and feedback (internal and external validation through multiple tools)
7- Define standards, assessment tools, and implementation through national ®ional networks
Evolving needs of Society and Stakeholders
• Public Trust• Professionalism• Quality Assurance• Performance• Risk Management• Information Technology
Sustainable infrastructure to drive competency based education, training, job profiles, performance, assessment, quality, safety and efficiency
12
IFAPP-PharmaTrain Competence Domains
• 57 competencies7 domains
IFAPP Full Core Competencies FrameworkExample:
C4: Ability to create a Clinical Development Plan for a new drug (CDP)
Applied knowledge of:• Construction of CDP:
– Key registration studies– Primary & secondary endpoints– Timelines for study and programme completion– Risks to plans
• Objectives of studies to support product life-cycle
Skills:• To write / contribute to CDP for a new drug• To create a target product profile (TPP)
Behaviours:• Recognises the role of team approach to clinical development• Recognises the integrated view on entire clinical development process
Initial validation of the full core competency framework conducted among the IFAPP membership and key stakeholders is underway
A framework for defining professional competence throughout the clinical research enterprise
The Joint Task Force on Clinical Research Competence
• Representatives met in person or via TC during the period October 2013-June 2014
• JTF Coordinators: S. Sonstein and R. Li
Joint Task Force – Competence Domains
Sonstein SA., Seltzer, J., Li,R., Jones, C.T., Silva, H., Daemen E. Moving form compliance to competency: A harmonized core competency framework for the clinical research professional.Clinical Researcher 2014 28(3); 17-23•Applied Clinical Trials. May 28, 2014•Journal of Clinical Research Best Practices,10(6); 1-12.•CenterWatch Whitepapers, June, 2014.
JTF Domains- Topics and Competencies
Domain KSA Topics Competencies
Scientific Concepts & Research Design
Biologic, Scientific Concepts, Design & Analyses of CT
5
Ethical & Participant Safety Considerations
Care of patients, Human Subject Protections, Safety
8
Medicines Dev’t & Regulation
How drugs, devices, biologics developed & regulated
7
Clinical Trials Operations Study management, GCP compliance, regulatory affairs, safety reporting, handing investigational product
12
Study & Site Mgt Site mgt (financial, personnel) operations 6
Data Mgt & Informatics How data are managed in a CT, databases 5
Leadership & Professionalism
Leadership, Professionalism, Mentoring 4
Communication & Teamwork
Communication- between site and externally, interdisciplinary teams
4
JTF Validation Objectives
To validate JTF domains/competencies through perceptions of clinical research professionals.
How do the competencies relate to:
• Roles, Job Titles and Experience
• Perceptions of competence and relevance
• Educational Levels
• Educational Needs
• Identify gaps for future study and workforce development
Methods
• Survey- Used SurveyMonkey® Structure
• URL/ 3 collectors (USA, Europe/Asia and Latin America)
• Support from academic institutions and professional organizations
• Cover letter- informational- informed consent
• Anonymous, untraceable
• Survey data- transferred to an excel database with pivot tables
• Additional analyses- Univ Michigan- SAS/SPSS
Scale of Self-Assessed Responses to Perception of Competence and Job relevance
Competence:
•0= Never been exposed to this content
•1= Aware of the content, but never needed to become further informed;
•2= Exposed and sufficiently aware of content that I can look up what might be necessary for my job;
•3= Competent–Able to interpret or discuss concepts and use knowledge to solve simple problems based
on application concepts;
•4 = Mastery–able to apply knowledge to complex problems, integrate information and create solutions
Relevance : •0 = unnecessary, no relevance to my role;
•1 = has some relevance to my role, but not my responsibility;
•2 = relevant to my role, but not a major component;
•3 = significant to my role and part of my job responsibilities;
•4 = major part of my responsibility or supervisory expectations
19
0- 2 3- 40 1
0.6
Scale Rearrangements for Statistical Analysis
Demographics by region and function20
Role #
Clinical Research Coordinator/Research
Nurse559
Research Administrator/Project
Manager357
Principal Investigator/Co-investigator 354
Clinical Research Associate/Monitor 177
Regulatory Affairs Professional 90
Data Management Professional 47
Others 154
Total 1738
Asia/Australia4%
Latin America/Carib
bean41%
US/Canada31%
Western Europe
24%
Regional Participants(n=1584)
Perception of Competence by Role
Perceived Domain Relevance by Role
Perception of Specific Competencies (%): Scientific Concepts and Research Design
Domain 1
% Competent
CRA CRC/
RN
DM PIs/
CoPIs
RA RM/P
M
Demonstrate
knowledge of
pathophysiology,
pharmacology and
toxicology as they
relate to medicines
discovery and
development
41.9 28.6 29.6 76.6 28.3 39.5
Identify clinically
important questions
that are potentially
testable clinical
research hypothesis,
through review of the
professional literature
37.2 31.3 40.7 83.4 30.4 38.4
Explain the elements
(statistical,
epidemiological and
operational) of clinical
and translational
study design
29.1 27.0 29.6 64.7 23.9 38.9
Design a clinical trial 31.8 22.9 37.0 69.6 37.0 39.1
Critically analyze study
results with an
understanding of
therapeutic and
comparative
effectiveness
40.7 21.0 37.0 83.7 31.1 35.5
Domain 1
% Relevant
CRA CRC/
RN
DM PIs/
CoPIs
RA RM/P
M
Demonstrate
knowledge of
pathophysiology,
pharmacology and
toxicology as they
relate to medicines
discovery and
development
51.2 30.6 22.2 73.0 21.3 28.6
Identify clinically
important questions
that are potentially
testable clinical
research hypothesis,
through review of the
professional literature
38.4 22.9 22.2 80.8 21.3 26.1
Explain the elements
(statistical,
epidemiological and
operational) of clinical
and translational
study design
29.1 24.4 25.9 69.6 17.0 33.2
Design a clinical trial 29.4 20.3 25.9 68.1 25.5 26.2
Critically analyze
study results with an
understanding of
therapeutic and
comparative
effectiveness
32.6 15.9 25.9 79.6 12.8 25.5
Years of Experience and competence
Experience influences perception of competence
Relevance to the position Vs. Experience
• Experience does not influence the relevance of the position
Academic degree Vs Perception of Competence
Perceived Additional Training Needs
Scientific Concepts and Research Design
ANOVA: Comparison between regions, *** p< 0.0001
***
***
Higher level of perceived competence/relevance in Latin America; low in Western Europe
Medicines Development and Regulation
ANOVA, ***p<0.0001
***
***
Low levels of perceived competence/relevance in all regions, but even lower in Asia
Ethical and Participant Safety Considerations
ANOVA: **p<0.001, ***p<0.0001
***
**
Communication and Teamwork
***
**
ANOVA: **p<0.001, ***p<0.0001
Lower perception of relevance in the USA and Europe
Some inferences from this first validation attempt
1. Lack of perceived competence in two key domains (scientific concept and medicines development)
• Could this reflect the lack of required educational preparation in clinical research content for CR staff and the lack of required experience for PIs?
2. Lack of educational preparation and low perceived competence levels over the first 5 years of employment and a requirement for “learning on the job.
• Could this be a major contributor to the high level of redundancy, delays, non-compliance, and poor quality seen in clinical research sites?
3. Those with the highest levels of competence have the greatest desire for more training/education.
• Could this mean that those with lower levels of perceived competence do not know what they do not know?
Sonstein S. et al. ACRP 2016
Comments to regional differences
• Need to confirm regional findings (focus groups) or surveys among representative samples from participating countries
• Need to explore further according to source of employment (academic, industry, CROs)
• Questionnaire was conducted in English. Need for language validation to confirm full understanding of the individual competency
• In spite the large sample size, internal regional demographics could skew the interpretation of results
• Several hypothesis can be formulated
True Core Competencies by function?
Function Number of competences perceived as relevant by > 60 % of respondents
Principal Investigators 45
Clinical Research Associates 25
Clinical Research Coordinator 24
All functions 21
• Implications for Job Descriptions and Job Portfolios• Core competencies are ever changing. A custodian group is necessary• Needs for further validation at the country/regional level
Impact of JTF Core Competencies in the USA
• ACRP appointed as Custodian of Core Competencies
• 16 related publications
• Presentations (26) in several conferences
• Institutional adoption by: CoAPCR; ACRP, CTSA, NBME, academic institutions, NIH-UK.
• Guide for accreditation of academic programs in clinical research (CoAPCR)
• Guide for training programs (ACRP) and certification exams (Academy of Clinical Research Professionals)
• Guide for job portfolio development (CTSA)
• Needs for further validation (CTSA DIAMOND Initiative)
