Produced by: FOI Services, Inc. 704 Quince Orchard Road • Suite 275 Gaithersburg MD 20878-1770 USA

Phone: 1-800-654-1147 or +1-301-975-9400Fax: 301-975-0702Email: infofoi@foiservices.comwww.foiservices.com

by a Distinguished Panel of Experts with FDA & Industry Experience

Date: Monday, March 28, 2011Time: 1:00pm – 2:30pm Eastern Daylight Time (GMT/UT 1700)

12:00pm – 1:30pm Central Time 11:00am – 12:30pm Mountain Time 10:00am – 11:30am Pacifi c Time

TC001539A

Training Programs:What FDA Expects to See

Teleconference Course MaterialsYou may duplicate this for each person attending the conference.

Call-In: Dial 1-888-848-0354 approximately 10-15 minutes prior to the start time. When prompted, enter the PIN code followed by the # key: 9822359#Outside the U.S. & Canada, you will receive special instructions by email. If you become disconnected and cannot reconnect through the number above, you can also call in on +1-801-853-0800.

At the conclusion of the conference, an audio CD will be made available for order. Attendees receive a special reduced price of $225. To order go to www.foiservices.com/training2359 or call 301-975-9400.

Important NoticeThe information provided in this course by the instructor is his/her personal opinion and does not necessarily

represent the opinions of FOI, Inc. or its staff. Companies relying on the information do so at their own risk and assume the risk and any subsequent liability that results from relying on the information.

The information provided does not constitute legal advice.

Training Programs:Training Programs: What FDA Expects to Seep

P t d bPresented by:  Nancy Singer, John Avellanet, Nicole Petty,  

Catherine Crismon, Marla Phillips & Cap UldriksCatherine Crismon,  Marla Phillips & Cap Uldriks

March 28, 2011An FOI Services, Inc. TeleconferenceAn FOI Services, Inc. Teleconference

Agenda• Training Requirements 

N Si C li Alli– Nancy Singer, Compliance‐Alliance

• Training Records J h A ll t C l A i t LLC– John Avellanet, Cerulean Associates LLC

• Training Best Practices

Ni l P S k I– Nicole Petty, Stryker Instruments

• The MLS Training Program C th i C i M idi Lif S i– Catherine Crismon, Meridian Life Science

• Learning vs. Training M l A Philli Ph D X i U i it– Marla A. Phillips, Ph.D., Xavier University

• Training: A View from a Former FDA OfficialCap Uldriks Olsson Frank Weeda Terman Bode Matz PC– Cap  Uldriks, Olsson Frank Weeda Terman Bode Matz PC

2

Training:Training:FDA Requirements & q

Warning Letter Citations

Nancy SingerNancy SingerCompliance‐Alliance, LLCNancy Singer@juno comNancy_Singer@juno.com

3

Device Regs: 21 CRF 820 20(b)Device Regs: 21 CRF 820.20(b)

(2) Resources. Each manufacturer shall provide adequate resources, including the assignment of trained personnel, for management, performance of work, and assessment activities, including internal quality audits, to meet the 

i f hirequirements of this part.

4

Device Regs: 21 CFR 820.25Sec. 820.25 Personnel. (a)General. Each manufacturer shall have sufficient personnel with the necessary education backgroundpersonnel with the necessary education, background, training, and experience to assure that all activities required by this part are correctly performed.(b)Training. Each manufacturer shall establish procedures for identifying training needs and ensure that all personnel are trained to adequately perform their assignedtrained to adequately perform their assigned responsibilities. Training shall be documented.

(1)  As part of their training, personnel shall be made         f d i d f hi h f haware of device defects which may occur from the 

improper performance of their specific jobs.(2)  Personnel who perform verification and validation ( ) e so e o pe o e ca o a d a da o

activities shall be made aware of defects and errors that may be encountered as part of their job functions.functions.

5

Drug Regs 21 CFR 211 25(a)Drug Regs. 21 CFR 211.25(a) 

Each person engaged in the manufacture, processing, packing, or holding of a drug product shall have d i i i d i bi ieducation, training, and experience, or any combination thereof, to enable that person to perform the assigned f ifunctions.

6

Drug Regs: 21 CFR 211.25(a) g g(continued)

Training shall be in the particular operations that the employee performs and in current good manufacturing practice (including the current good  manufacturing practice regulations in this chapter d itt d i d b thand written procedures required by these 

regulations) as they relate to the employee’s functions Training in current good manufacturingfunctions. Training in current good manufacturing practice shall be conducted by qualified individuals on a continuing basis and with sufficient frequency o a co t u g bas s a d t su c e t eque cyto assure that employees remain familiar with CGMP requirements applicable to them.

7

Drug Regs: 21 CFR 211 25(b)Drug Regs: 21 CFR 211.25(b)

Each person responsible for supervising the manufacture, processing, packing, or holding of a drug product shall have the education, training, and  experience, or any combination thereof, to perform assigned functions in 

h id h h dsuch a manner as to provide assurance that the drug product has the safety, identity, strength, quality, and 

i h i i dpurity that it purports or is represented to possess.

8

Establishment Inspection Reports (EIRs)

• Most EIRs contain a section on how the firm trains it personneltrains it personnel.

• The firm's training programs are of particular significance where inspectional findings find people may not be adequately trained.p p y q y

9

November 2010 Warning Letter to Optilasa SA

Failure to establish and maintain adequate procedures for identifying training needs and ensure that all personnel are trained to adequately perform their assigned responsibilities, to include documentation of h i i i d b 21 820 25(b) F lthe training, as required by 21 820.25(b). For example, Procedure PC‐05, Training, includes requirements for employee training However there is no documentationemployee training. However, there is no documentation to show that employees were trained.

10

December 2010 Warning Letterto  Airgas, Inc.

In your firm's response, your firm proposes training employees in "all applicable CGMP requirements" as a corrective action. Your response is inadequate because you have not defined what specific information will be i l d d i CGMP i iincluded in your CGMP training.

11

January 2011 Warning Letter to Lynn Peavey Company

F il t t bli h d f id tif i t i i d dFailure to establish procedures for identifying training needs and ensure that all personnel are trained to adequately perform their assigned responsibilities, as required by 21 CFR 820.25(b).

For example, your firm does not maintain a standard operating procedure for identifying personnel training needs and ensuring th t ll l t i d t d t l f th i i dthat all personnel are trained to adequately perform their assigned responsibilities with regard to the manufacturing of regulated medical devices including the Sexual Assault Evidence Collection Kit and the Blood Alcohol Collection Kit According to a documentand the Blood Alcohol Collection Kit. According to a document entitled: "Sexual Assault Assembly Certification" dated (b)(4). (b)(4)team members were certified to assemble Sexual Assault Evidence Collection Kits but the uncontrolled document does not identifyCollection Kits, but the uncontrolled document does not identify the training needs necessary for this certification.

12

February 2011 Warning Letter to Sanofi‐Aventis Deutschland GmbHY fi f il d h h d i hYour firm failed to ensure that each person engaged in the manufacture, processing, packing, or holding of a drug product has the education, training, and experience, or any combination thereof, to enable that person to perform the assigned functions. [21 C.F.R. § 211.25(a)].

For example, on September 6 and 9, 2010, operators involved in the cleaning operations and aseptic connections during filling were observed demonstrating incorrect asepticfilling, were observed demonstrating incorrect aseptic techniques to prevent product contamination. We expect that operators who conduct operations within aseptic processing areas be properly trained and monitored to ensure thatareas be properly trained and monitored to ensure that proper aseptic techniques are utilized during all operations.

13

February 2011 Warning Letter to Southern Implants (Pty) Ltd.

CAPA (b)(4) was opened in July 2010 as a preventive action regarding the lack of consistent procedures for applying laser marking on devices. The CAPA included categorizing laser marked products with information pertaining to the location of information for laser markings. The interim corrective action included using  (b)(4) as guidance for the laser marking operators yet no documentation of the use of the (b)(4) or training of the operators on these revised procedures was made.

14

February 2011 Warning Letter to RD Medical Manufacturing Holding Company

We found your response to this deficiency inadequate. You indicated that you have training records documenting the training of your personnel in specific functions and general g y p p gfunctions. Nevertheless, you failed to include evidence in your written response to show that your operators have been trained in the techniques described above. You also indicated qthat your training procedure will be revised to include a certification program as part of your training requirements. Your target completion date is the second quarter of 2011.Your target completion date is the second quarter of 2011. Please indicate which interim measures you have taken to ensure that your operators are adequately trained to perform the (b)(4) technique and the (b)(4). The release forperform the (b)(4) technique and the (b)(4). The release for distribution of several of your products depends in part on the operator's ability to adequately perform these tasks.

15

John AvellanetJohn Avellanetjohn@ceruleanllc.comwww.ceruleanllc.com

Cerulean Associates LLC

pathways to misfortune

US requirements

do’s and don’ts

TRAINING RECORDS

do’s and don’ts

lean compliance SOP elements

TRAINING RECORDS

16

Typical SOP ElementsTypical SOP Elements

• Reasons for trainingStandard Operating Procedure

• Types of trainingQuality Management System

• Tracking of trainingTraining SOP

• Attendance sign‐in sheetCerulean

17

Pathways to MisfortunePathways to Misfortune

Roles

• supervisor

Effectiveness

• learning

RecordsRetention

• company

ProcessLinks

• disciplinesupervisor• attendee

learning objectives

• metrics

company records

• personal records

• departmental

discipline• compliance• corporate

liabilityf ti l• departmental

records• cross-functional

activities• other QMS

processes

18

Questions to ConsiderQuestions to Consider

S i R lSupervisor Role• does the supervisor discuss the training and results with the attendee?• does the supervisor evaluate the knowledge sufficiency of thedoes the supervisor evaluate the knowledge sufficiency of the 

attendee?• does the supervisor document the training event?

d th i i di ll l t i di id l t i i d ?• does the supervisor periodically evaluate individual training needs?

Attendee RoleAttendee Role• does the attendee actively acknowledge and understand the training?• does the attendee transfer the training to job performance and/or does the attendee transfer the training to job performance and/or

product safety and efficacy responsibilities?

19

Questions to ConsiderQuestions to Consider

Eff iEffectiveness• how are training needs identified?

• how is the transfer of trained knowledge into the attendee’s daily responsibilities measured or tracked during and/or immediately after training?

• how is the transfer of trained knowledge measured or tracked 30‐90 days after training?

• does the training SOP provide sample effectiveness measures for trainers to draw upon?

• how is a trainer selected?how is a trainer selected?

• how is are drafted training materials assessed for appropriateness, level of detail, accuracy, and learning objectives?

20

U S Regulations & StatutesU.S. Regulations & Statutes• 21 CFR 58 195• 21 CFR 58.195• 21 CFR 110.10• 21 CFR 111.14• 21 CFR 211.25• 21 CFR 312.62

U.S. Code of Federal Regulations alone has

1 416 req irements• 21 CFR 600.10• 21 CFR 606.20• 21 CFR 812.140• 21 CFR 820.20, .25, .70 

1,416 requirementsfor training records

retention!• 29 CFR 825.500• 29 CFR 1601.31• 29 CFR 1602.14, .31• 29 CFR 1627.3• 40 CFR 264 73

retention!

• 40 CFR 264.73• FDCA• FAAA• FLSA• ERISA• Title VII (ADEA, GINA, EPA)• UCC 2‐725  (contractors only)• State statute of limitations  (contractors only)

21

Example Retention PeriodsExample Retention Periods

UC 2‐725

Title VII

40 CFR 264

UC 2 725

29 CFR 1627

21 CFR 820

21 CFR 312

21 CFR 820

0 2 4 6 8 10 12 14

Required Years after Event22

TranslationTranslation

retain a person’s training records untilthe date they leave the organizationthe date they leave the organization

+ 6 years+ 6 years

23

Be cautious aboutBe cautious about group attendancegroup attendance 

sheets!sheets!

24

Certificate of UnderstandingCertificate of Understanding

Document• active attendee participatory role

• supervisory review & approvesupervisory review & approve

Support personnel actions

Help limit company liability

Foster personal sense of ibilit f liresponsibility for compliance

25

Documentation Do’sDocumentation Do s

Wh d d h i iWho conducted the training

How the training was conductedHow the training was conducted

When the training occurred

Who attended the training

Summarize the training provided

Identify effectivenessmeasuresIdentify effectivenessmeasures

Why training related to job and/or product

26

Documentation Don’tsDocumentation Don ts

Segregate training records by department

Retain materials with every individual record

Forget supervisory role to review and decide

I d i t k i f t i i t i lIgnore desire to keep copies of training materials• attendees

i di id l t i (SME)• individual trainer (SME)

Exclude 3rd party‐provided training• organization’s official record

• individual’s own record

27

3rd Party Training Certificates3rd‐Party Training Certificates

Document attendance• may want to require agendas

Support personnel actions

Help limit company liability

Encourage individual responsibility for compliance

• continuous improvement

28

More Documentation Do’sMore Documentation Do s

Retain 1 set of training materials for future use

Rely upon the training matrixRely upon the training matrix

Archive yearly copy of the training matrix

Audit annually your training records

• current set of training materials – discrepancies?

• personnel training records (ANZI z1.4 sampling)

• relationship to other cross‐functional controls

• retrieval speed vs. retrieval accuracy

29

Lean Compliance Training SOPLean Compliance Training SOP

Elements• Supervisor’s roleStandard Operating Procedure

• Use of training matrix

• Effectiveness measuresQuality Management

System• Records to retain

• Training/re‐training triggersTraining SOP

• Roles of compliance teams

• Linkages to HR, IT, etc.

• Defining learning objectives

• 3rd party certificate usage

Cerulean

30

Training Process FlowTraining Process Flow

31

More SOPs to ConsiderMore SOPs to Consider

• Developing training materials

• Training matrices – creation and maintenance

• Training records – retention and archival

• Verification of training effectiveness• Verification of training effectiveness

• Overview of the training cycle  (guideline)

• Presentation design best practices  (guideline)

32

www ceruleanllc com/resources/foi2811/www.ceruleanllc.com/resources/foi2811/

14 i i SOP14‐step training SOP process map• from identifying training needs to storing the required records 

Training program self‐assessment26 d il d i ( i h i l )• 26 detailed questions (with a ratings scale)on training programs taken from FDA inspector training, FDA & EMA inspection experiences and audit best practicesexperiences, and audit best practices

Regulations impacting trainingg p g g• common US regulations and laws affecting training records

33

T i i B t P tiTraining Best Practices

Nicole PettySr. Manager Learning & Development

Stryker InstrumentsNicole.petty@stryker.com

34

S k I h ADDIEStryker Instruments uses the ADDIE model for our training programs

AAnalysis ‐ ‐ Annual Training Analysis to align training with budget and business strategy as well as at the individual training item level

DD i & DD l DDesign & DDevelopment ‐ ‐ Design training and evaluation plan and materials with SMEs ‐ ‐ including the appropriate method of training

IImplementation ‐ ‐ Learning Management System (LMS) coordinators communicate & assign training via curriculumsg g

EEvaluation ‐ ‐ Throughout the process, done both for individual training items as well as the overall training program

35

Training Responsibilities ‐ EVERYONE plays a part in making training effectivemaking training effective

36

Who must comply?Who must comply?

CompanyCompany ContractorsContractorsTempsTempsConsultantsConsultantsCompany PersonnelCompany Personnel

ContractorsContractorsTempsTempsConsultantsConsultants

37

Use curriculums to manage trainingUse curriculums to manage training assignments based on roles & responsibilities

Personal ComplianceDevelopmentPersonal ComplianceDevelopmentDevelopmentDevelopment

38

Monitoring training and build a learning culture

Management can see who is past dueManagement can see who is past dueManagement can see who is past due on training with a click of a button.

Management can see who is past due on training with a click of a button.

39

Working as a team to deliver effective training…

Annual employee survey for training, 

Monitor training and give feedback through Annual Training Plan & 

training evaluations, Individual 

Development Plans

management review

Review training to ensure it will meet the need & that

Work together to provide

the need & that the training is assigned to the appropriate peopleto provide 

appropriate learning solutions

people

Identify training needs for employees and help build training curriculums for their team

40

The MLS Training Programbyy

Catherine Crismon, B.S. M.S, RABQSA‐LA

Quality Systems ManagerMeridian Life Science

cathy.crismon@meridianlifescience.comy @901‐322‐9594

41

MLS Training ProgramMLS Training Program

• Different Types and Levels of Training

• Company Wide versus Job Specific

• Establishing Training RecordsEstablishing Training Records

• Meeting ISO and Regulatory Requirements

42

Taking Charge!Taking Charge!

• Who is in Charge of the Training Program?– Quality Assurance: task specific training, quality Q y p g, q yspecific training (ISO)

– Human Resources: EEOC and OSHA relatedHuman Resources: EEOC and OSHA related training

Wh i i Ch f M it i th T i i ?• Who is in Charge of Monitoring the Training?– Each supervisor: records must be reviewed at the end of each year at the same time performance evaluations are conducted.

43

Take Charge!Take Charge!

• Who is in charge of conducting the training?– Quality Assurance and Human Resources Q y

• Work in tandem on safety and quality related subjects

• QA arrange in house training seminars (e.g., how toQA arrange in house training seminars (e.g., how to write a CAPA)

• HR conducts lunch and learn sessions (e.g., conducting ( g , geffective interviews)

44

Who is Required to Be Trained?Who is Required to Be Trained?

• Everyone must attend the EEOC and OSHA related training.g

• Each employee must obtain training related to his or her job prior to doing that jobhis or her job prior to doing that job.

• No employee is exempt from getting the necessary training for his or her job or for EEOC/OSHA training.EEOC/OSHA training.

45

Training EffectivenessTraining Effectiveness

• Some training sessions require a test upon completion. p

• All jobs and associated documentation and records are audited by the QA staff forrecords are audited by the QA staff for accuracy and completeness. 

46

Participation/Buy InParticipation/Buy‐In

• Work with upper management on the importance of training.p g

• Well‐trained employees affect the bottom line in a positive wayin a positive way.

• Poorly‐trained or untrained employees are problematic and create extra work, scrap, nonconformities and unsafe workingnonconformities and unsafe working conditions.

47

Learningvs.

dBeing Trained

Marla A. Phillips, Ph.D.Xavier University

513‐745‐3073

phillipsm4@xavier.edu

wwwMedXU comwww.MedXU.com 

48

Typical Academic ModelsTypical Academic Models

• Grade School 

– Teaching is focused on drilling information

– Teaching methods include projects, field trips, and hands‐on activities

• Undergraduate

– Increase in the amount of hands‐on learning and self‐discovery.  

Teaching methods incl de papers st d– Teaching methods include papers, study groups, co‐ops, internships, laboratories, foreign exchange, residency, case studies, simulations, practicum, and seminars

49

Adult EducationAdult Education

• Tie the information delivered to an experience that they have encountered or with which they can relate

• Adults learn best through discussion,Adults learn best through discussion, sharing own experiences and asking questions

• Include variety, interaction, “entertainment”

• Interestingly, adults do not follow directions very well/completely when compared to students.  Why?

50

Employee TrainingEmployee Training

• “Read and Understand” approachRead and Understand  approach– What is the least effective academic learning model?

– Can be appropriate for very basic and simple processes

• Competency‐based training and qualification– Often not done properly 

– Often not tied to a risk assessment model

• Example: Qualification of chemists on everyExample:  Qualification of chemists on every analytical or microbiological method – All or nothing approach

– Why?  

– It is harder to think through and justify the right course of action, so instead just have everyone train on everything.

51

Risk ModelRisk Model

10

Qualification, Apprentice

Competency Based,Demonstrations

Probability

Read and C t B dRead andunderstand

Competency Based,Demonstrations

00

0 10

Impact

52

Risk Based Employee TrainingRisk Based Employee Training1. Read and Understand

H d H i L D i– How to conduct pH testing, Loss on Drying

2. Competency Based Training or Demonstrating Proficiency– Case studies on proper investigation techniques to identify root cause– Case studies on proper investigation techniques to identify root cause

– Demonstrating that simple analytical methods can be run using a live sample

3. Qualification– Complex analytical methods – equipment, method, formulation

4 Assess effectiveness – monitor human errors4. Assess effectiveness – monitor human errors– Listen to employee suggestions

– Involve employees in the training process

Effective Training

• Optimize learning:  self‐discovery, experiential

• Focus resources on areas of greatest impact 53

Training: A View from a FormerTraining: A View from a Former FDA Official

Cap  UldriksOl F k W d T B d M PCOlsson Frank Weeda Terman Bode Matz PC

Formerly CDRH's Associate Director for Regulatory Guidance and Regulatory Affairs

Culdriks@ofwlaw.com

54

FDA PerspectiveFDA Perspective

• Why does FDA believe that training is critical t fi ’ ti i t t f t l?to a firm’s operating in a state of control?

• What questions will an FDA investigator ask to q gdetermine whether people have received adequate training? 

55

FDA PerspectiveFDA Perspective

• How should you respond to a 483 observation i l tt th t lor warning letter that says employees were 

not trained adequately, if at all?

• How do FDA investigators feel when a firm’s hi h d i ihistory shows repeated promises to train or retrain employees, but no such training occurs?occurs?

56

FDA PerspectiveFDA Perspective

• Does FDA prefer one method of training over another (e.g., in‐person, online, read a ( g , p , ,document and take a test)?

• What is the best way to prove to an investigator that employees are trained?

57

FDA PerspectiveFDA Perspective

• If a trained employee makes a mistake, willIf a trained employee makes a mistake,  will FDA automatically assume that  the person was not trained?was not trained?

58

Thank You!Thank You! Questions?

If you prefer to submit a question to the panel later, please email y p q p , pyour inquiry to Nancy Singer at nancy_singer@juno.com or 

phone her at +1‐703‐525‐4159.

Your feedback is valued! 

Please complete & return the evaluation to enter a drawing for a $100 amazon.com gift card and/or receive a certificate 

if i 1 5 h f i i d iverifying your 1.5 hours of continuing education.

59

Training Evaluation FormTraining Programs: What FDA Expects to SeeMarch 28, 2011 — Presented by a Distinguished Panel of Experts

Please indicate your primary job responsibility (circle one):

Regulatory Affairs QA/QC Statistics Medical Writing Electronic Submissions General Management

Other (please specify):____________________________________

Please rate various aspects of this course (1 = Poor; 2=Fair; 3=Good; 4=Very Good; 5=Excellent – please circle your answers)

What is your overall rating of the course? 1 2 3 4 5

How do you rate the content of the course? 1 2 3 4 5

Were course materials clear and understandable? 1 2 3 4 5

Was the length of the course adequate to cover the content? 1 2 3 4 5

Were the instructors knowledgeable about the subject matter? 1 2 3 4 5 (please note any comments about specifi c instructors below)

How would you rate the instructors overall? 1 2 3 4 5

Other comments:

Please feel free to comment on any aspect of this course, including the speakers, content, and technical arrangements:

TC001539

Win a $100 Amazon.com Gift Certifi cate from FOI!Just complete this form and fax it to +1-301-975-0702 by April 4, 2011

and you will be entered in a drawing for an Amazon.com Gift Certifi cate

By providing the information below and faxing this page to +1-301-975-0702 by Monday, April 4, 2011you will be entered in a drawing for a $100 Amazon.com Gift Certifi cate from FOI.

Your Name: _______________________________________________________________________

Company Name: ___________________________________________________________________

Email Address: ____________________________________________________________

Check here to receive a Certifi cation of Attendance verifying your 1.5 training contact hours for this educational session. Please provide your legibly printed name with your signature next to it, email address, and fax this form to the number below. Your certifi cate will be emailed to you.