how the ich’s new inspection protocol program...about your presenter john avellanet trainer for...

How the ICH’s New InspectionProtocol Program

Helps Uncover Data Integrity Issues

About Your PresenterJohn Avellanet

Trainer for FDA and Health Canada inspectors and districtofficers on advanced data integrity inspection techniques anddetecting data fraud

Served on behalf of the US Department of Justice as theindependent overseer for the five-year, multi-million dollar Dr.Comfort Corporate Integrity Agreement

Industry reviewer for the international standard, BSI 10008Evidential Weight and Legal Admissibility of ElectronicInformation (2015)

Former lead expert for the ISPE GAMP Data Integrity WorkingGroup

Author of Get to Market Now! Turn FDA Compliance into aCompetitive Edge in the Era of Personalized Medicine (2010);co-author of Pharmaceutical Regulatory Inspections (2014)

Prior to founding Cerulean, John spent more than 15 yearsdesigning, implementing, and being accountable for qualitysystems and data compliance programs for FDA, DEA, BIS,ICH, IMDRF, and ISO

[email protected]

www.ceruleanllc.com

© 2018 Cerulean Associates LLC 2www.Ceruleanllc.com

During a regulatory inspection, theinvestigator asks for a specific record.After 24 hours, you cannot find it.Which response is best?

a) Give us another 24 hours to locate the record

b) The record is (truly) at another site

c) We noted a discrepancy and opened a CAPA

© 2018 Cerulean Associates LLC www.Ceruleanllc.com 3

Agenda

regulators’ concerns today

how the NIPP helps find data integrity issues

case study from the pilot

recent enforcement examples under the NIPP


This is not legal advice. Information in this presentation draws upon a variety of sources, including published FDA warningletters and Form FDA-483s, FDA regulations and statutes, FDA presentations, policies, and other regulatory guidance

documents, personal and client experiences, documents, interviews and research, all or any of which may or may not havebeen prepared or conducted by the presenter. Presenter does not provide a warranty concerning the accuracy of all the

information contained in the workshop presentations. The contents of this presentation are intended for general informationonly and should not be construed as legal advice. Presenter assumes no liability for actions taken or not taken as a result ofthe information in this presentation. The presentation is copyrighted 2018 by Cerulean Associates LLC. All rights reserved.

REGULATORS’ CONCERNS TODAY


Leiner Health Products

© 2018 Cerulean Associates LLC 6

“…inspections found many serious deviations, someof which involved data manipulation and

inadequate testing procedures.”Warning Letter to Leiner Health Products, August 2007

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2006/ucm076485.htm

www.Ceruleanllc.com

Vitamin maker$10 million fine$50 million sales revenue loss1 count of mail fraud540 employees laid offmultiple product liability lawsuitseventual bankruptcy


“One out of every 50 scientists

today falsify data.”- Daniele Fanelli, Stanford University, “How Many Scientists

Fabricate and Falsify Research? A Systematic Review andMeta-Analysis of Survey Data,” PLOS One, May 2009

www.Ceruleanllc.com

Software Bug Jeopardizes up to40,000 Clinical Studies


• 15-year old software bug infMRI software exaggeratesfalse positives (e.g., “noise”)by at least 10%

• Code fix implemented inDecember 2015

• All clinical studies involvingfMRI since 2000 may haveinflated drug and deviceefficacy by at least 10% - orunder measured suchefficacy

Sources:http://www.pnas.org/content/113/28/7900

https://arstechnica.com/science/2016/07/algorithms-used-to-study-brain-activity-may-be-exaggerating-results/

http://www.pnas.org/content/113/28/7900



Adobe PDF and Edge Browser Bug


“…the report shows that, when printingPDF documents, Edge can completely

transform their look, feel, and content.Note the numerous changes, including theshifting of cell numbers, the adding ofwords and symbols, and the substitution ofwords and characters….”

• https://arstechnica.com/information-technology/2017/05/microsoft-advises-use-of-xerox-copiers-to-mitigate-major-edge-bug/

• https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/11896203/

Microsoft and Adobefixed on June 5, 2017

https://arstechnica.com/information-technology/2017/05/microsoft-advises-use-of-xerox-copiers-to-mitigate-major-edge-bug/



https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/11896203/

https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/11896203/

11

Regulatory Agency Translation

what of an organization’s data can

we not trust?

© 2018 Cerulean Associates LLC www.Ceruleanllc.com

Data Integrity Spectrum

Wrong year entered shortlyafter new year turnover (i.e.,

“2018” instead of “2019”) “Read-only” data overwritten with randomvalues when reviewer closes databases (orprint-outs display random gibberish insteadof full reports, accurate lists, etc.)

Data and other information is lost fromdegradation of media, corruption of

archives, lack of good recordkeeping, etc.Data values are truncated,rounded up/down, etc.

Data is intentionally and fraudulentlymade up, edited to pass, destroyed,

manipulated, etc.


ICH’S NEW INSPECTION PROTOCOLPROGRAM (NIPP)


E-Inspections under FDASIA 2012

“Sec 706 Records for Inspection. (4)(A)Any records or other information that theSecretary may inspect under this sectionfrom a person that owns or operates anestablishment that is engaged in themanufacture, preparation, propagation,compounding, or processing of a drugshall, upon the request of the Secretary,be provided to the Secretary by suchperson, in advance of or in lieu ofan inspection, within a reasonabletimeframe, within reasonablelimits, and in a reasonablemanner, and in either electronic orphysical form, at the expense ofsuch person.”


New Inspection Protocol Program

• Leverages up to 10 years’ worth ofhistorical data with annual data,plus predictive analytics

• Uses algorithm to sort site data intoinspection priorities

• Goal: replaces routine inspections:– 50% PAI

– 50% postmarket surveillance (e.g., PV)inspections (within 3 years of NDA and5 years of ANDA)

– “for cause” will be one-offs

• FDA Piloted in 2015 and 2016 forICH; formal rollout through 2018


Example Post-Market Risk andSite Selection Criteria

site risk rankingsite risk ranking

inherent product riskfactors

inherent product riskfactors

product failures(reported, active,

shadow, pre-market)

product failures(reported, active,

shadow, pre-market)

dosage/usage factors(intravenous, life-sustaining, etc.)

dosage/usage factors(intravenous, life-sustaining, etc.)

exposure (volume,distribution, etc.)

exposure (volume,distribution, etc.)

postmarketsurveillance factors

postmarketsurveillance factors

see following examplepoints (next 3 slides)

see following examplepoints (next 3 slides)

facility risk factorsfacility risk factors

site hazard signals(recalls, MedWatch,

etc.)

site hazard signals(recalls, MedWatch,

etc.)

site inspection andenforcement history

(inspection w/in past 4years?)

site inspection andenforcement history

(inspection w/in past 4years?)

site demographics andgeographic risk (site

age, type, region, etc.)

site demographics andgeographic risk (site

age, type, region, etc.)

company risk factorscompany risk factors

company financialsignals (bankruptcy,

valuation, etc.)

company financialsignals (bankruptcy,

valuation, etc.)

company products(and revenue makeup)

company products(and revenue makeup)

company & executiveenforcement history

(FDA, SEC, OSHA, etc.)

company & executiveenforcement history

(FDA, SEC, OSHA, etc.)


Example Postmarket Risk andSite Selection Criteria

Post-market risk ranking surveillance factors:• reports of serious and/or unexpected adverse experiences

(within past 1-3 years) by applicant to FDA

• periodic safety reports (if any within past 1-3 years) byapplicant to FDA

• reports of serious problems sent directly to FDA by healthcareprofessionals and consumers

• reports of serious problems sent to FDA by other US–basedregulatory health organizations (CDC, state health board, etc.)

• record, history, and nature of recalls (if any) linked to the site



Post-market risk ranking surveillance factors:• medical and/or scientific literature (within past 1-3 years)

citing the product

• data from pre-market clinical trials

• data from any postmarket clinical trials (comparison to pre-market results)

• medication error analyses and analyses (over past 1-3 years)

• analyses and trends of product use (note: this may includeawareness from social media – consumer reported)

• risk management analysis and review of the above (plus anyoriginal FDA analysis from submission)



Post-market risk ranking factors (drugs only):

• data from other drugs within the samepharmacologic class

• reports of SAEs and unexpected SAEs from other ICHmember agencies (Health Canada, TGA, EMA, et al)

• review of postmarket epidemiologic study findings


Source:FDA, New Molecular Entity Postmarketing Safety Evaluation PilotProgram Final Report, 2008FDA, Manual of Policies and Procedures 5014.1: UnderstandingCDER’s Risk-Based Site Selection Model, 2018

“So what?”

20© 2018 Cerulean Associates LLC www.Ceruleanllc.com

Previously on….


QSIT and CAPA+2

• Quality System Inspection Technique (QSIT)

• Covered 5 different subsystems

• Pharma investigators used a “CAPA+2”approach (“CAPA+Production+1”)

• Examine 10 CAPAs and 10 production records

• Examine 1-2 other area such as:– design control – changes, validation, etc.

– raw material controls (incoming acceptance, supplierqualification, etc.)

– outsourced production-related controls (control overCMO, etc.)

– process validation

– records controls (records retention, data integrity –includes Part 11, etc.)

– distribution controls (anti-counterfeiting, etc.)

– postmarket surveillance (PV) and complaint-handling/MDR


Inspection War Room Setup


Meeting Room A

FDA investigatorsand firm’s host

Meeting Room B

firm’s support staffFDA request

firm’s bestresponse in4-24 hours

various document requests out to firm’s personnel

Firm’s response in15-30 MINUTES orless (“real-time”)

Inspection War Room Change


Meeting Room A

FDA investigatorsand firm’s host

Meeting Room B

firm’s support staffFDA request

firm’s bestresponse in4-24 hours

various document requests out to firm’s personnel

NIPP Tactics

25© 2018 Cerulean Associates LLC

• Select a system used principally for regulatory purposes

• Identify the personnel with administrative access– how were they qualified/trained to be administrators?

• Ask an individual with administrative access to walk throughthe system in real-time and its various data integrity controls –security settings, audit trail settings, data storage controls,user access control lists, etc.

• Sample various digital data files using binomial sampling –review each data file’s properties and an end-user’s ability tochange, edit, delete the data – along with corresponding audittrail documentation

www.Ceruleanllc.com

Be very cautious ifyou’re not trainingyour SMEs and ITadmins on FDA

inspection handling

There is NO time to previewdata in the war-room – it’s alllive in front of the investigator

Commonly Requested “Live”

• CAPA/deviation management and tracking system

• EDMS (for approved controlled docs)

• LIMS

• Electronic lab notebooks

• Complaint and AE management and tracking system

• Automated production systems (SAP, etc.)

Other Possibilities (not as frequent)• Change control management and tracking system• Facility (or equipment) monitoring (for temp, humidity, etc.)• Network file storage locations (including SharePoint)


NIPP Tactics


• On the system, check for folders commensurate with datafraud findings to date

• On the system, try to delete data:– use the application functionality

– use Windows operating system capability (e.g., ctrl+D)

• On the system, try to change time and date– within the application

– within Windows

• On the system, look for:– files in the desktop’s Recycle Bin

– failed/paused print jobs in the printer queue

– files in the Temp folder (e.g., Run%temp%)

www.Ceruleanllc.com

NIPP Tactics


• Request a copy on USB of the past X-years of data associatedwith a particular product (e.g., complaint files, batch files,SUSARs, CAPAs, etc.)…including any audit trails– make sure to capture any user-specific data if multiple logins on one

computerized system

• Ask the firm to provide a summary table of all such data (andcompare their summary to actual data)

• Offsite, examine the files provided:– look for data runs conducted or interrupted but not reported

– look for file annotations, multiple file saves and “last edited” dates

www.Ceruleanllc.com

Some NIPP Example Requests toExpect to Provide• A list of all computerized systems used for cGXP processes

(including Excel sheets with macros/UDFs, etc.)

• Most current Records Retention Schedule your firm follows

• Access Control List (ACL) for system (or the network sharedfolder) __________ for the following dates ____, ____, and____

• Current procedure (SOP, et al) on reviewing audit trails and data


NIPP Realities

• Team-based inspection (at least 1 Quality System/DataIntegrity expert and 1 Product Specialist)

• Heavy reliance on “live” access to your digital records andsystems (no time for “war room” reviews)

• Long-term goal is for ALL members of the ICH to use thismethodology by 2020 (FDA piloted this for ICH)

• Significantly increased likelihood of getting a FDA-483observation (wouldn’t be inspecting your site if not flagged asrisk OR as part of a one-off “for cause”)

• All FDA CPMs, Inspection Guides, etc. being re-written(including inspection policies….)


Case Study from 2016

• Firm makes and sells 5different OTC products

• Buys its APIs

• Onsite microbiology lab

• Onsite analytical chemistry lab

• Onsite distribution warehouse

• Runs two different shifts

• Approx. 350 personnel at site

• Had passed nine different FDAand Health Canada inspectionssince 2000


Pre-Arrival Requests

“Please complete the following three questions prior toour arrival onsite the week of […]:

1. Do you have a policy on data integrity? Yes | No(no need to supply now)


Why would they ask this?


2. Please confirm that computerized system ownersand personnel with administrative-level access willbe made available for the duration of theinspection. Note: If a corporate or global functionperforms this then a communication channel withremote access and visibility to all systems will besufficient.


And why would they need this?


3. Please complete the listing of computerizedsystems (e.g., ERP, LIMS, chromatography systems,MES, security control systems, spreadsheets withmacros, eBMR, EDMS, etc.) used principally inregulatory activities in the table below as follows.Please highlight any stand-alone systems.”


Regulator’s Computerized SystemInventory Format*Type Area/Site Product Name,

Purpose &Supplier

Versionor Model

LastValidationDate

Most RecentChanges(within past year)

Networked(onsite)

Labs (all) ChromeleonChromotographyData System(ThermoScientific)

v 6.9 Dec. 2014 Change controls#73, 76, 81

HostedSaaS

Corporate(all sites)

TrackWise EQMS(Sparta)

v 8.1 Nov. 2015 Change controls#81, 111

Stand-alone

QC Lab Excel SampleTrackingWorksheet(Microsoft withcustom macros)

v Office2013

April 2016 n/a


*Source:MHRA Inspection Notification Letter

¡Caution! Inventory Lists

• Regulators do NOT want a comprehensive listing ofall systems and software

• Regulators DO want a list of computerized systemsused principally for regulatory purposes – and notjust capital systems (e.g., must include spreadsheetswith macros, spreadsheets used for manipulatingraw data, local Access databases, software used formonitoring critical T° and pH, etc.)


Initial Records Requested

• Site data integrity compliance plan showing progress to date

• Inventory list of computerized system validations performed(completed) since last inspection

• List and copies of the CSV and data integrity-related SOPs andpolicies the site trains on and enforces, such as….

• Good data integrity practices (or Good documentation practices)

• Computerized system validations

• Change control

• Records retention and archiving

• Computerized system security

• Backups and disaster recovery

• The most recent change controls related to validated systems

• 18 months’ worth of CAPAs involving the validated systems,the word “data” and other key phrases


Does everyone seewhat this request

forces…?

What FDA was Looking for….

1) Data fraud - backdating, re-running samplesuntil they passed, etc.

2) Data loss – inadvertent or intentional, activedata, historical data

3) Ongoing oversight and verifications by sitebusiness management AND by Quality Unit

4) Consistency of controls – proof of a“consistent state-of-control” around data


What FDA was NOT Looking for….

• Perfection

• Use of the “right” risk methodology

• Detailed computerized system validations

• Who at the site to blame


Within One Day….


No periodic verificationsof data archives to prove

maintained records as per21 CFR §§211.68, 211.180,211.188, 211.194 and 21

CFR § 11.10(c)

Stand-alone labmachines and factoryfloor machines haveno backups as per 21

CFR §211.68(b) and 21CFR § 11.10(c)

No documented datareviews as per 21 CFR

§§211.22, 211.68,211.100, 211.160, 211.180

and 21 CFR § 11.10(e)

No investigations forfailed backups as per

21 CFR §§211.22,211.180, 211.188 and21 CFR § 11.10(b)(j)

No data integrity relatedSOPs or policies as per 21CFR §§211.22, 211.68(b),

211.180 and 21 CFR §11.10(j)

Validations were not“fit for use” (no PQ) asper 21 CFR §§211.68,211.110, 211.113(b)

and 21 CFR § 11.10(a)

Example Warning Letter Excerpt


“Failure to exercise sufficient controls over computerizedsystems to prevent unauthorized access or changes to data,and failure to have adequate controls to prevent omission ofdata. Your firm’s lack of data control calls the reliability of

your data into question.”Warning Letter to Kyowa Hakko Bio Co., Ltd., August 2018

https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/ucm617419.htm

www.Ceruleanllc.com

Notice: Not the“computer” or the “software”

but the computerized system

(the overall system)



“In your annual product reviews, you used unprotected Excelworksheets to perform calculations and statisticalevaluations of production data, such as standard deviationand process capability. These electronic files were notsecured to prevent unauthorized changes, and have no

change history.”Warning Letter to Kyowa Hakko Bio Co., Ltd., August 2018


www.Ceruleanllc.com

We are looking for the controlsover data used to make productsafety, quality and compliancedecisions…like in APR or QSMR



“Your firm failed to exercise appropriate controls overcomputer or related systems to assure that only authorizedpersonnel institute changes in master production andcontrol records, or other records (21 CFR 211.68(b)). Threeof your quality control team leaders had administratorprivileges within your HPLC computerized laboratorysoftware system. Because they review and approveCGMP data, their access level should preclude file deletion

or modification.”- Warning Letter to Hanlim Pharma Co., October 2018


www.Ceruleanllc.com

Notice how this speaks directly toALCOA+ characteristics of Complete,

Attributable, Original, etc.



“Your firm failed to exercise appropriate controls overcomputer or related systems to assure that only authorizedpersonnel could institute changes in master production andcontrol records, or other records (21 CFR 211.68(b)). ... Yourresponse is inadequate because the functions andadministrative privileges of the IT Head, QC Head, and otherpersonnel remain unclear. In your response, clarify the

specific user roles and associated privileges….”- Warning Letter to Sri Krishna Pharmaceuticals, April 2016

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2016/ucm495535.htm

www.Ceruleanllc.com

Notice: FDA clearly revieweddetailed permission levels on

the network and on thevarious computers….

Example FDA-483 Observation


“The integrity of manufacturing and analytical data is notguaranteed through current practices…. Data generated duringproduction is not secured from complete deletion of all datarecords through the Windows operating system. Productionsettings and recipe parameters can be altered or deletedthrough the Flexicon software. There is no review process tomonitor for recipe alteration. Additionally, the performance ofchallenges to user rights levels…were not definitely

documented as part of the initial qualifications of the system….”- FDA-483 to Institutt for Engergiteknikk, January 2016

(see reference materials)No challenge testing

in validationNo challenge testing

in validation

Who verifies – as part of batchproduction and/or release testing –

that production and formulationparameters have not been changed…?

Attendees Need To Vote – Is this aData Integrity Issue?

___ During the inspection, the FDA investigator finds lists ofmeasurements (clearly production related) in the Recycle Binof a computer on the factory floor

___ The IT department of a firm only backs up those systemsthat are attached to the company network – some of thesystems in the labs and on the factory floor are notconnected to the network


Key Point Recap

Investigators ask questions to determine the controlsaround your data from its creation through storage

Inspection questions are not IT-specific, but focus on“how can I trust this data you are showing me”

Expect FDA to request full copies of various electronicrecords and data folders (including relevant audit trails)

The New Inspection Protocol Program inspectiontechnique is designed for the 21st century and helpsinvestigators easily ferret out data integrity issues


Request Expert Training

Visit Cerulean’s website to learn more about our

private, tailored, highly interactive workshops

held onsite at your location

Sample agendas and Request form at

www.ceruleanllc.com/private-training/


thank you


Picture Credits

Photos, images and clip art that appear on these slides have been used to enhance this presentation and may NOTbe used for commercial or promotional purposes without permission from copyright holders.

Do not remove or copy from this presentation.

Contact:Microsoft Corporation

Imgarcade.comiStockPhoto

Cerulean Associates LLC


how the ich’s new inspection protocol program...about your presenter john avellanet trainer for...

Documents