holistic oversight of empower metadata as part of your ... · data integrity, such that ....

©2019 Waters Corporation 1COMPANY CONFIDENTIAL

Holistic Oversight of Empower metadata as part of your Periodic Review process

OR

Data Integrity Oversight through Analytics

Heather Longden & Ben Dearnaley


What is Involved in Data Integrity?People– Culture for data integrity– Governance and data review– Unique user accounts– Scientific skill– Training– Safeguards against fraud

Quality Separations– Quality standards & reagents– Instrument calibration & maintenance– Qualification– Method validation– System suitability– Quality Columns

IT Components– Secure centralized storage– Network qualification– Disaster recovery plan– Backup and restore process– Archiving electronic data

Laboratory Computerized Systems– Built-in data integrity controls– Computerized system validation– Maximum Automation

- Minimum Human Intervention– Traceability– Periodic review


Data Integrity Oversight Through Analytics - Agenda

What is Period Review and Quality Metrics

The Challenges

Leveraging Vendor Professional Services

Gathering Metrics from Empower Enterprise systems

Wrap Up




The Challenges



Wrap Up


MHRA GxP Guidance:Periodic Review

An increased risk may exist from poor organisational controls or data verification due to an overreliance on the system's validated state.

If data processing has been repeated with progressive modification of processing parameters this should be visible to ensure that the processing parameters are not being manipulated to achieve a more desirable result.

Periodic audits are capable of detecting opportunities for data integrity failures

GxP Data Integrity Definitions and Guidance for Industry March 2018


ISPE Europe Annual ConferenceRome, March 2018


MHRA GxP Guidance:Periodic Review

MHRA GXP Data Integrity Guidance and Definitions; Revision 1: March 2018 §6.15. Data review and approval

Routine data review should consider the integrity of an individual data set e.g. is this the only data generated as part of this activity? Has the data been generated and maintained correctly? Are there indicators of unauthorised changes?

Periodic audit of the data generated (encompassing both a review of electronically generated data and the broader organisational review) might verify the effectiveness of existing control measures and consider the possibility of unauthorised activity at all interfaces, – e.g. have there been IT requests to amend any data post review?

– Have there been any system maintenance activities and has the impact of that activity been assessed?


PIC/S Data Integrity Guidance DRAFT 3

Periodic surveillance, e.g. self inspection processes seek to verify the effectiveness of the data governance policy

The effectiveness of data integrity control measures should be assessed periodically as part of self inspection ( internal audit) or other periodic review processes. This should ensure that controls over the data lifecycle are operating as intended

There should be regular management review of quality metrics, including those related to data integrity, such that significant issues are identified, escalated and addressed in a timely manner.

Caution should be taken when key performance indicators are selected so as not to inadvertently result in a culture in which data integrity is lower in priority

Management can have an independent expert periodically verify the effectiveness of their systems and controls.


§5.6.1 The effectiveness of data integrity control measures should be assessed periodically as part of self-inspection (internal audit) or other periodic review processes. This should ensure that controls over the data lifecycle are operating as intended. §5.6.2 In addition to routine data verification checks, self-inspection activities should be extended to a wider review of control measures, including:

– A review for consistency of reported data/outcomes against raw data entries.

– In situations where routine computerised system data is reviewed by a validated ‘exception report’4, a risk-based sample of computerised system logs / audit trails to ensure that information of relevance to GMP activity is reported as expected

PIC/S Data Integrity Guidance DRAFT 3

GOOD PRACTICES FOR DATA MANAGEMENT AND INTEGRITY IN

REGULATEDGMP/GDP

ENVIRONMENTS

Also in DRAFT 330 November 2018


Routine Data Review: – By the normal reviewer and approver personnel – Before approving each and every result– Use exception flags in Empower to focus review on critical results

Periodic Audit of the Data:– As part of system Periodic Review – As part of Annual Product Review – As part of a wider Data Governance program– As part of an investigation– Can involve a wider team and/or a third party consultant

What Level of Review and When?




The Challenges



Wrap Up


Can You Find The 2 Issues Within These 52 Cards?


With the cards sorted into suits and then numerical order, it is easy to see:

– 5 ♣ is missing

– There are two 8 ♦

This example had 52 cards, with 2 known issues

Now consider looking for an unknown number of issues in THOUSANDS of data sets and audit trail entries

Sorting and filtering and querying the data are essential to find data integrity issues…

Now Find The 2 Issues After Sorting The Cards


Mark Newton and R.D McDowall articles in LCGC Dec 2017 http://www.chromatographyonline.com/data-integrity-metrics-chromatography

The scope of data integrity metrics can cover the four main areas within an organization:– Data governance such as data integrity policy(ies) and the associated training;– Assessment and remediation of laboratory manual processes and computerized systems;– Development and production activities including outsourcing of laboratory analysis;– Quality assurance oversight.

In addition to procedures and training, management must require metrics that permit both scientific and quality assurance (QA) personnel to monitor the laboratory data for troubling trends:

LC/GC Data Integrity Metrics Publication 2017

http://www.chromatographyonline.com/data-integrity-metrics-chromatography


Spotting and addressing potential issues before they affect product Quality Metrics can help FOCUS any periodic review exercise These metrics may

– give warning signs about DI concerns– give an indication of data that needs to be reviewed in more detail– must be selected scientifically– must be easy to collect automatically and– should not drive specific bad analyst behaviour i.e. to deliberately avoid triggering the metric

o E.g NO MANUAL INTEGRATIONo NO SINGLE INJECTIONSo ONLY PROCESS DATA ONCE

Periodic Review…




The Challenges



Wrap Up


CDS Traceabilityand Compliance Readiness

Technical Controls and Traceability

in the CDS

Periodic Review procedures

leveraging analytics

Instrument and Software Qualification Services

Computer System Validation Services


Essentials:– Appropriate Technical Controls

o Challenged and Supported by your Computer System Validation (CSV) process• Should be understood by your staff (Lab Analysts, Managers, QA, IT support)• Should be periodically reviewed and updated as required

– Appropriate Procedural Controlso To augment the technical controls

• Monitored to ensure procedures are followed• Should be periodically reviewed and updated as required

A Step Wise Process


Key Vendor Professional Services

Periodic Review Services– Leverage vendor tools and expert advise

o System Policy /User Privilege Review

o 21 CFR Part 11 / Annex 11 checklist

o Data Integrity Checklist and Gap Analysis

o Laboratory Analytics Implementation


Data Integrity Services

Workshop•Basics of Data Integrity•Culture of Compliance, Computer System Validation

•Manage CAPAs through Analytics

Laboratory Process Assessment•Data Integrity Needs Assessment•Review of Internal Processes and Procedures

•Summary Report of Observations and Remediation Plan

Training•Working Forward - For Analysts and Chemists

•Looking Backward - For Reviewers and QA•QA Empower Awareness training for DI

Consultation•Modernizing Chromatographic Methods•Streamlining Custom Calculations and Report Methods

•Updating SOPs•Computer System Validation Consultancy




The Challenges



Wrap Up


Should be used to understand the current Data Integrity statusThat means take action prior to any audit!

How well is the lab dealing with Data Integrity?– Can this be improved?– Can retrospective actions be taken?

Gives insight in– Processed vs. Unprocessed data– Aborted injections (any kind)– Multi tested injections– Calculation used (same name / formula)

Data Integrity Overview


The Laboratory Analytics team integrates instrument services with data driven laboratory analytics to provide unprecedented visibility into the utilization, efficiency and financial needs of your laboratory’s instruments and people.

Waters Laboratory Analytics

Business Workflow Analysis

Empower 3 Analytics

EDS 365

“Special”Resource

Management Reporting

Waters Laboratory Analytics


With this report we can you to help to check….– Are the technical controls sufficient to meet current data integrity regulations?– Is the data protected from unauthorized modification or access?– Are the analysts following company SOPs for multiple processing and manual

integration?– Are the audit trails configured as per regulatory requirements?– Is the data being managed over it’s life cycle in terms of archival and backup?

Data Integrity and Compliance Status Report


‘Point in time’ statistics are extracted from the Empower database in place, no backup needed

Information collected from installed database and delivered via presentation and report

Presented by Waters Informatics Specialists

Report deliverable detailing how Empower 3 compliance features are actually configured and what type of workflows are in place



While nothing

Automating the Periodic Metrics Review


Computers and software are good for:– Repeatability– Speed with large volumes of data– Verifying numbers against limits– ML tools for pattern identification

Use for general screening of all results to identify suspect patternsGuide any deeper review

Automating the Periodic Metrics Review

People are good for:– Spotting patterns, having instincts and

“gut feelings” – Following a complex trail of events (but

this takes time)

Use for investigating suspect data in careful detail to decide if it is evidence of a data integrity issue


Empower Driven services (EDS365) is a web-based dashboardsolution that provides detailed analyses of critical

chromatography performance data.

EDS365 is comprised of complete, prebuilt analyses that are flexible so that users can focus on what’s most important


Customer Insight From Complex Data

The 14 dashboards cover:– Lab Overview– Data Integrity– Project Usage– System Usage– Custom Fields– Message Center– Users

Example Dashboard: System usage patterns by day and hourShows areas of high/low usage for planning maintenance or

looking for efficiency increases


Analytics Across Your Empower Enterprise

LETS YOU:• Access critical system usage

information across projects.

• Maximize system utilization.

• Manage resources.

• Identify training needs.

• Identify process improvements.

• Plan for capital expenditures.

• Identify opportunities to improve sample turnover time using UPLC Technology

• Identify potential areas of Data Integrity concern


A Detailed View Of Your Empower System(s)

User Accounts

User groups and types

Systems and Injections

Clients and LAC/E boxes

Project information


Overall Score Card Single Injections Single Vial Sample Sets Unprocessed Injections

Aborted single Injections & Sample Sets

Multi-tested Injections and Sample Sets

‘Data Integrity’ Dashboard


Do you measure Processing Method robustness?

Examining the number of re-processes, and the % of manual processing gives an indication of method robustness


Do you measure Processing Method robustness?

Examining the number of re-processes, and the % of manual processing gives an indication of method robustness

Some methods show 100% manual processing but very

low ‘results per channel’


Identify details for injections not associated to a sample set, that have been aborted– Filter by node, Date Range, Project Name and/or System Name

Data Integrity OverviewAborted Single Injections

Sample names and ‘reasons’ entered may explain WHY

this occurs


Identify details for injections not associated to a sample set, that have been aborted– Filter by node, Date Range, Project Name and/or System Name

Data Integrity OverviewAborted Sample Sets

Critical to record a valid ‘reason” for the abort.

To answer “Why?”


Provides the ability to – Identify a list of injections/samples that are not associated to a sample set.– Filtered by Node, Date, Project Name and System Name– View associated information, including custom field information

Data Integrity OverviewSingle Injection Details

Procedures must describe when single injections are

usede.g. this is obviously a

bandspread Sys Suit test


Identify sample sets containing only a single vial– Filter by node, Date Range, Project Name and/or System Name

Data Integrity OverviewSingle Vial Sample Sets

Simple to obscure single injections by using a Sample

Set of one vial


Provides the ability to– Identify a list of Injections/channels that have not been processed– Filter by Node, Project name, Date range– Specific channels can be excluded (Channel NOT filter. e.g. Pressure, Temperature etc)– Sample sets which have not been processed can be highlighted and investigated if required

Data Integrity OverviewUnprocessed Injections

May be more correct to ensue that when Sys Suit fails data must NEVER be processed


Identify injections, sample sets and channels referencing a given Sample Name useful during investigations; sample, instrument, user, date range multiple filters available to minimise scope, e.g. Project, System, Sample Set name/id, user, vial etc

Data Integrity Overview Injections, Sample Sets,Unprocessed channels filtered by Sample Name


Identify same name samples that have been tested more than once (have multiple id’s)– Filters include Node, Date Range, also custom field values– ‘Excluded Values’ section allowing Sample name/s & custom field values to be excluded from the

search

Data Integrity OverviewMulti-tested Samples

Although relying on the ‘entered name’ is no

guarantee of the same solution being injected


Identify same name sample sets that have been tested more than once (have multiple id’s)– Filters include Node, Date Range, also custom field values– ‘Excluded Values’ section allowing Sample name/s & custom field values to be excluded from the

search

Data Integrity OverviewMulti-tested Sample Sets

Entirely legitimate to document and resubmit a Sample Set that had to be

aborted


May be used to gather injection count information per system Identify ‘obscure’ but well used projects

System Usage Information

Primarily used to measure productivity but also may

identify unusual user behavior using filters


Total Inventory– Same name differing

formulas & attributes– Locked vs Unlocked

Fields in Projects Non-Project Fields in

Report Methods Custom Field Usage

– In Sample Sets & Signed off Results

Custom Field Attribute

Custom Field Dashboard


EDS365: An Overview

Collect data from Empower databases

Support business decisions with data

Allow customers to query the data

Process data to generate analyses

Present data back in digestible dashboards




The Challenges



Wrap Up


Real Life Scenario…What is the Wider Impact?

I want to perform an assessment to ensure that individuals are adhering to the newly implemented procedures. I want to perform my assessment based upon risk, what projects should I review?

FDA Warning Letter Statement:“comprehensive assessment of the extent of your practice”

FDA Warning Letter Statement:“Include a detailed action plan ”


Periodic Review

It’s like an internal audit on the compliance of the system– Find concerns BEFORE the audit– Find ways to improve the efficiency of systems and processes– documented evidence of actively searching for data integrity issues

A formal report must be written about the review– Its a regulatory requirement

How often?– Frequency may depend of maturity and criticality (3-18monthly)

Review major and minor changes to determine if any retesting or additional testing of new functionality is required– Is the system still in control and in a validated state?

Review System Audit Trail for correct use of Admin functionalities


Analytical Method Services– Improve your methods to make data processing more automated

Compliance Services (Qualification and Validation) Consultation and Project Management Customer Education

– e-Learning, On-site, mentoring Data Integrity

– Confidence, reliability, and trust in your people, processes, and data

Informatics Services– Software challenges, Data migration, Disaster Recovery, Redundancy

Laboratory Analytics– Oversight though Metrics for your entire Empower Enterprise

Professional Services Provides:


Questions/Discussion

[email protected][email protected]

Waters Corporation34 Maple StreetMilfordMA 01757(508) 482 2000

mailto:[email protected]

mailto:[email protected]

holistic oversight of empower metadata as part of your ... · data integrity, such that ....

Documents