Data Integrity Gap Analysis Checklist

1. Show me an example of a change that shows previous value, last value and the data behind

it? – if this was for a bioprocessor – show me the set point (current) for temperature

control in the processor and the previous temperature set point and the collected data and

meta data.

2. Please show me the training records for the employee operating the bioprocessor /

instrument – looking for the record to show what aspects of data integrity were

documented as part of the operator training to operate the equipment. Ask the operator

some questions to see if they understand and remember and can apply the principles

covered in the training session?

3. Show me any firmware update to a bioprocessor or other instrument you are using – what

about automatic updates are they enabled? How do you validate these updates

(a) what exactly do we mean by firmware

(b) firmware can still communicate and therefore there is a software interface and that can

be updated for data collection and processing

(c) remote connections for automatic updates and bug fixing are RED FLAGS!!

4. What are the user types and access privilege levels for this piece of equipment?

how is it controlled? Is there a current approved controlled list? Who has it? Show me?

Does it include employees who were fired 6 years ago? Does it include new hires? Who

has to authorize addition / removal or alteration of privilege level? Where is that

documented?

5. Please attempt to delete this data – show them which data. Can it be deleted? Now ask

the administrator (who is the administrator – shouldn’t be a department member / head)

to delete data – can they? Is it recorded in the audit trail? Would that audit trail be

printed out or electronically approved as part of the relevant record review / approval.

When you look at the audit trail – does it name the administrator by name as the person

performing the deletion

6. For stand-alone equipment ask to see password privileges – are there assigned users, or a

single (or no) access controls? Is there automatic log out – when a user leaves a

workstation?

7. Do you have a POLICY for access to computerized systems of any kind? Show me the SOP –

show me a list of computerized systems in use in the lab / production / this area – show me

access controls, show me the audit trail, show me where raw data is stored, show me

where you have defined in the SOP what constitutes the raw data – is raw data stored with

metadata or just as numbers? Can the metadata be reconstructed later with the data?

Ask to retrieve and archived record – go back at least six months

8. Are your servers in a controlled environment – is the “cloud” a controlled environment?

9. Are system data and back up and recovery processes validated and periodically challenged

to make sure you aren’t getting garbage back – can recover when needed.

10. Show me the path to the (virtual?) server where data is backed up and show me how you

are certain that data is actually going to that pathway and cannot be diverted and the

types of data that are going there

11. With IT’s assistance run an automated check of files (program and data) comparing what

should be on the system, with what is there and comparing what is there now to what was

there 3 / 6 months ago and then analyze the discrepancies (look for “hidden” files)

look for the number (sequential number) of analyses performed and the number of data

files.

12. Show me recently reviewed data and the audit trail that was reviewed as part of the data

approval process; show me the system audit trail for the software used to generate the

data – compare the system files with the validation configuration – have there been any

changes – when were they made and by whom and were they qualified / validated

13. Show me the recycle bin – what’s in there?

14. Show me the garbage bin – the real one and explain why there are chromatograms in there

and also please explain why you are using chromatograms as waste paper for drawing and

making airplanes…cos I couldn’t shred it and I’m not allowed to put it in the paper recycling

bin – so I didn’t know what to do with it short of eating it!

[IMPORTANT NOTE: you should provide a reasonable way of handling this – such as a

locked waste bin – stuff goes in and is periodically destroyed by authorized personnel but it

is open for audit and should be audited. (unless you have a third party and no key)

POLICY could be “this is not raw data” and therefore it does not need to be retained – but

if annotated – that changes!

15. Tell me what is the raw data?

16. Are you performing any calculations external to your instrument software and if yes, show

me the validation and the data migration (manual or automated); revision control,

algorithm approval and periodic checks. NOTE: Excel is not really validatable but the

custom use can be sort of controlled.

17. Open up Excel please and then show me recent files

18. Show me how to change the date and time zone on the computer / instrument

19. Show me how to disconnect this instrument from the server

20. Explain why the server / network cable is disconnected at the back of this instrument

21. Are the instruments connected to Uninterruptable Power Supply and to an emergency

generator – and are they preventively maintained and periodically challenged to make sure

they work – if NOT, how is data protected from power loss. Is there power surge

protection on data collection / instruments

22. Are your suppliers and vendors on an approved supplier list, audited and have you

discussed data integrity with them? Where is this documented? Do you have quality

agreements in place with them addressing your data governance requirements? What

about contract laboratories doing testing for you?

23. Show me how worksheets are issued and reconciled – are worksheets numbered / kept in

a logbook – same for production records and forms

24. Show me the original and complete qualification data for this system

25. Show me what was done before and after the latest software update to retain data

26. Reintegrate the chromatogram from six months ago – do you use the new software or

retain a version of the old software to be used for this? If you use the new software – does

it give the same results as the old software?

Is there a policy for controlling this

27. For electronic documents how do you know the analyst is using the correct version and it is

still current? How do you tie in document / analytical method revisions with updating of

the method on the relevant instrument

same question for how do you tie in specification updates with COA templates – in a LIMs

system but what if still using ms word / excel for the COA and do you overwrite templates

28. Do you allow manual integration of chromatograms and how do you manage this? Is there

a policy and does the reviewer clearly see the automated integration and the manual one.

29. Show the procedure for control of macros including version controls.

30. Is the PC attached to the instruments connected to the internet? Is it used for e-mails? For

internet remote access? How is it controlled – firewalls, virus protection; automated

updates etc.

31. Policy on external drives (USB memory sticks, CDs etc.) – do the computers conform with

the policy e.g. are there USB ports available?

32. How do vendors access the instruments for preventive maintenance, calibration,

qualification – do they have a password – is it administrator level or ABOVE? How do you

control what they do? How is this documented? Show me the records of the most recent

visit or remote access – how is this logged – are all instances of remote access logged in the

system audit trail? Who is responsible for reviewing the system audit trail and at what

frequency? Are they trained (show me records) on the company’s SOPs

How do you control changes that the vendor may make at the software level including how

are the audit trails documented? If automatic, printout before and after. If not automatic

– print out the programming code before and after? Document if necessary by verbal

explanation. How do you manage vendor access to other interactive software which is not

theirs. That requires control – liase with your IT person – take a “golden image” so that

you can revert to the validated state.

33. Do you use shared passwords? Why is there a password pinned to the equipment? Why

does it say “guest”

34. Do you have the license number of the software noted – where? (IQ) do you have an

illegal software installed? Can software be downloaded from the internet? Is it? Is there a

company policy regarding free software downloads to controlled computers?

35. How do manage archiving – how long do you keep data, how long do you keep systems /

software after the version has been updated or the instrument after it has been retired at

least as a workstation where you could reintegrate raw data? Who has access to the

archive – where is the archive (if on “cloud”) you probably won’t know and some countries

take OWNERSHIP of your data if stored in their airspace!

36. Do you have database personnel roles and responsibilities defined