teaching students basic lab skills for a regulated environment

TEACHING STUDENTS BASIC LAB SKILLS FOR A

REGULATED ENVIRONMENT

BIOMAN 2007Lisa Seidman

Madison Area Technical CollegeMadison, WI

WHY THE BASICS?

• Needs of students

• Needs of employers

MYTH 1

• Basics means simple, easy, obvious

• If this were true, far fewer problems in companies and in research labs

BASIC MEANS:

• Vital

• Essential

• Fundamental

• Primary

• Staple

• Must

MYTH 2

• Most of this does not apply in research labs

MYTH 3

• We all learn the basics in high school, or someone else’s class, or by osmosis

MYTH 4

• Basics are boring

BASICS

• What are basics?

• Different answers, but some common themes

HOW TO TEACH BASICS?

• Consciously

• Systematically – model 1: way teach children music– model 2: way grad students are taught

• Underlying principles

THIS WORKSHOP

• Teaching basics consciously

• Systematically

• Underlying principles

TOPICS FOR THIS WORKSHOP

• Quality

• Basic lab task – making a solution

• Metrology (unifying principles)

STORY OF FRANCES KELSEY

• Case study

KELSEY

• Purpose:– introduce GMP– introduce process of developing drug– most important: idea of quality– Bureaucrat – who understood quality

QUALITY: THE BIG (BUT BRIEF) PICTURE

WHAT IS BIOTECHNOLOGY?

The biotechnology industry transforms scientific knowledge into useful products

OVERVIEW

• Talk about product quality systems

–In broad way

–Apply ideas to the various work places we talked about

QUALITY SYSTEMS

• Broad systems of regulations, standards, or policies that ensure the quality of the final product

• GMP/GLP/GCP are examples of quality systems

WHAT IS PRODUCT QUALITY?

• What is a “good” product in biotechnology?

• That depends…

• Consider biotech:

–Research labs

–Testing labs

–Production facilities

QUALITY PRODUCT: RESEARCH LAB

• Research lab, knowledge is product:

–Knowledge of nature (basic research)

–Understanding of technology (applied research, R&D)

QUALITY SYSTEMS IN RESEARCH LABS

• Quality system in research

• Ensure meaningful data–has been around a long time

• It is called:

• “DOING GOOD SCIENCE”

–Less formalized than other quality systems

–No one book spells it out–No laws to obey–But it exists

INFORMAL SYSTEM

• Consequences of poor quality product not life-threatening so–Government seldom involved in

monitoring research quality

–Oversight not generally by outside inspectors or auditors

BUT THERE IS OVERSIGHT

• Oversight is by peers

–Grant review

–Publications

–Reputation

• Compare and contrast situation in research labs and other work places

PRODUCT QUALITY: TESTING LAB

• Testing lab:

–Information about samples

–Good product = result that can be relied on when making decisions

CONSEQUENCES

• A poor quality product can be life-threatening or have serious effects

QUALITY SYSTEMS IN TESTING LABS

• Include most of what we call “doing good science” plus

• Specific formal requirements– Personnel– Equipment – Training– Facilities– Documentation…

• You can find a book that spells it out for:

–Clinical labs

–Forensic labs

–Environmental labs…

ENFORCEMENT: TESTING LABS

• Since consequences of poor product can be life-threatening

–Is outside oversight

• FBI

• EPA

• Etc.

PRODUCT QUALITY: PRODUCTION FACILITY

–Make tangible items

–Quality product fulfills intended purpose

• Ex.: reagent grade salt vs road salt vs table salt

QUALITY SYSTEMS IN PRODUCTION FACILITIES

• Depends on nature of product

• Poor product may or may not have life-threatening consequences

SO, FOR EXAMPLE

• Products for research use, not generally regulated

• Agricultural products are regulated in one way

• Pharmaceutical products are regulated in another

VOLUNTARY STANDARDS

• Companies that are not regulated may choose to comply with a product quality system for business reasons

ISO 9000

• ISO 9000–Formal product quality system–Extensive–Exists in a series of books–Companies comply voluntarily to

improve the quality of products–…and to make more money

OVERSIGHT: ISO 9000

• Oversight by outside auditors, paid by company

BIOTECH AND MEDICAL PRODUCTS

• Many biotech companies that make money make medical/pharmaceutical products

• Consequences of poor product can be life-threatening

SO…

• These products are highly regulated by the government

• But, it wasn’t always this way…

• history…

• CFR, handout

HOW IS QUALITY BUILT INTO A PRODUCT?

• No single answer• Requires:

– Skilled personnel– Well-designed and maintained facility– Well-constructed processes– Proper raw materials– Documentation– Change control– Validation

ENFORCEMENT

• Compliance is enforced by government

–FDA

QUALITY IS BASIC

• Details may not be essential right now

• Idea of quality is essential

LET’S “GO TO THE LAB”VERY BASIC LAB TASKS

1. Write procedure to make 100 mL of a buffer solution that is:

100 mM Tris, pH 7.5 2% NaCl10 μg/mL of proteinase K

2. QC “your solution” by checking its conductivity

3. Check the pH of a Tris buffer solution

PROCEDURE

• For 100 mL of 100 mM Tris solution (FW 121.1) weigh out 1.211 g of Tris base. Dissolve in about 60 mL of water and adjust pH to 7.5.

• Add 2g of NaCl

• 10 μg/mL of proteinase K X 100 mL = 1000 μg = 1 mg. Weigh and add to Tris.

• Dissolve, BTV, check pH

VARIABILITY IN APPROACHES?

• Value of SOPs in ensuring consistency

• Value of communicating among various lab workers

• Documentation

WHAT DO STUDENTS NEED TO KNOW?

• Conceptual– Why they are making solution, context– How to interpret recipe– Basic calculations

• Instrumentation– How to maintain, use, calibrate balance– How to maintain, use, calibrate pH meter– How to measure volume– How to maintain, use, calibrate conductivity meter

• Quality control– How to ensure that solution is what it should be—– How to document work

TEACHING

• Concrete skills– calculations– using equipment– etc.

• These are activities in the lab manual to systematically build these skills

VARIABILITY

• Mike Fino

UNDERLYING PRINCIPLES

• Quality ideas (e.g. reducing variability and documentation, following directions—SOPs)

• Math calculations/ideas that repeat over and over again

• Safety practices

• Metrology principles

INTRODUCTION TO METROLOGY

Lisa SeidmanBioman 2007

DEFINITIONS

• Metrology is the study of measurements

• Measurements are quantitative observations; numerical descriptions

OVERVIEW

• Begin with general principles

• Next: weight, volume, pH, light transmittance (spectrophotometry)

WE WANT TO MAKE “GOOD” MEASUREMENTS

• Making measurements is woven throughout daily life in a lab.

• Often take measurements for granted, but measurements must be “good”.

• What is a “good” measurement?

EXAMPLE

• A man weighs himself in the morning on his bathroom scale, 172 pounds.

• Later, he weighs himself at the gym,173 pounds.

QUESTIONS

• How much does he really weigh?

• Do you trust one or other scale? Which one? Could both be wrong? Do you think he actually gained a pound?

• Are these “good measurements”?

NOT SURE

• We are not exactly certain of the man’s true weight because:– Maybe his weight really did change – always

sample issues– Maybe one or both scales are wrong – always

instrument issues

DO WE REALLY CARE?

• Do you care if he really gained a pound?

• How many think “give or take” a pound is OK?

ANOTHER EXAMPLE

• Suppose a premature baby is weighed. The weight is recorded as 5 pounds 3 ounces and the baby is sent home.

• Do we care if the scale is off by a pound?

“GOOD” MEASUREMENTS

• A “good” measurement is one that can be trusted when making decisions.

• We just made judgments about scales.

• We make this type of judgment routinely.

IN THE LAB

• Anyone who works in a lab makes judgments about whether measurements are “good enough” – – but often the judgments are made

subconsciously– differently by different people

• Want to make decisions– Conscious– Consistent

QUALITY SYSTEMS

• All laboratory quality systems are concerned with measurements

• All want “good” measurements

NEED

• Awareness of issues so can make “good” measurements.

• Language to discuss measurements.

• Tools to evaluate measurements.

METROLOGY VOCABULARY

• Very precise science with imprecise vocabulary– (word “precise” has several precise meanings

that are, without uncertainty, different)

• Words have multiple meanings, but specific meanings

VOCABULARY

• Units of measurement

• Standards• Calibration• Traceability• Tolerance

• Accuracy • Precision • Errors • Uncertainty

Instrumentation

Measurement itself

UNITS OF MEASUREMENT

• Units define measurements

• Example, gram is the unit for mass

• What is the mass of a gram? How do we know?

DEFINITIONS MADE BY AGREEMENT

• Definitions of units are made by international agreements, SI system– Example, kilogram prototype in France– K10 and K20 at NIST

EXTERNAL AUTHORITY

• Measurements are always made in accordance with external authority

• Early authority was Pharaoh’s arm length

• A standard is an external authority

• Also, standard is a physical embodiment of a unit

STANDARDS ARE:

• Physical objects, the properties of which are known with sufficient accuracy to be used to evaluate other items.

STANDARDS ARE AFFECTED BY THE ENVIRONMENT

• Units are unaffected by the environment, but standards are– Example, Pharaoh’s arm length might change– Example, a ruler is a physical embodiment of

centimeters • Can change with temperature

• But cm doesn’t change

STANDARDS ALSO ARE:

• In chemical and biological assays, substances or solutions used to establish the response of an instrument or assay method to an analyte

• See these in spectrophotometry labs

STANDARDS ALSO ARE:

• Documents established by consensus and approved by a recognized body that establish rules to make a process consistent– Example ISO 9000– ASTM standard method calibrating

micropipettor

CALIBRATION IS:

• Bringing a measuring system into accordance with external authority, using standards

• For example, calibrating a balance– Use standards that have known masses– Relate response of balance to units of kg– Do this in lab

PERFORMANCE VERIFICATION IS:

• Check of the performance of an instrument or method without adjusting it.

• Do this in lab.

TOLERANCE IS:

• Amount of error that is allowed in the calibration of a particular item. National and international standards specify tolerances.

EXAMPLE

• Standards for balance calibration can have slight variation from “true” value– Highest quality 100 g standards have a

tolerance of + 2.5 mg– 99.99975-100.00025 g– Leads to uncertainty in all weight

measurements

TRACEABILITY IS:

• The chain of calibrations, genealogy, that establishes the value of a standard or measurement

• In the U.S. traceability for most physical and some chemical standards goes back to NIST

TRACEABILITY

• Note in this catalog example, “traceable to NIST”

VOCABULARY

• Standards

• Calibration

• Traceability

• Tolerance

• Play with these ideas in labs

MEASUREMENT

• What are the characteristics of good measurement?

• Accuracy

• Precision

ACCURACY AND PRECISION ARE:

• Accuracy is how close an individual value is to the true or accepted value

• Precision is the consistency of a series of measurements

EXPRESS ACCURACY

% error = True value – measured value X 100%

True value

Will calculate this in volume lab

EXPRESS PRECISION

• Standard deviation (p. 187-190)– Expression of variability– Take the mean (average)– Calculate how much each measurement

deviates from mean– Take an average of the deviation, so it is the

average deviation from the mean

• Try this in the volume lab

ERROR IS:

• Error is responsible for the difference between a measured value and the “true” value

CATEGORIES OF ERRORS

• Three types of error:– Gross– Random– Systematic

GROSS ERROR

• Blunders

RANDOM ERROR

• In U.S., weigh particular 10 g standard every day. They see:– 9.999590 g, 9.999601 g, 9.999592 g ….

• What do you think about this?

RANDOM ERROR

• Variability

• No one knows why

• They correct for humidity, barometric pressure, temperature

• Error that cannot be eliminated. Called “random error”

RANDOM ERROR

• Do you think that repeating the measurement over and over would allow us to be more certain of the “true” weight of this standard?

RANDOM ERROR

• Yes, because in the presence of only random error, the mean is more likely to be correct if repeat the measurement many times

• Standard is probably really a bit light

• Average of all the values is a good estimate of its true weight

RANDOM ERROR AND ACCURACY

• In presence of only random error, average value will tend to be correct

• With only one or a few measurements, may or may not be accurate

THERE IS ALWAYS RANDOM ERROR

• If can’t see it, system isn’t sensitive enough

• Less sensitive balance: 10.00 g,

10.00 g, 10.00 g

Versus 9.999600 g…

MeanMedianMode

SO…

• Can we ever be positive of true weight of that standard?

• No

• There is uncertainty in every weight measurement

RELATIONSHIP RANDOM ERROR AND PRECISION

• Random error –– Leads to a loss of precision

SYSTEMATIC ERROR

• Defined as measurements that are consistently too high or too low, bias

• Many causes, contaminated solutions, malfunctioning instruments, temperature fluctuations, etc., etc.

SYSTEMATIC ERROR

• Technician controls sources of systematic error and should try to eliminate them, if possible– Temperature effects– Humidity effects – Calibration of instruments– Etc.

• In the presence of systematic error, does it help to repeat measurements?

SYSTEMATIC ERROR

• Systematic error – – Does impact accuracy

• Repeating measurements with systematic error does not improve the accuracy of the measurements

Match these descriptions with the 4 distributions in the figure:

Good precision, poor accuracy

Good accuracy, poor precision

Good accuracy, good precision

Poor accuracy, poor precision

ANOTHER DEFINTION OF ERROR IS:

• Error = is the difference between the measured value and the “true” value due to any cause

Absolute error = “True” value - measured value

• Percent error is:“True” value - measured value (100 %)

“True” value

ERRORS AND UNCERTAINTY

• Errors lead to uncertainty in measurements

• Can never know the exact, “true” value for any measurement.

• Idea of a “true” value is abstract – never knowable.

• In practice, get close enough

UNCERTAINTY IS:

• Estimate of the inaccuracy of a measurement that includes both the random and systematic components.

UNCERTAINTY ALSO IS:

• An estimate of the range within which the true value for a measurement lies, with a given probability level.

UNCERTAINTY

• Not surprisingly, it is difficult to state, with certainty, how much uncertainty there is in a measurement value.

• But that doesn’t keep metrologists from trying …

METROLOGISTS

• Metrologists try to figure out all the possible sources of uncertainty and estimate their magnitude

• One or another factor may be more significant. For example, when measuring very short lengths with micrometers, care a lot about repeatability. But, with measurements of longer lengths, temperature effects are far more important

REPORT VALUES

• Metrologists come up with a value for uncertainty

• You may see this in catalogues or specifications– Example:

measured value + an estimate of uncertainty

UNCERTAINTY ESTIMATES

• Details are not important to us now

• But principle is: any measurement, need to know where the important sources of error might be

SIGNIFICANT FIGURES

• One cause of uncertainty in all measurements is that the value for the measurement can only read to a certain number of places

• This type of uncertainty. It is called “resolution error”. (It is often evaluated using Type B methods.)

SIIGNIFICANT FIGURE CONVENTIONS

• Significant figure conventions are used to record the values from measurements

• Expression of uncertainty

• Also apply to very large counted values– Do not apply to “exact” values

• Counts where are certain of value• Conversion factors

ROUNDING CONVENTIONS

• Combine numbers in calculations

• Confusing

• Look up rules when they need them

RECORDING MEASURED VALUES

• Record measured values (or large counts) with correct number of significant figures

• Don’t add extra zeros; don’t drop ones that are significant

• With digital reading, record exactly what it says; assume the last value is estimated

• With analog values, record all measured values plus one that is estimated

• Discussed in Laboratory Exercise 1

ROUNDING

• A Biotechnology company specifies that the level of RNA impurities in a certain product must be less than or equal to 0.02%. If the level of RNA in a particular lot is 0.024%, does that lot meet the specifications?

• The specification is set at the hundredth decimal place. Therefore, the result is rounded to that place when it is reported. The result rounded is therefore 0.02%, and it meets the specification.

GOOD WEB SITE FOR SIGNIFICANT FIGURES

• http://antoine.frostburg.edu/cgi-bin/senese/tutorials/sigfig/index.cgi

THERMOMETERS

• Look at the values for the thermometers on the board.

• Significant figure conventions can guide us in how to record the value that we read off any measuring instrument.

• With these thermometers, correct number of sig figs is _______.

THERMOMETERS

• Were they accurate?

• How could we figure out the “true” value for the temperature?

REPEATING MEASUREMENTS

• Would repeating measurements with these thermometers, assuming we did not calibrate them, improve our ability to trust them?

• Is their error an example of random or systematic error?

CALIBRATION

• Calibration of the thermometers could lead to increased accuracy

• This is a type of systematic error

• In the presence of systematic error, repeating the measurement will not improve its accuracy

TOLERANCE

• Here is a catalog description of mercury thermometers.

• Are these thermometers out of the range for which their tolerance is specified?

PRECISION

• Were they precise? How could precision be measured?

• Would calibration help to make them more precise?

CALIBRATION

• Calibration would probably not improve their precision

RETURN TO OUR ORIGINAL TYPE OF QUESTION

• Are our temperature measurements “good” measurements?

• How do you make that judgment?

• Can we trust them?

THERMOMETERS – GOOD ENOUGH?

• Are times that we need to be very close in temperature measurements. For example PCR is fairly picky.

• Other times we can be pretty far off and process will still work.

EXPLORE SOME OF THESE IDEAS

• In lab:– Calibrate instruments– Use standards– Check performance of pipettors– Record measurement values– Calculate per cent errors– Calculate repeatability

ASSAYS

SAME IDEAS APPLY

• A good assay is one can trust when making a decision

• Accuracy and precision

• Linearity

• Limits