developing a robust automated image-based …...©2006 millennium pharmaceuticals, inc. 1 techniques...

©2006 Millennium Pharmaceuticals, Inc. 1

Techniques in High-Content Screening and Assay Development

Workshop

Developing a Robust Automated Image-Based Assay

Doug Bowman,Sr. Mgr Imaging Sciences

Millennium Pharmaceuticals

Drug Discovery TechnologyAugust 7, 2006


The Big PictureThe Far Side by Gary Larson

• Cell-based assay• Immunofluorescence• Technologies / Instrumentation• Screening Assay• People


• IF Staining– Fix cells to substrate– Permeabilize membranes– Block – Add primary– Add secondary

• Image Acquisition• Image Analysis

HCS Assays using Immunofluorescence


• Provide physiological context for assaying drug activity via the quantitative analysis of cellular and sub-cellular events.

• Analysis also allows for the classification of cellular phenotypes based on morphological parameters.

“HIGH CONTENT”

Cellular “High Content” Imaging

Translocation

Protein Phosphorylation

NeuriteOutgrowth

Mitotic Index

Spindle Assembly / Morphology


Tissue…

• High content also!– Many of same assays

• Similar issues• High resolution

scanning

100um 20x objective


Technologies

• Instrumentation– Liquid handling robotics– Automated microscope-based

systems capable of acquiring high resolution images

• Software algorithms

OperaAutomatedMicroscope

Discovery-1

AcapellaProGuruMetaXpress

InCell1000

ArrayScan


Image-based assays

= 2λ / image * 5 sites / sample * 3 mice / timepoint= 30 images / timepoint= 400+MB / 5 timepoints

• Large volume of data

= 2λ / image * 9 sites / well * 96 wells / plate= 1728 images / plate = 1GB+ / plate3 plates / hr (Opera) 1 plate / hr (Discovery-1)

= 3λ / image * 15 sites / membrane= 30 images / sample= 120MB / sample


Working Group

Cell Biology Automation

Imaging

Assay Design

SoftwareDevelopment

ComputationalScience


Developing a Robust Automated Image-Based Assay:

Guidelines

• Use automation where possible• Visualize images• Develop robust process controls• Use data-driven approach


A data-driven approach to assay design

• Dissect components of the assay• Using control experiments

– Identify drivers of variability• Minimize them experimentally or account for them statistically

– Optimize parameter selection (robust)• Assess performance of assays in

project-relevant context


HCS Assay Components

• Dissect components• Using control experiments

– Identify drivers of variability– Minimize them experimentally

or account for them statistically

Wet LabProtocols

ImageAcquisition

Image Analysis

AssayAnalysis






Wet LabProtocols

ImageAcquisition

Image Analysis

AssayAnalysis


Wet Lab Protocols

Reagents

Automation: liquid handling IF Staining Protocol

Automation: cell plating

Reproducibility

Day to day variability


Older Multidrop tube cassettes found inaccurate – relative inaccuracy ~25% and CVs ~2%

New cassettes purchased and validated –relative inaccuracy ~2% and CVs ~0.7%

100ul manual (12 channel Multipipette) 4.43%100ul Multidrop 1.67%50ul Platemate 0.75%

50ul Tecan 0.80%10ul Precision 2000 1.34%

CVs

• Accuracy and Reproducibility– Compound addition / dilution– Cell Plating– Antibody addition / protocols

Wet Lab Protocols:

Testing liquid handling using Artel system

Artel MVS


Facilitate a systematic comparison of a large number of assay parameters to maximize data quality while minimizing the time required to find optimal parameters.

- fix- perm- block- primary ab dilution - secondary ab dilution- buffers- wash buffers- number of washes- time/temp of incubation….etc

Wet Lab Protocols:

IF Staining Protocol


Antibody Optimization

8 row trough

John Donovan

Matrix of 6 fixatives - 4 blocks - 4 abdilutions to assay

96 individual conditions


PBS TBS MTSB

1:500

1:750

1:1000

0

4 Abdilutions

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

PFA PBS PFA TBS PFA MTSB

Roche

BSA

PBS

Mixed Serum

MeOH Perm buf 1 MeOH Perm buf 2 MeOH Perm buf 3

4 Abdilutions

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

1

2

PFA PBS PFA TBS PFA MTSB

Roche

BSA

PBS

Mixed Serum

MeOH Perm buf 1 MeOH Perm buf 2 MeOH Perm buf 3

Wet Lab Protocols:



DNA / Tubulin

Before After

• Comparison of > 400 parameters (~2 weeks)• Enhances analysis of spindle morphology• Save $ Antibody concentration

Scatter P lot

Calculated Column (2)01 02 03 04 05 06 07 08 09 10 11 12

H

G

F

E

D

C

B

A

S/N

Wet Lab Protocols:







Wet LabProtocols

ImageAcquisition

Image Analysis

AssayAnalysis


Image Acquisition

Auto-Focus

Depth of Field Image Corrections

Instrument Choice

Reproducibility

Cell Plating: # sites

Illumination System


Image Acquisition

Auto-Focus


Instrument Choice

Reproducibility


Illumination System


Image Acquisition:

AutoFocus

• Image-based and Laser-based%Nuclear count in HT29 cells @ 24 hours

Millennium Pharmaceuticals Inc.

ML00618017-001-A

[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic0

10

20

30

40

50

60

70

80

90

100

95%CI = 0.1498 µMEC30 = 1.51734 µM


60 wells, 9 sites / well


AVG %Ratio Nucl/Cyto in U2OS cells

[CMPD] (µM)0.01 0.1 1 10

AV

G %

Int r

atio

0

20

40

60

80

100

12095%CI = 0.2107 µM

IC50 = 0.57859 µM

Image Acquisition:

AutoFocus


• Minimize experimentally– Instrument parameters– Plate selection

• Identify, Remove: Auto-focus Classifier– Classify several hundred images by hand– Use image processing measures to extract

quantities that vary based on edges and intensity

– Train a classifier to call out-of-focus wells

Image Acquisition:

AutoFocus


-40 um -20 um 0 um 20 um 40 um

60

70

80

90

100

110

-50 -40 -30 -20 -10 0 10 20 30 40 50

Z Position (um)

Tota

l Nuc

lei

Original

Segmented

Image Acquisition:

AutoFocus


• BayesNet classifier learns differences between in-focus and out-of-focus images

• Algorithm maximizes separation of classes on training set by:

– automatically assigning weights for each predictor

– automatically selecting threshold

Δ IntensitySharpness 2 Δ AreaSharpness 1

Total score

0.080.21 0.18

0.55

yes>1.4?in focus

noout-of-focus

Threshold

Belief parameters

Image Acquisition:

AutoFocus


• Results of BayesNet assessed using cross-validation:

– Algorithm is trained on 9/10ths of the data and tested on remaining 1/10th

– Process is repeated 10 times – Error rates are averaged

• 93% to 98% of the wells were called correctly (for 3 different cell types).

• Nearly all classification errors occurred in 'marginal' category

• Zero-R (baseline classification error)= 32%

•

Image Acquisition:

AutoFocus


Filtering out-of-focus images via machine learning

Millennium Pharmaceuticals Inc.[CMPD] (µM)

0.01 0.1 1 10

AV

G %

Int r

atio

0

10

20

30

40

50

60

70

80

90

100

110

120

130

95%CI = 0.2401 µM

IC50 = 0.58004 µM


0.01 0.1 1 10

AV

G %

Int r

atio

0

10

20

30

40

50

60

70

80

90

100

110

120

130

95%CI = 0.0711 µM

IC50 = 0.6034 µM

Original Data Filtered Data

Image Acquisition:

AutoFocus


Image Acquisition

Auto-Focus


Instrument Choice

Reproducibility


Illumination System


• What is it?– Fluorescence illumination system

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

1 33 65 97 129 161 193 225 257 289 321 353 385 417 449 481 513 545 577 609 641 673

Uniform sample

Image Acquisition:

Illumination system variation


• Is it a problem? ↓ Cell counting assays↑↑ Cell intensity assays↑↑ Phenotyping

Flat-Field Correction

0.94

0.96

0.98

1

1.02

1.04

1.06

1.08

1.1

1.12

Nuclei Count Nuclei Intensity Tubulin Intensity

%Non FF CorrectedFF Corrected

Image Acquisition:

Illumination system variation


Image Acquisition

Auto-Focus


Instrument Choice

Reproducibility


Illumination System


Image Acquisition:

Depth of Field

20x


- 4.0 um- 2.0 um0.0 um2.0 um4.0 um

Image Acquisition:

Depth of Field


Z = -4.0um Z = -2.0um Z = 0.0um Z = 2.0um Z = 4.0um

“Best Focus”Single Plane

Image Acquisition:

Depth of Field


Outof

focus

Multiple optical sections @ 2.0um intervals

In focus

“Best Focus” Image

Tissue Imaging• Challenge: depth of focus of the objective is less than

the flatness of the specimen. This results in portions of a single image plane being out of focus.






Wet LabProtocols

ImageAcquisition

Image Analysis

AssayAnalysis


Image Analysis

Cell Plating: density

Algorithm parameters Signal / Noise

Threshold algorithm

Reproducibility

Magnification


Image Analysis:

Cell Plating Density• Assess

– Optimal cell plating density to minimize segmentation errors

• DOE:– Serial dilution– DAPI– 6 replicates

XXXXXXXXXXXXH

XXG

XXF

XXE

XXD

XXC

XXB

XXXXXXXXXXXXA

121110987654321

Cell plating density







Image Analysis:

Cell Plating DensityCell Plating vs. Count

0

500

1000

1500

2000

2500

3000

Cell Plating Density

Nucl

ei C

ount


Image Analysis:

Threshold algorithm

0

200

400

600

800

1000

1200

1400

1 3 5 7 9

Drug Concentration

Nuc

lei C

ount

Object CountStd Area CountAdaptive Count


Image Analysis:

Signal / Noise• Assess

– Optimal exposure times to minimize acquisition time and maximize Signal/Noise

• Image-based autofocus• Intensity measurements• Segmentation

– DOE:– Uniformity Plate– DAPI– Multiple exposures

XXXXXXXXXXXXH

XXG

XXF

XXE

XXD

XXC

XXB

XXXXXXXXXXXXA

121110987654321


• Single Uniformity Plate, 9 sites/well

Decreasing exposure time

Image Analysis:

Signal / Noise


Adaptive Threshold

0

20

40

60

80

100120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12

Row

Nuc

lei C

ount

3.912.772.111.581.33

Fixed Threshold - Auto

020

4060

80100120

140160

180200

1 2 3 4 5 6 7 8 9 10 11 12

Row

Nuc

lei C

ount

Fixed Threshold - User

0

20

40

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12

Row

Nuc

lei C

ount

• Effect on thresholding

Image Analysis:

Signal / Noise


Count Nuclei

0

0.2

0.4

0.6

0.8

1

1.2

3.91 2.77 2.11 1.58 1.33

eSNR

% o

f 160

*: p<10-100, Welch’s two-tailed t-testError bars are standard deviations

*

*

Image Analysis:

Signal / Noise


• What objective to use?– Segmentation accuracy– Dose response curve re-sampling

Image Analysis:

Magnification


10x

20x

FKHR-GFPDNA

Image Analysis:

Magnification



0.01 0.1 1 10

AV

G %

Int r

atio

0

10

20

30

40

50

60

70

80

90

100

110

120

130

95%CI = 0.0744 µM

IC50 = 0.65515 µM


0.01 0.1 1 10

AV

G %

Int r

atio

0

10

20

30

40

50

60

70

80

90

100

110

120

130

95%CI = 0.0372 µM

IC50 = 0.64973 µM

10x 20x

Image Analysis:

Magnification


• Assess– Effect of variability of user-defined parameters– Different software algorithms (vendors)

• DOE:– Dose Response– Multiple algorithmparameters

XXXXXXXXXXXXH

XCXG

XCXF

XCXE

XCXD

XCXC

XCXB

XXXXXXXXXXXXA

121110987654321

dose response

dose response

dose response

dose response

dose response

dose response

Image Analysis:

Algorithm parameters


• User variability?

Image Analysis:



• Single Plate, 9 sites/well, dose-response• 30 variations of DNA, pH3 threshold

%Nuclear count in HT29 cells @ 24 hour


[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0499µMIC30= 0.65911µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0499µMIC30= 0.65911µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0594µMIC30= 0.65607µM



TEST

[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0462µMIC30= 0.60712µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0652µMIC30= 0.65776µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.05µMIC30= 0.61013µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.1432µMIC30= 0.66916µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0953µMIC30= 0.66289µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.198µMIC30= 0.66241µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0901µMIC30= 0.66033µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0605µMIC30= 0.65709µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0439µMIC30= 0.65786µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0439µMIC30= 0.65786µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0634µMIC30= 0.65637µM



TEST

[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0442µMIC30= 0.60945µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0664µMIC30= 0.65259µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0448µMIC30= 0.60799µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.1733µMIC30= 0.66231µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0858µMIC30= 0.65996µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.1274µMIC30= 0.66117µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0707µMIC30= 0.65613µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.046µMIC30= 0.65204µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0476µMIC30= 0.65429µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0463µMIC30= 0.6547µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0463µMIC30= 0.6547µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0627µMIC30= 0.6526µM



TEST

[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0404µMIC30= 0.6082µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0999µMIC30= 0.66593µM



[CMPD] (µM)0.1 1 10

AV

G %

Mito

tic

0

10

20

30

40

50

60

70

80

90

100

95%CI= 0.0685µMIC30= 0.65928µM

IC30 (µM)

0.1000.2000.3000.4000.5000.6000.7000.8000.9001.000

Algorithm Parameters (DNA, pH3 adaptive threshold)

IC30

(µM

)

DNApH3

Image Analysis:







Wet LabProtocols

ImageAcquisition

Image Analysis

AssayAnalysis


Assay Analysis

# of replicates

Transfer metrics Variance Component Analysis

(Plate-Plate / Day-Day)

Instrument Choice

Reproducibility

96 vs 60 wells

# images / well


Assay Analysis

# of replicates



Instrument Choice

Reproducibility

96 vs 60 wells

# images / well


Assay Analysis:

Variance Component Analysis

• Assess variability– Row / column– Plate-plate– Day-day

• DOE:– Single drug concentration (Uniformity plates)– 4 plates x 3 days

Day 1 Day 2 Day 3


96 wells 60 wells

• Inner 60 wells had median CV which was 35% lower than that for all 96 wells

Assay Analysis:

Uniformity Data


Single Plate

12 Plates •Variability among days was largest component followed by plate•Replicate dose response curves should be run on different plates on different days to account for largest amount of variability.

Total Mean: 27.7Total Variance: 32.0 (std dev: 5.7)Coefficient of Variation: 20.4%

Assay Analysis:

Uniformity DataVariance Component Analysis


Assay Analysis:

Re-sampling

• Determine– How many sites / well?– How many replicate wells?– Is the assay robust?– Does assay perform well across facilities?

• Sample– Uniformity Plates– Control Compound– Multiple Compounds


Bootstrapping

Population

• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample


Bootstrapping

Population Sample



Assay Analysis:

Control Compound• Assess

– Effect of number of replicate curves upon EC30 estimates

– Effect of number of images per well upon EC30 estimates

• DOE:– 6 dose response

curves– 5 plates

XXXXXXXXXXXXH

XCXG

XCXF

XCXE

XCXD

XCXC

XCXB

XXXXXXXXXXXXA

121110987654321

dose response

dose response

dose response

dose response

dose response

dose response


Assay Analysis:

Control Compound Data


Re-sampling Results

Distribution of EC30 values


Error bars are 95% CI All CVs are less than 12%

-Results incorporate differences among plates and there were fourobservations with high values.

Re-sampling Results:

Determine # wells


Going from 6 images to 9 has almost as much impact on standard error as going from 9 to 16.

10uM DMSO

Re-sampling Results:

Determine # sites/well


Assay Analysis

# of replicates



Instrument Choice

Reproducibility

96 vs 60 wells

# images / well


Assay Analysis:

Transfer

• Assess– Compare re-sampled EC30 estimates to estimates

generated in Oncology Discovery• DOE

– 4 compounds– 3 replicates / compound– Each compound was run on 4 plates = 12 curves– All plates run in two facilities


Four compound re-sampling results

•A and B were not significantly different, C and D were approximately 60% higher

• Three wells from each concentration were sampled.• Fit dose response curve and calculated EC30.• Total of 250 curves• Examine distribution of EC30 values

Compound

Log(

EC30

)

Assay Analysis:

Transfer


• Uniformity Plates

• Single Compound

Assay Analysis:

Instrument selection

Discovery-1 Opera

* determination based on signal/noise, speed, robustness, availability

• Signal/Noise, acquisition time, CVs

• Z’, EC50 CVs, confidence interval


Overall Assay Evaluation

• ~2 months• Confidence

– Immunofluorescence staining protocol

– Automation / liquid handling– Instrument / Instrument QC– Software Algorithm – Robust assay– Transfer from Discovery to

Platform Group

Platform validation

Variance ComponentsAnalysis

Bootstrapping on wellsand images

Bootstrapping withcontrol compound

Bootstrapping withreference panel

Identify sources of noise

Format assay

Determine robustness

Make site comparisons

Establish decision criteriaDecision criteria

simulation

Process controls

Platform validation

Variance ComponentsAnalysis

Bootstrapping on wellsand images

Bootstrapping withcontrol compound

Bootstrapping withreference panel

Identify sources of noise

Format assay

Determine robustness

Make site comparisons

Establish decision criteriaDecision criteria

simulation

Process controls


Summary

• Look at the images

• Run uniformity plates to identify and minimize noise (biology, instrument, and algorithm)

• Use analysis modules throughout process


Process Deliverables

• Development Guidelines– QC metrics: instruments, algorithm– IF staining protocol optimization– Image analysis optimization

• Production Guidelines – Statistical analysis for transfer– Ongoing QC metrics


Acknowledgements• Molecular & Cellular Oncology

– Ben Amidon– Jeff Ecsedy– Natalie Roy D’Amore– Liz Carideo– Kara Hoar – Claudia Rabino– Michelle Tighe-Nestor, – Denise Driscoll– Deborah Wysong

• Cancer Pharmacology– Arijit Chakravarty– Katherine Galvin

• Lead Discovery– John Ringeling– John Donovan

• Computational Sciences– Sudeshna Das– Mike Pickard


Breakthrough science. Breakthrough medicine.SM

developing a robust automated image-based …...©2006 millennium pharmaceuticals, inc. 1 techniques...

Documents