multicolour analysis and local image correlation in confocal

Multicolour analysis and local image correlation inconfocal microscopy*

D. DEMANDOLX & J. DAVOUSTCentre d’Immunologie CNRS-INSERM de Marseille-Luminy, Case 906, 13288 MarseilleCedex 9, France

Key words. Co-localization, confocal microscopy, fluorescence microscopy,fluorogram, image analysis, image correlation, multicolour analysis, pixelhistogram.

Summary

Multiparameter fluorescence microscopy is often used toidentify cell types and subcellular organelles according totheir differential labelling. For thick objects, the quantitativecomparison of different multiply labelled specimens requiresthe three-dimensional (3-D) sampling capacity of confocallaser scanning microscopy, which can be used to generatepseudocolour images. To analyse such 3-D data sets, wehave created pixel fluorogram representations, which areestimates of the joint probability densities linking multiplefluorescence distributions. Such pixel fluorograms alsoprovide a powerful means of analysing image acquisitionnoise, fluorescence cross-talk, fluorescence photobleachingand cell movements. To identify true fluorescence co-localization, we have developed a novel approach based onlocal image correlation maps. These maps discriminate thecoincident fluorescence distributions from the superimposi-tion of noncorrelated fluorescence profiles on a local basis,by correcting for contrast and local variations in back-ground intensity in each fluorescence channel. We believethat the pixel fluorograms are best suited to the qualitycontrol of multifluorescence image acquisition. The localimage correlation methods are more appropriate foridentifying co-localized structures at the cellular orsubcellular level. The thresholding of these correlationmaps can further be used to recognize and classify biologicalstructures according to multifluorescence attributes.

Introduction

In fluorescence microscopy, the simultaneous or sequentialdetection of several channels is of great importance inbiological applications where several markers have to besuperimposed and compared (Brelje et al., 1993). Proteinimmunofluorescence cytochemistry and fluorescence in situ

hybridization (FISH) of nucleic acids now permit thedetection of a wide variety of macromolecules within cellsand tissues. In conventional epifluorescence microscopy ofthin specimens, the multiple detection of nucleic acids byFISH can be achieved by combining one or severalfluorophores for each hybridization (Nederlof et al., 1990);the resulting diversity allows the identification of more than10 sites with three distinct labels mixed in multiple ratios(Dauwerse et al., 1992; Nederlof et al., 1992a,b).

For thick specimens, the images acquired with conven-tional microscopes suffer from the out-of-focus blur of thefluorescence emissions, impairing the estimation of co-localization between several markers. Confocal microscopyhas extended the capacity of optical microscopy by allowing3-D sampling of specimens while excluding out-of-focusblur (Wijnaendts-van-Resandt et al., 1985; White et al.,1987; Brakenhoff et al., 1989; Shotton, 1989). Theimproved axial resolution allows the 3-D localization offluorescence markers (Sheppard, 1989) and 3-D imagereconstruction (Van der Voort et al., 1989, 1993; White,1995). Moreover, the lateral resolution may, under idealconditions, be improved compared with nonconfocal opticalmicroscopy (Sheppard & Cogswell, 1990; Wilson, 1990).The confocal out-of-focus fluorescence rejection laid thefoundations for the quantitative comparison of multifluor-escence signals (Akner et al., 1991; Sandison & Webb,1994; Sandison et al., 1995). In multifluorescence confocalmicroscopy, the specimens are scanned with one or severalexcitation laser lines, allowing sequential or simultaneousdetection of several fluorescent markers through multipleacquisition channels (Mossberg et al., 1990; Brelje et al.,1993). Compared with flow cytometry, cells are analysed ata slower rate, yielding a confocal image of high spatialresolution of a limited number of cells. Multifluorescenceconfocal microscopy can also yield valuable information forthe identification of multiply labelled structures, or tomonitor the time evolution of intracellular Ca2+, H+ ion orcyclic AMP concentrations to be followed by fluorescence

Journal of Microscopy, Vol. 185, Pt 1, January 1997, pp. 21–36.Received 21 June 1996; accepted 16 September 1996

21q 1997 The Royal Microscopical Society

Correspondence to: Denis Demandolx. Tel: (+33) 4 91 26 94 36; Fax: (+33) 491

26 94 30; E-mail: [email protected]

* Paper presented at MICRO 96, London, 2–4 July 1996.

ratio imaging of dedicated probes (Poenie et al., 1986;Bright et al., 1989; Adams et al., 1991).

Double and triple immunofluorescence labelling permitsone to estimate the co-localization between fluorescentmarkers in the same field (Schubert, 1991; Brelje et al.,1993; Paddock et al., 1993; Paddock, 1995). Severalmethods based on the superimposition of colour channelshave been explored to reveal coincident immunofluores-cence labelling (Arndt-Jovin et al., 1990; Fox et al., 1991;Humbert et al., 1992, 1993; Dutartre et al., 1996) or tocompare FISH vs. immunofluorescence labelling (Leger etal., 1994). The local subtraction of appropriately scaledfluorescence channels (Akner et al., 1991) and the doublethresholding of fluorescence values (Mossberg et al., 1990)have also been used to compare confocal images. Globalcorrelation coefficients have been explored to estimate thedegree of correspondence between two related fluorescencemicrographs (Manders et al., 1992, 1993). In addition,several authors have identified fluorescence cross-talk andimage registration defects due to chromatic aberrations ormisalignment of confocal optics as fundamental problems inmultifluorescence acquisitions (Akinyemi et al., 1992;Sandison et al., 1995; White et al., 1996). A judicious choiceof fluorophores, laser types, objective lenses, fluorescencefilter sets, detection pinhole settings and photon detectorsoften reduces these limitations (Mossberg & Ericsson, 1990;Sheppard et al., 1992; Brelje et al., 1993; Keller, 1995; Tsien& Waggoner, 1995), which can further be compensated forby digital image processing (Carlsson & Mossberg, 1992;Brelje et al., 1993). Multicolour image analysis software ishighly desirable to control the acquisition conditions, toevaluate the problems of registration and aberrations and toreveal the co-localized fluorescence distributions.

To check the quality of image acquisition procedures andto study the correspondence between multifluorescencelabelled structures, we define here several levels of imageanalysis. We first propose the use of an improved dual-channel look-up table (LUT), second, we create statisticalrepresentations of multiple fluorescence data, and third wedevelop local image correlation methods. The enhanceddual-channel LUT allows a higher colour discrimination ofdouble fluorescence labelling, the pixel fluorograms providea statistical representation of multifluorescence distributions,and the local correlation maps reveal co-localized struc-tures. Use of the improved LUT, pixel fluorogram analysisand local image correlation have to be combined for theglobal and local analysis of fluorescence co-localization.

Materials and methods

Materials, cells and antibodies

All chemicals and proteins were purchased from Sigma Chemi-cal Co. (St Louis, MO, U.S.A.). Major histocompatibility

complex (MHC) class II IAk and invariant (Ii) chain positivefibroblast cells were obtained after transfection of IAka, IAkb

and Ii chains using the calcium phosphate precipitationmethod as described (Salamero et al., 1990; Humbert et al.,1993). Anti-MHC class II IAk 10.2.16 monoclonal anti-bodies and rabbit polyclonal anti-cathepsin D antibodieswere obtained and used as described (Humbert et al., 1993).Rabbit polyclonal anti-Ii chain antibodies were generouslyprovided by Nicolas Barois (Centre d’Immunologie CNRS-INSERM de Marseille-Luminy, France). Texas-Red-coupledwheat germ agglutinin (WGA) was obtained from MolecularProbes Inc. (Eugene, OR, U.S.A.).

Intracellular indirect immunofluorescence

Mouse fibroblast and B lymphoma cells were cultured for48 h on glass coverslips. After washing three times withPBS, cells were fixed for 15 min with a 4% paraformalde-hyde solution in phosphate-buffered saline (PBS) andpermeabilized with 0.1% Triton X100 solution. Cells weresubsequently washed and blocked with PBS containing0.2% gelatin. Antibodies were diluted to a 10 mg mL–1

concentration in PBS–gelatin. To perform indirect immuno-fluorescence labelling, cells were incubated for 20 min withprimary antibodies at room temperature, and then washedthree times, before incubation with either FITC- or Texas-Red-coupled anti-IgG antiserum (1/200 dilution in PBS–gelatin). The labelled cells adherent to the glass coverslipswere mounted on glass slides with Mowiol embeddingmedium (Humbert et al., 1993). To stain the surface ofliving cells, we incubated cells grown on coverslips with1 mg mL–1 of Texas-Red-coupled WGA for 20 min at 20 8C.The cells were mounted in optical chambers containing400 mL of medium as described (Pollerberg et al., 1986).

Configuration of the confocal laser scanning microscope

Confocal laser scanning microscopy was carried out using aLeica TCS 4D instrument (Leica Lasertechnik, Heidelberg,Germany), based on a Leitz DMRBE microscope interfacedwith an argon–krypton laser specified to have a maximumpower of 25 mW in each line (488, 568 and 647 nm). Thetotal laser output power was set to 25 mW. Selecting the488- and 568-nm lines simultaneously, we have 8 mW foreach line. Owing to an initial attenuation with filters, tooptical fibre coupling, to the confocal excitation apertureand to an over-illumination of the 1.4 numerical aperture(NA) of the microscope 100 PL APO objective, the inputpower is reduced <25-fold. The total illumination powerthus lies around 0.3 mW for each laser line on the specimenunder these conditions. To minimize the noise and to keepthe photobleaching rate below 1% for FITC, we selected anacquisition time of 1 s per scan and averaged 16 scans toproduce each 1024 × 1024-pixel image, as a standard

22 D. DEMANDOLX AND J. DAVOUST

q 1997 The Royal Microscopical Society, Journal of Microscopy, 185, 21–36

procedure for all the applications. (For the application ofFig. 2, where we wanted to accentuate the effects ofphotobleaching, we tripled the laser power.) For most of theapplications, we selected a field of interest of 512 × 512pixels from the raw micrographs. The Nyquist’s criterionindicates that to digitize a signal whose cut-off frequency isfc, it is necessary to sample at a frequency of at least 2fc. Thisprinciple can be applied in confocal microscopy to select theappropriate sampling rate of the detected signal (Young,1988). With the 100 PL APO objective, the lateral opticalresolution is about 0.18 mm and the axial resolution alongthe z-axis is 0.5 mm. We used a sampling step of 0.095 mm inthe plane of section and 0.25 mm in the axial direction.

In multicolour analysis, the image registration betweenchannels is of great importance. We checked thatwavelength-dependent distortions were small comparedwith the optical resolution for objects close to the coverslip.The possible chromatic aberrations (Akinyemi et al., 1992)are considerably reduced by using plan apochromaticobjective lenses, provided that for an oil-immersion lensimaging an aqueous specimen, the plane of focus is adjacentto the coverslip (Keller, 1995). Actual measurementsprovided by Leica Lasertechnik indicate that the relativeaxial shifts between 488 and 568 nm lines are never greaterthan 50 nm using 100 1.4-NA PL APO oil-immersionobjective. Experimental verifications using the detection ofdouble-labelled subcellular endosomal structures present inthe cells close to the coverslip/medium interface showed novisible shift between green and red fluorescence emissionseven at the periphery of the field. The use of a single argon–krypton laser feeding a monomode optical fibre guaranteesthe alignment of laser lines. Focal series of four horizontalplanes of section spaced at 0.25 mm have been monitoredsimultaneously for FITC and Texas Red. As we recordedseries of optical sections close to the coverslips, we haveminimized the optical aberrations due to differences ofrefractive index inside the specimen. Moreover, the Mowiolmounting medium has a refraction index close to the glasscoverslip (n = 1.52). For standard acquisitions, we used adouble dichroic mirror for the excitation beam, a band pass520–560 nm barrier filter for FITC, a long pass barrier filterabove 580 nm for Texas Red detection and two photo-multiplier tubes (PMT). Depending on the relative amountof FITC and Texas Red, and the gain settings of the twodetectors, simultaneous dual-channel acquisition can gen-erate a small amount of fluorescence cross-talk betweenFITC and Texas Red channels (Mossberg & Ericsson, 1990;Carlsson & Mossberg, 1992; Brelje et al., 1993).

Implementation of image processing programs

The individual 8-bit-encoded 1024 × 1024- or 512 × 512-pixel images, one from each channel for each plane of section,were combined and visualized with a 16-million-colour

display screen and printed using a Codonics NP-1600 photo-graphic network colour printer utilizing dye-sublimationtechnique (Codonics Inc., Middleburg Heights, OH, U.S.A.).Image processing, false-colour mapping, pixel fluorogramand local image correlation algorithms were implementedin C programming language using Microware’s C LanguageCompiler System (Microware, Des Moines, IA, U.S.A.) andthen linked with Leica’s image processing library. Ourcurrent programs run on a Motorola 68040 microprocessorsystem (Motorola, U.S.A.) with Microware’s OS-9 operatingsystem. To display dual-colour fluorescence images, we haveused both classical and extended LUT as described in theResults. Pixel fluorogram and local image correlationtechniques are extensively described in the Results and theAppendix.

Results

Improved look-up tables

Through all this work, we have used double fluorescenceconfocal micrographs obtained from fibroblast tissue culturecells expressing the IAkab MHC class II molecules and theassociated Ii chain (Humbert et al., 1993). Figure 1a showsa typical plane of section recorded 1 mm above the solidcoverglass support used for cell culture and simultaneouslyobserved with FITC and Texas Red acquisition channels.MHC class II and Ii chain are represented in green and red,respectively. This type of double fluorescence micrographcan be visualized with a standard red–green LUT producinga limited range of shades from green for FITC to red forTexas Red. Note that the plasma membrane is only labelledby indirect FITC immunofluorescence revealing the surfaceexpression of MHC class II molecules. Internal vesicles arelabelled with both fluorophores but with different ratios,indicating a partial segregation of MHC class II moleculesand Ii chains (Humbert et al., 1993).

As shown above, multifluorescence confocal micrographsare usually represented by the superimposition of their red,green and blue colour-coded channels. However, this directrepresentation leads to interpretation problems linked witha differential sensitivity of our perception of red, green andblue. To circumvent these difficulties, we have extended thisdual-channel LUT from cyan for FITC to magenta for TexasRed through green, yellow and red for halfway combina-tions, improving the discrimination of the fluorescenceratios between FITC and Texas Red (Fig. 1b). The conceptbehind this extended dual-channel LUT is inspired byconsideration of the classical hue–saturation–luminance(HSL) colour space. Here, a specific hue value is defined foreach ratio between the pure FITC and Texas Redfluorescence components x and y (see LUT bar in Fig. 1a).The corresponding luminance is here calculated by the ‘or’fuzzy logical operator x + y – xy/255 varying from 0 to 255


MULT IC OLOU R A NALYSIS IN CONFOCAL MICROSCOPY 23

for initial 8-bit-encoded fluorescence values. Other lumi-nance operators are conceivable, such as the maximumvalue of x and y. The final choice and the usefulness of suchcolour correction depends on accepted conventions. If thehues of the extended LUT are too far from conventional red–green LUT, their interpretation can require an adaptationtime.

Statistical representations of the fluorescence micrographs

To provide a statistical representation of the image data setsfor each channel, we can calculate the pixel histogramdistribution, showing the number n(k) of each pixel valuewithin the image:

nðkÞ ¼ cardinalfxði; j Þ ¼ kg; ð1Þ

where n(k) denotes the number of pixels whose fluorescencevalue x(i,j) equals k for all possible coordinates (i,j).

A large contribution of weakly fluorescent areas leads to asharp peak at the origin of the graphs for each channel(Fig. 1c,d). To classify the occurrences of double-labelledpixels in the micrographs, we decided to use second-orderhistograms to estimate the joint probability density linkingtwo fluorescence distributions (Pratt, 1991). Second-orderhistograms show the number n(k,l) of pixels for which thevalues of each channel x(i,j) and y(i,j) equal, respectively kand l:

nðk; l Þ ¼ cardinalfxði; j Þ ¼ k; yði; j Þ ¼ lg: ð2Þ

As an application, second-order semilogarithmic histogramshave been calculated for each cell circled on the micrographof Fig. 1b (Fig. 1e–h). By analogy with the well-knowncytofluorogram representation from flow cytometry, we areproposing to call this representation a pixel fluorogram.Pixel values of FITC and Texas Red components (k,l) areplaced, respectively, along the x- and y-axes. These graphscan also be represented by means of contour lines(Demandolx & Davoust, 1995). The sharp peak at theorigin of these pixel fluorograms corresponds to the lowluminance levels of the double fluorescence labelling. Itshould be noticed that these pixel fluorograms are composed

of a multitude of radial streaks showing the local correla-tions between fluorescence distributions. Each co-localizedstructure corresponds to a radial streak. On the other hand,horizontal or vertical clouds of dots lying along the x- and y-axes are related to single labelled structures in themicrographs. Usually, each combination of pixel values(k,l) corresponds to a specific colour on the dual-channelmicrographs. It is therefore possible to derive colour-codedpixel fluorograms from the actual semilogarithmic repre-sentation, as shown in Fig. 1i,k. Dots on such pixelfluorograms are represented with the same colour as inthe raw dual-colour image. A straightforward visualizationof double-labelled pixels in the micrograph can be achievedby selecting the pixels above a threshold for eachfluorescence channel in the pixel fluorogram (Fig. 1j,l).This is equivalent to a double thresholding using the imageamplitudes as attributes (Mossberg et al., 1990; Pratt, 1991;Pal & Pal, 1993). We preferred to limit ourselves to eachelliptical area of interest because it is often difficult to use aglobal threshold for a whole field containing several cells.Provided that the nonspecific fluorescence is uniformlydistributed within the object, this method allows us todistinguish double positive, single positive and backgroundareas according to the selected portion of the pixelfluorogram.

Application of the pixel fluorograms

The pixel fluorograms can be used to identify double-labelled pixels of interest and are helpful in the analysis ofsmall variations between closely related images. Signal-to-noise ratio (SNR) and basic movements can be analysed inthis way. As an example, a confocal fluorescence micro-graph of B lymphoma cells is shown in Fig. 2a. Cells arelabelled indirectly with primary anti-IAk MHC class IImolecules and FITC-coupled secondary antibodies. We haverepresented the pixel fluorograms obtained between twosequentially single scanned images (Fig. 2b), four-time-averaged scanned images (Fig. 2c) and 16-time-averagedscanned images (Fig. 2d). The diagonal cloud of dots on thepixel fluorogram gets thinner as the fluorescence signal and


Fig. 1. Double-labelling analysis with pixel fluorograms. (a) Double immunofluorescence labelling of fibroblast transfectants monitored byconfocal microscopy. IAkab MHC class II molecules are represented in green after FITC labelling and the Ii chain is represented in redafter Texas Red labelling. The additive superimposition of both channels gives a range of yellow shades as shown in the LUT bar in the bottomright corner. Field of view 50 × 50 mm. (b) Same field as in (a) but the combination between FITC and Texas Red is visualized using anextended LUT with more shades for different FITC and Texas Red ratios. Four elliptical areas of interest 1–4 have been defined aroundthe cells to compute the pixel fluorograms. (c,d) Semilogarithmic pixel histograms of the FITC and Texas Red components, respectively.(e–h) Second-order pixel fluorograms from the four elliptical areas of interest 1–4 displayed in (b). The grey level is inversely proportionalto the logarithm of pixel numbers. To distinguish isolated dots, we have used a black background. As a standard procedure, FITC and TexasRed components are represented along the x- and y-axes, respectively. (i) Colour-coded pixel fluorograms of the same elliptical areas of interest1 from panel (b). Pairs of pixel values present in the area of interest are represented by a dot of the same colour in the pixel fluorogram and inthe image. (j) Double-labelled pixels selected and displayed in white in the colour-coded pixel fluorogram (i) are visualized on the micrograph(j). (k,l) This is the same as (i,j) but for the area of interest 3 of (b).


the number of scans increase, indicating an improved SNR(Fig. 2b–d). The width of the cloud-of-dots is proportional tothe standard deviation of the acquisition noise anddecreases as 1/

��

np

where n denotes the number of scans(Mossberg et al., 1990). In confocal fluorescence micro-scopy, the noise originates mostly from the photon numberwhich follows Poisson statistics (Young, 1996) character-ized by a standard deviation proportional to the mean of thecorresponding signal. This explains the parabolic distribu-tion of the cloud-of-dots of Fig. 2. The photobleaching ofFITC between two acquisitions is also detected as a tilt of thediagonal as shown in Fig. 2d (Brakenhoff et al., 1994). Ifparts of the specimen undergo limited movement duringtime lapse recording of a living specimen, off-diagonal dotswill appear on the pixel fluorogram. Such an example isshown in the second part of Fig. 2, where we have presentedthe superimposition of the surface labelling of living cellsrecorded at time 0 and time 1 min (Fig. 2e) and at time 0and time 15 min (Fig. 2g). The corresponding colour-codedpixel fluorograms become more spread as a function of time(Fig. 2f,h, respectively). For a clear detection of themovement, good SNR and limited fluorescence photobleach-ing are obviously required. More sophisticated methods,optimized to track the cell movements, can also be

considered. However, the pixel fluorograms developed hereare very helpful in the analysis of SNR, photobleaching andcan reveal partial displacements in the specimens.

Application of intensity-weighted pixel fluorograms

To further quantify the fluorescence values for each class ofpixels identified on the pixel fluorograms, we have defined afluorescence intensity-weighted pixel fluorogram from thedouble confocal immunofluorescence micrograph of Fig. 3a.In such a pixel fluorogram, the number of pixels n(k,l) ismultiplied by the FITC or the Texas Red fluorescenceintensities k and l leading to the integrated FITC and TexasRed fluorescence values k.n(k,l) and l.n(k,l). Colour-codedintensity-weighted pixel fluorograms are then obtained bymerging FITC and Texas Red fluorescence integrated valuesusing dual-colour LUT (see Fig. 3b–e). As illustrated below,fluorescence intensity-weighted pixel fluorograms can beused to analyse fluorescence cross-talk and backgroundfluorescence.

The fluorescence cross-talk occurs when part of afluorophore emission is detected in the wrong fluorescencechannel. In particular, FITC and Texas Red fluorophores canemit photons of longer wavelength which are detected and

Fig. 2. Pixel fluorograms for noise and movement analysis. (a) Confocal immunofluorescence micrograph of IAk MHC class II molecules in Blymphoma cells indirectly labelled with FITC. Field of view 70 × 70 mm. (b-d) Pixel fluorograms of sequential acquisitions from the section (a)for 1-, 4- and 16-scan averaged images, respectively. Components from the first and second acquisition are represented along the x- and y-axis, respectively. The tilt of the cloud-of-dots (d) is due to the photobleaching of FITC after 16 scans with triple laser power. (e) Time-lapseconfocal image of living fibroblasts labelled on their surface with Texas Red-coupled WGA. Two time lapse acquisition with a 1-min intervalare superimposed. Field of view 50 × 50 mm. (f) Colour-coded pixel fluorogram of the two superimposed components of (e). (g,h) Same as (e,f),but with a time interval of 15 min.



added in another fluorescence channel (Carlsson & Moss-berg, 1992). FITC emission can leak into the Texas Redacquisition channel and Texas Red can leak into theCyanine 5 channel. Fluorescence cross-talk results in verycharacteristic pixel fluorograms. Everything happens as ifthe fluorophore x,y-base was no longer orthogonal. SinceFITC-stained regions of the specimen emitting greenphotons are also emitting a smaller number of red photonsdetected in the Texas Red channel, the FITC x-axis makes anacute angle as shown in Fig. 3b. Pixel fluorogramrepresentations are thus important to evaluate the cross-talk between fluorescence channels and to perform theappropriate corrections. Different approaches have beenproposed for the compensation of colour images (Carlsson etal., 1994). A practical approach implies the specification ofa matrix operator that will remove cross-talk contributionsas described by Brelje et al. (1993). Matrix coefficients canbe determined by estimating the cross-talk percentage fromthe slant in the contours of the intensity-weighted pixelfluorogram. The tangent of the angle made with the FITCaxis defines the ratio of the FITC intensity detected in theTexas Red channel to the FITC intensity detected in the FITCchannel (Fig. 3b) and consequently the matrix coefficientsfor its correction (Fig. 3c–e).

Most indirect immunofluorescence micrographs containnonspecific fluorescence levels referred to as backgroundfluorescence. This background fluorescence may be due toautofluorescence of the specimen, or to nonspecific absorp-tion of fluorescence coupled secondary antibodies usuallypresent all over cellular specimens. This gives a significantcontribution to the integrated fluorescence, as shown in theintensity-weighted pixel fluorogram close to the origin of thegraph. Therefore, we recalculated the intensity-weightedpixel fluorograms after appropriate fluorescence offsetting(Fig. 3d,e). In these representations, dots on the pixelfluorograms are weighted with offset-corrected fluorescenceintensities. Mossberg et al. (1990) proposed a method for thechoice offsets from each channel histogram. Ideally theoffsets should correspond to the averaged FITC and TexasRed background fluorescence recorded under the sameoptical conditions for a negative control specimen. Bycalculating intensity-weighted pixel fluorograms using off-set corrected fluorescence values, we can now resolvedifferently the bins of integrated fluorescence. The jointdistribution of fluorescence integrated values now revealsthe contribution of specific fluorescence measurements in abetter way. To minimize the local dispersion of the pixelfluorogram distributions, linear low-pass or nonlinear


Fig. 3. Application of intensity-weighted pixel fluorograms. (a) Double confocal immunofluorescence micrograph of fibroblasts. FITC-labelledantibodies directed against the lysosomal enzyme cathepsin D labels intracellular vesicles whereas Texas Red-coupled WGA binds mostly atthe cell surface. Field of view 50 × 50 mm. (b) Intensity-weighted pixel fluorogram of (a). The white line corresponds to the slant induced bythe 12% cross-talk of FITC to the Texas Red channel. (c) Same as (b) but with a cross-talk compensation. (d) Same as (c) but FITC and TexasRed fluorescence have been offset to correct for background fluorescence before the computation of the fluorogram. Offset values of 16 forFITC and 16 for Texas Red are shown by vertical and horizontal lines, respectively. (e) Same as (d) but using a preprocessing 13 × 13-pixellow-pass filter.


median filtering is usually advantageous. To remove fromthe pixel fluorograms the contribution of the high spatialfrequency components due to the acquisition noise, we haveapplied a 3 × 3-pixel low-pass convolution filter (Pratt,1991) throughout this paper. Using as a test a more drasticGaussian convolution filter whose standard deviation is setto 3 pixels on a 13 × 13-pixel window, all clouds-of-dots aresmoothed (Fig. 3e).

The intensity-weighted pixel fluorograms can revealseveral bins of pixels corresponding to statistically signifi-cant areas of interest, such as different levels of background,and single or double-labelled areas. However, very large orcomplex images contain many overlapping bins of pixelswhich leads to confused situations for the interpretation ofpixel fluorograms. Background fluorescence correction andlocal averaging are needed to extract more information onmultifluorescence images. The evaluation of specific vs.background fluorescence is always more accurate on a localbasis.

Local image correlation

In order to detect co-localization of fluorescence distribu-tions on a local basis, we have to extract more informationfrom the neighbouring pixels. In the strict sense, a co-localization means a correspondence between two fluores-cence distributions varying locally in the same way.Provided that the co-localized area stretches over relativelyfew pixels and that we do not reach fluorescence saturationlevel (Visscher et al., 1994), the cloud-of-dots of a very localpixel fluorogram is clustered along a straight line (Fig. 4a,b).The slope of such a cloud-of-dots is related to thefluorescence ratio between the markers used. The shapeand the width of the cloud depend on acquisition noises.The global pixel fluorograms are indeed composed of amultitude of radial streaks describing each individual double

positive structure. In the following, we have developed alocal approach using a sliding window to characterize thedispersion of the cloud-of-dots in second-order histograms.To find out the correspondences between local fluorescencedistributions, we have compared two methods based onlocal image correlation. The first is based on the calculationon the local correlation coefficient of raw images (CCRI) andthe second on the local joint moment of standardizedimages (JMSI). This second-order joint moment, also calledthe noncentral covariance or correlation function (Svedlowet al., 1978; Anderson, 1984), is computed between theimages using a sliding window after a statistical differencingfilter to standardize the two image data sets (Pratt, 1991).The local CCRI method consists of computing the correla-tion coefficient for a sliding window between the two rawimages. In the two methods, detailed in the Appendix, weobtain a complete correlation map expressing a local co-localization coefficient for each pixel. Both methods aresuited for registered image pairs.

To illustrate the principle of the JMSI method, we havechosen a small area of interest of about 100 pixels around avesicle in which both fluorophores are co-localized, zoomedand circled in Fig. 4a. If we form a pixel fluorogramrepresentation, the corresponding binary cloud-of-dots iselongated, showing a linear dependence between intensitiesand thus a significant correlation (Fig. 4b). To perform astatistical differencing, we have, respectively, subtractedfrom each pixel x(i,j) and y(i,j) of the raw images, theaveraged values x(i,j) and y(i,j) of the surrounding pixelscomputed with a 9 × 9-pixel Gaussian convolution filterwhose standard deviation is set to 1.33 pixel (Fig. 4c) (Pratt,1991). We have next divided the pixel values of x and yimages by their modified standard deviation jx(i,j) andjy(i,j) calculated over the same window at the coordinate(i,j) as defined in Eqs. (4)–(7) in the Appendix. The localmeans and standard deviations of the resulting images are

Fig. 4. Standardization of image data sets. (a) A subregion of Fig. 1b is selected to illustrate the principle of the local joint moment of stan-dardized images (JMSI). The circle identifies a small co-localized vesicle of about 100 pixel area. Field of view 12.5 × 12.5 mm. (b) Pixel fluor-ogram from the circled area of interest in the raw micrograph (a). (c) First step of statistical differencing showing the subtraction of the low-pass component of each raw image. (d) Second step of statistical differencing showing the normalization of the cloud-of-dots. By dividing eachcentral image by their modified local standard deviation, the cloud-of-dots has been straightened along the diagonal of the graph.



normalized (Fig. 4d). As a result of this standardization, thecloud-of-dots is now straightened along the bisecting line ofthe pixel fluorogram. The elongated shape along thediagonal shows the linear dependence between correlatedintensities in the source image of Fig. 4a. Noncorrelated

structures give rise to cloud-of-dots dispersed in alldirections. To visualize the resulting joint moment, wehave multiplied and locally averaged the two standardizedimages. Considering each pixel neighbourhood, this locallyaveraged joint moment is sensitive to the shape of the


Fig. 5. Application of local image correlation methods. (a) Correlation maps derived from the local correlation coefficient of raw images(CCRI), applied to the micrograph shown in Fig. 1a, using a black and white LUT between values 0 and 1 of the CCRI result. (b) Sameas (a) but for the correlation map of the local JMSI method displayed with the same LUT. (c,d) Same micrographs as in Fig. 1b, but the con-tours of the thresholded correlation maps (a) and (b), respectively, are shown as black lines to reveal co-localized vesicles.


standardized cloud-of-dots. Highly diagonal clouds give riseto maximum values of the joint moment.

Detection of co-localized areas with local image correlation

Image correlation maps contain positive and negative areasdepicting positive or inverse correlations between localfluorescence distributions. The local correlation maps of theCCRI and of the JMSI methods have been calculated fromthe original images of Fig. 1a (Fig. 5a and 5b, respectively).Here, we have only displayed the positive part of correlationmaps since it is the most significant one for our applications.Both methods use local and standardized data sets, and bothcorrelation maps reveal the co-localized structures ofinterest in the raw images. As shown in Fig. 5a, the localCCRI correlates the edges better than the top areas whosedistributions are less stretched out than the edges. In localJMSI method, the sliding estimation of local statisticalparameters allows a better discrimination of top areas andan easier determination of the threshold needed for furthersegmentation of co-localized structures (Fig. 5b). To com-pare the local image correlation maps with the correspond-ing multifluorescence micrographs, we have first segmentedthe co-localized areas from each correlation map (Pal & Pal,1993). Practically, for each correlation map, we haveextracted regions above a global correlation threshold of0.5. We have then superimposed in black the contours ofthese segmented areas on the raw image of Fig. 1b for theCCRI and JMSI correlation maps (Fig. 5c and 5d,respectively). For sake of clarity on the images, we do notshow the contours of correlated objects smaller than 9pixels. This segmentation allows the extraction of highlycorrelated structures whatever their fluorescence intensi-ties. Some cyan and magenta structures are scored as co-localized indicating a linear dependence between unevenlabelling.

Since the JMSI method reveals the co-localized vesiclesbetter than the CCRI, it facilitates subsequent segmentation.In our second example, we have applied the JMSI method toconfocal micrographs from other fibroblast cells having ahigher surface expression of MHC class II molecules labelledindirectly with Texas Red-coupled antibodies. The secondindirect labelling was carried out with antibodies coupledwith FITC and directed against the cathepsin D lysosomalenzyme. As in Fig. 5d, we have extracted and superimposedin black the contours of correlated regions on the raw image(Fig. 6a). Black-surrounded areas, thresholded from thelocal JMSI map, reveal the presence of a collection of co-localized structures located in the cytoplasm of doublepositive cells. Finally, to estimate the correspondencesbetween local correlation methods and pixel fluorograms,one can either classify pixels from individual segmentedobjects as shown in Fig. 4a or pool the segmented pixelssurrounded in Fig. 6a in an intensity-weighted pixel

fluorogram representation (Fig. 6b). On the other hand,the pixel fluorogram calculated on noncorrelated areas onlyis highly dominated by the background fluorescence(Fig. 6c). As expected from the correlation method, super-impositions of nonrelated FITC and Texas Red labelling areexcluded from the correlated areas as shown in themicrograph (Fig. 6a) and in the intensity-weighted pixelfluorogram (Fig. 6c).

Fig. 6. Pixel fluorograms and correlated areas. (a) Application ofthe JMSI method to identify and surround co-localized structuresin a double immunofluorescence labelling of fibroblasts. FITC-labelled antibodies directed against the lysosomal enzyme cath-epsin D reveal intracellular vesicles and Texas Red antibodies direc-ted against mutated MHC class II molecules now label the cellsurface and some internal vesicles. The JMSI correlation mapreveals colocalized vesicles but excludes nonrelated fluorescencesuperimposition. Field of view 50 × 50 mm. (b,c) Intensity-weightedpixel fluorograms of (b) correlated and (c) noncorrelated areasdefined in (a). The local JMSI correlation map extracts both weaklyand highly labelled structures (b). Double positive pixels that havenot been selected in colocalized areas in (a) are displayed in theintensity-weighted pixel fluorograms (c) with single positive andbackground pixels.



Discussion

Optical microscope users usually face difficulties during thecourse of multifluorescence imaging experiments carriedout with confocal or nonconfocal equipment. Among these,multicolour visualization, quality control of multifluores-cence acquisition and finally how to evaluate the statisticalsignificance of fluorescence co-localization are the mainpractical problems. For dual-channel imaging purposes, wehave proposed first to enhance the standard red–greensuperimposition with an extended dual-channel LUT,thereby allowing a higher discrimination between the twolabels. Second, we have defined pixel fluorograms, whichprovide a statistical representation of multiple fluorescenceacquisitions, in a similar way to flow cytometric cytofluoro-grams. Third, we have developed a local approach for thedetection of colocalization, by estimating the linear depen-dence between the fluorescence distributions on a slidingwindow assuming that intensity vs. binding follows thesame characteristics for the two probes.

Visualization of multiple fluorescence micrographs

For multicolour representations, when the emission spec-trum of each fluorophore produces a distinct colour whichis close to one of red–green–blue (RGB) primary colours, wegenerally associate each fluorescence channel to the RGBcolour plane that best matches the fluorophore emissioncolour. This straightforward method produces colourcombinations close to the ones observed through themicroscope oculars provided that the emission energy issimilarly distributed among different fluorescence detectors.In the other cases, an arbitrary RGB colour plane can beattributed to each channel. For double-labelling images, it ispossible to improve upon the standard red–green LUT byusing the third primary colour to create an extended dual-channel LUT, allowing a higher discrimination of fluores-cence ratios while conserving the luminance information.Triple-channel acquisition allows less flexibility. In theextended dual-channel LUT presented here, five hues definefive sectors of fluorescence ratios, and the luminance is afunction of both fluorescence intensities as defined in theResults section. We think that our sequence from magenta,red, yellow, green to cyan enhances our perception of colourcombination, as shown in Fig. 1. Other choices of hue andluminance formulae can be applied to visualize fluorescenceratio imaging, such as obtained with Ca2+ or H+ probes(Poenie et al., 1986; Bright et al., 1989). Anyhow, theinterpretation of colour superimposition remains linked tosubjective criteria. The extension of multicolour represen-tation to 3-D fluorescence data sets is not obvious. Still,some interesting approaches have been proposed (Van derVoort et al., 1989; Messerli et al., 1993; Messerli & Perriard,1995).

Statistical analysis of multichannel images

Despite using an improved LUT, it is generally difficult toestimate the occurrences of dual fluorescence measure-ments on multicolour micrographs. To visualize thefluorescence joint distribution, we have created second-order pixel fluorograms showing the frequency of registeredpairs of pixels coming from both fluorescence channels. Bydefining a threshold for each channel on these graphs, wecan now classify pixels according to their intensities in fourcategories: the background, the single-labelled areas, and thedouble-labelled areas. To provide a user-friendly correspon-dence between images and pixel fluorograms, we havecreated a colour-coded pixel fluorograms. As a complementto the spatial representation of images, we think thatsecond-order pixel fluorograms are appropriate to revealintrinsic properties of multifluorescence images that wouldotherwise be difficult to infer from the micrographs directly.The first group of applications concerns the analysis of thetime-dependence of single labelling including the SNR,fluorescence photobleaching and cell movement detection.A second group of applications deals with the multicolouranalysis of the channel registration, and measurements offluorescence cross-talk and background fluorescence. Thesetwo groups of applications require either time-resolved ormultifluorescence images. Photobleaching and fluorescencecross-talk can be corrected sequentially depending on theavailable data sets provided that single labelled structuresare present in the micrographs.

Owing to the low number of detected photons (Pawley,1995), the photon statistics have been proven to be themain limiting factor of the SNR (Young, 1996). As shown inour results, the pixel fluorograms allow one to visualize thenoise distribution for each grey level from two successivelyscanned images. The intrinsic dispersion of the cloud-of-dotsdue to the acquisition noise limits the precision with whichtwo images can be compared. Temporal and/or spatialfiltering techniques improve the SNR, as revealed by pixelfluorograms constructed from time-averaged images. On theother hand, a too long or too intense laser excitation offluorophores induces a fluorescence attenuation due tophotobleaching, which can also be estimated from pixelfluorograms. The acquisition noise can be detected as thewidth of the diagonal cloud-of-dots, and the photodamageas a tilt of the pixel distribution. Finally, limited movementsof specimens generate random distributions of points on thepixel fluorograms.

In multifluorescence acquisitions, image registration andfluorescence cross-talk are the main experimental problemsfor which a statistical analysis of multicolour pixels is highlydesirable. Relative distortions between multifluorescencechannels originate from chromatic aberrations in theobjective lenses, in the confocal optics, or from themovement of the specimen between sequential acquisitions.



Defect in the colour registration is detected by off-diagonaldots on pixel fluorograms if the specimen contains sharpand colocalized fluorescence structures.

To reveal the cross-talk between fluorescence channels,we have designed intensity-weighted pixel fluorograms.Nevertheless, the conditions of acquisition should first beoptimized to reduce cross-talk during the acquisition byadequate selection of laser lines, fluorophores and chro-matic filters (Mossberg & Ericsson, 1990; Brelje et al.,1993). For given acquisition settings, the cross-talkcoefficients can be evaluated by comparing the twoacquisition channels on second-order pixel fluorograms foreach single-labelled specimen (Mossberg et al., 1990). Ascarried out in fluorescence flow cytometry, a real time cross-subtraction method can be implemented at the level of thesignal acquisition electronics. Furthermore, the autofluor-escence and/or the nonspecific absorption of fluorescentantibodies generate a non-negligible part of the signal, inaddition to any instrumental acquisition noise. Ourintensity-weighted pixel fluorograms show that the lowluminance levels constitute the main part of the total imagefluorescence. Special intensity-weighted pixel fluorogramswere designed after offsetting the fluorescence values toremove a large part of the nonspecific backgroundfluorescence from the integration. However, the statisticalproperties of the noise vary within the observed areas, and itis generally difficult to associate bins for each component ofthe noise in wide-field pixel fluorograms. Therefore, werecommend others to restrict the area of interest fromwhich the pixel fluorogram is calculated. Within local areas,colocalized structures are defined by an elongated cloud-of-dots, the slope of which is related to the ratio of fluorescencelabelling. A single hue of the extended dual-channel LUT isattributed to each co-localized distribution, provided thatthe background fluorescence is small enough comparedwith the specific fluorescence.

Pixel fluorograms can easily be calculated on serialoptical sections, but again it is preferable to limit the volumeof interest in 3-D data sets. If needed, 3-D representationtechniques could be used to visualize third-order pixelfluorograms associated with triple fluorescence labelling.Pixel fluorogram analysis beyond the third-order presentsserious problems of representation and interpretation(Bonnet et al., 1995).

Detection of co-localization

To detect the local correlations of fluorescence distributions,we had to consider a collection of neighbouring pixels. Astandardization of the fluorescence values was required toobtain comparable data sets for each channel despite thedifferent acquisition conditions. Various advanced methodshave been proposed to compare images, either in patternrecognition to match a template and a portion of image

(Pratt, 1991) or for the spatial registration of multispectralor multitemporal satellite images (Svedlow et al., 1978).Among these, the correlation-based methods prove to bewell-suited to compare registered digital images, or todetermine the spatial shift separating two acquisitions.Manders et al. (1992, 1993) have applied correlationmethods to find out the degree of correspondence on entireconfocal micrographs. However, this approach does notprovide local information on the sites of colocalization. Toenlighten highly correlated structures in the images, wehave developed two image correlation methods adapted on alocal scale for the detection of fluorescence colocalization.The methods use standardized data sets to estimate thelinear dependence between the fluorescence distributions.One is based on the local correlation coefficient between rawimages (CCRI) and the second on the local second-orderjoint moment of standardized images (JMSI), also callednoncentral covariance or correlation function (Svedlow etal., 1978; Anderson, 1984).

To compute the correlation measurements locally, wehave chosen a Gaussian sliding window whose standarddeviation appears to be the main parameter of the methods.The standard deviation of the Gaussian window should belarge enough to estimate correctly the local statisticalparameters such as the mean and the standard deviation ofpixel values with regard to the optical resolution and thesampling frequency of the micrograph. Too broad a windowlowers the resolution of the correlation map, hampering thedetection of the small objects. The standard deviationdefining the Gaussian window should be chosen accordingto the size of objects to be colocalized, the optical resolutionand the sampling rate. We chose a 1.33-pixel standarddeviation on a 9 × 9-pixel window containing 99.9% of theGaussian distribution, which turned out to be large enoughto estimate local statistical parameters while conserving ahigh resolution in the correlation map. A compromiseshould be found between spatial resolution and validity ofstatistical measurements. Using standardized data sets, thecorrelation methods can reveal the correlation of veryweakly labelled areas. Indeed, we noted that the autofluor-escence of the specimen as well as a part of the nonspecificabsorption of antibodies give rise to highly correlatedstructures due to the normalization by very small standarddeviations. To prevent these nonsignificant outputs whenthe variance of very flat areas is low, we have added anadditional constant to the variance of each channel, whichmodifies the denominator of the correlation formulae. Instatistical differencing a constant is added to the denomin-ator (Pratt, 1991). This additional constant can be alsointerpreted as the variance of a noncorrelated noise addedto each channel. This noise constant can be adjusted todrown any type of weak correlated structures (autofluores-cence or correlated components of weak labelling). In our256-grey-level images, we have added a uniform noise



whose standard deviation was set to four quantization levelsfor both channels. The photon statistics and the darkcurrent give typical noncorrelated noises, but these weretoo small to mask nonsignificant structures in the lowfluorescence areas of the micrographs. As expected, thefluorescence cross-talk increases the correlation measure-ments and should be compensated for before computing thecorrelation maps. In addition to pixel fluorograms, imagecorrelation maps can be used to adjust acquisition settings,to eliminate fluorescence cross-talk and to optimize imageregistration locally. Among the two methods described inthe Results and the Appendix, the JMSI method betterdefines the tops of vesicular structures. Since this methoduses an image-orientated way of standardizing the data setscalled statistical differencing (Pratt, 1991), it providesintermediate images with normalized fluorescence distri-butions. Except for very low signals, for which localvariance is covered by the noise constant, both methodsdetect colocalization independently of the local mean of thesignal. Double positive pixels which do not correspond to alocal covariation of fluorescence distributions are notrevealed by these methods (White et al., 1996), as shownin Fig. 6. Local image correlation excludes the super-imposition of flat fluorescent profiles when a high spatialresolution is required for the analysis. At low spatialresolution, flat areas can be considered if they are smallerthan the local scanning window. We think that theidentification of colocalized structures is meaningful onlyfor a specified scale. Accordingly, unknown structures ofany size can be compared with known fluorescenceelements.

Perspectives in image cytometry

Image segmentation is required to carry out fluorescencemeasurements on recognized objects, and its qualitydepends on the selection of the attributes. The intensity,together with the variance and sometimes textural attri-butes, have been used to achieve improved segmentations(Santisteban & Brugal, 1995; Wu et al., 1995). Our twoconcurrent local correlation measurements prove to beappropriate attributes for multicolour image segmentationand are more discriminative than the double thresholding ofraw pixel values. The slope of the correlated cloud-of-dots(Fig. 4a) reveals the ratio between fluorescence distributionson a local window and characterizes the association ofmarkers. We propose to use the local correlation measure-ment as a basic attribute for further classifications andanalyses in multifluorescence image subcellular cytometry.The identification of segmented objects will allow theirclassification according to different parameters such as theirsize or fluorescence attributes, as performed at the cellularlevel in conventional fluorescence microscopy (Galbraith etal., 1991).

The extension of local correlation methods to serialconfocal sections requires the use of 3-D Gaussian windows.The fluorescence attenuation could then automatically becorrected by the standardization of the data. Independentmethods for the correction of this attenuation have beenproposed (Rigaut & Vassy, 1991; Visser et al., 1991;Strasters et al., 1994; Liljeborg et al., 1995). To applylocal image correlation methods to triple labelled specimensin a basic way, we propose to consider the three channelstwo-by-two, and then finally to superimpose the three mapsby associating each of them to a RGB colour plane.

Acknowledgments

This work was supported by the Centre National de laRecherche Scientifique (CNRS), the Institut National de laSante et de la Recherche Medicale (INSERM) and theAssociation pour la Recherche contre le Cancer (ARC). D.D.is the recipient of a predoctoral BDI fellowship from theCNRS and the Conseil Regional Provence-Alpes-Cotes-d’Azur. We wish to thank Dr Jacques Barbet (Centred’Immunologie de Marseille-Luminy, France) and especiallyDr David Shotton (University of Oxford, U.K.) for criticalreading of the manuscript. We also thank Dr Graca Raposo(Institut Curie, Paris, France) and Dr Patrizia Rovere (Centred’Immunologie de Marseille-Luminy, France) for the pre-paration of various specimens and especially Mr NicolasBarois (Centre d’Immunologie de Marseille-Luminy, France)for providing us with rabbit polyclonal anti-Ii chainantibodies.

References

Adams, S.R., Harootunian, A.T., Buechler, Y.J., Taylor, S.S. & Tsien,R.Y. (1991) Fluorescence ratio imaging of cyclic AMP in singlecells. Nature, 349, 694–697.

Akinyemi, O., Boyde, A., Browne, M.A., Hadravsky, M. & Petran, M.(1992) Chromatism and confocality in confocal microscopes.Scanning, 14, 136–143.

Akner, G., Mossberg, K., Wikstrom, A.C., Sundqvist, K.G. &Gustafsson, J.A. (1991) Evidence for colocalization of glucocor-ticoid receptor with cytoplasmic microtubules in human gingivalfibroblasts, using two different monoclonal anti-GR antibodies,confocal laser scanning microscopy and image analysis. J. SteroidBiochem. Mol. Biol., 39, 419–432.

Anderson, T.W. (1984) An Introduction to Multivariate StatisticalAnalysis. Wiley-Interscience, New York.

Arndt-Jovin, D.J., Robert-Nicoud, M. & Jovin, T.M. (1990) ProbingDNA structure and function with a multi-wavelength fluores-cence confocal laser microscope. J. Microsc. 157, 61–72.

Bonnet, N., Herbin, M. & Vautrot, P. (1995) Extension of thescatterplot approach to multiple images. Ultramicroscopy, 60,349–355.

Brakenhoff, G.J., Van der Voort, H.T.M., Van Spronsen, E.A. &Nanninga, N. (1989) Three-dimensional imaging in fluorescenceby confocal scanning microscopy. J. Microsc. 153, 151–159.



Brakenhoff, G.J., Visscher, K. & Gijsbers, E.J. (1994) Fluorescencebleach rate imaging. J. Microsc. 175, 154–161.

Brelje, T.C., Wessendorf, M.W. & Sorenson, R.L. (1993) Multicolorlaser scanning confocal immunofluorescence microscopy:practical application and limitations. Methods Cell Biol. 38,97–181.

Bright, G.R., Fisher, G.W., Rogowska, J. & Taylor, D.L. (1989)Fluorescence ratio imaging microscopy. Methods Cell Biol. 30,157–192.

Carlsson, K., Aslund, N., Mossberg, K. & Philip, J. (1994)Simultaneous confocal recording of multiple fluorescentlabels with improved channel separation. J. Microsc. 176,287–299.

Carlsson, K. & Mossberg, K. (1992) Reduction of cross-talk betweenfluorescent labels in scanning laser microscopy. J. Microsc. 167,23–37.

Dauwerse, J.G., Wiegant, J., Raap, A.K., Breuning, M.H. & VanOmmen, G.J.B. (1992) Multiple colors by fluorescence in situhybridization using ratio-labelled DNA probes create a molecularkaryotype. Hum. Mol. Genet. 1, 593–598.

Demandolx, D. & Davoust, J. (1995) Multicolor analysis in confocalimmunofluorescence microscopy. J. Trace Microprobe Tech. 13/3,217–225.

Dutartre, H., Davoust, J., Gorvel, J.P. & Chavrier, P. (1996)Cytokinesis arrest and redistribution of actin-cytoskeletonregulatory components in cells expressing the Rho GTPaseCDC42Hs. J. Cell Sci. 109, 367–377.

Fox, M.H., Arndt-Jovin, D.J., Jovin, T.M., Baumann, P.H. &Robert-Nicoud, M. (1991) Spatial and temporal distribution ofDNA replication sites localized by immunofluorescence andconfocal microscopy in mouse fibroblasts. J. Cell Sci. 99, 247–253.

Galbraith, W., Wagner, M.C.E., Chao, J., Abaza, M., Ernst, L.A.,Nederlof, M.A., Hartsock, R.J., Taylor, D.L. & Waggoner, A.S.(1991) Imaging cytometry by multiparameter fluorescence.Cytometry, 12, 579–596.

Humbert, C., Santisteban, M.S., Usson, Y. & Robert-Nicoud, M.(1992) Intranuclear co-location of newly replicated DNA andPCNA by simultaneous immunofluorescent labelling and con-focal microscopy in MCF-7 cells. J. Cell Sci. 103, 97–103.

Humbert, M., Raposo, G., Cosson, P., Reggio, H., Davoust, J. &Salamero, J. (1993) The invariant chain induces compact formsof class II molecules localized in late endosomal compartments.Eur. J. Immunol. 23, 3158–3166.

Keller, H.E. (1995) Objective lenses for confocal microscopy.Handbook of Biological Confocal Microscopy (ed. by J. B. Pawley),pp. 111–126. Plenum Press, New York.

Leger, I., Guillaud, M., Krief, B. & Brugal, G. (1994) Interactivecomputer-assisted analysis of chromosome 1 colocalization withnucleoli. Cytometry, 16, 313–323.

Liljeborg, A., Czader, M. & Porwit, A. (1995) A method tocompensate for light attenuation with depth in 3-D DNA imagecytometry using a confocal scanning laser microscope. J.Microsc. 177, 108–114.

Manders, E.M.M., Stap, J., Brakenhoff, G.J., Van Driel, R. & Aten,J.A. (1992) Dynamics of three-dimensional replication patternsduring the S-phase, analysed by double labelling of DNA andconfocal microscopy. J. Cell Sci. 103, 857–862.

Manders, E.M.M., Verbeek, F.J. & Aten, J.A. (1993) Measurement ofco-localization of objects in dual-colour confocal images. J.Microsc. 169, 375–382.

Messerli, J.M. & Perriard, J.C. (1995) Three-dimensional analysisand visualization of myofibrillogenesis in adult cardiomyo-cytes by confocal microscopy. Microsc. Res. Technique, 30, 521–530.

Messerli, J.M., Van der Voort, H.T.M., Rungger-Brandle, E. &Perriard, J.C. (1993) Three-dimensional visualization of multi-channel volume data: the amSFP algorithm. Cytometry, 14,725–735.

Mossberg, K., Arvidsson, U. & Ulfhake, B. (1990) Computerizedquantification of immunofluorescence-labeled axon terminalsand analysis of colocalization of neurochemicals in axonterminals with a confocal scanning laser microscope. J.Histochem. Cytochem. 38, 179–190.

Mossberg, K. & Ericsson, M. (1990) Detection of doubly stainedfluorescent specimens using confocal microscopy. J. Microsc.158, 215–224.

Nederlof, P.M., Van der Flier, S., Verwoerd, N.P., Vrolijk, J., Raap,A.K. & Tanke, H.J. (1992a) Quantification of fluorescence in situhybridization signals by image cytometry. Cytometry, 13, 846–852.

Nederlof, P.M., Van der Flier, S., Vrolijk, J., Tanke, H.J. & Raap, A.K.(1992b) Fluorescence ratio measurements of double-labeledprobes for multiple in situ hybridization by digital imagingmicroscopy. Cytometry, 13, 839–845.

Nederlof, P.M., Van der Flier, S., Wiegant, J., Raap, A.K., Tanke, H.J.,Ploem, J.S. & Van der Ploeg, M. (1990) Multiple fluorescence insitu hybridization. Cytometry, 11, 126–131.

Paddock, S.W. (1995) Imaging drosophila development with thelaser scanning confocal microscope. Microsc. Anal. 34, 9–11.

Paddock, S.W., Langeland, J.A., DeVries, P.J. & Carroll, S.B. (1993)Three-color immunofluorescence imaging of Drosophila embryosby laser scanning confocal microscopy. BioTechniques, 14, 42–48.

Pal, N.R. & Pal, S.K. (1993) A review on image segmentationtechniques. Pattern Recogn. 26, 1277–1294.

Pawley, J.B. (1995) Fundamental limits in confocal microscopy.Handbook of Biological Confocal Microscopy (ed. by J. B. Pawley),pp. 19–37. Plenum Press, New York.

Poenie, M., Alderton, J., Steinhardt, R. & Tsien, R. (1986) Calciumrises abruptly and briefly throughout the cell at the onset ofanaphase. Science, 233, 886–889.

Pollerberg, G.E., Schachner, M. & Davoust, J. (1986) Differentiationstate-dependent surface motilities of two forms of the neural celladhesion molecule. Nature, 324, 462–465.

Pratt, W.K. (1991) Digital Image Processing, 2nd edn. Wiley-Interscience, New York.

Rigaut, J.P. & Vassy, J. (1991) High-resolution three-dimensionalimages from confocal scanning laser microscopy. Quantitativestudy and mathematical correction of the effets from bleachingand fluorescence attenuation in depth. Anal. Quant. Cytol. Histol.13, 223–232.

Salamero, J., Humbert, M., Cosson, P. & Davoust, J. (1990) Mouse Blymphocyte specific endocytosis and recycling of MHC class IImolecules. EMBO J. 9, 3489–3496.

Sandison, D.R. & Webb, W.W. (1994) Background rejection and



signal-to-noise optimization in confocal and alternative fluores-cence microscopes. Appl. Opt. 33, 603–615.

Sandison, D.R., Williams, R.M., Sam Wells, K., Strickler, J. & Webb,W.W. (1995) Quantitative fluorescence confocal laser scanningmicroscopy (CLSM). Handbook of Biological Confocal Microscopy(ed. by J. B. Pawley), pp. 39–53. Plenum Press, New York.

Santisteban, M.S. & Brugal, G. (1995) Fluorescence image analysisof the MCF-7 cycle related changes in chromatin texture.Differences between AT- and GC-rich chromatin. Anal. Cell.Pathol. 9, 13–28.

Schubert, W. (1991) Triple immunofluorescence confocal laserscanning microscopy: spatial correlation of novel cellulardifferentiation markers in human muscle biopsies. Eur. J. CellBiol. 55, 272–285.

Sheppard, C.J.R. (1989) Axial resolution of confocal fluorescencemicroscopy. J. Microsc. 154, 237–242.

Sheppard, C.J.R. & Cogswell, C.J. (1990) Three-dimensional image-formation in confocal microscopy. J. Microsc. 159, 179–194.

Sheppard, C.J.R., Gu, M. & Roy, M. (1992) Signal-to-noise ratio inconfocal microscope systems. J. Microsc. 168, 209–218.

Shotton, D.M. (1989) Confocal scanning optical microscopy andits applications for biological specimens. J. Cell Sci. 94, 175–206.

Strasters, K.C., Van der Voort, H.T.M., Geusebroek, J.M. &Smeulders, A.W.M. (1994) Fast attenuation correction influorescence confocal imaging: a recursive approach. Bio-imaging, 2, 78–92.

Svedlow, M., McGillem, C.D. & Anuta, P.E. (1978) Imageregistration: Similarity measure and preprocessing methodcomparisons. IEEE Trans. Aerospace and Electronic Systems, 14,141–149.

Tsien, R.Y. & Waggoner, A. (1995) Fluorophores for confocalmicroscopy. Handbook of Biological Confocal Microscopy (ed. by J.B. Pawley), pp. 267–279. Plenum Press, New York.

Van der Voort, H.T.M., Brakenhoff, G.J. & Baarslag, M.W. (1989)Three-dimensional visualization methods for confocalmicroscopy. J. Microsc. 153, 123–132.

Van der Voort, H.T.M., Messerli, J.M. & Noordmans, H.J. (1993)Volume visualisation for interactive microscopical imageanalysis. Bioimaging, 1, 20–29.

Visscher, K., Brakenhoff, G.J. & Visser, T.D. (1994) Fluorescencesaturation in confocal microscopy. J. Microsc. 175, 162–165.

Visser, T.D., Groen, F.C.A. & Brakenhoff, G.J. (1991) Absorption andscattering correction in fluorescence confocal microscopy. J.Microsc. 163, 189–200.

White, J.G., Amos, W.B. & Fordham, M. (1987) An evaluation ofconfocal vs. conventional imaging of biological structures byfluorescence light microscopy. J. Cell Biol. 105, 41–48.

White, N.S. (1995) Visualisation systems for multidimensionalconfocal microscopy data. Handbook of Biological ConfocalMicroscopy (ed. by J. B. Pawley), pp. 211–254. Plenum Press,New York.

White, N.S., Errington, R.J., Fricker, M.D. & Wood, J.L. (1996)Aberration control in quantitative imaging of botanical speci-mens by multidimensional fluorescence microscopy. J. Microsc.181, 99–116.

Wijnaendts-van-Resandt, R.W., Marsman, H.J.B., Kaplan, R.,Davoust, J., Stelzer, E.H.K. & Stricker, R. (1985) Optical

fluorescence microscopy in three dimensions: microtomoscopy.J. Microsc. 138, 29–34.

Wilson, T. (1990) Confocal microscopy. Confocal Microscopy (ed. byT. Wilson), pp. 1–64. Academic Press, London.

Wu, K.H., Gauthier, D. & Levine, M.D. (1995) Live cell imagesegmentation. IEEE Trans. Biomed. Eng. 42, 1–12.

Young, I.T. (1988) Sampling density and quantitative microscopy.Anal. Quant. Cytol. Histol. 10, 269–275.

Young, I.T. (1996) Quantitative microscopy. IEEE Eng. Med. Biol.Mag. 15, 59–66.

Appendix: the local image comparison methods

The correlation coefficient of raw images

Considering a local area of interest in the pair offluorescence micrographs, the correlation coefficient esti-mates the linear dependencies between the two pixel valuedistributions independently of background fluorescencelevels. The correlation coefficient of raw images (CCRI),given in the correlation formula (3), is computed on slidingwindows across the whole image field, i.e. for each pixelneighbourhood, to produce a complete image correlationmap. All means on local windows are computed using aGaussian low-pass filter.

CCRI ¼xy ¹ xy

jxjy; ð3Þ

where x and y denote the digital fluorescence images asarrays of pixel values for both images. The image arrays j2

x

and j2y denote the modified local variances as indicated in

Eqs. (4) and (5). All arithmetical operations are calculatedpixel-to-pixel between pairs of images. The bar (¹) indicatesan image operation where each pixel value is replaced bythe averaged value of its neighbourhood. For example, theimage xy is the result of the image product between x and yconvoluted by a Gaussian convolution filter, whereas xy isthe pixel-to-pixel product between the filtered images x andy. In our example the experimental averages (¹) are allcalculated by smoothing the input images through a 9 × 9-pixel Gaussian convolution filter whose standard deviationis set to 1.33 pixel. The variances j2

x and j2y have been

modified by adding the constants ax and ay (Eqs. 4 and 5) toprevent the generation of nonsignificant correlation coef-ficient values due to very flat areas with small variances.The constants ax and ay can be also interpreted asvariances of noncorrelated noises added to the images. ax

and ay can be set to zero to obtain the exact variances andthe exact correlation coefficient varying from –1 to 1.

j2x ¼ x 2 ¹ x 2 þ ax ð4Þ

j2y ¼ y 2 ¹ y 2 þ ay: ð5Þ



The joint moment of standardized images

In an alternative approach, we applied a statisticaldifferencing (Pratt, 1991) to locally standardize the meanand the variance in the images. The standardized imagesare then compared by the calculation of their local second-order joint moment (Anderson, 1984). The statisticaldifferencing of images allows the generation of locallystandardized images successively by subtracting the locallyaveraged image x from the raw image x and dividing theresult by the standard deviation image jx defined above. Theoverall operator is defined by

xs ¼x ¹ xjx

ð6Þ

ys ¼y ¹ yjy

: ð7Þ

The notations are the same as for Eqs (3), (4) and (5). Allthe means (¯) are computed using the Gaussian convolutionfilter defined above.

Having defined two standardized image data sets xs andys, we can compute their joint moment between slidingwindows across the whole image field. This second-orderjoint moment between standardized images (JMSI) is theaveraged value of the product between images xs and ys:

JMSI ¼ xsys: ð8Þ

The JMSI values are not necessarily between –1 and 1 butare usually in this range.



multicolour analysis and local image correlation in confocal

Documents