SPECTROSCOPIC BB! TECHNIQUES

Simple and Efficient Method to Eliminate Spike Noise from Spectra Recorded on Charge-Coupled Device Detectors

H I D E O T A K E U C H I , * S H I N J I H A S H I M O T O , and ISSEI H A R A D A Pharmaceutical Inst i tute, Tohoku University, Aobayama, Sendai 980, Japan

Index Headings: Charge-coupled device; CCD; Data processing; Raman spectroscopy; Atomic emission spectroscopy.

INTRODUCTION

Charge-coupled device (CCD) detectors have several characteristics which are suitable for spectroscopic mea- surements at very low light levels. TM The quantum effi- ciency of a CCD is high over a wide wavelength range, and the dark count rate is greatly reduced by cooling the detector. Further reduction of the dark count can be made by setting the pixel binning frame to the detector area that is actually illuminated) If the detector is suf- ficiently cooled and the signal readout is performed after appropriate binning, the only significant noise source, aside from inevitable photon shot noise, is the noise pro- duced in the signal readout process. Since the readout noise is independent of the duration of exposure, longer exposure provides a better signal-to-noise (S/N) ratio, in principle. However, the practical limit to exposure time is set by spike noise, which is generated by cosmic rays as well as possible "y- and a-rays emitted from ma- terials around the detector chip2 Each quantum of such high-energy rays striking the CCD chip usually generates thousands of electrons in a pixel, while a single photon absorbed by the detector produces less than one electron on the average. Thus the spike noise sometimes obscures the very low light level spectrum. Distinction of spike noise from the true photon signals and reliable elimi- nation of the spikes are required in order to further elon- gate the exposure time and to achieve a better S/N ratio.

Received 15 June 1992. * Author to whom correspondence should be sent.

Occasionally, data processing with median filters is em- ployed to remove the spikes from spectra recorded on CCD detectors. 6,7 However, this filtering method is a kind of smoothing and has a possibility of distorting the spec- trum, particularly when spectral linewidths are compa- rable to or less than those of spikes. We have developed a simple and efficient method to eliminate the spike noise without losing the original spectral information. Here, we describe the method and demonstrate its effectiveness in Raman and atomic emission spectroscopy.

EXPERIMENTAL

Raman spectra were recorded on a single polychro- mator (JASCO, focal length 575 mm, f/6.3, 4320 grooves/ mm holographic grating for UV) equipped with a short- wavelength cut prism filter. The CCD detection system was a Princeton Instruments Model LN/CCD-1152 con- trolled by an ST-135 controller. The CCD array, a UV- coated EEV Model 88130 with 298 rows × 1152 columns of 22.5-ttm-square pixels, was cooled to - ll0°C. The cen- tral 180 row pixels in each column were binned, and charge packets were digitized to 16 bits. The CCD con- troller was linked with a host computer (NEC Model PC- 9801 DX, i80286 cpu, 12 MHz) through an IEEE-488 interface. Operational software RASP was coded in QuickBASIC 4.5 (Microsoft Inc.) and partly in machine language. Excitation of Raman scattering was effected by the fourth harmonics (266 nm) of an Nd:YAG laser (Quanta Ray, DCR-3G) operating at 30-Hz repetition rate.

Light from a low-pressure Hg lamp was faintly reflect- ed with a white paper and introduced to the spectrom- eter. Recording of atomic emission spectra was made in a way similar to that employed for Raman spectral mea- surements.

METHOD OF SPIKE ELIMINATION

In our CCD detection system, spike noise appears at a mean rate of about 8 × 10 -s s -1 per pixel. Since 180 row pixels are binned, each column may have spikes at a rate of 1.4 × 10 -~ s-L This means that a spectrum integrated for a 30-min exposure period will have about 30 spikes randomly distributed over 1152 data points. The positions of the spikes will change from measure- ment to measurement, while the true photon signals should be reproduced in every measurement. The pos-

Volume 47, Number 1, 1993 0003-7028/93/4701-012952.00/0 APPLIED SPECTROSCOPY 129 © 1993 Society for Applied Spectroscopy

sibility that a spike will appear at the same column in two successive recordings is about 7 × 10 -4 for a 30-min exposure. Therefore, if we record two spectra under iden- tical conditions, we can expect that the true photon signal at each data point is retained in at least one of the spec- tra. If one spectrum has a spike at a data point, the true photon signal at that point can be estimated from the intensity at the corresponding data point in the other spectrum. This is the basic idea behind our method of spike elimination.

Distinction of spike noise from noises of other sources (photon shot noise and readout noise) is rather simple. In the absence of spike, the root-mean-square overall noise, Ni, at the ith data point may be given by

Ni = (Si + Nr2) 1/2 (1)

where Si is the photon signal at that data point and Nr is the root-mean-square readout noise. The photon signal term includes the dark count when the exposure time is so long that the dark charge buildup cannot be neglected. Since the photon shot noise and readout noise are ran- dom and the peak-to-peak amplitude of the noise is ex- pected to show a Gaussian distribution with a standard deviation of 21/2Ni, more than 95 % of the noises will have amplitudes less than 5.6N~. If the intensity difference between two spectra is greater than 5.6N, it is highly probable that a spike overlaps at that data point in either of the two spectra. In such a case, the weaker intensity is a good estimate of the true photon signal because spikes are always positive peaks. The procedure of our spike elimination, which we call "robust summation," is described below.

1. Measure two spectra (1 and 2) under identical con- ditions.

2. Calculate the total intensity of each spectrum (T1 or T2) by summing up the intensities over the whole data points.

3. Take the weaker of the intensities, I1~ and I~, at the ith data point as a first guess of S~, where I~ denotes the scaled intensity of spectrum 2, I2i" TIT2 .

4. Calculate Ni according to Eq. 1 with S~ and predeter- mined Nr (1.4 for our detector).

5. If the absolute value of the difference I~ - I~ is greater than 5.6N~, replace the stronger intensity by the weak- er one. When I~i is replaced by Ili, multiply Ill by T2/ T1 to get I2i.

6. Take I1~ + I2~ as the intensity of the "true" sum spec- trum.

7. Repeat steps 3-6 for every data point.

The intensity scaling at step 3 is useful to compensate for the intensity fluctuation or drift during two successive measurements. In the final spectrum calculated, the S/N ratio is improved by a factor of 1.4 except at the data points where spikes were present. If spikes appear ac- cidentally at the same data point of both spectra, it is possible to eliminate the spikes by recording another spectrum and then adding the spectrum to the previous sum spectrum in the same manner as described above. The algorithm of robust summation is simple, and it is easily implemented into a data processing program. The spike elimination routine coded in QuickBASIC took less than 1 s for processing 1152 data points.

A

J

B

16'00 ' 14'00 ' 12.00 ' 10'00 ' 800 ' 6(}0 ' Raman Shift / cm -1

FiG. 1. The 266-nm excited Raman spectra of dioxane/ethyl acetate (1:2, v/v). (A and B) Raw data integrated for a 30-min exposure period with excitation of about 0.1 mW laser power; (C) calculated from the spectra in A and B by the robust summation method; (D) obtained by applying a 5-point median filter to the spectrum in A. Arrows in panel D indicate remaining spikes.

RESULTS AND DISCUSSION

Figure 1 shows 266-nm Raman spectra of dioxane/ ethyl acetate (1:2, v/v), which we usually use as a fre- quency standard sample for 266-nm Raman scattering. The upper two spectra (Fig. 1A and 1B) represent raw data integrated for 30 min with a very low laser power (~0.1 mW). These spectra contain about 30 spikes with varied peak heights, and some of the spikes overlap Ra- man bands. Figure 1C shows the spectrum calculated from the two sets of raw data by applying the robust summation and then dividing by 2 to get the average. The spikes are eliminated in the averaged spectrum, and this spectrum is identical to a spike-free raw spectrum recorded with shorter exposure time (3 min) and higher laser power (~ 1 mW).

To compare the effect of robust summation with that of median filtering, we have applied a 5-point median filter to the spectrum shown in Fig. 1A. The filter replaces the intensity at each data point by the median of inten- sity values at that and four neighboring data points. Although strong spikes are absent in the filtered spec- trum (Fig. 1D) and the curve is smoothed significantly, there remain a few weak spikes, as indicated by arrows in the figure. These remaining spikes originate from wide

130 Volume 47, Number 1, 1993

T + .

B

C

x2

1000 800 600 400 200 0 Channel No.

FiG. 2. Emission spectra of a low-pressure Hg lamp. The 253.7-nm line is located around the center of the abscissa. (A and B) Raw data recorded with a 10-rain exposure; (C) calculated from the spectra in A and B by the robust summation method; (D) effects of a 5-point median filter on the spectrum in A. The intensity is amplified by a factor of two in C and D. Note that the peak height of the 253.7-nm line is greatly reduced in D as a result of smoothing by the median filter.

spikes extending more than five da ta points. By increas- ing the width of filter (for example, to 11) we could elim- inate all the spikes, bu t the peaks of sharp Raman bands would also be cut out and the spectral resolution would deter iorate considerably as a result of excessive smooth- ing.

The advantage of our spike elimination method over the median filter me thod is more clearly demons t ra ted in Fig. 2, which shows emission spectra of a low-pressure Hg lamp. The Hg 253.7-nm line is located near the center of the channel number axis. In the raw spectra shown in Fig. 2A and 2B, it may be hard to confidently distinguish the 253.7-nm line from spikes. This is because most of

the spikes are stronger than the t rue spectral line, and the widths of spikes are comparable to tha t of the atomic emission line. The robust summat ion me thod is partic- ularly powerful in such a case. By this method, the t rue spectral line is conserved and all the spikes are elimi- nated, as shown in Fig. 2C. On the other hand, the median filter me thod is unsuccessful in eliminating the spikes. Figure 2D shows the result of 5-point filtering of the spect rum shown in Fig. 2A. The peak intensi ty of the 253.7-nm line is greatly reduced, and much higher spikes remain in the fil tered spectrum. Of course, a wider filter can remove the spikes, bu t the spectral line will also be erased altogether.

In the present robust summation, the peak height threshold for detect ing spikes is set to 5.6Ni. About 95% of the noises other than spikes are expected to have peak- to-peak ampli tudes under this threshold level. Alterna- t ion of the peak height threshold to a higher (7.3 N i, 99 % ) or lower (4.7 Ni, 90 % ) level does not significantly affect the general performance of robust summation. However, weak spikes t end to escape the detect ion with the 99 % threshold setting. When the 90% threshold is employed, on the other hand, some weak positive peaks due to readout noise and/or photon shot noise are re- garded as spikes and are excluded from the data, re- sulting in a failure to improve the S/N ratio by sum- mation. The 95 % threshold level seems optimal.

Of the parameters used in the robust summation, the readout noise Nr and the coefficient of the threshold level can be p rede te rmined without referring to the da ta to be processed. The remaining parameter N~ is calculated from the spectral da ta themselves. Thus the robust sum- mat ion can be made automatical ly without any param- eters to be input at the t ime of operation. Implemen- ta t ion of this rout ine into a repeated accumulat ion procedure makes it possible to achieve both the spike el imination and the S /N improvement while accumulat- ing the spectra on a spectrometer . The robust summat ion me thod may also be useful in acquiring spike-free two- dimensional images on CCD detectors.

1. C. A. Murray and S. B. Dierker, J. Opt. Soc. Am. 3, 2151 (1986). 2. R. B. Bilhorn, J. V. Sweedler, P. M. Epperson, and M. B. Denton,

Appl. Spectrosc. 41, 1114 (1987). 3. R. B. Bilhorn, J. V. Sweedler, P. M. Epperson, and M. B. Denton,

Appl. Spectrosc. 41, 1125 (1987). 4. J. V. Sweedler, R. D. Jalkian, and M. B. Denton, Appl. Spectrosc.

43, 953 (1989). 5. P.M. Epperson and M. Bonner Denton, Anal. Chem. 61, 1513 (1989). 6. P. J. Treado and M. D. Morris, Appl. Spectrosc. 44, 1 (1990). 7. J. J. Baraga, M. S. Feld, and R. P. Rava, Appl. Spectrosc. 46, 187

(1992).

APPLIED SPECTROSCOPY 131