1

Multi-spectral Image Acquisition and Spectral Reconstruction using a Trichromatic Digital

Camera System associated with absorption filters

Francisco H. ImaiMunsell Color Science Laboratory, Rochester Institute of Technology

Abstract

This report summarizes researches performed to evaluate the feasibility of using the IBM PRO\3000Digital Camera System associated with Kodak Wratten filters to reconstruct the spectral reflectances ofartwork images and analyzes the performance and discuss possible enhancements.

Introduction

The traditional techniques of image capture used to archive artwork in most of the museums of the worldrely on conventional photographic processes. Photography has the advantages of high-resolution andoptimal luminance (tone) reproduction and the disadvantage of poor color accuracy. The exception is theVASARI imaging system developed at the National Gallery, UK which employs a seven-channel multi-spectral 12 bit digital camera attached to a scanning device that traverses across the painting.1 Afterappropriate signal and spatial processing, 20K x 20K 10-bit L*, 11-bit a* and b* encoded images result.The National Gallery has been very successful in developing colorimetric image archives and using them toprovide the European community with accurate color reproductions in both soft-copy and hard-copy formsunder a defined set of illuminating and viewing conditions (i.e., colorimetric color reproduction).

We have an interest in drawing upon the European experiences and making two enhancements. Thefirst is alleviating the need to scan across the painting. This will greatly reduce the cost and complexity ofthe image acquisition system. The second is to define images spectrally and use the spectral information toprovide printed color reproductions that are close spectral matches to the original objects producing high-quality color matching under different illuminations and observers. The advantages of spectral systemshave been summarized by Berns2 and Hardeberg et al.3 Technical issues concerned with multi-spectralimage acquisition have been exhaustively studied.4-9 In particular, König and Praefcke10 analyzed practicalproblems of designing and operating a multi-spectral scanner using a set of narrow-band interference filtersand a monochrome CCD camera, the most common configuration for multi-spectral image capture. Whenusing interference filters for image acquisition, a major problem is caused by the transmittancecharacteristic of the filters that depends on the angle of incidence. For example, in order to image a paintingwith horizontal dimensions of 1 meter with a distance of 2 meters between the painting and the filter, thereis angle of incidence ~14° for points in the extremities. Simulations10 have shown that this causes color

differences of 2 ∆E*ab units in relation to the image obtained at 0° angle of incidence. Another problem isthat the surfaces of the interference filters are not exactly coplanar resulting in spatial shift and distortion ofthe captured image. We also need to consider that there are inter-reflections caused by reflections betweenthe spectral filters and the original image, and between the interference filters and the camera lens. Thesetechnical problems make it unrealistic and impractical for image acquisition using interference filters inmuseums without a considerable degree of expertise in multi-spectral imaging.

We believe that a conventional trichromatic digital camera combined with absorption filters can providean alternative way to capture multi-spectral images. The spectral reflectance of each pixel of the image canbe calculated by the camera signals using some non-linear iterative method. It makes the image acquisitioneasier and with relatively low cost since the performance-cost relation of commercial digital cameras hasincreased rapidly. In this research we also want to simplify the image acquisition system developed inEurope in order to avoid scanning across the painting.

2

A possible solution to the desired enhancements mentioned above can be achieved by the fusion of ahigh-spatial resolution lightness image with a low-spatial resolution multi-spectral image. It is possible tointerpolate each pixel of each multi-spectral image to high-spatial resolution while maintaining its colorinformation and changing the original lightness for the lightness data of the corresponding high-spatialresolution image subpixels. This can be accomplished without noticing the expected decrease of tonalresolution in the hybrid image, because the modulation of the light in the eye becomes progressivelysmaller as the spatial frequency increases,11 due to optical limitations and features of the retinal mosaic. Asa consequence, the chromatic channels have much lower spatial resolution than the luminance channel. Thisvisual feature of the human eye has been applied in broadcast television, and to devise very effectivecompression algorithms such as JPEG.12 In fact, pyramidal data structures that exploit the eye’s contrastsensitivity can be used for efficient data storage in the proposed system. Image fusion of a multi-spectralimage with a high-resolution image has been intensively researched in the field of remote sensing.13-15 Thelightness and color information can be codified respectively as L* and a*, b* in order to allow the systemto be easily optimized to have the least color difference in CIELAB ∆E*ab or ∆E*94 . Therefore, it isimportant to assure that the high-spatial resolution image capture system produces very accurate L* and themulti-spectral image capturing system provides high-accurate chromatic information. Conventionalphotography followed by high-quality scanning results in very high spatial resolution, large dynamic rangeand low noise, and the photometric response of the film-scanning system can be easily determined usinggray-scale targets alongside the imaged painting. A commercial digital camera with a set of filters can beused for the low-resolution chromatic data acquisition. This hybrid approach eliminates the need forscanning across artwork.

We can divide the hybrid multi-spectral generation into four parts: image acquisition, spectral analysis,image fusion, and spectral reconstruction.

In the image acquisition system shown in Figure 1, the lightness information, L*(x,y) is calculated foreach (x,y) pixel of the high-resolution image from a scanned photograph after proper photometric andspatial calibration, where (x,y) denotes the coordinates of the high-resolution image pixels. After properphotometric and spatial calibration, the digital counts of the multi-spectral camera ti, i = 1 to m, where m isthe number of filters, are used to estimate the spectral reflectance, R(x’,y’,λ), and the colorimetric values ofthe image, L*a*b*(x’,y’), where (x’,y’) denote the coordinates of the low-resolution image pixels.

One can model multi-spectral image acquisition using matrix-vector notation.9 Expressing the sampledillumination spectral power distribution as

S =

s1 0

s2

O0 sn

, (1)

and the object spectral reflectance as r=[r1, r2, ... rn]T, where the index indicates the set of n wavelengths over

the visible range and T the transpose matrix, representing the transmittance characteristics of the m filters ascolumns of F

F =f1,1 f1,2 L f1,m

M M L Mfn ,1 fn, 2 L fn ,m

(2)

and the spectral sensitivity of the detector as

D =

d1 0

d2

O0 dn

, (3)

then the captured image is given by t=(DF)TSr and the color vector can be represented as c=At=(X, Y, Z)T

where X, Y, Z are the CIE tristimulus values. The CIELAB L*, a*, b* are given by the non-lineartransformation ξ, where ξ( X, Y, Z ) = L*, a*, b*.

As shown in the figure 1, the spectral reflectance R(x’,y’) of the low-resolution image can be estimatedusing interpolation techniques such as cubic spline,8 modified-discrete-sine-transformation (MDST),7 or

3

spectral reconstruction methods based on statistical analyses such as principal-component analysis(PCA).16-18 The PCA method uses a set of a priori measured reflectance-based basis functions, E(λ,k),where k denotes the basis vector. Burns and Berns compared interpolation methods with PCA and foundthat PCA is more accurate than interpolation methods.19 However, König and Praefcke7 found that certaininterpolation techniques such as the smooth inverse may produce acceptable results and should beconsidered. The accuracy of spectral reconstruction depends on the number of basis functions.20 Thenumber of basis functions necessary for accurate spectral reconstruction also depends on the database usedfor PCA. However, 5 to 8 basis vectors seem to be sufficient for an accurate spectral reconstruction ofartwork. It is possible to optimize the filters7 but it is not considered in this stage of the research. TheL*a*b*(x’,y’) for the low-resolution image is calculated from the estimated spectral reflectance R(x’,y’)and the performance can be compared with direct linear transformation from the digital counts of theacquired multi-spectral image. 19

As a pilot experiment the IBM PRO/3000 Digital Camera System (IBM DCS), a trichromatic digitalcamera with good colorimetric performance, sufficient resolution (4,920 by 3,072 pixels), 12 bitsquantization and geometric stability for imaging is used. We need to test if this trichromatic digital cameracombined with absorption filters can provide an alternative way to capture multi-channel images.

This report will focus on the description, characterization and performance of the IBM digital camerasystem using Kodak Wratten filters.

Figure 1. Image acquisition and spectral analysis flowchart

4

Description of the IBM DCS

The IBM DCS Pro/3000 scanner21,22 is composed by copy stand, side lights, column, scanner head and acontroller as shown in Figures 2, 3 and 4 that connects to a PC and a high resolution color monitor shownin Figure 5. It also has a light for scanning transparencies on the copy stand.

Figure 2. Pro/3000 copy stand and controller Figure 3. Pro/3000 side lamp

Figure 4. Pro/3000 scanner head and column. Figure 5. IMB DCS PC and color monitorThe IBM Pro/3000 DCS supports 3072 by 4096 or 2048 by 3072 resolution pixels by a camera

mounted on a variable height copy stand with both reflective and transmissive illumination. This digitalcamera system has a sensor technology that samples the same pixel many times and performs a chipsummation and averaging allowing much more sensitivity than conventional CCD sensors. The CCD imagesensor is mounted on a moving slide located inside the scanner head. A color filter wheel with 5 positions:dark, clear, red, green, and blue, is used to filter the incoming light. There is a software that runs in the OS2operational system called PISA95 that is used to set parameters, calibrate and image using IBM Pro/3000DCS and produces 8 bit and 12 bit gray image as well as 8 bit color TIFF image calibrated using aMacbeth Color Chart under D50 illuminant and displayed on the IBM high-resolution monitor shown inFigure 5 with D65 white point. It is possible to set the scanner’s integration rates (analogous to exposuretime) for red, green and blue channels and they should be kept constant for all the calibration and imagingprocesses. It is also possible to extract 12 bit raw color images.

5

Experiments

Some experiments were conducted to characterize the tone reproduction and camera spectral components todetermine the feasibility of using this digital camera system to reconstruct the spectral reflectance ofartwork paintings.

1. The effect of quantization (bit depths) on tone reproduction.In a normal operation IBM DCS yields automatically 8 bit quantization R, G, B images with dark current

and spatial white correction. It is possible to extract 12 bit R,G,B with dark correction but still without whitespatial correction. A 12 bit R,G,B image of a white chromalin sheet was scanned by the IBM DCS in orderto perform a pixel by pixel white spatial correction for the 12 bit color image.

In this experiment, a Macbeth Color Checker was imaged and the digital counts of gray patches wereplotted against intensity (Y) shown in Figure 10a and 10b for red channel respectively for 8bit and 12 bitquantizations.

Figure 10a. Tone reproduction curve for 8 bit Figure 10b. Tone reproduction curve for

quantization 12 bit quantization

2.Characterization of the spectral components of IBM DCSThe knowledge of the spectral components of the camera systems gives us more flexibility in terms ofyielding simulations with different illuminants and allows us to use the camera as a “spectrophotometer”that gives us the spectral reflectance of each pixel of the image from the digital counts of the image.Then, the spectral characteristics of the systems should be measured (spectral power distribution of thesources and spectral sensitivity of the camera).

A)Measurement of the side lamps spectral radiant powerThe IBM DCS uses four halogen light bulbs (Sylvania J688 ENH, 350W 120V, color temperature of3,200 K) as side lamps.

At first, the spectral reflectance factor of a sheet of white chromalin was measured by BYK-GardnerTCS-35 spectrophotometer and the obtained curve is plotted in Figure 6.

6

Figure 6 Spectral Reflectance of the sheet of white chromalin

The PhotoResearch PR-704 SpectraScan Spectroradiometer was set vertically below the IBM DCSscanner head, 29.5 cm from the stand base, hold by a bridge of metal sticks mounted on tripods (strongenough to support its 14.5 kg) as shown in Figures 7a and 7b. The measured spectral radiant power of theside lamps reflected on the center of white chromalin was performed after 20 minutes warm up and plottedin the Figure 8. The camera system uses 45°/ 0° geometry to digitize images and this geometry was keptduring the measurements.

Figure 7a Measurement of spectral radiance of the side lamps Figure 7b Positioning of SpectraScan

For the measurement of the spectral radiance of the side lamps.

The resulting spectral radiant power of the illuminant was calculated considering a priori measuredspectral reflectance of the white chromalin surface. The absolute spectral radiant power of the illuminant isshown in Figure 9.

7

Figure 8. Measured spectra radiant power of the side lamps reflected on white chromalin

Figure 9. Absolute spectral radiant power of IBM DCS side lamps

B)Measurement of the transmissive light spectral radiant power

There is a square hole at the top of the copy stand of IBM DCS where an integration chamber for thetransmissive lamp source (Sylvania J688 ENH (350W 120V)) is inserted. A diffuser that provides a quiteuniform illumination as shown in Figure 10 covers the integration chamber. A black metal with arectangular hole of 4 by 6 inches was used to mask the illumination.

The PhotoResearch PR-704 SpectraScan Spectroradiometer was set vertically with a distance of 20 cmbetween its lens and the copy stand top and the lens of the SpectraScan. The spectroradiometer was holdby a bridge mounted on tripods like in the previous experiment. Figure 11a, and 11b show the measuredabsolute and normalized spectral radiant power of the transmissive light respectively.

8

Figure 10. Measurement of spectral radiance of the transmissive lamp source

Figure 11a. Absolute spectral radiance of Figure 11b. Normalized spectral radiance of

the transmissive lamp source the transmissive lamp source

C) The measurement of IBM DCS PRO/3000 Spectral Sensitivities.To estimate the spectral sensitivities of the IBM DCS two methods were employed: Monochromatormethod and Interference filters method.

Monochromator methodMeasurement of the spectral sensitivity of the IBM DCS was accomplished using a light source ModuleModel 740-20 (serial 8553) in conjunction with a double monochromator, part of the Optical RadiationMeasurement System Model 740 A (serial 185268-5) from Optronics Laboratories Inc.

Hewlett Packard Hamsom 6274A DC Power Supply (0-60V 0-15A) was set to provide 0.06A current tothe light source. This light source with monochromator provided narrowband illumination at the 10 nm exitslit, over a range 380-780 nm at 10 nm intervals. The monochromator, the light source, and the powersupply are shown in Figure 12.

The IEC TC-100: Audio, Video and multimedia systems and equipment had a project team 91966:

9

Colour measurement and management in multimedia systems and equipment and their Part 9 was dedicatedto digital cameras. The IEC recommends the use of optical fiber to connect the monochromator to a blackbox with a diffuser in front of the digital camera to measure its spectral responsivity. Otherwise, Burns alsomeasured a digital camera spectral sensitivity keeping a working distance of 70 cm from the exit slit of themonochromator to the surface of a digital camera lens.9 In the case of the IBM DCS, it is not easy tomodify the orientation of the scanner head to be positioned in front of the monochromator such as in theexperiment conducted by Burns, and a optical fiber was used to connect the monochromator output to theIBM DCS scanner head. The extremity of the optics fiber was kept attached to a holder hanging very closeto the camera lens (shown in Figure 13) and the diffuser recommended by IEC TC-100 was not usedbecause it would attenuate the weak signal from the monochromator.

Figure 12. Monochromator (white Figure 13. Optical fiber Figure 14. Image (8 bit) of thebox), light source (black box) extremity hold below the monochromator adjusted at 600 nm.over the power supply. scanner head.

The integration rate was set (R=20, G=18, and B=150) for “miscellaneous” 3072 by 2460 pixelsresolution, using Rodesnstock 105 mm lens, fstop 5.6, scanner head positioned 858 mm from the copystand and focus adjusted at 297.5 mm. Using these setting, a pilot experiment was performed digitizing 9images over the range of 400-720 nm in intervals of 40 nm. Figure 13 shows a scanner head with openview finder showing the light from the optical fiber for monochromator adjusted at 440 nm and Figure 14shows how an 8 bit image for monochromator adjusted at 600 nm looks like on the high resolution monitorof IBM DCS. Although this image is not uniform it is possible to crop the center portion of the image toget a quite uniform image. It is important to stress that 12 bit image is used in this experiment instead of the8 bit image because the 8 bit image suffer automatic white spatial correction and other colorimetriccorrections that are not desirable to determine the camera spectral sensitivity.

Twelve bit images of the monochromator light were taken in a dark environment over the range of 380-780 nm, in intervals of 10 nm. Then, each image was cropped in the same position centered in the light spotproducing a 200 by 200 pixels image. The averaged digital counts for each channel over the wavelengthrange are shown in Figure 15.

10

Figure 15. Averaged digital counts of a 200 by 200 image generated by a monochromator.

Next, the camera was replaced by a calibrated detector (Optronics Laboratories, Inc. OL730-5C SiliconPhoto detector SIN:1152 Hex Key with calibration certification date: May 30 1998) shown in Figure 16whose responsivity is given in Figure 17 and the source spectral irradiance was measured over the samerange used above using Optronics Laboratories Inc. radiometer 730 A (Serial # 850190).

Figure 16. Optronics Laboratories Inc. calibrated Figure 17. Responsivity of the calibratedSilicon Photodetector SIN:1152 photodetector SIN:1152

The experimental arrangement to measure the source irradiance is shown in Figures 18 a,b. Figure 18aalso shows the IBM high-resolution monitor displaying a cropped image for wavelength 520 nm. Figure18b shows the optical fiber from the monochromator attached to the calibrated detector that sends signals tothe radiometer.

11

Figure 18a. Experimental arrangement to Figure 18b. Optronics Laboratories Inc. measure the source irradiance (general view) Radiometer 730 A

From the averaged digital counts and the measured irradiance it is possible to calculate the spectralsensitivity of R, G, and B channels. The absolute and normalized spectral sensitivity is plotted in figures19a and 19b respectively.

Figure 19a. Absolute spectral sensitivity of the IBM DCS camera.

12

Figure 19b. Normalized spectral sensitivity of the IBM DCS camera.

Interference Filter MethodIt is possible to use the light filtered by interference filters put on the transmissive lamp at the copy standlevel of the IBM DCS to produce monochromatic light that is imaged by the camera. The averaged digitalcounts of the image of each filtered light source is divided by the measured radiance for each filter givingthe spectral sensitivity of each channel of the camera. The relative height for the spectral sensitivity of eachchannel can be adjusted comparing with the results obtained using monochromator method.

At first, the surface of the transmission lamp diffuser was covered by a 35 mm film mask and the filterswere put on the mask as shown in Figures 20a, b.

Figure20a. Measurement set. Figure 20b. Detail of the interference filter

Sixteen interference filters manufactured by Ealing centered over the range 400 to 700 nm in intervals of20nm were imaged in 12 bits using the same camera settings used in the monochromator method. The usedfilters are listed in the Table 1.

13

Table1. Ealing Interference Filters used to determine the spectral sensitivities of the IBM DCS.Filter Number Passband range (nm) Peak wavelength (nm)35-3219 385-415 401.635-3292 405-435 420.935-3359 425-455 439.935-3417 445-475 459.935-3458 465-495 481.435-3516 485-515 500.035-3599 505-535 520.535-3656 525-555 540.235-3714 545-575 561.735-3771 565-595 580.935-3839 585-615 601.235-3870 605-635 619.635-3938 625-655 640.835-4019 640-675 658.635-4076 665-695 679.636-4175 685-715 699.0

Figure 21 shows how looks like a digitized image (in 8 bits) for filter 35-3839. The spectral radiance ofthe transmissive source passing each filter was measured using the same setting for the Photo ResearchPR-704/714 SpectroScan Spectroradiometer used to measure the transmission lamp spectral radiance. Adistance of 15 cm was set between the spectroradiometer lens and the filter surface. It is illustrated in thefigures 22a and 22b. It is interesting to notice that the actual color of filter when it’s seem from the top isred like shown in Figure 22a and not green as shown in Figure 22b. This change of color is due to theinfluence of the angle of view on interference filter spectral properties as mentioned in the introduction.

Figure 21. Digitized Figure 22a. Spectral radiation measurement Figure 22b. Spectral radiationimage for filter 35-3839 of the light leaving interference filters measurement

The averaged digital counts for each channel for the images taken using interference filters can be seen infigure 23.

14

Figure 23. Averaged digital counts (12 bits) for the images taken using interference filters.

The measured spectral radiance of each filter is given in figure 24. The transmittance of each filter can beeither measured directly or can be calculated from the measured spectral radiance of the transmissionsource and the spectral radiance of the light leaving each filter. The calculated transmittance is given infigure 25.

Figure 24. Measured spectral radiance for each filter.

15

Figure 25. Calculated transmittance of interference filters

The absolute and relative spectral sensitivities of the IBM DCS given by this interference filters method isshown respectively in figure 26a and 26b.

Figure 26a. Absolute spectral sensitivities of the IBM DCS (by interference filters method).

16

Figure 26b. Relative spectral sensitivities of the IBM DCS (by interference filters method).

Comparison of the spectral sensitivities obtained by monochromator method and interference filtersmethodThe spectral sensitivities curves obtained by both monochromator method and interference filters methodare plotted together in Figure 27.

Figure 27. Comparison between monochromator and interference filters method to determine the spectralsensitivity of IBM DCS

It is possible to notice from the curves of figure 27 the curves match except for a shift in the lowerfrequencies of blue channel. In order to check the accuracy of the wavelength in the monochromator-

17

optical fiber system, a mercury vapor lamp (manufactured by Oriel Corp.) was used to illuminate the inputtube of the monochromator and its output was connected to a optical fiber that was linked to aphotodetector (Optronics Laboratories Model 740A-D Number 387) as shown in figure 28a. The mercuryvapor lamp was turned on using a Special Lamp Power Supply Model 65150 (serial number 244)manufactured by Oriel Corp. The same radiometer previously used to measure the radiant power of thelight from the optical fiber was used to determine the wavelength of the monochromator which gives peaksof radiant power as shown in Figure 28b. The experimental peak wavelength was compared to thewavelength that gives the peaks of radiant power for a typical mercury vapor lamp.23,24 This comparison issummarized in Table 2.

Figure 28a. Monochromator with a mercury Figure 28b. Measuring the radiant power of Vapor lamp source. the light from the fiber optics

Table 2. Comparison of measured and expected wavelengths of radiant power peaks for mercury vaporlamp

Expected Wavelength (nm) Measured Wavelength (nm)404.7 404.4435.8 435.4546.1 546.1579.1 579.4

From Table 2 it is possible to see that the monochromator gives quite accurate wavelength. I believe thatthe mismatch in the curves of the figure is due to the interference of noise in the low digital counts values inthe lower portion of spectra in the interference filter method. The spectral sensitivity determined bymonochromator method will be used in the following experiments.

3.Spectral reconstruction from IBM DCS digital counts

The IBM PRO/3000 trichromatic digital camera system and a set of Kodak Wratten gelatin filtersnumber 38 (Light blue) were used to capture two trichromatic images that combined with the threechannels without filters yielded six channels. Figure 29 shows the Filter number 38 and its holder. Figure30 shows how the filter was attached in front of the camera lens. The transmittance of the filter 38 isgiven in Figure 31.The spectral reflectance factors of the Macbeth color checker was measured andprincipal component analysis were performed to calculate the first nine eigenvectors. The chart wasimaged in 12 bits and the resulting digital counts were used to estimate a linear transformation matrixfrom digital counts to the eigenvalues corresponding to the eigenvectors of the sampled spectralreflectances. The resulting matrix was used to predict the spectral reflectance for the chart and thespectral and colorimetric accuracy was calculated for illuminant D50 and CIE 10 degree standard

18

observer. Figure 32 shows the comparison between measured and estimated spectral reflectance of theCyan patch of the Macbeth Color Checker. The average spectral mean error was 0.17 and the averageRMS error was 0.2. The mean ∆E*ab was 6.9.

Figure 29. Kodak Wratten filter number 38 and holder. Figure 30. Filter holder attached

to the scanner head.

Figure 31. Wratten filter number 38 transmittance

The Wratten filter number 44 (Light green-blue) was added to give three additional signals yieldingnine channels. In this case, the average spectral mean error was 0.029 and the average RMS error was0.049. The mean ∆E*ab was 2.2. As a comparison, Burns and Berns19 performed spectral reconstruction

19

by PCA from camera signals using seven interference filters and a monochrome camera. Their mean∆E*ab was 2.2 for the Color Checker presenting similar performance to the proposed scheme that

combines a trichromatic digital camera with Wratten filters.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

400 450 500 550 600 650 700Wavelength (nm)

Measured Spectral Reflectance

Estimated Spectral Reflectance

Figure 32. The spectral reconstruction from camera signal values for the Cyan sample of Color Checker.

At the present stage simulations using iterative and non-linear methods are been performed to reducethe number of channels and this methods will be applied for Macbeth Checker as well as paintings. Paintpatches (shown in figure 32a) can be imaged and their digital counts (the extraction of the digital countsfrom the 12 bits image using IDL is shown in figure 32b) used to establish a relationship between themeasured spectral reflectance and the digital counts. This a priori processing will be used to reproduceartwork image that uses the same pigments used in the patches (figure 32c).

As a pilot experiment to test image fusion, instead of using scanned photography, the originalimage will be blurred to produce a lower spatial resolution image and the estimated colorimetric valuesand spectral reflectance will be fused with the lightness information of the original image.

Figure 32a. Paint patches Figure 32b. Extraction of the Figure 32c.Artwork painting digital counts of the imaged patches using the same pigments of

a priori imaged patches

20

Future Research

Image Fusion

Figure 33 shows a flowchart of image fusion.25 Once the high-resolution L*(x,y) and the low-resolutionL*a*b*(x’,y’) are obtained, the image fusion is performed. The initial step of this process is thegeometric registration of the image (rotation, scaling, and translation, for example) that can be performedby a variety of commercial software such as ENVI. Among the fusion methods developed in the field ofremote sensing, we decided to combine high-resolution lightness images with low-resolution colorimetricimages in analogy to algorithms that use the hue, intensity and saturation (HIS) color space in remotesensing. We consider CIELAB as a reasonable first-order approximation to a vision model.

Spectral Reconstruction of Hybrid Image

The estimation of high-resolution spectral reflectance R(x,y,λ) from low-resolution reflectance R(x’,y’,λ)

is based on the Wyszecki hypothesis that any stimulus can be decomposed into a fundamental stimulus(with tristimulus values equal to the stimulus) and a metameric black (with tristimulus values equal tozero) whose mathematical technique, known as Matrix R, was developed by Cohen.26 The metamericblack from R(x’,y’) will be fused with the fundamental stimulus from L*a*b*(x,y,λ) resulting in a high-

resolution spectral image R(x,y,λ), and the same techniques used to combine CIELAB images will be

used to merge spectral information.

Figure 33. Image fusion and spectral reconstruction flowchart

21

Discussions

Based on preliminary experiments the IBM DCS was characterized. The experiments suggest that apractical solution to the inherent problems of using interference filters can be solved using trichromaticcamera and absorption filters as an alternative way to perform multi-spectral imaging. This camera inputsystem should be able to make a priori spectral analyses used as a spectrophotometer orspectroradiometer estimating spectral reflectance factors in a pixel basis presenting advantages over directmeasurements.

This camera system will be used in a pilot experiment of a hybrid image capture system. It consists of ahigh-resolution conventional photographic and digital scanning system and a low-resolution trichromaticdigital camera system. Using a priori spectral analyses, linear modeling techniques, and exploiting thehuman visual system's spatial properties, high-resolution spectral and colorimetric images can begenerated. These techniques should be able to be used in many of the photographic departments oftypical museums. Future research is aimed at further experimental optimization, verification, and testingwithin a museum context.

References

1.Saunders, D., Cupitt, J., Image processing at the National Gallery: The VASARI Project, NationalGallery technical bulletin, 14:72 (1993).

2.Berns, R. S., Challenges for color science in multimedia imaging, Proc. CIM’98 Colour Imaging inMultimedia, University of Derby, 123-133 (1998).

3.Hardeberg, J. Y., Schmitt, F., Brettel, H., Crettez, J-P. and Maitre, H., Multispectral imaging inmultimedia, Proc. CIM’98 Colour Imaging in Multimedia, University of Derby, 75-86 (1998).

4.Maitre, H., Schmitt, F. J. M., Crettez, J.-P., Wu, Y., Hardeberg, J. Y., Spectrophotometric imageanalysis of fine art paintings, Proc. IS&T/SID Fourth Color Imaging Conference: Color Science,Systems and Applications, 50-53 (1996).

5.Haneishi, H., Hasegawa, T., Tsumura, N., Miyake, Y., Design of color filters for recording artworks,IS&T’s 50th Annual Conference, 369-372 (1997).

6.Miyake, Y., Yokoyama, Y., Obtaining and reproduction of accurate color images based on humanperception, Proc. SPIE 3300: 190 (1998).

7.König, F., Præfke, W., A multispectral scanner, Proc. CIM’98 Colour Imaging in Multimedia,University of Derby, 63-73 (1998).

8.Vent, D. S. S., Multichannel analysis of object-color spectra, Master Degree Thesis, R.I.T., 1994.9.Burns, P. D., Analysis of image noise in multi-spectral color acquisition, Ph.D. Thesis, R.I.T., 1997.10.König, F., Præfke, W., The practice of multispectral image acquisition, in International symposium on

electronic capture and publishing, Proc. SPIE 3409 (1998).11.Boyton, R. M., Human Color Vision, Optical Society of America, 1992.12.Hunt, R. W. G., Bits, bytes, and square meals in digital imaging, Proc. IS&T/SID Fifth Color Imaging

Conference: Color Science, Systems, and Applications, 1-5 (1997).13.Haydn, R., Dalke, G. W., Henkel, J., and Bare, J. E., Application of IHS color transform to the

processing of multisensor data and image enhancement, Proc. International symposium on remotesensing of arid and semi-arid lands, Cairo, Egypt, 599-616, (1982).

22

14.Braun, G. J., Quantitative evaluation of six multi-spectral, multi-resolution image merger routines, Ph.D. Thesis, R.I.T., 1992.

15.Gross, H. N., An image fusion algorithm for spatially enhancing spectral mixture maps, Ph. D.Thesis, R.I.T., 1996.

16.Vrhel, M. J., Trussel, H. J., Color correction using principal components, Color Res. Appl. 17: 26(1992).

17.Præfke, W., Keusen, T., Optimized basis functions for coding reflectance spectra minimizing thevisual color difference, Proc. IS&T/SID 1995 Color Imaging Conference: Color Science, Systems andApplications, 37-40 (1995).

18.König, F., Reconstruction of natural spectra from a color sensor using nonlinear estimation methods,Proc. IS&T’s 50th annual conference, 454-458 (1997).

19.Burns, P. D., Berns, R. S., Analysis multispectral image capture, Proc. IS&T/SID 1995 ColorImaging Conference: Color Science, Systems and Applications, 19-22 (1996).

20.Vrhel, M. J., Gershon, R., Iwan, L. S., Measurement and analysis of object reflectance spectra, ColorRes. Appl. 19:4 (1994).

21.Pro/3000 Digital Imaging System PISA95 Operator’s Manual Version 6.0, IBM, 1996.22.Pro/3000 Digital Imaging System Reference Guide, IBM, 1996.23.Wyszecki, G., Stiles, W. S., Color Science: Concepts and Methods, Quantitative Data and Formulae,

2nd Edition, John Wiley & Sons, Inc., 1982.24.Handbook of chemistry and physics, 62nd edition, CRC press, Boca Raton, 1981.25.Imai, F. H., Berns, R. S., High-Resolution Multi-Spectral Image Archives – A Hybrid Approach,

Proc. IS&T/SID 1998 Color Imaging Conference: Color Science, Systems and Applications, (to bepublished).

26.Cohen, J. B., Kappauf, W. E., Metameric color stimuli, fundamental metamers, and Wyszecki’smetameric blacks, Am. J. Psychol. 95: 537 (1982).