IMAGE FUSION AND PSEUDO COLORING IN MRI ANALYSIS

Project Abstract: Providing a simple, single modality Image fusion and pseudo-coloring algorithm for better visual interpretation of multiple MRI scan images of the same section of the brain. The project is also compared with the ISODATA model for better visual perception of multiple MRI scan images of the same section. The constraints of both methods are also discussed. The paper also distinguishes between single modality and multiple modality based image fusion in MRI while citing examples of each.

I.INTRODUCTION

The purpose of an image fusion process is to combine a number of multimodal or multispectral images into a final entity that comprises the maximum possible information, which is present in the source images. The source images often exhibit a high degree of correlation since the same area is covered in different regions of the electromagnetic spectrum or with complementary imaging technologies. Thus, the same information can be found in more than one of the source images and is described as overlapping information.

The additional information of the panchromatic band in combination with the multi-spectral bands, allows the retrieval of maximum image information from the given image data set. According to Pohl, Van Genderen and Wald, ‘Image fusion is a process of performing the alliance of multi-

scale, multi-spectral or multi-temporal remote sensing data and generating a new data with higher information content’. The main objectives of image fusion are improved image reliability (by redundant information) and also improved image capability (by complementary information). Ideally, the method used to merge data sets with high spatial and high spectral resolution should not distort the spectral characteristics of the high-spectral resolution data.

II. CATEGORIZATION OF IMAGE FUSION ALGORITHMS

The algorithms available for image fusion, operate on a pixel-level, feature-level and decision-level. We concentrate on the pixel-based fusion which is performed at the level of spectral radiance values and offers minimum of original spectral information. This technique requires the input images to be registered with high accuracy of less than half a pixel, since incorrect registration can cause artificial colors in features of data, thereby leading to falsifying of interpretation.

Image fusion techniques can also be categorized into three types, color-related, numerical/ statistical-related and combined approaches. All color-related techniques employ slicing of original data into their respective layers, which can be basic RGB, human perceived IHS, HSV or more scientific luminance–chrominance. This is followed by substitution by a high resolution image in place of one of these channels and a back-transformation of this combination into the original RGB domain.

III. REVIEW OF IMAGE FUSION TECHNIQUES CURRENTLY IN USE IN MR IMAGING

There are several kinds of Image fusion techniques in MRI interpretation at the moment. One such technique which applies multimodal image fusion is the image fusion of an MRI scan and a CT scan for treatment planning in Tumor treatment. Treatment planning based on fused CT and MRI data enables better definition of target volume and risk structures as compared to treatment planning based on CT alone. Here, the image fusion technique only fuses a single MRI scan based image with a CT scan image and is useful for registration of the parts(to view the critical organs while viewing the bone-related information). The term ‘modalities’ here refers to either of ‘CT’, ‘MRI’, ‘PET’ and ‘SPECT’. The Image fusion techniques presently in use combine images of different modalities to form a fused image. This is often mentioned in publications as the 3TP method.

In terms of single modality based image fusion, a particular technique has been identified as well as published. In this publication, a multi-parametric MR image set was analyzed with the iterative self-organizing data (ISODATA) technique and it consisted of T1-weighted images, fat-suppressed T2-weighted images, and three-dimensional fat-suppressed T1- weighted images which were acquired before and during contrast material enhancement (see MR Imaging).These imaging sequences constitute the conventional breast MR imaging examination, and they were selected for ISODATA analysis because each sequence provides different contrast to disclose different tissue types.

If we assume that each tissue type has characteristic signal intensity on each MR image type, then each tissue type will form a

cluster in a feature space the axes of which represent the signal intensity of that tissue on MR images of that type. These clusters are then represented by different shades of gray and pseudo-colored using various color-maps until a suitable colormap is found for the fused image.

IV. CHANNEL BASED IMAGE FUSION AND PSEUDOCOLORING

An MRI is defined by its tissue selectivity for contrast, and physicians generally derive diagnostic information from an MRI because a particular tissue is displayed with a different contrast. However, each of these MRI scans having different diagnostically useful data can be differently identified using machine parameters such as Repetition Time, Echo Time and Inversion Time. Most of these parameters are available in the DICOM header file which is filled in by the machine. There are various commonly used MRI scans, namely, T1 weighted image scan, T2 weighted image scan, T1 FLAIR weighted image scan, Post Contrast T1 weighted image scan.

We can combine these different scans for better visual perception of data by Image fusion with channel based coloring of individual MRI scans. For example, for three MRI scans T2 weighted image, T1 FLAIR weighted image and Post Contrast T1 weighted scan we get the fused and channel colored image as:

R (m ,n)G(m , n)B (m,n)

= T 2(m, n)

FLAIR (m, n)PCT 1FLAIR(m, n)

These parameters were obtained by checking each of the physician identified images and seeking the parameters from the DICOM header file.

Parameter T1 FLAIR

T2 T2 FLAIR

PCT1

Echo Time

>60 60 <30 <30

Inversion Time

More than 1000

Less than 1000

500-1000

Less than 1000

RepetitionTime

>3000 > 3000

1500-3000

1500-3000

Contrast Info

- - - ‘IV’

Are the parameters for a ‘GE’ MRI scan machine of 1.5 T field strength.

V.MRI SEQUENCES AND THEIR INFORMATION

Each of these series provides a different form of information to the physician. For example, Fluid attenuated inversion recovery (FLAIR) is a pulse sequence an inversion recovery technique that nulls fluids. It can be used in brain imaging to suppress cerebrospinal fluid (CSF) effects on the image, so as to bring out the peri-ventricular hyper-intense lesions, such as multiple sclerosis (MS) plaques. Its usefulness is different from a T1 weighted image. T1 weighted image (also referred to as T1WI) is one of the basic pulse sequences in MRI and demonstrates the differences in the T1 relaxation time of tissues. T1WI relies upon the longitudinal relaxation of the Net Magnetization Vector. Fat has a large longitudinal and transverse magnetization vector and hence appears bright on a T1 weighted image. On the other hand, water appears to have less longitudinal magnetization prior to the RF pulse. Thus, water has low signals and appears dark. Therefore the T1 weighted image shows a

contrast between a fat tissue and a water tissue while a FLAIR sequence image forms a set of elements while nullifying the fluid data in the image. This can be thought of two sets of elements with a common factor. Identifying both these sets and their common elements must be done for diagnosis of the problem. This can be further simplified by fusing the images and giving each of these series a particular color channel. The physician can then diagnose the problem by identifying regions of problem occurrence in each of the underlying colors. This method, helps because, it provides greater amount of information in the image as well as better visual interpretation of the image.

T2 weighted image (also referred to as T2WI) is one of the basic pulse sequences in MRI and demonstrates the differences in the T2 relaxation time of tissues. The T2WI relies upon the transverse relaxation of the net magnetization vector (NMV). T2 weighting tend to have long TE and TR times. In a T2 weighted image, the fat portion of a tissue appears intermediate bright whereas the water portion appears very bright.

So if the doctor wants to identify between white matter and gray matter in the brain. He cannot do that purely on the basis of a T1 weighted image. He needs to look into both the T1 weighted image and the T2 weighted image. The white matter is wrapped in a fatty layer called myelin, which insulates the axons and allows them to conduct signals quickly, much like rubber insulation does for electrical wires. The type of fat in myelin makes it look white, so myelin-dense white matter takes on a white hue as well. Because gray cells are not surrounded by white

myelin, they take on the natural grayish color of the neurons and glial cells. Now suppose that the T1 weighted MRI was fused with the T2 weighted MRI. We could from this fused image directly distinguish between gray matter and white matter which was previously a function of a T1 weighted image alone. Also the function of a T2 weighted image could be seen as well. (Detecting bleeds, swellings at the same point of time.) Passing each of these images through a specific color channel (Red, Blue or Green) helps in fusing the three grayscale images without much loss of information.

Here, in each image, the parts of the image which are bright in a T1 weighted image will appear slightly red in the pseudo-colored fused image. The parts of the image which are bright in all the three images will appear as shades of gray, while those which are bright in two images will appear as a combination of those colors, while patches which are bright in a single image will appear bright in the input channel alone.

This method thus provides more information as well as easier visual interpretation of the boundaries. Also the edges of the tumors can be more easily detected in the combined image by applying edge detection algorithms to each of the individual images first followed by the same image fusion.

Some sample images are shown below with their processed equivalent images.

CONCLUSION

The proposed algorithm will make it easier for doctors to make informed diagnosis based on MRI scan information for

automated tumor identification and classification. This approach may enable the identification of specific tissue signatures characterisic of benign versus malignant tumors.

REFRENCES

[1] J.H. Jang and J.B. Ra,” Pseudo Color Image fusion based on Intensity-Hue-Saturation Color Space”, IEEE Conf. on Multisensor Fusion and Integration for Intelligent systems, TE 4-3.

[2] A. Toet, “Natural color mapping for multiband night vision imagery,” Information Fusion, vol. 4, pp. 155-166, 2003.

[3] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Prentice-Hall, 2002.

[4] J. H. Jang, Y. S. Kim, and J. B. Ra, “Image enhancement in multi-resolution multi-sensor fusion,” Proc. IEEE AVSS, pp. 289-294, Sep. 2007.

[5] Nargess Memaradeghi,”A Fast implementation of the ISODATA clustering Algorithm”, Itnl.J. Computing and Geometry, 2006.

[6] Tou J, Gonzales R. “Pattern recognitionprinciples. Reading, Mass: Addison-Wesley,1974.

[7] Michael A. Jacobs, PhD Peter B. Barker, D Phil, David A. Bluemke, et al, “ Benign and Malignant Breast Lesions: Diagnosis with Multi-parametric MR Imaging”, Radiology 2003; 229:225–232

[8] Thorsen Twellmann, Oliver Lichte, et al ,”An Adaptive extended color scale for comparison of pseudo-coloring techniques used for DCE-MRI Data”,Applied Neuroinformatics group, Department of Radiology, University of Munich.

(a) Post contrast T1 weighted image (b) T1 FLAIR weighted image(c) T2 FLAIR weighted image(d) Fused and Pseudo-colored image