Mapping functions of color image features and human KANSEI

Yen-Wei Chen1,2, Shouta Sobue2 and Xinyin Huang3

1Electronics & Information Eng., School ,Central South Univ. of Forestry and Tech., Changsha, 410004, China 2College of Information Science and Eng., Ritsumeikan Univ., Shiga, 525-8577, Japan

３School of Education, Soochow University, Suzhou, Jiangsu 215006, China Email: chen@is.ritsumei.ac.jp

Abstract

KANSEI is a Japanese term which means psychological feeling or image of a product. KANSEI engineering refers to the translation of consumers' psychological feeling about a product into perceptual design elements. Recently several researches have been done for image indexing or image retrieval based on KANSEI factors. In this paper, we report a quantitative study on relationship between image color features and human KNASEI factors. We use the semantic differential (SD) method to extract the KANSEI factors (impressions) such as bright, warm from 4 group subjects (Children, students, adults, elderly person) while they viewing an image (painting). A neural network is used to learn the mapping functions (relationships) from the image feature space to human KANSEI factor space (psychological space).

Keywords: color image, mapping function, impression, PCA, neural network, semantic differential 1. Introduction

Recently a growing interests has been seen in

KANSEI engineering or emotion design. KANSEI is an individual subjective impression from a certain artifact, environment or situation using all the senses of sight, hearing, feeling, smell, taste, recognition and balance [1]. KANSEI engineering refers to the translation of consumers' psychological feeling about a product into perceptual design elements. KANSEI can "measure" the feelings and shows the relationship to certain product properties. In consequence, products can be designed to bring forward the intended feeling. KANSEI is the integrated functions of the mind, and various functions exist in during receiving and sending. Filtering, acquiring information, estimating, recognizing, modeling, making relationship, producing, giving information, presenting and etc. are the contents of KANSEI. A person’s KANSEI will be expressed through physiological functions. There are 4 ways of measuring the KANSEI: (1) words; (2) physiological

response (heart rate, EEG); (3) people’s behaviors and actions; (4) Facial and body expressions [2]. The most common way of measuring the KANSEI is through the words. Recently several researches have been done for image indexing or image retrieval based on KANSEI factors (impression words) [3-5]. In this paper, we propose a method to study the relationship between image color features and human KNASEI factors. We use the semantic differential (SD) method to extract the KANSEI factors (impressions) such as bright, warm from 4 group subjects (Children, students, adults, elderly person) while they viewing an image (painting). A neural network is used to learn the mapping functions from the image feature space to human KANSEI factor space (psychological space). 2. Mapping Functions

In order to make a quantitative study on the relationship between the image features and KANSEI factors, we construct an image feature space and a KANSEI factor space (psychological space) as shown in Fig.1. One image (painting) has one point in the image feature space and has a corresponding point in the KANSEI factor space, which is an impression (KANSEI) of the subject to the image, just like a projection of the image feature to the psychological space. The relationship between the image features and the KANSEI can be described by the mapping function from the image feature space to the psychological space. The input of the mapping function is the image features and the output of the function is the KANSEI factors (impressions). Since different person, especially the person at the different age will have a different KANSEI or impression for a given image (painting), we have to use a proper mapping function for KANSEI-based applications such as impression words based image indexing or retrieval. The mapping functions (relationships) can be learned by finding the corresponding points in the image feature space and psychological space.

International Conference on Intelligent Information Hiding and Multimedia Signal Processing

978-0-7695-3278-3/08 $25.00 © 2008 IEEE

DOI 10.1109/IIH-MSP.2008.301

725

Fig.1 Mapping Functions from image feature space into psychological space.

In this paper, we use a neural network as a model of

mapping function as shown in Fig.2. The neural network can be used to approximate any nonlinear functions. The neural network or the mapping function can be learned by finding the corresponding points in the image feature space and psychological space. The input of the neural network is the image features and the output is the corresponding impression.

Fig.2 A neural network used as a model of mapping functions 3. Psychological Features (Impressions)

In order to find a corresponding point in psychological space, we use the semantic differential (SD) method to extract the KANSEI factors (impressions) such as bright, warm from 4 group subjects (Children, students, adults, elderly person) while they viewing an image (painting). In this research, we use words (adjective pairs) to measure the KANSEI. By careful selections, we chose 12 pairs of adjectives, which are shown in Table 1, as measures of KANSEI.

We chose 15 images (oil painting) shown in Fig.3 for learning of mapping functions and validations. We displayed the images to 4 group subjects. (group 1: children at the age of about 10; group 2: college students at the age of about 20; group 3: adults at the age of 40-50; group 4: elderly person at the age of 60-70). Each group consists of 30 subjects. We asked the subjects to rate the impression with 12 pairs of adjectives in 5-step scales (-2, -1, 0, 1, 2). The average values over 30 subjects are used as represent values of each group. The averaged standard deviations are 1.1, 0.6, 0.3, 0.4 for group 1, group 2, group 3 and group 4, respectively. The children group has a relatively large

variation. The typical impression data for image 1 is shown in Fig.4. The number of horizontal axis is the No. of adjective pair in Table 1. It can be seen that there are significant differences among groups.

Fig.3 15 images (oil painting) used for training

Table 1. 12 pairs of adjectives

1. bright - dark 2. vivid - subdued 3. clear - fainted 4. gaudy - plain 5. hard - soft 6. warm - cool 7. pure - impure 8. simple - complex 9. jaunty - placid 10. wide - narrow 11. dry - wet 12. like - dislike

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

1 2 3 4 5 6 7 8 9 10 11 12

Children Students Adults Elderly

Fig.4 The typical impression data for image 1

In order to remove the redundancy and reduce the dimensionality of the impression feature vector, the impression data are analyzed by Principal Component Analysis (PCA) [6] for each group. PCA is to linearly transform the data such that the outputs are uncorrelated. The impression data is expressed by a

726

12-dimensional vector Tzzz ),,( 1221=z . The linear image analysis process is to find a matrix W, so that the resulting vector:

)1(zWy T=

has uncorrelated components. W is constructed in such a way that IWW =T . Actually, the columns of W are the eigenvectors of the covariance matrix of z. The accumulated cover rate is shown in Fig.5. It can be seen that the 12-dimensional adjective feature space can be approximated with top 4 factors, while maintaining more than 90% of information. Thus we chose the top 4 factors as features of KANSEI and constructed a 4-dimensional psychological space. The KANSEI feature vector y is given by

)2(

4

3

2

1

4

3

2

1

==

=

zuzuzuzu

zWy

T

T

T

T

T

yyyy

where u1, u2, u3, u4 are top 4 eigenvectors of the covariance matrix of z, respectively.

0

0.2

0.4

0.6

0.8

1

1.2

1 2 3 4 5 6 7 8 9 10 11 12

Number of Components

Acc

um

ulat

ed c

over

rate

s

Fig.5 Accumulated cover rate.

4. Color Image Features

There are many low-level visual features such as color, texture and shape information in an image. Among them, color is perhaps the most dominant and distinguishing visual feature. In this paper, we focused our researches on the relationship between the color features and the KANSEI factors. Several color features have been proposed to represent color compositions of images [7]. Color histograms are widely used to capture the color information in an image [8]. They are easy to compute and tend to be robust against small changes of camera viewpoints. An image is given in some color space (e.g. red, green, blue). The color channels are quantized into a coarser

space with k bins for red, m bins for green and l bins for blue. Therefore the color histogram is a vector

),,( 21 nhhh=h , where lmkn ××= . In this paper, k=m=n=8 and n=512. Each element hj represents the number of pixels of the discretized color in the image and we normalize histogram elements as

)3(/'1∑ =

=n

j jjj hhh

The normalized histogram vector is used as image

features. The typical images and their color histograms are shown in Fig.6(a) and 5(b), respectively.

Fig.6 Typical images (a); their 512-D color histograms (b);

their 9-D PCA color features (c). The disadvantage of color histogram is its large

dimension, such as 512-dimension. Several methods have been proposed to transform the high dimensional data to low dimensional features. In this paper, we use Principal Component Analysis (PCA), which has been described in Sec.2.1, to reduce the large dimension of color histograms[9]. By using PCA, we can project the 512-dimensional feature space into a 9-dimensional subspace, while maintaining 85% of information. Thus, we select the top 9 components as image features and construct a 9-dimensional image feature space. The examples of PCA color features (9-dimension) are shown in Fig.5(c). It is clear that different color images have different features. 5. Experimental Results

As we described in the previous section, by constructing both image feature space (9-D) and psychological space (4-D), we can easily find a corresponding points for each input images. We use a three-layer neural network as shown in Fig.2 to learn the mapping function between the two spaces. The points (9-dimensional image feature vector) are used as input of the neural network. Thus the number of neurons in the input layer is 9. The output is the point (4-dimensional KANSEI factors) in the psychological space and the number of neuron in the output layer is 4.

727

The number of neurons in the middle layer is 32. The 15 images shown in Fig.3 are used for learning and validations (testing). One image is selected as testing image and remaining 14 images are used as training samples and the corresponding impression data obtained by SD method as described in Sec.3 are used as teaching signals. A backpropagation algorithm is used as a learning algorithm. The learning is done for each group. Finally, we got 4 trained neural networks for each group.

Leave-one-out method is used for validations. No.1 image is used as the testing image for validations. The remaining 14 images (except the testing image) are used for learning. We first calculate the image feature vector (the point in image feature space) and then the corresponding point (KANSEI feature vector) y=(y1, y2, y3, y4)T in psychological space is calculated by using the trained neural networks (mapping functions), which is the output of the neural network. The KANSEI features y are then transformed into impression data (12 pairs of adjectives) using inverse PCA as shown in Eq.(4).

)4(ˆ 44332211 uuuuWyz yyyy +++==

The estimated impressions (estimations) by using the trained neural networks (mapping functions) for each group are shown in Fig.8 and in order to make a comparison, the real impressions obtained by using SD method are also shown in Fig.8. The root mean square errors (RMSE) between the real impressions and the estimated impressions are 1.09, 0.55, 0.35, 0.31 for group 1, group 2, group 3 and group 4, respectively. It can be seen that good agreements between the real impression and estimated impression for college students, adults and elderly person have been achieved. The estimation to children is not so accurate. The main reason is that the impression data of children group (group 1) used for learning have relatively large variations as shown in Sec.3 (the standard deviation of group 1 is 1.1, while other groups are about 0.5). It is difficult to make an accurate estimation based on leaning from the samples which have large variations.

Fig.8 Comparison of estimated results with real impressions. (a) Children; (b) Students; (c) Adults and (d) Elderly person.

5. Conclusions In this paper, a quantitative study has been done for evaluation of relationship between image features and human KNASEI factors. We used the semantic differential (SD) method to extract the KANSEI factors (impressions) such as bright, warm from 4 group subjects (Children, college students, adults, and elderly person) while they viewing an image (painting). A neural network was used to learn the mapping functions from the image feature space to human KANSEI factor space (psychological space). Leave-one-out method has been used for testing. The testing results showed that it is possible to obtain a proper mapping function from the image feature space to the psychological space by using our proposed method. It is also important to use a proper mapping function for KANSEI based applications. This work is supported in part by the "Open Research Center" Project for Private Universities: matching fund subsidy from Japanese Ministry of Education, Culture, Sports, Science, and Technology. References 1. Schwte, S., Eklund, J., Axelsson, J.R.C., Nagamachi,

M.: “Concepts, methods and tools in Kansei Engineering,” Theoretical Issues in Ergonomics Science, Vol.5, pp.214-232, 2004.

2. Grimsæth, K.: “Linking emotions and product features,” KANSEI Engineering, pp.1-45, 2005.

3. Black, J., Kahol, K., Tripathi, P., Panchanathan, S.: “Indexing natural images for retrieval based on Kansei,” Proc. of Human Vision and Electronic Imaging conference, 2004.

4. Takagi, H., Noda, T.: “Media Converter with Impression Preservation Using a Neuro-Genetic Approach,” International J. of Hybrid Intelligent Systems, Vol.1, pp.49-56, 2004.

5. Takagi, H., Noda, T., Cho, S.: “Psychological Space to Hold Impression Among Media in Common for Media Database Retrieval System,” Proc. of IEEE International Conference on System, Man, and Cybernetics (SMC'99), Tokyo, Japan, vol.VI, pp.2630268, 1999.

6. Gianfelici, F., Biagetti, G., Crippa, P. e Turchetti, C.: “A Novel KLT Algorithm Optimized for Small Signal Sets,” Proceedings of 2005 IEEE International Conference of Acoustics, Speech and Signal Processing (IEEE ICASSP 2005), pp.18-23, 2005.

7. Sticker, M.A., Orengo, M.: “Similarity of color images,” Proc. SPIE Storage Retrieval Still Image Video Database, pp.381-392, 1996.

8. Deng, Y., Manjunath, B.S.: “An efficient color representation for image retrieval,” IEEE Trans. Image Processing, Vol.10, pp.140-147, 2001.

9. Zeng, X.Y., Chen, Y.W., Nakao, Z.: “Independent Component Analysis for Color Indexing,” IEICE Trans. Information & Systems, Vol.E87-D, pp.997-1003, 2004.

728