kansei reaction

7/29/2019 Kansei Reaction

http://slidepdf.com/reader/full/kansei-reaction 1/6

Physiological evaluation on KANSEI reaction caused by the different visual media

Emi NISHINA * **, Yoshitaka FUWAMOTO + Norie KAWAI", Tadao MAEKA WA ***,Reiko YAGI *,Motohiro TANAKA +++, and Tsutomu OOHASHI *

*ATR Human Information Processing Research Laboratories,2-2, Hikaridai, Seika-cho, Soraku-gun , Kyoto 6 19-0288 JAPAN

* * National Institute of Multimedia Education,

2-12, Wakaba, M ihama-ku, Chiba-shi, Chiba 261-0014 JAPAN* * *ATR Media Integration& Communications Research Laboratories,2-2, Hikaridai, Seika-cho, Soraku-gun, Kyo to 6 19-0288 JAPAN

+ Yokkaichi University,

1200, Kayou-cho, Yokkaichi-shi, M ie 5 12-8512 JAPAN

++ Foundation for Advancement of international Science,

3-9-1, Amakubo, Tsukuba-shi, Ibaraki 305-0005 JAPAN

+++Mitsubishi Electric Corporation,5-1 - 1, Oofuna, Kamakura-shi, Kanagawa 247-850 1 JAPAN

A BSTRA CT

Audio-visual information supplied by electronic media is

rapidly beginning to occupy a greater part of our

information environment than natural audio-visualinformation. While contact with visual media can havepositive effects on the user's, it is impossible to deny that

negative effects are also possible. To develop a basic and

effective method t o physiologically assess how visualmedia affect the human brain, we developed a physiological

assessment method based on electroencephalographic alpha

activity (alpha-EEG), by refemng to studies on soundamenities. Its effectiveness is examined in a comparisonbetween a High Definition video format and an NTSC video

format. The results suggest the effectiveness of the method,and the significantly higher conformity of the HD format

with human than that of the NTSC format.

1. I N TRO D U CTI O N

Expectations and dependence on audio-visual media are

increasing in many fields year by year. Much of the naturalaudio-visual information is being replaced by electronicallymediated aud io-visual information in our daily life. On the

one hand, the increasing contact with visual media isexpected to have positive effects on the user's brain. O n the

other hand, it will become necessary to face the problem of

whether mediated visual information negatively effects theuser's brain. Visual information very strongly affects the

brain, and this effect may be amp lified and reach the who lebody through the connection between the nervous systemand the internal secretion system. It is impossible to deny the

risk that unexpected problems will occur when thefkequency of exposure to mediated visual information

increases and mediated visual information occupies a

greater part of our information environment than naturalaudio-visual information.

In the field of visual media technology,

experimental psychology based assessment methods havealready been constructed [ I ] , and many results o f using them

have been accumulated. introspection, which is the

fimdamental approach of such methods, is characterized as

observation of consciousness. It is necessary to develop

evaluation methods for such phenom ena, which are difficultto detect and evaluate by u sing introspection appro ach (e.g.,reactions that affect the nervous system but are h ard for oneto be conscious of). We have investigated a Kansei

evaluation method, based on noninvasive measuring

techniques ofbrain activity to detect responses to sound [2].

We developed a physiological method to examme theresponse to mediated visual information, especially to

evaluate its conformity with the brain, based on the metho d

for sound evaluation. Using this method, we examined theKANSEI reaction caused by visual media in a comparison

between a High-definition (HD) video format and anNTS C

video format.

2. I N V ESTIG A TI O N O N PRESEN TED MA TERI A LS

Prohibition on Selection of Presented Visual Materials

Visual information generally has various aspects and can

give various impressions to individual watchers.

Consequently, if the selection of presented materials is not

adequate, a factor other than the quality of the visual image

may have a more important effect than the quality of thevisual image. Therefore, we ad opted following prohibitionson selection of presented visual materials in th ephysiological experiments.Proh ibitio n A: The factors other than the quality of

the visual image must not have more important effects than

0-7803-5731-0~9/!$10.00 199 9 IEEE W-348



the factor of the quality of visual image (e. g., a program

mainly occupi ed by printed text or speec h, pornography).

Prohibition B: Received information must notinvolve large-scale ch ange by repeated presentation (e. g., a

detective story, a sports game).Prohibition C: The information m ust not fail to keep

the subject‘s motivation or feel hard to watch repeatedly (e.g., intermittent app earanc e and extinction of a circle or a

triangle, a scen e of conversation w ith simple visual imageand contents).

Prohibition D: Differences in media must not fail toaffect the quality of the visual image (e. g., a simply

separated scre en by a few lines; each area of the separatedscreen being monotonously colored).

3. MATERIALS AND METHODSFO R EVALUATION

Presentation Materials

Visual materials: Three titles were chosen from NHK’s

HD video library as presented visual materials tophysiologically evaluate. While two titles were selectedwithout breaking the above prohibition, one title was

selected to dare to conflict with the prohibition. The

selected materials were as follows: (1) FLOWER-The

Mystery of Life (here called as FLOWER), considered

natural and comfortably viewing for human beings; (2)

Animals in Coral Reef (here called as SEA), chosenbecause although human beings may not normally see

underwater scenes, these shape s, colors and movem ents

were thought to be suitable for experiments; (3) “Zolotoi

Petushok” opera (here called OPERA), conflicting with theprohibition because it has a story and super-imposed speech,and it’s shapes and colors were simpler and more roughly

detailed than natural scenes. Three-minute segments were

edited from these materials onto master tapes forexperimental presentation.

Sound materials: The auditory sensory system is one of

the always-active human sensors. Complex and various air

vibrations always and plentihlly exist in nature.

Accordingly, when evalu ating visual media, it is important

to consider the relationship with sound information.

Gamelan music on Bali Island, Indonesia was used as sound

material. Gamelan sound seems to be in harmony with the

videos “FLOW ER’ and “SEA” .Gamelan sound is rich in

high-frequency natural sou nd above the audible range. The

hypersonic effect has been observed caused by the high

fiequency component o f gamelan sound [2]. Three sound

presentation conditions were used: full range sound (FRS),high cut so und (HCS), and no sound as a state very distantfrom natural sound states. Under the OPERA condition,provided digital so und tracks of the video w ere presented. Itdid not contam the high frequency components above theaudible range. Tha t is, only H CS and no sound conditionswere used under the OPERA condition.

Equipment

Visual presentation system: Video information was

reproduced by a 1- inch digital HD VTR and projected by a

53- nch back-projection video monitor in a room specially

design ed for audio-visual presentation. Th e tape used under

the HD condition was a digital copy from the originallyedited HD digital master tape. The tape used under theNT SC cond ition was up-converted after being down-converted from the HD master tape. That is, the quality o fth eNTSC format examined in this experiment was equivalent

to that of the conventional NTSC format in resolution and

was equivalent to that of the HD format in density of thescanning line, i.e. nearly the EDTV format. Therefore, ifsome difference were detected under this experimentalcondition, perhaps a greater difference would exist under

actual conditions.

Audio presentation system: With the FLOWER and

SEA videos, sound reproduced from a 2-inch analog tape

machine was divided into the high fiequency component(HFC) and the low frequency component (LFC). Cut-offfrequency of each high-pass and low -pass filter was 26 kHz.

Each compo nent was independently reproduced by deferentpower amplifie rs and loud speakers. We called this signal

pass design as Bi-channel Sound Reproducing System [2].

The condition reproducing only LFC was HCS, while HFC

+ LFC was FRS. With the OP ERA video, the digital sound(48 kHz, 16 bit) recorded in the so und track ofth e video tape

was reproduced.

Control room Audio-visual room

Figure 1 Experimental equipment

ProcedureEach kind of the three three-minute video materials,

FLOWER, SEA, and OPERA, were reproduced by twovideo format, HD and NTSC, combined with three soundreproducing conditions, FRS, HCS, an dn o sound. The orderof the presented conditions for each video material was as

follows: (1 ) NTSC (no sound); interval; (2) HD (no sound);interval; (3) NTSC (HCS); interval; (4) HD (HCS); interval;( 5 ) NTSC (FRS); interval; (6) H D (FRS).

Iv- 49



Subjects

The subjects for these experiments were twelve adult menand women. All of them had experienced EEG measurementin the past, so they were not overly 6ightened or

uncomfortable when th e experiments were conducted.

EEG Measurement and Analysis

Special attention was paid to the am enity of the subjects'

immediate environment to avoid discomfort The

equipm ent for EEG recordings was hidden from the subjects'

view, and all cables for the presentation were put in a pitbelow the floor.

out of all twelve subjects under the predetermined

6 Al l titles._E 1.

3a

2z{ O

c

m

g

-m

zI0

F5 0.

J NTSC HD \

Measurement: EEGs were recorded using a

telemetric system to minimize constraint of the subjects andstored on ma gnetic tapes for the following off-line analysis.EEGs were recorded continuously including intervalsbetween the conditions. Data were recorded 6 o m 12 scalp

according to the International 10-20 System using linked

earlobe electrodes as a reference.

Analysis: The obtained EEGs were subjected to powerspectra analysis. The power spectrum of EEGs at eachelectrode was calculated by Fast Fourier Transform ( F I T )

analysis in every 2-sec epoch with an overlap of one second

at the fiquency resolution of 0.5 Hz with sampling

frequency of 256 Hz.

The brain electric activity map (BEAM) wascalculated and illustrated at each 20-second epoch in th e180-second presentation for each sub ject According to the

examination of BEAMS, the EEG data for 140-second epoch

fiom 40 second s after the beginning of the presentation to

the end of the presentation was subjected to the following

analysis because of stability of alpha-EEG. the EEG data onthe fkontal area was omitted and that of the parietal and

occipital areas was subjected to the following analysis inorder to eliminate artifacts caused by eye movements. The

subjected EEG data was normalized with respect to the

mean valu e across all time epoch s, conditions and electrode

positions of each subject in order to eliminate possible

confounding inter-subject variability. Consequently, theparietal and occipital normalized alpha-EEG da ta during the

time epochs fiom 40- t o 180-second was subjected t o the

statistical evaluation of condition effects. Analysis ofvariance (ANOVA) followed by Fishers' PLSD post hoc

test was used for assessing the statistical significance across

different conditions.

4. RESULTS AND DISCUSSION

Confirmation of Alpha-EEG Generation in Subjects

When Their Eyes Are Open

To prepare for the a nalysis of the obtained data, we carefullystudied the alpha-EEG generation in each subject Ourexamination revealed distinct and continuous alpha-EEG inthe temporal waveforms of the electroencephalograms of ten

Figure 2 Difference of alpha-EEG caused by video formats

testing conditions. In the p ower spectra obtained by the FFTduring the presentation, we observed clear peaks in th ealpha-EEG ranges (8- 13 Hz) of the same ten subjects. These

results show th at clear alpha-EEG is generatedeven with theopen-eyes siaation, in the caretidly arranged condition to

reduce the stress to the subject caused by th e me asureme ntFollowing analyses were conducted forthe d ataof the above

ten subjects

Effects of Different Video Formats on Alpha-EEG

Potential

First, we focused ou r attention on th e effects of the HD andNTSC video formats under all test conditions. We

compared the normalized alpha-EEG potential obtained

during the presentation of HD and NTSC video imageswithout consideration of the video materials and sound

conditions. (Figure 2(a)) This comparison revealed a clear

tendency for the alpha-EEG potential to be significantlyhigher with the presentation of HD images than NTSC

images @<0.05). Very few experiments have beenconducted in the past to evaluate quality differences in visual

information using an objective method other than aquestionnaire-based survey (use of questionnaires reflectsthe subjectivity of subjects to a large extent). Our finding

verified that quality difference in visual information couldbe quantified by a physiological and objective index. We

believe that we achieved the main objective of our study,that is, to develop a physiological evaluation method forvisual information using EEG as an index.

lV- 50



A difference was observed in the alpha-EEG

potential resulting fiom the video format between thematerial which conflicted with the prohibitions establishedby u s and those which did not conflict with the prohibitions

(Figure 2(b) ) . [HD>NTS C] was observed in all ten subjectswith the “FLOWER” video, and in nine subjecls with the

“SEA” video. Combined, 19 cases out of 20 resulted inmD >N TSC ] with the “FLO WER ’ and “SEA” videos. We

exammed the measurements with the “FLOWER” and“SEA” videos using ANOVA and identified statistical

significance for [HD>NTSC] (p<0.05). In contrast, with the“OPERA” video, [HD>NTSC] was prominent in sixsubjec$, and pT SC >H D] in four subjects, thus indicating

that the “OPERA” video created no distinct tendency in thealpha-EEG potential. Statistical analysis of this particular

result also showed that the difference in the alpha-EEGpotential between HD an d NTSC did not reach a significant

lev el We consid erthese results to be proof of the adequacy

of th e prohibitions for test videos established by us. The real

images of “FLOWER’ and “SEA” videos, which do not

conflict with the prohibitions, have fine granularity andcomplex fluctuation which are too delica te and subtle to be

recorded and represented by any existing video format.Therefore, the different resolutions of the two video formats

would cause the significant effect. On the other hand, the“OPERA” video, which conflicts with the prohibitions,

includes many artificial materials and o ffers amuch simplerand homogeneou s spatial pattern when compared t o the

images o f natural elements.

The introduction o f HD may be thought to bring avalue-added improvement of quality and not to beindispensable. However, based on the present findings, wecannot deny the importance of promoting the conversion of

conventional systems to the HD system for health and safetyreasons, especially for people who spend many hours

watching TV images.

“OPERA’ “FLOWER”

prohibition- prohibitionconflicting non-confilicting

.“SEA”

Figure3 Difference of alpha-EEG caused by video contents

nosound H C S FRS

Figure4 Differenceof alpha-EEG causedby sound condition

Effects of Video C ontents on Alpha-EEG PotentialWe tried to exam ine the effects of video contents. The video

images were classified into those that conflicted with theprohibitions and those that did not, without any distinction of

the video formats or sound conditions. ANOVA revealed

that the alpha-EEG gen erated during the presentation of

prohibition non-conflicting images had a significantly

higher potential than the alpha-EEG generated during thepresentation o f prohibition-conflicting images 0><0.05)

(Figure 3). The images selected for this research were also

contrasted by non-visual elements such as dialogues and

story developments, in addition to the visual contrast of

natural occurrences and artificial products. Which elementhad an influence in creating the difference in the alpha-EEGpotential could not be detected. Nevertheless, these resultssuggested th e future possibility of research into the effects

of video content o n brain activities. When such evaluation

and analysis become possible, the resulting data will help tocreate more visually effective content for video information

media.

Effects of Image-Accompanying Sound on Alpha-EEG

Potential

We focused on three sound conditions: silence, HCS and

FRS. The data during the presentation of the “FLOWER’

and “SEA” videos were subjected to the analysis, bec ausethe “OPERA presentation did not include the FRS

condition. The comparison indicated that the alpha-EEGpotential was lowest when the image was shown withoutsoun d, followed by the ima ge with HCS, then the im age with

FRS (Figure 4). ANOVA revealed that the alpha-EEG

potential provided by th e image with FRS w as significantlyhigher than that achieved by the image with HCS (p<O.OI).

This confirmed that a hypersonic effect was producedduring the presentation of video images. ANOVA alsorevealed that silence noticeably reduced the alpha-EEGpotential as compared to the HC S or FR S conditions withconsidera bly hig h si gn ifica nce of p<O.OOOl. It is no tew orth ythat the alpha-EEG potential was restrained substantially in

Iv-351



.1 1 Video “FLOWER” Tm

C

Q,

0

Q

.-c

c

g 1.0-

2IsQ

(II

U

-

.-3 0.9

EC

0

Q,cn

Q,

.c

E! 0.8-

k

-

situation becomes1 more naturale

I I I I I IVideoformat NTSC HD NTSC HD NTSC HD

Sound condition no sound HCS FR Su u u

rebt inteCval

Figure 5 Alpha-EEG and deg ree of quality difference as

compared with the original and natural state of infomation

environment

intehral intehral intehral intehral intdrval

the silent condition as compared t o that when sound was

provided. We suspect that a significant load is imposed onthe brain by the condition of silence while subjects are

attentively viewing an video image, since it creates an

audio-visual information str ucture that is q uite differentfrom natural conditions.

Review of the Results Based on O riginal-Adaptive Model

We would like to intro duce a co ncept of ‘?he genetically

original and natural state where no adaptation is required

[3].” It is assumed that the brain is conformed to a

genetically original and natural state of information

environment. the brain seeks a condition in which no

inappropriate information input ex ists, and, at the same time,

demands continued input of essential information. The

higher load on the brain may occur at the larger distance

from the genetically original and natural state of information

environm ent. The alpha -EEG activity is suppressed by the

brain’s load [4]. Consequently, alpha-EEG may be a good

indicator of the distance between the present informationenvironm ent and the ge netically original and natural state ofinformation environment.

The degree of quality difference of the presentedaudio-visual information as compared with the original and

natural state of information environment is considered to be

[NTSC>HD] in terms of video system, and[Silence>HCS>FRS] in terms of sou nd condition. So, we

1.81 -r

Video “FLOWER”

$ 1. 61 ,” ~~~~~~ video 1,

0

a Video “OPERA”

W

A Interval between video “SEA”

1. 4 0 Interval between video “OPERA” /c

Figure 6 Alpha-EEG and deg ree of quality difference as

compared with t h e original and natural state of infonnation

environment

arranged the combinations of video format and sound

condition in an order correspon ding to the distance from the

original and natural state of information environment. In

other words, we hypothesized an axis of line coordinates on

which audio-video combinations were arranged in the orderof (1)NTSC + Silence, (2) HD + Silence, (3) NTSC + HCS,

(4) HD + HCS, (5) NTSC + FRS, (6) HD + FRS, in the

direction toward the original and natural audio-visual

information enviro nment. We then plotted the average value

of the normalized alpha-EEG potential of all subjects (10

persons) on the chart of coordinates according to visual

material types (Figure 5). The alpha-EEG w as lowest when

the “FLOWER” and “SEA” videos were shown under the[NTSC + Silence] conditions. The alpha-EEG potential

increased on the coordin ate axis towards the right, and the

alpha-EEG potential was largest at [HD + FRS]. This

complies with the prediction that the alpha-EEG potential

would become lower as the image and sound conditionsveered from the original and natural audio-visual

information state and that the alpha-EEG would increasewhen close to the original and natural state. This supports

the validity of our hypothesis that the alpha-EEG potentialserves as an index of closeness with respect to th e

genetically original and natural state of informationenvironment. On the other hand, the graph curve for the“OPERA” video had a small inclination and lacked th e

lV- 52



characteristic exhibited by the alpha-EEG potential

generated during the presentation of the other informationmaterials. It w ould be safer to regard this as an inconclusive

result. We suspect that the tendency observed during thepresentation of the “OPERA” video was attributed to theblunt composition and the lack of delicate expression that

was contained in the natural information environment. Weconcluded that, with such video images as did not require

high-quality reproduction of the audio-visual information,

the differences of the video format and sound conditions

would not lead to a physiological reaction of a detectable

level.

Comparison of Effects of Natural Visual Images and

Electronically Mediated Visual Images on Alpha-EEG

Potential

On the assumption that alpha-EEG serves as an index of the

closeness of an image to the original and natural state of

visual information based on the original-adaptive model, thepreviously stated results suggest that HD images generate

less load on the brain than NTSC images. However, we did

not obtain any evidence that HD images satisfied all

requirements for high conform ity to the human brain or thatHD images were problem-he for human viewing. In this

regard, we compared the alpha-EEG potential generated

during the presentation of electronically mediated visual

images and un processed natural images (real im ages). This

comparison was conducted as a supplementary experiment,

and the testing conditions were not strictly controlled.

We provided an interval of about one minu te

between each of two video imag es. During this intermission,subjecls did not watch images on TV, and only the roominterior with ordinary indoor lighting was in their sight We

continued the record ing of brain waves during the intervals.

Based on the measuremen$ taken during the intervals with

no image display, we calculated the average of thenormalized alpha-EE G potential of all subjects ( I O persons)

with their eyes open. Figure 6 shows this dataplotted on top

of Figure 5. In all except for one case, the results show edthat the alpha-EEG potential before and after a video

showing was clearly higher than the potential generatedduring an video im age presentation. This result indicated an

undeniable possibility that the image on TV, regardless ofwhether it was HD or NTSC , was recognizedby the brain as

quasi-information far enough from the original and natural

image, and that the vid eo image generated a certain level ofunconform ity. The alpha-EE G potential immed iately after

the presentation of an NTSC video image was lower thanthat obtained immediately before the image presentation in

many cases, while that immediately after an HD video imagepresentation was higher than that obtained immediately

before. We suspect that this difference was caused by the

different levels of load on the brain that the two video

formats created and that the HD system had a higher level ofconformity with brain activities.

At any rate, the above result suggests that even

high-defin tion video images, which are the highest videoquality standard currently provided, have not reach an

adequate conform ity level in terms ofhum an brain functions.During the development of the HD system, evaluation anddecisions were generally based on psychological

experiments. There was no adequate physiological

evaluation method suihble for practical application at thattime, and physiological aspects were neglected for practical

purposes in the establishment of the HD system. Therefore,

the physiological suitability of video systems to hum an brainactivities commands continued research. According to the

present findings, there is an undeniable need for further

assessment of physiological effects caused by HD video

images on the visual infom ation processing system of thehuman body, particularly the conformity with the brain.

However, the conformity of the HD format to humans isnoticeably higher than that of the NTSC format, and there

may be an urgent need t o replace NTSC video images with

higher-quality HD video images.

5. CONCLUSION

An alpha-EEG indicated physiological evaluation method

was developed to investigate the suitability of mediated

video information to the human brain. EffecB of HD and

NTSC video formats were examined and compared with

each other using this method. The results su ggested that theconformity of the HD format to humans isnoticeably higherthan that of the N TSC format, and verified the validity oftheevaluation method

REFERENCES

[ l] The Institute of Television Engineers of Japan ed.,Evaluation technrijues for TV pictures, Tokyo, CoronaPublishing Co. Ltd., 1986 (written in Japanese).

[2] T. Oohashi et al., “High Frequency Sounds above theAudible Range Affect Brain Electric Activity and Sound

Perception,” Audio Engineering Sociely 91st Convention

Preprint, 3207, 199 , p. 1-25.

[3] T. Oohashi, Science and technology fo r information

environment (Jouhou_Kankyou_gah), Tokyo, Asakura-

shoten, 1989 (written in Japanese).[4 ] S . M. Nowak and T. J . Marczynski, “Trait anxiety is

reflected in EEG alpha response to stress,”

Electroencephalography and C linical Neurophysiology,52 ,1981, pp. 1 75-191

Iv- 5 3

kansei reaction

Documents