hyeongrae, lee dept. of biomedical engineering, hanyang university, seoul, korea
DESCRIPTION
Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study. Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang University, Seoul, Korea. Introduction. Introduction. The social virtual environments (VEs). VEs. Virtual avatar. Social VEs. - PowerPoint PPT PresentationTRANSCRIPT
Development Virtual Environments for fMRI Development Virtual Environments for fMRI Socially to Interact with a virtual avatar Socially to Interact with a virtual avatar ; a pilot study; a pilot study
Hyeongrae, Lee
Dept. of Biomedical Engineering, Hanyang University,
Seoul, Korea
IntroductionIntroduction
3
Introduction
• The social virtual environments (VEs)The social virtual environments (VEs)
VEsVEs Virtualavatar
Virtualavatar Social VEsSocial VEs
4
Introduction
Schizophrenia
• The research and therapy using the social VEs The research and therapy using the social VEs
AlcoholismSocial phobia
Social VEsSocial VEsNeuroscienceNeuroscience
5
Displaydevice
Trackingdevice
Tactiledevice
Conventional device
…
Introduction
• The functional MRI environments
MR room
High magnetic and electronic fields
For social VEs, the other method is needed!!!
6
Introduction
•In this study, we purpose that In this study, we purpose that
1.1.development the social VEs for social cognitive neuroscience development the social VEs for social cognitive neuroscience
study using MR compatible devices.study using MR compatible devices.
2.2.a pilot study performed on the developed social VEs.a pilot study performed on the developed social VEs.
MethodsMethods
8
Outside of magnet roomInside of magnet room
Real-timevideo image
Marker detectionon the hand
Computer
Filter panel
Calculatemaker position
Apply hand positionto virtual environments
MR compatible HMD
MR compatible camera
marker
development the social VEs
• fMRI environment for social interactionfMRI environment for social interaction
9
development the social VEs
• Contents for the social VEsContents for the social VEs
(b) Acceptance
(a) Rejection
10
development the social VEs
11
A pilot study A pilot study
Normal groupNormal group
( n = 12 )( n = 12 )
AgeAge 20.92 (SD: 1.78)20.92 (SD: 1.78)
Sex (M:F)Sex (M:F) 5 : 75 : 7
• ParticipantsParticipants
12
A pilot study
• fMRI experiment design
13
A pilot study
• Questionnaires of social realism about the other avatar
(1) How real the other avatar’s action was?(2) Did you feel that you really shook hands with (or was
rejected by) the virtual other ?
• Questionnaires of social cognition about the other avatar
(1) Did you feel positive feeling about the other avatar’s response?
(2) Did you feel negative feeling about the other avatar’s response?
(3) Did you think that other avatar’s response was suitable in the situation?
(4) Did you feel that the other avatar rejected your offer?(5) Did you feel that the other avatar hated you or had
negative emotion about you?(6) Did you feel that the other avatar took threatening
attitude to you?(7) Did you feel that the virtual other ignored or excluded
you?(8) Did you feel that the virtual other was aware of you?
• Questionnaire (5 point response scale )Questionnaire (5 point response scale )
14
A pilot study
• fMRI data acquisitionfMRI data acquisition - The functional MRI conducted with 1.5T machine (GE Medical System).
• BOLD (Blood Oxygenate Level Dependent)
•EPI sequence, Gradient Echo•64x64x30 matrix with 3.75x3.75x5-mm spatial resolution, FOV: 240mm,•Slice thickness: 5mm, TR: 2s, TE: 14.3, FA=90, # of slices: 30
• High resolution anatomical images
•Fast Spoiled Gradient Echo sequence•256x256x116 matrix with 0.94x0.94x1.50-mm spatial resolution, FOV: 240mm,•Slice Thickness: 1.5mm, TR: 8.5s, TE: 1.8s, FA: 12, # of slices: 116).
15
CoregistrationCoregistration
Temporal NormalizationTemporal Normalization
3D construction3D construction
Preprocessing
Preprocessing
Temporal Smoothing(3-point Low pass filter )
Temporal Smoothing(3-point Low pass filter )
Spatial Normalization(MNI N27 templete,
2x2x2mm3)
Spatial Normalization(MNI N27 templete,
2x2x2mm3)
Spatial Smoothing(Gaussian filter, 8 mm
FWHM)
Spatial Smoothing(Gaussian filter, 8 mm
FWHM)
- Analysis of Functional NeuroImages (AFNI, Ver. 2006_06_30_1332)- subjects whose haed movements are more than 1mm are excluded
A pilot study
Statistical Analysis
Statistical Analysis
General Linear ModelAnalysis in each subjectGeneral Linear Model
Analysis in each subject
Group Analysis
Group Analysis
Paired t-test within group(uncorrected P < 0.001)
Paired t-test within group(uncorrected P < 0.001)
• fMRI analysisfMRI analysis
Results & Results & DiscussionsDiscussions
17
- Social realism about other avatars
Results & Discussions
(1) How real the other avatar’s
action
(2) really shake hands with (or
was rejected by) the virtual
other ?
• The questionnaire results.The questionnaire results.
- 1 point : not at all 5 point : very much
18
- social cognition about other avatars
Results & Discussions
• The questionnaire results.The questionnaire results.
(1) positive feeling(2) negative feeling(3) other avatar’s response was
suitable(4) the other avatar rejected
participant offer(5) hated participant or had
negative emotion(6) threatening attitude(7) ignored or excluded participant(8) awareness of participant
- 1 point : not at all 5 point : very much
19
Results & Discussions
uncorrected P <0.001, cluster size > 80 microliter
• Acceptance > Rejection
L thalamus (-11, -17, 8)R insula lobe (37, -3, 0)
• Rejection > Acceptance
L prefrontal gyrus (-17, 53, 20) R rolandic operculum (37, -3, 20)
L superior temporal sulcus (-37, -31, 6)L cerebelum (VI) (-25, -45, -22)
20
Results & Discussions
• Acceptance > Rejection
L thalamus
R insula lobe
• mirroring other’s emotion related to empathy
• Social cognition related friendship and teamwork
21
Results & Discussions
• Rejection > Acceptance
L prefrontal gyrus
L superior temporal sulcus
• understanding other’s mental states• Social emotion (ex. embarrassment)
• Social emotion (ex. embarrassment)
22
In this study, we develop the VEs for social cognitive In this study, we develop the VEs for social cognitive neuroscience study that it is possible to interact with the neuroscience study that it is possible to interact with the virtual avatars. virtual avatars.
This social VEs would provide social environment to This social VEs would provide social environment to patients with deficit in social cognition, such as patients patients with deficit in social cognition, such as patients with schizophrenia, social phobia and etc., even though just with schizophrenia, social phobia and etc., even though just only handshake with other.only handshake with other.
The social interaction methods using maker would be The social interaction methods using maker would be practicable in various social cognitive neuroscience studies.practicable in various social cognitive neuroscience studies.
Summary
23
Prof.Prof.
Ph.D Ph.D candidatecandidate
M.S candidateM.S candidate
Hyeongrae Lee Neuro-mechanism On Virtual Environment Lab.
Dept of Biomedical Engineering In Hanyang University
E-mail: [email protected]: http://bme.hanyang.ac.kr
FAX: +82-2-2296-5943 Tel: +82-2-2290-0691