emotion-based interactive installation
Post on 12-Apr-2017
174 Views
Preview:
TRANSCRIPT
Emotion-Based Interactive Installation
Francisco Ventura, António Pedro Oliveira and Amílcar Cardoso
University of Coimbra, Portugal
2
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
3
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
Introduction
4
We prepared an installation to explore the interactive capabilities of an Emotion-Driven Music Engine (EDME)
This installation allows participants to experience and influence the emotional state of a computational system
The system perceives emotions from the environment by using a camera and expresses emotions to the environment with music and images by using, respectively, loudspeakers and a projector
5
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
EDME
6
EDME is a computer program that intends to produce music expressing a desired emotion
It is composed by an offline stage and a real-time stage
At the offline stage, pre-composed music is segmented, emotionally classified and stored in a music base
At the real-time stage, EDME selects from a music base the segments that are closer to a desired emotion
EDME
7
Transformation and sequencing algorithms are applied on selected segments to form song-like structures
These song-like structures are then synthesized and given to the listener
Real-Time Stage
Offline Stage
8
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
Interactive Installation The installation is composed by a computer
system that uses a camera to take pictures of the environment
Interactive Installation The installation is composed by a computer system that uses a camera to take pictures of the environment
The system analyses these pictures to recognize the number of participants and quantity of movement
System recognizes:• Number of Participants• Quantity of Movement
Interactive Installation The installation is composed by a computer
system that uses a camera to take pictures of the environment
The system analyses these pictures to recognize the number of participants and quantity of movement
These two variables are used to define, respectively, the values of valence and arousal of the emotional state of the system
System recognizes:• Number of Participants• Quantity of Movement
Emotional State of the
System
Interactive Installation
12
The installation is composed by a computer system that uses a camera to take pictures of the environment
The system analyses these pictures to recognize the number of participants and quantity of movement
These two variables are used to define, respectively, the values of valence and arousal of the emotional state of the system
The emotional state influences the expression of music and images to the environment
System recognizes:• Number of Participants• Quantity of Movement
Emotional State of the
System
Emotional expression (Music & Images)
Interactive Installation
13
The number of participants is determined by the percentage of change between scenario (image taken when the system starts) and current image
Quantity of movement is determined by the change between current and last image
Interactive Installation
14
EDME generates music according to the values of valence and arousal of the system
As images are emotionally classified with valence, the selection of images is only based on this emotional dimension
Interactive Installation
15
The emotional behaviour of the system is controlled by a multi-agent system
16
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
Experiments
17
We made a preliminary experiment with students with the objective of observing the interaction between the system and participants
Experiments
18
The second experiment was focused in obtaining participants feedback via a questionnaire of 7 questions
Example of questions: Music expressed expected emotion? Importance of images in the emotional expression? System expressed expected emotion?
Experiments
19
Answers from the questionnaire show us that: the system correctly related arousal with the amount of
movement and valence with the number of participants the system was successful in the expression of emotions with
both music and images
20
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
21
Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion
Conclusion
22
EDME is prepared to be used in interactive contexts
People experienced and influenced the emotional state of the system
Both music and images are relevant in the emotional expression of the system
Installation assumes no variation of environmental conditions; however, it can be adapted to deal with this situation by changing the scenario
top related