sounds like play looks like play
Post on 23-Mar-2016
225 Views
Preview:
DESCRIPTION
TRANSCRIPT
SLPLLPl i k e P l a y L o o k s L i k e P L AYS o u n d s
See what we can do with a
... and suddenly Ben Schouten came up with a group of students who were crazy enough to work with Philips
Floris Provoost
““
F E S T I V A L
I N T E R V I E W
K I N E C T58
90
WITH FRENS FRIJNS and FLORIS PROVOoST
t h e i n s a n d o u t s o f t h e
108
We created a dynamic, interactive scenery of colours which could be influenced by the movement, speed and position of pas-sersby. The visitors are (subconsciously) making a dynamic ‘painting’, and in the process creating a performance for the lounging people in the surrounding area of the playground. The ‘painting’ is created through the collaboration of the movements of visitors, either con-sciously or unconsciously involved in the game. All individual parts of the concept already existed, Luminous Textile Panels (LTP) by Philips, Kinect depth-sensor by Microsoft, Processing by Ben Fry and Casey Reas, Kinect Library by Thomas Diewalt. The combination of these parts created a whole new platform for giving shape to an unlimited number of interactive activities and visualizations. This also means there was no (real) final design, it was evolving every day it was exhibited at the STRP-festival (and still is in our minds). Since Philips already has a number of corporations using LTP in their buildings, our concept might accelerate the process of interactive environments being incorporated in everyday-life.
“ “
Written by Marieke de Rooyhttp://youtu.be/wRSV0vU4VQc
01. Introduction- Students
- Construction of the report
- Approach
- Project
- Stakeholders
02. Midterm Exhibition- First year students
- Tijmen & Marieke
- Vivian
03. Explorations- Design for festival environment
- Communication and approval of STRP
- Video capturing
- Mock-up panel
- Mock-up panel 11
- Brainstorm interaction
04. Interaction Design- Interaction requirements
- Interaction designs
05. Technical Programming- Choosing a sensor
- Kinect library
- Blob detection
- Multiple Kinects
- How to connect multiple Kinects
- Multiple Kinects connected to one PC
- Pitfalls
- Using a database
- OSC
- Setup with multiple Kinects using OSC
- ArrayList
- From received data to visual
- MinDistBlob
Table of Contents
06. Visual Programming- Testing with Kinects
- Streaming content to Philips Luminous Textile Panels
- Streaming at STRP
- Designing the interaction
- Programming interaction during STRP
07. Production of the festival- Floor plan and technical drawings
- Production of the concept & stakeholders agreements
- Making a model of the festival area
- Mounting the hardware
- Electricity and network
08. STRP Festival- STRP festival
- Research and Development
- Observations
09. Interviews- Frens Frijns
- Floris Provoost
10. Conclusion and Recommendations- Conclusions
- Recommendations
11. Acknowledgments
12. References
13. Appendix
68
10
12
12
14
2022
24
26
2830
32
34
40
42
44
4648
52
5658
60
60
61
61
61
61
62
62
62
64
64
64
108110
112
116
118
120
122
126
130
6670
72
74
76
78
9092
93
94
96
96
98100
101
102
1
9
8 | Intro
du
ction
The Researcher and producer
Vivian Welten,Master Student Industrial DesignFocus: The understanding of the interction in search for behavioral patterns.
My special interest is in
designing for music concerts
and festivals. I think the
design should go beyond
the limits of the stage area
and involve the visitors in
the performance. For that
I strive to create surprising
and memorable experiences
through interactive media
design.
Let the spectator become
participant.
The Creative Director
Marieke De Rooy,Master Student Industrial DesignFocus: Trigger creativity and curiosity through playful experiences.
I aimed this project for
designing co-experiences;
visitors create meaning and
emotion together through
a social and playful activity.
Next to this I wanted visitors
of the festival to be able
to express themselves in a
non-auditive way; through
gestures and colors.
The Technical Programmer
Tijmen Van Gurp,Bachelor 3.2 Student Indus-trial DesignFocus: Creating physicy-cal dynamic interactions with surprizing factors
In this project my focus was
on technical parts of pro-
gramming. In the ideation
phase of this project my fo-
cus was on interactions with
physical cariers that would
suply for a direct interaction.
I believe that interactive con-
cepts should be intuative
and fun. One way to do that
is by giving people a sense
of control, for example the
control of a physical object.
The Visual Programmer
Anika van der Sanden,Bachelor 1.1 Student Indus-trial DesignFocus: creating a co-crea-
tion interaction that could
elite communication
Creating a co-creation inter-
action that could elite
communication. Because
our interactive system was
going to be placed at a
festival I found it interesting
to see if it could elite com-
munication between festival
visitors.
The Allrounder
Tom van ‘ T WesteindeBachelor 1.1 Student Indus-trial DesignFocus: Creating a perfor-mance interaction.
With this project I wanted to
make an interaction with a
unclear starting moment. I
wanted an interaction that
provokes a performance for
visitors in a lounge and
create a dynamic atmos-
phere. I wanted to par-
ticipate in every part of the
process to get an idea of
every aspect.
The Artistic Director
Max Verhoef,Bachelor 1.1 Student Indus-trial DesignFocus: Creating an inter-action which makes the player an artist.
At the early stage of the
project my focus was on an
interaction implemented
with a carrier, which makes it
clear whether you are play-
ing or not. Later on I found
it more important to create
the possibility for the player
to create his own things, so
everyone can be artistic in a
fun way.
The Communica-tion Designer
Alex Schepers,Bachelor 1.2 Student Indus-trial DesignFocus: Creating an in-teraction which triggers communication.
This project my focus was
on creating a social trig-
ger. During one of the first
brainstorm I had come up
with a way to make people
communicate and share
information.
Furthermore making sure
the creative process is
depicted as it is supposed
to be. Create something to
show what has been cre-
ated.
Seven Faces of Sounds like Play looks like Play
10 11ReportConstruction of theSince we have been working as a group
for most of the time, we agreed upon creating a group-report describing the pro-cess of us working in a team. The reporting will start from the moment we started to work together. How did we combine our in-dividual ideas into one concept? After the festival we split up again, all doing our own analysis of the project presented at STRP.
10 | Intro
du
ction
I n the group-report, we will discuss our group-exploration, technical and visual
realization, production of the festival and the exhibition at the STRP-festival. All parts that were done individually are described in a separate report. The con-clusions of these individual activities are combined to formulate one overall con-clusion. This conclusion is presented at the end of the group-report.
Approach
We started out individually, all with our own approach and goals
for this project. During the midterm exhibition we showed our own
concepts. After this exhibition we teamed up and started to formu-
late an overall concept, in which every one of us would be visible as
a designer. All strengths were bundled to create, organize and build
a playful, interactive lighting installation for the STRP festival. In four
weeks we succeeded to create an interactive lighting installation that
enhanced the atmosphere of the lounge area at the STRP-festival.
The Project
This project was set up to create an experience of expressiveness such
as artists perceive expressiveness. This experience would take place
during the STRP festival in the fall of 2011 in Eindhoven.
The project aimed at creating a fun, surprising, unconstrained inter-
active activity that made for a feeling of being creative in its users.
The design of the (playful) interaction was the main challenge of the
project, since the target-group, output device and location were fixed
at the outset. (1)
12
Introduction | 13
STRP’s requirements
Interactive lighting in the lounge/café area of the festival
Playful experience
Connecting visitors
Creating a good atmosphere
14
15 | Intro
du
ction
Stakeholder STRP festival
“The STRP-festival is a yearly festival in Eindhoven and is one
of the largest art & technology festivals in Europe, that focuses
on music, art and technology. The multidisciplinary program is
a mix of a 360-degree experience with adventures which ap-
peal to a wide audience. At STRP there are projects of young
game designers next to major works from the international
art circuit and experimental live cinema next to successful
pop artists and DJs. At STRP you find interactive art, light art,
robotics, concerts, DJs, theatrical and dance performances,
experimental music, interviews discussion, live cinema, films,
lectures, video art, animation and workshops. This year it is
mainly about an overview of the last 50 years of Dutch media
art and technology. The STRP festival took place from Novem-
ber 18 to November 27 at the ‘Klokgebouw’ in Eindhoven.” (2)
The client STRP is looking for interaction between unrelated
visitors. The STRP-festival has a wide variety of visitors. The visi-
tors of the STRP festival are looking for experiences, distrac-
tion, a good time with friends and inspiration, and they go
there out of curiosity. They like to create meaning and emo-
tion together by attending different events during this festival.
16
17 | Intro
du
ction
Stakeholder PhilipsThe client Philips Large Luminous Surfaces is looking
for the possibilities in interaction with their product.
They are interested in the reaction of people and the
communication between people and the Luminous
Textile Panels (LTP). This medium uses LED’s under
textile to create an atmosphere through blurry visuals.
The panels as they are now are static objects which
change appearance in a passive way. Philips provides
eight LTPs with black fabric to be used at the festival.
Their sizes are 250 x 120 cm. (3)
Philips’ requirements Interactivity with and through the LTPs
Exhibiting new product, expanding brand with interac-
tive lighting
Exploring the technical limits of the Panels
Beta-testing for the software
18 | Introduction Introduction | 19
Stakeholder TU/e Industrial Design
The goal of the faculty of Industrial Design is:
“Educating unique opportunity creators for
societal transformation through intelligent
systems. (Systems are networked technology,
products, services and users within a societal
context, and the interaction between them).”(4)
The focus of the theme Playful Interactions is
on changing people’s lives by designing playful
systems that seduce people to activities that
contribute to their health and well-being. As
Huizinga wrote in his book “Homo Ludens”
(1938) people are inherently playful beings.
How can we design products that allow for
playfulness in daily activities?
The playfull interaction group examines the
range between designing for play, such as
play environments and designing for a play-
ful approach to all sorts of other activities, for
example supporting playful communication
between older adults. (5)
TU/e ID’s Requirements Playful interaction
Intelligent systems
Societal transformation
2
Midterm Exhibition ResultsFor the midterm exhibition every student came up
with their own idea for an interaction. We worked out
our own ideas to present different possibilities at the
exhibition.
Anika, Tom, Alex, Max
The first year students want to stimulate communica-
tion between visitors. Every visitor is related to a light
spot on the panels which tries to get their attention.
These lights want to urge you to follow them and lead
you to another person so you can start a conversation.
Because of the interaction between the lights above
you, you might be triggered to interact with the person
standing in front of you.
22
23 | Mid
term Exh
ibitio
n
TijmenTijmen created a concept where people could manipulate con-
tent by movement of a physical object. Tijmen made use of the
accelerometer in his phone to test his interaction. By changing
the angle of the phone the content on the panels would move
accordingly. By putting the phone in a chair people were able to
manipulate the content by changing the angle of the chair. More
about this concept can be read in Tijmen’s individual part in the
latter part of this report.
24
25 | Mid
term Exh
ibitio
n
Marieke created a concept where people could paint on
the LTPs: the system would use the colour of your shirt, your
skin, your hair or the expression of your face to relate to
you on the panels. By looking at the panels you will leave a
trace of the colour of your shirt. If someone is just walking
through the field without looking at the panels he will only
mess up the colours or leave a rainbow of colours.
By working together with other visitors you can create and
change the colours on the panels and thus the lighting in
space. Your swatch would change according to your speed
and direction. By gathering all people underneath one
panel the game play will change.
Marieke
M1.2Vivian came with the idea of a visual world seen from underneath.
Design proposal for content creation on Luminous Textile panels
A direct mapping of the position at the ceiling as if seen from underneath
Motivation
I find it interesting to investigate how interactive multimedia design stimulates
user experiences within the context of public events and entertainment. Within
these event settings interactive media is recently introduced for commercial or
entertainment purposes. I am not directly referring to the social media. I would like
shows and performances to evolve in interactive play and let spectators become
participants by designing such interactive physical environments. Many people are
not used to this type of interaction yet, often the scope of interaction is limited to
human computer interaction. That is why I wonder what kind of input is interesting
for creating an interaction with the environment.
Research questionWhat behaviour is relevant for understanding the interaction between human and
environment?
Aim and objectivesLeads the possibility of interaction between visitor and installation in an architec-
tural context to playful behavior among festival visitors? I would like to find out
whether interactive media in the architectural setting is being recognized and how
it adds value to the experience of visitors in a temporary event setting.
Vivian
26
27 | Mid
term Exh
ibitio
n
Screen shot of Surface Film, part of thesis project MFA Design+ Technology ‘09
What changes are being recognised in the architectural context?
By the use of multimedia systems architectural spaces evolve in dynamic, respon-
sive or interactive environments. One of our biggest concerns is that the visitors will
not recognize the ceiling to be interactive. With subtle changes in the environment
we try to catch the attention of the ones who pass by.
To what extend are the visitors aware of their influence on the output?
The interaction between human and environment is difficult to define and will not
immediately be recognized. More changes in the architectural context might ques-
tion people what their influence could be on this.
What influence has the number of visitors on the behaviour of the participants?
When more people interact with the same medium it might become unclear who
is responsible for the effects. A lot of individuals creating input for the system can
result in the same large variety of output. The individual input and its effect on the
installation is than more difficult to relate. This could be one of the causes for visi-
tors not being aware of the interaction.
What were custom behaviours of the visitors interacting with the panels?
This might be important in making a difference between functional and playful
behavior. By observations we could identify the behavior which is performed to
discover new outputs and exceeds the custom behavior in that setting.
3
Designfor the festival environment 31
30 | Explorations
For this project we had 8 panels that we could use for the interaction. Before we came up with an interaction we decided to reach a clear agreement on how the panels were going to be placed. We made a template in Google SketchUp of the space where our panels would going to be placed at the festival. In this template we all presented different ideas of the positions of the panels. The only require-ments we got from STRP and Philips was that the panels had to be fixed on a minimal height of 3 meters. This was due to the varying public that would visit the festival and we could not afford any damage to the products of Philips.
of the PanelsPositioning
The spaceThe panels were going to be placed in the lounge of the light café of the STRP-festival. The light café is an area for the festival visitors where they have the pos-sibility to eat and drink something and escape from all the impulses from the other areas. In the light café there is a continuous passage of visitors.
After discussing every idea for the posi-tions of the panels we chose four ideas and sent these proposals to STRP so they could
give feedback. Of the four proposals we preferred option
one because of the fact that this option covers a large space. This option also offers many opportunities for interaction since it is effective regardless from which side it is
approached. In option 3 the panels are po-sitioned close to each other, which lights
up one area which is like a spotlight.
The feedback we received on proposal 1 was that with a half open curtain it would
give too much light pollution on the podi-um and proposal 3 was too squared. After this feedback we made one new proposal.
The feedback we received on our new pro-posal was that it would interfere with other light sources and that large parts of the space (two blocks) we had available were not cov-ered. STRP came up with a change in our new proposal. This proposal seems to them the best option because it is a bit asymmetrical and it covered the area we had completely. They also agreed with our idea to position the middle four panels horizontally and the other four a little tilted.
After we discussed the proposal of STRP we agreed that this would be a good option for the positions of the panels.
Communication and approval
of STRP
1. 2. 32 | Explo
ration
s
3. 4.
5.
6.
Video capturing
After the first client meeting with Philips we were much
more familiar with the panels. For the first explorations we
would experiment with different kinds of content for the
panels. We started with filming all sorts of footage in the
photo studio. We all brought different materials, such as
crepe, a disco ball, a flashlight, different colored liquids. We
used different light settings to play with shadows and light
effects. We switched between a black and white background
to see what was the best effect was but we soon agreed on
the black background because this gave the best contrast-
ing effects with the black textile on the panels. We also
filmed drinking glasses that were slowly filled with a liquid
and reversed by drinking through a straw. Finally we shoot
moving objects like rolling and bouncing balls.
With the footage we went to the High Tech Campus to test
this together with material from YouTube. We noticed that
using a variety of colors has a nice effect on the panels.
Movement of visual images only works when the movement
is very recognizable, for example a silhouette of a shadow
play of hands was easily recognizable, but the footage of
quick moving balls did not work out on the panels.
Shadows did have a nice effect but the silhouettes need to
be of such a size so that they are clearly visible and recogniz-
able.
For the next appointment with Philips we made some new
footage. This time we created footage based on the brain-
storm sessions which resulted in the use of natural elements
and upside down worlds. Max made a Flash animation of
walking footprints and Vivian used an enlargement of an ex-
isting video of YouTube to create the same idea as Max. We
also shot new shadow material using cloth and spotlights.
While we were testing this footage we noticed that the foot-
prints had just the same effect as the shadows we tested;
they really had to be big enough to be noticeable.
34
35 | Explo
ration
s
http://www.youtube.com/watch?v=W17Ce9TrD7M&list
=PL825402CE33CC8B00&index=8&feature=plpp_video
http://www.youtube.com/watch?v=X26kt53N35s&list=
PL825402CE33CC8B00&index=9&feature=plpp_video
36 | Conclusions and recomandations Conclusions and recomandations | 37
36 37
36 | Co
nclu
sion
s and
recom
and
ation
s
37 | Co
nclu
sion
s and
recom
and
ation
s
Exploringmovement
Philips Content ManagerIn the last exploration phase with footage we were able to test at the
university since we were equipped with software from Philips to use on
our own laptops. The software would show a preview of how uploaded
videos would look when uploaded to the panels. We used natural ma-
terial as visuals to test with that program. We filmed moving trees, fall-
ing leaves and flowing water. We also got a new idea about the upside
down world. We placed a camera with a fish eye lens on the floor and
walked around it to get the idea of a transparent floor. When importing
these files in the Philips preview-software it seemed that the leaves and
the water were hardly visible because of the blurry effect of the panels.
The trees worked out pretty well because of the color contrast and the
adapted speed of the video file.
38
40 | Explorations Explorations | 41
Mockup Panel
Motivation/reason:At this point in the project we wanted to go from paper
towards reality. Because we did not have a panel avail-
able to test content we decided to make a test-panel so
that we could get a better feeling of the situation at the
festival.
We built a spruce frame, and a surface of white sheets.
We chose the dimension of the panels we would get in
the final setup. We used a beamer with a slightly unfo-
cussed output to simulate content on the blurry panels.
This panel was installed in the hallway for a couple of
weeks. The purpose of this was the possibility to observe
the passing students in a very early stage in the project,
and see how they reacted to different footage. What we
concluded from these observations was that people only
noticed the visuals when there was an obvious variation
in color and movement.
ExamplesAt first we started to beam YouTube content on the
panel. This was just to see what kind of effect this had on
the people who walked by. The videos we played were
mainly very abstract and colorful. Bit by bit the concept
got shape, and we started to find out which visuals we
wanted to have on the panels.
With our requirements in mind Tom made content in
Particle Illusion to simulate this. There were two particles
on the panel which moved around each other and even-
tually met. With the sitting bags underneath the panel
we could experience the effect of video animation from
above just like it would be in the lounge at STRP festival.
But it still was not interactive yet. Tijmen made a Process-
ing sketch which was interactive. An iPhone was used
as accelerometer. Simply by moving the iPhone around
different kinds of balls danced around on the panel. As
soon as it got interactive, the effect of the combination
with interactivity and a large panel above you could be
experienced.
ConclusionThe decision to make a real physical panel brought us
one step closer to the real thing and helped us to test
our ideas in context. The further developed context also
narrowed down the ideas we had towards more concrete
ideas for the setting at the STRP festival. After the panel
was finished we all went our own way in developing our
ideas further. Because we tested our ideas on the panel
during this development we were able to give each
other feedback on the results.
Mockup Panel II
Motivation/reason:When we decided to use multiple Kinects we needed to
simulate a test situation with multiple Kinects that was as
close as possible to the real situation at the STRP festival.
Because we were not able to use the LTPs until the festival
we needed a test situation to find new opportunities and
possible pitfalls.
Construction/situationUntil now we had a panel hanging at a height of 2 meters,
at the festival it would hang at 3.5 meters. Therefore we
chose a spot at the Technical university where we could
make a construction from wood, some duct-tape and wires
to simulate the situation at the STRP festival. In this setup
we mounted 2 Kinects to the wood with a space of approxi-
mately 2.5 meters in between.
ConclusionWith this setup we were able see how large the new play-
ground was. We found out that we would need high quality
USB extension cables to prevent data loss of the Kinects. Be-
cause of this setup we could also decide the direction of the
Kinects, and the amount of space needed between them.
http://www.youtube.com/watch?v=ljrGmuCznKs
43
42 | Explorations
44
45 | Explo
ration
s
Brainstorm interaction
After the midterm exhibition we started thinking about the
concept for the interaction again. The best way to do this
as a group was through several brainstorms.
With the idea that colors really work well with the devices
of Philips we wrote down everything that came to our
minds concerning movement and different colors. Of all
these ideas we chose the ones
that we thought had the best potential. Using these ideas
as elements for another brainstorm we continued to
expand the options of these elements. This brainstorm
resulted in 5 new criteria. With the idea in mind that we had
to make a vague interaction because of the low resolution
of the panels in comparison to “normal panels” we did a
third brainstorm about vagueness. eparate from the last
brainstorm
we did a fourth brainstorm about ceilings because of the
way we are fixing the panels and this resulted in another
brainstorm about an upside down world. An important
part of the content was the way we would draw people’s at-
tention to a spot in the room that you normally hardly look
at. For that we closed the brainstorm sessions with three
brainstorms about seduction and deception.
Eventually there were seven brainstorms on the
table. The themes we thought were most fascinating
were the things you see from below and the ele-
ments of nature. These were the two concepts that
we started to work with, so we started to make foot-
age of this to experience the effect on the panels.
4
In the 6th week there was a group discus-sion about the concept requirements.
Everybody mentioned his or her personal requirements, interests and ambitions.
Vivian Likes the development from invol-
untary towards conscious interac-
tions. To implement this in the
concept we should take the actions
already present as point of origin for
the interaction. This project she will
set up test cases and work on the
analysis, strong visual language and
would like to develop video animat-
ing skills.
Max
Likes interactio
n through objects. H
e
wants to tu
rn ordinary objects into
something extraordinary by designing
its interactio
n. Curio
sity is an in
teresting
human characteristic to
be used. He has
no experience in
programming, so this
project could serve as a nice in
troductio
n
by observing the softw
are experiments
and developments of the concept.
Tom Wants the interaction to be not too obvi-
ous. He thinks there should not be a clearly
defined moment where people decide to take
part in the interaction. The system should do
something unexpected. Like Max he has no
experience in programming and will observe
the software development from the sideline.
He is interested in the overall development of
the concept, the relation between the input
and output and how people will react.
Marieke
Would like a more direct form of interac-
tion and the system should at the end
give rise to an intended and conscious
interaction. The focus is on connect-
ing people and creating together. She
would like to work on the visuals, and
think about the interaction, taking the
output as point of origin, and would
like to find out how people react to the
images.
Alex Thinks that the concept should be
present in a subtle way. It does not
need a clear purpose, but serves as
entertainment; give the visitors the
opportunity to create. He has some
experience in programming and
would like to develop this. He also
would like to improve his visualiza-
tion skills, but not necessarily within
the scope of this project.
Tijmen
Thinks the concept should have an
element of surpris
e and some vari-
ation by m
aking different levels of
complexity. H
e has experience in
programming, which can be useful in
developing the concept. H
e also has
to develop his form
and senses skills,
and this could be achieved by creat-
ing a report l
ay-out.
Anika
Would like the system to react to un-
intended actions. She likes program-
ming, has some experience in PHP
but likes to develop other languages
as well. She wants to develop her visu-
alization skills next semester.
Considering an interactive system consists out of three
elements - input, system processing and output - we sepa-
rated the team in terms of these categories. We would first
explore the possible solutions, then combine the system
and improve the quality.
INPUTTom and Anika
SOFTWARETijmen (towards input) and Alex (towards output)
OUTPUTMarieke, Max and Vivian
After discussing everybody’s concept require-ments we combined them into the following requirements:
Surprising element by random factor
Variable complexity
Subtle in its presence
Turn ordinary into extraordinary
No clear moment of participation
Stimulate co-creation
Trigger intended in-teractions at the end
Sketches
Since the previous brainstorms were more about content
for the panels instead of interaction scenarios, we decided
to do another brainstorm, more focused on interaction. This
time it was based on sketches. We thought about different
scenarios and immediately drew these on paper. This gave a
good impression of what a certain interaction will look like.
The actions that had to be performed for the interaction still
ranged from drinking out of a glass to waving the arms. Also
there were some options on how to start the interaction.
Results
The most important thing that came out of this brainstorm
was the principle of having at least two hot spots or play-
grounds. Using these playgrounds the user would be aware
of whether he/she was joining the interaction or not. The
second thing we wanted was to make a creator out of the
player. He or she would be able to control, paint or influence
the panel by just walking underneath it.
By thinking of these different scenarios we set a major step
towards our definitive concept. The hot spots and painting
principle would be essential for the end result.
Conclusion
5
Choosing a sensor
When reviewing the scenarios we found most interesting,
we could clearly see that most scenarios would use the
position and movement of people to create an interaction.
Using a Kinect as sensorial input sounded very promising,
it was the solution to track the movement and position of
people in space. Regarding the other options we had, this
would be the least fragile and most reliable sensor to use.
Technical specifications of the KinectWhen we choose the Kinect as the sensor for our interac-
tion the input team started to look at the specifications
and possibilities of this sensor with all its multiple options.
While searching the internet we discovered the versatility
of this sensor. Its basic properties satisfies the exact require-
ments of what we want to do with it. The combinations of an
infrared camera, a RGB camera and the infrared laser make
it possible to capture 3D images. The depth sensor consists
of an infrared laser projector combined with a monochrome
CMOS (image)sensor. The Kinect is capable of detecting the
skeleton from one up to 6 people. Because of the motorized
tilt the sensor can vertically detect in an angle of 47 degrees
and can be tilted up to 27 degrees up or down. Horizontally
the Kinect “sees” an angle of 57 degrees which provides a
playfield of 6 square meters. Next to this, we found out that
this sensor is a popular hardware product that has already
been hacked by hundreds of creative programmers. This
59 | Techn
ical Prog
ramm
ing
Technical Programmingmeans that beside the regular software of Microsoft© we
could find a lot of other software that could help us to
realize our ideas. (6)
We wanted to detect people from above, since this
would enable us to precisely locate the position of
multiple people in space. We had to find out what the
Kinect exactly detects when it is fixed parallel with the
ground. For that reason we made a small setup in which
we placed the sensor on a height of 3 meters and with
the software that the technical team had provided we
defined the playground for the interaction. Because of
the adjustable depth sensor we had to go through these
steps twice because the first time we detected the top
of the head during the measurement. This resulted in a
small, non-profitable rectangle in which hardly any inter-
action was possible.
The second time we defined a much bigger rectangle
because we changed the range to detect people’s feet
which gave more options for an interaction. We even
tried to enlarge this playfield using a fish eye lens espe-
cially made for the Kinect. It appeared that this had no
value for the detection because this lens also changed
the accuracy of the sensor in a negative way.
Kinect Library
After a lot of research about which drivers and libraries
to use (see Appendix ) we decided to go for the library
of Thomas Diewald. The main reason for this was that his
sketches from his OpenCV(source) kit worked without
calibration of hands or skeleton. Another advantage of this
library was that Tijmen could use Thomas Diewald as an ex-
pert during the process as he responded quickly to pro-
cessing-related questions via e-mail. The processing sketch
that we eventually used made use of the depth vision of the
Kinect and blob detection to detect persons in the scene. (7)
Blob Detection
Blob detection is a method that is used to filter objects and
people out of a map of pixels detected by a camera using a
certain threshold. Because the Kinect senses depth the use
of Blob detection is much more precise than when using a
normal camera. This is because in a normal camera needs
an contrasting background if you want to detect blobs.
A depth camera makes a high contrast image even in a
chaotic background. Because of this advantage of a depth
camera blob detection is faster with a depth camera than
with a normal camera.
60 | Technical Programming Technical Programming | 61
Multiple Kinects
When we first tested one Kinect searching for blobs from
above (from the ceiling to the floor) we measured the
amount of space in which it could detect blobs. This was
175 x 135 cm for detecting the head when the Kinect was
at a height of 3 m and 245 x 185 cm when the Kinect was at
3,5 m.
The total space that the panels would cover at the STRP
festival was 1000 x 420 cm. Therefore it was logical that we
needed more than one Kinect to extend our playground.
Because there was no budget it was not possible to buy
more than 3 Kinects (3*120 euro) from our own money, this
meant that we could work with a total surface of : 735 x 185
cm.
How to connect multiple Kinects
When we had decided that we wanted to use multiple
Kinects we had to find a way to combine the data into one
visual output. To do this Tijmen thought of several options
to get this done:
Multiple Kinects connected to one PCOne option would be to use one computer for connecting
the Kinects. The advantages would be that we would only
need one pc to receive the data and make a visualization.
Together with Marieke and Anika, Tijmen tried a sketch of
Thomas Diewald made especially for multiple Kinects.
PitfallsWe would need an extremely fast computer to run 2 Kinects
at the same time, we tried this on the newest computers
with a Core i7, on this computer it worked with 2 Kinects
but there was another problem. Adding 3 Kinects to one
motherboard was not possible as the USB slots of a mother-
board can only handle 1 Kinect per USB Slot. On these fast
computers there were only two USB slots available. So this
would not work with three Kinects.
Kinect Playground
Lumalive Panel
Using a databaseWhen was decided that we needed to connect all the data
of the Kinects in a different way Tijmen thought of an option
to use a MySQL database or to update XML files. We didn’t
choose any of these options because it was not practical for
the amount of data we had to transfer.
OSCAnother solution was to use OSC (Open Sound Control
which uses the UDP (User Datagram Protocol) protocol to
send data over a Local area network). Tijmen already had
some experience with this from the midterm exhibition
where an iPhone was connected with WIFI to a PC. During
the midterm exhibition this was done using an iPhone App
Touch OSC, and a library for processing called oscp5 (8).
OSC is also able to transfer data between multiple PC’s and
therefore extremely useful when we would connect multiple
PC’s with each its own Kinect to one PC. (9)
AdvantagesIn the end of the process OSC was the right choice for us.
We had no loss of data at all using the UDP protocol. With
OSC we were able to send the data of the Kinects towards
multiple IP addresses. Because of this it was possible to test
new visualizations during the STRP festival without having
to stop the visualization on the visual PC. OSC showed us
just a glimpse of what was possible, Resolume a VJ software
also has an option to receive OSC messages and thereby
many more like VVVV, MAX MSP and touch OSC. This tool is
highly recommended when you want software to communi-
cate with each other.
Setup with multiple Kinects using OSC
When we had decided to use OSC we had the following
setup:
This setup can be divided into 2 different parts, a sending
part and a receiving part. For each part a processing sketch
was written. Without going too much into detail the follow-
ing text will explain what these 2 parts did.
Sending messagesIn the Kinect sketch we incorporated OSC to send data
about each blob. We sent the data of each blob to the visual
pc. Each computer would send its data with a unique tag so
that in the receiving part on the visual pc the data could be
ordered and mapped in the right way.
Example adding message:
blobMessage.add(bb.xSize());
Example sending message:
oscP5.send(blobMessage, myRemoteLocation);
Example receiving message:
int blob_Xsize =(theOscMessage.get(i+1).intValue());
Receiving messagesThe visual PC received all the messages of the 3 computers.
The incoming data were in pure random order. For example
sometimes we first received 5 messages of one pc until an-
other pc came through. Therefore we had to filter out all the
messages and combine them into one set of data that could
be used for the visualization.
63
62 | Technical Programming
The Kinect sent the data of each blob it
sees in the order it detects these blobs,
and this in the same way that you read
a book. This could mean that in the first
case person one had position one in the
ArrayList. When a new person walks in
the playground this new person would
get position 1 and the one who had first
position 1 would get position 2.
This meant that the visual object that
belonged to the first blob would receive
new x and y values from the second
blob. This made it look as if this first blob
had moved a lot.
With one Kinect this would not be such
a big problem but when you have a
lot of persons and 3 Kinects you would
only get chaos and all the visuals would
change position to such an extent that it
would appear as random.
Therefore we had to find a way for the
position of each visual object to change
according to what was the nearest blob
(person).
To do this we wrote a function that
would calculate every distance between
all blobs and objects and engaged the
blobs to an object which had the small-
est distance. The x and y of this blob
would be used to update the visual ob-
ject. In this way the order of the ArrayList
of blobs could change without having
to change the order in which the visual
objects where stored in their ArrayList.
ArrayLists
Before we go into the visual programming a few things need
to explained first.
During our process of programming we bumped into a lot
of, for us new things. One of these things was ArrayLists.
One reason why we chose to work with ArrayLists was that
with an ArrayList data items can be easily added or removed
from the ArrayList.
Another reason we decided to choose to work with Ar-
rayLists instead of normal arrays was that a ArrayList is a
resizable-array implementation of the Java List interface.
Because of this a ArrayList has many functions we could use
to control or search for content in the ArrayList. Some of
the functions we used in our processing code were size(),
add(), remove(), get() and shuffle(). For instance the function
size() returns the length of the ArrayList and with the get()
function you can request the variable number/object from
a chosen position. We used the function shuffle() to create a
random Mondrian. (10)
From received data to visuals
Once the data received were filtered and ready to be sent to
the visualization there were several difficulties to overcome.
What will happen when someone new walks into the vision
of the Kinect?
Per blob a new visual object had to be created. Each of these
objects was assigned their own color and stored the X and Y
values of the blob in an arraylist.
MinDistBlob:
One big problem during programming was to assign one
specific visual to one person. For the amount of blobs there
were on all the 3 Kinects there were x and y values stored
in an ArrayList. To prevent this problem the order of the
blobs in the ArrayList had to be the same as the order of the
objects in the ArrayList.
65 | Techn
ical Prog
ramm
ing
6
68 | Visual Programming Visual Programming | 69
Visual ProgrammingTesting with mouseWhile Tijmen worked on the technical part of making the Ki-
nect communicate with Processing some first tests were done
with a computer-mouse as input. Marieke adjusted an already
existing code to fit the project better, and tested this on the
Philips Luminous Panels. This first code existed of “fireflies” cir-
cling around the position of the computer mouse. The speed
of the mouse influenced the way the fireflies circled around the
mouse. Since the Panels blurred the output considerably, the
details in the original code would be faded out. A simpler ver-
sion of this code was created to save processor-capacity. This
code had the same effect on the panels.
Marieke also tested her midterm idea, “Painting” in a simplified
“mouse”-version on the LTPs. When pointing to a color on one
panel one could choose the color of their swatch. By moving
your hand around you could “paint” on the panels. Immediately
it became clear that the chosen colors were clashing, and that
the stroke was not wide enough. These factors would have to
be taken into account when continuing with this concept.
Anika started with writing a collide function for the blobs
(objects perceived by the Kinect as people). In this function
different things could happen when blobs would collide, for
example: objects would change color when they collided. What
happens in this collide function is calculating the distance
between all objects; when this distance is under a certain
value collision has occurred. When objects collide this function
triggers another function, in which is written an action on this
event.
Testing with Kinects
These relatively simple codes were rewritten to react to the
output of one Kinect. The codes reacted to one variable X
and Y value (of the mouse), but now had to react to multiple
X and Y values of different blobs. (11)Classes were needed
to keep the code from getting too extensive. Since Tijmen,
Anika en Marieke were not that familiar with this kind of
programming, they asked Wouter van der Heide (2nd years
bachelor student Mathematics) for some help. Wouter
helped them setting up a processing sketch in which differ-
ent classes worked together in a structurized way.
Wouter wrote an AssignObjects class which had the func-
tion of adding or removing objects according to the amount
of blobs. Next to this class there was an Object class, which
contained the characteristics of the objects generated in the
AssignObjects class. Bit by bit they learned to use Object
Oriented Programming, and how they might be able to put
the envisioned interaction scenarios into practice
A problem occurred when they found out the Kinect
changes names of blobs when they have a different posi-
tion. More about this problem can be found in the technical
programming part of this report under mindistBlob. This
had as result that when 2 blobs switch position they will also
switch names, and as a consequence their colors will also be
switched.
Since your object was now always assigned to your blob,
Marieke saw the opportunity to put her mid-term idea into
practice; with a small adaption to the code painting became
possible. A stroke of your color would follow you along the
playground.
Meanwhile Anika focused on creating random objects when
there are no people within the Kinect’s sight. She added
a random blob class to the processing file with some help
from Bram van der Sanden (3th year bachelor student Infor-
matics). These random blobs are just like normal blobs but
are always present. These random blobs could collide with
each other and/or with the normal, “human” blobs (detected
by Kinect). Some things that can happen when a random
blob collided with a normal blob is that the normal blob
would take over the color of the random blob or the normal
blob killed the random blob.
When connecting all three Kinects to the visual PC a new
challenge arrived: combining these values into one visual
output. Marieke altered the incoming X and Y values in such
way that every Kinect had his own part of the output. For
71 | Visu
al Prog
ramm
ing
70
example, blobs of Kinect 1 were drawn in the first one-third
of the panel, and blobs of Kinect3 were drawn in the last
one-third of the panel.
After all the LTPs were installed, Marieke could start on fin-
ishing the mapping in the processing file to a final version.
It appeared that the blobs could have higher X and Y values
than expected. This meant that the resolution of the Kinect
output was not as expected (was not 640 x 480).
Blobs directly underneath the Kinect were not visible on the
Panels. These blobs had to be replaced to a visible area.
73
72 | Visu
al Prog
ramm
ing
Philips developed software for send-
ing content wireless from your pc to the
textile panels over a local network. Philips
content manager, which was the version
available at the beginning of this project,
was able to stream pre-created video
material to multiple panels. The setup of
the panels could be adjusted as long as
they are positioned in an angle of ninety
degrees in relation to each other.
This was actually not enough functional-
ity for the concept to be presented at
STRP festival. This time wireless stream-
ing of live content to multiple panels
was required. During a meeting at Philips
Lighting a programmer from Philips wrote
this custom piece of software which we
only had to adjust to our panel setup. The
software made it possible to stream live
content, selected from our laptop screens,
to the individual textile panels.
GraphEdit connects filters to a video
capture source, like a webcam or in this
case the laptop screen output. The filter
which sets this source to the laptop screen
output is the one we developed at Philips
lighting and was called ucapture.grf. This
filter has to be opened in GraphEdit. To
startup the streaming connection ucap-
ture.avs needs to run in a media player.
For this we used Media Player Classic. In
ucapture.avs the addresses of the panels
and the total pixel mapping are defined.
Several parts of the software were up-
dated during the design process to meet
our requirements.
Streaming content to Philips Luminous Textile Panels
When the LTPs were installed at the festival-area, together with
Philips Marieke started to adept the previously created file for
the streaming of the panels to the mapping of the panels as it
was now. A video file had to be created of the whole screen.
This was done using uCapture. From this file the pixels which
were meant to be sent to the Panels had to be downscaled. Also
the blank areas between the panels had to be calculated and
taken into account.
Streaming at STRP
Video file: 1600x1024 px
Processing output: 1600x720 px
Total resolution of all Panels together: 160x72 px
Resolution of one Panel: 20x40 px
Since the panels draw
their visual input in a
vertical way, horizontal
installed panels needed
to get the information of
the pixels also in a vertical
way. The pixels meant for
these panels needed to be
rotated 90degrees before
they could be send to the
panels.
All these calculations to
prepare the processing
output for sending towards
the LTPs where done in
AviSynth. Marieke worked
together with Floris Pro-
voost (Philips) to write a file
which does all these calcula-
tions.
In the first picture on the
right you can see the output
screen of processing, and
in the second picture you
can see the outcome of the
avisith file in uCapture. This
would then be the file which
would be send to the panels.
At the end of the avisynth file
all the data of the pixels was
send towards a panel with its
respectively ip address.
75
Example code rotation:
clip1 = TurnRight(rclip1)
Example code sending:
clip1 = SimpleSample(clip1, 40,20,2,”192.168.0.186”, 6038, 2,1)
Designing the Interaction
After the streaming was finished, Marieke could start
on finishing the mapping in the processing file to a
final version. It appeared that the blobs could have
higher X and Y values than expected. This meant
that the resolution of the Kinect output was not as
expected (was not 640 x 480).
Blobs directly underneath the Kinect were not visible
on the Panels. These blobs had to be replaced to a
visible area.
When Tijmen was moving around in the lounge area
he found out that he wanted the system to react on
him spreading his arms. This could be done easily by
measuring the width of the blob. Tijmen and Marieke
added different functionalities to this variable. When
pointing your arms to other Kinects you would cre-
ate a white flash when in the playground of Kinect1,
create a random Mondriaan-painting when being in
playground 2 or emit lightning when in area 3. If you
spreaded your arms in a different direction you would
create a huge blob above you. With this big blob you
could push other blobs off the panels.
Tijmen tested his Mid-term exhibition prototype too.
Using his mobile phone with accelerometer, he could
change the gravity of the balls on top of the panels.
These balls changed colors on the sound of the beat.
76 | Visual Programming
Programming interaction during STRP
During the festival Tijmen, Anika and Marieke kept programming additions to this basic interaction.
78
Visual Programming | 79
80
Visual Programming | 81
Painting The ‘painting’ code was tested and adapted several times. Marieke made
the brush size variable to the amount of people in the playground. More
people meant a smaller brush-size. Some experiments were done with
the colors. First random light colors were assigned to the blobs, but
these pastel-colors did not show much contrast with each other. This
had as result that your collaboration in the painting was not directly no-
ticeable. More bright colors were tested, but some colors really clashed
with each other. Some ‘ugly’ colors were filtered out, and in the end the
combination of colors felt satisfactory.
http://www.youtube.com/watch?v=f9kOuoorjds&list=
PL825402CE33CC8B00&index=4&feature=plpp_video
Programming interaction during STRP
82 | Visual Programming
Programming interaction during STRP
Micheal JacksonDorien van den Hurk (STRP) gave us the feedback that visitors of the
concerts during the night needed clearer interaction. Marieke created a
code in which panels would light up when someone was standing under-
neath that panel. When walking around panels would light up above you,
always in the same color. Since this interaction method did not work that
well when there are many people walking around in the area, an idea was
created to split the panels into parts; when four people are underneath
one panel the panel would split into four sections. A start for this code
was made (a panel splitting in two sections when more than one person
was underneath it). It had not been worked out more because of the
complexity of the idea.
http://www.youtube.com/watch?v=LqdCKmohGOY&list
=PL825402CE33CC8B00&index=2&feature=plpp_video
84
85 | Visu
al Prog
ramm
ing
Programming interaction during STRP
Extra Layer Hands During STRP festival we saw that children always used their hands to interact with the panels. In our
interactive installation nothing happened when they moved their hands. That was the reason why we
decided to add an extra layer (extra blob detection area) to the interaction.
When you put your hand in the air there would appear a pulse above you on the panel; a growing el-
lipse with no fill and a white stroke. When there was more than one blob detected in the second blob
detection area, processing would draw a line (connection) between the positions of those blobs. It
would connect the position of blob 0 with blob 1, blob 1 with blob 2, etc. .
When testing this at STRP festival we noticed that this wasn’t visible enough so decided to not develop
this further. (12)
http://www.youtube.com/watch?v=y5EhWBHdNf0&list=
PL825402CE33CC8B00&index=3&feature=plpp_video
87
86 | Visual Programming
Programming interaction during STRP
Adjusted Mapping for walkingWe also noticed that because the interaction was on the
panels above you people didn’t notice that they actually
where causing a change in visuals. They were more perform-
ing for the people who were sitting in the lounge that they
actually noticed that they did something. Therefore we
wanted to add that something would happen not directly
above you but a few meters in front of you. Tijmen made a
program that calculated the speed in a certain direction and
moved the visualization accordingly to the direction of the
walking route.
http://www.youtube.com/watch?v=x8kG6yG7pUM&list
=PL825402CE33CC8B00&index=7&feature=plpp_video
89
88 | Visual Programming
Programming interaction during STRP
Sound for SaturdaynightWe all noticed that during the concert on Saturday night the
atmosphere was completely different than during the day. There
were at least 4 times as many people as normal and they were
there not only for all the art but especially for all the music. This
music was most of the time with a very loud bass also in the
area where we had our setup. Tijmen had the idea to make a
visualization on the panels that would react on the sound. The
program automatically detected the right sound level and cre-
ate content that was created dynamically on the sound. More
sound meant brighter colors and thicker lines, but it was namely
the difference in sound that measured.
http://www.youtube.com/watch?v=e0cWmtvPBOs&list=
PL825402CE33CC8B00&index=1&feature=plpp_video
7
Floor plan and Technical drawings
Max and Tom did not get full approval from STRP for our
panel-mapping that we used to make our physical model.
So we adapted a new version into the old model. The final
position of the panels was set up as a unit which had a clear
division with three hot spots. We used this division to create
three separate playfields for the interaction. In this way the
playfields were divided over the surface so we could sepa-
rate different interactions. Because we received an updated
floor plan from STRP we were able to implement this new
mapping into the 2D map of the location. After we reached
an agreement about the positioning we had to further
elaborate this mapping with technical data. We made a list
with all the supplies we needed for the realization of the in-
stallation. This list contained all the extension cords, Ethenet
cables, power cables, routers, switches, computers and
sensors that made the installation into a whole. With this
information we could edit the aforementioned map. First we
did this by sketching over the original map and after that we
digitalized after on to make it easily editable and enable it to
be sent to the other parties. For this technical drawing Max
used a program called Microsoft Visio 2010.
Production of the concept and Stakeholder agreements When you are working with tree major parties, a lot of ar-
ranging is needed to let things run smoothly. When the festi-
val approached there was still some missing equipment and
due to some miscommunication there were still some am-
biguities about the division of tasks between the different
parties. As soon as the panels from Philips were transported
from Berlin to Eindhoven they had to be delivered to STRP
as soon as possible since there was no space to store them.
This concerns four boxes with approximately the size of two
refrigerators and a weight of 200 kilos. Nor the University
nor STRP could store these boxes in such a way that they
would be safe. Philips had to store them until the first day of
assembly. For the assembly neither parties had time or staff
to spare to coordinate the construction. That is why Alex and
Tom got instructions and material from Philips to guide this
process. There was still some confusion about the exact sup-
plies. Vivian set up a list with what each stakeholder needed
to provide.
List of materials and services
Philips
Eight Luminous Textile Panels (120x250)
Material for hanging up the panels
Power- and Ethernet cables
Steel cables
Router and two switches
Workshop for mounting to two students
STRP
Pliers and clamps
Available room for a computer on the ground
Power supply
Different color carpet tiles
Possibility for testing the installation
TU/e
4 computers
Cables
2 students for the assembly
Because the installation would hang on a height of 3,5 me-
ters we had to work with a servicing platform. We checked
who was liable if something went wrong. This seemed a bit
more complicated than we thought. Since STRP said that
they could not be responsible for that we informed the
University. They referred us to the insurance agent of the
University. Finally, after a couple of e-mails it seemed that
event insurance would cost us a lot of money and still gave
us a high risk. We decided that the costs of the insurance
had not enough benefit over the amount of risk we would
take with the servicing platforms.
According to the agreement we would arrange four laptops
that could stay at the festival for ten days. Because there was
no budget available within the project we had to arrange
these computers within our own network. From Jeanette
Schoumacher, who manages promotional material of the
faculty of Industrial Design, we could borrow two laptops,
which were capable enough for what we wanted to do with
it. Another two laptops were provided by the UCE-research
group of our faculty.
It was essential for the remote control of the computers that
we had an iPad. We could borrow two iPads from ID educa-
tion to easily turn on, access and apply changes into the
computers while they were on a height of 4 meters.
We made a schedule for the festival so that there was always
at least one person present that could reboot the system
when something went wrong.
Production of the Festival | 93
Making a model of the festival areaOnce we found out the first specifications of the sensor
Tom started to place it in different ways into the available
space. He tried to cover the complete field underneath
the map of the panels so that every spot under the panels
could be used for the interaction. It appeared that the size
of the playfield could not fit within the mapping of the
panels without overlapping. Even not without a combina-
tion of normal Kinects and Kinects with an extra zoom lens.
Besides
the fact that the overlapping part would become a dif-
ficult programming problem, there was no money to buy
at least five Kinects. With this information Tom started to
build a physical model of the situation at STRP. In an earlier
stadium of the process we voted for the panel mapping
and he used this to create a miniature of the conditions.
94
95 | Prod
uctio
n o
f the Festival
http://www.youtube.com/watch?v=2UJfpmQqYcw&list
=PL825402CE33CC8B00&index=6&feature=plpp_video
Mounting the hardware
The tools we were given were clamps, cable and all the electronics.
Gloves were needed to ensure that we could handle the panels safely.
Putting the panels up had some complication because of the size and
the technique with which they were hung. More can be read in de indi-
vidual part of Alex Schepers
Electricity and networkTwo electrical groups were made to ensure that if all were turned on at
once, no overloading would take place:
1. The panels
2. The data network
The first group consisted only and entirely out of the LTPs.
The eight panels were separated again at two sockets. Because the
panels could be interconnected, meaning that the power cable from
one could continue to the next, we were able to make two groups of
four without a need for long cables. The Ethernet cables between the
panels however could not be linked and had to be fed to every panel on
its own.
The second group was a mixture of electronics:
• Three Laptops that were used to collect data from the Kinect and send
it to the visuals laptop
• A LAN-router
• Two LAN-switches
Apart from all parts in this group needing power, ethernet cables were
also needed to establish a network.
Every laptop was connected through a power source which we could
individually power on or off. This meant that if a laptop would for some
reason fail, the rest of the system could still continue without a problem.
Should one of the groups have failed, the other would simply wait for
the other group to come back online.
97
96 | Production of the Festival
8
During the STRP festival our concept is exhibited at the Light
Café. This is the perfect opportunity to collect the opinions
and reactions of the visitors and participators on the interac-
tive play.
The festival will be visited by art lovers, creators, designers,
students, creative professionals, music fanatics, and many
others who are interested in art and technology.
The settingOur ceiling concept was positioned above a lounge interior.
The play field was in the centre of the lounge. The rectangu-
lar shaped area was circumcised by half room dividers, which
created lounge booths at the sides of the play field. The play
area was crossed by all those who were looking for a place
to sit down, met friends or were on their way to the next
exhibit.
DiversityThe program of STRP festival is very divers. During day time
the exhibition is the main activity, together with workshops,
readings and lectures. Throughout the week also several
schools will visit the festival, so in the mornings the overall
audience will exist of ten year olds. During the weekends the
festival is also opened during the evenings. The planning
states performances of great music artists and DJs and these
will attract a lot of festival visitors. Because the Light Café
is opened during all activities, we can be assured of a large
variety in visitors for our installation.
STRP festival
100
Production of the Festival | 101
Research and developmentThroughout the festival we changed and
updated the program for the interaction and
visualization. The presence and behavior of
visitors gave inspiration for improvements and
adjustments.
For research purposes we conducted some in-
terviews and did observations on the behavior
of the visitors at our installation. All interviews
and observations were acted out during the
finalized version of the program during the
last weekend of the festival. The functionality is
explained at page… of this report. During the
research we sat at the lounge on the side of
the play field and took notes on our IPad.
Data gatheringTo gather information for concept improvements, research
analysis and reflections we documented our observations
during our stay at the festival. Below are some results we
got from interviews and live observations. The observations
were conducted according to the categories proposed in
the outdoor play observation scheme for evaluating head-
up play (ref ).
102 | Production of the Festival Production of the Festival | 103
There appears to be a relation between the level of physical
activity and the understanding of the interaction. The more
movements are tried, the more functionality is discovered.
Most physical activity is shown in trying out the functional-
ity of the system. Repetition of the movements appears to
create awareness on how the system works, because the
movements then show the same effect. The behavior of
most visitors is actually limited to the standard behaviors
which are appropriate in the setting.
Intensive physical activityWhat stands out is the fact that only children are show-
ing intense physical activity. When the supervisors of the
school groups have given an explanation about the panels
and interaction, the children were given the opportunity to
explore the interaction for themselves. There are always a
couple of children waving and jumping below the panels.
This was one of the reasons that we added a second layer
which could detect this activity.
Non-intensive physical activityThis applies for almost all the visitors. They are walking in the
lounge unsuspectingly that they were causing an interac-
tion. This interaction, moving lines and changing colors, was
noticeable for the audience that enjoyed the sitting area of
the lounge.
But once they saw someone causing this interaction they
were more likely to copy the behavior. In the behavior of the
visitors it was remarkable that they started to wave above
their heads once they recognized that the system was inter-
active. They expected that the panels would react on this
movement, which did not happen in the beginning of the
festival. Later on the programmers adjusted the code so the
hands could also be used in the interaction.
No physical activityIf someone is standing still below the panels, a colored
circle appears above his or her head but mostly this person
doesn’t notice.(13)
ObservationsPhysical activity
FocusObservations 105
104 | Production of the Festival
There seems to be a big difference in focus between the
visitors who are walking underneath the installation and the
ones who are sitting down. Most people who are walking
under the panels only look up once, continue walking and
leave the play field. While sitting visitors look at the panels
as well as at other players and take more time examining the
surroundings. This might have to do with the fact that the
walking people have something else on their mind at that
moment.
Looking at other playersBoth adults and children are looking at the movements
made by other visitors. Movements of others that have a
new interaction as result were copied by bystanders. When
there are quite a lot of people this creates a certain conti-
nuity between watching and performing. People who are
watching want to try the interaction. With that they are the
new performers and provoke a new performers for other
people in the lounge. People who understand the interac-
tion can explain it to somebody else but they do not make it
more clear. We think that the most people were too embar-
rassed to take the initiative to make “shameful movements”,
but when we as experts of the interaction showed was pos-
sible by some extremer moves people were more likely to
step out of their comfort zone. In this way they see that they
can create something together.
Looking at game objectsUnder the panels there are hardly any objects. Only some
Fat Boy poufs pushed into the playfield. These poufs do not
have any influence on the interaction because the Kinect
only detects blobs above the shoulder. We think that this
is something that the visitors understand because they did
not try to get an interaction with any of the above named
objects.
Looking at something out of sight, possibly part of the gameWhen the visitors walk below the panel and like the thing
that they create, they often look at other visitors for confir-
mation.
ObservationsSocial Interaction
106 | Production of the Festival Production of the Festival | 107
As expected there was a lot of social interaction within the
setting of a lounge. We focused on the social interaction
which seemed to be related to the installation.
Notable was that standing people in the play field got
instructions from other visitors sitting on the side line, which
seems to origin from the fact that the sitting ones have
a better overview of the situation. And the level of social
interaction seems to decrease when the number of visitors
increases. It might be that people feel embarrassed to share
their findings or incomprehension when they are surround-
ed by a lot of people they don’t know.
Functional, with another playerThe special movements leading to erasing or coloring
require more explanation and work better individually than
with more people in the playing area. These movements are
not movements you make in daily life which is quite funny
in itself. People make these movements often unconsciously
by looking and pointing at something. Also when people
are toasting, shaking hands or greeting by giving three
kisses something happens. The same effect is reached when
somebody with a large waste is passing underneath the
sensor.
Non-functional positive/neutral, with another playerPlayers often proudly shout to other visitors what they have
discovered. They also repeated their discovery.
Unintended physical contactBecause people are looking up very often they sometimes
bump into another person, or accidently hit someone when
spreading their arms.
The differences in physical activity, focus and social interac-
tion clearly defined two different roles in our play field; the
one of active player, and the one of spectator.
9
FRENSFRIJNS
“In whatever way you look at it, the thing that makes STRP special are still the art pieces. And with 31000 visitors we surely can’t grumble.”
To find out more about the overall impression of the stakeholders about the pro-ject, we interviewed Frens Frijns (STRP Festival) and Floris Provoost (Philips) .
What did you like about the STRP festival? “I was very happy with the number of visitors and
the feedback of the artist world. I was also very
glad to see there was an increase in the number of
in art-viewers, symposium-visitors and all the other
visitors which came during the day. In whatever
way you look at it, the thing that makes STRP spe-
cial are still the art pieces. And with 31000 visitors
we surely can’t grumble.”
How did this project come about?“I came in touch with Floris Provoost for the panels
and I was already talking to Ben Schouten at the
time. Floris preferred artists to use his panels,
mostly regarding an interaction with the panels.
Because I thought the link with the TU/e was very
interesting, especially within the playful interac-
tions part, I was the entrepreneur as a manner of
speaking.”
Did we measure up to your ex-pectations/requirements?“Your determination during the festival was super.
It was really interesting to see the cooperation
between you and Philips and how that showed
itself at the STRP festival. The way you were at work
ensured me that it would all end well. To be honest
there was some chaos and some things that were
unclear at the beginning, but it turned out great at
the finish. “
What did you think of the final interaction at STRP festival with respect to your expecta-tions?“The interaction was quite underdeveloped in
the first weekend, but I saw a distinct improve-
ment towards the second weekend. De interaction
definitely increased, you could see a lot of people
starting to interact with it more. The continu-
ous variation was at least as important as how it
looked. I thought it fun to see how people didn’t
always notice what was happening above them.
That is much more fun than when it is all crystal
clear. That is the exact reason why I was of the
opinion that your project was far more interest-
ing than the other Philips project within the same
room. At this kind of festival it is very important
that the image is much more powerful and that
had happened a lot more during the second week.”
What was your general impres-sion of the collaboration be-tween all the parties (Philips, STRP, TUE) and everyone’s share?“I can’t really evaluate that. My cooperation with
Philips was limited to delivering the items neces-
sary. I was able to make some nice agreements
and that’s it. The miscommunications, as you have
stated, were not very clear on our side. You could
notice that you were not familiar with this kind
of situations and certainly not on such a location.
Overall it was very nice to work with you, every-
thing was agreeable. “
110
111 | Interview
s
What did you like about the STRP festival?“I really liked the festival this year. Probably because I’ve
been more involved in the construction of one of the art-
works. It was very versatile in the way that they used video, a
lot of good music and there were a lot of good media artists.
I thought that the line-up was of high quality and was also
very inspiring. Normally you have 4 or 5 stars and the rest
are pretty much in the middle but at this festival I found
everything of quite an appealing level of quality. “
How did this project come about?“Ad van der Meulen, someone here at Philips, wanted a
collaboration with STRP. He wanted to combine certain
research to exhibit something at the festival. Ad linked me
to Frens Frijns who sent me to the artist/designer Edwin van
der Heijden. Suddenly Ben Schouten came up with a group
of students who were crazy enough to work with Philips.
Finally Ben Schouten presented you to me as my experi-
ment team. The idea already ran for a while. At one point we
supported the idea of Edwin van der Heiden and then your
project came in and I found those two projects so interest-
ing that I wanted to support them both. In your project I had
a little more say. The thing I liked about your project was the
interactive part. For me, playful interactions has always been
something I’ve been involved with through previous col-
laboration with Ben Schouten and Rombout Frieling so I had
enough experience with it. Your group was well suited for
an experiment to expand our knowledge around a product
like your group used. This also gave you a nice opportunity
that you normally don’t get. On the other hand I will receive
feedback that I also normally would not get.
Of course I had some expectations with this project. I want-
ed to have a report with a piece of research I could link to
experiments, play and interaction. In general I find this pro-
ject very successful especially through the insights we got in
interaction. It was our first step into unfamiliar territory and
now I want to make some next steps with that. I definitely
see opportunities for further research to extend to personal
development and perceptions of a space combined with
animation and play. There are endless possibilities of such
products on which, together with the university, research
can be done. “
What did you learn from working together with different parties?“I know that you spilled a part of your valuable time on
figuring out how to deal with some technical issues, but this
certainly led to solutions. Finally, you made a one year dur-
ing project worthless in just two weeks. “
What did you think of the final inter-action at STRP festival with respect to your expectations?“To my mind the interaction was limited. People reacted to
the system but just not enough. It did not matter for the in-
teraction if you noticed your presence or not. Besides it was
too chaotic to be clearly recognizable. People understood
that the system was reacting to their presence but to me
they did not feel in control of the system. I would have used
less content. But it was certainly beautiful, decorative, fun
and it was what I wanted, also qualitatively. The interaction
was just too scantily developed. How I see it, there is too lit-
tle documentation about interaction. “
FLORIS PROVOOST 113 | In
terviews
What was your general impression of the collaboration between all the parties (Philips, STRP, TUE) and everyone’s share?“In the first place I thought it was a little bit chaotic. There
were seven students around the table, each with a dif-
ferent objective. You are people with all different per-
sonalities and goals within the project. For the long term
I noticed that team division was not evaluating equally.
The way I saw it I thought that not everybody was equally
involved. A reason for this could be that I did not see
everybody at the HTC when there was another meeting.
Secondly, I found that in the brainstorm phase there was
too little communication about the ideas that were cre-
ated, partly due to the lack of time. Some ideas may now
have slipped through the net. Generally I am as excited
about what eventually came out as the other people in-
volved, and this is definitely a reason to keep doing things
like this. “
What would you do differently next time in relation to the collabo-ration? (pro’s/con’s)?“I thought that there was too little time to properly work
out and test the concepts. You could partly blame that on
the panels that came available in a too late stadium of the
developments. Otherwise we could do more brainstorms
to gather more valuable information. If we had more time
we could have used a better, more systematical approach.
Therefore we had quite a slow start because it took a
while to create some clarity about the requirements. We
knew what making an interaction on the ceiling was dif-
ficult but we could have paid more attention to the differ-
ent options. As to the technical programming we should
have taken a better look at what already exists, instead of
diving into those problems. The communication was also
mainly about technical problems while more attention
might have been spent on the evaluation of the idea. “
115
115 | Interview
s
01
In this project we have developed a playful interaction with
Philips Luminous Textile Panels. This light-installation was
exhibited at the STRP-festival 2011.
We have established a feeling of curiosity for interacting
with our installation through vague visuals reacting inexpli-
cably on the presence and movement of visitors. We gave
meaning to a static light medium by enhancing the feeling
of connectedness between the people and the medium in
their environment.
Conclusions
Conclusion and Recommendations | 119
The provided interactive installation was not a game, but it
was more a medium to let people freely explore interactiv-
ity. There was not more to that than exploring, so people did
not make up any games. They would only explain to others
their knowledge about the interaction. No one fully under-
stood the interaction, but this gave the whole installation a
magical feeling, and gave the visitors the curiosity to discov-
er. We were able to trigger others’ interest in our installation
by performing ourselves, because we noticed people would
copy the behavior and movements of others.
Against our expectations, this relatively simple interactive
installation was still interesting enough for the festival-visi-
tors. We did not foresee the big role of the spectator. Since
the spectators were more aware of the interactivity and the
change the performers made to the content, which gave
them an important role in initiating the explorative interac-
tions.
Despite the limitations of the medium, being blurry, 2-di-
mensional, hanging on the ceiling, we came to a suitable
interaction.
Future perspective
The impact of the performance depended on the number of
people within the playground. The more people, the more
dynamic the visuals would get. This would also influence the
clearness of who was responsible for the change. The num-
ber of people on the playground for the optimal effect was
rather low; between 3 and 7 persons. In future perspectives,
the system should be more flexible in relating the interactiv-
ity to the amount of people.
The playground was not clear enough, since it is difficult to
keep an overview of the situation on the floor and the ceil-
ing at the same time. The playground was situated exactly
in-between the panels, which resulted in no direct action
above you (since there was no panel directly above you).
People would try to influence the visuals on the panels by
standing directly underneath them, but there was no play-
ground directly underneath the panels. It was interesting to
notice the system directed the movement of people in such
a way. Although for creating awareness in the interaction
the mapping should match the direct location.
Recommendations on multiple clients
Since we were not involved in the foundation of this project,
it was not clear to us which arrangements were made and
more importantly if there was something lacking in the
agreement. During a first meeting with STRP we got their in-
formation on the project and during a meeting with Philips
we heard their part. But we never had a group meeting, with
all parties involved. This resulted for us in unclearness about
regulations in responsibilities and arrangements. For future
projects it would be preferred to have all regulations down
in black and white.
Our planning and internal task divisions were not communi-
cated to the different parties. Floris (PHILIPS) mentioned in
his interview that he would have liked to be more involved
in our process, since he was especially interested in our
ideas for the interaction with the panels. Now he had only
seen our end-result, and some test-results. We were not
aware of his expectations of the project. A next time we
would have to ask our clients to what extent they would like
to be involved in the process.
Also we communicated with the clients through one con-
tact person, Vivian for STRP and Tijmen for Philips. Especially
Floris from Philips mentioned in his last interview that this
gave him the impression of some people not being involved
as much in the process as others. He would have liked to see
more from everyone.
Recommendations
Conclusion and Recommendations | 121
Recommendations on working in a team which consist of students from all years
Working together in a group of students from all years was
very interesting, and very challenging at the same time.
The first years did not have a clue how to run a project and
which steps are required in a design process. Working in
a team with senior students worked out quite well. You
have to make sure though that the senior students do not
become the clients of the first years. Not all steps should be
executed as a group, since the first years have to find their
way in the education and acquire a proactive attitude. While
the senior students should be able to go through their own,
individual, process as well.
For the more experienced students working in a group is
especially helpful when in the need of motivation. It was
useful for them to validate the knowledge they gained over
the years. By explaining the process and argumentation for
different decisions the senior students realized how much
they actually learned over the years. In this educational
model it is difficult to point out what you learned since there
are no exams or books required.
For Tijmen, a B3.2 student, it was very useful to see what stu-
dents do in the masters. Because you get a better overview
of the education through working together with master
students than through going to information sessions. We
think it will attract more students to our masters if they all
get a glimpse like Tijmen of what is expected and done in
this final part of the study.
11
124 | Conclusions and recomandations Conclusions and recomandations | 125
124 125
124 | Co
nclu
sion
s and
recom
and
ation
s
125 | Co
nclu
sion
s and
recom
and
ation
s
We are grateful that this project ended in such a success, this would not have been possible without the help of all the people in-volved in this project. Therefore our word of thanks goes to the STRP festival, Frens Frijns, Dorien van den Hurk, Jan-Bart van der Tuuk and Marnix de Nijs, who made it possible for us to showcase our work at this fantastic festival; to the Philips Large Luminous Surfaces team who lend us their fantastic panels; to Floris Provoost for providing the panels and his help with technical implications; to Ben Schouten for making this project possible and the guidance he found time to give us; to Frans Parthesius who guided us through the process and the process of communication between all the different parties; and Bram van der Sanden, Wouter van der Heide, and Thomas Diewald for their help in programming. All these people were indispensible for this project to be a success!
“ “
Acknowledgements
12
References1. Playful Interactions. Technische Universiteit Eindhoven: Intranet. [Online] 2011. http://w3.id.tue.nl/fileadmin/id/Educa-
tion_Documentation/Project_descriptions/1112-S1-Playful_Interactions-Longs.pdf.
2. About STRP. About STPR, context, STRP festival. [Online] 2011. http://strp.nl/en/context/about-strp/ .
3. Bring spaces alive. Bring spaces alive. [Online] 2011. http://www.lighting.philips.com/pwc_li/main/application_areas/as-
sets/luminous-textile/product_leaflet.pdf.
4. Foundation ID. Technische Universiteit Eindhoven: Foundation. [Online] http://w3.id.tue.nl/nl/education/foundation/ .
5. Current Themes and their Spaces. Technische Universiteit Eindhoven: Current Themes & their Spaces. [Online] http://
w3.id.tue.nl/nl/education/foundation/themes/current_themes_their_spaces/.
6. Xbox 360 manuals and specifications. Xbox 360 manuals | Xbox 360 Spec. [Online] http://support.xbox.com/en-US/xbox-
360/manuals-specs/manual-specs.
7. Processing Library – Computer Vision – diewald_CV_kit. http://thomasdiewald.com/blog/?p=1107. [Online] September 6,
2011. http://thomasdiewald.com/blog/?p=1107.
8. UDP (User Datagram Protocol). What is UDP (User Datagram Protocol)? [Online] October 2000. http://searchsoa.techtar-
get.com/definition/UDP .
9. A implementation of the OSC protocol for processing. oscP5. [Online] December 19, 2011 . http://www.sojamo.de/librar-
ies/oscP5/index.html.
10. ArrayList. ArrayList. [Online] August 30, 2010. http://www.processing.org/reference/ArrayList.html.
11. What Is a Class? What Is a Class? [Online] http://docs.oracle.com/javase/tutorial/java/concepts/class.html .
12. Processing Layers. Nootropic design. [Online] April 22, 2011 . http://nootropicdesign.com/processing-layers/.
13. Saskia Bakker, Panos Markopoulos, Yvonne de Kort. OPOS: An Observation Scheme for Evaluating. NordiCHI 2008. Octo-
ber 20-22, 2008, p. 3.
14. Blob Detection . Blob Detection - The Lab Book. [Online] http://www.labbookpages.co.uk/software/imgProc/blobDetec-
tion.html.
129 | Referen
ces
13
http://dl.dropbox.com/u/4067465/DATA_
PERMANENT/Receving_data_visuals.zip
http://dl.dropbox.com/u/4067465/DATA_
PERMANENT/Uitgaven.project.xls
http://dl.dropbox.com/u/4067465/DATA_
PERMANENT/Sketchup.zip
http://dl.dropbox.com/u/4067465/DATA_
PERMANENT/Sending_data.zip
Receving DataProcessing
Spendings Project Sketchup
Sending DataProcessing
Scenario’s
http://dl.dropbox.com/u/4067465/DATA_
PERMANENT/scenario%27s.html
Final SLPLLP Movie
http://youtu.be/wRSV0vU4VQc
top related