our process book
DESCRIPTION
This books is a collection of texts and images that emerged from the Independent Study Group working with the GSpeak System during Wintersession 2011.TRANSCRIPT
OUR PROCESS BOOK
UNFINISHED
Olivia Verdugo, Micah Barret, Catherine Siller, Camila Afanador, Keitaro Sakamoto, Niu MiaoGSpeak Wintersession Group 2011
OUR PROCESS BOOK
This books is a collection of texts and im-ages that emerged from the Independent Study Group working with the GSpeak System during Wintersession 2011.
GSPEAK WINTERSESSION GROUP RISD 2011
STUDENTS Olivia Verdugo, Micah Barret, Catherine Siller, Camila Afanador, Keitaro Sakamoto, Niu Miao.
INSTRUCTORSLucinda Hitchcock and Jamie Jewett.
PROCESSING WORKSHOP AND HELP: Clement Valla
Special thanks to Lane Myer for being part of our process.and to Oblong Industries for allowing us to use it and discover the potentials of the system.
33 DAYSFIRST MEETING
JAN 6 /2011-
FINAL SHOW
FEB 7 /2011
WEEK 1 & 2January 3 - 11 / 2011Processing Workshop by Clement Valla
06/01/2011
7
During 4 days be had an intensive Processing Workshop with Clement Valla in the GSpeak room.
8
void setup() { size(500,500); stroke(200); background(255);} void draw() { //background(255); fill(random(255),random(255),random(255)); noStroke(); ellipse(mouseX+random(15),mouseY+random(15),20,20); }
First sketch. By Keitaro.
Selected sketches produced during the Workshop
int xpos = 0;int ypos = 0;
void setup(){ size(400, 400); background(255); frameRate(10);} void draw(){for (int j=0; j<400; j = j+10) { ypos = j;for (int i=0; i<400; i= i+50) { xpos = i;
strokeWeight(2); stroke(0,0,0); fill(random(255),random(155),random(255), 85); rect(xpos,ypos,50,50); fill(random(155),random(155),random(255), 65); rect(xpos,ypos,20,20); fill(255,255,255,50); rect(xpos,ypos,10,10);
} }}
Loop. By Camila
9
First sketch. By Micah.
void setup(){ size(900, 500); background(77, 29, 29); smooth();}void draw(){ background(map(mouseY, 0, 500, 0, 100), 29, 29); stroke(255); strokeWeight(random(0,10)); line(0, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(100, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(200, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(300, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(400, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(500, 0, mouseX, mouseY);
stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10));
line(600, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(700, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(800, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(900, 0, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(0, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(100, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(200, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(300, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(400, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(500, 500, mouseX, mouseY);
10
void setup() { size(700,700); stroke(255); background(53,165,145);} void draw() { //background(53,165,145); strokeWeight(mouseX/25); stroke(mouseX/3,mouseY/3,mouseX/3,mouseY/3); fill(random(200,255),mouseY/3,mouseX/3,mouseX/3); triangle(mouseX, mous-eY, width-mouseX, (mouseX+1)/(mouseY+1),height-mouseY,mouseY/2);}
int xpos = 0;int ypos = 0; void setup(){ size(400, 400);// background(0);} void draw() { background(mouseX,mouseY,0); for(int j=0; j<=400; j = j+50) { for (int i=0; i<=450; i = i+30) { //i starts at 0, goes to 40, increases by 1 every time //starting condition, exit condition, incriment xpos=i; ypos=j; noStroke(); fill(255,0,155); rect(xpos,ypos,10,20); stroke(255);// fill(mouseX,mouseX,mouseY); fill(random(255),random(255),random(255)); ellipse(xpos-40, ypos, 10, 30); fill(255); ellipse(xpos+40, ypos, 10, 10); stroke(xpos,xpos,xpos); line(xpos-10, ypos+15, xpos+20, ypos+35); } }}
Loop. By Catherine. First sketch. By Olivia.
07/01/2011
11
int j;int x=20;int y=30;int w=40;int h=40; void setup(){ size(400,400); // shapes = new Shape[i]; background(255);} void draw(){ stroke(random(0,255),random(0,255),random(0,255),random(0,255)); for(int i=0; i<100; i++){ line(i,i+j,mouseX,mouseY); j++; } }
First sketch. By Niu.
First program for GSpeak. Basic drawing tool activated with the movement of the hand. By Camila
First program for GSpeak. By Keitaro
12
WEEK 3January 17 - 23 / 2011First Group Discussions and Work Sessions
07/01/2011
13
lucy:
ideas to think about: art, history, music, science, movies, games etc? what interests you?
system is performative, is it possible to turn the “performers” around such that we be able to engage with our audience.
even if our back is to our audience, how can we consider that audience?
framing and composition, how we make these connections. visualizing these ele-ments.
jamie:
choreography: move from a full body point of view, and then see how gesture emerges from that, or, see how whole-body choreography could emerge from specific gestural vocabulary
necessarily working with local space: what happens if we use up and down or back and fourth in a more extreme way? we tend not to use the volume of the space in its entirety when we use the system
what’s happening on stage that we may direct the audience’s attention to? some-thing we think about in any performative medium. thinking about spacial relation-ships not only between the performer and the screen, but also the audience, the demarcated space of the system etc.
where is the space for the person to talk? where is the space for the technology to talk?
camila:
primary interests: drawing tools as a starting point, discussion on the drawing experience, history, collaboration using g-speak. Documenting the whole experience.
keitaro:
so far, he has been able to connect his pro-gram with g-speak, if possible, he’d like to use type in an experiment. one interest is in data visualization using type/numbers, another possible interest is in using the whole field in a kinetic manner (tracking movement). a third interest might be using “real” imagery: building, stars, nature etc. to do something.
primary interests: type design & typogra-phy, gaming
catherine:
very interested in tracking movement (dance) and potentially attaching sensors to things other than hands. attaching sensors to things other than hands may be difficult, however, as sensors are reading the spatial relationships between tags, ergo it we put the tags somewhere else, it would not necessarily mean anything to the computer. but, maybe we should just try it?
if we place tags on parts of our body, would it be possible to contort into the x,y rela-tionships necessary to trigger the system?
approach from two angles, one, movement without worrying about what’s being made on the screen, two, the reverse: how to get a very specific result on the screen, what movements would be required? also inter-ested in the incorporation of language as well (written). perhaps a collaboration with some writers to “perform” a text? we must consider the 3d component of the system when we are creating.
how might something created in this environment stand up outside of this environment?
January 18th, 2011: G-Speak meeting notesNotes by Olivia Verdugo(please excuse typos, spelling mistakes and incomplete sentences! also, auto-correct kept changing G-speak to “speak” sorry)
14
what is the work beyond this room? how could it possibly exist in the worked at large?
primary interests: movement, language
evelyn:
haven’t thought about g-speak too much. traditionally before coming here very interested in thinking about tools for art-ists (digital tools for children and artists) very interested in the idea of collaborative tools. would like to develop as an artist as well as a tool maker. drawing tools would be something of interest, particularly if there could be a collaborative aspect involving more than one person at a time. taking how we work in the studio and ap-plying that here. thinks she should be able to spend a bit more time here now that the intense part of drawing marathon is over.
lucy:
should there be both a pragmatic project, and a poetic one? again, the interest in spatially exploring typography.
micah:
interested in how the impute devices affect how we interact with the content of the computer and think about the screen in general. for example, the new trackpad + hopw it has affected the way undergrads draw using the trackpad
how does the technology we use affect the very way we create/what we create
how would using both parts / both of our limbs to control content creation affect US
as with ipods/ipads, the content is shaped differently + our reading habits are changed
get away from making specific tools to-wards exploring the design of relationships and how it affects us as users of that tool
pitcher that can pitch both right and left handed
have made so far: exploring the practical technical side of things
primary interests: music, food, sports
lucy:
i know that you have a tendency for the
practical + like to frame things as they are
desire to be pushed into a less familiar place?
jamie:
defamiliarizing our physicality allows us to be more open/ expand what we do
o: (will add)
lucy:
when we talk about what g-speak was first designed for, would you say it was basically designed to be a bigger projection that we can get on our laptops
jamie:
the interface, not merely the projection
opec: used in a huge room to parse huge amts. of data pertaining to the mineral content of the earth
micah:
g-speak is more of a one-to-one relation-ship, quicker navigation etc.
non-linear, and spatial way to visualize/traverse information
information is not “chopped up” the same way (the power pt problem)
immediacy of information availability
lucy:
what are the kinds of artwork that get generated out of processing language
always look similar, time to move on?
o: (will add)
lucy:
what is drawing? old school thinker explains drawing, what is the limit that makes drawing not drawing anymore
what is a mark?
why is mark making so essential?
jamie:
from an art-making pt of view: one-to-one interactivity: drawing
what is the interactive paradigm that we are working with?
how may we define interactivity?
citrus bohn (sp?) “a system is only interac-tive if the user changes the system, and
07/01/2011
15
the system changes the user”
what if there are variables and diff func-tions that the computer responds to and then we resound to that? interface loop?
critique about non-understanding of interacting in dm
could the system be a part of the dialogue?
camila:
the potentials of creating, accidental creation, chance, not-knowing what the computer is going to do next?
lucy:
puppets on gloves? scrim/wyang kulit type shadow theater?
catherine:
julie taymor(sp?)
lucy:
shadows: camille utterback
catherine:
we want to explore something with speak that is more that you can just do with your computer with your mouse.
jamie:
3d
gestural
collaborative
micah:
but, we could create a drawing app on g-speak that does what you could never do on a computer
lucy:
would we be comfortable with being very open/not having a totally “finished” program/product at the end? perhaps a proposal?
jamie:
but, the “this is what i made” part is very important. without making, being here (in the g-speak space) is irrelevant
lane:
i have a huge list!
interesting things: thinking from purely a sculptural standpoint: space, light, simple investigations dealing with the fact that
there is a shadow and the machine is reading some response to the hands
hand:
puppet
shadow
tools
transcribing information vs navigating information
picking up the relationships of what your hand is doing as transcribing information
deafness and the way that a blind person exploders an object with their hands and how that-kind of tactile exploration might influence the way we create form
children’s games?
using mistakes: being able to uniformly obstruct information
mirrors
the fact that the machine creates scenery
assistance from others while “perform-ing”? (so, a kind of participation?)
andy warhol and his placing the camera on the doorknob, or just observing very basic things
taking the system in and out of the room,: how things begin in a salon or gallery be-fore they are able to move somewhere else
the extension of a projector - light film etc. without thinking about interface
“the one-eyed projector god”
art and design: history about what hap-pens when something is used “wrong”
mapping the inner-self: what would the battle-ground for one’s body be? what would happen if we could organize the 3d space that was self?
melding of household and self in an exter-nalized way
film before film? how people attempted to breathe life into an inanimate object before film
some connection btw watching slow tran-sitions between things that cause cracks in the way we see/understand information
drawing in the 3rd dimension: key things
16
to think about: what’s so cool about the word drawing is that it doesn’t exclude the notion of drawn (pulled out from) once you understand that notion of draw-ing you know that you are pulling it out and then shoving it forward
(vs rendering)
to draw and then project
extrusion
drawing as an activity
the translation between what we see in our open minds and the tapered tools with which we render those visions
(love that idea)
what does it take to get someone emotion-ally invested?
violation of security of understanding
evelyn:
PUSh on the system to see what it can do! it will push back
try to understand the parameters within which we are working
catherine:
do not undervalue the tools that we do have
lucy:
Richter Floating rectangles to sound?
work within our limits and do not be nega-tive about our simple gestures
lane:
is there a bulletin board to post just the simples thing that someone could create?
a way to just see what we’ve discovered
lucy:
ugop(sp?): flash pro guy
look at what he is doing
o: (will add)
lucy:
if we could take everything everyone has tossed out…
niu:
plays the accordion!
building sketches?
moving skyscrapers in manhattan
Q:
designing a typeface. speak could be a very interesting tool to explore the creation of a typeface through gesture
perhaps a script face? but very intuitive
creating a system to design a typeface?
micah:
body language - could the system respond to shifts in our body language?
lucy:
how could we move to the next level of creating our projects now?
catherine:
maybe we could start to mock things up?
could what we’re doing be BOTH practical and experimental?
lucy:
type, drawing, language, body & computer interface
what part of this can we really do in 4 weeks?
go have lunch together and mull over this question
maybe:
half of us: purely practical design?
half of us: purely exploratory exploring the connection btw hand gesture and image?
catherine:
first have a tool, then repurpose that tool for something experimental rather than just trying to make something impractical
micah:
experiential vs practical?
lane:
responsive system transcribes and then allows us to be responsive to what it transcribes
no matter what we do, it may be inter-preted into something we can interact with in a physical way
from general to specific
going from the self
11/01/2011
17
kind of like a three-dimensional tape recorder?
lucy:
the magic of what your hand can do is a bit less magical unless you kind of pull whatever is truly unique and surprising out of the system.
what can we do with this system/with our hands that is NOT just mouse like
how can we name a direction that will bring the amazingness of this experiential space forward?
lane:
can we storyboard the circumstances that cause what the dream is?
abstract thinking/feeling
next step: get specific, get visual and demonstrative
how can we get it out of a specific laptop into a communal space?
lucy:
independent study: urban curators?
using found gold frames draw attention to various things around town
this is different as we each/all want to be the architect of the programming and the recipient of the ah-ha
start with something that is a GIVEN is helpful, a dot, a line, a word, a poem, a prop?
if we could start with a thing we wanted to solve visually/expressively that might be able to start things moving/going
niu:
performace
game
two terminal directions
in the middle space there is a lot we may explore
lucy:
could we say that our primary goal is to use g-speak in a group situation?
two groups feels more manageable in terms of having some voice
our personal interests could be explored after wintersession
(now we are looking at our sketches/previ-ous work)
lucy:
specificity of hand gesture should be what we are working with
the limited from the limitedness
or vice versa
nancy dwyer - three dimensional typogra-phy um… extruded
how do we see type? when does type be-come something else? the portent of what is to be said?
lane:
orthographic drawing
hidden potentiality? mutations
catherine:
perhaps videotaping our hands close-up and projecting on another wall our hand gestures
18
The system has performative qualities, is it possible to turn the "performers" around and engage with an audience. How can we consider that audience?
∞∞∞
We are necessarily working with local space: what happens if we use up and down or back and fourth in a more extreme way?
∞∞∞
Think of spatial relationships, not only between the performer and the screen, but also with the audience. Where is the space for the person to talk? Where is the space for the technology to talk?
∞∞∞
Programming drawing tools as a starting point and discussion on the drawing experience. Documenting the whole experience as part of the learn-ing process.
∞∞∞
We could use type in experimental ways and work with data visualization.
∞∞∞
group discussion - january 18th, 2011Edited version (Original notes by Olvia Verdugo)
Approach from two angles: movement without worrying about what’s being made on the screen, and how to get a very specific result on the screen, what movements would be required? How might something created in this environment stand up outside of this environment? What is the work beyond this room?
∞∞∞
Exploring the idea of constructing collaborative tools. Development as artists as well as a tool-makers.
∞∞∞
Interests in how the input devices affect the way we interact with the content of the computer and think about the screen in general.
∞∞∞
How does the technology we use af-fect the very way we create, what we create? Get away from making specific tools towards exploring the design of relationships and how it affects us as users of that tool.
∞∞∞
Defamiliarizing from our physicality
18/01/2011
19
just do with your computer with your mouse.
∞∞∞
But, we could create a drawing appli-cation on GSpeak that does what you could never do on a computer.
∞∞∞
Explore the idea of transcribing infor-mation vs. navigating information.
Picking up the relationships of what your hand is doing as transcribing in-formation. Using mistakes: being able to uniformly obstruct information
the fact that the machine creates scenery.
∞∞∞
“The one-eyed projector God”
∞∞∞
Drawing in the 3rd dimension.
∞∞∞
Translation between what we see in our open minds and the tapered tools with which we render those visions.
∞∞∞
Push on the system to see what it can do! It will push back. Try to understand the parameters within which we are working.
∞∞∞
allows us to be more open and expand what we do.
∞∞∞
When we talk about what G-Speak was first designed for, would you say it was basically designed to be a big-ger projection that we can get on our laptops? The interface is not merely the projection. G-Speak is more than a one-to-one relationship; it allows quicker navigation and a non-linear, and spatial way to visualize/traverse information.
∞∞∞
What is the interactive paradigm that we are working with? How may we define interactivity?
Could the system be a part of the dialogue?
∞∞∞
What are the potentials of creating with code and not knowing what the computer is going to do next?
∞∞∞
We want to explore something with GSpeak that is more that you can
“A system is only interactive if the user changes the system, and the system changes the user”.
20
Designing a typeface? GSpeak could be a very interesting tool to explore the creation of a typeface through gesture. Perhaps a script face? but very intui-tive...
∞∞∞
Body language. Could the system re-spond to shifts in our body language?
∞∞∞
Could what we’re doing be both practi-cal and experimental?
∞∞∞
Exploring the connection between hand gesture and image?
Experiential vs practical?
∞∞∞
Responsive system transcribes and then allows us to be responsive to what it transcribes.
∞∞∞
What can we do with this system/with our hands that is NOT just mouse like
∞∞∞
Get specific, get visual and demonstra-tive
∞∞∞
How can we get it out of a specific laptop into a communal space?
Start with something that is a GIVEN is helpful, a dot, a line, a word, a poem, a prop?
∞∞∞
Could we say that our primary goal is to use g-speak in a group situation?
∞∞∞
Specificity of hand gesture should be what we are working with
perhaps videotaping our hands close-up and projecting on another wall our hand gestures
drawing applications, hand gestures, start-ing point, participa-tory drawing tools, hand gestures, con-struction of an experi-ence, group project, movement with visual responses, influence of movement, par-ticipation, languages, typography, stage, face an audience, body changes, perfor-mance, spatial rela-tionship, framing and composing
the input device affects how we ex-plore and generate the content
18/01/2011
21
Participants of the discussion: Micah Barrett, Olivia Verdugo, Catherine Siller, Niu Miao, Camila Afanador, Keitaro Sakamoto, Jamie Jewett, Lane Myer and Lucy Hitchcock
25 / 01 / 2011BRAINSTORMINGLane helped us writing down all the ideas in a brainstorming session we had. We proposed to develop a group project.
“...the proposal is to create a way to visually map (in 3d) gesture whether that gesture be behavioral or emotive with a focus on (spa-tial) relationship between hands and bod-ies and what info we may parse from those spatial relationships.” Catherine's description of our project
22
WEEK 4January 24 - 30 / 2011Group Discussions and Work Sessions
19/01/2011
23
first: catch-up
catherine: working directly on system rather than only on our computers
encountered problem: not able to save things once we’ve uploaded them to g-speak
jamie: there has been a new user set up for you guys (not clements)
we should find login info for this
catherine: perhaps we should ask evelyn?
jamie: if there is a problem with the system, let me know instead of going to clement directly b/c i can try to fix it first &/or alert Peter
interesting to publish the notes that i am taking
olivia: process document consisting of google docs notes, video, photos, etc.
jamie: who is hosting the wordpress blog? (we think risd/dm?)
catherine: worked on the technical aspect of developing our projects yesterday (on sunday)
1 group project (micah, Camila, catherine, olivia)
niu: doing own thing/ working on cityscape processing
keitaro: working on own typographic project
(edit - we all plan on working in the space at the same time so that we may help trouble shoot one another’s sketches etc.)
lucy: niu kept talking about fingers?
catherine: working on manhattan sketch as primary project
catherine: talking about the general group
project: what we ended up agreeing on as a group project was the idea of using the gspeak system to translate gestural expression into visuals
i.e. two people having a conversation and using gspeak to record a gestural record of what the hands are doing during a conversation
we each did a little visual brainstorming
lucy: makes me think about standard ty-pographic example: massin’s bald soprano (visualizing typography)
the other thing that brings to mind is the greek theater chorus trope - narrating and coloring what is happening on the stage
if something is happening here (in the gspeak space) in a small way (hands) where does that leave the rest of the space?
jamie: working 1:1 way
background of the environment that this is happening on top of could be impacted by secondary or tertiary levels of interactivity
still GENERATIVE thus
most of the gspeak stuff i’ve seen has been 1 layer- neutral
is there different layers of content/inter-polation of data that could create a layered record? mining the data from the hands in different ways to create layers? even just using time: slow down, speed up etc.
(edit: this is a really good point to think about!)
camila: how do the visuals affect the com-prehension of what someone is saying?
what kind of text or speech would we want to read? if the gestures will be enough
group discussion - january 24th, 2011Notes Jan 24th, morning meeting:. (sorry guys, this one is REALLY chock-full-o typos. i will try to edit it when i have a chance)
24
movement for us to produce the visual part of the project?
lucy: my dinner with andre? recording a conversation with two people
micah: making someone type/rip paper/cut paper - what do different actions LOOK like?
lucy: i like that this pulls out how we inter-act with tools
it’s saying something/making a statement culturally or sociologically
camila: VISUALS: how can the movement of the hands be visually represented, thinking about the movement of a ribbon (like olympic ribbon) element in 3d space etc. (shows us an image of two intertwined ribbons of color)
this was the basic idea of thinking of the seismograph but not thinking of the lines as “lines” but objects in a space with surface
(shows processing sketch) dimensional lines generated by the movement of the mouse across the screen
catherine: color will also play a role of course
camila: what can happen if we track the interaction btw. both hands?
jamie: one of the points of gspeak is that it applies DEFINITION and MEANING to gesture
most of the gestures we use in conversa-tion would have more than one meaning
most of the gripes we have defined are not conversational
lucy: maybe we need a middle ground/translating device?
(talks about friend’s project involving wind)
couldn’t you record the movement of people speaking with some other device and then feed that into gspeak to be ma-nipulated gesturally?
in order to get it to that place of pure gesture instead of just gripes...
jamie: i think thats going to be a rub that we come up against with gspeak often
you’re saying can i have a realtime motion capture data, which is the first layer, but we have access to the gripe info, not that
catherine: everything we do generates some kind of gripe information
if we can use this as a tool to identify what gripes are not in the packet that could help us
another thing that could help is figuring out how to assign, for example, 3 different gripes to the same effect, so that there is a little bit of leeway built into the gesture
camila: when you see the system, it us a little too rigid, we started to try to build in a level of flexibility
lucy: what if you had a mannequin, or someone that wasn’t human and did stop-frame animation while moving the fingers?
micah: i imagined programming little “traps”
catherine: someone should maybe jot down questions and concerns we have as we move along
two of the things you have brought up is the specificity of the system?
lucy: for me it’s not
(okay taking a mini break from typing to present my stuff - these notes are in the google doc olivia notes ideas )
lucy: i love how these things are hovering around real possibility for use in the world
i am thinking about the stem/steam thing
the idea that you could take an ephemeral gesture and turn it into a form
so that the thing ends up looking like something no one has ever seen
read christopher alexander
talks about the relationship of the whole system to the parts
what i love is the idea that the concrete aspect of the system (this moth) is in its nature to do what else works that way?
do two hands talking make some sensible form as this does? or will it be a mess?
what is the difference between us and this perfect being?
20/01/2011
25
26
25 / 01 / 2011KEITARO:I succeedED in controlling the camera angle, and now trying to figure out how to use ‘translate()’ at ‘hand.pos.z’.
24 / 01 / 2011MICAHMicah trying out his sketch in the GSpeak system,
20/01/2011
27
jamie: one of the differences btw this and a standard like quality is TEXTURE
tracking the relationships btw two hands and the fingers could create interesting texture
micah: trying to track relationship btw individual fingers etc.
is it possible to track each tag in space in-dividually? or do they only track in relation to one another?
jamie: what you are trying to do is subvert-ing the intent of gspeak - going behind the layer they have created. it might be as interesting to NOT do that
you can draw the path of the hands from the path of the hands
what happens if you draw the path of the gripes instead of the motion of the hands?
what happens if you track the path of the hands in relation to THEMSELVES?
from the standpoint of not doing what a lot of other people have done, how can we present it differently by taking advantage of what the gspeak system does best?
what can we do with the gripe informa-tion to create a different kind of positional data?
(edit: i agree)
lucy: what is the conversation thats being had and what does that do to the hand gestures?
micah: (showing sketch for dynamically drawing bezier curves) based on where the mouse goes it changes the curve
what i was envisioning is a more human looking line
i agree with what jamie said that this is very computer-looking
jamie: maybe its because i’m a geek but even so i find this so pleasing
micah: if the representation of it is some … what if all the dots/lines are coming out of the matrix of the tag, instead of just a single line? i’ve been thinking about what a seismograph does, it takes this really long record and
i’m really interested in the possibility of
maybe (the central square) never moves from the center, but the evidence of motion is captured behind it
so we would have this long trail of evidence
(sketch done in illustrator) bands of color emerge from the individual balls other tags creating very interesting paths
jamie: as opposed to just tracking a swishy line that follows my fingers there’s a “blob” of meaning that appears relating to specific gesture?
catherine: so the weight or quality of the line would be changed/determined by gesture?
micah: so the lines would all coalesce to make a shape.
one thing i’ve been really interested in is the relationship of people to the machine
example: fb: you have a transcript of the convo
with this you get something else
lucy: what if you traced the gestures of a convo, and sent that info to someone else? what would that look like?
catherine: a dm student wrote a program that drew squares according to sound
louder speaking-larger squares
ended up with a catalogue of conversa-tions; visual record of convo with mom, sister etc.
it was more intuitive/easier to read than one would initially think
lucy: video someone made of people de-scribing the relationship btw two people
hands playing with yarn
micah: i’m also interested in taking it out of the conversational concept: a pair of scissors, how do we use our hands as tools?
lucy: other situations where it would be interesting to look at what the hands are doing? political rally?
what other situation would we be able to look at?
catherine: something i thought was inter-
28
esting visually
(video with green type and threadlike structures)
dealing with space 3dimensionally
ribbons
all on the same page visually
how could this exist outside of the system?
i will be the voice for impracticality@!
how neat it would be to have some model for some kind of three-dimensionally thing but then what if i built it myself out of dowels and yarn instead of having it rapid prototyped
what if we did videotape the hands? so we could see what they are doing while we are tracking their movements?
what are the circumstances? this is the really big thing we haven;t figured out
are we going to put random people in here besides ourselves?
is it two people having a convo?
would we project text onto the other walls?
what about the performative aspect of this?
(viewpoints book) visual take on theater
one of the ways scientists use seismo-graphs is to map the inside of the earth that we can not see
lucy: t looks like a geologic map
catherine: exactly. what if we create this kind of tool that enables us to map some-one’s internal landscape?
jamie: interesting bc its one of the things that gspeak is being used to do: parse geological data?
lucy: would we overlay data? what’s hap-pening inside you (gesture) and whats happening in the ground in rhode island right now?
(edit: interesting)
catherine: whats the situation, who are the people that we are putting in this situ-ation? how can we begin to pin this down
cultural aspect:
how do gesture shift depending on what
language is being spoken?
another idea that camila had, that might be interesting to look into: there may be some kind of way that the content of the convo could somehow be built into that visual ribbon
dos it make sense for the person being mapped to be able to watch the map evolve?
jamie: that goes back to whether this is a stage or not. is this performative? or is this
lucy: you have e an interest in the record and the documentation
(note to self: olivia & lucy said something interesting at 20:03-21:00 listen to record-ing and add into notes)
[then we talk a little bit about how we each might be interested in being involved dur-ing the spring semester]
jamie: if we continue in the spring we might not have to continue working on this specific project, but we could build upon what you’ve learned and created
lucy: catherine before we move on what , in a nutshell, is the group project?
catherine: i think that the proposal is to create a visual / a way to visually map (in 3d) gesture whether that gesture be behavioral or emotive with a focus on (spatial) relationship between hands and bodies and what info we may parse from those spatial relationships.
keitaro: want to create processing applet to manipulate typefaces and converse through gesture
(shows examples of text manipulation by Peter Cho)
Jamie: have you guys talked about Daniel Howe’s Rita library?
keitaro(shows examples of how he hopes he may be able to manipulate single letter-forms through gripes)
lucy: why single letters rather than phrases?
keitaro: technical constraints
20/01/2011
29
24 / 01 / 2011CAMILA:Drawing sketch using 'easing'. Now the idea is to make it work in G-Speak. and make each hand draw in a different color.
lucy: it might be interesting if instead of squeezing for example the c, you went from something like a wide letter-form like a w to narrow letter-form like an l
catherine: we also worked on a sketch yesterday where the line will follow your hand’s x and y position and the z coordi-nate of your hand affects the width of the line
jamie (shows some fire calligraphy dance sketches to us as an example)
texture mapping flame on top of the rib-bons created by dancers
(also shows us troika ranch dance ex-ample)
lane (43:00-44:00 - fill this in)
jamie: ancillary movement why we have motion capture so that the movement we use in our animations are more real-
seeming
lucy: designer who created the string project: leslie kwok
http://lesliekwok.info <this is her main page
http://lesliekwok.info/mfa-thesis/socio-grams/ <this is the string example
[took a break here to talk about scheduling and decided that Monday at 6pm Feb 7th will be our final showing of our wintersesh work]
(camila shows her gspeak sketches - and they work! and look awesome)
we discuss the possibility of getting Pilot 1 working (right now only pilot 0 is function-ing)
(catherine shows her gspeak sketch - and it works! and looks awesome)
30
(jamie gives catherine’s sketch a try)
(edit: BRB)
catherine: making a plug for simplicity (edit: here here!! i agree. “K.I.S.S.”)
in my experience if we’re really particular and specific about how we set up the environment it will take us a long way in making the result / visual more impressive and valuable
jamie: and just to be clear, the point is not necessarilly to be “impressive”
you came in here, learned and went through a process that is dynamic and interesting and exciting, that = success. that’s why we’re here. that’s what risd’s agenda for the tool is.
I think its importatn/ what david wants to see is that there’s something happening taht’s interesting
we might think about inviting peter/david
since we’re doing agreat job at documen-tation, part of what could happen is having recordings of our steps along the way documented on the gspeak blog
one problem with our last classes is that we have no documentation of previous work done on the system! so it is as if it never existed
lucy: your research has been incredibly rich etc.
one of my goals in being part of this is to say this is here and lets keep using it and how can we continue to keep using it as a learning tool even if we end up making smaller tools/’sketches instead of some-thing “grand”
example of book containing lovely threads/layers connecting parts of the book: -> http://maria-fischer.com/en/traumgedan-ken_en.html
[niu now talks about his project]
niu: (shows us his cityscape sketch on the gspeak system) moving hands creates citys
jamie: maybe you should think about creating a gripe to clear the screen
lucy: and somehow i think it would be
lovely to record all the data thats being created (points to edge of screen where gripes are being recorded by “println”)
olivia: lets make a poster using all the gripe data generated by an interaction with the system!!
niu: i want to create an applet whereby with a very subtle gesture one can control the skyline/buildings flying away from the horizon line
lucy: where is that? wouldn’t it be interest-ing to create a sketch of a city that doesn’t have skyscrapers, or that needed them?
niu: (shows black and white photo) this is more like what i’d like to use in my sketch
jamie: what about asl?
potential problem: so much of what is communicated through asl is through emotional moement/facial expression etc. not just the letters/words formed by the hand but HOW they are formed
micah: paleography - developing a cal-ligraphic language for the hand?
jamie: might be interested to look into the stylus shorthand of palm pilots
lucy: if you aer going to do something where you trace two peoples hands in a convo what if one of these hand \-;..
what if we are already making marks with our hands that are somewhere represen-tative of an alphabet?
so, what do we want to sort of leave with?
continue to read each other’s notes on google docs
look back at that nice nugget that cath-erine expressed
do we need to specify things more pre-cisely in the next 20 mins before we walk out of here?
catherine: for me our interests seem to lie in somewhat different areas
should we agree upon that now, or decide to set it aside entirely?
lucy: could we make each of our ideas live in small sketches?
and the look to the process documentation
20/01/2011
31
to show what else we’re thinking about?
jamie: so today maybe figure out what you want to pursue and let us know
(1:36-37 - notes on what we want to do over the next two weeks)
lane: ideas for niu about being able to vi-sually traverse a city by moving things with our hands-shoving buildings apart etc. interacting organically witht he cityscape
lucy: be cognizant of the binoculars - do you need them if you are not using them?
niu: the binocs move with the car. not sure if coding ability is up to par with trying to get done what i want to get done
catherine: devils advocate we don;t have much coding experience either but we;’re getting it done we can make something simple and yet compelling
lane: don’t ever underestimate the cumu-lative effect showing these demonstra-tions in steps can have
people have a tendancy to understand the new by things that we already understand today
jamie: on interaction: you impact the sys-tem and the system impacts you
what about faking it???
(sorry everyone i am losing steam here a bit when it comes to typing - 3 hrs and my hands hurt!)
lucy: good conversation around the big project, tracking (1:47-1:48)
get some nourishment
come back and try to lay out one or two sentences that explain the goal
then maybe there’;s a way you can each work on different parts of the goal/real-izing different parts
jamie: its a great idea to think about working as a group to build the gestural motor, and then each can use it potentially differently
for next time:
developing the visual
exploring our understanding of code
basically, we are all interested in very similar visual parsing of the data:
where the hand(s) is in space, and how the hands relate to one another > how can we translate that information into a compel-ling visual?
& keep in mind what catherine stated earlier: “i think that the proposal is to cre-ate a visual / a way to visually map (in 3d) gesture whether that gesture be behav-ioral or emotive with a focus on (spatial) relationship between hands and bodies and what info we may parse from those spatial relationships.”
QUICK CHECK IN at 3:00pm:
olivia: interested in mapping gripes/rela-tionships rather than just hand in space (not just hand:hand but gripe:space). very interested in the REMNANT - what data is left over from a conversation articulated with our hands, and what can we DO with that data?
GSpeak screen dimensions
32
could we make a poster of gripes?
do we use the data in a video?
do we create a 3d prototype?
camila: like that idea - mapping btw two hands
catherine: might be interested to map btw behavioral gesture and … mapping behavior to mode of communication i.e. do we gesticulate more when speaking face to face than we do on the phone? over ichat etc.?
would like to have outside pressure (in
the form of people coming to our final showing)
as lane said, sharing our IDEAS RESEARCH
maybe (olivia) begin to brainstorm about how we could visually collate our
micah: excited to be playing with the visual aspect of this
keitaro: hmmmmmmmmm........hmmm-mmmm......want to keep coding and try to figure out how it might be possible to ma-nipulate typeface in 3d through process-ing. simple prototyping?
24 / 01 / 2011NIU:
20/01/2011
33
24 / 01 / 2011CATHERINE:
34
27 / 01 / 2011KEITARO:Considering again from the beginning, I made dot fonts on processing. This only works on PC so far, I want to connect with G-Speak and also make it 3D (like spheres).
20/01/2011
35
27 / 01 / 2011OLIVIA AND MICAH:We finally had a little breakthrough tonight! micah and i were able to get a bezier sketch running such that each of us could control one of the origin points with a gripe. camila took pictures, which we will post as soon as we have a chance to upload them to the laptop.
36
WEEK 5January 31 - February 6 / 2011Final Work Week
20/01/2011
37
Micah's sketch
38
Niu's sketch
Camila's drawing sketch
20/01/2011
39
-We need to update the library on the g-speak system
-there is now a second winter-session acct, perhaps we could upload the 2nd library there, so that if there is any problem with one, we can use the other
-via email, let’s compile a guestlist con-taining specific members of gd and d+m to invite to our showing next monday night
-kelly, evelyn, beth, dylan, andrew etc.?
-probably we should refrain from inviting the whole of each department so that the space is not overwhelmed.
-we can also always perform a demo for people who are not there at our “show-n-tell” next monday
please note:
the other g-speak acct info
acct name: risdwint2011
password: lines2001
--we might be able to save our sketches when working on this acct, as the prob-lems we’re having with saving on clement’s acct have to do with permissions.
everybody, check-in!:
keitaro:
-working on a sketch called “dotfont”
-the size of the dots composing the type are determined by how many times you click on the sketch with the mouse
-each of the circles jiggle slightly and leave a trail behind them when you move the word around the screen
-next step: upload to gspeak and control dot font through z-position of the hand
-also, try to control each letter separately
(one glove:one letter)
-the dots grow to a specific size, then return to their original small size
-lucy wonders if there could be one row of dots rather than two? if your investigation is about type, perhaps you could start it from the smallest version i.e. start with one dot? then bring in more dots (w mag-netic force?) and see the type assemble. that would create a narrative: how the type gets built.
-petri dish growth as a jumping off point?
-the “jiggling” of the dots suggests mo-lecular movement
-how could it be pushed forward? what if the dots were scattered, and went looking for each other?
-would it be possible (if each letter were a separate object/function) to manipulate them more carefully?
-lucy says: what word you’re teaching seems to matter to me. why risd? what about hotdog? blood? there ought to be some deeper connection between the meaning of the word and the form you’re giving it.
-lane says there’s almost something intestinal about the letter-forms, mentions cell mitosis. creation of a creature as a typographic image?
-growth is at the crux of this exploration
-when things grow, we don’t necessarily think of them as done. we’re just seeing them in one stage of their growth.
micah/olivia:
-THIS SKETCH (the one we just showed you all):
group discussion - january 31th, 2011
40
-the origin points of the bezier curves are controlled by the x and y position of pilot 1’s hands in the g-speak system.
-the bezier handles (aka control points) which control the curvature of the lines between the origin points controlled by the x and y position of pilot 0’s hands in the g-speak system. This is functional, but too exaggerated right now, so we need to scale it down.
-OUR NEXT STEP:
-we would like to map the bezier handles (aka control points) which control the curvature of the lines between the origin points to the x and y position of the hand in the g-speak syste m.
-OUR NEXT, NEXT STEP:
-get pilot 0 and pilot 1 working in concert such that each pilot’s left hand controls their respective origin point, and each pilot’s right hand controls their respective bezier handles (aka control points).
--notes--
-maybe we should incorporate a thin line of light into the bottom of the screen?
-how does having to kneel when interact-ing with the sketch change the meaning of the sketch?
-how doe we want to *FRAME* ourselves on “stage”? (or, is it even a stage?)
-could the control points of the bezier curves become something other than dots? letterforms? geometric shapes?
-micah shows beautiful sketch of rotating white circles
catherine:
-feeling a bit out of the loop
-b/c we are all new to coding, it is difficult for her to look at the code and understand what’s going on
-perhaps we can split up the work as we continue:
micah & olivia can continue trying to con-trol the bezier handles?
camila & catherine can work on replacing the circular bezier control points with
typographic characters, or perhaps other shapes?
-project catherine has been working on alone: understanding how the system fol-lows quality of touch
-videotaped an experiment this morning
-playing around with the idea of drawing by touch rather than sight -
-interested in using the computer as a barometer to measure quality of touch
-sketch shows catherine running g-speak gloved hands over her own body to pro-duce forms on the screen
-looks like duchamp’s nude descending a staircase (cool)
-what about incorporating an audio ele-ment?
-keep thinking about all the possible ways to enrich this artform
camila:
-has been working on exercises regarding grabbing points and moving them?
-working on a process book for her open research class
-it is awesome! incorporates notes, images etc. documenting our processes as a group
-please send camila any images + cap-tions you would like to see in the book
niu:
-demonstrating his sketch wherein the city line is controlled by niu’s gripes as he moves his hands across the screen
-there is also a sailboat(?) hanging out in the sky
-the ship is three dimensional! and it changes!
-perhaps the buildings could be stretched like taffy?
-visual NARRATIVE
-what does this mean? what’s it saying? what’s it about?
-also, think details - why upper left box? why nav(?) bar on right? smooth out the details in the graphics etc. so that the city looks like it is growing better.
20/01/2011
41
Olivia's Diffusion Limited Aggregation
Olivia's tree sketch
42
WEEK 6February 7, 2011, Final Presentation
20/01/2011
43
HH
H
M
M
M
M
M
aaa
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
nn
n
n
n
d
d
d
d
d
dd
d
d
e
e
e
e
e
e
e
e
ee
d
A collection of images produced through mapping the location and character of hands in space. Written in Processing at the Rhode Island School of Design in the winter of 2011.
Micah's Poster
44
(1) Two-hand drawing. Hand 1: Micah, hand 2: Camila. (2) Clemen Dancing Drawing
Camila's Poster
20/01/2011
45
Catherine's Poster
46